feat(job-registry): remplace /api/admin/internal/trigger-*

remplace /api/admin/internal/trigger-* to /api/admin/jobs/{...}/trigger
remove OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS
This commit is contained in:
Edouard Vanbelle
2026-07-27 23:40:40 +02:00
parent dfedde54a4
commit f66f7fa31f
24 changed files with 155 additions and 464 deletions
+5 -5
View File
@@ -360,11 +360,11 @@ grace window has zero effect on live access decisions — an expired grant is
invisible to `check(...)` even during the grace period. Cleanup only affects
storage bloat and the `list_grants_*` history surface.
The daemon runs inside the same process (`tokio::spawn` at startup, same
lifecycle as trash-cleanup / storage-usage sweep), so no external scheduler
is needed. An admin-triggered `POST /api/admin/internal/trigger-grant-cleanup`
lets operators force a purge in test or incident scenarios; the internal-
endpoints gate (`OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS`) applies.
The daemon runs inside the same process, registered with the periodic-job
scheduler (`docs/plan/job-registry.md`) on a 24-hour tick. An admin-
triggered `POST /api/admin/jobs/grant_cleanup/trigger?force=true` lets
operators force a purge in test or incident scenarios — `force=true`
collapses the grace window to zero for that call only.
The [Share Integration](/architecture/share-integration) doc's reverse
trigger takes it from there: when the daemon deletes the last `role_grants`
-1
View File
@@ -73,7 +73,6 @@ Most runtime variables use the `OXICLOUD_` prefix. A few build-time or allocator
| `OXICLOUD_ENABLE_SEARCH` | `true` | Full-text and metadata search |
| `OXICLOUD_ENABLE_MUSIC` | `true` | Music playlists and audio metadata |
| `OXICLOUD_EXPOSE_SYSTEM_USERS` | `true` | Expose other OxiCloud users as a read-only address book at `GET /api/address-books` |
| `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` | `false` | Expose `POST /api/admin/internal/trigger-sweep`, `POST /api/admin/internal/trigger-gc`, and `POST /api/admin/internal/trigger-grant-cleanup` — test-only synchronous triggers for the storage-usage reconciliation sweep, blob garbage collector, and expired-grant purge respectively. Used by the API test suite to assert convergence deterministically without waiting out the periodic tickers. Leave **off** in production: the routes return 404 even to an admin token when disabled. |
| `OXICLOUD_GRANT_CLEANUP_ENABLED` | `true` | Background daemon that deletes expired rows from `storage.role_grants`. The authorization engine already filters expired grants out of every permission check at read time (`expires_at IS NULL OR expires_at > NOW()`), so leaving expired rows in place is a hygiene issue — not a security one. This daemon garbage-collects them daily. Set to `false` to keep every expired grant row forever (uncommon; a fresh install rarely wants this). |
| `OXICLOUD_GRANT_CLEANUP_GRACE_DAYS` | `15` | Days past a grant's `expires_at` before the row is eligible for deletion. The grace window preserves the audit / support answer to "what happened to my access?" for a couple of weeks past expiration. Values below 1 are legal but discouraged — the recommendation is **≥ 15 days**. Values above the actual grant TTL used by clients waste index space; a few weeks is the sweet spot. |
| `OXICLOUD_GRANT_CLEANUP_INTERVAL_HOURS` | `24` | How often the grant-cleanup daemon fires. Clamped to a minimum of 1 hour. Adjusting this doesn't change what gets deleted — only how promptly. Daily is fine for any realistic grant volume. |
+34 -22
View File
@@ -426,7 +426,8 @@ pub trait CheckStore: Send + Sync {
///
/// See `BlobConsistencyCheck` for the canonical impl to copy-adapt.
pub trait StatefulAdapter: Send + Sync {
/// Subsystem slug — appears in `POST /api/admin/internal/consistency/{name}`
/// Subsystem slug — appears in the JobRegistry-registered
/// `job_name` (`consistency_<subsystem>`, e.g. `consistency_blobs`)
/// and in audit log `event` values. Lowercase snake_case, unique
/// per adapter. Convention: `"blobs"`, `"thumbnails"`, `"trash"`,
/// `"folder_tree"`, `"used_bytes"`.
@@ -483,26 +484,36 @@ impl ConsistencyRegistry {
## Admin surface
```
POST /api/admin/internal/consistency/{name}
→ 202 { run_id } (starts a new run)
Consistency runs are ordinary `RecoverableJob`s (see
`docs/plan/job-registry.md` Part 2), so most operator actions reach
them through the shared scheduler surface:
POST /api/admin/internal/consistency/runs/{id}/cancel
→ 200 { status: "CancelRequested" }
```
GET /api/admin/jobs
→ summary list — consistency runs appear as
`job_name = "consistency_<name>"`
POST /api/admin/jobs/consistency_{name}/trigger
→ 200 { ok, outcome: { run_id, status } }
(starts a new run or resumes the latest Paused one — see
Part 2's `run_or_resume`)
POST /api/admin/jobs/consistency_{name}/cancel
→ 200 { run_id, status: "CancelRequested" }
(cooperative — check finishes its current batch and returns Paused)
POST /api/admin/internal/consistency/runs/{id}/resume
→ 202 { run_id } (picks up cursor)
GET /api/admin/jobs/consistency_{name}/runs?status=<status>
→ 200 [{ id, status, scanned_count, last_progress_at, … }]
GET /api/admin/internal/consistency/runs?check=<name>&status=<status>
→ 200 [{ id, check_name, status, scanned_count, last_progress_at, … }]
GET /api/admin/internal/consistency/runs/{id}
GET /api/admin/jobs/consistency_{name}/runs/{id}
→ 200 { run: {...}, findings: [...paginated] }
```
Gated by `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` — same admin-guard
middleware as `trigger-sweep`, `trigger-gc`, `trigger-grant-cleanup`.
Findings enrichment on `runs/{id}` is consistency-specific — read
from `admin.consistency_findings` and joined into the response.
Everything else is generic Part 2 behaviour.
Production surface — always on, audit-logged. No feature-flag gate.
## Approach
@@ -630,8 +641,9 @@ Ship this PR without the actual checks. Grep `TODO(consistency)` = punch list.
- `list_runs(filter)` — SELECT with filters + paginate.
- `get_run(id)` — SELECT run + paginated findings.
Same admin-guard + `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` gate as
existing internal endpoints.
Same admin-guard as the JobRegistry surface (`trigger_job`,
`list_jobs`). Production surface — always on, audit-logged, no
feature-flag gate.
### 6. Boot-time crashed-run recovery
@@ -726,10 +738,10 @@ count decreases by one per PR.
## Reused existing utilities
- **Admin-guard + gate pattern** at
`src/interfaces/api/handlers/admin_handler.rs::internal_trigger_gc` —
same shape for the new endpoints.
- **`OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` gate** — same env var.
- **Admin-guard + audit-log pattern** at
`src/interfaces/api/handlers/admin_handler.rs::trigger_job` —
same shape for the new endpoints (production surface, always-on,
audit-logged; no feature-flag gate).
- **Dedup GC's orphan-detection logic** (`dedup_service.rs`) — the
algorithmic template for `BlobConsistencyCheck`'s orphan phase.
Reference impl, not a callsite — the check needs its own two-pass
@@ -762,8 +774,8 @@ count decreases by one per PR.
7. **Grace-window sanity**: run against a fresh 10 s window; upload a
file mid-scan; confirm the young blob does NOT surface as
`MissingInStorage` (grace window covers it).
8. **Env-flag off**: `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=false`
→ endpoints return 404, no leakage in the audit channel.
8. **AuthZ gate**: non-admin caller hits `POST /api/admin/jobs/consistency_blobs/trigger`
→ 403 from the admin middleware, audit line records the rejection.
## Out of scope
+16 -11
View File
@@ -769,21 +769,26 @@ accepts an optional `?force=<bool>` query param that maps to
mutations belong on the audit stream. Success/failure outcome fires
its own `oxicloud::scheduler` line via the existing supervisor path.
**Legacy shim retirement** (Stage 2 — follow-up PR after this one):
**Legacy shim retirement** (Stage 2 — landed):
The three existing internal endpoints map 1:1 to the new surface:
The three legacy internal endpoints have been retired in favour of
the JobRegistry surface. Kept here for archaeology / URL migration
reference for any external tool that still expects the old paths:
| Legacy | Replacement |
| Legacy (retired) | Replacement |
|---|---|
| `POST /admin/internal/trigger-sweep` | `POST /admin/jobs/storage_reconcile/trigger` |
| `POST /admin/internal/trigger-gc?force=X` | `POST /admin/jobs/dedup_gc/trigger?force=X` |
| `POST /admin/internal/trigger-grant-cleanup?force=X` | `POST /admin/jobs/grant_cleanup/trigger?force=X` |
| `POST /admin/internal/trigger-sweep` | `POST /admin/jobs/storage_reconcile/trigger` |
| `POST /admin/internal/trigger-gc?force=X` | `POST /admin/jobs/dedup_gc/trigger?force=X` |
| `POST /admin/internal/trigger-grant-cleanup?force=X` | `POST /admin/jobs/grant_cleanup/trigger?force=X` |
Rewritten as thin forwards to the new endpoints with a `Deprecation:
true` response header while Hurl suites migrate to the new paths. Once
all callers cut over, the shims are deleted AND the
`OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` env var is removed — its
sole purpose was gating those shims.
The `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS` env var was removed
alongside — its sole purpose was gating those shims.
Response shape also changed: the old endpoints returned custom fields
(`grants_deleted`, `blobs_deleted`, `bytes_freed`, `forced`); the new
endpoint returns a uniform `{ ok, outcome: JobOutcome }` envelope with
job-specific fields under `outcome.extra`. Any external caller reading
the old fields needs updating.
### Config surface — env vars