Commit Graph

17 Commits

Author SHA1 Message Date
Edouard Vanbelle a4101743e0 feat(jobs): jobs declare their own run parameters
`JobRunArgs` was a fixed struct — `force`, `deep`, `storage`, `repair` —
and six places hardcoded that same list: the engine's persist/restore,
the trigger endpoint's query type, the OXICLOUD_STARTUP_JOBS parser, the
frontend API wrapper, the panel's checkboxes, and `StartupTrigger` on
the wire.

Two costs. Adding a parameter meant editing all six, and forgetting one
dropped it silently — most damagingly in persist/restore, where a
resumed run lost it and a `?repair=true` migration came back as
discovery-only after a restart. And the panel offered the same knobs on
every job: only two jobs read `deep`, six read `repair`, so most of
those controls did nothing with no way to tell which.

Now `JobHandler::parameters()` returns `&'static [JobParam]` — name,
type (boolean/string/number), default, and the job's own description of
what it does. `JobRunArgs` holds a map keyed by those names.

Everything reads the declaration:

* `run_or_resume` iterates it to persist and restore, replacing
  `const FLAGS` plus a `storage` special case. `storage` stops being
  special — it was the one Option<String> among three bools.
* `dispatch` normalises every run against it, which is what makes "a
  handler sees its declared parameters with their declared defaults"
  true rather than usual. The periodic tick passes an empty
  `JobRunArgs::default()`, so a `default: true` parameter would
  otherwise read false on every scheduled run.
* The trigger endpoint takes free-form query params and rejects
  undeclared ones with a 400 naming the real set, instead of ignoring
  them.
* OXICLOUD_STARTUP_JOBS keeps raw pairs (config is parsed before the
  registry exists) and validates at dispatch, where the error can name
  the job's actual parameters. Still a boot panic, same as an unknown
  job name — a typo'd `?repare=true` must not leave a migration
  importing forever in discovery mode.
* `JobSummary.parameters` carries it to the panel, whose `supportsDeep`
  was a hardcoded name allowlist (`consistency_batch ||
  backend_consistency`). A job gaining a deep mode needed a frontend
  release; one losing it left a button that silently did nothing. The
  menu now renders from the declaration, so a newly-declared boolean
  appears with no frontend change.

Three consistency tenants were hand-rolling persist-on-fresh /
restore-on-resume for their own flag, under the same `params` key the
engine already used. Deleted — they read `args.get_bool(…)` now.

Fresh runs also filter to the declaration. `consistency_batch` forwards
its args verbatim to sub-jobs, so a tenant's `params` row could grow
`deep` with no deep mode, and the run-detail view would claim a mode the
job never had.

Two things found while wiring it, both worth knowing:

`RecoverableAdapter` bridges the two traits, and `parameters` has to be
forwarded there or the registry sees `&[]`. Both traits have defaults,
so omitting it compiled cleanly — and the trigger endpoint then rejected
`?repair=true` on the very jobs that declare it, with
OXICLOUD_STARTUP_JOBS panicking at boot. Now covered by
`adapter_forwards_job_metadata_from_inner_handler`.

`TriggerJobQuery` was briefly a newtype over the map. `serde_urlencoded`
cannot deserialize a newtype struct at the top level, so axum's `Query`
rejected EVERY trigger with a 400 — even one with no query string —
before the handler ran. It reads exactly like the new validation
rejecting something, which sent the first diagnosis to the wrong layer.
Now covered by `trigger_query_extracts_from_every_url_shape`.

Wire names are a compatibility surface: `params` rows are keyed by them
and the panel switches on them, so a rename breaks existing run history
the same way renaming a `Mutates` variant does. The JSON shape is pinned
in `snapshot_carries_job_metadata`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-07 22:23:30 +02:00
Edouard Vanbelle 63820c3c26 feat(jobs): the engine binds a run to the flags it started with
`run_or_resume` now records `JobRunArgs` in `params` on a Fresh open and
restores them on Resume, passing the restored args to the handler rather
than whatever the resuming caller supplied.

Two problems, and the engine is the only place both are guaranteed.

**A resumed run must not change mode.** Handlers read `args` on every
call, so a paused `?repair=true` import resumed by a plain trigger
silently continued as import-only: the deletion half never finished and
nothing reported it. `?deep=true` had the same hole — a paused bit-rot
scan resumed shallow while still presenting as the run that started
deep. `blobs_consistency` and `manifests_consistency` had hand-rolled
this for their own two flags; the three import jobs had not, and fixing
it per-handler means every future job remembering.

**The run row should say what it did.** For a destructive job, "did this
run delete anything?" is answerable only from `params`, which is what an
operator reads afterwards. Before this, a repair run and a discovery run
were indistinguishable in the history.

Deliberately not overridable on resume: adding `?repair=true` to a
resume would apply it to the remaining entries only, producing a run
that half-deleted. Cancel and start fresh is the honest way to change
your mind.

A missing key reads as false/None, so a run paused before this existed
resumes under-acting rather than deleting under a flag nobody gave it.
Failure to record the flags fails the run instead of guessing — acting
under unrecorded flags is the one thing worth refusing for jobs that
delete.

The flag list is hardcoded here. Letting each job declare its own
parameters — name, type, default — is the better shape and is its own
change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-30 22:07:27 +02:00
Edouard Vanbelle 1ea3826660 feat(jobs): jobs describe themselves — description, mutates, repair_description
The admin panel had no repair toggle wired to anything but a hardcoded
name list naming the two refcount tenants, so `thumb_derived_import` and
`thumb_attached_import` could not be run in repair mode from the UI at
all despite supporting it. And nothing in the job list said what any
given job does or whether clicking Run on production writes anything.

Three defaulted methods on `JobHandler` and `RecoverableJobHandler`:

    fn description(&self) -> &'static str
    fn mutates(&self) -> Mutates          // Never | Always | OnRepairOnly
    fn repair_description(&self) -> Option<&'static str>

`RecoverableAdapter` forwards them — the registry only holds
`dyn JobHandler`, so a tenant's metadata is invisible otherwise, and
falling back to the defaults would report every recoverable job as
read-only, including the ones that delete files.

Three values rather than a boolean because a job can be read-only by
default and destructive under `?repair=true`; a boolean answers wrongly
for one of its two modes, and `false` on something that unlinks files is
the dangerous direction to be wrong in. `repair_description` returning
`Option` collapses "does it repair" and "what does repair do" into one
method: presence gates the toggle, content is the confirmation text —
which the frontend cannot invent, since correcting a counter and
deleting sidecars are not the same warning.

`OnRepairOnly` with no `repair_description` is rejected at registration:
it claims to mutate only under a flag it does not support.

All 17 registered jobs declare all three. The panel now renders the
description under each name, badges read-only jobs, confirms before a
plain run of a mutating one, and offers the repair variant off the
backend flag instead of the name list.

Descriptions are English in the trait, next to the behaviour: one in
`locales/*.json` rots invisibly the moment a job changes, and a
translator cannot know what `manifests_consistency` reconciles. i18n can
layer on later keyed by job name with these as the fallback.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-30 13:41:05 +02:00
Edouard Vanbelle 8b0fb03b5c feat(recoverable jobs): clarify life cycle pause vs cancel
a job can be paused/resumed
    a job as an exclusibity by it's name
    if you want to run another job with same name:
    either cancel the first one, or wait of it's terminaison

    pause does not permit to run the other job, this can create
    race conditions
2026-08-02 17:38:44 +02:00
Edouard Vanbelle 5eec0fb36e fix(migration): recover the main banner progression on server restart 2026-08-02 16:25:47 +02:00
Edouard Vanbelle 07802e01f8 feat(recoverable job): add pause/resume capability 2026-08-02 16:17:15 +02:00
Edouard Vanbelle 015f2da0f7 refactor(backend): normalize naming convention to backend rather storage
no ambiguity with the backend rather storage
2026-08-02 14:56:29 +02:00
Edouard Vanbelle 0a41f561d0 feat(rotate-key): add report + key fingerprint in hexdigit fmt 2026-08-02 02:40:33 +02:00
Edouard Vanbelle a58351b7ad feat(storage key rot): add admin panel 2026-08-02 02:40:33 +02:00
Edouard Vanbelle 6b7bb67500 feat(storage): jobs can choose storage to migrate/scan 2026-08-01 13:06:09 +02:00
Edouard Vanbelle 9fe47a53ea feat(jobs): add admin call to purge jubs result 2026-07-30 00:39:11 +02:00
Edouard Vanbelle 5527d09618 feat(recoverable-job): check if old blob (no cdc) still remains: notice only 2026-07-29 22:47:07 +02:00
Edouard Vanbelle 0f12399a48 feat(recoverable-job): fix files_consistency to check blob chunk consistency 2026-07-29 22:17:15 +02:00
Edouard Vanbelle 5881968f50 feat(recoverable-job): add progress view 2026-07-29 08:41:19 +02:00
Edouard Vanbelle e1556e3d36 feat(recoverable-job): add findings 2026-07-29 07:57:06 +02:00
Edouard Vanbelle 8177b04786 feat(recoverable-job): wire API (cancel, view run, ...) 2026-07-28 23:25:37 +02:00
Edouard Vanbelle 302d2ff80f feat(recoverable-job): add core engine 2026-07-28 22:39:07 +02:00