feat(consistency): derived_consistency — the last coverage-matrix gap

Finds derived mappings whose Blob is gone on either side. Nothing else
can, and that is the point rather than an oversight: every other job
reasons from a Blob outwards, so a row whose SOURCE was reaped breaks
none of their invariants — valid reference, exactly correct refcount,
bytes present on the backend. Every check agrees the system is healthy
while the artifact is pinned forever. A leak that looks like
correctness, which is why it took four suite runs to name.

Two findings:

  derived_orphan_mapping (inconsistent) — source_hash has neither a
  manifest nor a blob row, so purge_derived_blobs can never fire for it.
  Storage that grows and never reclaims.

  derived_dangling_blob (data_loss) — blob_hash has no Blob behind it.
  The mapping promises an artifact that is gone, so a read finds the row
  and then fails.

Existence means EITHER table on both sides, since source_hash and
blob_hash each name a Blob: a manifest for CDC content, a bare blob row
for legacy whole-file content. Checking one would report every legacy
blob as missing.

Paged on the full primary key with a row-value comparison rather than
source_hash alone — a source has several variants, so a page boundary
can fall inside one and advancing by source would skip the rest. Both
existence probes fold into the page query, so a page is one round-trip
rather than 2xN. Cursor round-trip is tested, including that a malformed
one fails loudly: silently restarting would make a paged audit
under-report, which is the worst failure available to a job whose
purpose is finding what is missing.

e4c78ae0 stops new orphans at the write side; this finds the ones
already on disk, which that fix cannot reach. Added to the end-of-suite
sweep so it runs against real state every time.
This commit is contained in:
Edouard Vanbelle
2026-08-28 19:33:02 +02:00
parent de0f625d4c
commit 4fef34b230
4 changed files with 348 additions and 0 deletions
+13
View File
@@ -1494,6 +1494,19 @@ impl AppServiceFactory {
.register_recoverable_job(&core.job_registry, &job_store_provider_dyn)
.await;
// Finds derived mappings whose Blob is gone on either side. Nothing
// else can: a row whose SOURCE was reaped still holds a valid
// reference to a real artifact with a correct refcount, so every
// other check agrees the system is healthy while the artifact is
// pinned forever. Read-only.
let _ = Arc::new(
crate::infrastructure::services::derived_consistency_service::DerivedConsistencyCheck::new(
maintenance_pool.clone(),
),
)
.register_recoverable_job(&core.job_registry, &job_store_provider_dyn)
.await;
// Its file-keyed twin: `ext-{file_id}.jpg` previews the user uploaded,
// which no copy path duplicates today. Separate job, separate keying —
// routing these into the content-keyed table would share one user's