refactor(consistency): blobs_consistency reads only the database
`blobs_consistency` probed `blob_exists` once per row and, under `?deep=true`, read and re-hashed every blob. `backend_consistency` already reports the same `blob_missing_from_backend` from its merge-join — so the probe was duplicated work that found strictly less (a DB walk cannot see backend-only orphans by construction) at N round -trips instead of one enumeration. Every scheduled sweep paid for it. All three physical checks move to `backend_consistency`: * `blob_missing_from_backend` was already there; the duplicate is gone. * `blob_corrupted` / `blob_unreadable` hook the matched arm of the merge-join, which holds exactly the key pairs worth reading. Guarded by `in_range` so a pair past the horizon is not read twice, and `params.deep` is persisted on a fresh run and read back on resume so a paused deep scan does not silently continue shallow. Deep mode belongs there because it is backend work end to end: the only DB input is the hash. Keeping it in `blobs_consistency` forced that tenant to carry a backend for one flag. What remains is the half that needs no backend: `refcount_mismatch` and its repair. The constructor drops from five parameters to two — no backend, no storage_entries, no storage_path_fallback — and `?storage=<name>` / `?deep=true` are now inert there, which the job description says outright. `affected_files` is needed by both tenants, so it moves to a shared `blob_diagnostics` module rather than being copied. `PROBED_STORAGE_PARAM` moves to `backend_consistency`: it was defined in `blobs_consistency` and re-exported, which is backwards once the DB-only tenant has no entry to scope. The create-grace window goes with the probe — it existed to avoid flagging a blob whose bytes had landed before its row, and the refcount comparison reads one consistent snapshot. Known cost: `backend_consistency` returns `backend_unenumerable` on Azure and mid-migration, so on those configs missing bytes now go unreported where the per-row probe caught them. That argues for the Azure enumeration impl, not for keeping the probe. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+12
-15
@@ -1536,24 +1536,21 @@ impl AppServiceFactory {
|
||||
.register_recoverable_job(&core.job_registry, &job_store_provider_dyn)
|
||||
.await;
|
||||
|
||||
// Fourth recoverable-run tenant. Iterates `storage.blobs`
|
||||
// and verifies each row against the physical backend AND
|
||||
// against the reference-counting invariants that `dedup_gc`
|
||||
// relies on. Three per-row checks (subject-iteration in
|
||||
// action): `blob_missing_from_backend` (data_loss, bytes
|
||||
// gone from disk), `refcount_mismatch` (inconsistent,
|
||||
// dedup counter drift), and `blob_corrupted` (data_loss,
|
||||
// deep mode only — bit-rot). Complements
|
||||
// `files_consistency` without doubling work: probing
|
||||
// per-unique-blob preserves dedup savings vs probing
|
||||
// per-file-chunk. See memory
|
||||
// `project_cdc_dual_storage_registries` for the rationale.
|
||||
// Fourth recoverable-run tenant. Iterates `storage.blobs` and
|
||||
// checks the reference-counting invariant `dedup_gc` relies on:
|
||||
// `refcount_mismatch` (inconsistent — an under-count lets GC reap
|
||||
// a live blob, an over-count pins a dead one), repairable under
|
||||
// `?repair=true`.
|
||||
//
|
||||
// DB-only, and takes no backend. Physical checks — missing bytes,
|
||||
// orphaned bytes, bit-rot — all belong to `backend_consistency`,
|
||||
// which merge-joins the backend enumeration against this same
|
||||
// table in one pass. This tenant used to probe the backend once
|
||||
// per row for missing bytes, which found strictly less than the
|
||||
// merge-join at N round-trips instead of one enumeration.
|
||||
let _ = Arc::new(
|
||||
crate::infrastructure::services::blobs_consistency_service::BlobsConsistencyCheck::new(
|
||||
maintenance_pool.clone(),
|
||||
core.blob_backend.clone(),
|
||||
core.config.storage_entries.clone(),
|
||||
self.storage_path.clone(),
|
||||
// Same registry instance GC reaps from — see
|
||||
// DedupService::reference_registry.
|
||||
core.dedup_service.reference_registry(),
|
||||
|
||||
Reference in New Issue
Block a user