refactor(storage): drive blobs_consistency refcount from the registry

Completes step 1 of docs/plan/derived-blobs.md. The chunk-level
`actual_ref_count` recompute was two correlated subqueries written
inline; it now sums the registered reference sources instead, so
`blobs_consistency` and `dedup_gc` answer "what references this hash"
from one place. If they ever diverged the sweep would bless counts the
collector disagrees with — and the collector wins, destructively.

No behaviour change: the generated expression is the same legacy-files
term (guarded by NOT EXISTS) plus the same manifests-citing-this-chunk
term, and a golden test pins the whole statement byte-for-byte.

Built once at construction, like the reap statement, so the sweep runs
a fixed query per page rather than assembling SQL inside the loop. The
builder refuses an empty registry rather than emitting a query where
every blob looks unreferenced and the entire table reports
refcount_mismatch; there is a test.

DI now constructs one registry and hands the same instance to both
consumers — `DedupService::reference_registry()` is what
`BlobsConsistencyCheck` receives, so agreement is structural rather
than a convention someone has to maintain.

The long comment explaining the single-chunk double-count trap moved
from the query site to the builder's doc comment, where the NOT EXISTS
guard it describes actually lives.

fmt, clippy --all-features --all-targets and the 17 affected unit tests
all clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Edouard Vanbelle
2026-08-23 16:13:27 +02:00
parent a68c938400
commit f658e55751
3 changed files with 161 additions and 53 deletions
+21 -1
View File
@@ -440,6 +440,22 @@ impl AppServiceFactory {
// `blob_backend` into DedupService.
let blob_backend_for_consistency = blob_backend.clone();
// Every table holding blob references. Built ONCE and shared by the
// GC reap predicate and the consistency recompute so the two cannot
// disagree about what "referenced" means — a disagreement reaps live
// content. New blob-owning tables register here.
// See docs/plan/derived-blobs.md.
let blob_reference_registry = {
use crate::infrastructure::repositories::pg::blob_reference_sources::{
ChunksReferenceSource, FilesReferenceSource,
};
let mut registry =
crate::application::ports::blob_reference_ports::BlobReferenceRegistry::new();
registry.register(Arc::new(FilesReferenceSource::new(db_pool.clone())));
registry.register(Arc::new(ChunksReferenceSource::new(db_pool.clone())));
Arc::new(registry)
};
// Deduplication service — PRIMARY blob storage engine (PostgreSQL-backed index)
let dedup_service = Arc::new(
crate::infrastructure::services::dedup_service::DedupService::new(
@@ -447,7 +463,8 @@ impl AppServiceFactory {
db_pool.clone(),
maintenance_pool.clone(),
)
.with_blob_lifecycle(blob_lifecycle),
.with_blob_lifecycle(blob_lifecycle)
.with_reference_registry(blob_reference_registry.clone()),
);
dedup_service.initialize().await?;
@@ -1493,6 +1510,9 @@ impl AppServiceFactory {
core.blob_backend.clone(),
core.config.storage_entries.clone(),
self.storage_path.clone(),
// Same registry instance GC reaps from — see
// DedupService::reference_registry.
core.dedup_service.reference_registry(),
),
)
.register_recoverable_job(&core.job_registry, &job_store_provider_dyn)