feat: thumbnail dedup — store thumbnails by blob_hash instead of file_id (#233)
Thumbnails are now keyed by blob_hash on disk so identical files share
a single set of thumbnails (icon/preview/large). For 4000 duplicate
files with the same content, this reduces thumbnail storage from 12,000
files to just 3.
Changes:
- get_thumbnail_path() keys by blob_hash instead of file_id
- get_thumbnail(), get_cached_thumbnail(), generate_all_sizes_background()
accept blob_hash parameter for disk dedup
- generate_all_sizes_background() fast path: if blob-hash thumbnails
already exist on disk, skip image processing entirely and just
populate moka cache for the new file_id
- delete_thumbnails() only invalidates moka cache (shared disk
thumbnails must not be deleted when one file is removed)
- delete_blob_thumbnails() added for GC; garbage_collect() now cleans
up orphaned thumbnail files alongside blob files
- External thumbnails (video frames) stored as ext-{file_id}.jpg
since they are client-generated and not dedup-able
- ThumbnailPort trait updated with blob_hash parameters
- All handler call sites updated (file_handler, preview_handler)
- Tests updated for new signatures
This commit is contained in:
@@ -40,7 +40,7 @@ async fn generate_thumbnail_from_blob_path() {
|
||||
|
||||
// The key assertion: the service can read from a blob path (not a logical path)
|
||||
let result = svc
|
||||
.get_thumbnail("test-file-id", ThumbnailSize::Icon, &blob_path)
|
||||
.get_thumbnail("test-file-id", "ab1234567890", ThumbnailSize::Icon, &blob_path)
|
||||
.await;
|
||||
|
||||
let thumb_bytes = result.expect("thumbnail generation should succeed from blob path");
|
||||
@@ -67,7 +67,7 @@ async fn generate_thumbnail_nonexistent_path_returns_error() {
|
||||
|
||||
let bad_path = tmp.path().join("does-not-exist.png");
|
||||
let result = svc
|
||||
.get_thumbnail("missing-id", ThumbnailSize::Icon, &bad_path)
|
||||
.get_thumbnail("missing-id", "nonexistent-hash", ThumbnailSize::Icon, &bad_path)
|
||||
.await;
|
||||
|
||||
assert!(result.is_err(), "should fail for nonexistent file");
|
||||
|
||||
Reference in New Issue
Block a user