Thumbnails are now keyed by blob_hash on disk so identical files share
a single set of thumbnails (icon/preview/large). For 4000 duplicate
files with the same content, this reduces thumbnail storage from 12,000
files to just 3.
Changes:
- get_thumbnail_path() keys by blob_hash instead of file_id
- get_thumbnail(), get_cached_thumbnail(), generate_all_sizes_background()
accept blob_hash parameter for disk dedup
- generate_all_sizes_background() fast path: if blob-hash thumbnails
already exist on disk, skip image processing entirely and just
populate moka cache for the new file_id
- delete_thumbnails() only invalidates moka cache (shared disk
thumbnails must not be deleted when one file is removed)
- delete_blob_thumbnails() added for GC; garbage_collect() now cleans
up orphaned thumbnail files alongside blob files
- External thumbnails (video frames) stored as ext-{file_id}.jpg
since they are client-generated and not dedup-able
- ThumbnailPort trait updated with blob_hash parameters
- All handler call sites updated (file_handler, preview_handler)
- Tests updated for new signatures
- Move all sync I/O (id3::Tag, mp3_duration) into spawn_blocking via
extract_metadata_blocking() to avoid stalling Tokio worker threads
- Replace fetch_all with streaming .fetch() in reextract_all_audio_metadata
for O(1) memory usage regardless of audio file count
- Consolidate get_duration_secs into extract_metadata_blocking, eliminating
redundant file open (tag was read twice before)
- Replace blocking blake3::hash() with DedupService::hash_file() in
file_upload_service (create_file, update_file). Uses spawn_blocking +
mmap_rayon instead of blocking the Tokio async worker.
- Parallelize thumbnail resize+encode with rayon par_iter() for the 3
sizes (Icon, Preview, Large) inside spawn_blocking.
Replace hardcoded DEFAULT_MAX_CONCURRENT_DECODES=4 with adaptive
max_concurrent_decodes() that uses available_parallelism()/2 (min 2).
Matches the pattern already used in image_transcode_service.
Improves thumbnail throughput on 16+ core servers by 2-4x.
Resolves conflict between main and PR #229 by applying timeout support
on top of main's drop(data) memory optimization. Changes:
- Add thumbnail_generation_ms to TimeoutConfig (default 30s)
- Add generation_timeout field to ThumbnailService
- Wrap spawn_blocking in tokio::time::timeout to prevent hanging
- Update DI to pass timeout from config
- Update tests to provide timeout parameter
https://claude.ai/code/session_015FD62aAoMYx1XBZbzPvUu8
Explicitly drop the encoded image buffer after decoding and extracting
EXIF orientation data. This reduces peak memory consumption during
thumbnail generation by the size of the original file.
The encoded data is no longer needed once the image is decoded into
a DynamicImage, but it was being held in memory until the end of the
spawn_blocking scope.
Fixes excessive memory consumption in thumbnail generation.
The database schema defines owner_id as UUID, but the Calendar entity
stored it as String. This caused a type mismatch error when creating
calendars via CalDAV clients like DAVx:
column "owner_id" is of type uuid but expression is of type text
Changes:
- Change Calendar.owner_id from String to Uuid
- Update new() and with_id() to accept Uuid
- Update owner_id() getter to return &Uuid
- Update belongs_to() to accept &Uuid
- Update calendar_storage_adapter to pass Uuid directly
- Update tests to use Uuid::new_v4()
Fixes#200
The Nextcloud Android client compares ETags before and after upload to
verify its write landed. OxiCloud was returning the stable file UUID as
the ETag, which never changed on content updates, causing false
SYNC_CONFLICT errors on every upload.
Five fixes applied:
1. Thread blob_hash (SHA-256) through File entity, FileDto, all read/write
queries, and all WebDAV/PROPFIND responses as the ETag — changes on
every content update, no DB migration needed.
2. Honor X-OC-Mtime header: parse the client-supplied mtime and use it
for updated_at via COALESCE(to_timestamp($n), NOW()).
3. Disable phantom checksum capability (preferredUploadType/supportedTypes)
that the server never actually implemented, stopping retry loops.
4. Add nc:creation_time and nc:upload_time to PROPFIND responses.
5. Return oc-etag header in chunked upload MOVE (assemble) responses.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace manual schema.sql application with sqlx's built-in migration
system. Migrations are embedded at compile time and tracked in the
_sqlx_migrations table. Pending migrations run automatically on startup.
- Move db/schema.sql → migrations/20260307000000_initial_schema.sql
- Remove apply_schema() and split_sql_statements() from db.rs
- Add run_migrations() using sqlx::migrate!() macro
- Remove docker-compose schema.sql mount (app handles it now)
- Enable sqlx "migrate" feature in Cargo.toml
Future schema changes: add a new timestamped .sql in migrations/.
Closes#190
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix all clippy lints (collapsible if, clone on Copy, needless borrow,
redundant bindings, unused params) and apply rustfmt across the codebase.
Update test mocks to match Uuid-based trait signatures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace all 3 ImageFormat::WebP encode sites with JpegEncoder q=80.
Update fast-path to detect JPEG SOI instead of RIFF/WEBP magic.
Change file extension .webp -> .jpg, Content-Type headers, and
browser toBlob. Remove unused ImageFormat import and stale comments.
The webp feature stays for DECODING uploaded WebP images.
Schema (media_sort_date denormalization):
- Add media_sort_date column to storage.files with DEFAULT created_at
- Add trigger sync_media_sort_date: when file_metadata is upserted,
copies COALESCE(captured_at, created_at) into files.media_sort_date
- Add partial index idx_files_media_timeline on (user_id, media_sort_date DESC)
WHERE NOT is_trashed AND media type -- enables Index Scan + Limit (no Sort)
- copy_folder_tree now copies media_sort_date for copied files
- Remove dead idx_file_metadata_captured (no longer needed)
Query optimization (list_media_files):
- Rewrite to use fi.media_sort_date instead of COALESCE(fm.captured_at,...)
- Eliminates LEFT JOIN file_metadata -- one fewer table touch
- Plan: Limit to Index Scan O(LIMIT) instead of Sort O(N)
Video thumbnail pre-scaling (client + server):
- JS: pre-scale canvas to max 400px before toBlob -- 22x less RAM, 15x less BW
- Rust: fast-path in store_external_thumbnail -- if payload is already
WebP with dims within max_dim, store as-is (zero decode, zero encode)
Solution A — Cache-first path:
- get_thumbnail now tries moka (RAM) → disk BEFORE any DB query
- Ownership was verified at creation time; UUIDv4 prevents enumeration
- 146-photo timeline: 0 SQL queries instead of 146 per page load
Solution C — ETag / If-None-Match short-circuit:
- Deterministic ETag 'thumb-{id}-{size}' on every response
- If browser sends matching If-None-Match → 304 with zero I/O
- Cache-Control: immutable prevents even conditional requests
Additional improvements:
- Remove moka TTL (was 600s); thumbnails are immutable, weight-only eviction
- generate_all_sizes_background populates moka after disk write;
first GET after upload served from RAM with zero disk I/O
- DB path only taken on cache miss for images needing generation
Backend:
- get_thumbnail: for non-images, serve cached thumb or 204 (not 400)
- store_external_thumbnail: validate + re-encode to WebP + persist
- get_cached_thumbnail: memory → disk lookup without generation
- PUT /api/files/{id}/thumbnail/{size} endpoint for client uploads
- ThumbnailPort trait: add get_cached_thumbnail + store_external_thumbnail
Frontend (photos.js):
- On <img> error for video tiles, use <video> + <canvas> to extract
a frame at 25% duration using browser's native codec
- Show frame immediately via blob URL
- Fire-and-forget PUT to server for permanent caching
- Subsequent visits serve cached WebP instantly (no re-extraction)
Zero server-side dependencies — CPU distributed across clients.
- folder_db_repository: cast user_id to ::text in SELECT, ::uuid in INSERT/WHERE
- file_blob_write_repository: cast ::uuid in INSERT for user_id column
- file_blob_read_repository: bind Uuid natively instead of .to_string()
Fixes runtime errors:
- column user_id is of type uuid but expression is of type text
- operator does not exist: uuid = text
- Replace sql.chars().collect::<Vec<char>>() with direct byte-level
iteration over sql.as_bytes()
- Saves ~140 KB heap allocation for 35 KB schema (4× input size)
- Remove unused _start variable
- SQL is ASCII-safe so byte comparison is sufficient for all delimiters
- Replace std::fs::read() + update_rayon() with update_mmap_rayon()
for file hashing, eliminating full-file heap allocation (500MB file
no longer needs 500MB of RAM to hash)
- Enable blake3 'mmap' feature in Cargo.toml
- Lower hash_bytes rayon threshold from 10MB to 128KB
- Remove dead constants HASH_BLOCK_SIZE and RAYON_HASH_THRESHOLD
Add column-gap to list header/items, prevent rubber-band selection from
swallowing clicks on list headers and file items, preserve rubber-band
selection after mouseup, and extract type aliases to resolve clippy
type_complexity warnings.
Backend: new GET /api/photos endpoint with cursor-based pagination that
queries image/video files sorted by EXIF captured_at (falling back to
created_at), joining file_metadata for sort dates.
Frontend: dense photo grid grouped by day with lazy-loaded thumbnails,
IntersectionObserver infinite scroll, multi-select with batch
download/delete, and a full-screen lightbox with prev/next navigation,
EXIF metadata display, and download/favorite/delete toolbar.
Includes navigation wiring, CSS (with dark theme), and i18n translations
for all 9 locales.
Extract EXIF orientation, GPS coordinates, camera info, and timestamps
from uploaded images using kamadak-exif. Store metadata in a new
file_metadata PG table. Apply EXIF orientation to thumbnail generation
so images display correctly. Add /api/files/{id}/metadata endpoint.
- dedup_service: fall back to copy+delete when rename() fails with
EXDEV (os error 18), which occurs when temp and blob dirs are on
different filesystems
- NC webdav_handler: return 405 instead of 500 when MKCOL targets an
existing folder (RFC 4918 §9.3.1)
- JWT secret auto-generates and persists to <STORAGE_PATH>/.jwt_secret
- Remove setup token: first admin setup is open until system initialized
- Fix schema.sql: move CREATE EXTENSION pg_trgm/ltree to top
- Update login UI and auth.js to remove setup token fields
- Allow async_fn_in_trait lint crate-wide (internal project, 413 warnings)
- Add integration_tests feature to Cargo.toml to fix unexpected cfg warnings
- Collapse nested if statements into single conditions (13 locations)
- Add type_complexity allows on pg repository functions (12 locations)
- Fix dead code warnings in test modules with allow attributes
- Fix E0599 by gating new_stub() for integration_tests feature
- Add result_unit_err and result_large_err allows where appropriate
- Apply rustfmt formatting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement a complete Nextcloud client compatibility layer so that
Nextcloud desktop/mobile sync clients can connect to OxiCloud.
Key additions:
- Login Flow v2 (device auth) with OIDC bridge support
- WebDAV handler compatible with Nextcloud clients (PROPFIND, GET,
PUT, DELETE, MKCOL, MOVE, COPY, HEAD, PROPPATCH)
- OCS API endpoints (user info, capabilities, notifications stubs,
sharees, unified search)
- Basic Auth middleware with app password verification, account
lockout integration, and blake3-keyed auth cache
- App password management: create, list, revoke via both native
API (JWT-authenticated profile page) and Nextcloud OCS endpoints
- Nextcloud file ID mapping (oc:fileid) with persistent DB storage
- Chunked upload support (Nextcloud v2 chunking protocol)
- Trashbin WebDAV interface
- Avatar (SVG placeholder) and preview (redirect) handlers
- User profile page with app password management UI
- URL user validation on all DAV routes (403 on mismatch)
- Database schema for app_passwords and nextcloud_object_ids tables
All services are behind a `nextcloud.enabled` config flag and
cleanly separated under src/interfaces/nextcloud/.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Since the blob storage migration (3c7c16f), thumbnail generation failed
with "No such file or directory" because the handler constructed logical
file paths that don't exist on disk. Resolve the actual blob path via
get_blob_hash() + dedup_service.blob_path() in both upload and get
thumbnail handlers. Add regression tests.
- Enable HTTP/2 auto-detection in axum (add 'http2' feature)
- Remove direct hyper dependency with 'full' features (already transitive via axum/reqwest)
- Replace std::sync::RwLock with tokio::sync::RwLock in oidc_service.rs and file_system_i18n_service.rs to prevent async deadlocks
1. Share password bypass (HIGH): enforce password check in get_shared_link_by_token,
verify_shared_link_password now returns ShareDto only on correct password.
2. WebDAV MOVE ownership (MEDIUM): add assert_owner on destination parent folder
for file moves in both PathResolver and legacy branches.
3. Path traversal defense-in-depth (LOW): add reject_path_traversal() to WebDAV,
CalDAV, and CardDAV handlers rejecting '..' segments at HTTP boundary.
4. Setup race condition (LOW): atomic INSERT ... ON CONFLICT DO NOTHING in
try_claim_initialization prevents duplicate admin creation.
- Add type aliases (FileRow, FolderRow, FolderRowPaginated, FolderRowOptUser) to reduce type complexity
- Simplify redundant closures in app_password_handler and webdav_handler
- Remove needless borrow in auth_handler
- Collapse nested if/let chains in login_lockout, webdav_lock, auth, rate_limit
- Box LockEntry in acquire() Err variant to fix large enum variant warning
- Rename DeviceCodeStatus::from_str to parse to avoid should_implement_trait lint
- Add #[allow(clippy::too_many_arguments)] and #[allow(clippy::result_unit_err)] where appropriate
- Convert integration_tests from cargo feature to custom cfg attribute
- Add check-cfg lint config in Cargo.toml for integration_tests cfg
The standalone `md5` (0.8.0) crate is replaced with `md-5` (0.10) from
the RustCrypto ecosystem, which shares `digest v0.10` with sha2, argon2,
blake2 and other crates already in the dependency tree — eliminating one
redundant implementation.
https://claude.ai/code/session_01V23pGpfNw5ujZvtwRFG6qy
Replace String with Arc<str> for fields that contain repeated static
values (mime_type, icon_class, icon_special_class, category) in
FileDto, FolderDto, and OptimizedFileContent.
These fields are computed from ~40 static lookup tables and cloned
on every request. With Arc<str>, clone becomes O(1) atomic increment
instead of O(n) heap allocation — saving thousands of allocations/s
under load.
Fields kept as String: id, name, path, folder_id, owner_id
(unique per item, rarely cloned).
Zero API impact — serde serializes Arc<str> identically to String.
https://claude.ai/code/session_01EbAFEfyJNLRmJHmmYDX3Tt
Remove async-trait dependency and use native Rust async fn in traits.
Replace Arc<dyn Trait> with Arc<ConcreteType> throughout the codebase
to enable monomorphization and eliminate dynamic dispatch overhead.
Key changes:
- Remove write-behind cache (no implementation existed)
- Fix should_transcode static method call
- Use ContactStorageAdapter directly instead of dyn AddressBookUseCase
- Clean up unused trait imports across services and DI
https://claude.ai/code/session_01EbAFEfyJNLRmJHmmYDX3Tt
Diocrafts