initially the recent was done client side
recent files are now directly updated on serverside when accessing a file
note: nextcloud and webdav voluntary not included
now Drive is purely a metadata
each drive has always a root folder
this model minimize Oxicloud changes, and simplify
the Drive name is simply the folder's root's name
note: owner of Drive has more permission that an owner of the root folder
Threads `new_name: Option<&str>` through FileWritePort::copy_file and
FileManagementUseCase::copy_file_with_perms so a same-folder
COPY /a.txt → /b.txt picks up the destination name via a single
COALESCE($3::text, name) in the CTE. Without it the new row inherits
the source's filename and collides on the (folder_id, name, user_id)
unique index — the "Already Exists" 500 M8 was hitting.
handle_copy in the native WebDAV surface now passes
`(file.name != dest_name).then(|| dest_name.into())`, keeping the
"same name in a different folder" case at None so existing semantics
are preserved.
Extends the N2/PUT lock guard introduced earlier to the rest of the
native mutator surface. Same helper, same If: capture before body
consumption, same 423-on-reject shape:
- handle_delete : check source path
- handle_proppatch : check source path
- handle_move : check source AND destination paths
- handle_copy : check destination path only (source isn't
modified by a copy)
The class-2 DAV advertisement in OPTIONS is now honest across the
full surface, not just PUT.
New tests N2c-N2f run while n-locked.txt is still LOCKed (before the
existing N3 UNLOCK). Each asserts 423 without the token and verifies
the operation didn't half-apply: file present after DELETE-423,
source untouched + no destination after MOVE-423, locked destination's
content unchanged after COPY-423.
Positive (with-token) coverage is implicit via the M-series happy-
path tests that exercise each method on unlocked resources — a
regression that hard-rejected every call would fail there too.
Closes M5 / M7 / M8a / M8b.
The optimized PathResolver and the read-side find_*_by_path queries
disagree on what counts as 'a path that hits a row'. After the drive-
refactor migration rewrote the path column to drop the
"My Folder - <user>/" prefix, files PUT through the WebDAV surface
stayed reachable by GET (legacy lookup) but vanished from the
optimized resolver (strict path-match). MOVE/DELETE/COPY 404'd on
every root-level file as a result.
Introduces resolve_or_legacy: optimized resolver first, then the
GET-style legacy lookups as a strict superset. Ownership is enforced
in both branches. handle_delete / handle_move / handle_copy each
collapsed from two near-identical resolver-only + legacy-only branches
into a single match using the helper — fewer lines, identical
semantics, root-level paths now resolve.
handle_copy also fixes M8b: copy_file_with_perms takes no destination
name, so a copy to a different filename in the same folder collided
with the source. After the copy, rename the new file when dest_name
differs from source name. Mirrors what handle_move already does.
Closes N2. The native WebDAV PUT handler now consults the lock
store before accepting a write: if the target path is exclusively
locked, the request must carry the lock token in its If: header
or the server returns 423 Locked. Without a matching token, the
body is never consumed — a rejected PUT no longer wastes the
upload bandwidth or hits the CDC ingester.
Two helpers are introduced so the same enforcement plugs into
the other mutator methods (delete/move/copy/proppatch) when their
fixes land:
extract_if_header_tokens — angle-bracket-scoop view of If:
(sufficient for one-target writes;
full §10.4 tagged-list grammar would
only matter for multi-resource Ifs)
enforce_native_lock — Some(423) when locked + no/wrong
token, None otherwise
Test N2 flipped from pinned 204 to assert 423. Added N2b: same
PUT with the captured Lock-Token in If:(<...>) returns 204, so a
regression that hard-rejected every PUT would still fail loudly.
Every upload surface previously wrote each byte to disk twice: the HTTP
body was spooled to a temp file (or assembled from chunk parts), then
mmap-re-read for FastCDC analysis, and finally the new chunks were
written to the blob backend. CDC could not start until the last byte
arrived, so large uploads paid receive + reread + rewrite latency.
The dedup engine now chunks, hashes and settles the stream WHILE it
arrives (fastcdc AsyncStreamCDC + incremental BLAKE3):
- Each batch of distinct chunks is pinned-or-classified by ONE
`UPDATE … RETURNING` (no check-then-bump TOCTOU; pinned chunks can't
be reclaimed mid-upload), and only chunks the store doesn't have are
written — a full dedup hit performs zero content writes.
- Durability before visibility is preserved: one batched fsync sweep,
then one batched INSERT, then the manifest. Identical concurrent
uploads are resolved at the manifest INSERT via ON CONFLICT (the
loser releases its references and becomes a dedup hit).
- A drop guard rolls back pins and surfaces written-but-unregistered
chunks to GC if the request future is cancelled mid-stream.
- MIME sniffing now peeks the first bytes in-flight; client-requested
MD5/SHA-256 checksums are computed by a stream tee — the post-upload
re-read of the assembled file is gone.
All surfaces converge on the new interfaces::upload_ingest helper:
REST multipart, WebDAV PUT, NextCloud PUT, WOPI PutFile, the dedup
endpoint, and both chunked-upload completions (which now stream their
ordered parts straight into the store instead of writing an assembled
file — chunk parts persist until finalize, so completion is genuinely
retryable). The legacy blob re-chunk migration streams from the
backend with no spool file either.
Legacy removed: store_from_file + mmap CDC analysers + temp-path
plumbing through every port (pre_computed_hash, save_file_from_temp,
update_file_content_from_temp), upload_spool + assembled-file
assembly in both chunked services, create_file/update_file byte-slice
variants (no callers), common::temp, the OXICLOUD_UPLOAD_TMPDIR
config, and the memmap2 dependency.
Verified end-to-end against PostgreSQL 16: 8 MB upload (26 chunks),
identical re-upload (dedup hit, zero writes), 3-byte edit re-upload
(26 chunks, 1 written), byte-identical downloads, Range across chunk
boundaries, concurrent identical-upload race (manifest ref 2), and
trash-empty reclaiming exactly the unshared chunk while the shared 25
survive for the edited file. The empty/sub-8KB multipart path found a
post-EOF re-poll panic in the MIME peek (fixed with fuse + regression
test).
https://claude.ai/code/session_01WdNenpnujNR2sc32XVvwfS
Two DAV-surface fixes that replicate patterns the codebase already had:
NC PROPFIND (folder case) previously loaded EVERY child via unbounded
list_files/list_folders and serialized the entire multistatus into one
Vec (~2 KB per entry — a 50k-file folder meant ~100 MB of buffer per
request, repeated constantly by sync clients). It now mirrors the native
WebDAV handler's streaming builder: children are fetched in pages of
PROPFIND_BATCH_SIZE (500), each page's favorites and oc:fileids are
resolved with two batch queries, and the XML is yielded chunk by chunk —
memory stays O(batch) and the first byte flows immediately. The
single-file PROPFIND keeps a small buffered variant; the multistatus
opening tag is factored into a shared helper so the namespace set cannot
diverge.
WebDAV GET (native and NC) ignored the Range header and never compared
the ETag it emitted, so mount-style clients (rclone, davfs2, Finder)
re-transferred whole files on every seek, resume, or revalidation. New
shared `interfaces::range_requests` helpers — same semantics as the REST
download endpoint, which now reuses the 304 helper too — give both GETs
If-None-Match → 304, Range → 206/416, and Accept-Ranges advertising.
https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
webdav encode_uri_path runs on every PROPFIND href and did
.map(...).collect::<Vec<_>>().join("/"), allocating a String per segment plus
a joined Vec. Write each utf8_percent_encode Display adapter straight into a
single preallocated String. Behavior is identical (split on '/', encode each
segment, join with '/'), including leading/trailing-slash edge cases.
subject_group list / list_with_counts each issued a second SELECT COUNT(*)
round-trip for the total. Fold it into the page query via COUNT(*) OVER() —
the pattern folder_db_repository already uses — halving the round-trips.
total_count is read from the first row and is 0 on an empty page, matching
folder_db_repository's documented convention.
https://claude.ai/code/session_01UtfkS3nZF1vrF5jNAps6wV
ensure files does not exeed OXICLOUD_MAX_UPLOAD_SIZE, prefer to deny from header rather consuming bandwidth
add OXICLOUD_DIRECT_PUT_MAX_BYTES for direct PUT (non chunked), admins can fine tune their prefered values
Large uploads (e.g. ~800 MB ISOs) could OOMKill the process, even on
dedup hits, due to three separate full-file-in-memory paths:
- NextCloud PUT (/remote.php/dav) buffered the entire body in RAM via
body::to_bytes before any dedup logic, then re-wrote and re-hashed it.
Now streams the body to a temp file with incremental BLAKE3 and goes
through update_file_streaming (shared spool helper with the native
WebDAV PUT handler); peak heap is ~one HTTP frame regardless of size.
- DedupService::store_chunks materialized every new chunk's data in a Vec
before uploading. Now reads each new chunk by positioned I/O
(read_exact_at, off the runtime via spawn_blocking) just before its
upload; peak heap bounded to ~CHUNK_UPLOAD_CONCURRENCY x CDC_MAX_CHUNK.
- The upload spool used the OS temp dir, often tmpfs/RAM in containers
where its page-cache counts against the cgroup memory limit. Add
OXICLOUD_UPLOAD_TMPDIR to point the spool at real disk.
Also collapse a pre-existing clippy collapsible_else_if in carddav_handler.
Refs #404
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* make more coherent lifecycles
* remove specific implementation on different handlers (they do not need to know existence of ThumbnailSerice nor AudioMetadataService)
* reduce risk of orphean objects
* ensure additional services are correctly wired (ex: Thumbnail generation was not covering all upload cases)
* more details on docs/architecture/file-and-blob-lifecycle.md :
```rust
// application/ports/file_lifecycle.rs
pub trait FileLifecycleHook {
fn on_file_created(file_id, blob_hash, content_type, is_new_blob);
fn on_file_updated(file_id, blob_hash, content_type);
fn on_file_copied(file_id, blob_hash, content_type, source_id)
fn on_file_deleted(file_id);
}
// application/ports/blob_lifecycle.rs
pub trait BlobLifecycleHook {
fn on_blob_created(blob_hash, content_type);
fn on_blob_deleted(blob_hash);
}
```
When file or folder rename/move operations failed in WebDAV handlers,
all errors were incorrectly converted to HTTP 500 Internal Server Error
using AppError::internal_error(). This masked specific error types:
- AlreadyExists errors should return 409 CONFLICT
- NotFound errors should return 404 NOT FOUND
- AccessDenied errors should return 403 FORBIDDEN
Changed error handling to use AppError::from() which preserves the
original DomainError type and maps to appropriate HTTP status codes.
Fixes handling of incorrect status code for file rename failure.
Adds storage quota checking to the WebDAV PUT handler, which was missing
while present in other upload handlers (regular upload and chunked upload).
The quota check happens after the file is spooled to a temp file (so we
know the exact size) but before it's moved to permanent storage. If the
quota is exceeded, the temp file is cleaned up and a 507 Insufficient
Storage error is returned.
Fixes#104
The Nextcloud Android client compares ETags before and after upload to
verify its write landed. OxiCloud was returning the stable file UUID as
the ETag, which never changed on content updates, causing false
SYNC_CONFLICT errors on every upload.
Five fixes applied:
1. Thread blob_hash (SHA-256) through File entity, FileDto, all read/write
queries, and all WebDAV/PROPFIND responses as the ETag — changes on
every content update, no DB migration needed.
2. Honor X-OC-Mtime header: parse the client-supplied mtime and use it
for updated_at via COALESCE(to_timestamp($n), NOW()).
3. Disable phantom checksum capability (preferredUploadType/supportedTypes)
that the server never actually implemented, stopping retry loops.
4. Add nc:creation_time and nc:upload_time to PROPFIND responses.
5. Return oc-etag header in chunked upload MOVE (assemble) responses.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix all clippy lints (collapsible if, clone on Copy, needless borrow,
redundant bindings, unused params) and apply rustfmt across the codebase.
Update test mocks to match Uuid-based trait signatures.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- rate_limit: cache OXICLOUD_TRUST_PROXY_HEADERS in OnceLock<bool> to avoid
syscall on every request (~500ns → ~1ns)
- auth middleware: insert Arc<CurrentUser> instead of bare CurrentUser;
all 5 extractors now clone Arc (~1ns) instead of 4 Strings (~60-100ns)
- search_service: pre-compute query.to_lowercase() once before loops,
eliminating N redundant heap allocations per search
- Add resolve_webdav_path() helper that prepends user's home folder
prefix to raw WebDAV paths before routing to handlers
- Rewrite handle_mkcol to walk path segments, creating folders with
proper parent_id chain instead of relying on path-only lookup
- Fix double path stripping in update_file_streaming where
get_parent_folder_id received an already-stripped parent path
instead of the full file path it expects
Fixes#165
- JWT secret auto-generates and persists to <STORAGE_PATH>/.jwt_secret
- Remove setup token: first admin setup is open until system initialized
- Fix schema.sql: move CREATE EXTENSION pg_trgm/ltree to top
- Update login UI and auth.js to remove setup token fields
Update share_service test impl to match upstream trait changes
(requester_id params, verify_shared_link_password returns ShareDto).
Fix map_or, collapsible_if, dead_code, too_many_arguments warnings.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1. Share password bypass (HIGH): enforce password check in get_shared_link_by_token,
verify_shared_link_password now returns ShareDto only on correct password.
2. WebDAV MOVE ownership (MEDIUM): add assert_owner on destination parent folder
for file moves in both PathResolver and legacy branches.
3. Path traversal defense-in-depth (LOW): add reject_path_traversal() to WebDAV,
CalDAV, and CardDAV handlers rejecting '..' segments at HTTP boundary.
4. Setup race condition (LOW): atomic INSERT ... ON CONFLICT DO NOTHING in
try_claim_initialization prevents duplicate admin creation.
- Add type aliases (FileRow, FolderRow, FolderRowPaginated, FolderRowOptUser) to reduce type complexity
- Simplify redundant closures in app_password_handler and webdav_handler
- Remove needless borrow in auth_handler
- Collapse nested if/let chains in login_lockout, webdav_lock, auth, rate_limit
- Box LockEntry in acquire() Err variant to fix large enum variant warning
- Rename DeviceCodeStatus::from_str to parse to avoid should_implement_trait lint
- Add #[allow(clippy::too_many_arguments)] and #[allow(clippy::result_unit_err)] where appropriate
- Convert integration_tests from cargo feature to custom cfg attribute
- Add check-cfg lint config in Cargo.toml for integration_tests cfg
Replace String with Arc<str> for fields that contain repeated static
values (mime_type, icon_class, icon_special_class, category) in
FileDto, FolderDto, and OptimizedFileContent.
These fields are computed from ~40 static lookup tables and cloned
on every request. With Arc<str>, clone becomes O(1) atomic increment
instead of O(n) heap allocation — saving thousands of allocations/s
under load.
Fields kept as String: id, name, path, folder_id, owner_id
(unique per item, rarely cloned).
Zero API impact — serde serializes Arc<str> identically to String.
https://claude.ai/code/session_01EbAFEfyJNLRmJHmmYDX3Tt
Remove async-trait dependency and use native Rust async fn in traits.
Replace Arc<dyn Trait> with Arc<ConcreteType> throughout the codebase
to enable monomorphization and eliminate dynamic dispatch overhead.
Key changes:
- Remove write-behind cache (no implementation existed)
- Fix should_transcode static method call
- Use ContactStorageAdapter directly instead of dyn AddressBookUseCase
- Clean up unused trait imports across services and DI
https://claude.ai/code/session_01EbAFEfyJNLRmJHmmYDX3Tt
Replace the double-query pattern (get_folder_by_path + get_file_by_path)
across PROPFIND, HEAD, DELETE, MOVE, and COPY handlers with a single
UNION ALL query via PathResolverService.
PG Append node short-circuits on LIMIT 1: if the folder branch matches,
the file branch is never executed. Cuts WebDAV path resolution from
2 round-trips to 1 per request.
Also adds an exists() method using EXISTS subqueries for the Overwrite
header checks in MOVE/COPY (avoids constructing full DTOs).
Legacy double-query fallback retained when PathResolver is unavailable.
- WebDAV PUT and WOPI PutFile: replace sha2::Sha256 with blake3::Hasher (~5x faster hashing, compatible with dedup service)
- Fix TEXT↔UUID JOIN anti-pattern in favorites and recent_items repos (enables PK index usage)
- Add LIMIT 500 to get_favorites query to prevent unbounded memory allocation
- Remove unused lru crate from Cargo.toml (superseded by moka)
- Replace tokio features=["full"] with explicit feature list (removes signal, process, test-util)
Edouard Vanbelle