The KNOWN BUG pin for `GET /api/s/{folder-token}/file/{file_id}` was
stale — the route now returns 200 + body for files inside the share's
subtree and 404 for anything outside it. Replaces the sidestep
comment with two positive assertions:
- in-share: 200 + Content-Disposition references the file name
- out-of-share (caller-owned file in a different folder): 404,
matching "no such file" so the response can't be used to
enumerate foreign file ids
Coverage now exercises the actual recipient-side download path that
NC desktop and web clients use; the file-share variant (item_type=file)
moves down to test 8b.
Adds a teardown DELETE for the outsider hello.txt so the next test in
the runner (permissions.hurl) can re-upload its own hello.txt into
admin's home folder without hitting the live-name unique index.
handle_head was declaring `Content-Length: file.size` while writing
`Body::empty()` — on a keep-alive connection the client waits forever
for N bytes that never come. Hyper now derives Content-Length: 0 from
the actual body, which is honest about what's on the wire.
RFC 7231 §4.3.2 suggests HEAD return the same headers as GET, but
lying about Content-Length is worse than omitting it: NC and Sabre
clients use PROPFIND for size anyway, and curl -I (and any client
applying HEAD semantics) gets the same ETag/MIME/Last-Modified it
needs. Caught by the F6b test which uses `curl -X HEAD` to read the
current ETag before a conditional PUT.
Also adds `nc_status_propfind_depth0` to lib/dav_helpers.sh so the
F11/F11b assertions ("did the intermediate parent get auto-created?")
can compile.
When the client sends `MOVE /trashbin/{id}` with a `Destination` header,
handle_restore now resolves the destination path and returns 412
Precondition Failed if a live file or folder already sits there —
matching Sabre/DAV and the NC desktop client's expectation. There is
no `Overwrite: T` workflow for trash restore in either reference
implementation (silently replacing a live file with an undeleted one
is a footgun), so the refusal is unconditional.
The destination header is extracted at the dispatch site as an owned
String so the future stays Send-compatible (`&Request<Body>` is not
Sync because the body trait object is Send-only).
`extract_nc_subpath_from_dest` is promoted to `pub` so trashbin_handler
can share the same URL parser as handle_move.
Threads `new_name: Option<&str>` through FileWritePort::copy_file and
FileManagementUseCase::copy_file_with_perms so a same-folder
COPY /a.txt → /b.txt picks up the destination name via a single
COALESCE($3::text, name) in the CTE. Without it the new row inherits
the source's filename and collides on the (folder_id, name, user_id)
unique index — the "Already Exists" 500 M8 was hitting.
handle_copy in the native WebDAV surface now passes
`(file.name != dest_name).then(|| dest_name.into())`, keeping the
"same name in a different folder" case at None so existing semantics
are preserved.
Extends the N2/PUT lock guard introduced earlier to the rest of the
native mutator surface. Same helper, same If: capture before body
consumption, same 423-on-reject shape:
- handle_delete : check source path
- handle_proppatch : check source path
- handle_move : check source AND destination paths
- handle_copy : check destination path only (source isn't
modified by a copy)
The class-2 DAV advertisement in OPTIONS is now honest across the
full surface, not just PUT.
New tests N2c-N2f run while n-locked.txt is still LOCKed (before the
existing N3 UNLOCK). Each asserts 423 without the token and verifies
the operation didn't half-apply: file present after DELETE-423,
source untouched + no destination after MOVE-423, locked destination's
content unchanged after COPY-423.
Positive (with-token) coverage is implicit via the M-series happy-
path tests that exercise each method on unlocked resources — a
regression that hard-rejected every call would fail there too.
Closes F11.
The NC MKCOL handler previously had `mkdir -p` semantics:
sending MKCOL on /a/b/c/ where neither a nor b exists silently
created both intermediates and returned 201. Sabre/DAV and the
actual NC server both 409 on that — our auto-create deviated.
NC desktop walks ancestors one MKCOL at a time during sync so
nothing real depended on the old behaviour.
Drop the segment-walking creation loop. New flow:
target exists → 405
parent path missing → 409
parent ok, target new → 201
The race-recovery branch for the loop's per-segment create is
also gone — single parent lookup, single create, no window.
Test F11 flipped from pinned-201 to strict 409 and asserts the
intermediate parent was not silently created. F11b and F11c added
as regression guards for the success path and the 'target already
exists' case.
Closes G9.
DELETE on a folder now flips is_trashed on every descendant folder
and file under it in a single CTE pipeline (lpath <@ root.lpath
covers the whole subtree via the GiST index). Previously only the
root row was flipped — descendants stayed live, directly addressable
via their full path, and confused desktop-sync tree walks that
expected the parent-collection 404 to imply the children were gone
too.
restore_from_trash mirrors the cascade: descendants where the
original_*_parent_id column is NULL are the ones we cascade-trashed,
so they get cascade-restored too. Descendants that were independently
trashed before the parent went to trash have original_*_parent_id
set, so they stay in trash and remain visible as top-level entries
in storage.trash_items.
No schema migration needed — both original_parent_id (folders) and
original_folder_id (files) were already nullable and already encoded
'where this came from when it was independently trashed'; using NULL
as the cascade-marker reuses that existing distinction cleanly.
Test G9 flipped from KNOWN BUG to assert every descendant 404s after
the parent DELETE; new G9b proves the inverse cascade by restoring
the trashed root and verifying every descendant comes back at its
original path.
Closes M5 / M7 / M8a / M8b.
The optimized PathResolver and the read-side find_*_by_path queries
disagree on what counts as 'a path that hits a row'. After the drive-
refactor migration rewrote the path column to drop the
"My Folder - <user>/" prefix, files PUT through the WebDAV surface
stayed reachable by GET (legacy lookup) but vanished from the
optimized resolver (strict path-match). MOVE/DELETE/COPY 404'd on
every root-level file as a result.
Introduces resolve_or_legacy: optimized resolver first, then the
GET-style legacy lookups as a strict superset. Ownership is enforced
in both branches. handle_delete / handle_move / handle_copy each
collapsed from two near-identical resolver-only + legacy-only branches
into a single match using the helper — fewer lines, identical
semantics, root-level paths now resolve.
handle_copy also fixes M8b: copy_file_with_perms takes no destination
name, so a copy to a different filename in the same folder collided
with the source. After the copy, rename the new file when dest_name
differs from source name. Mirrors what handle_move already does.
Closes G4 / G5 / K5.
handle_move now resolves the destination once before the file/folder
dispatch and applies RFC 4918 §9.9.4:
- Overwrite: F on a collision → 412 Precondition Failed, source
untouched, destination untouched.
- Overwrite: T (or absent) on a collision → delete the existing
destination, then proceed → 204 No Content.
- No collision → 201 Created (unchanged).
The same destination lookup powers the 201-vs-204 status decision, so
adding the precondition guard adds zero extra DB hits on the happy
path.
handle_restore now catches the unique-index collision out of
restore_item and returns 412 instead of letting it bubble as 500.
Mirrors the G4 semantics for the trashbin surface (restore has no
Overwrite header so the refusal is unconditional; client resolves by
renaming the live file first).
Sabre/DAV's CorePlugin and our test pins agreed independently — NC
clients expect this exact behavior, and the new G5b/G5c positive-case
tests guard against a regression that hard-rejected every MOVE.
Closes N2. The native WebDAV PUT handler now consults the lock
store before accepting a write: if the target path is exclusively
locked, the request must carry the lock token in its If: header
or the server returns 423 Locked. Without a matching token, the
body is never consumed — a rejected PUT no longer wastes the
upload bandwidth or hits the CDC ingester.
Two helpers are introduced so the same enforcement plugs into
the other mutator methods (delete/move/copy/proppatch) when their
fixes land:
extract_if_header_tokens — angle-bracket-scoop view of If:
(sufficient for one-target writes;
full §10.4 tagged-list grammar would
only matter for multi-resource Ifs)
enforce_native_lock — Some(423) when locked + no/wrong
token, None otherwise
Test N2 flipped from pinned 204 to assert 423. Added N2b: same
PUT with the captured Lock-Token in If:(<...>) returns 204, so a
regression that hard-rejected every PUT would still fail loudly.
Closes F5/F6. The NC PUT handler now evaluates conditional
preconditions before body ingestion and returns 412 Precondition
Failed when they fail:
- If-None-Match: * on an existing target → 412 (create-if-absent)
- If-None-Match with matching ETag → 412 (weak compare)
- If-Match: * with no current representation → 412
- If-Match with no listed ETag strong-matching the current → 412
The lookup that drives the precondition reuses the same query the
handler already needed for the 201-vs-204 distinction, so this adds
no extra DB round-trip. Rejected requests skip body ingestion
entirely so a 412 doesn't waste megabytes of bandwidth + disk I/O.
Test F5/F6 flipped from 'pinned current 204' to assert 412, plus
mirror cases F5b/F6b/F6c/F6d covering the legitimate-success paths
so logical-operator regressions can't slip past silently.
- permit reuse of blob where file is trashed (by-hash and chunked)
- add hurl test on /api/files/by-hash
- add anti enumeration of blob (404 is always blob_not_owned_by_caller)
grid view is now displaying preview in 200px wide
icons are 150 px wide, this create a blur effec
preview is now more appropriated
the true solution would be to change icon size into 200px wide
After every upload, maybe_update_storage_usage spawned a full
`SUM(size) OVER all the user's non-trashed files` to refresh
auth.users.storage_used_bytes — O(N) in the user's file count per upload,
i.e. O(N²) for a bulk upload. (The covering index makes it index-only but
still scans N rows.)
Replace it with an O(1) incremental `storage_used_bytes += size`, keyed by the
file's owner_id (dropping the brittle "My Folder - <user>" path-parsing hack).
Deletes/trash never decremented this value — they already rely on the periodic
reconciliation sweep — so the model is unchanged: the sweep remains the
correctness backstop for every mutation, and the counter is clamped at 0.
Both stay fire-and-forget on a background task, off the upload's latency path.
Benchmarked (per-call, vs the user's existing file count N):
N=1k: full-SUM 202us vs incremental 123us
N=10k: full-SUM 1185us vs incremental 113us (10x)
N=50k: full-SUM 5397us vs incremental 114us (47x — incremental is flat O(1))
Bulk upload of 10k files (insert + usage update each):
full-SUM (O(N²)) 10.37s -> incremental (O(N)) 4.89s (>2x, diverges with scale)
https://claude.ai/code/session_01DCszkkU11LYxMEUWr4setK
save_file_with_blob_impl did three sequential DB round-trips per upload:
resolve_user_id (SELECT folders.user_id), the INSERT, then lookup_folder_path
(SELECT folders.path) — the first and third re-reading the same folders row.
Replace them with a single statement: a `parent` CTE reads the folder once and
the INSERT derives user_id from it and returns the path via the CTE, in one
round-trip. An empty CTE (folder vanished between ingest and insert) inserts
zero rows and now surfaces as a clean NotFound instead of a generic owner
error. Deadlock retry, blob-ref compensation, and the 23505 (duplicate name)
mapping are preserved; owner resolution + insert are now atomic (no TOCTOU).
What it does and doesn't buy (benchmarked, honest):
- Server CPU: UNCHANGED. A server-side 50k loop is identical (13.6s vs 13.6s)
— the two extra folder reads are cached point lookups, negligible against
the INSERT + per-statement triggers + 13 indexes.
- Client-observed latency: 2 fewer client<->DB round-trips per upload. At the
measured ~189us/round-trip on localhost that's ~0.38ms/upload; on a
networked DB (RTT 0.5-1ms) ~1-2ms/upload.
- Connection pool: the metadata phase holds a pooled connection for 1
round-trip instead of 3, freeing it ~3x sooner under upload concurrency.
So this is a latency + connection-utilization win (and an atomicity/cleanup),
not a server-CPU win. register_file_deferred keeps its own path.
https://claude.ai/code/session_01DCszkkU11LYxMEUWr4setK
storage.files carried 13 indexes; every INSERT/DELETE/rename maintains all of
them. Two are never chosen by the planner for any query the app issues —
verified statically (query text) AND empirically on a 50k-row table via
EXPLAIN + pg_stat_user_indexes over the real query shapes (idx_scan = 0):
- idx_files_name_search (user_id, name text_pattern_ops): file-name search is
`name ILIKE '%term%'` (served by the GIN trgm index); text_pattern_ops can
serve neither ILIKE, a leading-% substring, nor default-collation ORDER BY.
The one exact `name = $1` lookup is `WHERE folder_id=$1 AND name=$2`, served
by the UNIQUE (folder_id, name, user_id) index.
- idx_files_category_order (category_order): only emitted as a derived
type_order alias inside the folders⊎files UNION-ALL listing; the ORDER BY
runs post-UNION, so a single-column files index can't presort it. The real
listing uses idx_files_folder_id + a top-N sort.
Benchmark (50k single-row inserts, all triggers active): ~6% faster
(WITH: 10.46/10.71s; WITHOUT: 9.83/10.07s — every WITHOUT run beat every WITH
run) plus less disk and WAL on every file mutation. No query regression: the
planner never used these indexes. Reversible.
idx_folders_path (path text_pattern_ops) is intentionally KEPT — it serves
exact `WHERE path = $1` equality lookups.
https://claude.ai/code/session_01DCszkkU11LYxMEUWr4setK
FileContentCache (moka, 512 MiB) was keyed by the file UUID, so content that
the CDC store already deduplicates to ONE blob on disk was cached once PER
FILE in RAM: N files sharing a blob held N copies, all counting against the
512 MiB cap. With effective dedup the cache filled with duplicates and
thrashed.
Key it by the blob hash instead (already on FileDto::content_hash):
- The in-RAM cache now benefits from dedup like the disk does — each distinct
blob is cached once and shared across every file/user that references it,
so a download by user A warms the cache for user B's identical content.
- Content is immutable by hash, so entries never go stale; the existing
invalidate(file_id) calls become harmless no-ops (a UUID never matches a
hash key) and can be removed in a later cleanup.
- ETag is now the immutable content hash (strong validator).
- Guarded: a hash-less stub DTO disables caching for that request rather than
colliding every such file on the empty key.
Response Content-Type still comes from the DTO, not the cache, so keying does
not affect the served MIME (verified).
Benchmark (real moka, exact 512 MiB/weight config, 400 files x 4 MiB = 1600 MiB
working set, 4000 uniform-random accesses):
dedup 1x : file_id 35.8% hit / 2568 reads vs hash 35.6% / 2577 (no dedup -> no change; control)
dedup 5x : file_id 35.8% hit / 2570 reads vs hash 98.0% / 80 (32x fewer disk reads, RAM 512->320 MiB)
dedup 20x: file_id 35.1% hit / 2596 reads vs hash 99.5% / 20 (130x fewer disk reads, RAM 512->80 MiB)
The win scales with the dedup ratio; with no dedup it is a no-op.
https://claude.ai/code/session_01DCszkkU11LYxMEUWr4setK
Bradley Nelson