Commit Graph

51 Commits

Author SHA1 Message Date
Claude 992bdae898 perf: round 22 — hot-GET HeaderMap borrow, native-WebDAV/CalDAV etag borrowed quotes, FileDto content_hash move, CalendarEvent stamp, ShareItemType case-fold
Benchmark-gated, same rule as ROUND2-21: every change ships with a
BEFORE/AFTER counting-allocator benchmark and a byte/-value equivalence
gate; an AFTER that fails to reduce allocations exits non-zero (rollback).
See benches/ROUND22.md and examples/bench_round22_micro.rs. All arms
no-Postgres.

- H1: the hot GET handlers (get_thumbnail, download_file, list_files_query,
  list_photos, NextCloud preview, public-share download/access) take
  `req: Request` last and read `req.headers()` by borrow instead of axum's
  HeaderMap extractor, whose FromRequestParts impl clones the whole request
  header table just to read 1-3 headers (the ROUND14 §A4 middleware pattern,
  finally propagated to the handlers). 2 -> 0 allocs/req · 9.95x wall.
- W1: native WebDAV write_etag_quoted — the etag emitter for every /webdav/
  PROPFIND row (per file AND per folder, up to 500/page) — emits the quotes
  as borrowed pre-escaped " text events instead of escaping a "{etag}"
  String (the ROUND20 §C1 / ROUND21 §R4 pattern). 3 -> 0 allocs/row.
- C1: CalDAV getetag routed through a shared write_quoted_etag helper across
  all 5 sites (3 per-event + 2 per-calendar); the now-dead etag: &mut String
  buffer threaded through write_event_response/standard/requested props + the
  two per-page buffers removed. 2 -> 0 allocs/row.
- D1: FileDto::from reuses the moved parts.blob_hash instead of cloning it
  via the content_hash() getter (the ROUND19/20 move-not-clone sweep missed
  it — hash/etag are read before into_parts()). Per file row of every
  listing. 1 -> 0 allocs/row.
- E1: CalendarEvent::update_time_range/update_all_day stamp timed
  DTSTART/DTEND via fmt::compact_ical_utc stack render (chrono fallback out
  of range) instead of the %Y%m%dT%H%M%SZ strftime interpreter. 4 -> 0.
- S1: ShareItemType::try_from uses eq_ignore_ascii_case instead of a
  throwaway to_lowercase() String. 1 -> 0 allocs/parse.

Verified: cargo clippy --features bench --all-targets -D warnings clean,
cargo fmt --all --check clean, cargo test --lib --features bench = 529
passed / 0 failed (incl. the OpenAPI-spec-validity test guarding the H1
utoipa-handler signature change).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKyQ4AnYtgp1JtjzweyMeo
2026-07-20 13:48:47 +00:00
Claude 77f13ac643 perf: round 21 — CalDAV/CardDAV row-mapper pre-size, dedup hash-bind & digest-key dedup, CardDAV etag/BDAY emit, NC trashbin content-type
Round 21 of the benchmark-gated perf sweep. Six behaviour-preserving,
allocation-reducing changes, each with a BEFORE/AFTER counting-allocator
section in examples/bench_round21_micro.rs and a byte/-value equivalence
gate; all six pass their deterministic alloc gate (a non-winning AFTER
exits 1 = rollback).

- R1: pre-size the 16 CalDAV/CardDAV row-mapper Vecs (+1 HashMap) with
  Vec::with_capacity(rows.len()) — the ROUND20 §I1 file-side pattern
  extended to the calendar/contact repos it deferred. 7 → 1 allocs/op.
- R2: settle_batch binds a borrowed Vec<&str> instead of cloning every
  chunk hash into a Vec<String> (sqlx encodes &[&str] as text[]
  identically; favorites_pg_repository.rs:271 precedent). 33 → 1 allocs/op,
  39x wall.
- R3: store_loose_chunks keys its intra-request dedup set on the raw
  [u8;32] BLAKE3 digest and moves the hex on a duplicate (the ROUND17 §D2
  pattern applied to the delta-upload sibling). 401 → 209 allocs/op.
- R4: CardDAV getetag emits borrowed pre-escaped &quot; quotes via a shared
  write_quoted_etag helper (ROUND20 §C1 pattern, all 4 CardDAV etag sites).
  3 → 0 allocs/op.
- R5: BDAY stamped via the new fmt::compact_date stack renderer instead of
  chrono's strftime interpreter (chrono fallback out of the 4-digit-year
  range; byte-identical, unit-tested vs chrono). 2 → 0 allocs/op, 10.5x wall.
- R6: NC trashbin folder content-type via Cow::Borrowed instead of
  .to_string() on the constant (ROUND16 §M1 pattern). 1 → 0 allocs/op.

See benches/ROUND21.md for the full write-up and the deferred-items list
(HeaderMap-clone hot handlers, Query→typed-struct, WebDAV dead-props
HashSet, and others surfaced by the audit that want their own validated
pass). Validated: cargo fmt, cargo clippy --features bench --all-targets
-D warnings, cargo test --lib --features test_utils (529 passed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015gHVq5Wy2TzdWeSqtEmK6m
2026-07-20 08:48:42 +00:00
Claude 9754aecfa9 perf: round 19 — auth/WOPI/vCard/PROPFIND per-request & per-row alloc cuts
Benchmark-gated (examples/bench_round19_micro.rs, benches/ROUND19.md): every
section ships a BEFORE/AFTER counting-allocator arm with a byte/-value
equivalence gate and a GATE-FAIL-rollback exit. All eight pass. No Postgres.

- M1 verify_basic_auth cache key: blake3::hash(format!("{u}:{p}")) → incremental
  Hasher (byte-identical key, 2→0 allocs on every Basic-auth DAV request)
- M2 WopiTokenService: prebuild Validation/DecodingKey/EncodingKey in new()
  instead of per-call (mirrors JwtTokenService; 16→12 allocs/validate)
- V1/V2 vCard emit (contact_to_vcard/generate_vcard): FN fallback drops the
  throwaway to_string, NOTE skips the escape copy for newline-free notes, REV
  uses new common::fmt::compact_ical_utc stack renderer (11.5× vs chrono
  strftime, 3→0 allocs); per-contact 9→4 allocs
- M4 trash_service::row_to_item_dto: move name/path/blob_hash out of the owned
  row instead of cloning (3 clones/file row gone)
- M5 search cache key: Uuid::hyphenated().encode_lower stack buffer instead of
  to_string (identical u64 key, 1→0 allocs/request)
- M6 streaming PROPFIND: reuse one href buffer across the page instead of a
  format! per child (native + NC handlers; 192→3 allocs on a 64-child page)
- M7 nextcloud extract_url_user: return Cow instead of forcing into_owned
  (zero-alloc on the common ASCII-username path)

common::fmt::compact_ical_utc added with chrono-parity unit tests (CASES +
60-year sweep). cargo fmt + clippy --all-targets clean; 526 lib unit tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ront9bk7YMoffVQkGG47gh
2026-07-19 22:29:48 +00:00
Claude c207e66f4b perf: round 17 — dedup ingest/verify hash-clone purge, CardDAV vCard TYPE tokens
Targets the content-addressable dedup write path (the ROUND15-deferred
"dedup_service hash-String re-allocations") from both ends — the streaming
ingest loop and the delta-commit verification read — plus a CardDAV vCard
micro-cut. Every change is benchmark-gated with a hard rollback rule; no
PostgreSQL needed for any arm (benches/ROUND17.md).

Backend (counting-allocator, examples/bench_round17_micro.rs):
- D2 chunk-ingest (store_from_stream, the hottest write path — every chunk of
  every upload): the 64-char hex hash String was allocated 3x per chunk
  (to_hex + chunk_hashes clone + session_seen insert-clone, the last dropped on
  a duplicate). The intra-upload dedup set now keys on the raw 32-byte BLAKE3
  digest ([u8;32], Copy, no heap) and the manifest push is branch-split so a
  duplicate moves the hex in: 3 -> 2 allocs/new chunk, 3 -> 1/duplicate.
  Measured 214 -> 149 allocs/op (1.14x wall) on a 64-chunk 1-in-2-dup batch;
  smaller/faster set too (32B inline keys vs 64B heap Strings).
- D1 hash_chunk_sequence (delta-commit verification): took chunks by
  &[(String,u64)] and fed the backend stream with iter().cloned(), re-cloning
  every chunk hash a second time on top of the owned Vec the caller already
  built. Take the Vec by value + into_iter(): 65 -> 0 internal allocs/op, ~2.5us
  of clone work removed per verify.
- V1 vCard TYPE tokens (contact_to_vcard + generate_vcard, 5 sites): each
  EMAIL/TEL/ADR TYPE= param used ty.to_uppercase() — a throw-away String per
  token per contact. New shared fmt::push_upper writes the upper-cased chars
  straight into the buffer (byte-identical to str::to_uppercase, unit-tested):
  13 -> 5 allocs/op, 1.19x wall.

Gates: each section asserts byte/-value equivalence (D2 the ordered manifest +
sizes + write-set; D1 the removed clone is a pure copy; V1 the full vCard) and
exits non-zero if an AFTER arm fails to reduce allocations. push_upper is
unit-tested byte-equal to str::to_uppercase (fmt::tests). Verified end-to-end:
cargo fmt clean, clippy --release --all-targets --features bench -D warnings
clean, and the harness prints GATE PASS against the built release lib.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XMmt7vNETYUbEG3Hc17LDx
2026-07-19 19:33:41 +00:00
Claude c930f865b0 perf: round 14 — faces narrow projection, auth per-request allocs, CalDAV emit buffers, frontend set churn
Benchmark-gated (benches/ROUND14.md); every change ships a BEFORE/AFTER
benchmark with an equivalence gate and is rolled back on regression (the
rule is encoded as a GATE FAIL exit / threshold expect).

Backend
- Q1 faces_for_file → narrow face_boxes_for_file(id, person_id, bbox) with the
  caller filter pushed into SQL: drops the 2 KiB embedding BYTEA + 6 unused
  columns per face. 15-face lightbox open 0.312→0.219 ms, 32 KB→840 B/req.
- A1 cookie auth uses the borrow-only extract_cookie_str (already backs CSRF)
  instead of extract_cookie_value's owned String: -1 alloc/cookie request.
- A2 compute_relevance ASCII case-fold fast path vs name.to_lowercase() per
  result row (Unicode fallback preserved): 1.40x, 12→3 allocs/page.
- A3 sub pre-parsed to Uuid at decode time (TokenClaims.sub_id) vs re-parsing
  the 36-char claim on every request incl. cache hits: 22.7→0.7 ns.
- A4 auth + NextCloud middlewares borrow request.headers() instead of taking
  axum's HeaderMap extractor (a full map clone): 2→0 allocs/authed request.
- A5 CalDAV getlastmodified via the stack rfc2822_utc (byte-identical to
  chrono) vs a per-event to_rfc2822() heap String: 5→0 allocs.
- A6 CalDAV per-event href + quoted etag written into reused page buffers vs a
  fresh format! pair per event: 3.48x, 240→6 allocs/40-event page.

Frontend
- F1 t() shares one frozen EMPTY_PARAMS for the no-interpolation call forms vs
  a throwaway {} per call: -1 alloc/call.
- F2 favorites favoriteIds is a persistent SvelteSet with per-page add (clear
  on reset) vs a brand-new set over the whole accumulated list each page:
  22.3x over a 40-page drain (O(N^2)→O(N)).

Verified: cargo check --all-targets, cargo clippy -D warnings, both bench
packs (GATE PASS), frontend npm run check + vitest (4/4). ROUND14.md also
records the investigated-but-deferred backlog (music N+1, contact vcard
over-fetch, CachedBlobBackend syscalls, ResourceList.sections builder, etc.).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PymgCdK78NzUF3oRAQCJfN
2026-07-19 10:22:12 +00:00
Claude 221c1f31b0 perf: round 11 — StoragePath joined-only, classifier fusion, memoized bodies, query-shape pack, SPA fine-grained stars
Backend (each change benchmark-gated with BEFORE replicas + equivalence
gates; see examples/bench_round11_micro.rs, bench_round11_queries.rs,
bench_log_writer.rs and benches/ROUND11.md — final numbers land in the
follow-up doc commit):

- StoragePath re-representation: single canonical joined String, segments
  derived on demand; File/Folder drop the duplicated path_string field
  (4000→1000 allocs per 500-row listing page)
- Display classifier fusion: classify_display shares one stack-lowered
  extension across the three decision trees; call sites in FileDto,
  folder/favorites/recent handlers, trash, path-resolver (+ interning
  where Arc::from was still used)
- /status.php and /openapi.json memoized into OnceLock<Bytes> (openapi
  rebuilt a 171 KiB spec per request: 2.8 ms → 18 ns)
- NC upload-session PROPFIND: write! + pre-sized body + stack RFC2822
  dates (2.3-2.6x, 2582→772 allocs at 256 chunks)
- REST download: dead FileDto clone removed (capture mime/size + move)
- CalendarEventDto/TrashedItem into_parts moves (11 KiB ical_data memcpy
  gone per CalDAV row); CardDAV getlastmodified stack render
- 4xx path: borrowed ErrorResponse serialize, ErrorKind::as_str,
  not_found/already_exists clone kill
- vCard emit via write!; search page moved out with into_iter skip/take;
  content-hit UUIDs parsed once; group last-user check via HashSet
- RateLimiter: lock-free get + insert (and_upsert_with variant REJECTED
  by benchmark); CSRF token borrow-compare + borrowed cookie extraction
- Thumbnail/preview ETags built from as_str (Debug-identical bytes)
- Encrypted backend: encrypt_in_place_detached single-buffer write path,
  chunk-sized reserve in collect_stream; retry labels made lazy
- PG: deferred upload registration 3→1 round-trips (persist_file CTE
  template); direct_grant_cache for Calendar/AddressBook/Playlist authz
  (single-flight + set_role/clear_role invalidation); expand_user
  tokio::join!; geo clusters min(uuid)::text; recluster face assignment
  batched into one UNNEST update
- People recluster cosine: norms precomputed once (bit-identical gate)
- NC capabilities poll logs demoted to debug; tracing-appender dep added
  for the log-writer benchmark

Frontend:
- ResourceList.selectedEntries O(N)-per-toggle → id-index projection
  O(k log k); favorites/recent consume the batchToolbar snippet param and
  drop their duplicate filter + dead selectedIds mirror
- Recent: star state via new favoriteIds prop — a star click no longer
  rebuilds all N entries
- admin timeAgo >30d fallback uses the cached Intl.DateTimeFormat
- vitest gates in src/lib/components/round11.bench.test.ts

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7
2026-07-18 22:02:00 +00:00
Claude 9729f033b2 perf: round 6 backend — CardDAV cursor streaming, borrowed NC id chain, binary UUID decode, one-alloc hex
Benchmark-gated (equivalence + BEFORE/AFTER in examples/bench_*, results
and reproduce commands in benches/ROUND6.md):

- CardDAV whole-book REPORT + depth-1 PROPFIND stream through a PG
  cursor (stream_contacts_by_book, 500-contact pages) instead of
  materialising every vCard twice: 8 000 contacts TTFB 37.4 → 7.6 ms
  (4.9x), peak heap 19.0 → 7.0 MiB (2.7x), wall -23%; REPORT and
  PROPFIND byte-identical to the buffered writers.

- NC numeric-id chain fully borrowed: get_or_create_file_ids/folder_ids
  take &[&str] and return HashMap<Uuid, i64>; batch_resolve_ids callers
  (PROPFIND pages, REPORT, trashbin, OCS search) pass id slices and look
  up via nc_id_of. 2.006 → 0.006 allocs/child (334x), 1.53x wall per
  500-child page. batch_check_favorites binds &[&str] as text[].

- file_blob_read_repository listing SELECTs drop id::text/folder_id::text
  server casts: rows decode binary Uuid (16 vs 36 bytes on the wire) and
  render once in row_to_file. A/B on 500-row pages: 1.225 → 1.044 ms
  mean (1.17x), p95 1.686 → 1.345 (bench_uuid_text_cast; single-row,
  param and min() sites left as-is deliberately).

- IncrementalHasher::finalize_hex renders through common::fmt::hex_lower
  instead of one format! per digest byte: 18 → 1 (md5) / 35 → 1 (sha256)
  allocs per chunk finalize, 14-15x wall.

- Share landing overlaps the access-count UPDATE with the unlock fetch
  via tokio::join! (one round-trip off every public link hit).

- REJECTED by benchmark and reverted: try_join_all fan-out of the
  batch-favorites authz pre-check — 42.6 → 56.4 ms cold, 0.15 → 0.23 ms
  warm against local-socket PG (bench_favorites_authz kept as evidence).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
2026-07-18 09:03:33 +00:00
Claude 63cf6646d0 perf: round 5 — CalDAV cursor streaming, SPA interning gaps, NC href prefix, per-request micro-allocs
Seven benchmark-gated changes (benches/ROUND5.md; BEFORE/AFTER bench +
equivalence gate each, rollback rule as ROUND2-4 — two intermediate
CalDAV shapes measured worse and were themselves rolled back before
shipping):

- CalDAV whole-calendar responses (REPORT no-range/sync-collection,
  depth-1 collection PROPFIND, .ics GET): buffered double-residency →
  ONE window-ordered scan (MIN(start_time) OVER (PARTITION BY ical_uid))
  streamed through a PG cursor, pages cut at UID boundaries. TTFB
  23.3→11.0 ms (2.1x), peak heap 14.2→8.0 MiB at 4k events / 45→24 MiB
  at 12k, wall +9-15% (documented trade, ZIP-streaming class); both
  multistatus and ICS byte-identical to the buffered output. Rejected
  shapes kept in the doc: per-page GROUP-BY keyset (3-4x wall) and
  per-uid ANY hydration (~20 µs/index descent).
- SPA listing interning gaps: folder/recent/favorites resources handlers
  (and the WebDAV pseudo-root) called raw Arc::from per row for the
  closed display set ROUND3 interned — now intern_display/intern_mime,
  4→0 allocs/row, byte-identical Arc contents.
- NC PROPFIND child hrefs: username + parent path encoded once per
  request instead of per child (543→165 ns/row, 13→4 allocs); native
  WebDAV href drops its intermediate encode String.
- suggest enrichment: entity clone + field re-clones per keystroke row →
  consume + move (166.5→126.8 µs/200 rows, 20→7 allocs/row).
- list_readable_by returns the cache's Arc (246→128 ns warm hit, 4→0
  allocs) — deep Vec clone per DAV-selector request removed.
- CardDAV REPORT: borrowed props, reused href buffer, exact-size etag
  quoting (3.04→2.34 ms per 5k-contact getetag poll).
- Auth span records: user_id.to_string() per request ×3 →
  tracing::field::display.

Checks: cargo fmt, clippy --all-features --all-targets -D warnings,
cargo test --workspace (523 passed). Follow-ups (CardDAV streaming,
&[&str] id batches, ::text UUID casts A/B, share-landing join) recorded
in benches/ROUND5.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
2026-07-17 15:19:00 +00:00
Claude 12dc648cff perf: round 4 — one-pass row paths, drive-selector cache, CalDAV single-parse, streamed Azure, batched hydration
Nine benchmark-gated changes (benches/ROUND4.md; every one ships with a
BEFORE/AFTER bench + equivalence gate, rollback rule as ROUND2/3):

- Row→entity path build: one-pass StoragePath::from_folder_and_name /
  from_joined + normalize_storage_name_owned + alloc-free Display —
  743→417 ns/file-row (1.78x), −5 allocs/row on every listing surface.
- WebDAV drive-selector: per-user readable_cache (single-flight, 30 s
  TTL, explicit invalidation incl. membership + group changes) replaces
  the grants join per request — 441 µs → 0.8 µs (~550x), 0 queries warm.
- CalDAV from_ical/update_ical_data: 8 full IcalParser runs per VEVENT
  → 1 (7.1x per PUT, 4.4x on 50-event imports); alloc-free split_vevents,
  chunk scan without the whole-body uppercase copy (1.4x), borrowed-key
  UID grouping (1.3x), REPORT props no longer cloned.
- PROPFIND emit: partition Vecs dropped (single-pass 404 list) + stack
  rendered RFC 3339/2822 dates, sizes, quoted etags (common::fmt,
  chrono-byte-identical, sweep-tested) on both DAV surfaces — 1.22x
  per page, 17.9→12.0 allocs/row.
- Grant-listing hydration: calendars/address books/playlists batch
  hydrate via = ANY($1) — 15 serial queries → 1 (~13x per sync poll).
- user-flags cache: get→insert → try_get_with single-flight (32→1
  queries per cold herd).
- Azure downloads: whole-blob Vec buffering → streamed SDK pages —
  TTFB 349→4 ms (87x), peak heap 480→1.9 MiB (254x) on 256 MiB blobs;
  new OXICLOUD_AZURE_ENDPOINT_URL override (Azurite/bench hook).
- Face indexing: unbounded per-image tokio::spawn → core-count
  semaphore, permit before blob read — peak heap 1175→176 MiB (6.7x).

Checks: cargo fmt, clippy --all-features --all-targets -D warnings,
cargo test --workspace (523 passed) + --features test_utils. hurl API
suite and dockerized integration DB not runnable in this environment —
left to CI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
2026-07-17 13:48:37 +00:00
Claude cd4c62042a perf: keyset/LATERAL SQL shapes, auth+blob-cache single-flight, spool buffers, DTO interning
Round 3 of benchmark-gated optimizations (benches/ROUND3.md; every change
gated by a before/after benchmark — an AFTER that did not beat its BEFORE
was to be rolled back; none needed it. Equivalence gates assert identical
row sequences / byte-identical output on every behavior-preserving rewrite):

DB hot paths (local PG16, EXPLAIN-verified):
- Web-UI listing (list_resources_paged): cursor pushed INSIDE the
  folders/files UNION-ALL branches as sargable row-value comparisons with
  per-branch ORDER/LIMIT + two partial expression indexes
  (folder_id, LOWER(name), id). 20k-entry folder: 26.6 -> 1.3 ms/page
  (19.5x); other sort modes at parity or better. New migration
  20260918000000. [benches/LISTING-KEYSET.md section in ROUND3]
- Photos timeline (list_media_files): per-drive CROSS JOIN LATERAL top-N
  on the timeline index, joins moved above the top-N. 50k-photo library:
  97.4 -> 1.6 ms/page (55.7x). The old "LIMIT stops the scan early"
  comment was refuted by EXPLAIN.
- PROPFIND sub-folders (both DAV surfaces): keyset list_folders_batch off
  idx_folders_unique_name replaces COUNT(*) OVER() + LIMIT/OFFSET
  (5k dirs: 79.7 -> 17.9 ms full walk, 4.5x).

Concurrency:
- Basic-auth cache single-flight (moka try_get_with): 8 concurrent DAV
  connections at TTL expiry paid 8 Argon2id runs (2.6 s CPU + 8x64 MiB);
  now 1 (300 ms). Failed verifications remain uncached.
- CachedBlobBackend per-hash single-flight + unique tmp names: 16
  concurrent cold readers = 16 full remote downloads racing truncating
  writes on ONE deterministic .tmp (corruptible cache); now 1 download
  (16x less egress, 2.8x wall on a shared link) and torn files can never
  be renamed into the cache.

I/O and allocations:
- Chunk-assembly reads 64K -> 512K buffers (2.3x, 8x fewer syscalls);
  chunk-spool writes via BufWriter 512K (5.6x, 32x fewer syscalls).
- S3/Azure put_blob_from_bytes_unsynced overrides: dedup settle no longer
  pays a HEAD probe per new chunk (2 RTT -> 1, 1.8x); Azure stops copying
  every chunk (Bytes -> Body, -0.44 ms - 4 MiB alloc per 4 MiB chunk).
- Entity->DTO mapping: Arc<str> interning of closed-set display fields +
  common MIMEs, 1-alloc etag/size formatting, FolderDto moves instead of
  clones. File row: 11 -> 4 allocs; folder row: 11.8 -> 1 (2.1x faster).
- CardDAV REPORT: deleted dead per-contact vCard pre-generation and the
  O(N^2) uid scan whose result was discarded (5k contacts: 55.7 -> 5.7 ms,
  9.8x); byte-identical XML asserted.
- Search-results cache: byte weigher + 32 MiB budget
  (OXICLOUD_SEARCH_CACHE_MAX_BYTES) replaces the 1000-ENTRY cap that let
  ~300 MiB of enriched rows sit in RSS; read latency parity.
- Dropped aws-config + aws-smithy-types (zero references; -82 dep-graph
  nodes, three SDK stacks gone from every build). tokio "process" is now
  an explicit feature (was enabled transitively by aws-config).

Frontend:
- Cached Intl.DateTimeFormat keyed by (locale, options) in formatDate and
  4 sibling callsites: 20k dates 2612 -> 51 ms (51.6x); vitest gate
  asserts output identity across locales and a 3x floor.

Validation: cargo fmt + clippy --all-features --all-targets -D warnings
clean; 518 unit + 548 integration-cfg tests green; new-shape endpoints
smoke-tested end-to-end over HTTP (all 5 listing sort modes with cursor
walks, WebDAV PROPFIND Depth-1, photos timeline, Basic-auth DAV login);
frontend npm run check clean, new vitest gates green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EBsU2qEzny3A8WQUEuMNCr
2026-07-17 11:10:27 +00:00
Claude aba89c4f5d perf: eliminate N+1 hot-path queries, cache immutable lookups, stop re-compressing compressed bytes
Every change is benchmark-verified (harness + before/after numbers in
benches/, measured on this branch; reproduction commands in each doc):

DAV / sync-client hot paths
- PROPFIND dead-properties: one = ANY($1) query per 500-child page instead
  of one sequential query per child, and indexable `=` predicates instead
  of IS NOT DISTINCT FROM (seq scans). 2,000-child folder: 1.07-4.54 s of
  DB chatter -> 4-6 ms (258-773x). Applied to native + NC PROPFIND and
  both NC REPORT handlers. [benches/DEAD-PROPS.md]
- Folder paging: keyset cursor (name > $last) + new partial index
  (folder_id, name) replaces LIMIT/OFFSET full-folder rescan per page.
  Full 20k-file walk: 1266 ms -> 77 ms (16.5x). New migration
  20260917000000. [benches/PROPFIND-PAGING.md]
- NC chroot / default-drive resolution: moka caches (30 s TTL, explicit
  invalidation on drive mutations) for find_default_for_user and the
  markerless chroot FolderDto. 2 uncached queries + 2 pool checkouts per
  NC/WebDAV/WOPI request -> sub-us moka hit (p50 0.7-3.6 ms -> ~1 us).
  [benches/CHROOT-CACHE.md]
- Quota: PROPFINDs whose prop list never names a quota prop skip the
  2-query resolution entirely (wants_quota()); the remaining lookups read
  2 columns instead of the full auth.users row with its <=512 KiB avatar
  (11-16x, p50 3.4 ms -> 0.29 ms). Same narrow read now gates every
  upload quota check. [benches/QUOTA-PATH.md]

CPU on the request path
- ZIP exports (folder download, share ZIP, batch download): entries whose
  MIME says already-compressed (JPEG/MP4/zip/pdf/...) are Stored instead
  of Deflate - deflate ran inline on the tokio writer task at ~41 MB/s
  for ~0% size gain. Mixed media corpus: 4.31x wall and CPU, archive size
  unchanged. Shared predicate in common::mime_detect. [benches/ZIP-MEDIA.md]
- Compression layers: tower-http's default maps to Brotli QUALITY 11
  (verified in brotli-8.0.2 source and empirically: 90 ms per 64 KiB JSON
  response, 1.3 s per 700 KiB bundle). Both layers pinned to Precise(4):
  99x less CPU for ~15% more bytes. SPA assets are now precompressed at
  build time (scripts/precompress.mjs, 77% smaller) and served via
  ServeDir::precompressed_br/gzip: 2016x less per-request work, and
  clients get the better q11 bytes. [benches/STATIC-PRECOMPRESSED.md]

Batched / cached backend paths [benches/NPLUS1-AND-CACHES.md]
- Content-search ReBAC re-verification: new
  AuthorizationEngine::check_files_read_batch (default = old loop;
  PgAclEngine override batches drive resolution + reuses role cache).
  200 sequential point SELECTs per search -> 1-2 queries.
- Batch-ZIP subtree downloads: drop per-file re-authz + per-file Recent
  recording (2 writes/file) for subtree entries already authorized at the
  root - mirrors the native folder-download path. ~6,000 statements
  removed from a 2,000-file archive.
- CDC chunk manifests: immutable by content address, now moka-cached
  (weight-bounded 32 MiB, 60 s TTL, positive-only, invalidated on delete)
  - removes one manifest query (p50 0.44-4.4 ms) from every stream,
  range and full blob read.
- People tab: grouped COUNT + batched cover lookup instead of dragging
  every face row with its 2 KiB embedding (10k faces: 30.4 ms & 21 MB ->
  3.8 ms & 1.3 KB, 8.1x); merge() is one set-based UPDATE.
  [benches/PEOPLE-LIST.md]
- Photos timeline cursor: raw timestamptz comparison instead of
  EXTRACT(EPOCH ...) wrapper + IS NULL OR disjunction - cursor is an
  index boundary again, deep scroll stops re-scanning skipped rows.
- Public share landing: one atomic UPDATE ... access_count + 1 (was
  SELECT + full-row write-back: racy, lost updates, clobbered concurrent
  owner edits) - 3 round-trips -> 2 per visit.
- move_to_trash: dead full-entity SELECT feeding a documented no-op
  removed from both branches; dead fields dropped from TrashService.
- NFC normalization: is_nfc_quick fast path skips the decompose/recompose
  state machine for the ~100% already-NFC case (every row loaded from PG).

Frontend
- Large folders paint after page one (~200 items) via fetchFolderListing's
  new onPage hook instead of waiting for every sequential page.
- Tested-and-reverted (kept for the record): cached Intl.Collator for name
  sorts - vitest showed it 2x SLOWER than V8's argument-less localeCompare
  fast path (5.6 ms vs 12.1 ms / 5k names). Sort order untouched.

New bench harnesses under examples/ (bench feature): zip_media,
dead_props, chroot_cache, quota_path, people_list, propfind_paging,
static_precompress.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CBK1RdtzyP6759Muqe1K1w
2026-07-16 14:20:20 +00:00
Edouard Vanbelle cb6c29a063 fix(caldav): fix generation of events
keep information of: ATTENDEE, ORGANIZER, CATEGORIES, STATUS, TRANSP, VALARM, X-*

    this fix answer in all calldav GET
2026-07-15 07:55:31 +02:00
Edouard Vanbelle a67fcadeea fix(ical): fix timerange issue
accept iCal DATE-TIME + RFC 3339 fallback
2026-07-14 23:13:50 +02:00
Edouard Vanbelle 7966c7178a fix(528): pass3: PUT with RECURRENCE-ID 2026-07-14 20:42:18 +02:00
M.Schmidt 6701ddfc17 Merge branch 'main' into rfc-4331-quota-properties
# Conflicts:
#	src/interfaces/nextcloud/report_handler.rs
#	src/interfaces/nextcloud/webdav_handler.rs
#	tests/api/run.sh
2026-07-13 20:32:01 +02:00
M.Schmidt 7011fdff5a Merge origin/main into webdav-litmus-compliance 2026-07-12 22:25:12 +02:00
M.Schmidt 3cb12c6fc1 refactor(readability): and reducing condition evaluation twice on same condition 2026-07-12 21:04:37 +02:00
M.Schmidt f017c700f1 feat(webdav): add RFC 4331 quota-available-bytes/quota-used-bytes properties
Threads the caller's account-wide (used, available) storage figures through
PROPFIND for the plain-file WebDAV surface, resolved once per request via
StorageUsagePort::get_user_storage_info and reused for every folder entry
in the response. Unlimited accounts (quota <= 0) omit quota-available-bytes
entirely per RFC 4331 §3, rather than disclosing a sentinel value.
Properties are only advertised as known when the quota subsystem is enabled
and the lookup succeeds; otherwise they fall through to the standard 404
propstat.
2026-07-12 20:22:57 +02:00
Edouard Vanbelle 92d1a10d45 security(quick-xml): bump version to 0.41.0
- and protect amount of properties
 - azure_core 0.21.0 is using quick-xml 0.31.0  which is Dos-able
   azure_core is no more maintained, would migrte to official azure lib
   later
2026-07-03 02:12:36 +02:00
Edouard Vanbelle 37467ed9d3 feat(drive): remove all owner_id from {File,Folder}Dto 2026-07-03 01:13:53 +02:00
M.Schmidt 0ad0ea1a43 fix(webdav): reject PROPPATCH on protected DAV:/oc:/nc:/ocs: props
DeadPropertyStore let PROPPATCH set any namespace/name verbatim,
incl. names the server itself emits as live state (DAV: entirely,
plus oc:/nc:/ocs: names used by write_file_response /
write_folder_response). That either forges a live prop or stores
dead rows nothing ever reads. is_protected_property() denylists
them; both PROPPATCH handlers (native + NC) now return per-property
403 instead of storing. oc:favorite stays writable via its existing
special-case, which runs before the protection check.
2026-07-01 22:54:48 +02:00
M.Schmidt f220217659 feat(nc-webdav): generic dead-props on PROPPATCH/PROPFIND/REPORT
NC PROPPATCH only special-cased oc:favorite, silently dropped every
other prop while claiming 200. PROPFIND had no dead-prop lookup at
all. Reuse native /webdav/ dead-prop plumbing (now pub(crate)) so NC
surface gets real RFC 4918 §4.2 storage. Missing resource on
PROPPATCH is now 404 instead of a fake success no-op.
2026-07-01 19:59:16 +02:00
M.Schmidt 6ca6649e50 fix(webdav): achieve 100% litmus compliance (59/59 tests)
Complete RFC 4918 litmus test suite compliance (basic 16/16,
copymove 13/13, props 30/30) by fixing six categories of failures:

- PropPatchOp: parse PROPPATCH in document order (RFC 4918 §9.2)
  so that remove-then-set and set-then-remove yield different results
- Null namespace: resolve_name now correctly maps xmlns="" to empty-ns
  QualifiedName instead of falling through to DAV: namespace
- Unicode chars: handle quick-xml 0.39's Event::GeneralRef for character
  references (&#65536;) — the parser emits these as GeneralRef, not Text
- MOVE preserves dead props: call rename_resource on MOVE, clearing
  stale destination data even when source has no registered properties
- Malformed PROPFIND: return 400 when body lacks a complete <propfind>
  element (test 2: bare <foo>, test 3: invalid xmlns:prefix="" binding)
- PROPPATCH document order: process ops via Vec<PropPatchOp> instead of
  separate (sets, removes) to honour interleaved remove/set sequences
2026-06-30 08:58:13 +02:00
M.Schmidt 1cf48b0dfa fix(webdav): RFC 4918 PROPFIND adapter compliance
- Return unknown properties in a separate 404 propstat (RFC 4918 §9.2)
  via folder_prop_is_known/file_prop_is_known classifiers
- Detect unclosed <propfind> element and return 400 Bad Request
- Add write_folder/file_entry_with_dead_props to include stored dead
  properties in PROPFIND responses
- Change write_folder/file_requested_props to accept &[&QualifiedName]
  to avoid clone overhead when using partition output
2026-06-28 23:51:38 +02:00
M.Schmidt b5afa1a855 feat(webdav): implement RFC 4918 dead property store
Add an in-memory DeadPropertyStore backed by RwLock<HashMap> that
stores arbitrary client-supplied XML properties per resource path.
Wire it into AppState so PROPPATCH can persist dead props and PROPFIND
can retrieve them across requests.
2026-06-26 21:19:03 +02:00
M.Schmidt 5913fff047 feat(webdav): RFC 4918 prop classification and 404 propstat 2026-06-24 23:08:51 +02:00
Claude fe852d3b79 fix(dav): repair CalDAV/CardDAV client connectivity (#480)
Standard CalDAV/CardDAV clients (Thunderbird, DAVx5, Apple
Calendar/Contacts) failed to connect, mounted collections read-only, or
could not discover address books, even though curl worked. Three
protocol-compliance gaps caused this:

1. Missing Basic-auth challenge on /caldav and /carddav.
   The 401 returned for these surfaces carried no `WWW-Authenticate`
   header (only /webdav did). Spec-compliant clients never send
   credentials preemptively the way `curl -u` does — they wait for the
   challenge — so Thunderbird never authenticated and failed with
   "discovery failed" / 401. Extend the challenge to all DAV surfaces via
   shared `is_dav_path` / `dav_basic_auth_challenge` helpers.

2. Calendars always advertised read-only.
   The `current-user-privilege-set` write gate compared `owner_id`
   against the literal string "current_user_id", which never matched a
   real UUID, so `<D:write/>` was never emitted and clients mounted every
   calendar read-only. Thread the caller's id through the CalDAV adapter
   and grant write when the caller owns the calendar.

3. CardDAV discovery was incomplete.
   There was no `/.well-known/carddav` route and the root PROPFIND
   exposed neither `current-user-principal` nor `addressbook-home-set`,
   so clients could not locate address books. Add the well-known redirect
   and root/principal discovery responses mirroring the CalDAV adapter.

Adds unit tests for the auth challenge predicate, the calendar
owner/non-owner privilege split, and the CardDAV root/principal discovery
responses.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016cVV9nRQjP6G6a8zbNUWMw
2026-06-19 08:38:27 +00:00
Bradley Nelson b3e1e42e93 Clean up 2026-06-17 00:28:10 -06:00
Bradley Nelson 803150635c add frontend 2026-06-16 21:26:36 -06:00
Bradley Nelson 87d68c5b6f init plugins 2026-06-16 17:57:57 -06:00
Claude 6a27c742d4 Fix CalDAV/CardDAV multiget loading entire collections into memory
A REPORT multiget for a handful of resources previously listed the
whole calendar/address book (every row incl. ical_data/vcard) and
filtered by href in Rust with an O(N×M) substring scan. Large
collections paid full-table latency, RAM and DB CPU on every sync.

- Add find_events_by_ical_uids / get_contacts_by_uids through every
  layer (domain repo trait → PG repo → storage port/adapter → use case
  → handler) using one indexed `= ANY($2)` query, mirroring the
  existing single-UID lookups.
- Extract UIDs from multiget hrefs with a shared, tested
  uid_from_multiget_href helper (percent-decoding, case-insensitive
  extension strip, collection hrefs rejected). Exact UID matching also
  removes the false positives the old substring filter allowed.
- CalDAV PROPFIND single-event path now uses the existing indexed
  get_event_by_ical_uid instead of listing all events.
- Honour the already-documented limit/offset query params on
  GET /api/address-books/{id}/contacts: optional ListQuery fields
  thread through ContactUseCase::list_contacts to a paginated repo
  query. Omitted params keep returning the full book (frontend and
  DAV listing/sync paths unchanged).

https://claude.ai/code/session_0193Hff42gaA962wThxMGSd1
2026-06-11 09:21:35 +00:00
Edouard Vanbelle 0135930da9 refactor(file|folder): separate etag and blob_hash 2026-06-06 18:49:14 +02:00
Edouard Vanbelle 6e1c435098 style(server): correct clippy warnings 2026-05-04 12:58:30 +02:00
Edouard Vanbelle badf35f08f chore: remove all executable attributes on non bash files 2026-04-01 23:14:42 +02:00
Dionisio f2d35ca792 feat: auto-persist JWT secret, remove setup token requirement
- JWT secret auto-generates and persists to <STORAGE_PATH>/.jwt_secret
- Remove setup token: first admin setup is open until system initialized
- Fix schema.sql: move CREATE EXTENSION pg_trgm/ltree to top
- Update login UI and auth.js to remove setup token fields
2026-03-05 22:12:53 +01:00
zjean 54eedf5483 feat(nextcloud): add Nextcloud-compatible API layer
Implement a complete Nextcloud client compatibility layer so that
Nextcloud desktop/mobile sync clients can connect to OxiCloud.

Key additions:
- Login Flow v2 (device auth) with OIDC bridge support
- WebDAV handler compatible with Nextcloud clients (PROPFIND, GET,
  PUT, DELETE, MKCOL, MOVE, COPY, HEAD, PROPPATCH)
- OCS API endpoints (user info, capabilities, notifications stubs,
  sharees, unified search)
- Basic Auth middleware with app password verification, account
  lockout integration, and blake3-keyed auth cache
- App password management: create, list, revoke via both native
  API (JWT-authenticated profile page) and Nextcloud OCS endpoints
- Nextcloud file ID mapping (oc:fileid) with persistent DB storage
- Chunked upload support (Nextcloud v2 chunking protocol)
- Trashbin WebDAV interface
- Avatar (SVG placeholder) and preview (redirect) handlers
- User profile page with app password management UI
- URL user validation on all DAV routes (403 on mismatch)
- Database schema for app_passwords and nextcloud_object_ids tables

All services are behind a `nextcloud.enabled` config flag and
cleanly separated under src/interfaces/nextcloud/.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 20:46:07 +01:00
Dionisio efcf88c4d7 style: cargo fmt --all 2026-03-03 01:49:18 +01:00
Claude b9de342223 fix: resolve CalDAV PROPFIND returning empty property values (#153)
Root cause: XML namespace prefixes (e.g. "D", "C") were not being
resolved to their actual namespace URIs (e.g. "DAV:",
"urn:ietf:params:xml:ns:caldav") during PROPFIND parsing. This caused
all property match arms to fall through to the catch-all, producing
empty XML elements.

Changes:
- Add namespace-aware XML parsing (collect_ns_decls + resolve_name) to
  WebDavAdapter, used by all DAV protocol parsers (WebDAV, CalDAV,
  CardDAV)
- Add /.well-known/caldav -> /caldav/ redirect (RFC 6764)
- Add root /caldav/ PROPFIND response with current-user-principal and
  calendar-home-set discovery properties
- Add /caldav/principals/{username}/ PROPFIND handler (was 500 error)
- Add /caldav/{username}/ user calendar home handler (calendar-home-set
  target)
- Respect Depth header at root: depth 0 returns only root entry, depth
  1+ includes calendar children
- Fix pre-existing TRANSCODE_POOL_THREADS test compilation error
- Add 8 new tests covering namespace resolution, discovery properties,
  and principal responses

https://claude.ai/code/session_01T49VBJSimgo28APxbucHzq
2026-03-02 20:40:41 +00:00
Dionisio 71c2cb5edb perf: Arc<AppState>, streaming PROPFIND, spawn_blocking SHA-256
- Issue #4: Wrap AppState in Arc — eliminates 42 Arc::clone + 16 String::clone per request
- Issue #2: Reject Depth:infinity with 403 + streaming XML with paginated DB queries
- Issue #5: Move chunked upload assembly (SHA-256 hash-on-write) to spawn_blocking
- Remove ~270 lines dead code from di.rs (unused builders, Default impl, stubs)
- Clean up unused tokio imports in chunked_upload_service.rs
2026-02-24 15:11:56 +01:00
Dionisio 0be7ef8c0b style: fix clippy collapsible_if + cargo fmt 2026-02-15 18:04:32 +01:00
Dionisio 1ed20f425f perf: Phase 4+5 optimizations — uploads 10x, downloads 2x, concurrent 2x. moka cache, 512KB buffers, remove sync_all, hash-on-write, preloaded queries, bench.sh v3, gitignore storage/. 500MB upload 12.6s->1.3s (392MB/s). RSS 69-113MB, 0 swap. 2026-02-15 17:56:47 +01:00
Dionisio 4c98c5a657 style: apply cargo fmt to entire codebase
Standardize code formatting across all 173 Rust source files
using rustfmt. No functional changes - purely cosmetic.

This establishes a consistent code style baseline for the
project going forward.
2026-02-14 01:29:34 +01:00
Dionisio 67137a3ef2 refactor: apply clippy auto-fixes (162 warnings resolved)
- Fix needless borrows and references
- Collapse nested if statements
- Replace manual strip_prefix with str::strip_prefix()
- Remove redundant closures in map/unwrap_or_else
- Use Iterator::next_back() instead of rev().next()
- Simplify map_or patterns
- Use std::io::Error::other() instead of new(ErrorKind::Other, ..)
- Use div_ceil() instead of manual ceiling division
- Consolidate format! string arguments
- Various other idiomatic Rust improvements

39 files changed, 220 insertions(+), 320 deletions(-)
2026-02-14 01:26:02 +01:00
Dionisio 5bf1e4b607 chore: migrate to Rust Edition 2024
- Update edition from 2021 to 2024 in Cargo.toml
- Remove explicit `ref` bindings in pattern matches (di.rs, carddav_adapter.rs)
  Edition 2024 uses implicit ref binding modes
- Refactor folder_handler.rs: change `impl IntoResponse` return types to
  concrete `axum::response::Response` to avoid lifetime capture issues
  (Edition 2024 captures all in-scope lifetimes in `impl Trait`)
- All 101 tests pass, zero warnings
2026-02-13 23:00:16 +01:00
Dionisio ef9ed2cc31 feat: complete CalDAV (RFC 4791) and CardDAV (RFC 6352) implementation
- CalDAV: MKCALENDAR, PROPFIND, PUT/GET/DELETE events, REPORT calendar-query
- CardDAV: MKCOL, PROPFIND, PUT/GET/DELETE vCards, REPORT addressbook-query
- Fix routing: move CalDAV/CardDAV to top-level merge() with explicit routes
- Fix DB schema: VARCHAR(36) -> UUID for entity IDs, vcard_data -> vcard
- Fix 15 repository stub methods that returned empty results
- Fix vCard parser in ContactStorageAdapter (was hardcoded stub)
- All operations tested end-to-end in Docker (201/207/200/204 as expected)
2026-02-10 18:46:59 +01:00
Dionisio 52840e57df refactor: remove serde from domain entities for Clean Architecture compliance
- Remove Serialize/Deserialize from File, Folder, Session, User, Contact entities
- Create contact_persistence_dto.rs for JSONB persistence in infrastructure layer
- Update contact_pg_repository to use persistence DTOs
- Fix dependency on zip crate (downgrade from 7.2.0 to 2.1.0)
- Fix unused variable warnings in main.rs
- Move PathService import from domain to infrastructure
- Add missing fields to CoreServices and RepositoryServices
- Create proper service initialization in main.rs

Clean Architecture improvements:
- Domain layer no longer depends on serde framework
- Persistence concerns isolated to infrastructure layer
- TokenClaims in auth_service.rs is only exception (required for JWT)
2026-02-02 23:56:40 +01:00
DioCrafts 52d8250d51 adding card dav and cald dav 2025-04-13 01:04:04 +02:00
DioCrafts 5d67bc4d84 fix several bugs 2025-04-10 01:43:25 +02:00
DioCrafts 8f1d213526 improve postgresql performance 2025-04-09 00:21:20 +02:00
DioCrafts ba3ef2a530 adding webdav features 2025-04-04 21:31:41 +02:00