Commit Graph

29 Commits

Author SHA1 Message Date
Claude aba89c4f5d perf: eliminate N+1 hot-path queries, cache immutable lookups, stop re-compressing compressed bytes
Every change is benchmark-verified (harness + before/after numbers in
benches/, measured on this branch; reproduction commands in each doc):

DAV / sync-client hot paths
- PROPFIND dead-properties: one = ANY($1) query per 500-child page instead
  of one sequential query per child, and indexable `=` predicates instead
  of IS NOT DISTINCT FROM (seq scans). 2,000-child folder: 1.07-4.54 s of
  DB chatter -> 4-6 ms (258-773x). Applied to native + NC PROPFIND and
  both NC REPORT handlers. [benches/DEAD-PROPS.md]
- Folder paging: keyset cursor (name > $last) + new partial index
  (folder_id, name) replaces LIMIT/OFFSET full-folder rescan per page.
  Full 20k-file walk: 1266 ms -> 77 ms (16.5x). New migration
  20260917000000. [benches/PROPFIND-PAGING.md]
- NC chroot / default-drive resolution: moka caches (30 s TTL, explicit
  invalidation on drive mutations) for find_default_for_user and the
  markerless chroot FolderDto. 2 uncached queries + 2 pool checkouts per
  NC/WebDAV/WOPI request -> sub-us moka hit (p50 0.7-3.6 ms -> ~1 us).
  [benches/CHROOT-CACHE.md]
- Quota: PROPFINDs whose prop list never names a quota prop skip the
  2-query resolution entirely (wants_quota()); the remaining lookups read
  2 columns instead of the full auth.users row with its <=512 KiB avatar
  (11-16x, p50 3.4 ms -> 0.29 ms). Same narrow read now gates every
  upload quota check. [benches/QUOTA-PATH.md]

CPU on the request path
- ZIP exports (folder download, share ZIP, batch download): entries whose
  MIME says already-compressed (JPEG/MP4/zip/pdf/...) are Stored instead
  of Deflate - deflate ran inline on the tokio writer task at ~41 MB/s
  for ~0% size gain. Mixed media corpus: 4.31x wall and CPU, archive size
  unchanged. Shared predicate in common::mime_detect. [benches/ZIP-MEDIA.md]
- Compression layers: tower-http's default maps to Brotli QUALITY 11
  (verified in brotli-8.0.2 source and empirically: 90 ms per 64 KiB JSON
  response, 1.3 s per 700 KiB bundle). Both layers pinned to Precise(4):
  99x less CPU for ~15% more bytes. SPA assets are now precompressed at
  build time (scripts/precompress.mjs, 77% smaller) and served via
  ServeDir::precompressed_br/gzip: 2016x less per-request work, and
  clients get the better q11 bytes. [benches/STATIC-PRECOMPRESSED.md]

Batched / cached backend paths [benches/NPLUS1-AND-CACHES.md]
- Content-search ReBAC re-verification: new
  AuthorizationEngine::check_files_read_batch (default = old loop;
  PgAclEngine override batches drive resolution + reuses role cache).
  200 sequential point SELECTs per search -> 1-2 queries.
- Batch-ZIP subtree downloads: drop per-file re-authz + per-file Recent
  recording (2 writes/file) for subtree entries already authorized at the
  root - mirrors the native folder-download path. ~6,000 statements
  removed from a 2,000-file archive.
- CDC chunk manifests: immutable by content address, now moka-cached
  (weight-bounded 32 MiB, 60 s TTL, positive-only, invalidated on delete)
  - removes one manifest query (p50 0.44-4.4 ms) from every stream,
  range and full blob read.
- People tab: grouped COUNT + batched cover lookup instead of dragging
  every face row with its 2 KiB embedding (10k faces: 30.4 ms & 21 MB ->
  3.8 ms & 1.3 KB, 8.1x); merge() is one set-based UPDATE.
  [benches/PEOPLE-LIST.md]
- Photos timeline cursor: raw timestamptz comparison instead of
  EXTRACT(EPOCH ...) wrapper + IS NULL OR disjunction - cursor is an
  index boundary again, deep scroll stops re-scanning skipped rows.
- Public share landing: one atomic UPDATE ... access_count + 1 (was
  SELECT + full-row write-back: racy, lost updates, clobbered concurrent
  owner edits) - 3 round-trips -> 2 per visit.
- move_to_trash: dead full-entity SELECT feeding a documented no-op
  removed from both branches; dead fields dropped from TrashService.
- NFC normalization: is_nfc_quick fast path skips the decompose/recompose
  state machine for the ~100% already-NFC case (every row loaded from PG).

Frontend
- Large folders paint after page one (~200 items) via fetchFolderListing's
  new onPage hook instead of waiting for every sequential page.
- Tested-and-reverted (kept for the record): cached Intl.Collator for name
  sorts - vitest showed it 2x SLOWER than V8's argument-less localeCompare
  fast path (5.6 ms vs 12.1 ms / 5k names). Sort order untouched.

New bench harnesses under examples/ (bench feature): zip_media,
dead_props, chroot_cache, quota_path, people_list, propfind_paging,
static_precompress.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CBK1RdtzyP6759Muqe1K1w
2026-07-16 14:20:20 +00:00
M.Schmidt 6701ddfc17 Merge branch 'main' into rfc-4331-quota-properties
# Conflicts:
#	src/interfaces/nextcloud/report_handler.rs
#	src/interfaces/nextcloud/webdav_handler.rs
#	tests/api/run.sh
2026-07-13 20:32:01 +02:00
M.Schmidt 7011fdff5a Merge origin/main into webdav-litmus-compliance 2026-07-12 22:25:12 +02:00
M.Schmidt 3cb12c6fc1 refactor(readability): and reducing condition evaluation twice on same condition 2026-07-12 21:04:37 +02:00
M.Schmidt f017c700f1 feat(webdav): add RFC 4331 quota-available-bytes/quota-used-bytes properties
Threads the caller's account-wide (used, available) storage figures through
PROPFIND for the plain-file WebDAV surface, resolved once per request via
StorageUsagePort::get_user_storage_info and reused for every folder entry
in the response. Unlimited accounts (quota <= 0) omit quota-available-bytes
entirely per RFC 4331 §3, rather than disclosing a sentinel value.
Properties are only advertised as known when the quota subsystem is enabled
and the lookup succeeds; otherwise they fall through to the standard 404
propstat.
2026-07-12 20:22:57 +02:00
Edouard Vanbelle 92d1a10d45 security(quick-xml): bump version to 0.41.0
- and protect amount of properties
 - azure_core 0.21.0 is using quick-xml 0.31.0  which is Dos-able
   azure_core is no more maintained, would migrte to official azure lib
   later
2026-07-03 02:12:36 +02:00
M.Schmidt 0ad0ea1a43 fix(webdav): reject PROPPATCH on protected DAV:/oc:/nc:/ocs: props
DeadPropertyStore let PROPPATCH set any namespace/name verbatim,
incl. names the server itself emits as live state (DAV: entirely,
plus oc:/nc:/ocs: names used by write_file_response /
write_folder_response). That either forges a live prop or stores
dead rows nothing ever reads. is_protected_property() denylists
them; both PROPPATCH handlers (native + NC) now return per-property
403 instead of storing. oc:favorite stays writable via its existing
special-case, which runs before the protection check.
2026-07-01 22:54:48 +02:00
M.Schmidt f220217659 feat(nc-webdav): generic dead-props on PROPPATCH/PROPFIND/REPORT
NC PROPPATCH only special-cased oc:favorite, silently dropped every
other prop while claiming 200. PROPFIND had no dead-prop lookup at
all. Reuse native /webdav/ dead-prop plumbing (now pub(crate)) so NC
surface gets real RFC 4918 §4.2 storage. Missing resource on
PROPPATCH is now 404 instead of a fake success no-op.
2026-07-01 19:59:16 +02:00
M.Schmidt 6ca6649e50 fix(webdav): achieve 100% litmus compliance (59/59 tests)
Complete RFC 4918 litmus test suite compliance (basic 16/16,
copymove 13/13, props 30/30) by fixing six categories of failures:

- PropPatchOp: parse PROPPATCH in document order (RFC 4918 §9.2)
  so that remove-then-set and set-then-remove yield different results
- Null namespace: resolve_name now correctly maps xmlns="" to empty-ns
  QualifiedName instead of falling through to DAV: namespace
- Unicode chars: handle quick-xml 0.39's Event::GeneralRef for character
  references (&#65536;) — the parser emits these as GeneralRef, not Text
- MOVE preserves dead props: call rename_resource on MOVE, clearing
  stale destination data even when source has no registered properties
- Malformed PROPFIND: return 400 when body lacks a complete <propfind>
  element (test 2: bare <foo>, test 3: invalid xmlns:prefix="" binding)
- PROPPATCH document order: process ops via Vec<PropPatchOp> instead of
  separate (sets, removes) to honour interleaved remove/set sequences
2026-06-30 08:58:13 +02:00
M.Schmidt 1cf48b0dfa fix(webdav): RFC 4918 PROPFIND adapter compliance
- Return unknown properties in a separate 404 propstat (RFC 4918 §9.2)
  via folder_prop_is_known/file_prop_is_known classifiers
- Detect unclosed <propfind> element and return 400 Bad Request
- Add write_folder/file_entry_with_dead_props to include stored dead
  properties in PROPFIND responses
- Change write_folder/file_requested_props to accept &[&QualifiedName]
  to avoid clone overhead when using partition output
2026-06-28 23:51:38 +02:00
M.Schmidt b5afa1a855 feat(webdav): implement RFC 4918 dead property store
Add an in-memory DeadPropertyStore backed by RwLock<HashMap> that
stores arbitrary client-supplied XML properties per resource path.
Wire it into AppState so PROPPATCH can persist dead props and PROPFIND
can retrieve them across requests.
2026-06-26 21:19:03 +02:00
M.Schmidt 5913fff047 feat(webdav): RFC 4918 prop classification and 404 propstat 2026-06-24 23:08:51 +02:00
Edouard Vanbelle 0135930da9 refactor(file|folder): separate etag and blob_hash 2026-06-06 18:49:14 +02:00
Edouard Vanbelle 6e1c435098 style(server): correct clippy warnings 2026-05-04 12:58:30 +02:00
Edouard Vanbelle badf35f08f chore: remove all executable attributes on non bash files 2026-04-01 23:14:42 +02:00
Dionisio f2d35ca792 feat: auto-persist JWT secret, remove setup token requirement
- JWT secret auto-generates and persists to <STORAGE_PATH>/.jwt_secret
- Remove setup token: first admin setup is open until system initialized
- Fix schema.sql: move CREATE EXTENSION pg_trgm/ltree to top
- Update login UI and auth.js to remove setup token fields
2026-03-05 22:12:53 +01:00
zjean 54eedf5483 feat(nextcloud): add Nextcloud-compatible API layer
Implement a complete Nextcloud client compatibility layer so that
Nextcloud desktop/mobile sync clients can connect to OxiCloud.

Key additions:
- Login Flow v2 (device auth) with OIDC bridge support
- WebDAV handler compatible with Nextcloud clients (PROPFIND, GET,
  PUT, DELETE, MKCOL, MOVE, COPY, HEAD, PROPPATCH)
- OCS API endpoints (user info, capabilities, notifications stubs,
  sharees, unified search)
- Basic Auth middleware with app password verification, account
  lockout integration, and blake3-keyed auth cache
- App password management: create, list, revoke via both native
  API (JWT-authenticated profile page) and Nextcloud OCS endpoints
- Nextcloud file ID mapping (oc:fileid) with persistent DB storage
- Chunked upload support (Nextcloud v2 chunking protocol)
- Trashbin WebDAV interface
- Avatar (SVG placeholder) and preview (redirect) handlers
- User profile page with app password management UI
- URL user validation on all DAV routes (403 on mismatch)
- Database schema for app_passwords and nextcloud_object_ids tables

All services are behind a `nextcloud.enabled` config flag and
cleanly separated under src/interfaces/nextcloud/.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 20:46:07 +01:00
Claude b9de342223 fix: resolve CalDAV PROPFIND returning empty property values (#153)
Root cause: XML namespace prefixes (e.g. "D", "C") were not being
resolved to their actual namespace URIs (e.g. "DAV:",
"urn:ietf:params:xml:ns:caldav") during PROPFIND parsing. This caused
all property match arms to fall through to the catch-all, producing
empty XML elements.

Changes:
- Add namespace-aware XML parsing (collect_ns_decls + resolve_name) to
  WebDavAdapter, used by all DAV protocol parsers (WebDAV, CalDAV,
  CardDAV)
- Add /.well-known/caldav -> /caldav/ redirect (RFC 6764)
- Add root /caldav/ PROPFIND response with current-user-principal and
  calendar-home-set discovery properties
- Add /caldav/principals/{username}/ PROPFIND handler (was 500 error)
- Add /caldav/{username}/ user calendar home handler (calendar-home-set
  target)
- Respect Depth header at root: depth 0 returns only root entry, depth
  1+ includes calendar children
- Fix pre-existing TRANSCODE_POOL_THREADS test compilation error
- Add 8 new tests covering namespace resolution, discovery properties,
  and principal responses

https://claude.ai/code/session_01T49VBJSimgo28APxbucHzq
2026-03-02 20:40:41 +00:00
Dionisio 71c2cb5edb perf: Arc<AppState>, streaming PROPFIND, spawn_blocking SHA-256
- Issue #4: Wrap AppState in Arc — eliminates 42 Arc::clone + 16 String::clone per request
- Issue #2: Reject Depth:infinity with 403 + streaming XML with paginated DB queries
- Issue #5: Move chunked upload assembly (SHA-256 hash-on-write) to spawn_blocking
- Remove ~270 lines dead code from di.rs (unused builders, Default impl, stubs)
- Clean up unused tokio imports in chunked_upload_service.rs
2026-02-24 15:11:56 +01:00
Dionisio 0be7ef8c0b style: fix clippy collapsible_if + cargo fmt 2026-02-15 18:04:32 +01:00
Dionisio 1ed20f425f perf: Phase 4+5 optimizations — uploads 10x, downloads 2x, concurrent 2x. moka cache, 512KB buffers, remove sync_all, hash-on-write, preloaded queries, bench.sh v3, gitignore storage/. 500MB upload 12.6s->1.3s (392MB/s). RSS 69-113MB, 0 swap. 2026-02-15 17:56:47 +01:00
Dionisio 4c98c5a657 style: apply cargo fmt to entire codebase
Standardize code formatting across all 173 Rust source files
using rustfmt. No functional changes - purely cosmetic.

This establishes a consistent code style baseline for the
project going forward.
2026-02-14 01:29:34 +01:00
Dionisio 67137a3ef2 refactor: apply clippy auto-fixes (162 warnings resolved)
- Fix needless borrows and references
- Collapse nested if statements
- Replace manual strip_prefix with str::strip_prefix()
- Remove redundant closures in map/unwrap_or_else
- Use Iterator::next_back() instead of rev().next()
- Simplify map_or patterns
- Use std::io::Error::other() instead of new(ErrorKind::Other, ..)
- Use div_ceil() instead of manual ceiling division
- Consolidate format! string arguments
- Various other idiomatic Rust improvements

39 files changed, 220 insertions(+), 320 deletions(-)
2026-02-14 01:26:02 +01:00
Dionisio 52840e57df refactor: remove serde from domain entities for Clean Architecture compliance
- Remove Serialize/Deserialize from File, Folder, Session, User, Contact entities
- Create contact_persistence_dto.rs for JSONB persistence in infrastructure layer
- Update contact_pg_repository to use persistence DTOs
- Fix dependency on zip crate (downgrade from 7.2.0 to 2.1.0)
- Fix unused variable warnings in main.rs
- Move PathService import from domain to infrastructure
- Add missing fields to CoreServices and RepositoryServices
- Create proper service initialization in main.rs

Clean Architecture improvements:
- Domain layer no longer depends on serde framework
- Persistence concerns isolated to infrastructure layer
- TokenClaims in auth_service.rs is only exception (required for JWT)
2026-02-02 23:56:40 +01:00
DioCrafts 52d8250d51 adding card dav and cald dav 2025-04-13 01:04:04 +02:00
DioCrafts 5d67bc4d84 fix several bugs 2025-04-10 01:43:25 +02:00
DioCrafts 8f1d213526 improve postgresql performance 2025-04-09 00:21:20 +02:00
DioCrafts ba3ef2a530 adding webdav features 2025-04-04 21:31:41 +02:00
DioCrafts 868119e405 adding documentation 2025-04-04 01:48:55 +02:00