Commit Graph

19 Commits

Author SHA1 Message Date
Edouard Vanbelle a95a6b106c feat(username): normalize username into lowercase
- normalize username into lowercase (this is already ASCII only)
- permit users to login with their username with insensitive case
- if a disabled account is reactivated and got a collision, it will normalize it too
- server will stop on collision (ex: 2 entries with `Alice` and `alice`)
  in a such case admin can run:

```
oxicloud migrate lowercase-usernames --dry-run
```
then
```
oxicloud migrate lowercase-usernames
```
2026-09-13 19:53:09 +02:00
Edouard Vanbelle 8d6e03a4bb feat(DPoP): check requests and 401 on failure 2026-08-09 01:56:07 +02:00
Claude 8e55caa8a9 perf(round29): cache-serve borrow-probe, NC REPORT href buffer, auth per-req allocs, DB over-fetch
Seven behaviour-preserving allocation / copy / bandwidth cuts, each behind a
counting-allocator BEFORE/AFTER gate that exit(1)s unless AFTER allocates
strictly fewer than BEFORE (benches/ROUND29.md, examples/bench_round29_micro.rs).

- [B] Content-cache serve fast path (optimized_inner Tier 1 +
  get_file_range_preloaded — the video-scrub hot path): probe the cache with a
  borrow first and build the owned get_or_load args (quoted-etag / key / id
  Strings) only on a miss, instead of allocating them before every probe and
  discarding them on a hit. 6 -> 0 allocs per cache hit. Splits get_or_load into
  get + load_and_cache so the miss path is not re-probed and the hit/miss stat
  counters stay byte-identical. Also drops the unconditional content_hash/name
  clones that ran for the >=10 MB streaming tier that used neither.
- [A] NextCloud REPORT emit loops: per-row href String (and format! per folder
  row) -> one reused href_buf via nc_href_into / nc_collection_href_into with the
  URL-encoded user computed once per page. 1497 fewer allocs on a 500-row page.
- [C] read_full: a single-frame blob is returned zero-copy instead of a second
  whole-payload memcpy into a fresh BytesMut; multi-frame path unchanged.
- [D] login-lockout key: to_lowercase()+format! -> one pre-sized ASCII buffer
  (non-ASCII keeps str::to_lowercase). 3 -> 1 alloc/req, byte-identical key.
- [E] NC composite-username parse: owned clone/to_string -> &str borrow of the
  already-owned raw_username. 1 -> 0 alloc on the common no-marker path.
- [F] get_contacts_in_group: stop SELECTing the discarded multi-KB vcard column
  (the live method ROUND25 §Q2 missed; ContactDto has no vcard field).
- [G] count_admin_users: add count_users_by_role -> scalar COUNT(*) instead of
  hydrating every admin's full row (incl. up-to-512 KiB avatar + ui_preferences
  JSONB) only to .len() it, on a bootstrap-polled status endpoint.

All seven gates pass; cargo fmt --check and cargo clippy --all-features
--all-targets -D warnings clean. §F/§G additionally validated against a live
PostgreSQL 16 with the full migration set (query validity, result equivalence,
600000 -> 8 byte wire delta on the admin count).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LhpDZxSQTAGnAqCHUdtG5N
2026-07-21 10:25:36 +00:00
Claude c930f865b0 perf: round 14 — faces narrow projection, auth per-request allocs, CalDAV emit buffers, frontend set churn
Benchmark-gated (benches/ROUND14.md); every change ships a BEFORE/AFTER
benchmark with an equivalence gate and is rolled back on regression (the
rule is encoded as a GATE FAIL exit / threshold expect).

Backend
- Q1 faces_for_file → narrow face_boxes_for_file(id, person_id, bbox) with the
  caller filter pushed into SQL: drops the 2 KiB embedding BYTEA + 6 unused
  columns per face. 15-face lightbox open 0.312→0.219 ms, 32 KB→840 B/req.
- A1 cookie auth uses the borrow-only extract_cookie_str (already backs CSRF)
  instead of extract_cookie_value's owned String: -1 alloc/cookie request.
- A2 compute_relevance ASCII case-fold fast path vs name.to_lowercase() per
  result row (Unicode fallback preserved): 1.40x, 12→3 allocs/page.
- A3 sub pre-parsed to Uuid at decode time (TokenClaims.sub_id) vs re-parsing
  the 36-char claim on every request incl. cache hits: 22.7→0.7 ns.
- A4 auth + NextCloud middlewares borrow request.headers() instead of taking
  axum's HeaderMap extractor (a full map clone): 2→0 allocs/authed request.
- A5 CalDAV getlastmodified via the stack rfc2822_utc (byte-identical to
  chrono) vs a per-event to_rfc2822() heap String: 5→0 allocs.
- A6 CalDAV per-event href + quoted etag written into reused page buffers vs a
  fresh format! pair per event: 3.48x, 240→6 allocs/40-event page.

Frontend
- F1 t() shares one frozen EMPTY_PARAMS for the no-interpolation call forms vs
  a throwaway {} per call: -1 alloc/call.
- F2 favorites favoriteIds is a persistent SvelteSet with per-page add (clear
  on reset) vs a brand-new set over the whole accumulated list each page:
  22.3x over a 40-page drain (O(N^2)→O(N)).

Verified: cargo check --all-targets, cargo clippy -D warnings, both bench
packs (GATE PASS), frontend npm run check + vitest (4/4). ROUND14.md also
records the investigated-but-deferred backlog (music N+1, contact vcard
over-fetch, CachedBlobBackend syscalls, ResourceList.sections builder, etc.).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PymgCdK78NzUF3oRAQCJfN
2026-07-19 10:22:12 +00:00
Claude fdf445d2b0 perf: round 9 — decorator PUT reactivation, session/search/dedup alloc purges, PROPFIND join!, folder-level cascade
Benchmark-gated round (benches/ROUND9.md): every change carries a
BEFORE/AFTER bench with equivalence/safety gates; verdicts below are from
the committed harnesses on 4 cores / local PG 16.

Backend:
- Blob decorators (Retry/Cached) now forward put_blob_from_bytes_unsynced
  + sync_blobs — the trait default had silently reinstated HEAD-before-PUT
  per chunk on decorated remote stacks, undoing ROUND3 §8. Full production
  stack: 500 probes -> 0, 1.9x wall at 10 ms RTT (bench_s3_put §3).
- NC PROPFIND per-page enrichment triple (favorites / oc:fileid / dead
  props) overlapped with tokio::join!: 2.07x local, 2.86x at 5 ms RTT
  (bench_nc_enrich_join, injected-latency decide-by-bench).
- Search enrichment consumes its DTOs and carries the interned Arc<str>
  display fields end-to-end (SearchFileResultDto type change, OpenAPI
  shape preserved): enrich_file 2.0x, 11.6 -> 2.2 allocs/row; the NC
  REPORT conversion stops re-running all three classifiers per row
  (bench_search_enrich).
- NC session Arc end-to-end: SharedNcSession extractor (8 -> 0 allocs),
  Arc<FolderDto> chroot cache (4 -> 0/hit), single shared Arc<CurrentUser>
  + lazy span render (11 -> 6/build) (bench_nc_session).
- Storage micro-pack: atomic create_new chunk writes (2.1x fresh),
  stream_chunks over the manifest Arc (4097 -> 0 allocs/read incl. the
  Range path), manifest single-flight (herd 64 -> 1 loads), hex_lower for
  chunk Content-MD5 (18 -> 1 allocs) (bench_storage_micro).
- OCS capabilities memoized into OnceLock<[Bytes;2]>: 237x, 102 -> 0
  allocs/poll, byte-identical (bench_capabilities_static).
- Drive::is_empty COUNT(*) sum -> EXISTS: 34.4x on a 100k-file drive
  (bench_drive_is_empty).
- favorites/recents row-map ROUND7 port: path/name/blob_hash moved,
  -2.75 allocs/row (bench_resource_row_map §2).
- Folder rows decode binary UUIDs (ROUND6 §10 port): 1.03-1.07x page
  fetch, honest verdict incl. one noise-band wash documented
  (bench_folder_uuid_decode).
- Authz: file cascade decision decomposed into memoized folder-level
  decision + direct-grant lookup (ROUND8 deferred item). Cold shared-album
  first view 592 -> 418 µs/thumb; warm path unchanged; safety gates incl.
  new direct-grant sibling isolation, revoke-flush re-verified, full
  integration authz suite green (bench_thumbnail_cascade_cache).

Frontend (vitest gates committed beside the code):
- resolveLabel/resolveRecipient O(directory) scan -> id-keyed Map: 13.9x
  (recipients.bench.test.ts).
- ResourceList selection-prune effect skips when nothing is selected
  (100 -> 0 Set builds per drain) and the photos timeline reads a
  listener-fed mobile flag instead of matchMedia per recompute
  (listDerives.bench.test.ts).

Verification: cargo fmt + clippy --all-features --all-targets -D warnings
clean; 524 unit + 554 integration (--cfg integration_tests) tests pass;
frontend npm run check clean with 293 vitest tests green.

Deferred with rationale in ROUND9.md: CalDAV authz-before-fetch reorder
(maintainer sign-off), per-page batched parent resolution, JWT-claims
Arc<str>, batch_operations signature widening.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XDc9VtXvskJ6dnMRraSndn
2026-07-18 16:12:04 +00:00
Claude aba89c4f5d perf: eliminate N+1 hot-path queries, cache immutable lookups, stop re-compressing compressed bytes
Every change is benchmark-verified (harness + before/after numbers in
benches/, measured on this branch; reproduction commands in each doc):

DAV / sync-client hot paths
- PROPFIND dead-properties: one = ANY($1) query per 500-child page instead
  of one sequential query per child, and indexable `=` predicates instead
  of IS NOT DISTINCT FROM (seq scans). 2,000-child folder: 1.07-4.54 s of
  DB chatter -> 4-6 ms (258-773x). Applied to native + NC PROPFIND and
  both NC REPORT handlers. [benches/DEAD-PROPS.md]
- Folder paging: keyset cursor (name > $last) + new partial index
  (folder_id, name) replaces LIMIT/OFFSET full-folder rescan per page.
  Full 20k-file walk: 1266 ms -> 77 ms (16.5x). New migration
  20260917000000. [benches/PROPFIND-PAGING.md]
- NC chroot / default-drive resolution: moka caches (30 s TTL, explicit
  invalidation on drive mutations) for find_default_for_user and the
  markerless chroot FolderDto. 2 uncached queries + 2 pool checkouts per
  NC/WebDAV/WOPI request -> sub-us moka hit (p50 0.7-3.6 ms -> ~1 us).
  [benches/CHROOT-CACHE.md]
- Quota: PROPFINDs whose prop list never names a quota prop skip the
  2-query resolution entirely (wants_quota()); the remaining lookups read
  2 columns instead of the full auth.users row with its <=512 KiB avatar
  (11-16x, p50 3.4 ms -> 0.29 ms). Same narrow read now gates every
  upload quota check. [benches/QUOTA-PATH.md]

CPU on the request path
- ZIP exports (folder download, share ZIP, batch download): entries whose
  MIME says already-compressed (JPEG/MP4/zip/pdf/...) are Stored instead
  of Deflate - deflate ran inline on the tokio writer task at ~41 MB/s
  for ~0% size gain. Mixed media corpus: 4.31x wall and CPU, archive size
  unchanged. Shared predicate in common::mime_detect. [benches/ZIP-MEDIA.md]
- Compression layers: tower-http's default maps to Brotli QUALITY 11
  (verified in brotli-8.0.2 source and empirically: 90 ms per 64 KiB JSON
  response, 1.3 s per 700 KiB bundle). Both layers pinned to Precise(4):
  99x less CPU for ~15% more bytes. SPA assets are now precompressed at
  build time (scripts/precompress.mjs, 77% smaller) and served via
  ServeDir::precompressed_br/gzip: 2016x less per-request work, and
  clients get the better q11 bytes. [benches/STATIC-PRECOMPRESSED.md]

Batched / cached backend paths [benches/NPLUS1-AND-CACHES.md]
- Content-search ReBAC re-verification: new
  AuthorizationEngine::check_files_read_batch (default = old loop;
  PgAclEngine override batches drive resolution + reuses role cache).
  200 sequential point SELECTs per search -> 1-2 queries.
- Batch-ZIP subtree downloads: drop per-file re-authz + per-file Recent
  recording (2 writes/file) for subtree entries already authorized at the
  root - mirrors the native folder-download path. ~6,000 statements
  removed from a 2,000-file archive.
- CDC chunk manifests: immutable by content address, now moka-cached
  (weight-bounded 32 MiB, 60 s TTL, positive-only, invalidated on delete)
  - removes one manifest query (p50 0.44-4.4 ms) from every stream,
  range and full blob read.
- People tab: grouped COUNT + batched cover lookup instead of dragging
  every face row with its 2 KiB embedding (10k faces: 30.4 ms & 21 MB ->
  3.8 ms & 1.3 KB, 8.1x); merge() is one set-based UPDATE.
  [benches/PEOPLE-LIST.md]
- Photos timeline cursor: raw timestamptz comparison instead of
  EXTRACT(EPOCH ...) wrapper + IS NULL OR disjunction - cursor is an
  index boundary again, deep scroll stops re-scanning skipped rows.
- Public share landing: one atomic UPDATE ... access_count + 1 (was
  SELECT + full-row write-back: racy, lost updates, clobbered concurrent
  owner edits) - 3 round-trips -> 2 per visit.
- move_to_trash: dead full-entity SELECT feeding a documented no-op
  removed from both branches; dead fields dropped from TrashService.
- NFC normalization: is_nfc_quick fast path skips the decompose/recompose
  state machine for the ~100% already-NFC case (every row loaded from PG).

Frontend
- Large folders paint after page one (~200 items) via fetchFolderListing's
  new onPage hook instead of waiting for every sequential page.
- Tested-and-reverted (kept for the record): cached Intl.Collator for name
  sorts - vitest showed it 2x SLOWER than V8's argument-less localeCompare
  fast path (5.6 ms vs 12.1 ms / 5k names). Sort order untouched.

New bench harnesses under examples/ (bench feature): zip_media,
dead_props, chroot_cache, quota_path, people_list, propfind_paging,
static_precompress.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CBK1RdtzyP6759Muqe1K1w
2026-07-16 14:20:20 +00:00
Edouard Vanbelle 50fb34659e feat(drive): limit NC request by disk 2026-06-19 16:06:38 +02:00
Edouard Vanbelle aa155dffa0 fix_next_cloud 2026-06-19 16:06:38 +02:00
Edouard Vanbelle 42510d94f3 POC(nextcloud): add chroot on nextcloud
Bring chroot to nextcloud capability: login on Nextcloud via username="{user}~{folder-uuid}
    Doing a such login will chroot the folder folder-uuid

    If user has several folder as root (parent=None), the login flow will request which
    folder user want to chroot
2026-06-19 16:06:38 +02:00
Claude 8a42b07cbe perf(auth): cached image-free user-flags lookup for per-request guards
Every WebDAV / CalDAV / CardDAV request paid one full-row user fetch in
require_internal_user_layer just to read `is_external` (and the NC Basic
Auth middleware repeated it right after its own cache hit). That SELECT
includes the `image` column — a data URI of up to 512 KiB — so a sync
client issuing hundreds of PROPFINDs per minute dragged hundreds of MB
of avatar bytes out of Postgres to evaluate a boolean.

- New `UserFlags { role, is_external, active }` + a repo query selecting
  only those three columns (inherent method, mirroring `update_image`).
- `AuthApplicationService::get_user_flags`: moka cache, 30 s TTL,
  10k capacity. `change_user_role` / `set_user_active` invalidate
  eagerly, so admin changes still apply immediately; anything else is
  visible within the TTL — preserving the documented "no token rotation
  needed" semantics at a per-request cost of zero DB round-trips when
  warm.
- `require_internal_user`, `require_admin_user` and the NC Basic Auth
  external check now go through the flags lookup.

https://claude.ai/code/session_01Dp3oWon5GBMVn4j3QXZdgx
2026-06-10 09:27:32 +00:00
SAY-5 b9af3092be chore: remove em-dashes from comments 2026-06-09 15:11:16 -07:00
SAY-5 9dfb29bdda fix(auth): scope lockout key to (account, IP) to prevent DOS by login flood
Closes #323.

LoginLockoutService cached failed-attempt counters keyed only on
the username, so any caller that could reach the auth endpoint and
guess (or enumerate) a username could lock that account out for the
entire lockout window — the rate limiter happily lets each IP make
its share of bad-password attempts before clamping, which is enough
to trip the per-account threshold in seconds. The reporter
demonstrated a complete DOS by spoofing X-Forwarded-For with
OXICLOUD_TRUST_PROXY_HEADERS=true.

Fix: change the lockout cache key from `username` to `username|ip`.
A flood from one IP locks that IP out of that account, but a
legitimate user coming from a different IP is unaffected.

Changes:
- LoginLockoutService::{check, record_failure, record_success} take
  client_ip as a second argument; cache key is built via Self::key
  (`format!("{username}|{ip}")`).
- middleware/rate_limit.rs: factor out extract_client_ip_from_parts
  (HeaderMap + Option<&SocketAddr>) so handlers that don't take a
  full Request<B> can still derive the same client identifier
  extract_client_ip uses. extract_client_ip now delegates to it.
- auth_handler.rs login: derive client_ip from headers (the only
  signal available without ConnectInfo) and pass it through to all
  three lockout calls.
- nextcloud/basic_auth_middleware.rs: do the same with the full
  Request via extract_client_ip.

Tests:
- Updated existing 4 unit tests to thread an IP arg.
- New does_not_lock_out_other_ips_for_same_account: lock from IP1,
  assert IP2 still allowed (the #323 regression).
- New success_resets_only_the_acting_ip: a successful login from
  IP2 must NOT clear an attacker's lockout from IP1.

Verification:
- `cargo build` ✅
- `cargo test login_lockout` → 6 passed (4 existing thread an IP
  arg without behaviour change, 2 new pin the per-IP scoping).

Signed-off-by: SAY-5 <say.apm35@gmail.com>
2026-06-09 14:47:47 -07:00
Edouard Vanbelle 04a4868cd2 refactor(api): ensure trace are for all HTTP req.
- ensure Nextcloud handler has also traces
    - ensure that user_id is provisionned by the nextcloud auth method
2026-06-06 18:49:14 +02:00
Edouard Vanbelle 6d70a7000e security(external_user): protect unnecessary route access to external users 2026-06-03 00:32:00 +02:00
Edouard Vanbelle badf35f08f chore: remove all executable attributes on non bash files 2026-04-01 23:14:42 +02:00
Diocrafts 9f08460027 perf: OnceLock for env var, Arc<CurrentUser> in auth, pre-compute query lowercase
- rate_limit: cache OXICLOUD_TRUST_PROXY_HEADERS in OnceLock<bool> to avoid
  syscall on every request (~500ns → ~1ns)
- auth middleware: insert Arc<CurrentUser> instead of bare CurrentUser;
  all 5 extractors now clone Arc (~1ns) instead of 4 Strings (~60-100ns)
- search_service: pre-compute query.to_lowercase() once before loops,
  eliminating N redundant heap allocations per search
2026-03-07 11:23:56 +01:00
Dionisio 9aa35aa0ea quick fix 2026-03-06 13:18:36 +01:00
zjean 45c60faeb5 fix: resolve all clippy warnings for CI (async_fn_in_trait, collapsible_if, type_complexity, dead_code)
- Allow async_fn_in_trait lint crate-wide (internal project, 413 warnings)
- Add integration_tests feature to Cargo.toml to fix unexpected cfg warnings
- Collapse nested if statements into single conditions (13 locations)
- Add type_complexity allows on pg repository functions (12 locations)
- Fix dead code warnings in test modules with allow attributes
- Fix E0599 by gating new_stub() for integration_tests feature
- Add result_unit_err and result_large_err allows where appropriate
- Apply rustfmt formatting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 20:48:03 +01:00
zjean 54eedf5483 feat(nextcloud): add Nextcloud-compatible API layer
Implement a complete Nextcloud client compatibility layer so that
Nextcloud desktop/mobile sync clients can connect to OxiCloud.

Key additions:
- Login Flow v2 (device auth) with OIDC bridge support
- WebDAV handler compatible with Nextcloud clients (PROPFIND, GET,
  PUT, DELETE, MKCOL, MOVE, COPY, HEAD, PROPPATCH)
- OCS API endpoints (user info, capabilities, notifications stubs,
  sharees, unified search)
- Basic Auth middleware with app password verification, account
  lockout integration, and blake3-keyed auth cache
- App password management: create, list, revoke via both native
  API (JWT-authenticated profile page) and Nextcloud OCS endpoints
- Nextcloud file ID mapping (oc:fileid) with persistent DB storage
- Chunked upload support (Nextcloud v2 chunking protocol)
- Trashbin WebDAV interface
- Avatar (SVG placeholder) and preview (redirect) handlers
- User profile page with app password management UI
- URL user validation on all DAV routes (403 on mismatch)
- Database schema for app_passwords and nextcloud_object_ids tables

All services are behind a `nextcloud.enabled` config flag and
cleanly separated under src/interfaces/nextcloud/.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 20:46:07 +01:00