Commit Graph

28 Commits

Author SHA1 Message Date
Claude 221c1f31b0 perf: round 11 — StoragePath joined-only, classifier fusion, memoized bodies, query-shape pack, SPA fine-grained stars
Backend (each change benchmark-gated with BEFORE replicas + equivalence
gates; see examples/bench_round11_micro.rs, bench_round11_queries.rs,
bench_log_writer.rs and benches/ROUND11.md — final numbers land in the
follow-up doc commit):

- StoragePath re-representation: single canonical joined String, segments
  derived on demand; File/Folder drop the duplicated path_string field
  (4000→1000 allocs per 500-row listing page)
- Display classifier fusion: classify_display shares one stack-lowered
  extension across the three decision trees; call sites in FileDto,
  folder/favorites/recent handlers, trash, path-resolver (+ interning
  where Arc::from was still used)
- /status.php and /openapi.json memoized into OnceLock<Bytes> (openapi
  rebuilt a 171 KiB spec per request: 2.8 ms → 18 ns)
- NC upload-session PROPFIND: write! + pre-sized body + stack RFC2822
  dates (2.3-2.6x, 2582→772 allocs at 256 chunks)
- REST download: dead FileDto clone removed (capture mime/size + move)
- CalendarEventDto/TrashedItem into_parts moves (11 KiB ical_data memcpy
  gone per CalDAV row); CardDAV getlastmodified stack render
- 4xx path: borrowed ErrorResponse serialize, ErrorKind::as_str,
  not_found/already_exists clone kill
- vCard emit via write!; search page moved out with into_iter skip/take;
  content-hit UUIDs parsed once; group last-user check via HashSet
- RateLimiter: lock-free get + insert (and_upsert_with variant REJECTED
  by benchmark); CSRF token borrow-compare + borrowed cookie extraction
- Thumbnail/preview ETags built from as_str (Debug-identical bytes)
- Encrypted backend: encrypt_in_place_detached single-buffer write path,
  chunk-sized reserve in collect_stream; retry labels made lazy
- PG: deferred upload registration 3→1 round-trips (persist_file CTE
  template); direct_grant_cache for Calendar/AddressBook/Playlist authz
  (single-flight + set_role/clear_role invalidation); expand_user
  tokio::join!; geo clusters min(uuid)::text; recluster face assignment
  batched into one UNNEST update
- People recluster cosine: norms precomputed once (bit-identical gate)
- NC capabilities poll logs demoted to debug; tracing-appender dep added
  for the log-writer benchmark

Frontend:
- ResourceList.selectedEntries O(N)-per-toggle → id-index projection
  O(k log k); favorites/recent consume the batchToolbar snippet param and
  drop their duplicate filter + dead selectedIds mirror
- Recent: star state via new favoriteIds prop — a star click no longer
  rebuilds all N entries
- admin timeAgo >30d fallback uses the cached Intl.DateTimeFormat
- vitest gates in src/lib/components/round11.bench.test.ts

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7
2026-07-18 22:02:00 +00:00
Claude fdf445d2b0 perf: round 9 — decorator PUT reactivation, session/search/dedup alloc purges, PROPFIND join!, folder-level cascade
Benchmark-gated round (benches/ROUND9.md): every change carries a
BEFORE/AFTER bench with equivalence/safety gates; verdicts below are from
the committed harnesses on 4 cores / local PG 16.

Backend:
- Blob decorators (Retry/Cached) now forward put_blob_from_bytes_unsynced
  + sync_blobs — the trait default had silently reinstated HEAD-before-PUT
  per chunk on decorated remote stacks, undoing ROUND3 §8. Full production
  stack: 500 probes -> 0, 1.9x wall at 10 ms RTT (bench_s3_put §3).
- NC PROPFIND per-page enrichment triple (favorites / oc:fileid / dead
  props) overlapped with tokio::join!: 2.07x local, 2.86x at 5 ms RTT
  (bench_nc_enrich_join, injected-latency decide-by-bench).
- Search enrichment consumes its DTOs and carries the interned Arc<str>
  display fields end-to-end (SearchFileResultDto type change, OpenAPI
  shape preserved): enrich_file 2.0x, 11.6 -> 2.2 allocs/row; the NC
  REPORT conversion stops re-running all three classifiers per row
  (bench_search_enrich).
- NC session Arc end-to-end: SharedNcSession extractor (8 -> 0 allocs),
  Arc<FolderDto> chroot cache (4 -> 0/hit), single shared Arc<CurrentUser>
  + lazy span render (11 -> 6/build) (bench_nc_session).
- Storage micro-pack: atomic create_new chunk writes (2.1x fresh),
  stream_chunks over the manifest Arc (4097 -> 0 allocs/read incl. the
  Range path), manifest single-flight (herd 64 -> 1 loads), hex_lower for
  chunk Content-MD5 (18 -> 1 allocs) (bench_storage_micro).
- OCS capabilities memoized into OnceLock<[Bytes;2]>: 237x, 102 -> 0
  allocs/poll, byte-identical (bench_capabilities_static).
- Drive::is_empty COUNT(*) sum -> EXISTS: 34.4x on a 100k-file drive
  (bench_drive_is_empty).
- favorites/recents row-map ROUND7 port: path/name/blob_hash moved,
  -2.75 allocs/row (bench_resource_row_map §2).
- Folder rows decode binary UUIDs (ROUND6 §10 port): 1.03-1.07x page
  fetch, honest verdict incl. one noise-band wash documented
  (bench_folder_uuid_decode).
- Authz: file cascade decision decomposed into memoized folder-level
  decision + direct-grant lookup (ROUND8 deferred item). Cold shared-album
  first view 592 -> 418 µs/thumb; warm path unchanged; safety gates incl.
  new direct-grant sibling isolation, revoke-flush re-verified, full
  integration authz suite green (bench_thumbnail_cascade_cache).

Frontend (vitest gates committed beside the code):
- resolveLabel/resolveRecipient O(directory) scan -> id-keyed Map: 13.9x
  (recipients.bench.test.ts).
- ResourceList selection-prune effect skips when nothing is selected
  (100 -> 0 Set builds per drain) and the photos timeline reads a
  listener-fed mobile flag instead of matchMedia per recompute
  (listDerives.bench.test.ts).

Verification: cargo fmt + clippy --all-features --all-targets -D warnings
clean; 524 unit + 554 integration (--cfg integration_tests) tests pass;
frontend npm run check clean with 293 vitest tests green.

Deferred with rationale in ROUND9.md: CalDAV authz-before-fetch reorder
(maintainer sign-off), per-page batched parent resolution, JWT-claims
Arc<str>, batch_operations signature widening.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XDc9VtXvskJ6dnMRraSndn
2026-07-18 16:12:04 +00:00
Claude 63cf6646d0 perf: round 5 — CalDAV cursor streaming, SPA interning gaps, NC href prefix, per-request micro-allocs
Seven benchmark-gated changes (benches/ROUND5.md; BEFORE/AFTER bench +
equivalence gate each, rollback rule as ROUND2-4 — two intermediate
CalDAV shapes measured worse and were themselves rolled back before
shipping):

- CalDAV whole-calendar responses (REPORT no-range/sync-collection,
  depth-1 collection PROPFIND, .ics GET): buffered double-residency →
  ONE window-ordered scan (MIN(start_time) OVER (PARTITION BY ical_uid))
  streamed through a PG cursor, pages cut at UID boundaries. TTFB
  23.3→11.0 ms (2.1x), peak heap 14.2→8.0 MiB at 4k events / 45→24 MiB
  at 12k, wall +9-15% (documented trade, ZIP-streaming class); both
  multistatus and ICS byte-identical to the buffered output. Rejected
  shapes kept in the doc: per-page GROUP-BY keyset (3-4x wall) and
  per-uid ANY hydration (~20 µs/index descent).
- SPA listing interning gaps: folder/recent/favorites resources handlers
  (and the WebDAV pseudo-root) called raw Arc::from per row for the
  closed display set ROUND3 interned — now intern_display/intern_mime,
  4→0 allocs/row, byte-identical Arc contents.
- NC PROPFIND child hrefs: username + parent path encoded once per
  request instead of per child (543→165 ns/row, 13→4 allocs); native
  WebDAV href drops its intermediate encode String.
- suggest enrichment: entity clone + field re-clones per keystroke row →
  consume + move (166.5→126.8 µs/200 rows, 20→7 allocs/row).
- list_readable_by returns the cache's Arc (246→128 ns warm hit, 4→0
  allocs) — deep Vec clone per DAV-selector request removed.
- CardDAV REPORT: borrowed props, reused href buffer, exact-size etag
  quoting (3.04→2.34 ms per 5k-contact getetag poll).
- Auth span records: user_id.to_string() per request ×3 →
  tracing::field::display.

Checks: cargo fmt, clippy --all-features --all-targets -D warnings,
cargo test --workspace (523 passed). Follow-ups (CardDAV streaming,
&[&str] id batches, ::text UUID casts A/B, share-landing join) recorded
in benches/ROUND5.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA
2026-07-17 15:19:00 +00:00
Edouard Vanbelle b95e740b2f security(music): ensure read permission via authz 2026-07-05 22:53:05 +02:00
Edouard Vanbelle 37467ed9d3 feat(drive): remove all owner_id from {File,Folder}Dto 2026-07-03 01:13:53 +02:00
Edouard Vanbelle 206a780494 feat(drive): stop writing in {file,folder}.user_id
plan:

    ```
    What step 5 is actually doing

    Goal: stop writing to storage.files.user_id / storage.folders.user_id on new INSERTs. This is a prerequisite for step 6 (dropping the column entirely). The column has existed since the pre-D0 single-owner era; every read that used to key on it has already been migrated to drive-membership grants over the last N days.

    What has to change together for this to be safe:

    ┌──────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────┐
    │    Piece     │                                                          What changes                                                           │                                                  Why                                                  │
    ├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
    │ Migration    │ Swap storage.files uniqueness indexes from (folder_id, name, user_id) to (drive_id, folder_id, name)                            │ Otherwise, two new rows with user_id = NULL would both be allowed (PG treats NULLs as distinct) —     │
    │ (a)          │                                                                                                                                 │ uniqueness silently breaks                                                                            │
    ├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
    │ Migration    │ ALTER user_id DROP NOT NULL on both tables                                                                                      │ Otherwise, dropping the INSERT bind violates NOT NULL and every write 500s                            │
    │ (b)          │                                                                                                                                 │                                                                                                       │
    ├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
    │ Migration    │ Drop dead user_id-leading indexes                                                                                               │ Cheap cleanup — nothing scans them anymore                                                            │
    │ (c)          │                                                                                                                                 │                                                                                                       │
    ├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
    │ PL/pgSQL (d) │ Rewrite storage.copy_folder_tree without user_id in the INSERT column list                                                      │ Cross-drive copy runs entirely in SQL, needs the same treatment                                       │
    ├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
    │ Rust (e)     │ ~12 INSERT column-list drops across folder_db_repository, file_blob_write_repository, drive_pg_repository, dedup_service,       │ Actual write path                                                                                     │
    │              │ folder_service, load-seed.rs                                                                                                    │                                                                                                       │
    ├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
    │ Rust (f)     │ Simplify folder_db_repository::create_folder's parent lookup to only fetch drive_id (was fetching (user_id, drive_id))          │ It's fetching a value it no longer needs                                                              │
    └──────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────┘
    ```
2026-07-03 00:23:35 +02:00
Edouard Vanbelle 583af517c2 fix(drive): fix drive_id on copy to other drive
- fix alsot drive_id exposition
2026-06-29 23:21:17 +02:00
DioCrafts 2b5339b73e refactor(api): remove 5 deprecated list endpoints superseded by /resources
The normalized cursor-paginated /resources API is live and the bundled
frontend already uses it for favorites, recent, trash and grants. These five
deprecated old-format endpoints had no remaining frontend or protocol
consumers (the Nextcloud handlers call the service layer directly, not these
HTTP routes), so remove them for a uniform API and less duplicate listing
logic:

- GET /api/folders/{id}/contents          → use /api/folders/{id}/resources
- GET /api/folders/{id}/contents/paginated → use /api/folders/{id}/resources
- GET /api/favorites                       → use /api/favorites/resources
- GET /api/recent                          → use /api/recent/resources
- GET /api/trash                           → use /api/trash/resources

Removes the HTTP handlers, their routes, OpenAPI path registrations, and the
now-dead list_folder_contents{,_paginated}_impl helpers + unused imports. The
underlying service methods (favorites_service.get_favorites,
trash_service.get_trash_items, etc.) are KEPT — the Nextcloud OCS/trashbin
handlers depend on them.

Deliberately NOT removed: GET /api/folders/{id}/listing. It is still the Files
view's primary data path and offers ETag/304 conditional caching plus
one-shot favorite/share badge sets that /resources does not yet provide;
migrating it needs a separate parity pass on /resources first.

Updates the OpenAPI structure test and the two docs that referenced the
removed paths. `cargo clippy -D warnings` clean; 443 lib tests pass; OpenAPI
regenerates with the 5 paths gone and the /resources replacements present.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-19 23:46:58 +02:00
Edouard Vanbelle e7f4826778 feat(drive): complete updated_by created_by 2026-06-19 16:06:38 +02:00
Edouard Vanbelle 50fb34659e feat(drive): limit NC request by disk 2026-06-19 16:06:38 +02:00
Edouard Vanbelle bde7c83932 feat(content_hash): propagate etag and hash_content to */resources 2026-06-06 20:11:13 +02:00
Edouard Vanbelle 0135930da9 refactor(file|folder): separate etag and blob_hash 2026-06-06 18:49:14 +02:00
Edouard Vanbelle 6e30353c4b chore(deprecated api): log a warning on serverside if a deprecated route is still used 2026-05-29 13:13:23 +02:00
Edouard Vanbelle d88b3a313f fix(recent): prevent recent API having non UUID entries + correct any invalid entry in recent database
note: issue may have been on my side during test implementation, in any case the protection is required
2026-05-29 13:13:23 +02:00
Edouard Vanbelle 73e78904e7 feat(recent): add cursor + groupby support 2026-05-28 12:12:22 +02:00
Edouard Vanbelle dfb21b746d feat(openapi): show explicitly path requiring bearerAuth 2026-05-28 00:48:20 +02:00
Edouard Vanbelle badf35f08f chore: remove all executable attributes on non bash files 2026-04-01 23:14:42 +02:00
iltumio bf7e030cd6 feat: add OpenAPI spec generation with utoipa and justfile
- Add utoipa v5 dependency with ToSchema derives on all REST API DTOs
- Annotate free-function handlers with #[utoipa::path] (trash, share, favorites, recent)
- Create ApiDoc struct with OpenApi derive registering 37 schemas across 7 tags
- Add generate-openapi binary outputting resources/gen/openapi.json
- Serve OpenAPI spec at GET /api/openapi.json (public, no auth)
- Add justfile with common dev commands (build, test, lint, check, openapi, db)
2026-03-29 18:49:10 +02:00
Diocrafts 06ed0455ce perf: migrate all user/session/auth IDs from VARCHAR(36) to native UUID
- Schema: all ~15 VARCHAR(36) columns → UUID with DEFAULT gen_random_uuid()
- Domain entities: User, Session, DeviceCode, AppPassword, Share → id: Uuid
- DTOs: CurrentUser.id → Uuid (API boundary DTOs keep String for JSON)
- Auth middleware: parse JWT claims.sub (String) → Uuid at boundary
- All repository traits, port traits, service impls updated end-to-end
- Handlers: pass Uuid by value (Copy, 16 bytes) instead of String refs
- Settings chain: updated_by column → Uuid (was text, caused setup crash)
- Removed ~650 lines of String↔Uuid conversion boilerplate
- Eliminates per-request heap allocations for ID cloning
- 16-byte binary comparison vs 36-byte string comparison in all queries
- Native UUID indexing in PostgreSQL (btree on 16 bytes vs 36-char text)

85 files changed, 1090 insertions(+), 1739 deletions(-)
2026-03-07 14:59:32 +01:00
Dionisio f2d35ca792 feat: auto-persist JWT secret, remove setup token requirement
- JWT secret auto-generates and persists to <STORAGE_PATH>/.jwt_secret
- Remove setup token: first admin setup is open until system initialized
- Fix schema.sql: move CREATE EXTENSION pg_trgm/ltree to top
- Update login UI and auth.js to remove setup token fields
2026-03-05 22:12:53 +01:00
Dionisio b503e08384 security: fix vulnerabilities 1-7 from security audit
- Fix #1: Share handler IDOR - enforce owner check on share operations
- Fix #2: list_files_query IDOR - bind folder queries to authenticated user
- Fix #3: Dedup handler IDOR - restrict dedup operations to file owner
- Fix #4: Trash handler OptionalAuthUser - require full AuthUser
- Fix #5: Error info leakage - sanitize 500 error responses
- Fix #6: Chunked upload IDOR - bind upload sessions to user_id,
  add verify_session_owner() check on all session operations
- Fix #7: CSP unsafe-inline removal - migrate all inline scripts,
  styles and event handlers to external files, tighten CSP to
  script-src 'self'; style-src 'self'

New files:
  - static/js/core/theme-init.js (render-blocking theme init)
  - static/js/core/sw-register.js (service worker registration)
  - static/css/views/device-verify.css (extracted inline styles)
  - static/js/views/device-verify/device-verify.js (extracted inline script)
2026-03-05 13:15:34 +01:00
Diocrafts ee86c3a128 fix: resolve all clippy warnings and convert integration_tests to custom cfg
- Add type aliases (FileRow, FolderRow, FolderRowPaginated, FolderRowOptUser) to reduce type complexity
- Simplify redundant closures in app_password_handler and webdav_handler
- Remove needless borrow in auth_handler
- Collapse nested if/let chains in login_lockout, webdav_lock, auth, rate_limit
- Box LockEntry in acquire() Err variant to fix large enum variant warning
- Rename DeviceCodeStatus::from_str to parse to avoid should_implement_trait lint
- Add #[allow(clippy::too_many_arguments)] and #[allow(clippy::result_unit_err)] where appropriate
- Convert integration_tests from cargo feature to custom cfg attribute
- Add check-cfg lint config in Cargo.toml for integration_tests cfg
2026-03-04 23:55:08 +01:00
Claude 1b49135ca9 perf: replace dyn trait objects with concrete types to eliminate vtable overhead
Remove async-trait dependency and use native Rust async fn in traits.
Replace Arc<dyn Trait> with Arc<ConcreteType> throughout the codebase
to enable monomorphization and eliminate dynamic dispatch overhead.

Key changes:
- Remove write-behind cache (no implementation existed)
- Fix should_transcode static method call
- Use ContactStorageAdapter directly instead of dyn AddressBookUseCase
- Clean up unused trait imports across services and DI

https://claude.ai/code/session_01EbAFEfyJNLRmJHmmYDX3Tt
2026-03-03 15:36:42 +00:00
Dionisio 4c98c5a657 style: apply cargo fmt to entire codebase
Standardize code formatting across all 173 Rust source files
using rustfmt. No functional changes - purely cosmetic.

This establishes a consistent code style baseline for the
project going forward.
2026-02-14 01:29:34 +01:00
Dionisio d31a413e57 chore: translate all Spanish comments and log messages to English 2026-02-12 09:41:25 +01:00
Dionisio b9d10c7b5c fix 2026-02-07 04:02:38 +01:00
DioCrafts ba3ef2a530 adding webdav features 2025-04-04 21:31:41 +02:00
DioCrafts a79c335b73 adding recent feature + bug fixed 2025-04-02 05:08:30 +02:00