Two items from the ROUND17 deferred list, each benchmark-gated with a
BEFORE/AFTER equivalence gate and a rollback-on-regression check (ROUND2–17
discipline). See benches/ROUND18.md.
[C1] backend — CalendarEvent::update_ical_property / remove_ical_property
rewrote the ENTIRE ical_data body with format!("{}{}{}") on every call and
allocated two search needles per call. calendar_storage_adapter::update_event
fans a multi-field edit out into one call per changed field, so a full REST
edit paid one full-body (up to ~11 KB) allocation per property. The body is
now mutated in place (replace_range for an existing property, four insert/
insert_str for a new one, byte-identical spans) and the single "\nNAME:"
needle is built on the stack (the "\r\nNAME:" needle was redundant — the LF
form is its suffix). bench_round18_micro [C1]: 70 -> 2 allocs/op (68 fewer),
2.46x wall, emitted body byte-identical.
[F1] frontend — ResourceList itemIndexById rebuilt a fresh Map over the whole
accumulated list every infinite-scroll page (O(N)/page, O(N^2) drain) and,
being a new instance each page, re-fired the reap-stale effect (another O(N)
id Set/page). New ItemIndexBuilder extends a persistent Map with the fresh
page only and reuses the reference across appends; the reap-stale effect now
tests membership against it. round18.bench.test.ts [F1]: 40x50 drain 74.1 ->
6.4 ms (11.5x), deep-equal to the reference at every page, reference-contract
gate (same-ref append / new-ref rebuild).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FoxFtikahM1N4PE5s3ZVH3
Benchmark-gated, same rule as rounds 2–14: every change ships with a
BEFORE/AFTER benchmark and an equivalence/safety gate; an AFTER that doesn't
beat its BEFORE is rolled back. The rule is encoded per harness (GATE FAIL
non-zero exit in the Rust examples, threshold expect() in vitest).
F1 — Grouped listings (trash / recent / favorites / shared-with-me)
re-bucketed the WHOLE accumulated list on every infinite-scroll page.
ResourceSectionsBuilder (new, off the reactive graph) re-buckets only the
fresh page and hands VirtualList the same rows array reference for untouched
buckets. 50×50 (2 500-item) drain: 63 750 → 2 500 bucketOf calls (25.5×),
12.5 → 1.3 ms wall (9.9×); O(N²/page) → O(N). Deep-equal to the full-rebuild
reference at every page for both a contiguous (date) and a non-contiguous
(trash-by-drive) group-by; reference-stability + fallback gated.
B1 — exif Make/Model: the display String was thrown away to allocate the
trimmed copy; display_value_trimmed trims in place (drain + truncate), 2 → 1
alloc per field (8 → 4 allocs/op, 1.26×).
B2 — content-index worker: text_extractor::supports (lowercases MIME +
extension) was called twice per file per drain batch; classify once into a
Vec<bool> and thread it through both uses. 256-file batch: 704 → 353 allocs,
34.5 → 16.7 µs (2.07×).
B3 — tantivy: skip SnippetGenerator::create on a zero-hit content search
(return Ok(vec![]) once top_docs.is_empty()); the per-hit loop was empty.
400-doc index: 1 575.6 → 1 237.2 ns (1.27×), widens with index size.
Harnesses: examples/bench_round15_micro.rs, examples/bench_round15_tantivy.rs,
frontend resourceSections.bench.test.ts; writeup in benches/ROUND15.md. Also
normalizes two round14 bench examples that were committed unformatted
(cargo fmt --all).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012o47jSrtL7xuNGTHXmtiYL
Benchmark-gated (BEFORE/AFTER + equivalence/safety gate per change), same
discipline as rounds 2-12. Full write-up in benches/ROUND13.md.
Shipped:
- V1 Grouped views windowed (files route + ResourceList). The grid arm was
the last unwindowed path (trash is grouped-by-default in grid): each
swimlane now feeds its own VirtualList, outer container a flex stack.
vitest gate: 800-item grouped grid mounts <120 .file-item (was 800).
- Q1 get_users_by_ids drops the <=512 KiB avatar image + ui_preferences
JSONB (notification path never reads them). 30-member fan-out 8.60 ->
0.25 ms (34.3x), ~7.7 MB off the wire.
- Q2 Login provisioning is_empty() -> SELECT EXISTS for calendar + address
book (every login). 0.193 -> 0.170 ms, widens with owned-row count.
- Q3 Recent-access prunes only when the upsert inserted (RETURNING xmax=0)
— a re-access can't grow the set. 0.567 -> 0.324 ms (1.75x).
- L1 Locale supported-codes precomputed once vs rebuilt per anonymous
request. 616 -> 17.3 ns (35.7x), 18 -> 1 allocs.
- H1 Duplicate /api TraceLayer removed (global stack already wraps it).
1.86 -> 1.42 us/request, -6 allocs.
- H2 client_ip span field: borrow-only ClientIpDisplay vs owned String.
187 -> 173 ns, -1 alloc.
Not shipped (discipline): the "media hooks read the blob 3x" lead was a
correctness bug, not a perf dup — the raw-path metadata/faces readers
resolve only for local+unencrypted+single-chunk blobs and silently produce
nothing otherwise. Flagged for maintainers; routing through read_blob_bytes
is a correctness fix (perf-neutral-to-negative), not a benchmark-gated
perf change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB
Benchmark-gated round (benches/ROUND12.md; every change ships with a
BEFORE/AFTER harness + equivalence gates, one candidate rejected by its
own bench):
DB / query shapes (bench_round12_queries):
- NC sharee search: username-only projection instead of the 21-column row
(incl. the <=512 KiB avatar) per match, + gin_trgm_ops indexes on
auth.users for the leading-wildcard ILIKE (4.98x; 54.7x with index).
- Password login: delete the redundant full-row update_user — create_session
already stamps last_login_at in its own txn (4.45x per login).
- Email-verified stamp: narrow conditional UPDATE (8.9x); OIDC repeat login
now compares profile state in memory and issues ZERO queries when nothing
changed (was: full 17-column rewrite per login).
- Refresh rotation: revoke+insert+stamp fused into one transaction via new
rotate_session port method (1.18x).
- WOPI CheckFileInfo / authorize_wopi_access: require(Read) + get_file +
check(Update) overlapped with tokio::join!, original result precedence
(cold 1.34x).
- Upload quota gate: user-envelope + drive-cap checks fused into ONE
round-trip (check_upload_quotas) — the NC chunked PUT pays this per
chunk (1.81x, 2 -> 1 queries/chunk); shared verdict evaluators keep
error shapes byte-identical.
CPU / allocs (bench_round12_micro):
- sized_json: pre-sized listing serialization replacing axum Json's 128 B
seed + doubling-realloc chain on files/folder-resources/photos/search
responses (1.40x, 13 -> 2 allocs per 500-row page; byte-identical).
- Security headers: 4 SetResponseHeaderLayer folded into the CSP middleware
pass (5 layers -> 1; 1.43x per request, -26 allocs; header set gated
byte-identical incl. 304s).
- Media capture-metadata: single-read extraction — nom-exif now parses the
buffer kamadak already read (zero-copy Bytes) and videos open once with a
kind() dispatch; per-image opens 2-3 -> 1 (1.44x warm geomean, 1.6-3.2x
cold cache; extraction outputs gated identical incl. the MIME-mislabel
track fallback).
- Chunked-upload session ops: owner gate folded into the operation's own
DashMap lookup + stack-encoded uuid compare (5 -> 3 lookups, -2 allocs,
1.28x per chunk).
Blob cache (bench_blob_cache_index + round-3 regression guard):
- CachedBlobBackend index: tokio::sync::Mutex<LruCache> -> moka::sync::Cache
with byte weigher. The mutex serialized every cached chunk read and scaled
NEGATIVELY (2.08 -> 1.07 Mops/s from 1 -> 2 readers); moka probes are
lock-free (2.17x at K=2). Byte budget now enforced by moka (manual
current_size + collect_evictions machinery deleted); eviction listener
unlinks size-evicted files only (Replaced entries keep their file —
gated). Single-flight miss gate unchanged (16 concurrent misses -> 1
fetch re-verified via the round-3 harness).
- put_blob now populates the cache BEFORE the inner backend consumes the
source file (the old order failed 100% of the time — local renames,
S3/Azure delete the source — so the first read after a whole-file put
re-downloaded from the remote); inner-put failure invalidates the entry.
Frontend (vitest gates):
- List-view thumbnails request the 150px icon rendition instead of 400px
preview into a 40px slot (~7.1x fewer pixels, ~4-5x fewer bytes per
thumbnail across list views); grid keeps preview.
Rejected by its own bench (kept as evidence in bench_round12_micro §2):
- Single-pass compression predicate: the monomorphized And-chain already
costs ~4.6 ns / 0 allocs total; the fused node measured within noise.
New migration: 20260719000000_users_search_trgm.sql (trgm indexes).
Deferred with prepared design: grouped file/grid view virtualization
(single-VirtualRows flatten, the photos pattern) — next round's headline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB
Conflict resolution re-applies the round-11 SPA optimizations onto the
refactored item-based ResourceList (which independently converged on the
favoriteIds-prop star — S2 is now upstream's shape):
- ResourceList.selectedItems: upstream reintroduced the full O(N)
items.filter per selection toggle; replaced with the round-11
id-index projection — O(k·log k), item order preserved
(benches/ROUND11.md §S1)
- favorites/recent pages: upstream's rewrite kept the host-side
selectedItems shadow + unused selectedIds mirror and ignored the
batchToolbar snippet param; re-applied the param-consuming shape and
deleted the shadows
Validation on the merged tree: clippy --all-features --all-targets
-D warnings clean; cargo test --workspace 524 passed; frontend npm run
check 0 errors; vitest 310 passed (upstream's new tests + round-11 gates)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7
Backend (each change benchmark-gated with BEFORE replicas + equivalence
gates; see examples/bench_round11_micro.rs, bench_round11_queries.rs,
bench_log_writer.rs and benches/ROUND11.md — final numbers land in the
follow-up doc commit):
- StoragePath re-representation: single canonical joined String, segments
derived on demand; File/Folder drop the duplicated path_string field
(4000→1000 allocs per 500-row listing page)
- Display classifier fusion: classify_display shares one stack-lowered
extension across the three decision trees; call sites in FileDto,
folder/favorites/recent handlers, trash, path-resolver (+ interning
where Arc::from was still used)
- /status.php and /openapi.json memoized into OnceLock<Bytes> (openapi
rebuilt a 171 KiB spec per request: 2.8 ms → 18 ns)
- NC upload-session PROPFIND: write! + pre-sized body + stack RFC2822
dates (2.3-2.6x, 2582→772 allocs at 256 chunks)
- REST download: dead FileDto clone removed (capture mime/size + move)
- CalendarEventDto/TrashedItem into_parts moves (11 KiB ical_data memcpy
gone per CalDAV row); CardDAV getlastmodified stack render
- 4xx path: borrowed ErrorResponse serialize, ErrorKind::as_str,
not_found/already_exists clone kill
- vCard emit via write!; search page moved out with into_iter skip/take;
content-hit UUIDs parsed once; group last-user check via HashSet
- RateLimiter: lock-free get + insert (and_upsert_with variant REJECTED
by benchmark); CSRF token borrow-compare + borrowed cookie extraction
- Thumbnail/preview ETags built from as_str (Debug-identical bytes)
- Encrypted backend: encrypt_in_place_detached single-buffer write path,
chunk-sized reserve in collect_stream; retry labels made lazy
- PG: deferred upload registration 3→1 round-trips (persist_file CTE
template); direct_grant_cache for Calendar/AddressBook/Playlist authz
(single-flight + set_role/clear_role invalidation); expand_user
tokio::join!; geo clusters min(uuid)::text; recluster face assignment
batched into one UNNEST update
- People recluster cosine: norms precomputed once (bit-identical gate)
- NC capabilities poll logs demoted to debug; tracing-appender dep added
for the log-writer benchmark
Frontend:
- ResourceList.selectedEntries O(N)-per-toggle → id-index projection
O(k log k); favorites/recent consume the batchToolbar snippet param and
drop their duplicate filter + dead selectedIds mirror
- Recent: star state via new favoriteIds prop — a star click no longer
rebuilds all N entries
- admin timeAgo >30d fallback uses the cached Intl.DateTimeFormat
- vitest gates in src/lib/components/round11.bench.test.ts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ABhTEHuGujvwoodh67Kga7
- add resource kind filter (file, folder, drive) in shared section (localStorage stored)
- add user preferences serverside store
- add client side dotfile filter (show/hide dotfiles) (user perf stored, default: dotfiles are shown)
for security trashed dotfile are always displayed
protection added: if a folder has only hidden items, a notification invite user to display it
if a user rename or create a hidden item, a notification tells it to user
Add an end-to-end and unit test suite for the SvelteKit frontend:
- Playwright e2e specs (tests/e2e/spa) with a throwaway container stack,
codegen scenarios, and an Istanbul-based coverage report pipeline.
- Vitest unit tests across API endpoints, components, stores and composables.
- `data-testid` hooks on interactive elements (AppShell, FileViewer,
ShareDialog, search, photos, files breadcrumbs, login/Nextcloud flows,
public share pages) so the e2e suite can target them deterministically.
- Serve the SPA app-shell CSP from a <meta> policy (svelte.config.js) plus a
middleware that skips the CSP header on HTML; move the Nextcloud Login Flow
v2 grant page to the SvelteKit /nextcloud/login route.
- `just front-codegen` recipe and start-server-spa.sh harness.
Make the test environment robust and consistent:
- Install a deterministic in-memory localStorage/sessionStorage in the Vitest
setup so storage behaves identically across Node versions (Node 26 ships a
native Web Storage global that otherwise shadows jsdom's).
- Pin devenv to Node 26 + PostgreSQL 18 and pin every CI job to Node 26.3.0
so the dev shell and CI run the same toolchain versions.
Repair the API/WebDAV (hurl) suite, which had drifted from the backend:
- Migrate the removed `/api/folders/{id}/listing` endpoint to `/resources`
(cursor-paginated `{items:[{resource_type,resource}]}` shape) across the
batch-copy, grants, nested-group, and WebDAV NC tests + the dav_helpers
wipe routine.
- Stop photos_etag from uploading the dedup-tracked fixture so the dedup
blob-lifecycle test can own its content-addressed blob exclusively.
- dedup_create now asserts the idempotent same-content re-upload (201 +
existing file id) instead of the stale 409 expectation.
Generated coverage reports, nyc output and the e2e server runtime data dir
are gitignored rather than committed.
Checkpoint of the in-progress frontend toolchain work (Vite pinned to ^6 after
the 7/8 rolldown build break, eslint-plugin-svelte v3 navigation/reactivity
fixes, CI/Dockerfile/manifest updates) together with three UI performance
optimizations (verified on the Vite 6 build):
- Critical CSS: move auth.css/music.css off the global path into their route
chunks (login/device/nextcloud-login, music) -- -25% gzipped critical CSS
(~5.4 KB) on every non-auth/non-music page load.
- relativeTimeAgo: cache the Intl.RelativeTimeFormat (was rebuilt per call, once
per row per render) -- 22.7x faster date formatting in large lists.
- Virtualize search results and grouped trash (list view) via VirtualList -- DOM
rows mounted stay ~constant (~27) instead of O(N) (94.6% fewer for 500 hits).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bundles the backend+frontend delta-upload (content-dedup) feature with a
batch of frontend fixes from this session.
Upload / dedup:
- Client-hashed delta & instant upload (deltaUpload, hashWasm vendor shim)
- Backend dedup batch endpoint (dedup_service, dedup_handler, routes)
- session store owned-hash helpers; unit tests + upload-strategy bench
Frontend UI/UX:
- Colour file-type icons in grid/list (per-type tinted tiles + glyph hue)
- Robust thumbnail fallback; PDFs now show their type icon (backend
generates no PDF thumbnails) instead of a blank tile
- Fix PDF preview: load via a same-origin blob: iframe — the API URL is
blocked by the global X-Frame-Options: DENY in the browser's framed
PDF viewer, matching the existing CSP `frame-src blob:` design
- Groups: localized virtual-group description (no DB schema-note leak),
add nav.groups to the 15 missing locales, fix primary-button contrast
- Repoint --color-text-light → --color-on-accent (was faint grey on accent)
- Nudge the admin role badge off the user-menu header divider
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends windowing to the remaining O(n)-DOM surfaces: the card-grid view of
ResourceList (recent / favorites / shared / shared-with-me / trash / search) and
the main file browser (`files/[...path]`), in both list and grid layouts.
- VirtualList gains a real grid mode: its inner window carries the caller's grid
class (`files-grid-view`) and lays out `columns` cards per windowed row. The
row pitch is auto-measured (grid card height tracks column width via the 4/3
aspect-ratio thumbnail) and re-measured on resize.
- `useVirtualWindow` now distinguishes scroll from resize and exposes a
`resizeTick`, so size-dependent layout (the grid pitch) is recomputed only when
it can actually change.
- `gridColumns(width)` (new util) mirrors the CSS `auto-fill` / `--grid-card-min`
/ gap so the windowed row count matches the browser's real wrapping exactly;
shared by both grid callers.
- The files browser flattens folders-then-files into one discriminated `entries`
list rendered through VirtualList (list: columns=1; grid: columns from width).
Grouped (swimlane) views stay fully rendered, as before — they're bounded.
ResourceList GRID, headless Chromium (1280x900), synthetic rows, before/after:
rows | mount→paint | DOM nodes | JS heap | scroll frame | jank frames
------+-------------+-----------+---------+--------------+------------
1000 | 979→75 ms | 19k→879 | 16→3 MB | 16→17 ms | 0→0
5000 | 4005→67 ms | 95k→879 | 72→3 MB | 37→17 ms | 26→0
20000 |12015→65 ms | 380k→879 |281→7 MB |197→17 ms |1249→19
Rendered DOM and heap are flat (O(visible)) regardless of dataset size; mount is
~185x faster and scroll holds ~60fps. Verified visually mid-scroll (5 columns,
4/3 thumbnail tiles, cards land at the expected indices). The files browser
shares the same VirtualList path (it can't be mounted headless — it depends on
$app routing/session — so it's validated via svelte-check + the production build
+ the shared, separately-benchmarked component).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8Vb9QHmLZnEMzHz7MrFy6
Large folders/collections rendered every row into the DOM, so memory and
scripting time grew linearly with item count. A 20k-row list took ~12.5s to
first paint, held ~380k DOM nodes / ~281MB JS heap, and scrolled at ~4fps.
Add a reusable, dependency-free `VirtualList` that windows rows against the
nearest scrollable ancestor (the existing `.content-area`), so the
single-scrollbar UX, sticky header and end-of-list sentinel are unchanged. It
reserves the full scroll height with a sized spacer and translates a small
window of rows; row height is auto-measured for the single-column case.
Wire it into ResourceList's flat list view (recent, favorites, shared,
shared-with-me, trash, search). Grouped sections and grid view are unchanged
for now and are the next callers of the same primitive.
Measured in headless Chromium (1280x900) with synthetic rows, before/after:
rows | mount→paint | DOM nodes | JS heap | scroll frame | jank frames
------+-------------+-----------+---------+--------------+------------
1000 | 632→70 ms | 19k→351 | 16→2 MB | 17→17 ms | 0→0
5000 | 4401→54 ms | 95k→351 | 72→3 MB | 47→17 ms | 136→0
20000 | 12577→56 ms | 380k→351 |281→6 MB |258→17 ms | 1443→0
Rendered DOM and heap are now flat (O(visible)) regardless of dataset size;
scroll holds 60fps with zero jank. Correctness verified by probing a mid-list
scroll (rows land at the expected indices, positioned within the viewport).
eslint: disable core `no-undef` for `.svelte` (TypeScript/svelte-check already
resolve identifiers, including `<script generics>` type params it can't see).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M8Vb9QHmLZnEMzHz7MrFy6
Claude