purpose is to design a job registry with a scheduler
thi aim to drive in the same way any services requiring execution of periodic background tasks
plugins could benefeciate it
purpose is mostly to add a normative way to implement consistency check per services
this is to edutcate implementors adding any new services
goal is to ensure data quality with oxicloud and resumable jobs by default
add /api/folders/{id}/ancestors
this API to iterate parent up to the drive root or the shared folder
this will help UI to build the breadcrumb in 1 API call
and to identify the root element (is it a drive users has access to or
a shared folder ?)
ui: now only 1 API call is now required to build the breadcrumb
This machine-local Claude Code runtime lock was committed long ago in
e3823ce4 (shared main history) and later removed on upstream/main. Our
fork's main never got that removal, so the branch still carried it and
the PR diff was re-introducing it into upstream/main.
Untrack it (`git rm --cached`, local copy kept) and gitignore it so it
never returns. Since the base (upstream/main) already lacks the file,
removing it at the branch tip drops it from the PR diff entirely.
Close the two coverage gaps in the external-mounts feature — the admin
REST surface and the admin UI tab both previously had no automated tests.
Backend (hurl):
- tests/api/external_mounts.hurl — black-box coverage of the admin mount
endpoints (list / create / delete): input validation (empty name,
non-existent host path → 400), create → 201 with the full view, the
mount appears in the list, delete → 204 then 404, and the /api/admin
middleware denies a non-admin on every verb → 403.
- server.env: enable OXICLOUD_ENABLE_EXTERNAL_MOUNTS for the test server;
run.sh: register the new scenario.
Frontend (vitest):
- admin/page.test.ts — opening the mounts tab loads + lists mounts,
the create form calls createExternalMount, and delete goes through the
confirm modal to deleteExternalMount.
Tooling:
- devenv.nix: add b3sum (used by tests/api/run.sh's storage-integrity check).
- provide is_shared and is_favorite information in DTO, information propagated as badge/buttons per items
regarding performances I try to be minimalis on SQL to prevent any perf regression
- remove old set of sharedids and favoriteids (was not functionnal anymore)
- fix date picker (no past selection) in grant
- fix contextmenu close on /files section
- rustfmt the integration-test FolderService::new callsites added during
the upstream merge (long single-line args wrapped).
- Cargo.lock: rebuild from upstream/main's lock so only the
testcontainers-modules dev-dep subtree and the memmap2 0.9.10→0.9.11
security bump differ (avoids churning upstream's pins).
- .cargo/audit.toml: ignore the four astral-tokio-tar tar-extraction
advisories — dev-only, transitive via testcontainers-modules
(integration-test harness), never in the production binary.
Resolve conflicts between the external-file-mounts feature and upstream's
D5/D7 refactor (per-file provenance, keyset pagination, cross-drive move
gates, resource-access hook, folder-cascade lifecycle hook).
Key resolutions:
- FolderService::new now takes (repo, authz, file_lifecycle, mount_router);
all callers + DI updated.
- FileRetrievalService / FileManagementService keep both the mount_router
and the new resource_access_hook / drive_repo / storage_usage wiring.
- list_files_batch_with_perms: adapt the mount branch from offset- to
keyset (after_name) pagination, mirroring paginate_mount_entries.
- download_file_impl: keep upstream's &HeaderMap + `impl IntoResponse + use<>`
signature, retain the mount-download branch.
- Mount DTOs: the retired `owner_id` field maps onto created_by/updated_by
(the mount owner) — the fields the frontend now uses for owner display.
- admin/+page.svelte: keep upstream's user-delete modal + the 'mounts' tab.
- Bump memmap2 0.9.10 -> 0.9.11 (RUSTSEC critical advisory fix) and
regenerate Cargo.lock against the merged Cargo.toml.
Seven behaviour-preserving allocation / copy / bandwidth cuts, each behind a
counting-allocator BEFORE/AFTER gate that exit(1)s unless AFTER allocates
strictly fewer than BEFORE (benches/ROUND29.md, examples/bench_round29_micro.rs).
- [B] Content-cache serve fast path (optimized_inner Tier 1 +
get_file_range_preloaded — the video-scrub hot path): probe the cache with a
borrow first and build the owned get_or_load args (quoted-etag / key / id
Strings) only on a miss, instead of allocating them before every probe and
discarding them on a hit. 6 -> 0 allocs per cache hit. Splits get_or_load into
get + load_and_cache so the miss path is not re-probed and the hit/miss stat
counters stay byte-identical. Also drops the unconditional content_hash/name
clones that ran for the >=10 MB streaming tier that used neither.
- [A] NextCloud REPORT emit loops: per-row href String (and format! per folder
row) -> one reused href_buf via nc_href_into / nc_collection_href_into with the
URL-encoded user computed once per page. 1497 fewer allocs on a 500-row page.
- [C] read_full: a single-frame blob is returned zero-copy instead of a second
whole-payload memcpy into a fresh BytesMut; multi-frame path unchanged.
- [D] login-lockout key: to_lowercase()+format! -> one pre-sized ASCII buffer
(non-ASCII keeps str::to_lowercase). 3 -> 1 alloc/req, byte-identical key.
- [E] NC composite-username parse: owned clone/to_string -> &str borrow of the
already-owned raw_username. 1 -> 0 alloc on the common no-marker path.
- [F] get_contacts_in_group: stop SELECTing the discarded multi-KB vcard column
(the live method ROUND25 §Q2 missed; ContactDto has no vcard field).
- [G] count_admin_users: add count_users_by_role -> scalar COUNT(*) instead of
hydrating every admin's full row (incl. up-to-512 KiB avatar + ui_preferences
JSONB) only to .len() it, on a bootstrap-polled status endpoint.
All seven gates pass; cargo fmt --check and cargo clippy --all-features
--all-targets -D warnings clean. §F/§G additionally validated against a live
PostgreSQL 16 with the full migration set (query validity, result equivalence,
600000 -> 8 byte wire delta on the admin count).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LhpDZxSQTAGnAqCHUdtG5N
report_handler's four REPORT emit loops (the filter-files favorites REPORT and the
search REPORT, each a file loop + a folder loop) shared the same per-row oc:id
String that ROUND27 §H1 replaced in the two PROPFIND page loops. Apply the
identical, already-validated transformation: hoist one oc_buf per handler (reused
across both its loops) and compute the id into it via format_oc_id_into instead of
a fresh format_oc_id String per child. 1 String/row -> 0 (amortized). The
write_{file,folder}_response fns already take Option<&str>, so their signatures are
unchanged and the emitted oc:id bytes are byte-identical.
Same transformation benchmarked in ROUND27 §H1 (bench_round27_micro: 998 -> 0
per-row allocs, 2.16x wall), so no new bench. Verified: cargo fmt clean, cargo
clippy --features bench -D warnings clean, cargo test --lib --features bench =
529 passed / 0 failed across 5 consecutive runs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
Two behaviour-preserving allocation cuts (benches/ROUND27.md), each with a
counting-allocator BEFORE/AFTER gate that rolls back if AFTER does not allocate
fewer than BEFORE:
- H1 NextCloud PROPFIND: the streaming page loops built oc:id as a fresh String
per child (format_oc_id -> format!("{:08}{}", id, instance)). Add
format_oc_id_into(&mut buf, id, svc) and compute into one oc_buf reused across
the page (next to the existing href buffer) — 1 String/row -> 0. 998->0
per-row allocs on a 500-row page, 2.16x wall. The write fns still take
Option<&str>, so no signature change; oc:id bytes identical. Scoped to the two
PROPFIND page loops (the hot directory-listing path); REPORT/trashbin deferred.
- P2 contact create/update: bind sqlx::types::Json(&dtos) (Encode runs to_writer
straight into the JSONB buffer) instead of serde_json::to_value(&dtos) + bind,
which built a throwaway Value DOM per JSONB column. Write-side twin of ROUND23
J1. 21->2 allocs, 4.68x wall for a 3-entry column. Behaviour-preserving:
to_value sorts keys and direct serialize keeps struct order, but Postgres
normalizes JSONB key order so the stored value is identical (verified via psql:
'{...alpha...}'::jsonb = '{...struct...}'::jsonb -> t), and reads decode by
field name; the etag comes from the domain entity, not the stored JSONB.
Adds bench_round27_micro. Verified: cargo fmt clean, cargo clippy --features
bench -D warnings clean (real exit), cargo test --lib --features bench = 529
passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
The BEFORE/AFTER rollback gates wrote `if !(after < before)`, which
clippy::neg_cmp_op_on_partial_ord flags on f64 (a negated comparison on a
partially-ordered type). Under CI's `cargo clippy --all-targets -- -D warnings`
this fails. Replace with the equivalent `if after >= before` (fail the gate when
AFTER does not strictly beat BEFORE). No behavioral change to any benchmark.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
Three benchmark-gated optimizations from the ROUND25 backlog (benches/ROUND26.md),
each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE:
- P1 drive_pg_repository policy reads: decode d.policies through
sqlx::types::Json<DrivePolicies> (one from_slice over the raw JSONB bytes)
instead of a throwaway serde_json::Value DOM + DrivePolicies::from_value —
6 -> 0 allocs/read, 2.77x wall. A shared policies_from_row helper preserves
the lenient unwrap_or_default fallback (malformed bag -> all-false).
- D1 CachedBlobBackend: pre-create the 256 {00..ff} shard dirs at initialize()
(mirroring LocalBlobBackend, reusing HEX_PREFIXES) and drop the redundant
per-write create_dir_all on already-existing shards — ~45us + a blocking-pool
dispatch removed per cache write on cached-remote deployments.
- G1 delta-upload have/need hash sets (distinct_hashes, authorize_chunk_download):
SipHash -> foldhash::quality::RandomState — a fast hasher that stays
DoS-resistant via a per-instance random seed, the required property for the
attacker-controlled 64-hex client hashes — 2.37x wall on a 40k-hash
negotiation. foldhash was already in the lockfile transitively (hashbrown).
Tested and REVERTED (kept as-is): moving the moka eviction unlink off the reactor
via spawn_blocking. The benchmark refuted it — on the local cache dir the
spawn_blocking dispatch (~20us) costs more than the inline unlink (~7us) it would
replace. See ROUND26.md §D2.
Adds bench_round26_{micro,diskio,hasher} (counting allocator / async wall / wall).
Verified: cargo fmt clean, cargo clippy --features bench -D warnings clean,
cargo test --lib --features bench = 529 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
Five benchmark-gated optimizations from a fresh six-way audit (benches/ROUND25.md),
each with a BEFORE/AFTER gate that rolls back if AFTER does not beat BEFORE:
- M1 EncryptedBlobBackend::decrypt_bytes: replace split_off (a fresh Vec + full
ciphertext memcpy on every decrypted chunk, contradicting its own "in place"
doc) with in-place detached decrypt + a zero-copy Bytes::slice past the nonce.
Peak RAM per read drops from ~2x to ~1x the payload (-262KB/op at 256KiB;
scales with blob size). Plaintext byte-identical; tamper/wrong-key tests pass.
- M2 delta commit: move-unzip the owned chunk list instead of a third
per-occurrence hash clone (-4000 allocs on a 4000-chunk commit).
- M3 folder ZIP download: drop the dead Query<HashMap> extractor it never read
(byte-identical response; 5->0 allocs/request).
- Q1 public-playlist listing: fold the per-playlist COUNT(*) N+1 into one
LEFT JOIN ... GROUP BY via a new inherent repo method (101 -> 1 round-trips,
36x wall on a 100-playlist page).
- Q2 contact REST listings (paginated/search/by-group): stop over-fetching the
multi-KB vcard TEXT the ContactDto discards, via a shared lite row mapper and
narrowed SELECTs (6.4x wall on 1000 contacts with 8KiB vcards). The whole-book
vCard export and CardDAV sync paths keep the column.
Adds bench_round25_micro (counting allocator tracking count+bytes) and
bench_round25_queries (live Postgres), both with equivalence gates and a
rollback exit(1). Verified: cargo fmt clean, cargo clippy -D warnings clean,
cargo test --lib --features bench = 529 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8gs91AhmazoxMsDcNk3KT
Edouard Vanbelle