Oxicloud

cjw/Oxicloud

Fork 0

Commit Graph

Author	SHA1	Message	Date
Edouard Vanbelle	7663f803d3	fix: fix services accessig directly to localstorage Prevent services accessing directly to localstorage and prefer using an astraction layer to expose full blob. The abstraction layer (dedup services) will cover backend storage election (local, s3, ...), encryption, etc This change permit audio_metadata_service, media_metadaa_service, face_indexing_service to handle blobs without worring of the backend. note: prefered way to handle blob is the streamed way. Some services may not have this possibility	2026-08-02 22:20:46 +02:00
Claude	12dc648cff	perf: round 4 — one-pass row paths, drive-selector cache, CalDAV single-parse, streamed Azure, batched hydration Nine benchmark-gated changes (benches/ROUND4.md; every one ships with a BEFORE/AFTER bench + equivalence gate, rollback rule as ROUND2/3): - Row→entity path build: one-pass StoragePath::from_folder_and_name / from_joined + normalize_storage_name_owned + alloc-free Display — 743→417 ns/file-row (1.78x), −5 allocs/row on every listing surface. - WebDAV drive-selector: per-user readable_cache (single-flight, 30 s TTL, explicit invalidation incl. membership + group changes) replaces the grants join per request — 441 µs → 0.8 µs (~550x), 0 queries warm. - CalDAV from_ical/update_ical_data: 8 full IcalParser runs per VEVENT → 1 (7.1x per PUT, 4.4x on 50-event imports); alloc-free split_vevents, chunk scan without the whole-body uppercase copy (1.4x), borrowed-key UID grouping (1.3x), REPORT props no longer cloned. - PROPFIND emit: partition Vecs dropped (single-pass 404 list) + stack rendered RFC 3339/2822 dates, sizes, quoted etags (common::fmt, chrono-byte-identical, sweep-tested) on both DAV surfaces — 1.22x per page, 17.9→12.0 allocs/row. - Grant-listing hydration: calendars/address books/playlists batch hydrate via = ANY($1) — 15 serial queries → 1 (~13x per sync poll). - user-flags cache: get→insert → try_get_with single-flight (32→1 queries per cold herd). - Azure downloads: whole-blob Vec buffering → streamed SDK pages — TTFB 349→4 ms (87x), peak heap 480→1.9 MiB (254x) on 256 MiB blobs; new OXICLOUD_AZURE_ENDPOINT_URL override (Azurite/bench hook). - Face indexing: unbounded per-image tokio::spawn → core-count semaphore, permit before blob read — peak heap 1175→176 MiB (6.7x). Checks: cargo fmt, clippy --all-features --all-targets -D warnings, cargo test --workspace (523 passed) + --features test_utils. hurl API suite and dockerized integration DB not runnable in this environment — left to CI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA	2026-07-17 13:48:37 +00:00
Claude	1dca9d7d05	feat(faces): indexing pipeline + DI wiring Phase 2 increment 5: - FaceIndexingService: a FileLifecycleHook that, on image upload, detects + embeds faces in a background task and stores them. Dedup-aware (clones an identical blob's faces instead of re-running inference), reindexes on overwrite, and relies on the DB cascade for deletes. Completely inert when no model is ready. - DI: registers the hook in the FileLifecycleService chain and exposes PeopleService in AppState — both gated on OXICLOUD_ENABLE_FACES, both using the default no-op analyzer until the operator wires a real ONNX model. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JW6ghFMDtnRYuYNzZhb47M	2026-06-19 11:46:57 +00:00

Author

SHA1

Message

Date

Edouard Vanbelle

7663f803d3

fix: fix services accessig directly to localstorage

Prevent services accessing directly to localstorage and prefer using an astraction layer
to expose full blob. The abstraction layer (dedup services) will cover backend storage
election (local, s3, ...), encryption, etc

This change permit audio_metadata_service, media_metadaa_service, face_indexing_service to handle
blobs without worring of the backend.

note: prefered way to handle blob is the streamed way. Some services may not have this possibility

2026-08-02 22:20:46 +02:00

Claude

12dc648cff

perf: round 4 — one-pass row paths, drive-selector cache, CalDAV single-parse, streamed Azure, batched hydration

Nine benchmark-gated changes (benches/ROUND4.md; every one ships with a
BEFORE/AFTER bench + equivalence gate, rollback rule as ROUND2/3):

- Row→entity path build: one-pass StoragePath::from_folder_and_name /
  from_joined + normalize_storage_name_owned + alloc-free Display —
  743→417 ns/file-row (1.78x), −5 allocs/row on every listing surface.
- WebDAV drive-selector: per-user readable_cache (single-flight, 30 s
  TTL, explicit invalidation incl. membership + group changes) replaces
  the grants join per request — 441 µs → 0.8 µs (~550x), 0 queries warm.
- CalDAV from_ical/update_ical_data: 8 full IcalParser runs per VEVENT
  → 1 (7.1x per PUT, 4.4x on 50-event imports); alloc-free split_vevents,
  chunk scan without the whole-body uppercase copy (1.4x), borrowed-key
  UID grouping (1.3x), REPORT props no longer cloned.
- PROPFIND emit: partition Vecs dropped (single-pass 404 list) + stack
  rendered RFC 3339/2822 dates, sizes, quoted etags (common::fmt,
  chrono-byte-identical, sweep-tested) on both DAV surfaces — 1.22x
  per page, 17.9→12.0 allocs/row.
- Grant-listing hydration: calendars/address books/playlists batch
  hydrate via = ANY($1) — 15 serial queries → 1 (~13x per sync poll).
- user-flags cache: get→insert → try_get_with single-flight (32→1
  queries per cold herd).
- Azure downloads: whole-blob Vec buffering → streamed SDK pages —
  TTFB 349→4 ms (87x), peak heap 480→1.9 MiB (254x) on 256 MiB blobs;
  new OXICLOUD_AZURE_ENDPOINT_URL override (Azurite/bench hook).
- Face indexing: unbounded per-image tokio::spawn → core-count
  semaphore, permit before blob read — peak heap 1175→176 MiB (6.7x).

Checks: cargo fmt, clippy --all-features --all-targets -D warnings,
cargo test --workspace (523 passed) + --features test_utils. hurl API
suite and dockerized integration DB not runnable in this environment —
left to CI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aJu9ghvuT8WqC31ZEGTBA

2026-07-17 13:48:37 +00:00

Claude

1dca9d7d05

feat(faces): indexing pipeline + DI wiring

Phase 2 increment 5:
- FaceIndexingService: a FileLifecycleHook that, on image upload, detects +
  embeds faces in a background task and stores them. Dedup-aware (clones an
  identical blob's faces instead of re-running inference), reindexes on
  overwrite, and relies on the DB cascade for deletes. Completely inert when
  no model is ready.
- DI: registers the hook in the FileLifecycleService chain and exposes
  PeopleService in AppState — both gated on OXICLOUD_ENABLE_FACES, both
  using the default no-op analyzer until the operator wires a real ONNX
  model.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JW6ghFMDtnRYuYNzZhb47M

2026-06-19 11:46:57 +00:00

3 Commits