Commit Graph

279 Commits

Author SHA1 Message Date
Edouard Vanbelle 1b9d812175 feat(telemetry): add /metrics prommetheus exporter 2026-08-09 10:41:34 +02:00
Edouard Vanbelle 5c4354bc67 feat(DPoP): add anti replay cache 2026-08-09 01:56:07 +02:00
Edouard Vanbelle 2e6789e506 feat(DPoP): add verification + X-Forwarded-Host X-Forwarded-Proto 2026-08-09 01:56:07 +02:00
Edouard Vanbelle e9495a63ad feat(oidc): permit auto/manual oidc account link/unlink
link are checking that email matches, +email alias are normalize into email
if email is already used on another account, link is not possible
not usurpation risk as the IDP is choosen by the admin
2026-08-08 19:21:13 +02:00
Edouard Vanbelle c4bf2568ba feat(opaque): lower KSF values to support old clients (memory intensive) 2026-08-05 20:48:55 +02:00
Edouard Vanbelle 6965855388 reactor(opaque): normalize to OXICLOUD_AUTH_OPAQUE_* variables 2026-08-04 07:31:32 +02:00
Edouard Vanbelle 168d94370a feat(opaque): legacy login refuses migrated OPAQUE users 2026-08-04 07:03:08 +02:00
Edouard Vanbelle 033146a6c9 feat(opaque): add login opaque exchange 2026-08-04 07:03:08 +02:00
Edouard Vanbelle 0e395ae15f feat(auth): bring opaque (RFC 9807) auth
OPAQUE (RFC 9807) implementation (using `opaque-ke` crate)

    with opaque authentfication, server will never receive the password (in the auth=password mode)
    this is a must have to create trust with users to permit end to end encryption in the future
    (we cannot know if user use the same password/passphrase for his asymetric key or his oxicloud auth,
    this is why server must never have the password)

    pass1: prepare server
2026-08-04 07:03:08 +02:00
Edouard Vanbelle d065f99952 feat(oidc): deprecate OXICLOUD_OIDC_DISABLE_PASSWORD_LOGIN 2026-08-03 22:26:46 +02:00
Edouard Vanbelle 5ebe2d3bae feat(oidc): add auto-redirect for OIDC
add `auto_redirect_if_standalone_oidc` in `OXICLOUD_AUTH_POLICIES`
let admin decide to redirect immediately to IdP if OIDC is the only auth method enabled
2026-08-03 01:01:00 +02:00
Edouard Vanbelle b91f2fab2b feat(oidc): add oidc method in OXICLOUD_AUTH_METHODS
permit an admin to specify `oidc` only as the only method to login/register
note that if OIDC is enabled, the engine always append oidc in OXICLOUD_AUTH_METHODS
we could move to an explicit declaration in a major release
2026-08-03 00:49:40 +02:00
Edouard Vanbelle 7663f803d3 fix: fix services accessig directly to localstorage
Prevent services accessing directly to localstorage and prefer using an astraction layer
to expose full blob. The abstraction layer (dedup services) will cover backend storage
election (local, s3, ...), encryption, etc

This change permit audio_metadata_service, media_metadaa_service, face_indexing_service to handle
blobs without worring of the backend.

note: prefered way to handle blob is the streamed way. Some services may not have this possibility
2026-08-02 22:20:46 +02:00
Edouard Vanbelle 015f2da0f7 refactor(backend): normalize naming convention to backend rather storage
no ambiguity with the backend rather storage
2026-08-02 14:56:29 +02:00
Edouard Vanbelle dd1528de92 feat(rotate-key): show finger print + way to know if can remove key 2026-08-02 04:26:47 +02:00
Edouard Vanbelle a9d5aae781 feat(storage key rot): add rotate services 2026-08-02 02:40:33 +02:00
Edouard Vanbelle 9485ee5540 feat(storage key rot): add blob header engine 2026-08-02 02:40:33 +02:00
Edouard Vanbelle e164689771 feat(storage key rot): remove dead born OXICLOUD_STORAGE_<N>_ENCRYPTION_CIPHER
+ alway ovewrite on storage migration (got issue when migrating with blob already existing and a key change)
2026-08-02 02:40:33 +02:00
Edouard Vanbelle 03c8f87f1f feat(storage key rot): prepare format <cipher1>:<key2>,<cipher2>:<key2>,... 2026-08-02 02:40:33 +02:00
Edouard Vanbelle bbfb106a32 feat(maintenance): add a maintenance notification during backend migration 2026-08-01 18:46:17 +02:00
Edouard Vanbelle f409658c96 feat(storage): hot swap backend on migrate 2026-08-01 17:11:07 +02:00
Edouard Vanbelle 8329b4aa56 feat(storage): improve admin panel 2026-08-01 17:10:06 +02:00
Edouard Vanbelle d7c19570a5 feat(storage): wire choice of storage 2026-08-01 14:19:14 +02:00
Edouard Vanbelle 2de71b6d9a feat(storage): add readonly during storage migration 2026-08-01 13:43:04 +02:00
Edouard Vanbelle 6b7bb67500 feat(storage): jobs can choose storage to migrate/scan 2026-08-01 13:06:09 +02:00
Edouard Vanbelle 354e058114 feat(storage): add multi entry in config 2026-08-01 12:58:43 +02:00
Edouard Vanbelle 82a50936d4 feat(storage-migration): move storage mig. to recoverable job 2026-08-01 11:53:51 +02:00
Edouard Vanbelle 507bc2e98d feat(recoverable-job): add backend_consistency (storage) 2026-07-30 00:15:20 +02:00
Edouard Vanbelle 44356b14f2 feat(recoverable-job): add blobs_consistency job 2026-07-29 22:25:05 +02:00
Edouard Vanbelle 41d83b3053 feat(recoverable-job): add consistency_batch (runs all consistency check) 2026-07-29 01:44:52 +02:00
Edouard Vanbelle 782a5c99bd feat(recoverable-job): add folder_consistency 2026-07-29 01:39:58 +02:00
Edouard Vanbelle 394708c9a1 feat(recov. job): add drive consistency service 2026-07-29 01:25:15 +02:00
Edouard Vanbelle 8177b04786 feat(recoverable-job): wire API (cancel, view run, ...) 2026-07-28 23:25:37 +02:00
Edouard Vanbelle 302d2ff80f feat(recoverable-job): add core engine 2026-07-28 22:39:07 +02:00
Edouard Vanbelle 0e8b1fbbeb refactor(job-registry): simplify the job registering* 2026-07-28 21:13:09 +02:00
Edouard Vanbelle f66f7fa31f feat(job-registry): remplace /api/admin/internal/trigger-*
remplace /api/admin/internal/trigger-* to /api/admin/jobs/{...}/trigger
remove OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS
2026-07-27 23:54:43 +02:00
Edouard Vanbelle dfedde54a4 feat(job-registry): wire /api/admin/jobs/* 2026-07-27 23:38:12 +02:00
Edouard Vanbelle 72b99f5b9a refactor: apply clippy 2026-07-27 23:06:02 +02:00
Edouard Vanbelle 1190dfef87 feat(job-registry): wire grant cleaner 2026-07-27 23:03:22 +02:00
Edouard Vanbelle f740de3c10 feat(job-registry): wire dedub_gc 2026-07-27 22:56:02 +02:00
Edouard Vanbelle f5f794fde5 feat(job-registry): handle job without periodicity but with trigger 2026-07-27 22:53:13 +02:00
Edouard Vanbelle e29f3ba067 feat(job-registry): wire storage usage update 2026-07-27 22:33:04 +02:00
Edouard Vanbelle d0f4e1ef5a feat(job-registry): wire engine + wire trash cleaner 2026-07-27 22:24:15 +02:00
Edouard Vanbelle 99e6de6fce feat(search): invalidate user's cache on share or favorite change 2026-07-26 18:18:15 +02:00
Edouard Vanbelle c22741bc7f refactor(search): normalize answer to /resources format 2026-07-26 18:18:15 +02:00
Bradley Nelson 9f4bc54113 Merge upstream/main into feat/external-file-mounts
Resolve conflicts between the external-file-mounts feature and upstream's
D5/D7 refactor (per-file provenance, keyset pagination, cross-drive move
gates, resource-access hook, folder-cascade lifecycle hook).

Key resolutions:
- FolderService::new now takes (repo, authz, file_lifecycle, mount_router);
  all callers + DI updated.
- FileRetrievalService / FileManagementService keep both the mount_router
  and the new resource_access_hook / drive_repo / storage_usage wiring.
- list_files_batch_with_perms: adapt the mount branch from offset- to
  keyset (after_name) pagination, mirroring paginate_mount_entries.
- download_file_impl: keep upstream's &HeaderMap + `impl IntoResponse + use<>`
  signature, retain the mount-download branch.
- Mount DTOs: the retired `owner_id` field maps onto created_by/updated_by
  (the mount owner) — the fields the frontend now uses for owner display.
- admin/+page.svelte: keep upstream's user-delete modal + the 'mounts' tab.
- Bump memmap2 0.9.10 -> 0.9.11 (RUSTSEC critical advisory fix) and
  regenerate Cargo.lock against the merged Cargo.toml.
2026-07-21 17:09:36 -06:00
Dionisio Pozo 67fe944c2a Merge pull request #596 from swissiety/rfc-5789-http-patch 2026-07-22 00:32:27 +02:00
Claude 77f13ac643 perf: round 21 — CalDAV/CardDAV row-mapper pre-size, dedup hash-bind & digest-key dedup, CardDAV etag/BDAY emit, NC trashbin content-type
Round 21 of the benchmark-gated perf sweep. Six behaviour-preserving,
allocation-reducing changes, each with a BEFORE/AFTER counting-allocator
section in examples/bench_round21_micro.rs and a byte/-value equivalence
gate; all six pass their deterministic alloc gate (a non-winning AFTER
exits 1 = rollback).

- R1: pre-size the 16 CalDAV/CardDAV row-mapper Vecs (+1 HashMap) with
  Vec::with_capacity(rows.len()) — the ROUND20 §I1 file-side pattern
  extended to the calendar/contact repos it deferred. 7 → 1 allocs/op.
- R2: settle_batch binds a borrowed Vec<&str> instead of cloning every
  chunk hash into a Vec<String> (sqlx encodes &[&str] as text[]
  identically; favorites_pg_repository.rs:271 precedent). 33 → 1 allocs/op,
  39x wall.
- R3: store_loose_chunks keys its intra-request dedup set on the raw
  [u8;32] BLAKE3 digest and moves the hex on a duplicate (the ROUND17 §D2
  pattern applied to the delta-upload sibling). 401 → 209 allocs/op.
- R4: CardDAV getetag emits borrowed pre-escaped &quot; quotes via a shared
  write_quoted_etag helper (ROUND20 §C1 pattern, all 4 CardDAV etag sites).
  3 → 0 allocs/op.
- R5: BDAY stamped via the new fmt::compact_date stack renderer instead of
  chrono's strftime interpreter (chrono fallback out of the 4-digit-year
  range; byte-identical, unit-tested vs chrono). 2 → 0 allocs/op, 10.5x wall.
- R6: NC trashbin folder content-type via Cow::Borrowed instead of
  .to_string() on the constant (ROUND16 §M1 pattern). 1 → 0 allocs/op.

See benches/ROUND21.md for the full write-up and the deferred-items list
(HeaderMap-clone hot handlers, Query→typed-struct, WebDAV dead-props
HashSet, and others surfaced by the audit that want their own validated
pass). Validated: cargo fmt, cargo clippy --features bench --all-targets
-D warnings, cargo test --lib --features test_utils (529 passed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015gHVq5Wy2TzdWeSqtEmK6m
2026-07-20 08:48:42 +00:00
Claude 867e1fe259 perf: round 20 — iCal/vCard parse allocs, owned-DTO moves, Result-collect pre-size, NC etag/favorites emit
Benchmark-gated (benches/ROUND20.md), same rule as rounds 2-19: every change
ships with a BEFORE/AFTER counting-allocator micro-benchmark and a byte-value
equivalence gate; a non-winning AFTER is rolled back (never applied). The
rollback rule is encoded in the harness (GATE FAIL exit). All 8 sections pass.

Reproduce: cargo run --release --features bench --example bench_round20_micro

- A1 CalendarEvent iCal parse: replace the throwaway per-property
  HashMap<String,Vec<String>> (DTSTART/DTEND/RECURRENCE-ID) with a direct
  VALUE=DATE scan; prop_with_params kept #[cfg(test)] (6->2 allocs/event, 4.2x)
- A2 UserDto::from: add User::into_parts and MOVE image (<=512 KiB data URI)
  + ui_preferences JSON instead of cloning on every /api/auth/me (27->14 allocs)
- A3 parse_vcard: drop the per-line to_ascii_uppercase copy + the lines Vec;
  promote ascii_ci_contains to common::text and share it (8->1 allocs/contact)
- A4 Calendar/AddressBook DTO: into_parts move incl. custom_properties map (18->10)
- I1 file-listing repos: collect::<Result<Vec>>() size-hints to 0 and grows from
  capacity 0; pre-size with Vec::with_capacity (8->1 container reallocs, 4 sites)
- I4 plaintext_stream: lazy emit iterator instead of eager Vec collect (43x wall)
- C1 NC write_etag_element: borrowed pre-escaped quote events, no owned quoted
  String/escape re-alloc; byte-identical output (3->0 allocs/PROPFIND row)
- C3 NC favorites REPORT: map.remove() move instead of get().clone() (~7 allocs/fav)

Deferred (documented in ROUND20.md): NC oc:id/trashbin buffer reuse, I1 sibling
CardDAV/CalDAV listing paths, Contact JSONB Json<Vec<_>> decode, dedup
settle_batch &str bind, and a fast DoS-resistant hasher for hot trusted-key maps
(needs a dependency decision).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JsJjcVX9RoN96DMa35Wqzd
2026-07-20 00:18:54 +00:00
Claude 9754aecfa9 perf: round 19 — auth/WOPI/vCard/PROPFIND per-request & per-row alloc cuts
Benchmark-gated (examples/bench_round19_micro.rs, benches/ROUND19.md): every
section ships a BEFORE/AFTER counting-allocator arm with a byte/-value
equivalence gate and a GATE-FAIL-rollback exit. All eight pass. No Postgres.

- M1 verify_basic_auth cache key: blake3::hash(format!("{u}:{p}")) → incremental
  Hasher (byte-identical key, 2→0 allocs on every Basic-auth DAV request)
- M2 WopiTokenService: prebuild Validation/DecodingKey/EncodingKey in new()
  instead of per-call (mirrors JwtTokenService; 16→12 allocs/validate)
- V1/V2 vCard emit (contact_to_vcard/generate_vcard): FN fallback drops the
  throwaway to_string, NOTE skips the escape copy for newline-free notes, REV
  uses new common::fmt::compact_ical_utc stack renderer (11.5× vs chrono
  strftime, 3→0 allocs); per-contact 9→4 allocs
- M4 trash_service::row_to_item_dto: move name/path/blob_hash out of the owned
  row instead of cloning (3 clones/file row gone)
- M5 search cache key: Uuid::hyphenated().encode_lower stack buffer instead of
  to_string (identical u64 key, 1→0 allocs/request)
- M6 streaming PROPFIND: reuse one href buffer across the page instead of a
  format! per child (native + NC handlers; 192→3 allocs on a 64-child page)
- M7 nextcloud extract_url_user: return Cow instead of forcing into_owned
  (zero-alloc on the common ASCII-username path)

common::fmt::compact_ical_utc added with chrono-parity unit tests (CASES +
60-year sweep). cargo fmt + clippy --all-targets clean; 526 lib unit tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ront9bk7YMoffVQkGG47gh
2026-07-19 22:29:48 +00:00