Commit Graph

376 Commits

Author SHA1 Message Date
Dionisio Pozo 22e09c09be Merge pull request #538 from swissiety/webdav-litmus-compliance
implement dead properties for nextcloud handler and fixup frontend migration leftover
2026-07-13 09:36:43 +02:00
M.Schmidt 7011fdff5a Merge origin/main into webdav-litmus-compliance 2026-07-12 22:25:12 +02:00
Edouard Vanbelle ba620166ee feat(grant): clean up expired grants 2026-07-12 18:37:13 +02:00
Edouard Vanbelle c1e46910b0 feat(music): move playlist to authz engine 2026-07-08 01:04:03 +02:00
Edouard Vanbelle a2ad7757c3 feat(calendar,addressbook): add tests for authz 2026-07-08 01:03:25 +02:00
Edouard Vanbelle 0fcd617fd1 feat(calendar,addressbook): migrate share to authz engine
migrate DB entries to authz engine
    and wire authz engine to caldav and carddav
2026-07-08 01:03:25 +02:00
Edouard Vanbelle b5881c7114 feat(calendar,addressbook): prepare authz engine 2026-07-08 01:03:25 +02:00
Edouard Vanbelle 7e34045ff8 feat(drive): fix webdav back-compat
add env variable `OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX`
    which is by default:
    `OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX="@drive"`

    so `/webdav/` -> points to user's personal drive (**backward compatibilit**y)
    `/web/dav/@drive/{uuid|drive name}/` points to the respective drive

    if admins want directly `/webdav/` pointing to list of drives they need to:
    `OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX=""`

    + ensure lock is per user (RFC 4918 §9.11)

    fix: #554
2026-07-06 22:14:50 +02:00
Edouard Vanbelle 75601beb43 security(wopi): add authz to Wopi 2026-07-05 23:31:44 +02:00
Edouard Vanbelle 1786fe4111 security(nextcloud): chroot-aware display paths + recent race fix
strip_chroot_prefix replaces the hardcoded "Personal/" strip
    in NC trashbin PROPFIND, OCS unified search, and REPORT
    (favorites + search). Handles composed chroots, drops
    cross-chroot items instead of surfacing malformed paths, and
    fixes the leading-slash mismatch (FolderDto path has '/', DB
    paths don't) that silently dropped every NC trashbin item
    post-D3. OCS keeps a first-segment fallback (results
    legitimately span drives, no single chroot).

    uploads_handler switches to nc_to_internal_path(chroot, …)
    for the two remaining hardcoded "Personal/" sites, closing
    the D1 TODO markers.

    RecentService::record_item_access is split from a new
    record_item_access_internal (no authz) used by
    RecentRecordingHook. Round 1's authz.require widened the
    tokio::spawn race past tests/api/recent.hurl step 7; the
    internal path skips the redundant Read gate — upstream
    _with_perms already enforced it.

    Tests: 8 unit tests pin strip_chroot_prefix (leading slash,
    composed chroots, sibling-leak rejection, partial-prefix,
    empty-chroot). drives_membership.hurl step 21b/22b cover
    Editor upload → 201 / Viewer upload → 404 fresh + overwrite
    with fixture cleanup at 30c. test_nc_move_copy_delete_trash
    K1 pins the actual original-location value.
2026-07-05 22:53:05 +02:00
Edouard Vanbelle 0342bae300 security(nextcloud): add authz to PUT verb 2026-07-05 22:53:05 +02:00
Edouard Vanbelle b95e740b2f security(music): ensure read permission via authz 2026-07-05 22:53:05 +02:00
Edouard Vanbelle 37467ed9d3 feat(drive): remove all owner_id from {File,Folder}Dto 2026-07-03 01:13:53 +02:00
Edouard Vanbelle 206a780494 feat(drive): stop writing in {file,folder}.user_id
plan:

    ```
    What step 5 is actually doing

    Goal: stop writing to storage.files.user_id / storage.folders.user_id on new INSERTs. This is a prerequisite for step 6 (dropping the column entirely). The column has existed since the pre-D0 single-owner era; every read that used to key on it has already been migrated to drive-membership grants over the last N days.

    What has to change together for this to be safe:

    ┌──────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────┐
    │    Piece     │                                                          What changes                                                           │                                                  Why                                                  │
    ├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
    │ Migration    │ Swap storage.files uniqueness indexes from (folder_id, name, user_id) to (drive_id, folder_id, name)                            │ Otherwise, two new rows with user_id = NULL would both be allowed (PG treats NULLs as distinct) —     │
    │ (a)          │                                                                                                                                 │ uniqueness silently breaks                                                                            │
    ├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
    │ Migration    │ ALTER user_id DROP NOT NULL on both tables                                                                                      │ Otherwise, dropping the INSERT bind violates NOT NULL and every write 500s                            │
    │ (b)          │                                                                                                                                 │                                                                                                       │
    ├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
    │ Migration    │ Drop dead user_id-leading indexes                                                                                               │ Cheap cleanup — nothing scans them anymore                                                            │
    │ (c)          │                                                                                                                                 │                                                                                                       │
    ├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
    │ PL/pgSQL (d) │ Rewrite storage.copy_folder_tree without user_id in the INSERT column list                                                      │ Cross-drive copy runs entirely in SQL, needs the same treatment                                       │
    ├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
    │ Rust (e)     │ ~12 INSERT column-list drops across folder_db_repository, file_blob_write_repository, drive_pg_repository, dedup_service,       │ Actual write path                                                                                     │
    │              │ folder_service, load-seed.rs                                                                                                    │                                                                                                       │
    ├──────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────┤
    │ Rust (f)     │ Simplify folder_db_repository::create_folder's parent lookup to only fetch drive_id (was fetching (user_id, drive_id))          │ It's fetching a value it no longer needs                                                              │
    └──────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────┘
    ```
2026-07-03 00:23:35 +02:00
Edouard Vanbelle a0e5c34950 feat(webdav): impl if header RFC 4918 §10.4
this fix an issue raised by webdav litmus test
2026-07-02 20:59:18 +02:00
Edouard Vanbelle 91435e6be1 feat(drive): webdav handler based on drive
and add authz
2026-07-02 20:41:33 +02:00
Edouard Vanbelle f5f5b1167f feat(drive): remove _for_owner() and use authz
remove related IDOR protection as Hurl tests are covering this surface
2026-07-02 01:14:02 +02:00
Edouard Vanbelle 09790644f3 feat(drive): repl user_id by caller has read access
repl user_id by caller has read access in readonly functions
    using CALLER_CAN_READ_DRIVE constant

    ensure webdav preview is using the permission handler
2026-07-02 01:03:39 +02:00
Edouard Vanbelle 79bc60899b feat(drive): improvepermission check and simplify functions 2026-07-02 00:29:33 +02:00
Edouard Vanbelle 858139ef3b feat(drive): prepare removal of user_id
this commit changes GET /api/<resources> to return resource caller has access to
    this is not anymmore resources users is owner of
2026-07-02 00:01:20 +02:00
M.Schmidt 0ad0ea1a43 fix(webdav): reject PROPPATCH on protected DAV:/oc:/nc:/ocs: props
DeadPropertyStore let PROPPATCH set any namespace/name verbatim,
incl. names the server itself emits as live state (DAV: entirely,
plus oc:/nc:/ocs: names used by write_file_response /
write_folder_response). That either forges a live prop or stores
dead rows nothing ever reads. is_protected_property() denylists
them; both PROPPATCH handlers (native + NC) now return per-property
403 instead of storing. oc:favorite stays writable via its existing
special-case, which runs before the protection check.
2026-07-01 22:54:48 +02:00
Edouard Vanbelle 01ff7dab0b feat(drive): impl policy photo + music policies
add `include_in_photo_index` and `include_in_music_index` policies
    both true for default personal drive

    photo is implemented
    music is not yet implemented
2026-07-01 21:57:37 +02:00
M.Schmidt f220217659 feat(nc-webdav): generic dead-props on PROPPATCH/PROPFIND/REPORT
NC PROPPATCH only special-cased oc:favorite, silently dropped every
other prop while claiming 200. PROPFIND had no dead-prop lookup at
all. Reuse native /webdav/ dead-prop plumbing (now pub(crate)) so NC
surface gets real RFC 4918 §4.2 storage. Missing resource on
PROPPATCH is now 404 instead of a fake success no-op.
2026-07-01 19:59:16 +02:00
Edouard Vanbelle cb7b653a15 feat(webdav): bind dead prop to res. id rather path 2026-06-30 23:35:25 +02:00
Edouard Vanbelle d52c1a2397 fix(webdav): partial fix: remove dead props on del
ensure that dead properties are correctly deleted on resource deletion

    **IMPORTANT**: this is a partial fix:
    dead properties are not deleted if resource is deleted from API !
    code need to be reviewed to attach property directly to resource + use on delete cascade
2026-06-30 21:00:02 +02:00
Edouard Vanbelle 5631b7e062 fix(sql macro): permit code to compile 2026-06-30 19:46:19 +02:00
M.Schmidt 6ca6649e50 fix(webdav): achieve 100% litmus compliance (59/59 tests)
Complete RFC 4918 litmus test suite compliance (basic 16/16,
copymove 13/13, props 30/30) by fixing six categories of failures:

- PropPatchOp: parse PROPPATCH in document order (RFC 4918 §9.2)
  so that remove-then-set and set-then-remove yield different results
- Null namespace: resolve_name now correctly maps xmlns="" to empty-ns
  QualifiedName instead of falling through to DAV: namespace
- Unicode chars: handle quick-xml 0.39's Event::GeneralRef for character
  references (&#65536;) — the parser emits these as GeneralRef, not Text
- MOVE preserves dead props: call rename_resource on MOVE, clearing
  stale destination data even when source has no registered properties
- Malformed PROPFIND: return 400 when body lacks a complete <propfind>
  element (test 2: bare <foo>, test 3: invalid xmlns:prefix="" binding)
- PROPPATCH document order: process ops via Vec<PropPatchOp> instead of
  separate (sets, removes) to honour interleaved remove/set sequences
2026-06-30 08:58:13 +02:00
M.Schmidt 43cf4a2b2b fix(webdav): RFC 4918 handler compliance (COPY/MOVE/MKCOL/PROPFIND/PROPPATCH)
- COPY: return 403 on self-copy, 409 when destination parent missing,
  409 (not 500) when overwriting a locked or conflicting resource
- MOVE: same parent-missing and conflict handling as COPY
- MKCOL: return 405 when collection already exists, 409 when parent
  is missing (no auto-creation of ancestors)
- PUT: return 201 Created for new resources, 204 No Content for overwrites
- PROPFIND: use client-facing URI path (not internal home-folder path)
  for DAV:href values so responses match the request URI
- PROPFIND: include dead properties from DeadPropertyStore in responses
- PROPPATCH: persist set/remove operations to DeadPropertyStore
2026-06-29 22:33:09 +02:00
M.Schmidt b5afa1a855 feat(webdav): implement RFC 4918 dead property store
Add an in-memory DeadPropertyStore backed by RwLock<HashMap> that
stores arbitrary client-supplied XML properties per resource path.
Wire it into AppState so PROPPATCH can persist dead props and PROPFIND
can retrieve them across requests.
2026-06-26 21:19:03 +02:00
M.Schmidt 5a001c4120 fix(webdav): remove spurious calendar_rest_handler module declaration
The declaration was brought in by a cherry-pick but that handler does
not exist on this branch, causing a compilation error.
2026-06-25 22:44:27 +02:00
M.Schmidt 5913fff047 feat(webdav): RFC 4918 prop classification and 404 propstat 2026-06-24 23:08:51 +02:00
M.Schmidt f9999cdd0f fix(webdav): RFC 4918 litmus compliance 2026-06-22 21:37:14 +02:00
Edouard Vanbelle 934d6b4af0 feat(drive): NC chunked upload respect drive's quota
personal drive are based on user's quota
2026-06-29 23:21:17 +02:00
Edouard Vanbelle 583af517c2 fix(drive): fix drive_id on copy to other drive
- fix alsot drive_id exposition
2026-06-29 23:21:17 +02:00
Edouard Vanbelle 66f2aaa250 feat(drive): add drive policie
- add policy forbid_external_sharing
    - add policy forbid_sharing
    - add polocy forbid_cross_drive_move
    - add policy forbid_owner_role_change
2026-06-26 14:04:36 +02:00
Edouard Vanbelle ddb131da8b feat(drive): add policy forbid_public_links 2026-06-26 13:59:02 +02:00
Edouard Vanbelle cfd783cbd3 feat(drive): personal drives have the user's quota in commun 2026-06-26 13:59:02 +02:00
Edouard Vanbelle f4032e9e23 feat(recent): update recent list server side
initially the recent was done client side
    recent files are now directly updated on serverside when accessing a file

    note: nextcloud and webdav voluntary not included
2026-06-26 01:44:19 +02:00
Edouard Vanbelle 2604ac3309 refactor: format syntaxt 2026-06-25 00:07:16 +02:00
Edouard Vanbelle 90474aa885 chore(test): blob lifecyclc with thumbnail cleanup
renable thumbnail test, ensure that blob lifecycle correctly
    trigger thumbnail cleanup on blob deletion

    need to call `/api/admin/internal/trigger-gc?force=true`
2026-06-24 23:50:37 +02:00
Edouard Vanbelle 6b8e2ba49c test(drive): check quota calculus 2026-06-24 23:01:34 +02:00
Edouard Vanbelle a9a8604322 feat(drive): add quota calculus per drive
- quota per drive
    - add 2 internal API endpoints to test purpose
        disabled by default, enable it via `OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS=true`

        this enable:

            /api/admin/internal/trigger-sweep
                to sweep the trash and recalculated quota

            /api/admin/internal/trigger-gc
                to garbage orphan blobs

        use full for end to end tests and validate lifecycles
2026-06-24 22:28:31 +02:00
Edouard Vanbelle b73f176024 feat(drive): permanent deletion per drive 2026-06-24 21:56:27 +02:00
Edouard Vanbelle 7d24015fc4 feat(drive): add drive deletion
- conditions: drive must be empty
    - deletion forbidden on main personal drive
2026-06-24 21:17:27 +02:00
Edouard Vanbelle 6034dd47d9 feat(drive): improve drive edition from owners 2026-06-24 02:03:17 +02:00
Edouard Vanbelle d77846119f feat(drive): UI: add drive edition for admin 2026-06-24 01:20:44 +02:00
Edouard Vanbelle a5b24a7453 feat(drive): add /api/drive
- permit shared drive creation from oxicloud admin (for now)
    - prepare other personal drive creation (Not implemented), need to validate
    quota policies and strategy first
    - add hurl test to verify permissions
2026-06-23 23:50:57 +02:00
Edouard Vanbelle a82bae4136 feat(drive): api to handle Drive grants 2026-06-22 23:53:00 +02:00
Claude 6e26d1c694 refactor: remove dead OptimizedFileContent::Mmap variant
OptimizedFileContent::Mmap was constructed nowhere — the documented "Tier 2:
memory-mapped I/O (10-100 MB)" path was never wired, so optimized_inner only
ever returns Bytes (<10 MB) or Stream (>=10 MB). The variant survived only as
an enum case plus two dead match arms in the file and share download handlers.

Remove the variant and its arms, and fix the now-misleading retrieval-service
tier docs (everything >=10 MB streams via CDC chunk reassembly with the
backend read-ahead; there is no mmap tier). Behaviour is unchanged — the
deleted arms were unreachable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JG5yYZ9s868mJwqT2Qz7ez
2026-06-22 08:10:54 +00:00
Bradley Nelson e3823ce470 test(e2e): Playwright + Vitest coverage harness and test instrumentation
Add an end-to-end and unit test suite for the SvelteKit frontend:

- Playwright e2e specs (tests/e2e/spa) with a throwaway container stack,
  codegen scenarios, and an Istanbul-based coverage report pipeline.
- Vitest unit tests across API endpoints, components, stores and composables.
- `data-testid` hooks on interactive elements (AppShell, FileViewer,
  ShareDialog, search, photos, files breadcrumbs, login/Nextcloud flows,
  public share pages) so the e2e suite can target them deterministically.
- Serve the SPA app-shell CSP from a <meta> policy (svelte.config.js) plus a
  middleware that skips the CSP header on HTML; move the Nextcloud Login Flow
  v2 grant page to the SvelteKit /nextcloud/login route.
- `just front-codegen` recipe and start-server-spa.sh harness.

Make the test environment robust and consistent:
- Install a deterministic in-memory localStorage/sessionStorage in the Vitest
  setup so storage behaves identically across Node versions (Node 26 ships a
  native Web Storage global that otherwise shadows jsdom's).
- Pin devenv to Node 26 + PostgreSQL 18 and pin every CI job to Node 26.3.0
  so the dev shell and CI run the same toolchain versions.

Repair the API/WebDAV (hurl) suite, which had drifted from the backend:
- Migrate the removed `/api/folders/{id}/listing` endpoint to `/resources`
  (cursor-paginated `{items:[{resource_type,resource}]}` shape) across the
  batch-copy, grants, nested-group, and WebDAV NC tests + the dav_helpers
  wipe routine.
- Stop photos_etag from uploading the dedup-tracked fixture so the dedup
  blob-lifecycle test can own its content-addressed blob exclusively.
- dedup_create now asserts the idempotent same-content re-upload (201 +
  existing file id) instead of the stale 409 expectation.

Generated coverage reports, nyc output and the e2e server runtime data dir
are gitignored rather than committed.
2026-06-22 00:05:06 -06:00