Commit Graph

19 Commits

Author SHA1 Message Date
Edouard Vanbelle ba620166ee feat(grant): clean up expired grants 2026-07-12 18:37:13 +02:00
Edouard Vanbelle 7e34045ff8 feat(drive): fix webdav back-compat
add env variable `OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX`
    which is by default:
    `OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX="@drive"`

    so `/webdav/` -> points to user's personal drive (**backward compatibilit**y)
    `/web/dav/@drive/{uuid|drive name}/` points to the respective drive

    if admins want directly `/webdav/` pointing to list of drives they need to:
    `OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX=""`

    + ensure lock is per user (RFC 4918 §9.11)

    fix: #554
2026-07-06 22:14:50 +02:00
Edouard Vanbelle 6b8e2ba49c test(drive): check quota calculus 2026-06-24 23:01:34 +02:00
Claude e3f04d58aa Stream uploads directly into the CDC chunk store (no spool, single write)
Every upload surface previously wrote each byte to disk twice: the HTTP
body was spooled to a temp file (or assembled from chunk parts), then
mmap-re-read for FastCDC analysis, and finally the new chunks were
written to the blob backend. CDC could not start until the last byte
arrived, so large uploads paid receive + reread + rewrite latency.

The dedup engine now chunks, hashes and settles the stream WHILE it
arrives (fastcdc AsyncStreamCDC + incremental BLAKE3):

- Each batch of distinct chunks is pinned-or-classified by ONE
  `UPDATE … RETURNING` (no check-then-bump TOCTOU; pinned chunks can't
  be reclaimed mid-upload), and only chunks the store doesn't have are
  written — a full dedup hit performs zero content writes.
- Durability before visibility is preserved: one batched fsync sweep,
  then one batched INSERT, then the manifest. Identical concurrent
  uploads are resolved at the manifest INSERT via ON CONFLICT (the
  loser releases its references and becomes a dedup hit).
- A drop guard rolls back pins and surfaces written-but-unregistered
  chunks to GC if the request future is cancelled mid-stream.
- MIME sniffing now peeks the first bytes in-flight; client-requested
  MD5/SHA-256 checksums are computed by a stream tee — the post-upload
  re-read of the assembled file is gone.

All surfaces converge on the new interfaces::upload_ingest helper:
REST multipart, WebDAV PUT, NextCloud PUT, WOPI PutFile, the dedup
endpoint, and both chunked-upload completions (which now stream their
ordered parts straight into the store instead of writing an assembled
file — chunk parts persist until finalize, so completion is genuinely
retryable). The legacy blob re-chunk migration streams from the
backend with no spool file either.

Legacy removed: store_from_file + mmap CDC analysers + temp-path
plumbing through every port (pre_computed_hash, save_file_from_temp,
update_file_content_from_temp), upload_spool + assembled-file
assembly in both chunked services, create_file/update_file byte-slice
variants (no callers), common::temp, the OXICLOUD_UPLOAD_TMPDIR
config, and the memmap2 dependency.

Verified end-to-end against PostgreSQL 16: 8 MB upload (26 chunks),
identical re-upload (dedup hit, zero writes), 3-byte edit re-upload
(26 chunks, 1 written), byte-identical downloads, Range across chunk
boundaries, concurrent identical-upload race (manifest ref 2), and
trash-empty reclaiming exactly the unshared chunk while the shared 25
survive for the edited file. The empty/sub-8KB multipart path found a
post-EOF re-poll panic in the MIME peek (fixed with fuse + regression
test).

https://claude.ai/code/session_01WdNenpnujNR2sc32XVvwfS
2026-06-11 13:06:33 +00:00
Edouard Vanbelle c7b359eae3 chore(vitepress): link new /config/storage-fine-tuning doc 2026-06-09 11:12:18 +02:00
Edouard Vanbelle 50ea406719 feat(OXICLOUD_DIRECT_PUT_MAX_BYTES): add a security limit on direct PUT
ensure files does not exeed OXICLOUD_MAX_UPLOAD_SIZE, prefer to deny from header rather consuming bandwidth
    add OXICLOUD_DIRECT_PUT_MAX_BYTES for direct PUT (non chunked), admins can fine tune their prefered values
2026-06-09 11:06:44 +02:00
Edouard Vanbelle 4e36de49eb feat(uploads): add OXICLOUD_CHUNK_DIR + documentation for admins
explain OXICLOUD_CHUNK_DIR and OXICLOUD_UPLOAD_TMPDIR
    and also the OXICLOUD_CHUNK_MAX_BYTES & OXICLOUD_UPLOAD_TMPDIR
    to help administratorrs to defined correctly their storage architecture
2026-06-09 11:06:44 +02:00
Edouard Vanbelle 8cc21f17c5 feat(notify): add notif to internal users when granted
- add coalesced protection to avoid mail bombing if an invited goes many grant in a short period
    - add resentd method in share menu item (work for both internal and external users)
    - user can disable email notification via his properties
    - add env variable from admin to disable notifications
2026-06-05 11:25:06 +02:00
Edouard Vanbelle 044bd76738 feat(i18n): add i18n on server side
- remove the hardcoded list of locales in favor of a discovry on start time
    - server will stop on badly formatted locale .json
    - add server.* entries for serer side translation

    server side translation will be used for templating and email
    note: no json in some embded html (like in /magic), amount of work was similar
2026-06-03 13:27:12 +02:00
Edouard Vanbelle 03f63ad103 feat(external users): email sanity + mock SMTP
- SMTP has a mock to enable end to end test and validate the whole path
     (via OXICLOUD_SMTP_MOCK)
    - add email normalisation ( including punicode)
    - api to share to external user
2026-06-03 00:31:59 +02:00
Edouard Vanbelle 408b084ee6 chore: SMTP is a simple implementation, recommand local server to have a spool 2026-06-03 00:31:59 +02:00
Edouard Vanbelle 8a05ee00cb doc: add OXICLOUD_SMTP_* variables + external variables user options 2026-06-03 00:31:59 +02:00
Edouard Vanbelle d273146138 feat(server): add OXICLOUD_REUSE_PORT env parameter (false by default)
initially server was accepting multiple instances on same port (Linux and MacOS only)
    this can create issues during development (if a dev forget another running instance...)

    Add OXICLOUD_REUSE_PORT variable to activate it, so only admins knowing this feature can activate itt
    (permits multiple instance + let the OS schedduler to decide which process will handle a request)

    if not enabled, other instance will receive exit with a:
    `Error: Os { code: 48, kind: AddrInUse, message: "Address already in use" }`
2026-05-28 12:10:01 +02:00
Jurek ffcbbf99c1 Adding OXICLOUD_DISABLE_REGISTRATION in documentation 2026-05-18 10:45:25 +02:00
Edouard Vanbelle 815c9df613 feat(server): add support of IPv6 (no dual stack) 2026-05-14 20:11:12 +02:00
Edouard Vanbelle 8b08ee165d doc: update example.env and docs/config/env.md 2026-05-10 22:12:55 +02:00
onthebed 0629177278 docs(wopi): document public and callback base URLs
Clarify how OXICLOUD_WOPI_PUBLIC_BASE_URL and OXICLOUD_WOPI_BASE_URL
should be set when the office editor runs on a different hostname.
2026-04-26 13:16:04 +08:00
Diocrafts c0cb86c273 docs: migrate legacy docs to official site 2026-04-22 07:50:41 +02:00
Diocrafts f08cffc0e8 feat: add VitePress docs site, music file picker modal, Dockerfile optimization, i18n keys for 14 locales
- Add docs/ with VitePress site (19 pages): guide, config, architecture, FAQ
- Add GitHub Actions workflow for auto-deploy to GitHub Pages
- Replace music 'Add Tracks' upload picker with in-app audio file browser modal
- Add music picker CSS styles with dark theme support
- Add missing i18n keys (search_audio, no_audio_files, etc.) to all 14 locales
- Optimize Dockerfile: shared base stage, COPY --chmod, consolidated RUN, HEALTHCHECK
- Improve README: docs links, updated stats (222+ tests, 14 languages), feature status
2026-04-11 20:58:34 +02:00