Compare commits

16 Commits

Author SHA1 Message Date
cjw b7640e9be4 x
CI / changes (push) Has been cancelled
CI / Build (push) Has been cancelled
Deploy Docs / build (push) Has been cancelled
Docker Build and Test / Build and Test Docker Image (push) Has been cancelled
Docker Publish (release, main, dry-run) / Pre-publish Tests (push) Has been cancelled
CI / Frontend — svelte-check, ESLint, Stylelint, Prettier (push) Has been cancelled
CI / Message-bus spec — AsyncAPI + TypeScript DTO drift (push) Has been cancelled
CI / Migration ordering (new migrations postdate target branch) (push) Has been cancelled
CI / Rustfmt (push) Has been cancelled
CI / Clippy (push) Has been cancelled
CI / Wasm — fmt + clippy (push) Has been cancelled
CI / Wasm — release tests (push) Has been cancelled
CI / Plugins — fixtures + runtime tests (push) Has been cancelled
CI / Server Unit and Functionnal Tests (push) Has been cancelled
CI / Security Audit (push) Has been cancelled
CI / API, WebDAV & OIDC tests (push) Has been cancelled
CI / Bundled-assets binary — embed + SPA-serve integration (push) Has been cancelled
CI / WebDAV RFC 4918 — litmus (59/59) (push) Has been cancelled
CI / CalDAV + CardDAV — python-caldav (push) Has been cancelled
CI / Frontend end-to-end tests (via Playwright) (push) Has been cancelled
Deploy Docs / deploy (push) Has been cancelled
Docker Publish (release, main, dry-run) / Build & Push Multi-Arch (push) Has been cancelled
2026-09-20 00:28:52 +08:00
cjw d33d1932b6 perf(thumbnail): cache attached-blob lookups on the request path
Every thumbnail request paid an uncached storage.file_attached_blobs
point query before it could answer — including 304 revalidations and
RAM thumbnail hits, where the ETag path (thumbnail_content_id) probes
the row every time and tier 2b probes it again with the same key. A
photos grid revalidating 60 thumbnails per visit meant 60+ point
queries per browse, repeated on every visit.

find_attached_blob now reads through a process-local moka cache in
DedupService, keyed by the row's (file_id, kind, variant) PK, holding
positive and negative entries (most files have no attached preview, so
the negative side carries the win). Two rules keep it honest:

- DB faults are surfaced as Err and never cached — a transient outage
  cannot freeze "no attached blob" into a negative entry (a read
  failure is never proof that data is absent). The public signature is
  unchanged; the SQL body moved to find_attached_blob_uncached.
- Writes invalidate eagerly: store_attached_blob and the Inserted arm
  of store_attached_blob_if_absent on success, and deletions via
  ThumbnailRefreshHook::on_file_deleted, which all three production
  delete paths (single file, folder cascade, trash clear) fire after
  the DELETE commits. The 60s TTL bounds only what the process cannot
  see (bare SQL, copy_file_satellites races).

The Nextcloud preview endpoint rides the same lookup and benefits
identically. Five in-memory contract tests pin the cache behaviour,
including the fault-not-cached rule.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-20 00:28:00 +08:00
cjw 68e21f4bef fix(share): stream single-file shares through /api/s/{token}/file/{id}
CI / changes (push) Has been cancelled
CI / Build (push) Has been cancelled
Docker Build and Test / Build and Test Docker Image (push) Has been cancelled
Docker Publish (release, main, dry-run) / Pre-publish Tests (push) Has been cancelled
CI / Frontend — svelte-check, ESLint, Stylelint, Prettier (push) Has been cancelled
CI / Message-bus spec — AsyncAPI + TypeScript DTO drift (push) Has been cancelled
CI / Migration ordering (new migrations postdate target branch) (push) Has been cancelled
CI / Rustfmt (push) Has been cancelled
CI / Clippy (push) Has been cancelled
CI / Wasm — fmt + clippy (push) Has been cancelled
CI / Wasm — release tests (push) Has been cancelled
CI / Plugins — fixtures + runtime tests (push) Has been cancelled
CI / Server Unit and Functionnal Tests (push) Has been cancelled
CI / Security Audit (push) Has been cancelled
CI / API, WebDAV & OIDC tests (push) Has been cancelled
CI / Bundled-assets binary — embed + SPA-serve integration (push) Has been cancelled
CI / WebDAV RFC 4918 — litmus (59/59) (push) Has been cancelled
CI / CalDAV + CardDAV — python-caldav (push) Has been cancelled
CI / Frontend end-to-end tests (via Playwright) (push) Has been cancelled
Docker Publish (release, main, dry-run) / Build & Push Multi-Arch (push) Has been cancelled
The public landing page's inline media preview (added in 6ee26e46)
requests /api/s/{token}/file/{item_id}, but assert_file_in_share went
through resolve_folder_share, which hard-rejects non-folder shares —
so for a single-file share the video src got a 400 and the player
rendered empty: the preview box appeared but nothing would play.

The AuthZ gate now branches on item_type instead: a file share only
accepts file_id == share.item_id, a folder share still requires the
file to live in the shared subtree, and anything else is NotFound
(same shape as "file doesn't exist", preserving anti-enumeration).
Password/expiry checks still happen inside
get_shared_link_with_unlock, unchanged.

Also extend public_shares.hurl section 8b: the file-share token must
stream its own item (200 + inline disposition) and reject an outsider
file id with 404.

NOTE: fmt/clippy/api-test could not run on the authoring machine (no
Rust toolchain or Docker) — run `just check` + `just api-test` before
pushing.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-15 00:53:34 +08:00
cjw d677f92b7f fix(main): consume reuse_port param on Windows builds
Docker Publish (release, main, dry-run) / Pre-publish Tests (push) Has been cancelled
Docker Publish (release, main, dry-run) / Build & Push Multi-Arch (push) Has been cancelled
Docker Build and Test / Build and Test Docker Image (push) Has been cancelled
CI / Frontend — svelte-check, ESLint, Stylelint, Prettier (push) Has been cancelled
CI / Message-bus spec — AsyncAPI + TypeScript DTO drift (push) Has been cancelled
CI / Migration ordering (new migrations postdate target branch) (push) Has been cancelled
CI / Rustfmt (push) Has been cancelled
CI / Clippy (push) Has been cancelled
CI / Wasm — fmt + clippy (push) Has been cancelled
CI / Wasm — release tests (push) Has been cancelled
CI / Plugins — fixtures + runtime tests (push) Has been cancelled
CI / Security Audit (push) Has been cancelled
CI / Bundled-assets binary — embed + SPA-serve integration (push) Has been cancelled
CI / CalDAV + CardDAV — python-caldav (push) Has been cancelled
CI / changes (push) Has been cancelled
CI / Build (push) Has been cancelled
CI / Server Unit and Functionnal Tests (push) Has been cancelled
CI / API, WebDAV & OIDC tests (push) Has been cancelled
CI / WebDAV RFC 4918 — litmus (59/59) (push) Has been cancelled
CI / Frontend end-to-end tests (via Playwright) (push) Has been cancelled
SO_REUSEPORT is Unix-only; the parameter is only read inside the
#[cfg(not(windows))] block, so -D warnings fails the build with an
unused-variable error on Windows hosts while Linux CI stays green.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-14 16:02:03 +08:00
cjw 5f9f91ee2f fix(blob): fsync blob files via a write handle so Windows works
The sync sweep (and the EXDEV copy fallback) opened blob files with
File::open — a read-only handle — before calling sync_all. POSIX fsync
accepts read-only fds, so Linux never noticed, but Windows
FlushFileBuffers requires a GENERIC_WRITE handle and fails with
ACCESS_DENIED (os error 5) on every call. On Windows deployments the
strict sweep therefore failed every deferred sync, and the post-copy
fsync silently never happened.

Files now open via OpenOptions::write(true); the best-effort directory
fsyncs keep the read-only POSIX dirent idiom unchanged.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-14 16:02:02 +08:00
cjw 6ee26e46e6 feat(share): inline media preview on single-file share landing
The public share page only rendered media previews for FOLDER shares —
a single-file share got a bare icon + download button, even for images
and videos the browser can play natively.

Backend: resolve the shared file's mime_type + size at read time and
expose them on ShareDto (meta + password-verify endpoints, one shared
enrichment helper). Display-only enrichment: a failed file lookup
leaves the fields None instead of failing the response — the download
endpoint still surfaces the real error.

Frontend: the 'file' view now reuses the folder grid's lazyVideo
(poster-seek + retry) for video and imageRetry for images, with
Range-aware streaming already provided by /api/s/{token}/file/{id}.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-14 16:00:30 +08:00
Dionisio Pozo 4d067b3fc6 Merge pull request #727 from EdouardVanbelle/chore/ambition
Docker Publish (release, main, dry-run) / Pre-publish Tests (push) Has been cancelled
Docker Publish (release, main, dry-run) / Build & Push Multi-Arch (push) Has been cancelled
CI / changes (push) Has been cancelled
CI / Build (push) Has been cancelled
Docker Build and Test / Build and Test Docker Image (push) Has been cancelled
CI / Frontend — svelte-check, ESLint, Stylelint, Prettier (push) Has been cancelled
CI / Message-bus spec — AsyncAPI + TypeScript DTO drift (push) Has been cancelled
CI / Migration ordering (new migrations postdate target branch) (push) Has been cancelled
CI / Rustfmt (push) Has been cancelled
CI / Clippy (push) Has been cancelled
CI / Wasm — fmt + clippy (push) Has been cancelled
CI / Wasm — release tests (push) Has been cancelled
CI / Plugins — fixtures + runtime tests (push) Has been cancelled
CI / Server Unit and Functionnal Tests (push) Has been cancelled
CI / Security Audit (push) Has been cancelled
CI / API, WebDAV & OIDC tests (push) Has been cancelled
CI / Bundled-assets binary — embed + SPA-serve integration (push) Has been cancelled
CI / WebDAV RFC 4918 — litmus (59/59) (push) Has been cancelled
CI / CalDAV + CardDAV — python-caldav (push) Has been cancelled
CI / Frontend end-to-end tests (via Playwright) (push) Has been cancelled
2026-09-14 10:23:33 +08:00
Dionisio Pozo fd9e1e669f Merge pull request #728 from EdouardVanbelle/feat/username-lowercase 2026-09-14 10:23:20 +08:00
Dionisio Pozo b5648b82f9 Merge pull request #726 from EdouardVanbelle/docs/message-bus 2026-09-14 02:24:24 +08:00
Edouard Vanbelle a95a6b106c feat(username): normalize username into lowercase
- normalize username into lowercase (this is already ASCII only)
- permit users to login with their username with insensitive case
- if a disabled account is reactivated and got a collision, it will normalize it too
- server will stop on collision (ex: 2 entries with `Alice` and `alice`)
  in a such case admin can run:

```
oxicloud migrate lowercase-usernames --dry-run
```
then
```
oxicloud migrate lowercase-usernames
```
2026-09-13 19:53:09 +02:00
Edouard Vanbelle 0b9e8bfe23 docs(plan): case-insensitive usernames
Records the design for issue #691 (make usernames case-insensitive):
silently lowercase on ingest, explicit `oxicloud migrate
lowercase-usernames [--dry-run]`, refuse-to-boot until DB is fully
lowercase. Includes the chunked-upload directory rename step and the
OIDC JIT lowercase fix surfaced during the design sweep.

Design deferred pieces (display_name split, WebDAV URL redesign) are
listed under "Not in scope" so the boundary is explicit.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-13 17:00:36 +02:00
Edouard Vanbelle 03f2956457 docs(message-bus + notifications): add arch doc 2026-09-13 14:12:47 +02:00
Edouard Vanbelle 47346b9309 chore: ambition and target 2026-09-13 13:21:05 +02:00
Dionisio Pozo 4840930767 Merge pull request #724 from EdouardVanbelle/fix/ref_count 2026-09-13 16:29:34 +08:00
Dionisio Pozo 69fc88982e Merge pull request #723 from EdouardVanbelle/feat/notifications 2026-09-13 16:29:14 +08:00
Edouard Vanbelle ca85ac7307 fix(file_attached): doe not increment ref_count if new attachement has same hash
- do not increment ref_count if new attachement to a file with same data
- add audit log to help identifying other future issue in ref_count
- prevent race condition while attaching a blob
2026-09-13 02:00:32 +02:00
68 changed files with 5677 additions and 426 deletions
+129
View File
@@ -2,6 +2,72 @@
This file provides guidance to coding agents (Claude Code, Codex, Cursor, Aider, …) working with this repository. Claude Code reads it via `@AGENTS.md` in `CLAUDE.md`. This file provides guidance to coding agents (Claude Code, Codex, Cursor, Aider, …) working with this repository. Claude Code reads it via `@AGENTS.md` in `CLAUDE.md`.
# Purpose — what OxiCloud is, and what it is not
Read this before designing anything. Most "should we…" questions are answered by
the scale target rather than by taste.
- **Open source, and it stays that way.** MIT (`LICENSE`). This is a constraint on
what you may add, not a footnote: dependencies must be license-compatible — a
GPL/AGPL crate or npm package would force the whole project to relicense, and
that is not on the table. No feature may be gated behind a licence key or an
"enterprise edition", and nothing core may hard-depend on a proprietary service
or SDK. Vendored frontend assets (`frontend/static/vendors/`) carry the same
rule; record the licence when vendoring.
- **Self-hosted**, for an individual or an enterprise. The operator is not an SRE
team: defaults must be safe, failures loud, and nothing may silently depend on a
cloud service.
- **Target scale is up to ~10k users.** Design against that number in both
directions. Do not build million-user machinery (sharding, eventual consistency,
service decomposition) for a load that will never arrive; equally, never ship
anything O(users) per request, or a table scan that is fine at 50 users and
fatal at 10k.
- **Not a mass hoster.** OxiCloud does not claim to serve millions of users on one
deployment, and trade-offs should not pretend otherwise.
- **Decentralised by intent.** Many instances federating beats one large instance —
OpenCloudMesh is one route. Prefer designs that survive "this is one of many
instances" over ones assuming a single authoritative deployment.
Targets:
- **Feature ambition: Google Workspace / Office 365.** Breadth of capability is a
goal, not scope creep.
- **Collaboration is the main feature axis.** OxiCloud is not a personal backup
drive that happens to have sharing bolted on — sharing, shared drives, grants,
co-editing (WOPI) and live updates are the product. When choosing what to build
or how to build it, the multi-user case is the primary one, not the case to
generalise to later. A feature that works only for a single owner is unfinished.
- **Customer target: NextCloud users.** Hence the NextCloud-compatible API surface
(`/remote.php`, `/ocs`, `/status.php`) — compatibility is a feature, and breaking
it costs adopters.
## Design axes
Four things decide an open design question. **Security and resilience are
absolute** — they are not traded against anything. Performance is measured against
the 10k target. Privacy is a direction with a stated endpoint.
- **Resilience.** This is a storage product: **no data loss, no data corruption,
ever.** Anything that can silently drop or alter bytes is a top-severity defect,
not a trade-off. In practice that means: a job that skips work must never report
success (pause at a cursor instead — `docs/plan/jobs-handling-recoverable-error.md`);
a read failure is never proof that data is absent; content-addressing and
ref-counting are load-bearing, not decoration; and consistency checks are
discovery-only unless repair is explicitly requested.
- **Security.** Prefer deny-by-default over assert-later; a guarantee enforced by
the type system or the router beats one a reviewer must remember. AuthZ lives in
the service layer, never in handlers. See `src/AGENTS.md` § AuthZ enforcement
points.
- **Performance.** Measure against 10k users, not a dev instance. The hot paths are
listing, thumbnails and auth — a per-row query or an extra round trip there is a
real regression even when it looks harmless.
- **Privacy.** When the backend belongs to a third party (S3, Azure), encryption at
rest is a *should-have*; **end-to-end encryption is the target.** Designs that
assume the server can always read plaintext will have to be undone — the `Vault`
drive kind is reserved for the E2E case.
Where two conflict, resilience and security win, and the cost is documented.
# Architecture # Architecture
This project is split into two parts: This project is split into two parts:
@@ -258,3 +324,66 @@ CI runs the same `npm run check` (plus Vitest) — commits that fail will not me
- Leave debug `console.log` statements in code - Leave debug `console.log` statements in code
- Use raw color values in CSS — always use CSS custom properties - Use raw color values in CSS — always use CSS custom properties
- Commit without passing all linters (`npm run check` for the frontend; `cargo fmt` + `cargo clippy` for the backend) - Commit without passing all linters (`npm run check` for the frontend; `cargo fmt` + `cargo clippy` for the backend)
# 本地 fork 维护规则(Local fork rules)
> 本节是仅本地追加的内容,不属于上游 OxiCloud。与上游合并时,若本节之外的部分发生冲突,
> 以上游为准;本节始终保留在文件末尾以减少冲突面。
## 背景
本仓库是开源项目 OxiCloud 的本地副本,上游会持续更新。本地修改必须
**可追溯、可合并**:任何时候都要能知道"我们改了什么",以便与上游主分支合并。
## 规则 1:计划与进度必须记录在 `status.md`
- 每次接到非琐碎任务,**开始前**先在 `status.md` 顶部("进行中"区域)写下计划。
- `status.md` 条目格式(每条任务一个区块):
```markdown
### [YYYY-MM-DD] 任务标题
- **状态**: 进行中 / 已完成 / 已放弃(写明原因)
- **计划**: 要做什么、分几步
- **改动文件**: 列出修改/新增的上游文件(相对路径)+ 一句话说明
- **仅本地文件**: 新增的不属于上游的文件(合并时无需处理)
- **上游冲突风险**: 高 / 中 / 低,以及可能与上游哪些文件冲突
```
- 状态只允许进行中/已完成/已放弃三种;完成的任务移入"已完成"区域,保留记录不删除。
## 规则 2:每次修改后立即更新 `status.md`
- **不需要用户提醒**。任何一次代码/文档修改完成后,agent 必须同步更新
`status.md` 中对应条目的状态、改动文件列表和冲突风险。
- 即使任务中途被打断,也要把当前进度写清(做到哪一步、剩下什么),保证
任何 agent(或人)读了 `status.md` 就能接手。
## 规则 3:与上游主分支合并
- **小步提交**:一个任务一个 commit(或少量 commit),commit message 说清楚改了什么。
不要把多天的工作堆成一个巨型 commit,否则合并时无法选择性丢弃。
- **少改上游文件**:能用新增文件解决的(新组件、新模块、新 endpoint)就不要改上游现有文件;
必须改时尽量小而集中,并在 `status.md` 的"上游冲突风险"里注明。
**`AGENTS.md` 本身也因此只允许在文件末尾追加内容,不得改动上游已有的章节。**
- **不改无关格式**:不要顺手重排上游代码、改无关 import 顺序——纯噪音,制造冲突。
- 合并上游的流程:
```bash
git remote add upstream <上游仓库地址> # 只需配置一次
git fetch upstream
git merge upstream/main # 或 rebase,按团队习惯;首次建议 merge
# 解决冲突时:先读 status.md 的"改动文件"列表,逐个文件核对本地意图
git status # 确认没有遗漏的冲突标记
cargo fmt --all && cargo clippy --all-features --all-targets -- -D warnings
just test
```
- 合并完成后,在 `status.md` 新增一条"上游合并"记录:合并到的 upstream commit、
解决过的冲突文件、是否有本地修改被上游覆盖/废弃。
- 若上游已用别的方式实现了某个本地功能(导致本地补丁不再需要),在 `status.md`
把对应条目标为"已放弃(上游已实现)",并考虑回退本地补丁。
## 规则 4:其他
- `status.md` 属于仅本地文件,不向上游提 PR(除非团队明确决定);
`AGENTS.md` 中仅本节("本地 fork 维护规则")是本地内容,向上游提 PR 时应剔除。
+4
View File
@@ -176,6 +176,10 @@ export default defineConfig({
{ text: "Authentication model", link: "/architecture/auth-model" }, { text: "Authentication model", link: "/architecture/auth-model" },
{ text: "Magic-link auth", link: "/architecture/magic-link-auth" }, { text: "Magic-link auth", link: "/architecture/magic-link-auth" },
{ text: "Background jobs", link: "/architecture/jobs" }, { text: "Background jobs", link: "/architecture/jobs" },
{
text: "Message bus & notifications",
link: "/architecture/message-bus-and-notifications",
},
{ text: "UI diagnostics", link: "/architecture/ui-diagnostics" }, { text: "UI diagnostics", link: "/architecture/ui-diagnostics" },
], ],
}, },
+25
View File
@@ -16,8 +16,33 @@ The two layers are orthogonal — the moka caches shave query round-trips regard
| Thumbnail cache | configurable | 1 000 | Generated WebP/AVIF thumbnails | | Thumbnail cache | configurable | 1 000 | Generated WebP/AVIF thumbnails |
| Image transcode | configurable | 500 | On-the-fly image transcoding results | | Image transcode | configurable | 500 | On-the-fly image transcoding results |
| Blob hash | 30 s TTI | 5 000 | BLAKE3 hashes for dedup lookups | | Blob hash | 30 s TTI | 5 000 | BLAKE3 hashes for dedup lookups |
| Attached blob | 60 s TTL | 50 000 | `file_attached_blobs` row lookups on the thumbnail hot path (ETag + tier-2b, also the Nextcloud preview endpoint) |
| Audio metadata | — | 2 000 | ID3 tags and duration | | Audio metadata | — | 2 000 | ID3 tags and duration |
### The attached-blob cache
Every thumbnail request pays a `storage.file_attached_blobs` point query
before it can even answer "304 Not Modified" — the ETag names the attached
blob's hash. A photos grid revalidating 60 thumbnails per visit means
60+ point queries per browse. The cache sits in `DedupService` in front of
that lookup (`find_attached_blob`), keyed by the row's `(file_id, kind,
variant)` primary key, and caches **both directions**: `Some(row)` and
`None` (most files have no attached preview, so the negative side is where
most of the win is).
Two rules keep it honest:
- **DB faults are never cached.** The uncached lookup surfaces errors as
`Err`; only a genuine `Ok(None)` fills a negative entry. A transient
outage must not freeze "no attached blob" into place for a full TTL —
a read failure is never proof that data is absent.
- **TTL is the bound, not the invalidation strategy.** Writes invalidate
eagerly — `store_attached_blob` / `store_attached_blob_if_absent` on
success, deletions via `ThumbnailRefreshHook::on_file_deleted` (which
all three production delete paths fire). The 60 s TTL only bounds what
the process cannot see: bare SQL, the `copy_file_satellites` race
window, a hypothetical second instance.
### How it works ### How it works
1. **Read path:** check cache → if hit, return immediately (sub-ms); if miss, query PostgreSQL, populate cache, return 1. **Read path:** check cache → if hit, return immediately (sub-ms); if miss, query PostgreSQL, populate cache, return
@@ -248,6 +248,19 @@ on `DELETE`. The trigger fires on DELETE only; replacing a preview
updates `blob_hash` in place and the Rust path handles that reference updates `blob_hash` in place and the Rust path handles that reference
swap. swap.
**Reads are cached; the cache never outlives the truth by design.**
`DedupService::find_attached_blob` — the lookup the thumbnail ETag path
pays on *every* request, 304 or not — reads through an in-process moka
cache keyed by the row's PK, positive and negative entries alike. Two
properties make that safe rather than merely fast: a DB fault is
surfaced as an error and never fills a negative entry (a failed lookup
is not a missing row), and every write path that can change an answer
invalidates first — the two `store_attached_blob*` variants on success,
deletes via `ThumbnailRefreshHook::on_file_deleted` after the CASCADE
committed. The 60 s TTL exists for the residual cases the process
cannot observe (bare SQL, `copy_file_satellites` racing a concurrent
new file), not as the primary coherence mechanism.
**Writing a derived row requires its source to exist.** **Writing a derived row requires its source to exist.**
`store_derived_blob` guards the insert with an `EXISTS` on `store_derived_blob` guards the insert with an `EXISTS` on
`chunk_manifests`/`blobs`. Without it, a row written just after its `chunk_manifests`/`blobs`. Without it, a row written just after its
+1
View File
@@ -77,3 +77,4 @@ src/
- [Backend Storage →](/architecture/backend-storage) - [Backend Storage →](/architecture/backend-storage)
- [Derived and Attached Blobs →](/architecture/derived-and-attached-blobs) — thumbnails, transcodes and uploaded previews: why content-keyed and file-keyed artifacts need separate tables - [Derived and Attached Blobs →](/architecture/derived-and-attached-blobs) — thumbnails, transcodes and uploaded previews: why content-keyed and file-keyed artifacts need separate tables
- [Background Jobs →](/architecture/jobs) - [Background Jobs →](/architecture/jobs)
- [Message Bus & Notifications →](/architecture/message-bus-and-notifications) — real-time WebSocket bus (topics, AuthZ scopes, tab-visibility grace-close), persistent notifications (bell), AsyncAPI vs OpenAPI schema ownership
@@ -0,0 +1,645 @@
# Message Bus & Persistent Notifications
OxiCloud has two coupled subsystems that together power its real-time
UX — a **live message bus** over WebSocket for "something just
happened, refresh your view", and a **persistent notifications
table** for "you need to know about this even if you weren't
online." This document explains how both work, how they authenticate
and authorize subscribers, and why the frontend deliberately drops
the WebSocket while a tab is hidden.
Design docs the shipped code implements: [`docs/plan/message-bus.md`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/message-bus.md)
+ [`docs/plan/templated-messages.md`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/templated-messages.md).
---
## The two channels
| | **Message bus (WebSocket)** | **Persistent notifications (REST + DB)** |
|---|---|---|
| Purpose | "Something changed, refresh your view" | "You need to know about this — later is fine" |
| Transport | JSON-RPC 2.0 over `/api/rt/ws` | `GET/POST/DELETE /api/notifications/*` + `notif.notifications` table |
| Delivery | Best-effort, in-memory, no replay | Durable, per-user rows, survive reboot / offline |
| Payload | Thin "poke" facts (id + verb) | Full per-kind DTO with all render data |
| Loss on disconnect | Yes (events during outage window are dropped) | No (rows are the source of truth) |
| Schema owner | AsyncAPI (`resources/gen/asyncapi.json`) | OpenAPI (`resources/gen/openapi.json`) |
The two work together: an ingester that wants to notify a user
writes **both** — the DB row (for durability + the bell's history)
AND publishes a bus event on `user:{u}:notifications` (so online
sessions refetch instantly instead of waiting for the next
mount). The wire event on that topic is a **pure poke** — empty
`data: {}`. The row's real content only ever crosses the REST wire.
---
## Schema ownership — AsyncAPI vs OpenAPI
The bus and the REST endpoints have separate wire specs. The rule
the codebase adopts to keep them from drifting:
> **AsyncAPI defines the envelope + transport for clients.
> OpenAPI defines the payload.**
Concretely:
| Type | Home | How it stays in sync with Rust |
|---|---|---|
| **Bus events** (`MessageBusEvent`, subscribe / unsubscribe frames, revoked notifications, envelope shape) | **AsyncAPI** — `resources/gen/asyncapi.json` | Hand-written in `src/bin/generate-asyncapi.rs` via `json!` macros; kept in lockstep with `MessageBusEvent`'s serde shape. Small drift risk — Rust is truth. |
| **REST DTOs** (response bodies, request bodies, per-kind notification payloads) | **OpenAPI** — `resources/gen/openapi.json` | `#[derive(utoipa::ToSchema)]` on the Rust struct. Utoipa walks `#[utoipa::path(...)]` handlers + registered schemas. **No drift possible** — projection is derived from Rust. |
| **Types on both wires** (rare; none today) | Would live as one Rust struct with both derives, or wait for single-source codegen | — |
### Why this split, not one unified spec
The instinct is to put the notification payload schema in AsyncAPI
alongside the bus event that triggers a refetch. It looks cleaner
until you realize the payload **never travels on the bus wire** —
the bus event is `NotificationReceived` with empty `data: {}`, a
pure cache-invalidation poke. The FE fetches the payload from
`GET /api/notifications`, which is REST → OpenAPI's territory.
Putting the payload schema in AsyncAPI would mean "documenting
this shape on a transport it doesn't travel on" — a conceptual
stretch that adds a drift risk for zero gain.
### Rejected alternatives
- **Dual-spec (same type declared in both AsyncAPI + OpenAPI).**
Guaranteed drift unless both come from a single codegen. Nothing
in the tooling today produces both, so we'd hand-maintain two
copies of every shared type. Bug factory.
- **Cross-spec `$ref`** — AsyncAPI 3.0 allows `"$ref":
"openapi.json#/components/schemas/Foo"`, and utoipa's
`components(schemas(...))` can publish orphan types (no
`#[utoipa::path]` reference) so OpenAPI advertises "internal"
schemas. Technically workable but: Modelina + Swagger UI + Redoc
handle external refs inconsistently, OpenAPI stops being "the
REST contract" and becomes "a general schema registry",
reviewers get confused. Legal, fragile, avoided.
- **Bus event carries the full payload** (revert the pure-poke
design). Would put per-kind payload schemas in AsyncAPI as
`MessageBusEvent::NotificationReceived { granter_id, resource_id,
… }`. Rejected because the FE has to REST-fetch anyway (bell
reads from DB for history + persistence), so the fields on the
wire are dead weight — same-content overlap between the two
specs, no consumer benefit.
- **Session-resume tokens** (`rt.subscribe { since: N }` +
server-side ring buffer). Would let the bus deliver missed rows
directly on reconnect, saving one REST round-trip. Rejected for
**backward compatibility with `OXICLOUD_MESSAGEBUS_ENABLE=false`**:
ops who disable the WS rely on the bell falling back to REST;
bus-only replay would leave those deployments with no catch-up
path. The REST `?after=` cursor works in every mode (bus on, bus
off, network gap); the bus stays purely "instant-poke".
### What this looks like in the tree
- `src/application/ports/message_bus_ports.rs` — `MessageBusEvent`
enum (Rust source of truth for bus wire shapes).
- `src/bin/generate-asyncapi.rs` — projects those Rust variants
into `resources/gen/asyncapi.json`.
- `src/domain/entities/notification.rs` — `SharegrantedPayload`
and its siblings, `#[derive(ToSchema)]`, source of truth for
REST payload shapes.
- `src/interfaces/api/mod.rs` — utoipa `#[openapi(components(schemas(SharegrantedPayload, ...)))]`
registers the payload in OpenAPI even though `NotificationDto.payload`
stays `serde_json::Value` on the response type. (FE type-narrows
on `row.kind` and casts to the right shape.)
- Nothing lives in both specs today.
### Adding a new bus event
1. Add a variant to `MessageBusEvent`.
2. Add the variant to `generate-asyncapi.rs`'s `event_kind` enum
and (if the variant has payload fields) a schema function.
3. Regenerate AsyncAPI + FE DTOs via `just asyncapi` +
`npm run gen:message-bus`.
4. **Do not** add the variant to OpenAPI. Bus events don't
travel on REST.
### Adding a new notification kind's payload
1. Add a Rust struct in `domain/entities/notification.rs` with
`#[derive(Serialize, Deserialize, ToSchema)]`.
2. Register it in `src/interfaces/api/mod.rs`'s
`components(schemas(...))` list.
3. Regenerate OpenAPI via `just openapi`.
4. **Do not** add the struct to AsyncAPI. Notification payloads
only cross the REST wire.
---
## Message bus
### Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ SERVICE LAYER │
│ │
│ ShareService.grant() ── after commit ──▶ bus.publish(…) │
│ FileMgmtService.…() ── after commit ──▶ bus.publish(…) │
│ NotificationService ── after commit ──▶ bus.publish(…) │
│ Scheduler engine ── on run start/end ──▶ bus.publish │
└──────────────────────────┬──────────────────────────────────┘
▼
┌─────────────────────────────────────────────────────────────┐
│ MessageBus port (application/ports/message_bus_ports.rs) │
│ │
│ InProcessMessageBus │
│ DashMap<Topic, tokio::broadcast::Sender<Event>> │
└──────────────────────────┬──────────────────────────────────┘
│
│ (optional replicator seam)
▼
NoopReplicator (v1)
PgListenReplicator (deferred)
BrokerReplicator (RabbitMQ / NATS, deferred)
│
▼
┌─────────────────────────────────────────────────────────────┐
│ WS handler (interfaces/api/handlers/rt_ws.rs) │
│ │
│ One session per socket: │
│ HashMap<wire_key, Sub> + outbound mpsc │
│ │
│ rt.subscribe / rt.unsubscribe frames │
│ rt.event / rt.revoked / rt.pong notifications │
└─────────────────────────────────────────────────────────────┘
```
### Topics — a typed enum, not a string
```rust
pub enum Topic {
Folder(Uuid), // "folder:{uuid}"
UserAuthz(Uuid), // "user:{uuid}:authz"
UserNotifications(Uuid), // "user:{uuid}:notifications"
Job(String), // "job:{name}"
}
```
Defined in `application/ports/message_bus_ports.rs`. Encoded to a
stable dotted wire form; parsed back with strict validation. The
wire form doubles as a routing key for future broker replicators
(RabbitMQ topic exchanges, NATS subjects).
### Events
`MessageBusEvent` (same module) is the discriminated union of every
payload a publisher can produce — `FileCreated`, `FolderMoved`,
`AuthzChanged`, `JobRunStarted / Progress / Ended`,
`NotificationReceived`, etc. Serde tags with `#[serde(tag = "event",
rename_all = "snake_case")]`, so the wire is
`{"event": "file_created", "file_id": "...", "actor": "..."}`.
Payloads are **thin facts**: the ID of the changed resource + the
actor + the verb. Clients refetch details via REST if they need
them. Keeps the AuthZ surface small (thin payloads can't leak
fields the caller couldn't already read via REST) and keeps events
well under any future broker's message-size cap.
### Wire protocol
JSON-RPC 2.0 over text frames. Full protocol in [`docs/plan/message-bus.md § Wire protocol`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/message-bus.md).
```jsonc
// Client → server
{"jsonrpc":"2.0","id":1,"method":"rt.subscribe","params":{"topic":"folder:abc-…"}}
// Server → client (ack)
{"jsonrpc":"2.0","id":1,"result":{"subscribed":"folder:abc-…"}}
// Server → client (push, id-less notification)
{"jsonrpc":"2.0","method":"rt.event","params":{
"topic":"folder:abc-…",
"event":"file_created",
"data":{"file_id":"…","name":"…","parent_id":"…","actor":"…"}
}}
```
### Authentication for the WebSocket upgrade
Two paths, both accepted by the same handler:
| Client kind | Path | Why |
|---|---|---|
| **Programmatic** (CLI, test helper) | `Authorization: Bearer <jwt>` on the upgrade | The `new WebSocket()` API in browsers can attach `Sec-WebSocket-Protocol` but NOT arbitrary headers, so browsers can't do this. |
| **Browser** | `POST /api/rt/ticket` (with the full DPoP + CSRF middleware chain) mints a one-shot 30-second opaque UUID; the browser opens the WS with `Sec-WebSocket-Protocol: oxi.ticket.<uuid>` | DPoP-bound sessions cannot attach a `DPoP:` header to `new WebSocket()`. The ticket flow moves the DPoP check to a normal POST that DOES support headers, and the WS upgrade just redeems the opaque token. |
Tickets are single-use, TTL 30 s, stored in a `RtTicketStore`
(in-memory). Redemption removes the entry — replay is impossible.
The WS route is deliberately mounted **outside** the
`protected_api` middleware stack — otherwise the DPoP-required
layer would 401 every browser on upgrade before the ticket flow
could kick in.
---
## The three AuthZ scopes
`Topic::required_perm(&self) -> AuthzCheck` dispatches every
subscribe attempt into exactly one of three classes. This is the
authoritative diagram of what the WS handler enforces:
```
┌──────────────────────────────────────────┐
│ Topic::required_perm() │
└─────┬─────────────┬─────────────┬────────┘
│ │ │
ResourceRead │ IdentityMatch │ RoleAdmin
▼ ▼ ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Class 1 │ │ Class 2 │ │ Class 3 │
│ per-resource│ │ per-user │ │ per-session │
│ │ │ │ │ (role) │
└─────────────┘ └─────────────┘ └─────────────┘
```
### Class 1 — Per-resource (`AuthzCheck::ResourceRead`)
**Topics:** `folder:{id}`, and (future) `file:{id}`,
`drive:{id}`, `calendar:{id}`, `addressbook:{id}`.
**Rule:** the caller must hold **`Read`** on the resource via the
same `AuthorizationEngine` that guards every REST endpoint. Owner
short-circuits pass; direct grants pass; group-mediated grants
pass; drive-membership cascades pass. Everything else is denied.
**Wire response on denial:** JSON-RPC error object with
`code = -32001`, `message = "no_read"`. Same shape whether the
resource doesn't exist OR the caller lacks the grant — **anti-
enumeration invariant**. Audit reason (`no_read` /
`no_such_resource`) distinguishes internally.
**On grant revocation:** the WS handler auto-subscribes each
session to `user:{caller}:authz` (Class 2 below). When a
`MessageBusEvent::AuthzChanged { affected_folders }` fires, the
session's reader translates it to an internal `EvictFolders`
signal → the main loop walks the sub set and drops any Class-1
subscription whose resource was affected, emitting a client-visible
`rt.revoked` notification per evicted topic. Same pattern applies
to any Class-1 topic when the AuthZ model widens beyond folders.
### Class 2 — Per-user, strict privacy (`AuthzCheck::IdentityMatch`)
**Topics:** `user:{u}:authz`, `user:{u}:notifications`.
**Rule:** direct UUID equality — `caller_id == u`. **No admin
bypass, no group indirection, no owner short-circuit.** Admins
cannot subscribe to other users' `:authz` or `:notifications`
streams; that's a privacy invariant, not a mere policy choice.
**Wire response on mismatch:** `topic_forbidden` — the **same
wire shape as an unknown topic**. An attacker probing
`user:{someone_else_uuid}:authz` cannot distinguish "user exists
but not me" from "no such user."
**Auto-subscription:** the WS handler auto-subscribes every session
to its own `user:{caller}:authz` AND `user:{caller}:notifications`
at session open. No `rt.subscribe` frame is needed from the client
for these — they're always active for the caller's own UUID.
### Class 3 — Per-session role (`AuthzCheck::RoleAdmin`)
**Topics:** `job:{name}` today. Future `admin:*` topics land here.
**Rule:** the session's snapshotted role at open time must be
`admin`. The handler resolves `caller_role` once during session
setup via `resolve_live_role` and stores it on the session state —
no per-subscribe DB round-trip.
**Wire response on non-admin:** `topic_forbidden` — same anti-enum
shape as Class 2. A non-admin probing job topics cannot enumerate
which jobs are registered.
**Why snapshot at session open, not per subscribe:** admin role
loss is rare + trivially recoverable (the user closes the tab and
reopens, hitting the fresh role check). Per-subscribe checks would
be an extra DB round-trip on every frame with no meaningful
security gain — the WS session itself was authenticated at upgrade
time under the current role.
### Adding a new topic
Every new topic variant must decide which class it belongs to at
`Topic::required_perm`. The compiler enforces exhaustiveness — a
new variant with no branch fails to build, which is deliberate. New
topics get audited before shipping precisely because the
`required_perm` match forces the author to state the class
explicitly.
---
## Tab-visibility grace-close — reducing idle connections
Every open browser tab holds one WebSocket to the server. A user
with five tabs open holds five sockets. A user who leaves a tab
open all day but only uses one holds five sockets, four of them
serving nothing.
The frontend closes the WebSocket **after 60 seconds of tab
hidden** and reopens it when the tab becomes visible again. The
subscription state is preserved locally through the outage — every
subscriber's release handle stays live, the reactive store still
holds the last-known list — but the wire is silent while the tab
is hidden.
### Implementation
Frontend `MessageBusClient` (`frontend/src/lib/message-bus/client.svelte.ts`)
attaches a `visibilitychange` listener on construction:
- **Tab hidden** → starts a 60-second timer.
- **Timer fires while still hidden** → close the WS via a
`#closeForHidden` path that sets state to `disconnected` but
preserves `#subs` for later replay. The `#onClose` handler is
guarded by `#tabIsHidden()` — an auto-reconnect won't fire while
the tab remains hidden.
- **Tab visible again** → cancels the timer if it hadn't fired
yet; if the WS was closed, kicks off a normal reconnect.
- **On reopen**, the WS handler auto-subscribes to `:authz` and
`:notifications` again, and the client replays every entry in
`#subs` as `rt.subscribe` frames. From the user's POV, the state
is identical to what they left behind.
### What this trades
- **Saved**: N-1 idle sockets per user with N tabs open, over the
hidden-tab window. Meaningful at scale (100 users × 5 tabs × 8
idle hours = 4000 idle-tab-hours of connection state to keep
alive per day).
- **Lost**: bus events published during the 60-second delay
(transient) + the whole grace-close window (indefinite while
hidden) are dropped for that session. **Recovery**: on reopen,
every consumer that cares refetches. See "Reconnect catch-up"
below.
### Why 60 seconds
Short enough that leaving a tab for a coffee break doesn't burn
the connection. Long enough that the momentary focus-shifts
users do all day (Cmd-Tab to another app, back within seconds)
don't churn the socket. Not tunable per-user — the value is
hard-coded in `HIDDEN_GRACE_MS`.
### Reconnect catch-up — the `?after=` cursor
For consumers whose state can't be reconstructed by refetching a
current listing (specifically: **notifications**, whose bell must
show rows that landed during the outage), the FE issues a delta
fetch:
- Store tracks `#lastReceivedAt` — the newest `created_at` seen
before disconnect.
- On `messageBus.onReconnect(...)` fire, calls
`GET /api/notifications?after=<lastReceivedAt>&limit=100`.
- Merges the returned rows into local state via `mergeById` —
duplicates are resolved with **incoming wins** (server value
overrides local, so a `read_at` flip on another device shows
up correctly).
The server-side `?after=` predicate is **strict `>`** — a row at
exactly `lastReceivedAt` is excluded. This makes the WS push (which
delivers a row at time T) and the delta fetch (which asks for
"anything after T") non-overlapping by construction. `mergeById`
handles the case where the two paths race and both deliver the
same row.
For consumers whose state IS a current listing (folder view: the
files/subfolders in a folder), reconnect just refetches the listing
via the normal REST endpoint. `useReconnect` composable exposes
`onReconnect(cb)` as a one-liner for that pattern.
---
## Persistent notifications (the bell)
### Data model
```sql
notif.notifications (
id UUID PRIMARY KEY,
user_id UUID NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
kind TEXT NOT NULL, -- 'share_granted' | 'new_login_from_new_device' | …
payload JSONB NOT NULL, -- per-kind shape (see below)
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
read_at TIMESTAMPTZ -- NULL = unread
)
```
Two indexes:
- `(user_id, created_at DESC) INCLUDE (read_at, kind)` covers the
bell's list query + the mark-all-read filter.
- `(read_at) WHERE read_at IS NOT NULL` — partial, tiny on healthy
DBs; feeds the retention job's DELETE.
### Ingesters
An ingester is a code path that calls
`NotificationApplicationService::create(NewNotification)`. The
service atomically:
1. `INSERT INTO notif.notifications RETURNING …` — durable row.
2. `bus.publish(Topic::UserNotifications(user_id), NotificationReceived)` — the fast-path poke.
Publish happens **after** the DB write succeeds, never inside a
transaction — a rolled-back INSERT would otherwise fan out a lie.
**Currently shipped ingester:** `share_granted` in
`interfaces/api/handlers/grant_handler.rs::create_grant`. Fires
after `authz.set_role(...)` succeeds. Fans out to every resolved
recipient user:
- `Subject::User(id)` → one row for that user.
- `Subject::Group(id)` → one row per transitive member (via
`SubjectGroupService::list_transitive_users`).
- `Subject::Token(_)` → no row (anonymous share links have no
target user).
Self-shares (owner grants themselves via a group they belong to)
skip. Failure to write is best-effort — a warn log; the grant row
stays durable in `role_grants`, the recipient can still discover
the share via `/api/grants/shared-with-me`.
**Planned but not wired** (each needs a prerequisite subsystem
listed in [`docs/plan/message-bus.md § Deferred`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/message-bus.md)):
| Kind | Prerequisite |
|---|---|
| `new_login_from_new_device` | Device-fingerprint tracking table |
| `job_completed_for_you` | Scheduler engine threading the trigger caller's `caller_id` through `dispatch()` |
| `storage_quota_threshold` | Per-user usage/quota comparator with threshold-crossing detection |
### Payload shape — typed per kind
The `payload` column is JSONB (schema-free at the DB layer). Each
kind's Rust shape lives in `domain/entities/notification.rs` with
`#[derive(Serialize, Deserialize, ToSchema)]`. OpenAPI picks up the
struct automatically. Adding a new field is additive on JSONB — no
migration.
Example — `share_granted`:
```rust
pub struct SharegrantedPayload {
pub granter_id: Uuid,
pub resource_type: String, // 'folder' | 'file' | 'drive' | …
pub resource_id: Uuid,
pub resource_name: Option<String>, // snapshot at grant time
pub resource_path: Option<String>, // storage path snapshot
pub navigate_folder_id: Option<Uuid>, // FE routing target for drives
pub role: String,
pub expires_at: Option<DateTime<Utc>>,
}
```
The name/path fields are **snapshotted at grant time**. If the
folder is later renamed or moved, the notification still reflects
what it was called when the share happened. Same principle as
email invitations or activity feeds: the record is a fact about
what was true at the moment, not a live pointer.
### Schema-ownership rule — AsyncAPI vs OpenAPI
> **AsyncAPI defines the envelope + transport for clients.
> OpenAPI defines the payload.**
The bus event `MessageBusEvent::NotificationReceived` is a **unit
variant** — it serializes to `{"event":"notification_received",
"data":{}}` with no fields on the wire. The topic identifies the
semantic; the FE responds by refetching from REST.
The payload's shape lives in OpenAPI via `ToSchema` on
`SharegrantedPayload` (and its future siblings), auto-derived from
Rust. AsyncAPI never sees these types — payloads don't travel on
the bus wire.
**Why this split** — it eliminates schema drift between the two
specs. A payload edit changes Rust → OpenAPI updates on regen
(mechanical). AsyncAPI stays stable (hand-written, but it never
touches payloads). Same rule applies to any future bus consumer
that also has a REST DTO — Rust is the source of truth; each spec
projects the parts of Rust that travel on its transport.
Design rationale + rejected alternatives (dual-spec, cross-`$ref`,
per-kind Svelte components) in [`docs/plan/templated-messages.md § Schema ownership`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/templated-messages.md).
### REST surface — `/api/notifications`
| Method | Path | Purpose |
|---|---|---|
| `GET` | `/api/notifications` | List newest-first. Query params: `unread` (bool), `before` / `after` (cursors), `limit` |
| `GET` | `/api/notifications/unread` | Badge-only fast path (returns just `unread_count`) |
| `POST` | `/api/notifications/{id}/read` | Mark one row read (idempotent, always 204) |
| `POST` | `/api/notifications/read-all` | Bulk mark-all-read (returns rows updated) |
| `DELETE` | `/api/notifications/{id}` | Hard-delete one row (idempotent, always 204) |
**Anti-enumeration:** mark-read and delete always respond 204 —
whether the row existed and belonged to the caller, or didn't
exist at all, or belonged to someone else. Every mutating endpoint
scopes on `caller_id` at the SQL layer; the response shape is
identical across the three outcomes.
### Retention
The `notifications_cleanup` scheduled job (daily, same tier as
`trash_cleanup`) DELETEs read rows older than
`OXICLOUD_NOTIFICATIONS_RETENTION_DAYS` (default 30). **Unread
rows are preserved unconditionally** — a user offline for a month
still sees the share-granted notice when they log back in.
Runtime override via the job's `retention_days` parameter on the
admin panel's trigger — the env default seeds it, the panel
overrides at trigger time.
### Frontend rendering
`frontend/src/lib/composables/useNotifications.svelte.ts` owns the
module-scoped store — one instance per SPA session. Exposes:
- `notifications.items` — reactive list (newest first)
- `notifications.unread` — reactive badge count
- `notifications.refresh()` / `refreshDelta()` / `markRead(id)` /
`markAllRead()` / `delete(id)`
`NotificationRow.svelte` handles the actual rendering — one file,
one `switch` on `row.kind`, one rich template per shipped kind
(`share_granted` today; the others fall back to a generic string).
Extraction into per-kind components is deferred until a single
kind's block exceeds ~30 lines or two kinds start needing the same
sub-component (see [`docs/plan/templated-messages.md § Rendering`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/templated-messages.md)).
### Notification click routing
| `resource_type` | Route | Data used |
|---|---|---|
| `folder` | `/files/{resource_id}` | `resource_id` |
| `file` | `/shared-with-me?file={resource_id}` | `resource_id` — the `/files/{uuid}` route requires a **folder** id, and a file-scoped grant may not include parent-folder access. `/shared-with-me` is the guaranteed-accessible home for every recipient of a `share_granted`, and its `?file=` deep link opens the inline `FileViewer`. |
| `drive` | `/files/{navigate_folder_id}` | Drives have no browsable URL of their own; `navigate_folder_id` is the drive's `root_folder_id`, enriched at ingest via `DriveRepository::get_by_id`. |
| `calendar` / `address_book` / `playlist` | no link (bold text) | Not addressable via `/files/*`. |
The bell also fires a **transient toast** (via the existing
`ui.notify(...)` mechanism) on every fresh row that arrives via
delta — the toast fades out after ~4 s while the persistent row
stays in the bell's history section. Same bell icon, same badge
count, no duplicate UX.
---
## Feature flags & config
| Env var | Default | Effect |
|---|---|---|
| `OXICLOUD_MESSAGEBUS_ENABLE` | `true` | Master switch. When `false`, the `/api/rt/ws` and `/api/rt/ticket` routes are **not registered** at boot (Axum returns 404), and the FE `useNotifications` composable skips the WS setup entirely. The bell falls back to REST-only mode — polling on mount, delta on manual refresh. Zero client-side error spam. |
| `OXICLOUD_MESSAGEBUS_KEEPALIVE_SECONDS` | `30` | Server-initiated RFC 6455 Ping interval on each WS connection. Prevents intermediate proxies (nginx, Traefik, Cloudflare) from reaping the TCP session as idle. |
| `OXICLOUD_NOTIFICATIONS_RETENTION_DAYS` | `30` | Retention window for read notifications. Unread rows are always preserved. The `notifications_cleanup` job clamps to a minimum of 1 day. |
Client discovers all of these via `GET /api/config` — no in-band
"does the server support the bus?" probe needed. The FE reads
`serverConfig.features.message_bus` at boot and skips WS setup
entirely when it's false.
---
## Failure modes
| Scenario | Behavior |
|---|---|
| Bus is disabled server-side (`OXICLOUD_MESSAGEBUS_ENABLE=false`) | `/api/rt/ws` returns 404. `useTopic` in the FE returns early. Bell works via REST only. |
| Network drops mid-session | Client-side jittered exponential backoff (250 ms → 30 s cap, 20-failure circuit breaker). On reconnect, WS handler re-auto-subscribes to `:authz` + `:notifications`; `useReconnect` composable fires `onReconnect` callbacks so views refetch. |
| Server restart | Same as network drop — the WS breaks, client backs off, reconnects when server is back. Events published during the outage are lost (no persistent event log by design); consumers refetch. |
| Tab hidden > 60 s | WS closed via `visibilitychange` grace-close. State preserved locally. On visibility return, reconnect + replay subscriptions. |
| Publish before commit | Not allowed. Every publish site is documented as "after commit". A publish inside a transaction that rolls back would fan out a lie. |
| Broker replicator failure (future) | The local `InProcessMessageBus` publishes still succeed — the replicator is beside the bus, not in front. Broker-hop failures affect multi-instance fanout but never local delivery. |
---
## Testing
The api-test suite exercises the full stack end-to-end via `rt-hurl-helper` (a small Rust binary gated on `test_utils`) — Hurl alone can't drive a WebSocket. Sixteen scenarios in `tests/api/rt_bus_check.sh`:
- Positive delivery, topic isolation
- AuthZ denial (Class 1 folder), unknown-topic anti-enum
- Server keepalive, delete emits, move fan-out
- Grant-revoke eviction (`AuthzChanged` → `rt.revoked`)
- Cross-user identity gate (Class 2)
- Ticket happy path + single-use replay refused
- Admin-only job topic (Class 3)
- `notification_received` wire push, DB row via `GET /api/notifications`
- Cross-user notifications identity gate (Class 2, notifications topic)
- `?after=` cursor with strict-`>` boundary invariant
`mergeById` — the FE's WS/reconnect race dedup — has its own
Vitest with 5 covered cases (empty, non-overlap, exact-dup,
stale-read-at overwrite, mixed overlap).
---
## Further reading
- Plan doc: [`docs/plan/message-bus.md`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/message-bus.md) — full design rationale, roadmap, and deferred slices (Yjs collab, broker replicator, SharedWorker, Web Push).
- Plan doc: [`docs/plan/templated-messages.md`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/templated-messages.md) — schema-ownership rule, rendering shape, notification routing decision.
- [ReBAC Authorization](/architecture/rebac-authorization) — the engine every Class-1 topic gate calls.
- [Background jobs](/architecture/jobs) — `notifications_cleanup` is one of them; `Topic::Job` publishes on job lifecycle.
+37 -2
View File
@@ -36,8 +36,43 @@ What used to live on the share row but is now resolved through ReBAC:
| Method | Path | Description | | Method | Path | Description |
| --- | --- | --- | | --- | --- | --- |
| `GET` | `/api/s/{token}` | Access a shared item | | `GET` | `/api/s/{token}` | Share landing metadata (see [landing enrichment](#share-landing-metadata-enrichment)) |
| `POST` | `/api/s/{token}/verify` | Verify a password-protected share | | `POST` | `/api/s/{token}/verify` | Verify a password-protected share (sets the unlock-JWT cookie) |
| `GET` | `/api/s/{token}/download` | Download a **file share** (Range / 206 / 304 / 416 aware) |
| `GET` | `/api/s/{token}/contents` | List a **folder share's** root (folders + files) |
| `GET` | `/api/s/{token}/contents/{folder_id}` | List a subfolder inside the shared subtree |
| `GET` | `/api/s/{token}/file/{file_id}` | Stream one file — the landing page's inline preview and per-file download path (Range aware) |
| `GET` | `/api/s/{token}/zip` | ZIP archive of a **folder share's** root |
| `GET` | `/api/s/{token}/zip/{folder_id}` | ZIP archive of a subfolder inside the shared subtree |
#### File scoping on `/file/{file_id}`
The AuthZ gate (`ShareBrowseService::assert_file_in_share`) branches on the
share's `item_type` — a single-file share and a folder share scope the
endpoint differently:
- **File share** — only the shared item itself may be streamed
(`file_id == share.item_id`). This is what renders the public landing
page's inline media preview (video player / image) and it is also the
NextCloud-desktop-style per-file fetch path.
- **Folder share** — the file must live inside the shared subtree
(ltree `is_file_in_subtree` against the share's root folder).
- Anything else → **404**, the same shape as "file doesn't exist", so the
endpoint cannot be used to enumerate ids.
Password and expiry checks happen inside `get_shared_link_with_unlock`
before the scope decision; a password-protected share answers 401 with
`requiresPassword: true` until the unlock cookie is presented.
#### Share landing metadata enrichment
`GET /api/s/{token}` resolves the shared **file's** `mime_type` + `size`
at read time so anonymous viewers get an inline media preview (video
player / image) instead of a bare download button. The enrichment is
display-only and never fails the response: a failed file lookup (transient
DB error, race with a delete) leaves the fields absent and the download
endpoints surface the real error — a read failure is never proof that the
data is absent. Folder shares pass through unenriched.
## Service Responsibilities ## Service Responsibilities
+64
View File
@@ -123,6 +123,70 @@ rather than as a visible error.
| `OXICLOUD_NOTIFICATIONS_RETENTION_DAYS` | `30` | Retention window for **read** notification rows (`notif.notifications`). The `notifications_cleanup` scheduled job runs daily and deletes rows where `read_at IS NOT NULL` and `read_at < now() - retention_days`. Unread rows are preserved unconditionally — the whole point of the durable table is that a user offline for a month still sees the share-granted notice on next login. Clamped to a minimum of 1 (0 would purge every read row on every tick). Adjust down for compliance-sensitive deployments where "cleared once seen" matters; adjust up when operators expect users to reference old notifications for support. | | `OXICLOUD_NOTIFICATIONS_RETENTION_DAYS` | `30` | Retention window for **read** notification rows (`notif.notifications`). The `notifications_cleanup` scheduled job runs daily and deletes rows where `read_at IS NOT NULL` and `read_at < now() - retention_days`. Unread rows are preserved unconditionally — the whole point of the durable table is that a user offline for a month still sees the share-granted notice on next login. Clamped to a minimum of 1 (0 would purge every read row on every tick). Adjust down for compliance-sensitive deployments where "cleared once seen" matters; adjust up when operators expect users to reference old notifications for support. |
| `OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX` | `@drive` | Native WebDAV URL segment that renders the caller's drive list. Sanitized by trimming leading/trailing `/`. Three shapes: (1) default `@drive` — `/webdav/…` addresses the caller's default personal drive (back-compat), `/webdav/@drive/` returns the drive listing, `/webdav/@drive/<uuid\|name>/…` targets a specific drive. (2) empty string `""` — `/webdav/` IS the drive listing, `/webdav/<uuid\|name>/…` targets a specific drive, no default-drive shortcut. (3) any other string (e.g. `drives`) — same shape as `@drive` with that segment substituted. Only drives the caller has Read on via `role_grants` resolve. | | `OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX` | `@drive` | Native WebDAV URL segment that renders the caller's drive list. Sanitized by trimming leading/trailing `/`. Three shapes: (1) default `@drive` — `/webdav/…` addresses the caller's default personal drive (back-compat), `/webdav/@drive/` returns the drive listing, `/webdav/@drive/<uuid\|name>/…` targets a specific drive. (2) empty string `""` — `/webdav/` IS the drive listing, `/webdav/<uuid\|name>/…` targets a specific drive, no default-drive shortcut. (3) any other string (e.g. `drives`) — same shape as `@drive` with that segment substituted. Only drives the caller has Read on via `role_grants` resolve. |
## Boot-time refuse-to-boot checks
Some upgrades add invariants the running database must satisfy
BEFORE the new binary can serve traffic. These are enforced by
read-only checks that run after `sqlx::migrate!()` and before the
server binds a listen socket. If a check fails, the server exits
with a FATAL message spelling out the exact CLI command to run.
The server **never silently mutates data** at boot — every fix is
an explicit `oxicloud migrate <name>` invocation. Follows the
"discovery-only by default, mutation opt-in" rule that also
governs the consistency-check jobs.
### `lowercase-usernames`
Three outcomes at boot, only one of which stops the server:
- **All lowercase (or `NULL`)** — the check is a no-op, boot
proceeds unchanged.
- **Mixed-case rows exist, no `LOWER(username)` collision** —
boot **auto-lowercases** them in one atomic transaction, emits
a structured audit line per rename
(`user.username_lowercased_on_boot`, INFO) plus an INFO summary
(`user.usernames_lowercased_on_boot_summary` with `renamed=N`),
and continues. Silent action is confined to the case with
exactly one correct move: `Alice` (with no `alice` row) becomes
`alice`.
- **`LOWER(username)` collision** — two or more active rows share
the same lowercase form (e.g. `Alice` + `alice`). Boot
**refuses to start** with a FATAL message spelling out every
collision group and the exact CLI command to resolve it.
Tiebreak needs a human.
Soft-deleted / disabled accounts (`active = false`) and `NULL`
usernames (OPAQUE-migrated) are skipped in every case.
**Refusal message pattern:** `FATAL: cannot start — N colliding
username group(s) (M affected account(s) in total)`, followed by
each group's canonical form and its members with `id` +
`last_login`. Up to 10 groups shown; the `--dry-run` CLI reveals
the full list.
**Fix (only required when the server refused):**
```
oxicloud migrate lowercase-usernames --dry-run # preview the tiebreak
oxicloud migrate lowercase-usernames # apply
```
**What the migration does:** lowercases every mixed-case
username. On collision (`Alice` + `alice` both exist), the
tiebreak `(last_login_at DESC NULLS LAST, created_at ASC)` picks
a winner; losers get `alice-2`, `-3`, … as a suffix. Sessions
and grants survive the rename — both key on the user's UUID.
**Client compat:** NextCloud clients that cached URLs like
`/remote.php/dav/files/Alice/…` continue to work indefinitely —
the Basic Auth middleware and URL parser both lowercase on
decode. NC desktop clients will prompt a one-time re-sync on
first PROPFIND after upgrade; DAVX5 and NC mobile handle it
silently. See `docs/install/binary.md § Upgrading from a
case-sensitive-usernames release` for the full upgrade flow.
**Design:** `docs/plan/username-lowercase.md`.
## Storage Entries (multi-entry, recommended) ## Storage Entries (multi-entry, recommended)
Declare one or more **named** storage backends. The one the app runs on is picked from the DB (`admin_settings.storage.active_backend_name`); the admin panel's storage tab flips the pointer, and cross-backend migration is a recoverable job that copies blobs between two entries with a read-only safety window. See [Admin Settings — Storage & Migration](/config/admin-settings) for the operator flow and the [multi-entry design doc](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/storage-multi-entry.md) for the full model. Declare one or more **named** storage backends. The one the app runs on is picked from the DB (`admin_settings.storage.active_backend_name`); the admin panel's storage tab flips the pointer, and cross-backend migration is a recoverable job that copies blobs between two entries with a read-only safety window. See [Admin Settings — Storage & Migration](/config/admin-settings) for the operator flow and the [multi-entry design doc](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/storage-multi-entry.md) for the full model.
+13
View File
@@ -47,6 +47,19 @@ If you need to let someone make changes, share with their **email**
instead. They'll receive an invitation, and from then on every change instead. They'll receive an invitation, and from then on every change
they make is recorded under their name. they make is recorded under their name.
## What recipients see
Opening a **single-file** public link shows the file right on the
landing page — images display inline, and videos play in the browser
with a working seek bar (streamed, so no full download before
playback). A **Download** button always sits below the preview. For
files the browser can't display, recipients get the download button as
usual.
Opening a **folder** public link shows a browsable listing — folders
and files as cards, with a grid/list toggle and a **Download ZIP**
button. Images and videos open in a lightbox viewer.
## Expiration ## Expiration
When you share, you can set an **expiration date**. After that date, When you share, you can set an **expiration date**. After that date,
+69
View File
@@ -217,6 +217,75 @@ supported by sqlx's migration model; if you need to roll back, stop
the server, roll back your Postgres data directory to a snapshot, and the server, roll back your Postgres data directory to a snapshot, and
install the previous binary. install the previous binary.
### Upgrading from a case-sensitive-usernames release
Releases that predate the case-insensitive-usernames change stored
`Alice`, `alice`, and `ALICE` as three separate accounts. The
current release treats usernames as case-insensitive (canonical
lowercase in the database). What happens on the first boot after
upgrade depends on your data:
**No mixed-case usernames.** The check is a no-op; the server
starts normally. Nothing to do.
**Mixed-case usernames with no collision.** The server
**auto-lowercases** them in one atomic transaction at boot and
continues. Each rename is recorded in the audit log
(`user.username_lowercased_on_boot`) and a WARN summary line
names the total count. `Alice` (with no `alice` row alongside)
becomes `alice`; no ops action needed. This covers the vast
majority of single-admin self-hosted deployments.
**Mixed-case usernames WITH a collision** (`Alice` + `alice`
both exist as active accounts). The server **refuses to boot**
— tiebreak requires a human. Run the migration:
```
# Preview the tiebreak — no writes.
sudo -u oxicloud DATABASE_URL="postgres://..." \
/usr/local/bin/oxicloud migrate lowercase-usernames --dry-run
# Apply. Renames run in a single transaction; safe to re-run if aborted.
sudo -u oxicloud DATABASE_URL="postgres://..." \
/usr/local/bin/oxicloud migrate lowercase-usernames
```
On collision, the migration picks a winner by `(last_login_at
DESC NULLS LAST, created_at ASC)` — most recently active keeps
the canonical lowercase name; the loser gets `alice-2`, `-3`, …
as a suffix. **Sessions and grants survive the rename** — both
key on the user's UUID, not the username.
Skipped in every path above: soft-deleted / disabled accounts
and OPAQUE-migrated users whose `username` column is NULL.
Neither blocks boot.
After the migration completes, restart the service:
```
sudo systemctl start oxicloud
```
**Nextcloud desktop clients** will prompt a one-time re-sync on
their first PROPFIND after upgrade — the account URL case
changed. Data is safe (files re-verify via ETag, not re-uploaded).
DAVX5 (calendars, contacts) and NC mobile handle the URL case
change silently. **No client upgrade or reconfiguration is
required** — the server accepts uppercase URL segments (`Alice`
in `/remote.php/dav/files/Alice/...`) indefinitely.
The refusal message printed by the server on boot (collision
path only) includes the exact CLI command above, so you can't
miss it. Full plan and rationale in
`docs/plan/username-lowercase.md`.
**Explicit-preview path.** If you'd rather run the migration
before the binary swap — to review renames on your own schedule
or to gate a backup step — run `oxicloud migrate
lowercase-usernames --dry-run` against the OLD binary's DB
first, then apply. On next boot the new binary sees a
lowercase-clean DB and the auto-rename path is a no-op.
## Installing via `cargo binstall` ## Installing via `cargo binstall`
If you already have the Rust toolchain and just want the binary If you already have the Rust toolchain and just want the binary
+581
View File
@@ -0,0 +1,581 @@
# Plan — Case-insensitive usernames (lowercase-on-ingest)
## Context
Feature ask: [issue #691](https://github.com/AtalayaLabs/OxiCloud/issues/691).
Usernames are currently case-sensitive, so `Alice`, `alice`, and `ALICE`
refer to three different accounts. Users hit this as a login friction —
they type their name with different capitalization on different clients
and get "invalid credentials" instead of a successful login.
## Why this is simpler than it looks in this codebase specifically
- `validate_username` in `src/domain/entities/user.rs:884` already
restricts usernames to ASCII-only `[a-zA-Z0-9._-]{2,64}` with no `@`.
The Unicode case-folding minefield (Turkish dotted-I, German ß, Greek
final sigma, NFC vs NFD) does not apply — ASCII case-folding is
trivial (`to_ascii_lowercase`), deterministic, and locale-independent.
- OIDC identity binds via `(iss, sub)` in
`get_user_by_federation_subject` at
`src/infrastructure/repositories/pg/user_pg_repository.rs:1257-1303` —
case-sensitivity of the local username is orthogonal to OIDC identity
matching. No OIDC breakage risk.
- Password verification runs through Argon2's `verify_password`
(constant-time by construction). Not affected.
- `@`-forbidden rule in usernames is the disjoint namespace with email
lookup (`dispatch_login` at `auth_application_service.rs:1018`).
Case-insensitive usernames align semantics with email addresses
(already case-insensitive in practice), so any future
`groupname@domain` composition stays consistent.
- NextCloud URL `/remote.php/dav/files/{user}/…` uses `{user}` as an
informational / consistency-check marker, not a security boundary —
the chroot ACL is the real authz. Handling case in the URL segment is
a small local change (documented in `session.rs:33-34`).
## Design decisions
1. **Silently lowercase on ingest** (registration, admin-create, OIDC
provisioning, rename). Never reject uppercase input from clients —
accept liberally, store strictly (Postel's Law).
2. **Explicit migration** (`oxicloud migrate lowercase-usernames
[--dry-run]`). The server never mutates `auth.users` at boot. Ops
MUST run the migration explicitly. Follows the
[[feedback_no_silent_auto_repair]] rule: consistency tenants are
discovery-only by default; mutation is opt-in.
3. **Refuse-to-boot** if any active-user username is not already
lowercase. Boot error message shows the exact CLI command to run.
Boot performs a read-only verification only.
4. **Collision tiebreak** on migration: `(last_login_at DESC NULLS
LAST, created_at ASC)`. Winner keeps the canonical lowercased name.
Losers get `-2`, `-3`, … suffix (increment until free), matching the
pattern in `oxicloud migrate nfc-filenames`.
5. **Active accounts only** in the boot check + migration. Soft-deleted
/ disabled rows are skipped (they don't block usable logins). NULL
usernames (OPAQUE-migrated accounts) are skipped in every layer —
the boot verifier, the migration UPDATE, the CLI report. The
`WHERE username <> LOWER(username)` predicate is already NULL-safe
by SQL semantics (NULL comparisons yield NULL, filtered out); state
it explicitly so a reviewer isn't left wondering.
6. **Rename-only** on collision resolution — sessions are not
invalidated. Sessions key on `user_id` so they survive the rename.
7. **Un-soft-delete of a mixed-case account uses the SAME suffix
scheme.** If `Alice` is soft-deleted (skipped by migration) and
later un-soft-deleted while `alice` already exists, the un-soft-
delete path re-normalizes via `set_username` and, on collision,
assigns `alice-2` / `alice-3` / … — the same helper the migration
CLI calls. Both callers reach for a shared
`find_free_username_suffix(pool, base) -> String` in
`src/common/username_migration.rs` so migration + un-soft-delete
agree by construction. Without this, an un-soft-delete could
create a fresh collision that the next boot's auto-rename
couldn't resolve (auto-rename handles singletons only) — the
server would then refuse-to-boot until an admin resolves the
tiebreak.
8. **Boot-time behaviour has three outcomes, not two.** The
verifier categorises the DB into: (a) clean — nothing to do;
(b) mixed-case rows with no `LOWER(username)` collision — the
server **auto-lowercases them in one atomic transaction and
continues**, emitting an audit line per rename; (c) at least
one `LOWER(username)` collision — the server **refuses to
boot** because tiebreak requires human judgement. Silent
action is bounded to (b), where there is exactly one correct
move. This is a narrower reading of
[[feedback_no_silent_auto_repair]] than "no silent action
ever": the rule targets consistency-check jobs where drift is
a bug signal; a schema-adjacent boot invariant with a
unique-correct-fix is a different situation. Making the
trivial-case common path a no-op massively lowers upgrade
friction for the 90% self-hosted deployment.
## Not in scope
- Unicode case-folding (usernames are ASCII-only by validation).
- `display_name` split (usernames were already just identifiers;
free-form display is a separate future feature if a user asks for
it — deferred pending real demand signal).
- OIDC provisioning behaviour change beyond the ingest-normalize point.
- Case-insensitivity for emails (already achieved in practice; not
touched).
- Any change to `validate_username`'s character-class rules.
- Any change to the WebDAV URL shape `/dav/files/{user}/…` (client
compat; drop deferred separately per [[project_nc_multidrive_poc]]).
- Group names (`SubjectGroup`). Lowercase by convention today; no
runtime enforcement, no migration. If group-name case-insensitivity
becomes a real ask, it lands as a sibling plan doc with the same
shape.
## Deliverables
### 1. Ingest normalization
Change `validate_username` to return the canonical form instead of
`()`:
```rust
// src/domain/entities/user.rs — new signature
fn validate_username(username: &str) -> UserResult<String> {
let normalized = username.trim().to_ascii_lowercase();
// ... existing length + charset + boundary checks apply to `normalized` ...
Ok(normalized)
}
```
Every caller that today does `Self::validate_username(u)?;` becomes
`let u = Self::validate_username(&u)?;` — the returned canonical form
is what gets stored. Because the return type changes from `Result<()>`
to `Result<String>`, any caller that ignores the result now becomes a
compile error — the type system forces every write path through the
normalizer.
Write sites all funnel through `User::new` (`src/domain/entities/user.rs:313`)
or `User::set_username` (`:811`), so the signature change catches the
entity-write path automatically. Callers to touch:
- Application services calling `User::new`:
- `auth_application_service.rs:840` — `register()` public signup
- `auth_application_service.rs:944` — `setup_create_admin()`
first-boot admin
- `auth_application_service.rs:3520`, `:3532` — `admin_create_user()`
external + internal branches
- `auth_application_service.rs:4692` — OIDC JIT provisioning
- `magic_link_invite_service.rs:233` — magic-link external invite
- User-driven rename (calls `User::set_username`):
- `auth_application_service.rs:2756-2801` — `update_profile()`
- Repository-write compile-error catches:
- `src/infrastructure/repositories/pg/user_pg_repository.rs:281`
(`create_user` INSERT) and `:740` (`update_user` UPDATE) — these
bind `user_clone.username()`, which is now guaranteed lowercase by
the entity constructor.
### 2. OIDC JIT derivation
`auth_application_service.rs:4649-4690` derives a local username from
the OIDC `preferred_username` / `name` / `sub` claims, filters to
`[a-zA-Z0-9._-]`, and truncates. **It does not currently lowercase.**
Add `to_ascii_lowercase()` on the derived string before passing to
`User::new`. This is beyond what the entity signature change catches —
explicit fix required.
### 3. Lookup normalization
Repository `find_by_username`-style methods internally lowercase the
input before the SQL query, so callers don't have to remember. One-line
change per method:
- `src/infrastructure/repositories/pg/user_pg_repository.rs:487`
(`get_user_by_username`) — add `let username = username.trim().
to_ascii_lowercase();` before the `.bind(&username)` at line 488.
- `src/infrastructure/repositories/pg/user_pg_repository.rs:1043`
(`search_users`) — `ILIKE` is already case-insensitive by
construction; verify nothing regresses.
- `src/infrastructure/repositories/pg/user_pg_repository.rs:1504`
(`search_usernames`) — same as above.
- `src/application/services/storage_usage_service.rs:145-149`
(`update_user_storage_usage_by_username`) — raw SQL bind; normalize
before `.bind()`.
- `src/cli/opaque.rs:125`, `:202` — `opaque reset` CLI identifier
dispatch on `@`; lowercase the username branch input.
Post-migration, the DB is fully lowercase so `WHERE username = 'alice'`
matches. The mixed-case-DB-during-transition state cannot serve
traffic because the boot flow either (a) auto-renames the singleton
rows before `AppState` assembles, or (b) refuses to boot on collision
groups.
### 4. NextCloud DAV surface
Two coordinated changes on the NC surface:
- `src/interfaces/nextcloud/basic_auth_middleware.rs:94-134` — decoded
`raw_username` from the Basic Auth header, lowercase the whole
string. Safe for the `user~drive_uuid` multi-drive format because
UUID hex is `[0-9a-f-]` which lowercases to itself.
- `src/interfaces/nextcloud/basic_auth_middleware.rs:307-323`
(`parse_basic_auth` helper) — lowercase the username portion before
returning.
- `src/interfaces/nextcloud/session.rs:90-111`
(`extract_url_user`) — lowercase the returned `Cow<'_, str>` value
from URL decode. The cross-check comparison at `session.rs:157-161`
(`url_user != session.raw_username`) then compares normalized vs
normalized — no change needed at the comparison site itself.
Downstream `session.raw_username` consumers (WebDAV / OCS href
builders, MOVE Destination parsers, avatar / trashbin handlers) all
pass through and emit lowercase automatically — no per-site change
needed.
**Client compatibility:** NC / DAVX5 clients that cached URLs like
`/remote.php/dav/files/Alice/…` continue to work through the migration
because the server accepts uppercase URL segments **indefinitely**
(the Basic Auth middleware + `extract_url_user` both lowercase on
decode). No forced client upgrade or reconfiguration. PROPFIND
response bodies emit lowercase hrefs (from canonical
`session.raw_username`), which well-behaved clients update on next
sync.
Expected per-client behavior on first PROPFIND after upgrade:
- **Nextcloud desktop** — prompts a one-time re-sync notification
when it notices the account URL case changed. Files re-verify
via ETag, so no re-upload; the re-sync completes in
seconds-to-minutes depending on file count. Users click through
the reconnect dialog.
- **DAVX5** (calendars, contacts) — silent update of the internal
`principal-URL`; user sees no dialog.
- **NC mobile app** — silent refresh of the account tile.
- **Older / misbehaving clients** — may create a duplicate account
profile (rare, cosmetic, not destructive).
**Zero data risk in every path.** The chroot ACL keys on
`user_id`, not username, so files, calendars, contacts, and
grants all follow the user across the rename. The blast radius
is a one-time UX notification, not lost bytes.
**Power-user pre-emption** (worth documenting in CHANGELOG): ops
who want to avoid the re-sync prompt entirely can, before
upgrading, log into each NC desktop client and manually update
the account URL from `.../USERNAME` to lowercase. Cheap
prophylactic for organizations rolling out to non-technical
users.
### 5. Chunked-upload directory rename
`src/infrastructure/services/nextcloud_chunked_upload_service.rs:99-103`
uses `user.username` as an on-disk directory name AND as an in-memory
cache key. Post-migration, `user.username` becomes lowercase; any
in-flight upload for `Alice` at migration time strands the on-disk
`base_dir/Alice/upload_xxx/` directory and orphans its cache entry.
The migration command SHOULD walk `base_dir/*/` and rename any
mixed-case subdirectory to its lowercase form. Collision handling
(both `Alice/` and `alice/` present) → merge contents; else simple
rename. In practice this is likely a no-op — chunked-upload state
is ephemeral, and simultaneous mixed-case uploads by the same user
are rare.
**Implementation status:** deferred. Chunked-upload state is
ephemeral: any in-flight upload that gets stranded is retryable
by the client (the upload session's timeout eventually purges the
stale dir; the client retries with a fresh `upload_id`, this time
under the lowercase username). Wiring the dir-walk into the CLI
adds ~40 lines of async filesystem code (walk, collision merge,
mtime-preserving move) and a new `--chunk-dir <path>` arg — the
CLI otherwise doesn't need to know about the storage-path
config layer. Not worth it for a rare no-op; add if user reports
show a real problem.
**Ops manual step** — if a migration is run WHILE an upload is
in flight, ops can either restart the affected client (the
upload session is stateful across a `create → chunks → complete`
cycle, so the client will retry from scratch) or manually
`mv base_dir/Alice base_dir/alice` after the DB migration
completes.
### 6. Boot-time verification
New module `src/common/username_migration.rs` exposing:
```rust
pub async fn verify_all_usernames_lowercase(pool: &PgPool) -> Result<(), String>
```
Runs after `sqlx::migrate!()` completes, before `AppState` is
assembled. Query:
```sql
SELECT id, username, created_at, last_login_at
FROM auth.users
WHERE username <> LOWER(username)
-- NULL usernames (OPAQUE-migrated accounts) are already filtered
-- out by SQL semantics: NULL <> anything yields NULL, which
-- WHERE excludes. Explicit for the reviewer's benefit.
-- add is_deleted / disabled filter if such a flag exists
ORDER BY LOWER(username),
(last_login_at IS NULL),
last_login_at DESC NULLS LAST,
created_at ASC
LIMIT 200; -- soft cap on error-message size
```
If empty → boot proceeds. If non-empty → format the FATAL error and
return `Err(String)`. `main.rs` propagates via `?` to a non-zero
process exit.
Boot only READS `auth.users`; never WRITES. This is the "explicit
migration required" enforcement layer.
**Error message format** (self-sufficient — no docs required at 3 AM):
```
FATAL: cannot start — <N> user account(s) have non-lowercase usernames.
Before this version can boot, run the migration:
oxicloud migrate lowercase-usernames --dry-run # preview
oxicloud migrate lowercase-usernames # apply
Affected accounts (up to 20 shown; full list via the dry-run):
Alice (id: a1b2c3d4-... last_login: 2026-08-01)
BOB (id: 9abc0000-... last_login: never)
...
The migration handles case-collisions (Alice + alice → alice keeps
the name based on most recent login; the other gets alice-2 suffix).
Sessions and grants survive the rename (they key on user_id).
```
### 7. Migration CLI
New action under `oxicloud migrate`:
```rust
// src/cli/migrate.rs — extend the Action enum
Action::LowercaseUsernames { dry_run: bool }
```
Following the shape of `run_nfc_filenames`:
- Load all active users (skip soft-deleted / disabled AND rows
where `username IS NULL` — OPAQUE-migrated accounts have no
username string to normalize)
- Group by `LOWER(username)`
- For each group:
- Single-member group with mixed-case name → UPDATE to lowercase
- Multi-member group (collision) → apply tiebreak
`(last_login_at DESC NULLS LAST, created_at ASC)`, winner UPDATEs
to lowercase, losers UPDATE to `<lowercase>-2`, `-3`, … (increment
until free)
- Per-row `println!` log:
`NORMALIZE user=<uuid> '<before>' ({}B) → '<after>' ({}B)`
- Summary at end: scanned / already-lowercase / normalized /
collision-resolved / renamed-to-suffix
- `--dry-run` guards all UPDATEs
After the DB pass, the chunked-upload directory rename step (see
Deliverable 5) is deferred; run manually only if in-flight uploads
were live at migration time.
Suffix search reuses the pattern from
`find_free_folder_duplicate_name` in the existing NFC migration —
increment-until-free loop, starting at `-2`, probing until an
unused suffix is found. Robust against pre-existing rows like
`alice-2` already being taken (the probe just steps past them
to `-3`, `-4`, …).
Extracted into a shared public helper in
`src/common/username_migration.rs`:
```rust
pub async fn find_free_username_suffix(pool: &PgPool, base: &str) -> Result<String, sqlx::Error>
```
Both the migration CLI AND the un-soft-delete API (Design decision
7) call this helper — same collision-resolution behavior by
construction, no drift risk between the two paths.
Bounded at 10,000 as a safety cap. The probability of reaching
that in a real deployment is negligible — it would require ~10 K
distinct accounts all originally cased differently but sharing
the same lowercase form (a normal collision is 2-3 accounts, not
10 K). If the cap ever fires, something is very wrong with the
account universe and the migration ABORTs with a loud error
rather than silently truncating — the loud abort IS the
detection mechanism.
### 8. Test seed audit
Sweep-verified: existing test seeds all produce lowercase or NULL
usernames. Worth one more grep pass to ensure no test fixture INSERTs
`INSERT INTO auth.users … 'AliceTest'` — if any exist, lowercase them
in the same commit to avoid CI refuse-to-boot regressions.
Files verified (all safe):
- `src/infrastructure/repositories/pg/user_pg_repository.rs:1786`
- `src/infrastructure/repositories/pg/opaque_pg_repository.rs:339`
(NULL)
- `src/application/services/auth_application_service.rs:4982` (NULL)
- `src/application/services/subject_group_service.rs:796` (NULL)
- `src/bin/load-seed.rs:414`, `:446` (`load_user_XXXX` — lowercase)
- `src/mount_it_support.rs:61` (`make_user(name)` — verify callers)
- `tests/common/init-test-schema.sh:40` (`ci-admin` — lowercase)
### 9. Cosmetic side-effects (worth noting in CHANGELOG, non-blocking)
- `src/interfaces/nextcloud/avatar_handler.rs:283` — `pick_color`
derives a deterministic tile color from username bytes. Users whose
canonical username had uppercase letters will get a different
fallback-avatar tile color after the migration. One-time cosmetic
change.
- **NC desktop may perform a one-time re-sync** — see Deliverable 4.
### 10. Documentation
Release notes / CHANGELOG entry is NOT part of this PR — the
canonical repo's maintainer handles release notes at version-bump
time. This PR just leaves the notes-worthy items enumerated here
so the maintainer has the bullets to pick from when the next
version ships:
- Server auto-lowercases non-colliding mixed-case usernames at
first boot. No ops action needed for the common case.
- On `LOWER(username)` collision (`Alice` + `alice` both active),
the server refuses to boot; ops runs `oxicloud migrate
lowercase-usernames`. Exact CLI command shown in the refusal.
- Nextcloud desktop clients will prompt for a one-time re-sync on
first PROPFIND after upgrade. Files are ETag-verified, not
re-uploaded. DAVX5 and NC mobile handle the URL case change
silently. **No forced client upgrade or reconfiguration** —
server accepts uppercase URL segments indefinitely.
- Optional pre-emption for non-technical users: ops can manually
update the account URL to lowercase in each NC desktop client
before upgrading, avoiding the re-sync prompt entirely.
- Usernames become lowercase in ALL UI display surfaces (share
dialogs, activity feeds, admin panels, PROPFIND response
bodies, notification bell). Login identity unchanged from the
user's POV (they can still type any case at the login form).
- Avatar fallback color may change for users with previously-
uppercase usernames.
- Note: `display_name` is a possible follow-up if users miss
capitalisation for display — deferred pending demand signal, no
compat cost to adding later.
The two docs that DO ship with this PR:
- `docs/config/env.md` — note the boot-time check + migration command.
- `docs/install/binary.md` — upgrade-from-case-sensitive section.
### 11. Test coverage
- **Unit**: `validate_username("Alice")` returns `Ok("alice")`;
`validate_username("alice-")` returns `Err(...)` unchanged;
`validate_username(" Alice ")` returns `Ok("alice")`.
- **Unit**: `format_refusal_message_collisions` — 1 group renders
canonical + members + CLI; > 10 groups renders overflow tail;
total-affected-count sums across groups.
- **Hurl** (`tests/api/lowercase_usernames.hurl`, new): register a
user with `MixedCase`, assert DB stores `mixedcase`; log in with
`MIXEDCASE` and `mixedcase` — both succeed; rename to `NewName`,
assert `newname` stored; NC Basic Auth accepts `MixedCase:pass`,
`MIXEDCASE:pass`, `mixedcase:pass`.
- **Manual** (against dev DB, not CI):
- Auto-rename path: `UPDATE auth.users SET username='Alice' WHERE
username='alice'` (no collision); boot server → verify audit log
line + WARN summary, row is `alice` after boot, service starts.
- Collision path: `INSERT INTO auth.users … 'Alice'` on top of
existing `alice`; boot server → verify refusal message names both
rows + exact CLI shown, server exits non-zero.
- `oxicloud migrate lowercase-usernames --dry-run` → verify report
of collision + tiebreak decision
- `oxicloud migrate lowercase-usernames` → verify apply, one row
keeps `alice`, other gets `alice-2`
- Boot again → succeeds (Clean outcome)
- `curl -u ALICE:pass https://oxicloud/remote.php/dav/files/ALICE/…`
→ succeeds (accepts uppercase input, resolves to lowercase user)
## Cache-and-consistency observations (informational)
- `src/infrastructure/services/login_lockout_service.rs:33,68-89` —
already lowercases the key at line 58. No code change; comment
becomes factual not incidental.
- `src/application/services/app_password_service.rs:89,317-323` —
BLAKE3-keyed cache using the raw wire username. Post-normalization,
both sides normalize consistently → cache stays coherent. 300 s TTL
self-heals any transitional window.
- `NC_CHROOT_CACHE` in `basic_auth_middleware.rs:34-40` — keyed on
`Uuid`, not username. Unaffected.
## Critical files
Full enumeration in the Deliverables sections above. Grouped summary:
**Ingest normalizer:**
- `src/domain/entities/user.rs` (signature change + callers)
**Application services (write callers):**
- `src/application/services/auth_application_service.rs`
- `src/application/services/magic_link_invite_service.rs`
**Repositories (lookup normalization):**
- `src/infrastructure/repositories/pg/user_pg_repository.rs`
- `src/application/services/storage_usage_service.rs`
- `src/cli/opaque.rs`
**NextCloud DAV surface:**
- `src/interfaces/nextcloud/basic_auth_middleware.rs`
- `src/interfaces/nextcloud/session.rs`
- `src/infrastructure/services/nextcloud_chunked_upload_service.rs`
**New files:**
- `src/common/username_migration.rs`
- `tests/api/lowercase_usernames.hurl`
**Main entry:**
- `src/main.rs` (call verifier after `sqlx::migrate!()`)
**Migration CLI:**
- `src/cli/migrate.rs`
**Docs:**
- `CHANGELOG.md`
- `docs/config/env.md`
- `docs/install/binary.md`
## Delivery order
1. Change `validate_username` signature to return `Result<String>` —
one file.
2. Fix OIDC JIT derivation
(`auth_application_service.rs:4649-4690`) to lowercase before
passing to `User::new` — explicit change beyond the entity
normalizer's compile-time catches.
3. Iterate on compile errors — the return-type change catches every
downstream write-site.
4. Update repository lookup methods (`user_pg_repository.rs`,
`storage_usage_service.rs`, `cli/opaque.rs`) to internally
lowercase input before `.bind()`.
5. Update NC `basic_auth_middleware.rs` (lowercase `raw_username` at
decode) + `session.rs::extract_url_user` (lowercase return).
6. Add the boot-time verification helper (`src/common/username_migration.rs`)
+ wire into `main.rs`.
7. Extend `oxicloud migrate` with `lowercase-usernames [--dry-run]` —
DB pass + chunked-upload directory rename.
8. Test seed audit (grep pass).
9. Add hurl coverage.
10. Admin docs update (`docs/config/env.md` boot-check subsection +
`docs/install/binary.md` upgrade section). CHANGELOG is Dio's
job at version-bump time — not part of this PR.
11. Manual smoke test against dev DB.
12. PR to canonical.
## Total scope estimate
~6-8 hours of careful work. Larger than the initial estimate because
of these sweep-surfaced additions:
- OIDC JIT explicit fix (small).
- Chunked-upload directory rename step in the migration (~30 min).
- Test-seed audit (~15 min).
- More lookup callsites than initially thought.
The shape is uniform (`to_ascii_lowercase()` at every touchpoint) and
the compiler catches missed entity-write sites via the
`Result<String>` signature change. The parts NOT caught by the
compiler (OIDC JIT, lookup normalizers, NC URL segment, chunked-upload
directory) are the ones needing careful review — enumerated above.
## References
- Issue: [#691](https://github.com/AtalayaLabs/OxiCloud/issues/691)
- Related feature restrictions today:
- `validate_username` at
`src/domain/entities/user.rs:884-916`
- `@`-disjoint dispatch at
`src/application/services/auth_application_service.rs:1018`
- Related project docs:
- `docs/plan/auth-simplification.md` — the broader auth surface this
fits within
- Prior similar migration:
`oxicloud migrate nfc-filenames` in `src/cli/migrate.rs`
+19
View File
@@ -33,3 +33,22 @@ export function copyFolders(folderIds: string[], targetFolderId: string | null):
target_folder_id: targetFolderId target_folder_id: targetFolderId
}); });
} }
/**
* Stream a multi-item selection as a server-built zip (`POST /api/batch/download`
* — folders included, unlike the legacy per-item loop). The caller names and
* saves the returned blob.
*/
export async function downloadBatch(fileIds: string[], folderIds: string[]): Promise<Blob> {
const res = await apiFetch('/api/batch/download', {
method: 'POST',
credentials: 'same-origin',
headers: { ...JSON_HEADERS, ...getCsrfHeaders() },
body: JSON.stringify({ file_ids: fileIds, folder_ids: folderIds })
});
if (!res.ok) {
const e = (await res.json().catch(() => ({}))) as { error?: string; message?: string };
throw new Error(e.error || e.message || `batch download failed: ${res.status}`);
}
return res.blob();
}
@@ -139,3 +139,24 @@ export async function removeFavorite(type: ItemType, id: string): Promise<void>
}); });
if (!res.ok) throw new Error(`remove favorite failed: ${res.status}`); if (!res.ok) throw new Error(`remove favorite failed: ${res.status}`);
} }
/** One item for the batch favorites call. */
export interface FavoriteBatchItem {
item_id: string;
item_type: ItemType;
}
/**
* Batch-add favorites via `POST /api/favorites/batch` — a single round trip
* for the whole selection (used by the files page and search results batch bar).
*/
export async function addFavoritesBatch(items: FavoriteBatchItem[]): Promise<void> {
if (items.length === 0) return;
const res = await apiFetch('/api/favorites/batch', {
method: 'POST',
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json', ...getCsrfHeaders() },
body: JSON.stringify({ items })
});
if (!res.ok) throw new Error(`batch favorites failed: ${res.status}`);
}
+10
View File
@@ -9,6 +9,16 @@ import type { ItemType } from '$lib/api/types';
export interface ShareMeta { export interface ShareMeta {
item_type: ItemType; item_type: ItemType;
item_name: string; item_name: string;
/** The shared item's id — file shares use it to build the preview src. */
item_id: string;
/**
* File shares only: resolved by the server at read time so the landing
* page can inline a media preview (video player / image) instead of a
* bare download button. Absent for folder shares.
*/
mime_type?: string;
/** File shares only: the shared file's size in bytes. */
size?: number;
} }
export interface ShareFolderEntry { export interface ShareFolderEntry {
@@ -0,0 +1,354 @@
<script lang="ts">
import Icon from '$lib/icons/Icon.svelte';
import { t } from '$lib/i18n/index.svelte';
import {
defaultFilterState,
type DateKey,
type ResourceFilterState,
type SizeKey,
type TypeKey
} from '$lib/utils/searchFilters';
interface Props {
/** Bindable full filter state (keyword + toggles + presets). */
value?: ResourceFilterState;
/** Advanced section (type/size/date/recursive) expanded. */
expanded?: boolean;
placeholder?: string;
/** Debounce for the keyword input (ms). */
debounceMs?: number;
/** Hide the recursive toggle (a surface where scope is owned elsewhere). */
hideRecursive?: boolean;
}
let {
value = $bindable(defaultFilterState()),
expanded = $bindable(false),
placeholder = t('filter.placeholder', 'Search this folder and subfolders…'),
debounceMs = 300,
hideRecursive = false
}: Props = $props();
// Option label lists reuse the /search page's i18n keys — the vocabularies
// themselves (extensions, byte bounds) live in the shared searchFilters util.
const TYPES: { v: TypeKey; l: string }[] = [
{ v: 'all', l: t('search.type.all', 'All types') },
{ v: 'image', l: t('search.type.image', 'Images') },
{ v: 'video', l: t('search.type.video', 'Videos') },
{ v: 'document', l: t('search.type.document', 'Documents') },
{ v: 'audio', l: t('search.type.audio', 'Audio') },
{ v: 'archive', l: t('search.type.archive', 'Archives') }
];
const SIZES: { v: SizeKey; l: string }[] = [
{ v: 'all', l: t('search.size.all', 'Any size') },
{ v: 'small', l: t('search.size.small', '< 1 MB') },
{ v: 'medium', l: t('search.size.medium', '1–100 MB') },
{ v: 'large', l: t('search.size.large', '> 100 MB') }
];
const DATES: { v: DateKey; l: string }[] = [
{ v: 'all', l: t('search.date.all', 'Any time') },
{ v: 'day', l: t('search.date.day', 'Past 24 hours') },
{ v: 'week', l: t('search.date.week', 'Past week') },
{ v: 'month', l: t('search.date.month', 'Past month') },
{ v: 'year', l: t('search.date.year', 'Past year') }
];
// Keyword buffer: typing updates the buffer immediately (responsive input)
// and pushes into `value.query` debounced, so a keystroke doesn't fire a
// backend search per character. `lastPushed` disambiguates our own pushes
// from external writes (e.g. the page's clear-filter Escape path), which
// flow back into the buffer via the sync effect below.
let keyword = $state(value.query);
let lastPushed = value.query;
let timer: ReturnType<typeof setTimeout> | null = null;
$effect(() => {
// External `value.query` change → adopt it into the input buffer.
const external = value.query;
if (external !== lastPushed) {
if (timer) {
clearTimeout(timer);
timer = null;
}
keyword = external;
lastPushed = external;
}
});
$effect(() => {
// No reactive deps — teardown-only, clearing a pending debounce on destroy.
return () => {
if (timer) clearTimeout(timer);
};
});
function pushKeyword(v: string) {
if (timer) {
clearTimeout(timer);
timer = null;
}
value.query = v;
lastPushed = v;
}
function handleInput(e: Event) {
const v = (e.target as HTMLInputElement).value;
keyword = v;
if (timer) clearTimeout(timer);
timer = setTimeout(() => {
timer = null;
pushKeyword(v);
}, debounceMs);
}
function onInputKeydown(e: KeyboardEvent) {
// Escape clears the keyword locally and never reaches the page-level
// handler (which would otherwise also clear the selection / filters).
if (e.key === 'Escape') {
e.stopPropagation();
clearKeyword();
} else if (e.key === 'Enter') {
// Enter flushes the debounce for an immediate search.
e.preventDefault();
pushKeyword(keyword);
}
}
function clearKeyword() {
keyword = '';
pushKeyword('');
}
const activeCount = $derived(
(value.type !== 'all' ? 1 : 0) + (value.size !== 'all' ? 1 : 0) + (value.date !== 'all' ? 1 : 0)
);
</script>
<div class="sfb" data-testid="search-filter-bar">
<div class="sfb__row">
<div class="sfb__input-wrap">
<span class="sfb__magnifier"><Icon name="search" /></span>
<input
class="sfb__input"
type="search"
{placeholder}
aria-label={t('filter.keyword', 'Keyword')}
data-testid="filter-keyword-input"
value={keyword}
oninput={handleInput}
onkeydown={onInputKeydown}
/>
{#if keyword.length > 0}
<button
class="sfb__clear"
type="button"
aria-label={t('filter.clear_keyword', 'Clear search')}
data-testid="filter-clear-keyword-btn"
onclick={clearKeyword}
>
<Icon name="times" />
</button>
{/if}
</div>
<button
class="sfb__toggle"
class:sfb__toggle--active={activeCount > 0}
type="button"
aria-expanded={expanded}
aria-label={t('filter.advanced', 'Filters')}
title={t('filter.advanced', 'Filters')}
data-testid="filter-advanced-toggle-btn"
onclick={() => (expanded = !expanded)}
>
<Icon name="sliders-h" />
{#if activeCount > 0}
<span class="sfb__badge">{activeCount}</span>
{/if}
</button>
</div>
{#if expanded}
<div class="sfb__advanced" data-testid="filter-advanced-row">
<label class="sfb__field">
<span class="sfb__label">{t('search.type_label', 'Type')}</span>
<select bind:value={value.type}>
{#each TYPES as opt (opt.v)}
<option value={opt.v}>{opt.l}</option>
{/each}
</select>
</label>
<label class="sfb__field">
<span class="sfb__label">{t('search.size_label', 'Size')}</span>
<select bind:value={value.size}>
{#each SIZES as opt (opt.v)}
<option value={opt.v}>{opt.l}</option>
{/each}
</select>
</label>
<label class="sfb__field">
<span class="sfb__label">{t('search.date_label', 'Date')}</span>
<select bind:value={value.date}>
{#each DATES as opt (opt.v)}
<option value={opt.v}>{opt.l}</option>
{/each}
</select>
</label>
{#if !hideRecursive}
<label class="sfb__check">
<input type="checkbox" bind:checked={value.recursive} />
<span>{t('filter.recursive', 'Include subfolders')}</span>
</label>
{/if}
</div>
{/if}
</div>
<style>
.sfb {
display: flex;
flex-direction: column;
gap: var(--space-1);
width: 100%;
}
.sfb__row {
display: flex;
align-items: center;
gap: var(--space-1);
}
.sfb__input-wrap {
position: relative;
display: flex;
align-items: center;
flex: 1;
min-width: 0;
}
.sfb__magnifier {
position: absolute;
left: 0.6rem;
color: var(--color-text-secondary);
pointer-events: none;
}
.sfb__input {
width: 100%;
padding: 0.45rem 2rem 0.45rem 2.1rem;
border: 1px solid var(--color-border);
border-radius: var(--radius-md);
background: var(--color-bg-surface);
color: var(--color-text);
font-size: var(--text-sm);
}
.sfb__input:focus {
outline: none;
border-color: var(--color-accent);
}
.sfb__input::-webkit-search-cancel-button {
-webkit-appearance: none;
appearance: none;
}
.sfb__clear {
position: absolute;
right: 0.4rem;
display: flex;
align-items: center;
justify-content: center;
width: 1.4rem;
height: 1.4rem;
padding: 0;
border: none;
border-radius: var(--radius-sm);
background: none;
color: var(--color-text-secondary);
cursor: pointer;
}
.sfb__clear:hover {
background: var(--color-bg-hover);
color: var(--color-text);
}
.sfb__toggle {
position: relative;
display: flex;
align-items: center;
justify-content: center;
width: 2.2rem;
height: 2.2rem;
padding: 0;
border: 1px solid var(--color-border);
border-radius: var(--radius-md);
background: var(--color-bg-surface);
color: var(--color-text);
cursor: pointer;
flex-shrink: 0;
}
.sfb__toggle:hover {
background: var(--color-bg-hover);
}
.sfb__toggle--active {
border-color: var(--color-accent);
color: var(--color-accent);
}
.sfb__badge {
position: absolute;
top: -0.4rem;
right: -0.4rem;
min-width: 1rem;
height: 1rem;
padding: 0 0.2rem;
border-radius: var(--radius-sm);
background: var(--color-accent);
color: var(--color-on-accent);
font-size: var(--text-xs, 0.7rem);
line-height: 1rem;
text-align: center;
}
.sfb__advanced {
display: flex;
flex-wrap: wrap;
align-items: center;
gap: var(--space-2);
}
.sfb__field {
display: flex;
align-items: center;
gap: 0.4rem;
}
.sfb__label {
color: var(--color-text-secondary);
font-size: var(--text-sm);
white-space: nowrap;
}
.sfb__field select {
padding: 0.3rem 0.4rem;
border: 1px solid var(--color-border);
border-radius: var(--radius-sm);
background: var(--color-bg-surface);
color: var(--color-text);
font-size: var(--text-sm);
max-width: 9rem;
}
.sfb__check {
display: flex;
align-items: center;
gap: 0.4rem;
color: var(--color-text);
font-size: var(--text-sm);
cursor: pointer;
white-space: nowrap;
}
</style>
@@ -0,0 +1,241 @@
// Shared batch-actions composable for resource list views (files page,
// search results, …).
//
// Extracted verbatim from the files page so every surface that can select
// items shares one implementation of the batch favorite / download /
// delete / move / copy flows. Surfaces differ in (a) the id→item index,
// (b) what "refresh" means (folder reload vs search re-run) and (c) how a
// favorites flip reaches the rows — those differences are injected via
// the callbacks in `ResourceActionsOptions`.
import type { FileItem, FolderItem, ItemType } from '$lib/api/types';
import { downloadBatch } from '$lib/api/endpoints/batch';
import { deleteFile, fileDownloadUrl } from '$lib/api/endpoints/files';
import { deleteFolder } from '$lib/api/endpoints/folders';
import { addFavoritesBatch } from '$lib/api/endpoints/favorites';
import { t } from '$lib/i18n/index.svelte';
import { confirmDialog } from '$lib/stores/dialogs.svelte';
import { ui } from '$lib/stores/ui.svelte';
import { errorToast } from '$lib/utils/errors';
import { mapLimit } from '$lib/utils/mapLimit';
/** A minimal actionable item reference (dialog props, favorites payload). */
export interface ActionTarget {
id: string;
name: string;
kind: ItemType;
}
export interface ResourceActionsOptions {
/** Current on-screen rows, read at action time (fresh, never stale). */
getItems: () => ReadonlyArray<FileItem | FolderItem>;
/** Selected ids — the page's SvelteSet mirror of the list's selection. */
getSelected: () => ReadonlySet<string>;
clearSelection: () => void;
/** After a successful delete: reload the listing or re-run the search. */
onChanged: () => void | Promise<void>;
/** Extra bookkeeping after a delete (e.g. the session/quota refresh). */
afterDelete?: () => void;
/**
* After favorites succeed, update rows in place (keeps scroll position on
* infinite-scroll pages). Defaults to flipping `is_favorite` on the items
* returned by `getItems()` — sufficient for plain DTO `$state` arrays.
*/
onFavoritesApplied?: (ids: ReadonlySet<string>) => void;
}
/** Name for a server-zipped multi-item archive (matches the legacy format). */
export function batchZipName(): string {
// eslint-disable-next-line svelte/prefer-svelte-reactivity -- filename stamp, never read reactively
const stamp = new Date().toISOString().replace('T', ' ').replace(/\..*/, '').replace(/:/g, '-');
return `oxicloud ${stamp}.zip`;
}
/** Trigger a browser download of `blob` as `name`. */
function saveBlob(blob: Blob, name: string): void {
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = name;
document.body.appendChild(a);
a.click();
a.remove();
URL.revokeObjectURL(url);
}
export function useResourceActions(opts: ResourceActionsOptions) {
// Move/copy dialog state, owned here so both surfaces bind one dialog to
// `moveDialog.*` instead of re-implementing the open/mode/items triple.
const moveDialog = $state({
open: false,
mode: 'move' as 'move' | 'copy',
item: null as ActionTarget | null,
items: null as ActionTarget[] | null
});
function openBatchDialog(mode: 'move' | 'copy', items: ActionTarget[]): void {
moveDialog.items = items;
moveDialog.item = null;
moveDialog.mode = mode;
moveDialog.open = true;
}
/** Context-menu / single-row entry points. */
function openMove(target: ActionTarget): void {
moveDialog.item = target;
moveDialog.items = null;
moveDialog.mode = 'move';
moveDialog.open = true;
}
function openCopy(target: ActionTarget): void {
moveDialog.item = target;
moveDialog.items = null;
moveDialog.mode = 'copy';
moveDialog.open = true;
}
function selectionTargets(): ActionTarget[] {
// One O(M) index build instead of an O(N·M) `find` per selected id.
// Folders win id collisions, matching the old folder-first probe.
// eslint-disable-next-line svelte/prefer-svelte-reactivity -- ephemeral local index, discarded before any reactive read
const byId = new Map<string, ActionTarget>();
for (const f of opts.getItems()) byId.set(f.id, { id: f.id, name: f.name, kind: kindOf(f) });
return [...opts.getSelected()]
.map((id) => byId.get(id) ?? null)
.filter((x): x is ActionTarget => x !== null);
}
/**
* Download the whole selection as a single zip via POST /api/batch/download —
* folders are included (the old per-item loop silently skipped them). A lone
* file still streams directly so it keeps its original name/extension.
*/
async function batchDownload(): Promise<void> {
const targets = selectionTargets();
if (targets.length === 0) return;
const fileTargets = targets.filter((it) => it.kind === 'file');
const folderTargets = targets.filter((it) => it.kind === 'folder');
// Single file, no folders → direct download (preserves the real name).
if (fileTargets.length === 1 && folderTargets.length === 0) {
const file = opts.getItems().find((f) => f.id === fileTargets[0].id);
if (file) {
const a = document.createElement('a');
a.href = fileDownloadUrl(file.id);
a.download = file.name;
document.body.appendChild(a);
a.click();
a.remove();
}
return;
}
try {
const blob = await downloadBatch(
fileTargets.map((it) => it.id),
folderTargets.map((it) => it.id)
);
saveBlob(blob, batchZipName());
} catch (e) {
errorToast(e);
}
}
/** Batch add the selection to favorites — single /api/favorites/batch call. */
async function batchFavorites(): Promise<void> {
const items = opts.getItems();
// Build an id → item index so the "already favorite" filter is
// O(1) per selection member instead of an O(N·M) scan. Reused
// after success to flip `is_favorite` in place on each row.
// eslint-disable-next-line svelte/prefer-svelte-reactivity -- ephemeral local index, discarded before any reactive read
const byId = new Map<string, FileItem | FolderItem>();
for (const it of items) byId.set(it.id, it);
const targets = selectionTargets().filter((it) => !(byId.get(it.id)?.is_favorite ?? false));
if (targets.length === 0) {
ui.notify(t('files.already_favorites', 'All selected items are already favorites'), 'info');
opts.clearSelection();
return;
}
try {
await addFavoritesBatch(targets.map((it) => ({ item_id: it.id, item_type: it.kind })));
// eslint-disable-next-line svelte/prefer-svelte-reactivity -- ephemeral local index, discarded before any reactive read
const flipped = new Set(targets.map((it) => it.id));
if (opts.onFavoritesApplied) opts.onFavoritesApplied(flipped);
else
for (const id of flipped) {
const row = byId.get(id);
if (row) row.is_favorite = true;
}
ui.notify(t('files.added_favorites', 'Added to favorites'), 'success');
opts.clearSelection();
} catch (e) {
errorToast(e);
}
}
async function batchDelete(): Promise<void> {
const selected = opts.getSelected();
const ids = [...selected];
const ok = await confirmDialog({
title: t('files.batch_delete', 'Delete selected'),
message: t('files.confirm_batch_delete', { n: ids.length }, 'Move {{n}} items to trash?'),
confirmText: t('common.delete', 'Delete'),
danger: true
});
if (!ok) return;
// Bounded fan-out instead of a serial await per item: 100 deletes at
// ~30 ms RTT collapse from ~3 s of waterfall to a few round-trip
// windows. Failures toast individually and the rest still proceed.
const items = opts.getItems();
// eslint-disable-next-line svelte/prefer-svelte-reactivity -- ephemeral local index, discarded before any reactive read
const folderIdSet = new Set(items.filter((it) => !isFileItem(it)).map((it) => it.id));
await mapLimit(ids, 6, async (id) => {
try {
if (folderIdSet.has(id)) await deleteFolder(id);
else await deleteFile(id);
} catch (e) {
errorToast(e);
}
});
opts.clearSelection();
await opts.onChanged();
opts.afterDelete?.();
}
function batchMove(): void {
const items = selectionTargets();
if (items.length) openBatchDialog('move', items);
}
function batchCopy(): void {
const items = selectionTargets();
if (items.length) openBatchDialog('copy', items);
}
/** Pass as the MoveDialog `onmoved` handler. */
async function handleMoved(): Promise<void> {
opts.clearSelection();
await opts.onChanged();
}
return {
selectionTargets,
batchFavorites,
batchDownload,
batchDelete,
batchMove,
batchCopy,
openMove,
openCopy,
handleMoved,
moveDialog
};
}
function kindOf(item: FileItem | FolderItem): ItemType {
return isFileItem(item) ? 'file' : 'folder';
}
/** `FileItem | FolderItem` uses duck typing (`mime_type`) rather than a tag field. */
export function isFileItem(item: FileItem | FolderItem): item is FileItem {
return 'mime_type' in item;
}
@@ -0,0 +1,196 @@
import { describe, it, expect, vi, beforeEach } from 'vitest';
const { confirmDialog, ui } = vi.hoisted(() => ({
confirmDialog: vi.fn(),
ui: {
notify: vi.fn(),
startProgress: vi.fn(() => 1),
updateProgress: vi.fn(),
finishProgress: vi.fn()
}
}));
vi.mock('$lib/stores/dialogs.svelte', () => ({ confirmDialog, promptDialog: vi.fn() }));
vi.mock('$lib/stores/ui.svelte', () => ({ ui }));
vi.mock('$lib/api/client', () => ({ apiFetch: vi.fn(), apiJson: vi.fn() }));
vi.mock('$lib/api/csrf', () => ({ getCsrfHeaders: () => ({}) }));
vi.mock('$lib/api/endpoints/batch', () => ({
downloadBatch: vi.fn(),
copyFiles: vi.fn(),
copyFolders: vi.fn()
}));
vi.mock('$lib/api/endpoints/files', () => ({ deleteFile: vi.fn(), fileDownloadUrl: () => '/dl' }));
vi.mock('$lib/api/endpoints/folders', () => ({ deleteFolder: vi.fn() }));
vi.mock('$lib/api/endpoints/favorites', () => ({ addFavoritesBatch: vi.fn() }));
import { downloadBatch } from '$lib/api/endpoints/batch';
import { deleteFile } from '$lib/api/endpoints/files';
import { deleteFolder } from '$lib/api/endpoints/folders';
import { addFavoritesBatch } from '$lib/api/endpoints/favorites';
import { useResourceActions, type ActionTarget } from './useResourceActions.svelte';
import type { FileItem, FolderItem } from '$lib/api/types';
const m = (fn: unknown) => fn as ReturnType<typeof vi.fn>;
// The composable only reads `id` / `name` / `mime_type` / `is_favorite`; the
// full DTO shapes are satisfied via casts to keep the fixtures minimal.
function fileItem(id: string, overrides: Record<string, unknown> = {}): FileItem {
return {
id,
name: `${id}.txt`,
mime_type: 'text/plain',
is_favorite: false,
...overrides
} as unknown as FileItem;
}
function folderItem(id: string, overrides: Record<string, unknown> = {}): FolderItem {
return {
id,
name: id,
is_favorite: false,
...overrides
} as unknown as FolderItem;
}
function harness(items: Array<FileItem | FolderItem>, selected: string[]) {
const selection = new Set(selected);
const actions = useResourceActions({
getItems: () => items,
getSelected: () => selection,
clearSelection: () => selection.clear(),
onChanged: vi.fn(),
afterDelete: vi.fn()
});
return { actions, selection };
}
beforeEach(() => {
vi.clearAllMocks();
});
describe('selectionTargets', () => {
it('maps selected ids to targets with kind', () => {
const { actions } = harness([folderItem('d1'), fileItem('f1')], ['f1']);
expect(actions.selectionTargets()).toEqual<ActionTarget[]>([
{ id: 'f1', name: 'f1.txt', kind: 'file' }
]);
});
it('lets folders win id collisions', () => {
const { actions } = harness([fileItem('x'), folderItem('x')], ['x']);
expect(actions.selectionTargets()[0].kind).toBe('folder');
});
it('drops ids that are no longer on screen', () => {
const { actions } = harness([fileItem('f1')], ['gone']);
expect(actions.selectionTargets()).toEqual([]);
});
});
describe('batchFavorites', () => {
it('skips items that are already favorites and flips the rest in place', async () => {
const folder = folderItem('d1');
const file = fileItem('f1', { is_favorite: true });
const { actions, selection } = harness([folder, file], ['d1', 'f1']);
m(addFavoritesBatch).mockResolvedValue(undefined);
await actions.batchFavorites();
expect(addFavoritesBatch).toHaveBeenCalledWith([{ item_id: 'd1', item_type: 'folder' }]);
expect(folder.is_favorite).toBe(true);
// already-favorite rows keep their state
expect(file.is_favorite).toBe(true);
expect(selection.size).toBe(0);
expect(ui.notify).toHaveBeenCalledWith(expect.anything(), 'success');
});
it('notifies when every selected item is already a favorite', async () => {
const file = fileItem('f1', { is_favorite: true });
const { actions, selection } = harness([file], ['f1']);
await actions.batchFavorites();
expect(addFavoritesBatch).not.toHaveBeenCalled();
expect(ui.notify).toHaveBeenCalledWith(expect.anything(), 'info');
expect(selection.size).toBe(0);
});
});
describe('batchDownload', () => {
it('uses the batch zip endpoint for a mixed selection', async () => {
const { actions } = harness(
[fileItem('f1'), fileItem('f2'), folderItem('d1')],
['f1', 'f2', 'd1']
);
m(downloadBatch).mockResolvedValue(new Blob(['zip']));
await actions.batchDownload();
expect(downloadBatch).toHaveBeenCalledWith(['f1', 'f2'], ['d1']);
});
it('streams a lone file directly, without the zip endpoint', async () => {
const { actions } = harness([fileItem('f1')], ['f1']);
await actions.batchDownload();
expect(downloadBatch).not.toHaveBeenCalled();
});
});
describe('batchDelete', () => {
it('fans out per item after confirmation and calls onChanged + afterDelete', async () => {
const selection = new Set(['d1', 'f1']);
const onChanged = vi.fn();
const afterDelete = vi.fn();
const actions = useResourceActions({
getItems: () => [folderItem('d1'), fileItem('f1')],
getSelected: () => selection,
clearSelection: () => selection.clear(),
onChanged,
afterDelete
});
confirmDialog.mockResolvedValue(true);
await actions.batchDelete();
expect(deleteFolder).toHaveBeenCalledWith('d1');
expect(deleteFile).toHaveBeenCalledWith('f1');
expect(onChanged).toHaveBeenCalled();
expect(afterDelete).toHaveBeenCalled();
expect(selection.size).toBe(0);
});
it('does nothing when the confirm dialog is dismissed', async () => {
const { actions } = harness([fileItem('f1')], ['f1']);
confirmDialog.mockResolvedValue(false);
await actions.batchDelete();
expect(deleteFile).not.toHaveBeenCalled();
});
});
describe('move/copy dialogs', () => {
it('batch move opens the dialog with the selection', () => {
const { actions } = harness([fileItem('f1')], ['f1']);
actions.batchMove();
expect(actions.moveDialog.open).toBe(true);
expect(actions.moveDialog.mode).toBe('move');
expect(actions.moveDialog.items).toEqual<ActionTarget[]>([
{ id: 'f1', name: 'f1.txt', kind: 'file' }
]);
expect(actions.moveDialog.item).toBeNull();
});
it('batch copy opens the dialog in copy mode', () => {
const { actions } = harness([fileItem('f1')], ['f1']);
actions.batchCopy();
expect(actions.moveDialog.open).toBe(true);
expect(actions.moveDialog.mode).toBe('copy');
});
it('openMove sets a single item and handleMoved clears selection', async () => {
const selection = new Set(['f1']);
const onChanged = vi.fn();
const actions = useResourceActions({
getItems: () => [fileItem('f1')],
getSelected: () => selection,
clearSelection: () => selection.clear(),
onChanged
});
actions.openMove({ id: 'f1', name: 'f1.txt', kind: 'file' });
expect(actions.moveDialog.open).toBe(true);
expect(actions.moveDialog.item).toEqual({ id: 'f1', name: 'f1.txt', kind: 'file' });
await actions.handleMoved();
expect(selection.size).toBe(0);
expect(onChanged).toHaveBeenCalled();
});
});
+49
View File
@@ -0,0 +1,49 @@
import { describe, expect, it, vi } from 'vitest';
import { mapLimit } from './mapLimit';
describe('mapLimit', () => {
it('preserves result order regardless of completion order', async () => {
const result = await mapLimit([200, 50, 100, 10], 4, async (ms) => {
await new Promise((r) => setTimeout(r, ms));
return ms;
});
expect(result).toEqual([200, 50, 100, 10]);
});
it('never exceeds the concurrency cap', async () => {
let active = 0;
let peak = 0;
await mapLimit([1, 2, 3, 4, 5, 6, 7, 8], 3, async () => {
active++;
peak = Math.max(peak, active);
await new Promise((r) => setTimeout(r, 1));
active--;
return null;
});
expect(peak).toBe(3);
});
it('propagates rejections', async () => {
await expect(
mapLimit([1, 2, 3], 2, async (n) => {
if (n === 2) throw new Error('boom');
return n;
})
).rejects.toThrow('boom');
});
it('handles empty input', async () => {
const fn = vi.fn(async (n: number) => n);
await expect(mapLimit([], 4, fn)).resolves.toEqual([]);
expect(fn).not.toHaveBeenCalled();
});
it('runs items sequentially when limit is 1', async () => {
const calls: number[] = [];
await mapLimit([1, 2, 3], 1, async (n) => {
calls.push(n);
return n;
});
expect(calls).toEqual([1, 2, 3]);
});
});
+24
View File
@@ -0,0 +1,24 @@
/**
* Map `fn` over `items` with at most `limit` concurrent calls, preserving
* result order regardless of completion order.
*
* Extracted from the files page so batch fan-out paths (delete, drag-move,
* upload probing) and the shared resource-actions composable all use the
* same bounded-concurrency primitive.
*/
export async function mapLimit<T, R>(
items: readonly T[],
limit: number,
fn: (item: T) => Promise<R>
): Promise<R[]> {
const out = new Array<R>(items.length);
let next = 0;
const worker = async () => {
while (next < items.length) {
const i = next++;
out[i] = await fn(items[i]);
}
};
await Promise.all(Array.from({ length: Math.max(0, Math.min(limit, items.length)) }, worker));
return out;
}
@@ -0,0 +1,101 @@
import { describe, expect, it } from 'vitest';
import {
clearFilterState,
dateBound,
defaultFilterState,
filterToSearchOptions,
isFilterActive,
sizeBounds,
TYPE_EXT
} from './searchFilters';
const MB = 1024 * 1024;
describe('TYPE_EXT', () => {
it('covers the five non-all type keys', () => {
expect(Object.keys(TYPE_EXT).sort()).toEqual(
['archive', 'audio', 'document', 'image', 'video'].sort()
);
});
it('uses bare lowercase extensions', () => {
for (const exts of Object.values(TYPE_EXT)) {
for (const ext of exts) expect(ext).toBe(ext.toLowerCase());
}
});
});
describe('sizeBounds', () => {
it('maps the presets to byte ranges', () => {
expect(sizeBounds('all')).toEqual({});
expect(sizeBounds('small')).toEqual({ maxSize: MB });
expect(sizeBounds('medium')).toEqual({ minSize: MB, maxSize: 100 * MB });
expect(sizeBounds('large')).toEqual({ minSize: 100 * MB });
});
});
describe('dateBound', () => {
it('maps day to 24h ago', () => {
expect(dateBound('day')).toBe(Math.floor(Date.now() / 1000) - 86400);
});
it('returns undefined for all', () => {
expect(dateBound('all')).toBeUndefined();
});
});
describe('isFilterActive', () => {
it('is false for the default state', () => {
expect(isFilterActive(defaultFilterState())).toBe(false);
});
it('is true for a non-empty keyword (even whitespace-only counts as empty)', () => {
expect(isFilterActive({ ...defaultFilterState(), query: 'x' })).toBe(true);
expect(isFilterActive({ ...defaultFilterState(), query: ' ' })).toBe(false);
});
it('is true when any preset differs from all', () => {
expect(isFilterActive({ ...defaultFilterState(), type: 'image' })).toBe(true);
expect(isFilterActive({ ...defaultFilterState(), size: 'small' })).toBe(true);
expect(isFilterActive({ ...defaultFilterState(), date: 'week' })).toBe(true);
});
it('ignores the recursive toggle', () => {
expect(isFilterActive({ ...defaultFilterState(), recursive: false })).toBe(false);
});
});
describe('clearFilterState', () => {
it('resets every field in place', () => {
const f = { ...defaultFilterState(), query: 'a', recursive: false, type: 'video' as const };
clearFilterState(f);
expect(f).toEqual(defaultFilterState());
});
});
describe('filterToSearchOptions', () => {
it('omits everything for the default state', () => {
expect(filterToSearchOptions(defaultFilterState())).toEqual({
fileTypes: undefined,
minSize: undefined,
maxSize: undefined,
modifiedAfter: undefined,
recursive: true
});
});
it('maps each active dimension onto the wire options', () => {
const opts = filterToSearchOptions({
query: 'report',
recursive: false,
type: 'archive',
size: 'medium',
date: 'month'
});
expect(opts.fileTypes).toEqual(TYPE_EXT.archive);
expect(opts.minSize).toBe(MB);
expect(opts.maxSize).toBe(100 * MB);
expect(opts.modifiedAfter).toBe(dateBound('month'));
expect(opts.recursive).toBe(false);
});
});
+96
View File
@@ -0,0 +1,96 @@
// Shared resource-filter model for search-backed list views.
//
// Extracted from the /search page so the files page's filter bar and
// /search's filter selects share one source of truth for the preset
// vocabularies (type / size / date) and their mapping onto
// `SearchOptions`. Pure functions only — no runes here, so the module is
// unit-testable without component scaffolding.
import type { SearchOptions } from '$lib/api/endpoints/search';
export type TypeKey = 'all' | 'image' | 'video' | 'document' | 'audio' | 'archive';
export type SizeKey = 'all' | 'small' | 'medium' | 'large';
export type DateKey = 'all' | 'day' | 'week' | 'month' | 'year';
/** Full filter state for a search-backed resource list. */
export interface ResourceFilterState {
query: string;
/** Search subfolders too (backend default is true; exposed explicitly on /files). */
recursive: boolean;
type: TypeKey;
size: SizeKey;
date: DateKey;
}
export function defaultFilterState(): ResourceFilterState {
return { query: '', recursive: true, type: 'all', size: 'all', date: 'all' };
}
export const TYPE_EXT: Record<Exclude<TypeKey, 'all'>, string[]> = {
image: ['jpg', 'jpeg', 'png', 'gif', 'webp', 'svg', 'bmp', 'heic', 'avif', 'tiff'],
video: ['mp4', 'mov', 'mkv', 'avi', 'webm', 'm4v', 'wmv', 'flv'],
document: ['pdf', 'doc', 'docx', 'xls', 'xlsx', 'ppt', 'pptx', 'txt', 'md', 'odt', 'rtf', 'csv'],
audio: ['mp3', 'wav', 'flac', 'aac', 'ogg', 'm4a', 'opus'],
archive: ['zip', 'rar', '7z', 'tar', 'gz', 'bz2', 'xz']
};
const MB = 1024 * 1024;
export function sizeBounds(k: SizeKey): { minSize?: number; maxSize?: number } {
switch (k) {
case 'small':
return { maxSize: MB };
case 'medium':
return { minSize: MB, maxSize: 100 * MB };
case 'large':
return { minSize: 100 * MB };
default:
return {};
}
}
export function dateBound(k: DateKey): number | undefined {
const day = 86400;
const now = Math.floor(Date.now() / 1000);
switch (k) {
case 'day':
return now - day;
case 'week':
return now - 7 * day;
case 'month':
return now - 30 * day;
case 'year':
return now - 365 * day;
default:
return undefined;
}
}
/** True when any filter dimension would change the result set. */
export function isFilterActive(f: ResourceFilterState): boolean {
return f.query.trim() !== '' || f.type !== 'all' || f.size !== 'all' || f.date !== 'all';
}
/** Reset every dimension in place (runes-friendly — mutates the $state proxy). */
export function clearFilterState(f: ResourceFilterState): void {
f.query = '';
f.recursive = true;
f.type = 'all';
f.size = 'all';
f.date = 'all';
}
/**
* Map the filter state onto the search-wire options. Scope (folderId) and
* sorting stay the caller's concern — they differ per surface. The date
* preset maps to `modifiedAfter` (its labels read "Past N", which matches
* modified-time semantics); created-time bounds are a possible follow-up.
*/
export function filterToSearchOptions(
f: ResourceFilterState
): Pick<SearchOptions, 'fileTypes' | 'minSize' | 'maxSize' | 'modifiedAfter' | 'recursive'> {
return {
fileTypes: f.type === 'all' ? undefined : TYPE_EXT[f.type],
...sizeBounds(f.size),
modifiedAfter: dateBound(f.date),
recursive: f.recursive
};
}
+16 -1
View File
@@ -1,7 +1,22 @@
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; import { describe, it, expect, vi, beforeEach, afterEach, beforeAll } from 'vitest';
import { relativeTimeAgo } from './time'; import { relativeTimeAgo } from './time';
describe('relativeTimeAgo', () => { describe('relativeTimeAgo', () => {
// The formatter resolves the runtime default locale (`undefined`), so on a
// non-English dev machine (e.g. zh-CN Windows) the output is localized and
// these English-unit regexes fail. Pin English for the tests; vitest's
// per-file isolation keeps the module-level formatter cache from leaking.
const RealRelativeTimeFormat = Intl.RelativeTimeFormat;
beforeAll(() => {
// A regular function, not an arrow: time.ts calls the mock via `new`.
vi.spyOn(Intl, 'RelativeTimeFormat').mockImplementation(function (
locales?: string | string[],
options?: Intl.RelativeTimeFormatOptions
) {
return new RealRelativeTimeFormat('en', options);
} as unknown as typeof Intl.RelativeTimeFormat);
});
beforeEach(() => { beforeEach(() => {
vi.useFakeTimers(); vi.useFakeTimers();
vi.setSystemTime(new Date('2024-06-15T12:00:00Z')); vi.setSystemTime(new Date('2024-06-15T12:00:00Z'));
+267 -220
View File
@@ -40,11 +40,9 @@
import { canEditWithWopi, getEditorUrlWithFallback } from '$lib/api/endpoints/wopi'; import { canEditWithWopi, getEditorUrlWithFallback } from '$lib/api/endpoints/wopi';
import { addTracks, createPlaylist, listPlaylists } from '$lib/api/endpoints/music'; import { addTracks, createPlaylist, listPlaylists } from '$lib/api/endpoints/music';
import { copyFiles, copyFolders } from '$lib/api/endpoints/batch'; import { copyFiles, copyFolders } from '$lib/api/endpoints/batch';
import { apiFetch } from '$lib/api/client';
import { getCsrfHeaders } from '$lib/api/csrf';
import { countHidden, filterDotfiles } from '$lib/utils/dotfileFilter'; import { countHidden, filterDotfiles } from '$lib/utils/dotfileFilter';
import { preferences } from '$lib/stores/preferences.svelte'; import { preferences } from '$lib/stores/preferences.svelte';
import type { FileItem, FolderItem, ItemType } from '$lib/api/types'; import type { FileItem, FolderItem, ItemType, SortBy } from '$lib/api/types';
import ReadOnlyBanner from '$lib/components/ReadOnlyBanner.svelte'; import ReadOnlyBanner from '$lib/components/ReadOnlyBanner.svelte';
import FolderBreadcrumb from '$lib/components/FolderBreadcrumb.svelte'; import FolderBreadcrumb from '$lib/components/FolderBreadcrumb.svelte';
import ResourceList, { import ResourceList, {
@@ -62,6 +60,21 @@
import { ui } from '$lib/stores/ui.svelte'; import { ui } from '$lib/stores/ui.svelte';
import { dateBucket, sizeBucket, typeLabel } from '$lib/stores/files.svelte'; import { dateBucket, sizeBucket, typeLabel } from '$lib/stores/files.svelte';
import { replaceSet } from '$lib/utils/sets'; import { replaceSet } from '$lib/utils/sets';
import { mapLimit } from '$lib/utils/mapLimit';
import {
defaultFilterState,
clearFilterState,
isFilterActive,
filterToSearchOptions,
type ResourceFilterState
} from '$lib/utils/searchFilters';
import { searchResources } from '$lib/api/endpoints/search';
import {
useResourceActions,
batchZipName,
type ActionTarget
} from '$lib/composables/useResourceActions.svelte';
import SearchFilterBar from '$lib/components/SearchFilterBar.svelte';
// Message-bus logger. Users can tune with // Message-bus logger. Users can tune with
// oxi.setLogLevel('oxi:message-bus', 'debug') // oxi.setLogLevel('oxi:message-bus', 'debug')
@@ -180,16 +193,11 @@
let fileInput = $state<HTMLInputElement | null>(null); let fileInput = $state<HTMLInputElement | null>(null);
let uploading = $state(false); let uploading = $state(false);
interface ActionTarget { // Move/copy dialog state lives in the shared `useResourceActions`
id: string; // composable (`resActions.moveDialog.*`); this page only keeps the share
name: string; // dialog target, which the composable doesn't own.
kind: ItemType;
}
let moveOpen = $state(false);
let moveMode = $state<'move' | 'copy'>('move');
let shareOpen = $state(false); let shareOpen = $state(false);
let actionTarget = $state<ActionTarget | null>(null); let actionTarget = $state<ActionTarget | null>(null);
let moveItems = $state<ActionTarget[] | null>(null);
// Favorite / shared state now lives inline on every `FileItem` / // Favorite / shared state now lives inline on every `FileItem` /
// `FolderItem` DTO (`is_favorite`, `is_shared` — see // `FolderItem` DTO (`is_favorite`, `is_shared` — see
@@ -200,16 +208,10 @@
// inside `orderedItems`. No more `SvelteSet` shadowing. // inside `orderedItems`. No more `SvelteSet` shadowing.
function openMove(kind: ItemType, id: string, name: string) { function openMove(kind: ItemType, id: string, name: string) {
actionTarget = { id, name, kind }; resActions.openMove({ id, name, kind });
moveItems = null;
moveMode = 'move';
moveOpen = true;
} }
function openCopy(kind: ItemType, id: string, name: string) { function openCopy(kind: ItemType, id: string, name: string) {
actionTarget = { id, name, kind }; resActions.openCopy({ id, name, kind });
moveItems = null;
moveMode = 'copy';
moveOpen = true;
} }
function openShare(kind: ItemType, id: string, name: string) { function openShare(kind: ItemType, id: string, name: string) {
actionTarget = { id, name, kind }; actionTarget = { id, name, kind };
@@ -389,9 +391,15 @@
* Fetch and append the next page. Invoked by ResourceList's * Fetch and append the next page. Invoked by ResourceList's
* IntersectionObserver when the bottom sentinel enters the viewport. * IntersectionObserver when the bottom sentinel enters the viewport.
* The `loadingMore` guard collapses a double-fire (the observer can * The `loadingMore` guard collapses a double-fire (the observer can
* tick twice on the same intersection edge). * tick twice on the same intersection edge). Mode-aware: appends to
* the search results while the filter bar is active, to the folder
* page otherwise.
*/ */
async function loadMore() { async function loadMore() {
if (searchActive) {
await loadMoreSearch();
return;
}
if (loadingMore || pageCursor === undefined) return; if (loadingMore || pageCursor === undefined) return;
loadingMore = true; loadingMore = true;
try { try {
@@ -437,6 +445,74 @@
} }
} }
// ── Filter / search mode ─────────────────────────────────────────────────
// While any SearchFilterBar dimension is active the listing switches from
// the folder page (`fetchFolderPage`) to a scoped search (`searchResources`
// with folder_id = currentId). The two data paths keep independent cursors
// and stale guards; entering/leaving the mode neutralizes the other path's
// in-flight response so a slow folder page can never clobber fresh search
// rows (and vice versa).
//
// The backend treats an absent/empty `query` as "match everything"
// (`SearchResourcesQuery.query` is `Option<String>`), so filter-only
// searches (type/size/date, no keyword) work; its Tantivy content index
// additionally requires ≥2 chars before it engages, so empty queries stay
// name/filter-driven.
let filter = $state<ResourceFilterState>(defaultFilterState());
const searchActive = $derived(isFilterActive(filter));
let searchItems = $state<Array<FileItem | FolderItem>>([]);
let searchCursor = $state<string | undefined>(undefined);
let searchSeq = 0;
let searchAbort: AbortController | null = null;
async function runSearch(reset: boolean = true) {
const folderId = currentId;
if (!searchActive || !folderId) return;
error = null;
const seq = ++searchSeq;
searchAbort?.abort();
const ctl = new AbortController();
searchAbort = ctl;
loading = true;
const activeAtStart = searchActive;
try {
// The search wire has no `type` order (that's a client-side group-by)
// and calls modified time `updated_at` — map both before sending.
// Relevance is meaningless with a (possibly empty) filter query, so
// the current sort field is always sent instead.
const sortBy: SortBy =
sortField === 'type' ? 'name' : sortField === 'modified_at' ? 'updated_at' : sortField;
const res = await searchResources(filter.query.trim(), {
folderId,
...filterToSearchOptions(filter),
sortBy,
reverse: reversed,
limit: 50,
cursor: reset ? undefined : searchCursor,
signal: ctl.signal
});
if (seq !== searchSeq || searchActive !== activeAtStart) return; // superseded
// Unwrap the search envelope: each hit's `resource` is already the
// shared FileItem | FolderItem shape ResourceList consumes.
const hits = res.items.map((it) => it.resource);
searchItems = reset ? hits : [...searchItems, ...hits];
searchCursor = res.next_cursor;
loading = false;
} catch (e) {
if (seq !== searchSeq || searchActive !== activeAtStart) return;
loading = false;
if ((e as Error)?.name !== 'AbortError') error = errorMessage(e);
} finally {
if (searchAbort === ctl) searchAbort = null;
}
}
/** Append the next search page — the search-mode twin of `loadMore()`. */
async function loadMoreSearch() {
if (searchCursor === undefined) return;
await runSearch(false);
}
// ── Live folder updates (message bus) ──────────────────────────── // ── Live folder updates (message bus) ────────────────────────────
// Subscribe to `folder:{currentId}` and refresh when THIS session's // Subscribe to `folder:{currentId}` and refresh when THIS session's
// tabs, another tab of the same user, or another user with a share // tabs, another tab of the same user, or another user with a share
@@ -465,7 +541,11 @@
// event upload without feeling laggy. // event upload without feeling laggy.
setTimeout(() => { setTimeout(() => {
reloadScheduled = false; reloadScheduled = false;
void reload(); // In search mode re-run the SEARCH, not the folder page: a
// recursive filter covers subfolders, and a mutation in any of
// them (or of a matched row itself) can invalidate the results.
if (searchActive) void runSearch(true);
else void reload();
}, 100); }, 100);
} }
useFolderTopic(() => currentId, { useFolderTopic(() => currentId, {
@@ -670,24 +750,6 @@
} }
} }
/** Map `fn` over `items` with at most `limit` concurrent calls, preserving order. */
async function mapLimit<T, R>(
items: T[],
limit: number,
fn: (item: T) => Promise<R>
): Promise<R[]> {
const out = new Array<R>(items.length);
let next = 0;
const worker = async () => {
while (next < items.length) {
const i = next++;
out[i] = await fn(items[i]);
}
};
await Promise.all(Array.from({ length: Math.min(limit, items.length) }, worker));
return out;
}
/** Split items into the readable ones and the unreadable (FIFO/socket/…) ones. */ /** Split items into the readable ones and the unreadable (FIFO/socket/…) ones. */
async function partitionReadable<T>( async function partitionReadable<T>(
items: T[], items: T[],
@@ -1117,7 +1179,11 @@
// viewer-state changes, so a user-initiated close can't be re-opened here. // viewer-state changes, so a user-initiated close can't be re-opened here.
$effect(() => { $effect(() => {
const fileId = page.url.searchParams.get('file'); const fileId = page.url.searchParams.get('file');
const files = listing.files; // Deep links must also resolve while the filter bar is active — the
// hit may only exist in the search results, not the folder page.
const files = searchActive
? searchItems.filter((it): it is FileItem => isFile(it))
: listing.files;
untrack(() => { untrack(() => {
if (!fileId) { if (!fileId) {
if (viewerOpen) viewerOpen = false; if (viewerOpen) viewerOpen = false;
@@ -1167,140 +1233,35 @@
selected.clear(); selected.clear();
} }
/** // Shared batch actions (favorite / download / delete / move / copy),
* Download the whole selection as a single zip via POST /api/batch/download — // extracted so this page and the /search results page share one
* folders are included (the old per-item loop silently skipped them). A lone // implementation. `getItems` switches with the view mode: batch
* file still streams directly so it keeps its original name/extension. // operations act on search hits while the filter is active, on the
*/ // folder listing otherwise. (`orderedItems` rather than `rlItems` —
/** Name for a server-zipped multi-item archive (matches the legacy format). */ // hidden dotfiles can never be selected, and the raw array keeps the
function batchZipName(): string { // lone-file download name lookup working.)
const stamp = new Date().toISOString().replace('T', ' ').replace(/\..*/, '').replace(/:/g, '-'); const resActions = useResourceActions({
return `oxicloud ${stamp}.zip`; getItems: () => (searchActive ? searchItems : orderedItems),
} getSelected: () => selected,
clearSelection,
async function batchDownload() { onChanged: () => (searchActive ? runSearch(true) : reload()),
const fileIds: string[] = []; afterDelete: () => void session.refresh()
const folderIds: string[] = []; });
// One O(M) pass over the listing instead of an O(N·M) `some` per id.
const folderIdSet = new Set(listing.folders.map((f) => f.id));
const fileIdSet = new Set(listing.files.map((f) => f.id));
for (const id of selected) {
if (folderIdSet.has(id)) folderIds.push(id);
else if (fileIdSet.has(id)) fileIds.push(id);
}
if (fileIds.length === 0 && folderIds.length === 0) return;
// Single file, no folders → direct download (preserves the real name).
if (fileIds.length === 1 && folderIds.length === 0) {
const file = listing.files.find((f) => f.id === fileIds[0]);
if (file) {
const a = document.createElement('a');
a.href = fileDownloadUrl(file.id);
a.download = file.name;
document.body.appendChild(a);
a.click();
a.remove();
}
return;
}
const zipName = batchZipName();
try {
const res = await apiFetch('/api/batch/download', {
method: 'POST',
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json', ...getCsrfHeaders() },
body: JSON.stringify({ file_ids: fileIds, folder_ids: folderIds })
});
if (!res.ok) throw new Error(`Server returned ${res.status}`);
const blob = await res.blob();
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = zipName;
document.body.appendChild(a);
a.click();
a.remove();
URL.revokeObjectURL(url);
} catch (e) {
errorToast(e);
}
}
/** Batch add the selection to favorites — single /api/favorites/batch call. */
async function batchFavorites() {
// Build an id → item index so the "already favorite" filter is
// O(1) per selection member instead of an O(N·M) scan. Reused
// after success to flip `is_favorite` in place on each row.
// eslint-disable-next-line svelte/prefer-svelte-reactivity -- ephemeral local index, discarded before any reactive read
const byId = new Map<string, FileItem | FolderItem>();
for (const it of orderedItems) byId.set(it.id, it);
const items = selectionTargets().filter((it) => !(byId.get(it.id)?.is_favorite ?? false));
if (items.length === 0) {
ui.notify(t('files.already_favorites', 'All selected items are already favorites'), 'info');
clearSelection();
return;
}
try {
const res = await apiFetch('/api/favorites/batch', {
method: 'POST',
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json', ...getCsrfHeaders() },
body: JSON.stringify({
items: items.map((it) => ({ item_id: it.id, item_type: it.kind }))
})
});
if (!res.ok) throw new Error(`Server returned ${res.status}`);
for (const it of items) {
const row = byId.get(it.id);
if (row) row.is_favorite = true;
}
ui.notify(t('files.added_favorites', 'Added to favorites'), 'success');
clearSelection();
} catch (e) {
errorToast(e);
}
}
function selectionTargets(): ActionTarget[] {
// One O(M) index build instead of an O(N·M) `find` per selected id.
// Folders win id collisions, matching the old folder-first probe.
// eslint-disable-next-line svelte/prefer-svelte-reactivity -- ephemeral local index, discarded before any reactive read
const byId = new Map<string, ActionTarget>();
for (const f of listing.files) byId.set(f.id, { id: f.id, name: f.name, kind: 'file' });
for (const f of listing.folders) byId.set(f.id, { id: f.id, name: f.name, kind: 'folder' });
return [...selected]
.map((id) => byId.get(id) ?? null)
.filter((x): x is ActionTarget => x !== null);
}
function batchMove() {
const items = selectionTargets();
if (items.length) {
moveItems = items;
moveMode = 'move';
moveOpen = true;
}
}
function batchCopy() {
const items = selectionTargets();
if (items.length) {
moveItems = items;
moveMode = 'copy';
moveOpen = true;
}
}
function onKeydown(e: KeyboardEvent) { function onKeydown(e: KeyboardEvent) {
const tag = (e.target as HTMLElement)?.tagName; const tag = (e.target as HTMLElement)?.tagName;
if (tag === 'INPUT' || tag === 'TEXTAREA' || tag === 'SELECT') return; if (tag === 'INPUT' || tag === 'TEXTAREA' || tag === 'SELECT') return;
// Escape priority: selection first, then an active filter. The filter
// input handles its own Escape (clears the keyword, stopPropagation)
// so reaching here means the focus is outside the search bar.
if (e.key === 'Escape' && selected.size) { if (e.key === 'Escape' && selected.size) {
clearSelection(); clearSelection();
} else if (e.key === 'Escape' && searchActive) {
clearFilterState(filter);
} else if (e.key === 'Delete' && selected.size) { } else if (e.key === 'Delete' && selected.size) {
// Delete only — Backspace was dropped: it triggered accidental deletes. // Delete only — Backspace was dropped: it triggered accidental deletes.
e.preventDefault(); e.preventDefault();
void batchDelete(); void resActions.batchDelete();
} }
// Ctrl+A "select all" moved to the list-header checkbox owned by // Ctrl+A "select all" moved to the list-header checkbox owned by
// ResourceList — the row-level selection UX now lives entirely // ResourceList — the row-level selection UX now lives entirely
@@ -1309,33 +1270,6 @@
// gestures that reference the local `selected` mirror. // gestures that reference the local `selected` mirror.
} }
async function batchDelete() {
const ids = [...selected];
const ok = await confirmDialog({
title: t('files.batch_delete', 'Delete selected'),
message: t('files.confirm_batch_delete', { n: ids.length }, 'Move {{n}} items to trash?'),
confirmText: t('common.delete', 'Delete'),
danger: true
});
if (!ok) return;
// Bounded fan-out instead of a serial await per item: 100 deletes at
// ~30 ms RTT collapse from ~3 s of waterfall to a few round-trip
// windows. Failures toast individually and the rest still proceed,
// exactly like the old serial loop.
const folderIdSet = new Set(listing.folders.map((f) => f.id));
await mapLimit(ids, 6, async (id) => {
try {
if (folderIdSet.has(id)) await deleteFolder(id);
else await deleteFile(id);
} catch (e) {
errorToast(e);
}
});
clearSelection();
await reload();
void session.refresh();
}
// ── Drag-to-move ───────────────────────────────────────────────────────── // ── Drag-to-move ─────────────────────────────────────────────────────────
const DRAG_TYPE = 'application/x-oxi-item'; const DRAG_TYPE = 'application/x-oxi-item';
let dropFolderId = $state<string | null>(null); let dropFolderId = $state<string | null>(null);
@@ -1380,7 +1314,7 @@
*/ */
function onItemDragStart(e: DragEvent, kind: ItemType, id: string, name: string) { function onItemDragStart(e: DragEvent, kind: ItemType, id: string, name: string) {
const items: ActionTarget[] = const items: ActionTarget[] =
selected.has(id) && selected.size > 1 ? selectionTargets() : [{ id, name, kind }]; selected.has(id) && selected.size > 1 ? resActions.selectionTargets() : [{ id, name, kind }];
e.dataTransfer?.setData(DRAG_TYPE, JSON.stringify(items)); e.dataTransfer?.setData(DRAG_TYPE, JSON.stringify(items));
if (e.dataTransfer) { if (e.dataTransfer) {
// `copyMove` advertises both operations; the drop-target's // `copyMove` advertises both operations; the drop-target's
@@ -1575,7 +1509,7 @@
$effect(() => { $effect(() => {
if (viewerOpen) void fileViewer.load(); if (viewerOpen) void fileViewer.load();
if (wopiOpen) void wopiEditor.load(); if (wopiOpen) void wopiEditor.load();
if (moveOpen) void moveDialog.load(); if (resActions.moveDialog.open) void moveDialog.load();
if (shareOpen) void shareDialog.load(); if (shareOpen) void shareDialog.load();
}); });
// Editability of the current context-menu target file, resolved async. // Editability of the current context-menu target file, resolved async.
@@ -1798,6 +1732,10 @@
// user has active. First-appearance bucketing in // user has active. First-appearance bucketing in
// `buildResourceSections` keys off the item order in the input list. // `buildResourceSections` keys off the item order in the input list.
const rlItems = $derived.by<Array<FileItem | FolderItem>>(() => { const rlItems = $derived.by<Array<FileItem | FolderItem>>(() => {
// Search mode: the rows are the scoped search hits; the swimlane
// hoist never applies there (it's cleared on mode entry, see the
// filter effect) so the dotfile filter passes straight through.
if (searchActive) return filterDotfiles(searchItems, preferences.hideDotfiles);
const filtered = filterDotfiles(orderedItems, preferences.hideDotfiles); const filtered = filterDotfiles(orderedItems, preferences.hideDotfiles);
if (newlyAdded.size === 0) return filtered; if (newlyAdded.size === 0) return filtered;
const hoisted: Array<FileItem | FolderItem> = []; const hoisted: Array<FileItem | FolderItem> = [];
@@ -1809,6 +1747,23 @@
return [...hoisted, ...rest]; return [...hoisted, ...rest];
}); });
// ── Folder content stat ──────────────────────────────────────────────────
// Counts of what the listing actually renders (post-dotfile-filter, i.e.
// exactly the rows on screen), shown next to the breadcrumb. Listing is
// cursor-paginated, so while more pages exist (`pageCursor` defined) the
// numbers are partial — a "+" suffix says so instead of claiming exactness
// for a folder larger than one page.
const folderStat = $derived.by(() => {
const files = rlItems.filter(isFile).length;
// Search mode shows a flat result count instead of the folders/files
// split (the recursive result set isn't "this folder's content");
// partial/+ semantics carry over via the active mode's cursor.
if (searchActive) {
return { folders: 0, files: rlItems.length, partial: searchCursor !== undefined };
}
return { folders: rlItems.length - files, files, partial: pageCursor !== undefined };
});
// Group-by state (bound to <ResourceList>). Kept as a `string` prop // Group-by state (bound to <ResourceList>). Kept as a `string` prop
// value; the current `sortField` mirrors from the picked group's // value; the current `sortField` mirrors from the picked group's
// `orderBy` so a group-by change also drives the sort. // `orderBy` so a group-by change also drives the sort.
@@ -1965,10 +1920,37 @@
// you just added"; carrying it across folders would surface // you just added"; carrying it across folders would surface
// stale ids that don't belong to the new listing. // stale ids that don't belong to the new listing.
newlyAdded.clear(); newlyAdded.clear();
// Always re-load the folder page: even in search mode `load()`
// resolves the canonical folder id + breadcrumbs that the
// scoped search below is anchored to.
void load(true); void load(true);
}); });
}); });
// Scoped-search driver. Re-runs the search from page 1 whenever its
// inputs change: any filter dimension, the folder it's scoped to
// (`currentId`, resolved by `load()` above), or the sort dimension.
// Inactive (plain folder listing) is the no-op fast path. `filter` is
// a `$state` proxy — the field reads are what register the deps.
$effect(() => {
void filter.query;
void filter.type;
void filter.size;
void filter.date;
void filter.recursive;
void currentId;
void sortField;
void reversed;
const active = searchActive;
untrack(() => {
if (!active) return;
// Mode entry / re-run → drop the swimlane so its hoisting
// never fights the search ordering.
newlyAdded.clear();
void runSearch(true);
});
});
// The command palette's "Upload files" action navigates here then dispatches // The command palette's "Upload files" action navigates here then dispatches
// this event so the hidden file picker opens (the input lives on this page). // this event so the hidden file picker opens (the input lives on this page).
$effect(() => { $effect(() => {
@@ -2001,6 +1983,14 @@
<ReadOnlyBanner driveName={currentDrive.name} /> <ReadOnlyBanner driveName={currentDrive.name} />
{/if} {/if}
<!-- Fuzzy filter bar: scoped keyword + type/size/date presets over the
current folder (recursive toggle inside). While any dimension is
active the listing below switches from the folder page to the
search results; clearing it returns to the plain folder view. -->
<div class="files-filter-row">
<SearchFilterBar bind:value={filter} />
</div>
<!-- Hidden upload inputs stay mounted even while the batch bar is shown. <!-- Hidden upload inputs stay mounted even while the batch bar is shown.
Kept OUTSIDE ResourceList so the split-button dropdown in the Kept OUTSIDE ResourceList so the split-button dropdown in the
`actions` snippet can click() them without ResourceList's internal `actions` snippet can click() them without ResourceList's internal
@@ -2026,16 +2016,20 @@
<ResourceList <ResourceList
title={t('nav.files', 'Files')} title={t('nav.files', 'Files')}
items={rlItems} items={rlItems}
emptyText={hiddenCount > 0 emptyText={searchActive
? t('files.empty_hidden_title', { n: hiddenCount }, '{{n}} hidden item(s) in this folder') ? t('search.no_results', 'No results found for this search')
: t('files.empty_title', 'This folder is empty')} : hiddenCount > 0
emptyHint={hiddenCount > 0 ? t('files.empty_hidden_title', { n: hiddenCount }, '{{n}} hidden item(s) in this folder')
? t( : t('files.empty_title', 'This folder is empty')}
'files.empty_hidden_hint', emptyHint={searchActive
"Files whose name starts with '.' are hidden. Toggle the setting to see them." ? t('search.prompt', 'Type a query in the search bar above.')
) : hiddenCount > 0
: t('files.empty_hint', 'Drop files here or use the Upload button to add files.')} ? t(
emptyIcon={hiddenCount > 0 ? 'eye-slash' : undefined} 'files.empty_hidden_hint',
"Files whose name starts with '.' are hidden. Toggle the setting to see them."
)
: t('files.empty_hint', 'Drop files here or use the Upload button to add files.')}
emptyIcon={searchActive ? 'search' : hiddenCount > 0 ? 'eye-slash' : undefined}
{loading} {loading}
error={error ?? undefined} error={error ?? undefined}
selectable selectable
@@ -2052,7 +2046,7 @@
groupBys={rlGroupBys} groupBys={rlGroupBys}
bind:groupBy bind:groupBy
bind:reversed bind:reversed
hasMore={pageCursor !== undefined} hasMore={searchActive ? searchCursor !== undefined : pageCursor !== undefined}
onloadmore={loadMore} onloadmore={loadMore}
onreload={(orderBy) => { onreload={(orderBy) => {
sortField = orderBy as SortField; sortField = orderBy as SortField;
@@ -2074,8 +2068,10 @@
<!-- Surfaces only when the folder isn't really empty — it's just <!-- Surfaces only when the folder isn't really empty — it's just
filtered because the user chose to hide dotfiles. Clicking filtered because the user chose to hide dotfiles. Clicking
flips the app-wide `preferences.hideDotfiles` back off, flips the app-wide `preferences.hideDotfiles` back off,
re-populating the list without a hunt through settings. --> re-populating the list without a hunt through settings.
{#if hiddenCount > 0} Suppressed in search mode: an empty result there is a real
"nothing matched", not a hidden-items artifact. -->
{#if !searchActive && hiddenCount > 0}
<button <button
class="btn btn-secondary" class="btn btn-secondary"
onclick={() => preferences.setHideDotfiles(false)} onclick={() => preferences.setHideDotfiles(false)}
@@ -2106,6 +2102,29 @@
onDrop={(target, e) => onCrumbDrop(e, target)} onDrop={(target, e) => onCrumbDrop(e, target)}
dragMime={DRAG_TYPE} dragMime={DRAG_TYPE}
/> />
{#if searchActive}
<!-- Filter mode: a flat result count replaces the folders/files
split — the recursive result set isn't "this folder's
content". "+" keeps the partial-pages meaning. -->
{#if rlItems.length > 0}
<span class="folder-stat" data-testid="files-folder-stat">
{t('filter.results_count', { n: rlItems.length }, '{{n}} results')}{folderStat.partial
? '+'
: ''}
</span>
{/if}
{:else if folderStat.folders + folderStat.files > 0}
<!-- Item count for the folder on screen. Sits in the same sticky
strip as the breadcrumb so it stays visible while scrolling.
"+" = more pages are still loading via infinite scroll. -->
<span class="folder-stat" data-testid="files-folder-stat">
{t(
'files.folder_stat',
{ folders: folderStat.folders, files: folderStat.files },
'{{folders}} folders · {{files}} files'
)}{folderStat.partial ? '+' : ''}
</span>
{/if}
{/snippet} {/snippet}
{#snippet actions()} {#snippet actions()}
@@ -2155,6 +2174,21 @@
<Icon name="folder-plus" class="icon-mr" /> <Icon name="folder-plus" class="icon-mr" />
<span>{t('actions.new_folder', 'New folder')}</span> <span>{t('actions.new_folder', 'New folder')}</span>
</button> </button>
<!-- Manual reload of the current folder: resets pagination to page 1
and refetches (listing accumulator + folder stat + dotfile
filter all recompute). Disabled while a load is already in
flight — the button is a convenience, not a hammer. -->
<button
class="btn btn-secondary"
data-testid="files-refresh-btn"
title={t('common.refresh', 'Refresh')}
aria-label={t('common.refresh', 'Refresh')}
disabled={loading}
onclick={() => void load(true)}
>
<Icon name="repeat" class="icon-mr" />
<span>{t('common.refresh', 'Refresh')}</span>
</button>
{/snippet} {/snippet}
{#snippet batchActions(_sel)} {#snippet batchActions(_sel)}
@@ -2162,7 +2196,7 @@
class="batch-btn" class="batch-btn"
title={t('files.add_favorites', 'Add to favorites')} title={t('files.add_favorites', 'Add to favorites')}
data-testid="files-batch-favorite-btn" data-testid="files-batch-favorite-btn"
onclick={() => void batchFavorites()} onclick={() => void resActions.batchFavorites()}
> >
<Icon name="star" /> <Icon name="star" />
<span>{t('files.add_favorites', 'Add to favorites')}</span> <span>{t('files.add_favorites', 'Add to favorites')}</span>
@@ -2171,7 +2205,7 @@
class="batch-btn" class="batch-btn"
title={t('files.move', 'Move')} title={t('files.move', 'Move')}
data-testid="files-batch-move-btn" data-testid="files-batch-move-btn"
onclick={batchMove} onclick={resActions.batchMove}
> >
<Icon name="arrows-alt" /> <Icon name="arrows-alt" />
<span>{t('files.move', 'Move')}</span> <span>{t('files.move', 'Move')}</span>
@@ -2180,7 +2214,7 @@
class="batch-btn" class="batch-btn"
title={t('files.copy', 'Copy')} title={t('files.copy', 'Copy')}
data-testid="files-batch-copy-btn" data-testid="files-batch-copy-btn"
onclick={batchCopy} onclick={resActions.batchCopy}
> >
<Icon name="copy" /> <Icon name="copy" />
<span>{t('files.copy', 'Copy')}</span> <span>{t('files.copy', 'Copy')}</span>
@@ -2189,7 +2223,7 @@
class="batch-btn" class="batch-btn"
title={t('common.download', 'Download')} title={t('common.download', 'Download')}
data-testid="files-batch-download-btn" data-testid="files-batch-download-btn"
onclick={() => void batchDownload()} onclick={() => void resActions.batchDownload()}
> >
<Icon name="download" /> <Icon name="download" />
<span>{t('common.download', 'Download')}</span> <span>{t('common.download', 'Download')}</span>
@@ -2198,7 +2232,7 @@
class="batch-btn batch-btn-danger" class="batch-btn batch-btn-danger"
title={t('common.delete', 'Delete')} title={t('common.delete', 'Delete')}
data-testid="files-batch-delete-btn" data-testid="files-batch-delete-btn"
onclick={batchDelete} onclick={() => void resActions.batchDelete()}
> >
<Icon name="trash" /> <Icon name="trash" />
<span>{t('common.delete', 'Delete')}</span> <span>{t('common.delete', 'Delete')}</span>
@@ -2210,14 +2244,11 @@
{#if moveDialog.component} {#if moveDialog.component}
{@const MoveDialog = moveDialog.component} {@const MoveDialog = moveDialog.component}
<MoveDialog <MoveDialog
bind:open={moveOpen} bind:open={resActions.moveDialog.open}
item={actionTarget} item={resActions.moveDialog.item}
items={moveItems} items={resActions.moveDialog.items}
mode={moveMode} mode={resActions.moveDialog.mode}
onmoved={() => { onmoved={resActions.handleMoved}
clearSelection();
void reload();
}}
/> />
{/if} {/if}
{#if shareDialog.component} {#if shareDialog.component}
@@ -2434,6 +2465,17 @@
z-index: 1000; z-index: 1000;
} }
/* Item count next to the breadcrumb (same sticky strip; `.rl-breadcrumb`
is already a flex row, so the span just flows beside the crumbs).
`flex-shrink: 0` keeps long crumb trails from squeezing the digits
into a vertical stack on narrow viewports — the crumbs wrap instead. */
.folder-stat {
flex-shrink: 0;
color: var(--color-text-muted);
font-size: 0.8125rem;
white-space: nowrap;
}
.ctx-menu { .ctx-menu {
position: fixed; position: fixed;
z-index: 1001; z-index: 1001;
@@ -2471,4 +2513,9 @@
rendered near-invisible here). Mirrors the user-menu logout red. */ rendered near-invisible here). Mirrors the user-menu logout red. */
color: var(--color-danger-alt); color: var(--color-danger-alt);
} }
.files-filter-row {
padding: 0 var(--space-2);
margin-bottom: var(--space-1);
}
</style> </style>
+85 -8
View File
@@ -27,14 +27,19 @@ vi.mock('$app/state', () => ({ page: pageState }));
vi.mock('$lib/stores/session.svelte', () => ({ session })); vi.mock('$lib/stores/session.svelte', () => ({ session }));
vi.mock('$lib/stores/ui.svelte', () => ({ ui })); vi.mock('$lib/stores/ui.svelte', () => ({ ui }));
vi.mock('$lib/stores/dialogs.svelte', () => ({ confirmDialog, promptDialog })); vi.mock('$lib/stores/dialogs.svelte', () => ({ confirmDialog, promptDialog }));
vi.mock('$lib/api/client', () => ({ apiFetch: vi.fn() })); vi.mock('$lib/api/client', () => ({ apiFetch: vi.fn(), apiJson: vi.fn() }));
vi.mock('$lib/api/csrf', () => ({ getCsrfHeaders: () => ({}) })); vi.mock('$lib/api/csrf', () => ({ getCsrfHeaders: () => ({}) }));
vi.mock('$lib/api/endpoints/search', () => ({ searchResources: vi.fn() }));
vi.mock('$lib/api/endpoints/deltaUpload', () => ({ vi.mock('$lib/api/endpoints/deltaUpload', () => ({
instantUploadOwned: vi.fn(), instantUploadOwned: vi.fn(),
resolveOwnedHashes: vi.fn(), resolveOwnedHashes: vi.fn(),
tryDeltaUpload: vi.fn() tryDeltaUpload: vi.fn()
})); }));
vi.mock('$lib/api/endpoints/favorites', () => ({ addFavorite: vi.fn(), removeFavorite: vi.fn() })); vi.mock('$lib/api/endpoints/favorites', () => ({
addFavorite: vi.fn(),
removeFavorite: vi.fn(),
addFavoritesBatch: vi.fn()
}));
vi.mock('$lib/api/endpoints/wopi', () => ({ vi.mock('$lib/api/endpoints/wopi', () => ({
canEditWithWopi: () => false, canEditWithWopi: () => false,
getEditorUrlWithFallback: vi.fn() getEditorUrlWithFallback: vi.fn()
@@ -77,7 +82,8 @@ vi.mock('$lib/api/endpoints/folders', () => ({
import { fetchFolderPage, createFolder, deleteFolder } from '$lib/api/endpoints/folders'; import { fetchFolderPage, createFolder, deleteFolder } from '$lib/api/endpoints/folders';
import { deleteFile, uploadFileWithProgress } from '$lib/api/endpoints/files'; import { deleteFile, uploadFileWithProgress } from '$lib/api/endpoints/files';
import { resolveOwnedHashes, tryDeltaUpload } from '$lib/api/endpoints/deltaUpload'; import { resolveOwnedHashes, tryDeltaUpload } from '$lib/api/endpoints/deltaUpload';
import { apiFetch } from '$lib/api/client'; import { addFavoritesBatch } from '$lib/api/endpoints/favorites';
import { searchResources } from '$lib/api/endpoints/search';
import { files as filesStore } from '$lib/stores/files.svelte'; import { files as filesStore } from '$lib/stores/files.svelte';
import FilesPage from './[...path]/+page.svelte'; import FilesPage from './[...path]/+page.svelte';
@@ -196,6 +202,41 @@ it('loads the home folder listing on mount and renders its contents', async () =
await screen.findByTestId('files-new-folder-btn'); await screen.findByTestId('files-new-folder-btn');
}); });
it('shows the folder item count next to the breadcrumb', async () => {
withListing(); // 1 folder + 1 file, `nextCursor` undefined → last page
render(FilesPage);
const stat = await screen.findByTestId('files-folder-stat');
expect(stat.textContent).toContain('1 folders · 1 files');
// No "+" suffix — the listing is complete, the count is exact.
expect(stat.textContent!.trim().endsWith('+')).toBe(false);
});
it('marks the folder count as partial while more pages exist', async () => {
const folder = folderItem('sub1', 'Sub');
const file = fileItem('f1', 'hello.txt');
m(fetchFolderPage).mockResolvedValue({
items: [folder, file],
folders: [folder],
files: [file],
nextCursor: 'page-2'
});
render(FilesPage);
const stat = await screen.findByTestId('files-folder-stat');
// Counts reflect the pages loaded so far; the trailing "+" says more
// are on the way via infinite scroll instead of claiming exactness.
expect(stat.textContent).toContain('1 folders · 1 files');
expect(stat.textContent!.trim().endsWith('+')).toBe(true);
});
it('reloads the listing when the refresh button is clicked', async () => {
withListing();
render(FilesPage);
await screen.findByTestId('files-refresh-btn');
// Initial mount load = 1 call; the click resets pagination and refetches.
await fireEvent.click(screen.getByTestId('files-refresh-btn'));
await waitFor(() => expect(fetchFolderPage).toHaveBeenCalledTimes(2));
});
it('shows an error when the listing fails with no cache', async () => { it('shows an error when the listing fails with no cache', async () => {
m(fetchFolderPage).mockRejectedValue(Object.assign(new Error('nope'), { status: 500 })); m(fetchFolderPage).mockRejectedValue(Object.assign(new Error('nope'), { status: 500 }));
render(FilesPage); render(FilesPage);
@@ -233,14 +274,50 @@ it('batch-deletes the whole selection after confirmation', async () => {
it('batch-favorites the selection via the favorites batch endpoint', async () => { it('batch-favorites the selection via the favorites batch endpoint', async () => {
withListing(); withListing();
m(apiFetch).mockResolvedValue({ ok: true }); m(addFavoritesBatch).mockResolvedValue(undefined);
render(FilesPage); render(FilesPage);
await fireEvent.click(await screen.findByTestId('resource-list-select-all-checkbox')); await fireEvent.click(await screen.findByTestId('resource-list-select-all-checkbox'));
await fireEvent.click(await screen.findByTestId('files-batch-favorite-btn')); await fireEvent.click(await screen.findByTestId('files-batch-favorite-btn'));
await waitFor(() => await waitFor(() =>
expect(apiFetch).toHaveBeenCalledWith( expect(addFavoritesBatch).toHaveBeenCalledWith([
'/api/favorites/batch', { item_id: 'sub1', item_type: 'folder' },
expect.objectContaining({ method: 'POST' }) { item_id: 'f1', item_type: 'file' }
) ])
); );
}); });
it('runs a scoped recursive search when the filter keyword is set', async () => {
withListing();
m(searchResources).mockResolvedValue({ items: [], query_time_ms: 1 });
render(FilesPage);
await waitFor(() => expect(fetchFolderPage).toHaveBeenCalled());
vi.useFakeTimers();
await fireEvent.input(screen.getByTestId('filter-keyword-input'), {
target: { value: 'hello' }
});
await vi.advanceTimersByTimeAsync(400);
vi.useRealTimers();
expect(searchResources).toHaveBeenCalledWith(
'hello',
expect.objectContaining({ folderId: 'home', recursive: true })
);
});
it('shows the search result count while the filter is active', async () => {
withListing();
const hit = fileItem('s1', 'found.txt');
m(searchResources).mockResolvedValue({
items: [{ resource_type: 'file', resource: hit, meta: { score: 50 } }],
query_time_ms: 1
});
render(FilesPage);
await waitFor(() => expect(fetchFolderPage).toHaveBeenCalled());
vi.useFakeTimers();
await fireEvent.input(screen.getByTestId('filter-keyword-input'), {
target: { value: 'found' }
});
await vi.advanceTimersByTimeAsync(400);
vi.useRealTimers();
const stat = await screen.findByTestId('files-folder-stat');
expect(stat.textContent).toContain('1 results');
});
+30 -1
View File
@@ -287,7 +287,29 @@
</form> </form>
{:else if view === 'file'} {:else if view === 'file'}
<div class="share__center"> <div class="share__center">
<Icon name="file" class="share__big-icon" /> {#if meta && mediaKind(meta.mime_type) === 'video'}
<!-- Inline player: Range-aware endpoint streams the video, so the
timeline seeks without downloading the whole file first.
lazyVideo defers the load, seeks a few frames in for a poster
and retries once on error (same behaviour as the folder grid). -->
<video
class="share__media"
data-testid="public-share-file-video"
use:lazyVideo={shareFileUrl(token, meta.item_id)}
controls
playsinline
></video>
{:else if meta && mediaKind(meta.mime_type) === 'image'}
<img
class="share__media"
data-testid="public-share-file-image"
src={shareFileUrl(token, meta.item_id)}
alt={meta.item_name}
use:imageRetry
/>
{:else}
<Icon name="file" class="share__big-icon" />
{/if}
<h1>{meta?.item_name}</h1> <h1>{meta?.item_name}</h1>
<a <a
class="share__btn" class="share__btn"
@@ -486,6 +508,13 @@
text-align: center; text-align: center;
} }
.share__media {
max-width: 100%;
max-height: min(70vh, 40rem);
border-radius: var(--radius-2xl);
border: 1px solid var(--color-border);
}
:global(.share__big-icon) { :global(.share__big-icon) {
font-size: 3rem; font-size: 3rem;
color: var(--color-text-muted); color: var(--color-text-muted);
+107 -67
View File
@@ -22,10 +22,18 @@
import type { FileItem, FolderItem, SearchResourceItem, SortBy } from '$lib/api/types'; import type { FileItem, FolderItem, SearchResourceItem, SortBy } from '$lib/api/types';
import { lazyComponent } from '$lib/composables/lazyComponent.svelte'; import { lazyComponent } from '$lib/composables/lazyComponent.svelte';
import { folderAccessCached, probeFolderAccess } from '$lib/utils/folderAccess'; import { folderAccessCached, probeFolderAccess } from '$lib/utils/folderAccess';
import { TYPE_EXT, dateBound, sizeBounds } from '$lib/utils/searchFilters';
import { SvelteSet } from 'svelte/reactivity';
import { replaceSet } from '$lib/utils/sets';
import {
useResourceActions,
type ActionTarget
} from '$lib/composables/useResourceActions.svelte';
import Icon from '$lib/icons/Icon.svelte'; import Icon from '$lib/icons/Icon.svelte';
import { t } from '$lib/i18n/index.svelte'; import { t } from '$lib/i18n/index.svelte';
import { confirmDialog, promptDialog } from '$lib/stores/dialogs.svelte'; import { confirmDialog, promptDialog } from '$lib/stores/dialogs.svelte';
import { files as filesStore } from '$lib/stores/files.svelte'; import { files as filesStore } from '$lib/stores/files.svelte';
import { session } from '$lib/stores/session.svelte';
import { ui } from '$lib/stores/ui.svelte'; import { ui } from '$lib/stores/ui.svelte';
const query = $derived(page.url.searchParams.get('q') ?? ''); const query = $derived(page.url.searchParams.get('q') ?? '');
@@ -145,7 +153,9 @@
void goto(target, { replaceState: true, keepFocus: true, noScroll: true }); void goto(target, { replaceState: true, keepFocus: true, noScroll: true });
} }
// Filters // Filters — the preset vocabularies and their SearchOptions mapping live
// in the shared `searchFilters` util (also consumed by the files page's
// filter bar); only the i18n label lists stay local since they need `t()`.
type TypeKey = 'all' | 'image' | 'video' | 'document' | 'audio' | 'archive'; type TypeKey = 'all' | 'image' | 'video' | 'document' | 'audio' | 'archive';
type SizeKey = 'all' | 'small' | 'medium' | 'large'; type SizeKey = 'all' | 'small' | 'medium' | 'large';
type DateKey = 'all' | 'day' | 'week' | 'month' | 'year'; type DateKey = 'all' | 'day' | 'week' | 'month' | 'year';
@@ -153,26 +163,6 @@
let sizeFilter = $state<SizeKey>('all'); let sizeFilter = $state<SizeKey>('all');
let dateFilter = $state<DateKey>('all'); let dateFilter = $state<DateKey>('all');
const TYPE_EXT: Record<Exclude<TypeKey, 'all'>, string[]> = {
image: ['jpg', 'jpeg', 'png', 'gif', 'webp', 'svg', 'bmp', 'heic', 'avif', 'tiff'],
video: ['mp4', 'mov', 'mkv', 'avi', 'webm', 'm4v', 'wmv', 'flv'],
document: [
'pdf',
'doc',
'docx',
'xls',
'xlsx',
'ppt',
'pptx',
'txt',
'md',
'odt',
'rtf',
'csv'
],
audio: ['mp3', 'wav', 'flac', 'aac', 'ogg', 'm4a', 'opus'],
archive: ['zip', 'rar', '7z', 'tar', 'gz', 'bz2', 'xz']
};
const TYPES: { v: TypeKey; l: string }[] = [ const TYPES: { v: TypeKey; l: string }[] = [
{ v: 'all', l: t('search.type.all', 'All types') }, { v: 'all', l: t('search.type.all', 'All types') },
{ v: 'image', l: t('search.type.image', 'Images') }, { v: 'image', l: t('search.type.image', 'Images') },
@@ -195,36 +185,6 @@
{ v: 'year', l: t('search.date.year', 'Past year') } { v: 'year', l: t('search.date.year', 'Past year') }
]; ];
const MB = 1024 * 1024;
function sizeBounds(k: SizeKey): { minSize?: number; maxSize?: number } {
switch (k) {
case 'small':
return { maxSize: MB };
case 'medium':
return { minSize: MB, maxSize: 100 * MB };
case 'large':
return { minSize: 100 * MB };
default:
return {};
}
}
function dateBound(k: DateKey): number | undefined {
const day = 86400;
const now = Math.floor(Date.now() / 1000);
switch (k) {
case 'day':
return now - day;
case 'week':
return now - 7 * day;
case 'month':
return now - 30 * day;
case 'year':
return now - 365 * day;
default:
return undefined;
}
}
const hasFilters = $derived(typeFilter !== 'all' || sizeFilter !== 'all' || dateFilter !== 'all'); const hasFilters = $derived(typeFilter !== 'all' || sizeFilter !== 'all' || dateFilter !== 'all');
function clearFilters() { function clearFilters() {
typeFilter = 'all'; typeFilter = 'all';
@@ -294,6 +254,8 @@
scope === 'folder' && filesStore.section !== 'trash' scope === 'folder' && filesStore.section !== 'trash'
? (effectiveFolder ?? undefined) ? (effectiveFolder ?? undefined)
: undefined; : undefined;
// TYPE_EXT / sizeBounds / dateBound come from `$lib/utils/searchFilters`
// (shared with the files-page filter bar).
return { return {
recursive: true, recursive: true,
sortBy: orderByForGroup() as SortBy, sortBy: orderByForGroup() as SortBy,
@@ -383,16 +345,46 @@
// all reuse the same lazy dialogs. // all reuse the same lazy dialogs.
let viewerOpen = $state(false); let viewerOpen = $state(false);
let viewerFile = $state<FileItem | null>(null); let viewerFile = $state<FileItem | null>(null);
let moveOpen = $state(false);
let moveTarget = $state<{ id: string; name: string; kind: 'file' | 'folder' } | null>(null);
let shareOpen = $state(false); let shareOpen = $state(false);
let shareTarget = $state<{ id: string; name: string; kind: 'file' | 'folder' } | null>(null); let shareTarget = $state<{ id: string; name: string; kind: 'file' | 'folder' } | null>(null);
const fileViewer = lazyComponent(() => import('$lib/components/FileViewer.svelte')); const fileViewer = lazyComponent(() => import('$lib/components/FileViewer.svelte'));
const moveDialog = lazyComponent(() => import('$lib/components/MoveDialog.svelte')); const moveDialog = lazyComponent(() => import('$lib/components/MoveDialog.svelte'));
const shareDialog = lazyComponent(() => import('$lib/components/ShareDialog.svelte')); const shareDialog = lazyComponent(() => import('$lib/components/ShareDialog.svelte'));
// ── Multi-select + batch actions ─────────────────────────────────────
// Same wiring as the files page: ResourceList owns the row-level
// selection UX and mirrors it out via `onselectionchange`; the shared
// composable owns the batch favorite/download/delete/move/copy flows
// and the MoveDialog state. After any mutation the search re-runs —
// a move/delete can shift rows in or out of the current scope and
// filter set, so patching in place would go stale.
const selected = new SvelteSet<string>();
function clearSelection() {
selected.clear();
}
const resActions = useResourceActions({
getItems: () => items,
getSelected: () => selected,
clearSelection,
onChanged: () => run(query),
afterDelete: () => void session.refresh()
});
function onKeydown(e: KeyboardEvent) {
const tag = (e.target as HTMLElement)?.tagName;
if (tag === 'INPUT' || tag === 'TEXTAREA' || tag === 'SELECT') return;
if (e.key === 'Escape' && selected.size) {
clearSelection();
} else if (e.key === 'Delete' && selected.size) {
// Delete only — Backspace was dropped: it triggered accidental deletes.
e.preventDefault();
void resActions.batchDelete();
}
}
$effect(() => { $effect(() => {
if (viewerOpen) void fileViewer.load(); if (viewerOpen) void fileViewer.load();
if (moveOpen) void moveDialog.load(); if (resActions.moveDialog.open) void moveDialog.load();
if (shareOpen) void shareDialog.load(); if (shareOpen) void shareDialog.load();
}); });
@@ -446,8 +438,8 @@
} }
function openMoveDialog(item: FileItem | FolderItem) { function openMoveDialog(item: FileItem | FolderItem) {
moveTarget = { id: item.id, name: item.name, kind: kindOf(item) }; const target: ActionTarget = { id: item.id, name: item.name, kind: kindOf(item) };
moveOpen = true; resActions.openMove(target);
} }
function downloadItem(item: FileItem | FolderItem) { function downloadItem(item: FileItem | FolderItem) {
@@ -615,7 +607,7 @@
<svelte:head><title>{t('search.title', 'Search')} · OxiCloud</title></svelte:head> <svelte:head><title>{t('search.title', 'Search')} · OxiCloud</title></svelte:head>
<svelte:window ondragover={onWindowDragOver} ondrop={onWindowDrop} /> <svelte:window ondragover={onWindowDragOver} ondrop={onWindowDrop} onkeydown={onKeydown} />
{#if !query} {#if !query}
<EmptyState title={t('search.prompt', 'Type a query in the search bar above.')} /> <EmptyState title={t('search.prompt', 'Type a query in the search bar above.')} />
@@ -629,6 +621,9 @@
emptyText={t('search.no_results', 'No results found for this search')} emptyText={t('search.no_results', 'No results found for this search')}
hasMore={!!cursor} hasMore={!!cursor}
onloadmore={loadMore} onloadmore={loadMore}
selectable
shiftRangeSelect
onselectionchange={(ids) => replaceSet(selected, ids)}
showPath showPath
showViewToggle showViewToggle
onopen={open} onopen={open}
@@ -750,6 +745,56 @@
<FolderBreadcrumb folderId={scopeFolderId} /> <FolderBreadcrumb folderId={scopeFolderId} />
{/if} {/if}
{/snippet} {/snippet}
{#snippet batchActions(_sel)}
<!-- Same five batch buttons as the files page (shared
`useResourceActions` composable); testids are search-prefixed
so the two pages' tests stay unambiguous. -->
<button
class="batch-btn"
title={t('files.add_favorites', 'Add to favorites')}
data-testid="search-batch-favorite-btn"
onclick={() => void resActions.batchFavorites()}
>
<Icon name="star" />
<span>{t('files.add_favorites', 'Add to favorites')}</span>
</button>
<button
class="batch-btn"
title={t('files.move', 'Move')}
data-testid="search-batch-move-btn"
onclick={resActions.batchMove}
>
<Icon name="arrows-alt" />
<span>{t('files.move', 'Move')}</span>
</button>
<button
class="batch-btn"
title={t('files.copy', 'Copy')}
data-testid="search-batch-copy-btn"
onclick={resActions.batchCopy}
>
<Icon name="copy" />
<span>{t('files.copy', 'Copy')}</span>
</button>
<button
class="batch-btn"
title={t('common.download', 'Download')}
data-testid="search-batch-download-btn"
onclick={() => void resActions.batchDownload()}
>
<Icon name="download" />
<span>{t('common.download', 'Download')}</span>
</button>
<button
class="batch-btn batch-btn-danger"
title={t('common.delete', 'Delete')}
data-testid="search-batch-delete-btn"
onclick={() => void resActions.batchDelete()}
>
<Icon name="trash" />
<span>{t('common.delete', 'Delete')}</span>
</button>
{/snippet}
{#snippet itemActions(item)} {#snippet itemActions(item)}
<!-- <!--
Per-row "Open parent folder" quick-action — search results Per-row "Open parent folder" quick-action — search results
@@ -787,16 +832,11 @@
{#if moveDialog.component} {#if moveDialog.component}
{@const MoveDialog = moveDialog.component} {@const MoveDialog = moveDialog.component}
<MoveDialog <MoveDialog
bind:open={moveOpen} bind:open={resActions.moveDialog.open}
item={moveTarget} item={resActions.moveDialog.item}
onmoved={() => { items={resActions.moveDialog.items}
// A move can shift the row out of the current scope (`?in=<uuid>`) mode={resActions.moveDialog.mode}
// or into it, and the SQL name-match count may change. Reload onmoved={resActions.handleMoved}
// page 1 rather than trying to patch state in place — search
// state is already reactive on query/scope so a fresh `run()`
// is cheap and correct.
void run(query);
}}
/> />
{/if} {/if}
{#if shareDialog.component} {#if shareDialog.component}
+100 -4
View File
@@ -1,24 +1,108 @@
import { it, expect, vi, beforeEach } from 'vitest'; import { it, expect, vi, beforeEach } from 'vitest';
import { render, screen, waitFor } from '@testing-library/svelte'; import { render, screen, waitFor, fireEvent } from '@testing-library/svelte';
const { goto, pageState } = vi.hoisted(() => ({ const { goto, pageState, session, ui, confirmDialog, promptDialog } = vi.hoisted(() => ({
goto: vi.fn(), goto: vi.fn(),
pageState: { url: new URL('http://localhost/search?q=report') } pageState: { url: new URL('http://localhost/search?q=report') },
session: {
user: { id: 'me', username: 'admin', is_external: false },
isExternalUser: false,
loadHomeFolder: vi.fn(async () => 'home'),
refresh: vi.fn(async () => {})
},
ui: {
notify: vi.fn(),
startProgress: vi.fn(() => 1),
updateProgress: vi.fn(),
finishProgress: vi.fn()
},
confirmDialog: vi.fn(),
promptDialog: vi.fn()
})); }));
vi.mock('$app/navigation', () => ({ goto })); vi.mock('$app/navigation', () => ({ goto }));
vi.mock('$app/state', () => ({ page: pageState })); vi.mock('$app/state', () => ({ page: pageState }));
vi.mock('$lib/api/endpoints/search', () => ({ searchResources: vi.fn() })); vi.mock('$lib/api/endpoints/search', () => ({ searchResources: vi.fn() }));
vi.mock('$lib/api/endpoints/files', () => ({ fileInlineUrl: () => '/in' })); vi.mock('$lib/api/client', () => ({ apiFetch: vi.fn(), apiJson: vi.fn() }));
vi.mock('$lib/api/csrf', () => ({ getCsrfHeaders: () => ({}) }));
vi.mock('$lib/stores/session.svelte', () => ({ session }));
vi.mock('$lib/stores/ui.svelte', () => ({ ui }));
vi.mock('$lib/stores/dialogs.svelte', () => ({ confirmDialog, promptDialog }));
vi.mock('$lib/api/endpoints/files', () => ({
fileInlineUrl: () => '/in',
fileDownloadUrl: () => '/dl',
fileThumbnailUrl: () => '/thumb',
thumbSizeForView: () => 'preview' as const,
deleteFile: vi.fn(),
moveFile: vi.fn(),
renameFile: vi.fn()
}));
vi.mock('$lib/api/endpoints/folders', () => ({
deleteFolder: vi.fn(),
moveFolder: vi.fn(),
renameFolder: vi.fn()
}));
vi.mock('$lib/api/endpoints/favorites', () => ({
addFavorite: vi.fn(),
removeFavorite: vi.fn(),
addFavoritesBatch: vi.fn(),
dateBucket: () => 'bucket',
sizeBucket: () => 'bucket'
}));
import { searchResources } from '$lib/api/endpoints/search'; import { searchResources } from '$lib/api/endpoints/search';
import { deleteFile } from '$lib/api/endpoints/files';
import { deleteFolder } from '$lib/api/endpoints/folders';
import { files as filesStore } from '$lib/stores/files.svelte';
import SearchPage from './+page.svelte'; import SearchPage from './+page.svelte';
const m = (fn: unknown) => fn as ReturnType<typeof vi.fn>; const m = (fn: unknown) => fn as ReturnType<typeof vi.fn>;
const searchHit = () => ({
items: [
{
resource_type: 'file',
resource: {
id: 'f1',
name: 'a-report.txt',
mime_type: 'text/plain',
folder_id: 'p',
category: 'Document',
created_at: 0,
modified_at: 0,
size: 4,
created_by: 'me',
updated_by: 'me',
path: '/a-report.txt'
},
meta: { score: 50 }
},
{
resource_type: 'folder',
resource: {
id: 'd1',
name: 'reports',
parent_id: 'p',
category: 'Folder',
created_at: 0,
modified_at: 0,
is_root: false,
created_by: 'me',
updated_by: 'me',
path: '/reports'
},
meta: { score: 50 }
}
],
query_time_ms: 1,
total: 2
});
beforeEach(() => { beforeEach(() => {
vi.clearAllMocks(); vi.clearAllMocks();
pageState.url = new URL('http://localhost/search?q=report'); pageState.url = new URL('http://localhost/search?q=report');
m(searchResources).mockResolvedValue({ items: [], query_time_ms: 0, total: 0 }); m(searchResources).mockResolvedValue({ items: [], query_time_ms: 0, total: 0 });
// List view renders the select-all header + per-row checkboxes; grid hides them.
filesStore.viewMode = 'list';
}); });
it('runs a search from the q query parameter on mount', async () => { it('runs a search from the q query parameter on mount', async () => {
@@ -41,3 +125,15 @@ it('surfaces a search error', async () => {
await waitFor(() => expect(searchResources).toHaveBeenCalled()); await waitFor(() => expect(searchResources).toHaveBeenCalled());
await waitFor(() => expect(screen.getByText('search boom')).toBeTruthy()); await waitFor(() => expect(screen.getByText('search boom')).toBeTruthy());
}); });
it('batch-deletes the selected search results after confirmation', async () => {
m(searchResources).mockResolvedValue(searchHit());
confirmDialog.mockResolvedValue(true);
render(SearchPage);
await waitFor(() => expect(searchResources).toHaveBeenCalled());
await fireEvent.click(await screen.findByTestId('resource-list-select-all-checkbox'));
await fireEvent.click(await screen.findByTestId('search-batch-delete-btn'));
await waitFor(() => expect(deleteFile).toHaveBeenCalledWith('f1'));
await waitFor(() => expect(deleteFolder).toHaveBeenCalledWith('d1'));
await waitFor(() => expect(session.refresh).toHaveBeenCalled());
});
+8
View File
@@ -416,6 +416,14 @@
"shared_notificationSent": "تم إرسال الإشعار بنجاح", "shared_notificationSent": "تم إرسال الإشعار بنجاح",
"shared_notificationFailed": "فشل إرسال الإشعار" "shared_notificationFailed": "فشل إرسال الإشعار"
}, },
"filter": {
"placeholder": "ابحث في هذا المجلد والمجلدات الفرعية…",
"keyword": "الكلمة المفتاحية",
"advanced": "المرشحات",
"recursive": "تضمين المجلدات الفرعية",
"results_count": "{{n}} نتائج",
"clear_keyword": "مسح البحث"
},
"files": { "files": {
"name": "الاسم", "name": "الاسم",
"type": "النوع", "type": "النوع",
+8
View File
@@ -416,6 +416,14 @@
"shared_notificationSent": "Benachrichtigung erfolgreich gesendet", "shared_notificationSent": "Benachrichtigung erfolgreich gesendet",
"shared_notificationFailed": "Benachrichtigung konnte nicht gesendet werden" "shared_notificationFailed": "Benachrichtigung konnte nicht gesendet werden"
}, },
"filter": {
"placeholder": "Diesen Ordner und Unterordner durchsuchen…",
"keyword": "Suchbegriff",
"advanced": "Filter",
"recursive": "Unterordner einbeziehen",
"results_count": "{{n}} Ergebnisse",
"clear_keyword": "Suche löschen"
},
"files": { "files": {
"name": "Name", "name": "Name",
"type": "Typ", "type": "Typ",
+10
View File
@@ -451,11 +451,20 @@
"shared_notificationSent": "Notification sent successfully", "shared_notificationSent": "Notification sent successfully",
"shared_notificationFailed": "Failed to send notification" "shared_notificationFailed": "Failed to send notification"
}, },
"filter": {
"placeholder": "Search this folder and subfolders…",
"keyword": "Keyword",
"advanced": "Filters",
"recursive": "Include subfolders",
"results_count": "{{n}} results",
"clear_keyword": "Clear search"
},
"files": { "files": {
"name": "Name", "name": "Name",
"type": "Type", "type": "Type",
"size": "Size", "size": "Size",
"modified": "Modified", "modified": "Modified",
"folder_stat": "{{folders}} folders · {{files}} files",
"no_files": "No files in this folder", "no_files": "No files in this folder",
"empty_hint": "Upload files or create folders to get started", "empty_hint": "Upload files or create folders to get started",
"drop_to_upload": "Drop files here to upload", "drop_to_upload": "Drop files here to upload",
@@ -1645,6 +1654,7 @@
"toggle_theme": "Toggle theme" "toggle_theme": "Toggle theme"
}, },
"common": { "common": {
"refresh": "Refresh",
"add": "Add", "add": "Add",
"cancel": "Cancel", "cancel": "Cancel",
"clear": "Clear", "clear": "Clear",
+8
View File
@@ -416,6 +416,14 @@
"mit_license": "Licencia MIT", "mit_license": "Licencia MIT",
"title": "Menú de usuario" "title": "Menú de usuario"
}, },
"filter": {
"placeholder": "Buscar en esta carpeta y subcarpetas…",
"keyword": "Palabra clave",
"advanced": "Filtros",
"recursive": "Incluir subcarpetas",
"results_count": "{{n}} resultados",
"clear_keyword": "Borrar búsqueda"
},
"files": { "files": {
"name": "Nombre", "name": "Nombre",
"type": "Tipo", "type": "Tipo",
+8
View File
@@ -416,6 +416,14 @@
"shared_notificationSent": "آگاه‌سازی با موفقیت ارسال شد", "shared_notificationSent": "آگاه‌سازی با موفقیت ارسال شد",
"shared_notificationFailed": "ارسال آگاه‌سازی ناموفق بود" "shared_notificationFailed": "ارسال آگاه‌سازی ناموفق بود"
}, },
"filter": {
"placeholder": "جستجو در این پوشه و زیرپوشه‌ها…",
"keyword": "کلیدواژه",
"advanced": "فیلترها",
"recursive": "شامل زیرپوشه‌ها",
"results_count": "{{n}} نتیجه",
"clear_keyword": "پاک کردن جستجو"
},
"files": { "files": {
"name": "نام", "name": "نام",
"type": "نوع", "type": "نوع",
+8
View File
@@ -416,6 +416,14 @@
"shared_notificationSent": "Notification envoyée avec succès", "shared_notificationSent": "Notification envoyée avec succès",
"shared_notificationFailed": "Erreur lors de l'envoi de la notification" "shared_notificationFailed": "Erreur lors de l'envoi de la notification"
}, },
"filter": {
"placeholder": "Rechercher dans ce dossier et ses sous-dossiers…",
"keyword": "Mot-clé",
"advanced": "Filtres",
"recursive": "Inclure les sous-dossiers",
"results_count": "{{n}} résultats",
"clear_keyword": "Effacer la recherche"
},
"files": { "files": {
"name": "Nom", "name": "Nom",
"type": "Type", "type": "Type",
+8
View File
@@ -416,6 +416,14 @@
"shared_notificationSent": "सूचना सफलतापूर्वक भेजी गई", "shared_notificationSent": "सूचना सफलतापूर्वक भेजी गई",
"shared_notificationFailed": "सूचना भेजने में विफल" "shared_notificationFailed": "सूचना भेजने में विफल"
}, },
"filter": {
"placeholder": "इस फ़ोल्डर और सबफ़ोल्डर में खोजें…",
"keyword": "कीवर्ड",
"advanced": "फ़िल्टर",
"recursive": "सबफ़ोल्डर शामिल करें",
"results_count": "{{n}} परिणाम",
"clear_keyword": "खोज साफ़ करें"
},
"files": { "files": {
"name": "नाम", "name": "नाम",
"type": "प्रकार", "type": "प्रकार",
+8
View File
@@ -416,6 +416,14 @@
"shared_notificationSent": "Notifica inviata con successo", "shared_notificationSent": "Notifica inviata con successo",
"shared_notificationFailed": "Impossibile inviare la notifica" "shared_notificationFailed": "Impossibile inviare la notifica"
}, },
"filter": {
"placeholder": "Cerca in questa cartella e nelle sottocartelle…",
"keyword": "Parola chiave",
"advanced": "Filtri",
"recursive": "Includi sottocartelle",
"results_count": "{{n}} risultati",
"clear_keyword": "Cancella ricerca"
},
"files": { "files": {
"name": "Nome", "name": "Nome",
"type": "Tipo", "type": "Tipo",
+8
View File
@@ -416,6 +416,14 @@
"shared_notificationSent": "通知が正常に送信されました", "shared_notificationSent": "通知が正常に送信されました",
"shared_notificationFailed": "通知の送信に失敗しました" "shared_notificationFailed": "通知の送信に失敗しました"
}, },
"filter": {
"placeholder": "このフォルダとサブフォルダを検索…",
"keyword": "キーワード",
"advanced": "フィルター",
"recursive": "サブフォルダを含める",
"results_count": "{{n}} 件の結果",
"clear_keyword": "検索をクリア"
},
"files": { "files": {
"name": "名前", "name": "名前",
"type": "種類", "type": "種類",
+8
View File
@@ -416,6 +416,14 @@
"shared_notificationSent": "알림이 성공적으로 전송되었습니다", "shared_notificationSent": "알림이 성공적으로 전송되었습니다",
"shared_notificationFailed": "알림 전송에 실패했습니다" "shared_notificationFailed": "알림 전송에 실패했습니다"
}, },
"filter": {
"placeholder": "이 폴더와 하위 폴더 검색…",
"keyword": "키워드",
"advanced": "필터",
"recursive": "하위 폴더 포함",
"results_count": "{{n}}개 결과",
"clear_keyword": "검색 지우기"
},
"files": { "files": {
"name": "이름", "name": "이름",
"type": "유형", "type": "유형",
+8
View File
@@ -416,6 +416,14 @@
"shared_notificationSent": "Notificatie verzonden", "shared_notificationSent": "Notificatie verzonden",
"shared_notificationFailed": "Notificatie verzenden mislukt" "shared_notificationFailed": "Notificatie verzenden mislukt"
}, },
"filter": {
"placeholder": "Deze map en submappen doorzoeken…",
"keyword": "Trefwoord",
"advanced": "Filters",
"recursive": "Submappen opnemen",
"results_count": "{{n}} resultaten",
"clear_keyword": "Zoekopdracht wissen"
},
"files": { "files": {
"name": "Naam", "name": "Naam",
"type": "Type", "type": "Type",
+8
View File
@@ -416,6 +416,14 @@
"shared_notificationSent": "Powiadomienie wysłane pomyślnie", "shared_notificationSent": "Powiadomienie wysłane pomyślnie",
"shared_notificationFailed": "Nie udało się wysłać powiadomienia" "shared_notificationFailed": "Nie udało się wysłać powiadomienia"
}, },
"filter": {
"placeholder": "Szukaj w tym folderze i podfolderach…",
"keyword": "Słowo kluczowe",
"advanced": "Filtry",
"recursive": "Uwzględnij podfoldery",
"results_count": "Wyniki: {{n}}",
"clear_keyword": "Wyczyść wyszukiwanie"
},
"files": { "files": {
"name": "Nazwa", "name": "Nazwa",
"type": "Typ", "type": "Typ",
+8
View File
@@ -416,6 +416,14 @@
"shared_notificationSent": "Notificação enviada com sucesso", "shared_notificationSent": "Notificação enviada com sucesso",
"shared_notificationFailed": "Falha ao enviar a notificação" "shared_notificationFailed": "Falha ao enviar a notificação"
}, },
"filter": {
"placeholder": "Pesquisar nesta pasta e subpastas…",
"keyword": "Palavra-chave",
"advanced": "Filtros",
"recursive": "Incluir subpastas",
"results_count": "{{n}} resultados",
"clear_keyword": "Limpar pesquisa"
},
"files": { "files": {
"name": "Nome", "name": "Nome",
"type": "Tipo", "type": "Tipo",
+8
View File
@@ -416,6 +416,14 @@
"shared_notificationSent": "Уведомление успешно отправлено", "shared_notificationSent": "Уведомление успешно отправлено",
"shared_notificationFailed": "Не удалось отправить уведомление" "shared_notificationFailed": "Не удалось отправить уведомление"
}, },
"filter": {
"placeholder": "Поиск в этой папке и подпапках…",
"keyword": "Ключевое слово",
"advanced": "Фильтры",
"recursive": "Включая подпапки",
"results_count": "Результатов: {{n}}",
"clear_keyword": "Очистить поиск"
},
"files": { "files": {
"name": "Имя", "name": "Имя",
"type": "Тип", "type": "Тип",
+10
View File
@@ -416,11 +416,20 @@
"shared_typeFile": "檔案", "shared_typeFile": "檔案",
"shared_typeFolder": "資料夾" "shared_typeFolder": "資料夾"
}, },
"filter": {
"placeholder": "搜尋此資料夾及子資料夾…",
"keyword": "關鍵字",
"advanced": "篩選",
"recursive": "包含子資料夾",
"results_count": "{{n}} 個結果",
"clear_keyword": "清除搜尋"
},
"files": { "files": {
"name": "名稱", "name": "名稱",
"type": "型別", "type": "型別",
"size": "大小", "size": "大小",
"modified": "修改日期", "modified": "修改日期",
"folder_stat": "{{folders}} 個資料夾 · {{files}} 個檔案",
"no_files": "此資料夾中沒有檔案", "no_files": "此資料夾中沒有檔案",
"empty_hint": "上傳檔案或建立資料夾以開始使用", "empty_hint": "上傳檔案或建立資料夾以開始使用",
"drop_to_upload": "將檔案拖放到此處上傳", "drop_to_upload": "將檔案拖放到此處上傳",
@@ -1598,6 +1607,7 @@
"videos": "影片" "videos": "影片"
}, },
"common": { "common": {
"refresh": "重新整理",
"add": "新增", "add": "新增",
"cancel": "取消", "cancel": "取消",
"clear": "清除", "clear": "清除",
+10
View File
@@ -416,11 +416,20 @@
"shared_typeFile": "文件", "shared_typeFile": "文件",
"shared_typeFolder": "文件夹" "shared_typeFolder": "文件夹"
}, },
"filter": {
"placeholder": "搜索此文件夹及子文件夹…",
"keyword": "关键词",
"advanced": "筛选",
"recursive": "包含子文件夹",
"results_count": "{{n}} 个结果",
"clear_keyword": "清除搜索"
},
"files": { "files": {
"name": "名称", "name": "名称",
"type": "类型", "type": "类型",
"size": "大小", "size": "大小",
"modified": "修改日期", "modified": "修改日期",
"folder_stat": "{{folders}} 个文件夹 · {{files}} 个文件",
"no_files": "此文件夹中没有文件", "no_files": "此文件夹中没有文件",
"empty_hint": "上传文件或创建文件夹以开始使用", "empty_hint": "上传文件或创建文件夹以开始使用",
"drop_to_upload": "将文件拖放到此处上传", "drop_to_upload": "将文件拖放到此处上传",
@@ -1598,6 +1607,7 @@
"videos": "视频" "videos": "视频"
}, },
"common": { "common": {
"refresh": "刷新",
"add": "添加", "add": "添加",
"cancel": "取消", "cancel": "取消",
"clear": "清除", "clear": "清除",
+15
View File
@@ -15,6 +15,15 @@ Non-obvious rules that trip up new code. Terse on purpose.
- Any new endpoint that mints or consumes credentials/tokens must consult one of the `is_*_login_allowed()` helpers, not the raw allowlist. - Any new endpoint that mints or consumes credentials/tokens must consult one of the `is_*_login_allowed()` helpers, not the raw allowlist.
- Any new "policy-disabled" refusal must emit an `audit`-target line before returning — matches `auth.login_rejected`, `magic_link.redemption_rejected` conventions. - Any new "policy-disabled" refusal must emit an `audit`-target line before returning — matches `auth.login_rejected`, `magic_link.redemption_rejected` conventions.
## AuthZ enforcement points
- **The extractors are NOT a choke point.** Four paths authenticate without ever touching `AuthUser` / `CurrentUserId`: `middleware/admin.rs::require_authenticated` (re-parses the JWT from header *or* cookie itself), the three DAV handlers' hand-rolled `extract_user` (`webdav_handler.rs:154`, `caldav_handler.rs:421`, `carddav_handler.rs:173`), `POST /api/auth/refresh` (mounted outside `auth_middleware`, `main.rs:775`), and `GET /api/rt/ws` (self-auths from a raw Bearer and never reads `claims.role`, `rt_ws.rs:240-255`). A rule added to an extractor does not hold until it is added to these too.
- **Never hand-roll principal extraction.** `req.extensions().get::<Arc<CurrentUser>>()` inside a handler is exactly the anti-pattern above — it bypasses every `FromRequestParts` guard. Take the extractor, or call the shared assertion.
- **A method check is not an authorization check.** GETs that mint credentials or write exist: `GET /api/wopi/editor-url` returns a WOPI token usable for `POST /wopi/files/{id}/contents`; `GET /api/s/{token}` writes via `register_shared_link_access`; `GET /api/auth/device/verify` is an oracle on live device codes; `GET /api/batch/download` builds an arbitrary ZIP from a querystring. Never gate on verb alone.
- **An auth helper's `_ =>` arm must DENY.** Two fail-open gates exist and are bugs, not patterns to copy: `require_internal_user` (`middleware/user.rs:64-71`) admits the caller on *any* `get_user_flags` error, and `decide_live_role` (`:169-176`) resurrects the claim role on a transient DB error. The first is the only middleware guarding all three DAV surfaces.
- **Prefer deny-by-default over assert-in-handler.** A restriction enforced inside the extractor covers ~200 call sites with no edits; the same restriction as "handler takes an optional principal and asserts" is one forgotten call away from silently accepting. `OptionalUserId` (`middleware/auth.rs:85-98`) is the cautionary tale — it exists, it is dead code, and nothing ever used it.
- Anonymous-session direction (share links as a principal): `docs/plan/rationalize-publicshare.md`.
## Storage backend access ## Storage backend access
- **Read blob content through `Arc<DedupService>`.** It's the ONE canonical read abstraction — CDC-manifest-aware (`file.blob_hash` may reference a chunk manifest, not a blob), backend-agnostic (Local/S3/Azure), wrapper-transparent (encryption/retry/cache). Never take `Arc<dyn BlobStorageBackend>` directly in a service that reads content; you'll silently break on any file ≥ 64 KiB (`CDC_MIN_CHUNK`). Follow `thumbnail_service`, `audio_metadata_service`, `media_metadata_service`, `face_indexing_service`, `search_index::content_index_worker` as reference impls. - **Read blob content through `Arc<DedupService>`.** It's the ONE canonical read abstraction — CDC-manifest-aware (`file.blob_hash` may reference a chunk manifest, not a blob), backend-agnostic (Local/S3/Azure), wrapper-transparent (encryption/retry/cache). Never take `Arc<dyn BlobStorageBackend>` directly in a service that reads content; you'll silently break on any file ≥ 64 KiB (`CDC_MIN_CHUNK`). Follow `thumbnail_service`, `audio_metadata_service`, `media_metadata_service`, `face_indexing_service`, `search_index::content_index_worker` as reference impls.
@@ -30,3 +39,9 @@ Non-obvious rules that trip up new code. Terse on purpose.
- **After adding: `cargo run --bin generate-openapi`** to regenerate `resources/gen/openapi.json`, then `git diff resources/gen/openapi.json` — the new path + its request/response schemas must be present. Zero-diff means you missed the registration. - **After adding: `cargo run --bin generate-openapi`** to regenerate `resources/gen/openapi.json`, then `git diff resources/gen/openapi.json` — the new path + its request/response schemas must be present. Zero-diff means you missed the registration.
- Sanity check for the whole surface: `diff <(grep -oE 'path = "/api[^"]+"' src/interfaces/api/handlers/*.rs | grep -oE '/api[^"]+' | sort -u) <(jq -r '.paths | keys | .[]' resources/gen/openapi.json | sort -u)` — should always be empty. Non-empty diff = drift. - Sanity check for the whole surface: `diff <(grep -oE 'path = "/api[^"]+"' src/interfaces/api/handlers/*.rs | grep -oE '/api[^"]+' | sort -u) <(jq -r '.paths | keys | .[]' resources/gen/openapi.json | sort -u)` — should always be empty. Non-empty diff = drift.
- Handlers referenced by the `paths(...)` list MUST be `pub` (module-visible from the paths list). Private `async fn` compiles at the router mount but breaks the paths list with a visibility error — see `get_smtp_info`, `send_smtp_test`, `get_user_profile` for the retrofit. - Handlers referenced by the `paths(...)` list MUST be `pub` (module-visible from the paths list). Private `async fn` compiles at the router mount but breaks the paths list with a visibility error — see `get_smtp_info`, `send_smtp_test`, `get_user_profile` for the retrofit.
### Security / scope in the spec
- **`security(("bearerAuth" = []))` — the empty array is the SCOPES list**, not decoration. Every route currently declares the same thing, so the spec claims "a session is required" for `GET /api/version` and `PUT /api/admin/users/{id}/role` alike: true, and useless. If a route's gate differs from the default, declare it there.
- OpenAPI has **no field for a minimum role** — OAuth2 has no role concept, so the spec has nowhere to put one. Use a pseudo-scope (`["role:admin"]`) rather than a vendor extension no tooling renders.
- **Declaring is not enforcing.** utoipa's `security` wires nothing, so it drifts from the real gate silently. Any scope worth declaring is worth a test cross-checking it against the actual mount — otherwise the spec becomes a parallel description of the authorization boundary rather than a picture of it.
+11
View File
@@ -16,6 +16,15 @@ pub struct ShareDto {
pub created_at: u64, pub created_at: u64,
pub created_by: String, pub created_by: String,
pub access_count: u64, pub access_count: u64,
/// File shares only: the shared file's MIME type, resolved at read time
/// so anonymous viewers can render an inline media preview (video player
/// / image) instead of a bare download button. Absent for folder shares
/// and whenever the file lookup fails (display-only enrichment).
#[serde(skip_serializing_if = "Option::is_none")]
pub mime_type: Option<String>,
/// File shares only: the shared file's size in bytes (see `mime_type`).
#[serde(skip_serializing_if = "Option::is_none")]
pub size: Option<u64>,
} }
#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)] #[derive(Debug, Clone, Serialize, Deserialize, ToSchema)]
@@ -49,6 +58,8 @@ impl ShareDto {
created_at: share.created_at(), created_at: share.created_at(),
created_by: share.created_by().to_string(), created_by: share.created_by().to_string(),
access_count: share.access_count(), access_count: share.access_count(),
mime_type: None,
size: None,
} }
} }
} }
@@ -3425,34 +3425,37 @@ impl AuthApplicationService {
&self, &self,
dto: crate::application::dtos::settings_dto::AdminCreateUserDto, dto: crate::application::dtos::settings_dto::AdminCreateUserDto,
) -> Result<FullUserDto, DomainError> { ) -> Result<FullUserDto, DomainError> {
// Validate username length // Normalise the username up-front — trim + lowercase — and use
if dto.username.len() < 3 || dto.username.len() > 254 { // the canonical form for every downstream check + generated
return Err(DomainError::new( // value below. `User::new` also normalises internally, but the
ErrorKind::InvalidInput, // placeholder-email fallback and the duplicate-check error
"User", // message live above that call, so they'd otherwise capture the
"Username must be between 3 and 254 characters".to_string(), // raw wire input (e.g. `UpperCase@oxicloud.local`).
)); // See docs/plan/username-lowercase.md § Design decision 5.
} let username = User::validate_username(&dto.username).map_err(|e| {
DomainError::new(ErrorKind::InvalidInput, "User", format!("Username: {e}"))
})?;
// Check for duplicate username // Check for duplicate username
if self if self
.user_storage .user_storage
.get_user_by_username(&dto.username) .get_user_by_username(&username)
.await .await
.is_ok() .is_ok()
{ {
return Err(DomainError::new( return Err(DomainError::new(
ErrorKind::AlreadyExists, ErrorKind::AlreadyExists,
"User", "User",
format!("User '{}' already exists", dto.username), format!("User '{username}' already exists"),
)); ));
} }
// Email: use provided or generate placeholder // Email: use provided or generate placeholder from the
// canonical (lowercase) username.
let email = dto let email = dto
.email .email
.filter(|e| !e.trim().is_empty()) .filter(|e| !e.trim().is_empty())
.unwrap_or_else(|| format!("{}@oxicloud.local", dto.username)); .unwrap_or_else(|| format!("{username}@oxicloud.local"));
// Check email uniqueness // Check email uniqueness
if self.user_storage.get_user_by_email(&email).await.is_ok() { if self.user_storage.get_user_by_email(&email).await.is_ok() {
@@ -3519,7 +3522,7 @@ impl AuthApplicationService {
let user = if is_external { let user = if is_external {
User::new( User::new(
email, email,
Some(dto.username.clone()), Some(username.clone()),
Some(password_hash), Some(password_hash),
None, // federation_kind: admin-created external, no federation link yet None, // federation_kind: admin-created external, no federation link yet
None, // federation_issuer None, // federation_issuer
@@ -3531,7 +3534,7 @@ impl AuthApplicationService {
} else { } else {
User::new( User::new(
email, email,
Some(dto.username.clone()), Some(username.clone()),
Some(password_hash), Some(password_hash),
None, // federation_kind: admin-created local user None, // federation_kind: admin-created local user
None, // federation_issuer None, // federation_issuer
@@ -3750,8 +3753,33 @@ impl AuthApplicationService {
Ok(()) Ok(())
} }
/// Activate or deactivate a user (admin only) /// Activate or deactivate a user (admin only).
///
/// **Reactivation collision handling** — when a deactivated account
/// holds a mixed-case username from before the lowercase-usernames
/// migration (its row was skipped by that migration precisely
/// because it was deactivated), reactivating it can produce a
/// username collision if `LOWER(other.username) == LOWER(this.username)`
/// for an active row. We resolve the collision by:
///
/// 1. Re-normalising via `User::set_username` (returns the
/// canonical lowercase form on success).
/// 2. If the canonical form is already taken by another active
/// row, probe `<lower>-2`, `-3`, … via the shared
/// `find_free_username_suffix` helper.
/// 3. Persist the resolved name BEFORE flipping `active = true`
/// so no time window has two-active-users with the same
/// lowercase form.
///
/// See `docs/plan/username-lowercase.md § Design decision 7`.
/// Without this, un-soft-deleting the only pre-migration
/// mixed-case survivor would refuse-to-boot on the next restart.
pub async fn set_user_active(&self, user_id: Uuid, active: bool) -> Result<(), DomainError> { pub async fn set_user_active(&self, user_id: Uuid, active: bool) -> Result<(), DomainError> {
// Only the activate direction needs the collision-resolution
// dance — deactivation just flips a bit.
if active {
self.resolve_reactivation_collision(user_id).await?;
}
self.user_storage self.user_storage
.set_user_active_status(user_id, active) .set_user_active_status(user_id, active)
.await?; .await?;
@@ -3759,6 +3787,93 @@ impl AuthApplicationService {
Ok(()) Ok(())
} }
/// Pre-flight for `set_user_active(active = true)`: ensures the
/// target user's username is canonical (lowercase) AND unique
/// against currently-active accounts. Renames the target row if
/// either invariant would break.
///
/// NULL usernames (OPAQUE-migrated accounts) are a no-op — nothing
/// to normalise, nothing to collide.
async fn resolve_reactivation_collision(&self, user_id: Uuid) -> Result<(), DomainError> {
let target = self.user_storage.get_user_by_id(user_id).await?;
let Some(current) = target.username().map(str::to_string) else {
return Ok(());
};
let canonical = current.to_ascii_lowercase();
// Look for another ACTIVE user holding the canonical form.
// The migration CLI's `find_free_username_suffix` probes
// directly via a pool; here we don't have the pool
// (`AuthApplicationService` holds a `dyn UserRepository`
// trait object). Use `get_user_by_username` — the repo
// normalises input to lowercase before bind, so this
// resolves against the canonical row.
let collision = self
.user_storage
.get_user_by_username(&canonical)
.await
.ok()
.filter(|other| other.id() != user_id && other.is_active());
let chosen_name = match collision {
None => canonical,
Some(_) => {
// Collision — probe `<canonical>-2`, `-3`, … via
// repository lookups. Same shape as the migration
// CLI's `find_free_username_suffix`, just against
// the repo trait instead of a raw pool. Both paths
// agree by construction on the numbering scheme.
//
// Cap at 10_000 (matches the shared helper's cap —
// see `docs/plan/username-lowercase.md § 3. Suffix-
// collision robustness`). Reaching the cap means
// the account universe has an anomaly worth
// investigating; loud abort beats silent truncation.
const SUFFIX_PROBE_CAP: i32 = 10_000;
let mut chosen: Option<String> = None;
for n in 2..=SUFFIX_PROBE_CAP {
let candidate = format!("{canonical}-{n}");
match self.user_storage.get_user_by_username(&candidate).await {
Ok(_) => continue,
Err(_) => {
chosen = Some(candidate);
break;
}
}
}
let suffixed = chosen.ok_or_else(|| {
DomainError::internal_error(
"User",
format!(
"reactivation-collision suffix probe exhausted \
{SUFFIX_PROBE_CAP} candidates for base '{canonical}'"
),
)
})?;
tracing::info!(
target: "audit",
event = "user.reactivation_renamed",
reason = "collision_with_active",
target_id = %user_id,
from = %current,
to = %suffixed,
"🔄 user reactivation renamed to avoid username collision",
);
suffixed
}
};
if target.username() != Some(chosen_name.as_str()) {
let mut renamed = target;
renamed
.set_username(chosen_name)
.map_err(|e| DomainError::internal_error("User", format!("set_username: {e}")))?;
self.user_storage.update_user(renamed).await?;
}
Ok(())
}
/// Change user role (admin only). /// Change user role (admin only).
/// ///
/// Refuses `role = "admin"` when the target is external (grant-only). /// Refuses `role = "admin"` when the target is external (grant-only).
@@ -4435,10 +4550,14 @@ impl AuthApplicationService {
.clone() .clone()
.or(claims.name.clone()) .or(claims.name.clone())
.unwrap_or_else(|| format!("oidc_{}", &claims.sub[..8.min(claims.sub.len())])); .unwrap_or_else(|| format!("oidc_{}", &claims.sub[..8.min(claims.sub.len())]));
// Placeholder-email fallback when the IdP omits `email` from
// the claim set. Lowercase the local-part so the fake address
// matches the storage convention for other placeholder-email
// paths (see `admin_create_user`'s `<username>@oxicloud.local`).
let oidc_email = claims let oidc_email = claims
.email .email
.clone() .clone()
.unwrap_or_else(|| format!("{}@oidc.local", oidc_username)); .unwrap_or_else(|| format!("{}@oidc.local", oidc_username.to_ascii_lowercase()));
// 5. Look up existing user by OIDC subject. // 5. Look up existing user by OIDC subject.
// //
@@ -4652,16 +4771,30 @@ impl AuthApplicationService {
&oidc_username &oidc_username
}; };
// Filter to valid username characters only, then truncate to 32 chars // Lowercase at JIT derivation. `validate_username` in
// `User::new` would lowercase too, but the collision
// check below (`get_user_by_username`) needs the
// canonical form BEFORE `User::new` is called —
// otherwise `Alice` from an IdP claim would look
// "free" against an `alice` row on the first pass
// and fail the DB unique constraint at INSERT time.
// See `docs/plan/username-lowercase.md § 2. OIDC JIT
// derivation`.
//
// ASCII-only by the char-filter below, so
// `to_ascii_lowercase()` is deterministic and
// locale-safe.
let mut username = base_username let mut username = base_username
.chars() .chars()
.filter(|c| { .filter(|c| {
c.is_ascii_alphanumeric() || *c == '-' || *c == '_' || *c == '.' c.is_ascii_alphanumeric() || *c == '-' || *c == '_' || *c == '.'
}) })
.take(32) .take(32)
.collect::<String>(); .collect::<String>()
.to_ascii_lowercase();
// Filter helper: removes any chars that are not valid in a username // Filter helper: removes any chars that are not valid in a username.
// Lowercases too so the collision-suffix path below writes canonical form.
let filter_username_chars = |s: &str| { let filter_username_chars = |s: &str| {
s.chars() s.chars()
.filter(|c| { .filter(|c| {
@@ -4669,6 +4802,7 @@ impl AuthApplicationService {
}) })
.take(32) .take(32)
.collect::<String>() .collect::<String>()
.to_ascii_lowercase()
}; };
// Ensure minimum length (the padding suffix must also be filtered) // Ensure minimum length (the padding suffix must also be filtered)
@@ -116,19 +116,35 @@ impl ShareBrowseService {
self.list_inner(folder_id, resolved.owner_id).await self.list_inner(folder_id, resolved.owner_id).await
} }
/// AuthZ gate for `/api/s/{token}/file/{file_id}`: the requested file
/// must either BE the shared item (single-file share — the public landing
/// page's inline media preview streams through here) or live inside the
/// shared folder's subtree (folder share). Anything else is NotFound —
/// the same shape as "file doesn't exist", so the endpoint can't be used
/// to enumerate file ids.
pub async fn assert_file_in_share( pub async fn assert_file_in_share(
&self, &self,
token: &str, token: &str,
file_id: &str, file_id: &str,
unlock_jwt: Option<&str>, unlock_jwt: Option<&str>,
) -> Result<(), DomainError> { ) -> Result<(), DomainError> {
let resolved = self.resolve_folder_share(token, unlock_jwt).await?; let share = self
.share_service
.get_shared_link_with_unlock(token, unlock_jwt)
.await?;
if !self let in_scope = match share.item_type.as_str() {
.folder_repo // Single-file share: only the shared item itself may be streamed.
.is_file_in_subtree(file_id, &resolved.root_folder_id) "file" => file_id == share.item_id,
.await? // Folder share: the file must live in the shared subtree.
{ "folder" => {
self.folder_repo
.is_file_in_subtree(file_id, &share.item_id)
.await?
}
_ => false,
};
if !in_scope {
return Err(DomainError::not_found("File", file_id)); return Err(DomainError::not_found("File", file_id));
} }
Ok(()) Ok(())
+122 -1
View File
@@ -249,6 +249,18 @@ impl ShareService {
}; };
self.fetch_share_resolved(token, unlocked).await self.fetch_share_resolved(token, unlocked).await
} }
/// Public share landing payload: share metadata enriched with the shared
/// file's `mime_type` + `size` so anonymous viewers get an inline media
/// preview (video player / image) instead of a bare download button.
pub async fn get_shared_link_meta_with_unlock(
&self,
token: &str,
unlock_jwt: Option<&str>,
) -> Result<ShareDto, DomainError> {
let dto = self.get_shared_link_with_unlock(token, unlock_jwt).await?;
Ok(enrich_share_dto_with_file_info(dto, self.file_repository.as_ref()).await)
}
} }
impl ShareUseCase for ShareService { impl ShareUseCase for ShareService {
@@ -538,7 +550,8 @@ impl ShareUseCase for ShareService {
} }
// Password verified (or not required) — return full share metadata // Password verified (or not required) — return full share metadata
Ok(ShareDto::from_entity(&share, &self.base_url)) let dto = ShareDto::from_entity(&share, &self.base_url);
Ok(enrich_share_dto_with_file_info(dto, self.file_repository.as_ref()).await)
} }
async fn register_shared_link_access(&self, token: &str) -> Result<(), DomainError> { async fn register_shared_link_access(&self, token: &str) -> Result<(), DomainError> {
@@ -558,6 +571,31 @@ impl ShareUseCase for ShareService {
} }
} }
/// Fill `mime_type`/`size` on file-share DTOs so anonymous viewers get an
/// inline media preview (video player / image) on the public share page
/// instead of a bare download button.
///
/// Display-only enrichment that never fails the response: a failed file
/// lookup (transient DB error, race with a delete) leaves the fields `None`
/// and the download endpoint surfaces the real error — a read failure is
/// never proof the data is absent. Folder shares pass through untouched.
///
/// A free function (not a method) so the `integration_tests` mirror of the
/// service exercises the exact same logic instead of re-implementing it.
async fn enrich_share_dto_with_file_info<FR: FileReadPort>(
mut dto: ShareDto,
file_repository: &FR,
) -> ShareDto {
if dto.item_type != "file" {
return dto;
}
if let Ok(file) = file_repository.get_file(&dto.item_id).await {
dto.mime_type = Some(file.mime_type().to_string());
dto.size = Some(file.size());
}
dto
}
#[cfg(feature = "integration_tests")] #[cfg(feature = "integration_tests")]
#[allow(dead_code)] #[allow(dead_code)]
mod tests { mod tests {
@@ -642,6 +680,18 @@ mod tests {
})?; })?;
self.password_hasher.hash_password(password).await self.password_hasher.hash_password(password).await
} }
/// Mirror of `ShareService::get_shared_link_meta_with_unlock` —
/// fetch by token (the mirror has no unlock-JWT machinery) plus the
/// shared file-info enrichment.
async fn get_shared_link_meta_with_unlock(
&self,
token: &str,
_unlock_jwt: Option<&str>,
) -> Result<ShareDto, DomainError> {
let dto = self.get_shared_link_by_token(token).await?;
Ok(enrich_share_dto_with_file_info(dto, self.file_repository.as_ref()).await)
}
} }
impl<SR, FR, FoR, PH> ShareUseCase for ShareServiceForTest<SR, FR, FoR, PH> impl<SR, FR, FoR, PH> ShareUseCase for ShareServiceForTest<SR, FR, FoR, PH>
@@ -1261,4 +1311,75 @@ mod tests {
assert!(share_dto.has_password); assert!(share_dto.has_password);
assert!(share_dto.url.starts_with("http://127.0.0.1:8086/s/")); assert!(share_dto.url.starts_with("http://127.0.0.1:8086/s/"));
} }
/// The share-landing meta endpoint enriches file shares with the shared
/// file's mime type + size so anonymous viewers can render an inline
/// preview (video player / image) instead of a bare download button.
#[tokio::test]
async fn test_get_shared_link_meta_enriches_file_shares() {
let config = Arc::new(AppConfig::default());
let service = ShareServiceForTest::new(
config,
Arc::new(MockShareRepository::new()),
Arc::new(MockFileRepository),
Arc::new(MockFolderRepository),
Arc::new(MockPasswordHasher),
);
let share = service
.create_shared_link(
Uuid::new_v4(),
CreateShareDto {
item_id: "test_file_id".to_string(),
item_name: Some("movie.mp4".to_string()),
item_type: "file".to_string(),
password: None,
expires_at: None,
},
)
.await
.unwrap();
let meta = service
.get_shared_link_meta_with_unlock(&share.token, None)
.await
.unwrap();
assert_eq!(meta.mime_type.as_deref(), Some("text/plain"));
assert_eq!(meta.size, Some(123));
}
/// Folder shares must NOT gain a bogus mime type — the enrichment is a
/// file-share-only passthrough for them.
#[tokio::test]
async fn test_get_shared_link_meta_leaves_folder_shares_unenriched() {
let config = Arc::new(AppConfig::default());
let service = ShareServiceForTest::new(
config,
Arc::new(MockShareRepository::new()),
Arc::new(MockFileRepository),
Arc::new(MockFolderRepository),
Arc::new(MockPasswordHasher),
);
let share = service
.create_shared_link(
Uuid::new_v4(),
CreateShareDto {
item_id: "test_folder_id".to_string(),
item_name: Some("pictures".to_string()),
item_type: "folder".to_string(),
password: None,
expires_at: None,
},
)
.await
.unwrap();
let meta = service
.get_shared_link_meta_with_unlock(&share.token, None)
.await
.unwrap();
assert_eq!(meta.mime_type, None);
assert_eq!(meta.size, None);
}
} }
@@ -125,10 +125,16 @@ impl StorageUsageService {
} }
/// Same as [`Self::update_user_storage_usage`], keyed by username. /// Same as [`Self::update_user_storage_usage`], keyed by username.
///
/// Lowercases input before bind — same rule as
/// `UserRepository::get_user_by_username`. Usernames are canonical
/// (lowercase) in the DB post-migration; callers may pass any case.
/// See `docs/plan/username-lowercase.md`.
pub async fn update_user_storage_usage_by_username( pub async fn update_user_storage_usage_by_username(
&self, &self,
username: &str, username: &str,
) -> Result<i64, DomainError> { ) -> Result<i64, DomainError> {
let username = username.trim().to_ascii_lowercase();
let total_usage: Option<i64> = sqlx::query_scalar( let total_usage: Option<i64> = sqlx::query_scalar(
r#" r#"
UPDATE auth.users u UPDATE auth.users u
@@ -146,7 +152,7 @@ impl StorageUsageService {
RETURNING u.storage_used_bytes RETURNING u.storage_used_bytes
"#, "#,
) )
.bind(username) .bind(&username)
.fetch_optional(self.pool.as_ref()) .fetch_optional(self.pool.as_ref())
.await .await
.map_err(|e| { .map_err(|e| {
+314
View File
@@ -82,11 +82,35 @@ pub enum Action {
#[arg(long)] #[arg(long)]
dry_run: bool, dry_run: bool,
}, },
/// Lowercase every active user's username in `auth.users`.
///
/// Enforcement-companion for the case-insensitive-usernames
/// migration (see `docs/plan/username-lowercase.md`). The server
/// refuses to boot after upgrade until this has run. Data touched
/// is `auth.users.username` only.
///
/// Collision handling: when `Alice` and `alice` both exist,
/// tiebreak `(last_login_at DESC NULLS LAST, created_at ASC)` —
/// the winner keeps the canonical lowercased name, losers get
/// `<lowercase>-2`, `-3`, … via the shared
/// [`common::username_migration::find_free_username_suffix`]
/// probe. Sessions and grants survive the rename (they key on
/// `user_id`).
///
/// Skipped: soft-deleted / disabled rows and rows where
/// `username IS NULL` (OPAQUE-migrated accounts).
LowercaseUsernames {
/// Print what would change without touching the DB.
#[arg(long)]
dry_run: bool,
},
} }
pub async fn run(action: Action) -> u8 { pub async fn run(action: Action) -> u8 {
match action { match action {
Action::NfcFilenames { dry_run } => run_nfc_filenames(dry_run).await, Action::NfcFilenames { dry_run } => run_nfc_filenames(dry_run).await,
Action::LowercaseUsernames { dry_run } => run_lowercase_usernames(dry_run).await,
} }
} }
@@ -758,3 +782,293 @@ async fn run_folders(pool: &PgPool, dry_run: bool, stats: &mut Stats) -> Result<
} }
Ok(()) Ok(())
} }
// ════════════════════════════════════════════════════════════════════════════
// lowercase-usernames
// ════════════════════════════════════════════════════════════════════════════
#[derive(Default)]
struct UsernameStats {
scanned: u64,
already_lowercase: u64,
normalized_in_place: u64,
/// Multi-member `LOWER(username)` group where the tiebreak
/// winner kept the canonical name.
collision_winners: u64,
/// Multi-member losers renamed to `<lowercase>-N`.
renamed_to_suffix: u64,
/// Rows the scan touched but the loop declined to modify. Today
/// this is inactive rows (soft-deleted / admin-disabled). NULL
/// usernames never enter the scan so they don't contribute here.
skipped: u64,
}
#[derive(Debug)]
struct UsernameRow {
id: Uuid,
username: String,
/// Inactive rows (soft-deleted / admin-disabled) are read but not
/// modified — normalising a name we can't reach anyway risks
/// creating a `<lower>-N` conflict with a future re-activation of
/// the same handle. The loop uses this flag to skip and count.
active: bool,
// Kept for the SQL row-shape roundtrip (the SELECT ordering
// depends on them) even though the Rust-side grouping only
// reads `id` and `username`. Marked `#[allow(dead_code)]`
// so clippy doesn't nag; renaming to `_last_login_at` would
// work too but the SQL column names are load-bearing for the
// sqlx `Row::get` calls below.
#[allow(dead_code)]
last_login_at: Option<DateTime<Utc>>,
#[allow(dead_code)]
created_at: DateTime<Utc>,
}
async fn run_lowercase_usernames(dry_run: bool) -> u8 {
let database_url = match env::var("DATABASE_URL") {
Ok(v) => v,
Err(_) => {
eprintln!("migrate lowercase-usernames: DATABASE_URL not set");
return 2;
}
};
let pool = match PgPool::connect(&database_url).await {
Ok(p) => p,
Err(e) => {
eprintln!("migrate lowercase-usernames: failed to connect: {e}");
return 2;
}
};
if dry_run {
println!("migrate lowercase-usernames: DRY RUN (no writes)");
} else {
println!("migrate lowercase-usernames: applying changes");
}
// Load every user with a non-NULL username, active and inactive
// alike. Inactive rows are surfaced (not filtered at scan-time)
// so the `skipped (inactive)` counter can report them honestly —
// an operator reading the summary sees "10 rows scanned, 2
// inactive were passed on" instead of a phantom 0.
//
// NULL usernames stay out of the scan: there's nothing to
// normalise for OPAQUE-migrated rows, and pulling them would
// inflate `scanned` with rows the migration has no verb for.
//
// Ordering: alphabetic by `LOWER(username)` groups collisions
// together, then the intra-group order is the tiebreak
// (`last_login_at DESC NULLS LAST, created_at ASC` — most
// recently active wins the canonical name).
let rows: Vec<UsernameRow> = match sqlx::query(
r#"
SELECT id, username, active, last_login_at, created_at
FROM auth.users
WHERE username IS NOT NULL
ORDER BY LOWER(username),
(last_login_at IS NULL),
last_login_at DESC NULLS LAST,
created_at ASC
"#,
)
.fetch_all(&pool)
.await
{
Ok(rs) => rs
.into_iter()
.map(|r| UsernameRow {
id: r.get::<Uuid, _>("id"),
username: r.get::<String, _>("username"),
active: r.get::<bool, _>("active"),
last_login_at: r.try_get::<DateTime<Utc>, _>("last_login_at").ok(),
created_at: r.get::<DateTime<Utc>, _>("created_at"),
})
.collect(),
Err(e) => {
eprintln!("migrate lowercase-usernames: initial scan failed: {e}");
return 2;
}
};
let mut stats = UsernameStats::default();
// Group by `LOWER(username)`. Order preserved from the SQL query
// → within a group, the FIRST row is the tiebreak winner.
//
// Inactive rows are filtered OUT of the grouping (not just
// skipped inside the loop) so they can't create a phantom
// collision with an active row sharing their lowercase form.
// Example: inactive `Alice` + active `alice` would otherwise
// look like a two-member group; filtering inactive first leaves
// `alice` as a clean singleton no-op. The count goes to
// `stats.skipped`, surfaced in the summary as `skipped (inactive)`.
let mut groups: Vec<(String, Vec<UsernameRow>)> = Vec::new();
for row in rows {
stats.scanned += 1;
if !row.active {
stats.skipped += 1;
continue;
}
let key = row.username.to_ascii_lowercase();
match groups.last_mut() {
Some((k, v)) if k == &key => v.push(row),
_ => groups.push((key, vec![row])),
}
}
for (lower, members) in groups {
if members.len() == 1 {
let row = &members[0];
if row.username == lower {
stats.already_lowercase += 1;
continue;
}
// Single-member group with a mixed-case name → straight
// rename to the lowercase form. No collision.
if !dry_run && let Err(e) = update_username(&pool, row.id, &lower).await {
eprintln!(
"migrate lowercase-usernames: UPDATE failed for {}: {e}",
row.id
);
return 1;
}
println!(
"NORMALIZE user={} '{}' → '{}'",
row.id, row.username, lower
);
stats.normalized_in_place += 1;
continue;
}
// Multi-member group → collision. Members are already ordered
// by the tiebreak. Winner takes the canonical lowercase name,
// losers get `<lower>-2`, `-3`, … via the shared suffix helper.
//
// ORDER MATTERS: losers must be renamed FIRST. If we renamed
// the winner to `<lower>` while a loser still holds that
// exact name (the common case where the winner is mixed-case
// and the loser is already-lowercase), the UNIQUE constraint
// `users_username_key` fires. Freeing the canonical form by
// suffixing every non-winner member first eliminates the
// race entirely.
let (winner, losers) = members.split_first().expect("non-empty by construction");
// Suffixes assigned inside this group during this run. Used
// to keep dry-run consistent (no DB writes → the shared
// suffix helper would hand the same probe back for every
// loser). During apply, the DB itself deduplicates, but
// tracking here keeps the two modes structurally identical.
let mut reserved_this_group: Vec<String> = Vec::new();
for loser in losers {
let suffixed =
match pick_free_suffix_avoiding(&pool, &lower, &reserved_this_group).await {
Ok(s) => s,
Err(e) => {
eprintln!(
"migrate lowercase-usernames: suffix probe failed for {}: {e}",
loser.id
);
return 1;
}
};
if !dry_run && let Err(e) = update_username(&pool, loser.id, &suffixed).await {
eprintln!(
"migrate lowercase-usernames: loser UPDATE failed for {}: {e}",
loser.id
);
return 1;
}
println!(
"RENAME user={} '{}' → '{}' (collision suffix)",
loser.id, loser.username, suffixed
);
stats.renamed_to_suffix += 1;
reserved_this_group.push(suffixed);
}
if winner.username == lower {
// Winner already holds the canonical name (a lowercase
// row happened to be the most recently active; other
// members are the ones needing renames).
stats.already_lowercase += 1;
} else {
if !dry_run && let Err(e) = update_username(&pool, winner.id, &lower).await {
eprintln!(
"migrate lowercase-usernames: winner UPDATE failed for {}: {e}",
winner.id
);
return 1;
}
println!(
"NORMALIZE user={} '{}' → '{}' (collision winner)",
winner.id, winner.username, lower
);
stats.collision_winners += 1;
}
}
println!();
println!("Summary:");
println!(" scanned: {}", stats.scanned);
println!(" already-lowercase: {}", stats.already_lowercase);
println!(" normalized in place: {}", stats.normalized_in_place);
println!(" collision winners: {}", stats.collision_winners);
println!(" renamed to suffix: {}", stats.renamed_to_suffix);
println!(" skipped (inactive): {}", stats.skipped);
if dry_run
&& (stats.normalized_in_place + stats.collision_winners + stats.renamed_to_suffix) > 0
{
println!();
println!("(dry-run — re-run without --dry-run to apply)");
}
0
}
async fn update_username(pool: &PgPool, id: Uuid, new_name: &str) -> Result<(), sqlx::Error> {
sqlx::query("UPDATE auth.users SET username = $1, updated_at = NOW() WHERE id = $2")
.bind(new_name)
.bind(id)
.execute(pool)
.await?;
Ok(())
}
/// Pick the next free `<base>-<N>` suffix, skipping any suffix already
/// reserved earlier in this migration run.
///
/// Wraps [`crate::common::username_migration::find_free_username_suffix`]
/// with an additional local guard: in dry-run mode, no UPDATEs land so
/// the DB probe would return the same suffix for every loser in a
/// multi-loser group. The `reserved` slice lets the caller feed back
/// the suffixes it has already announced, and the probe steps past
/// them. In apply mode the DB probe alone would be enough (each real
/// UPDATE moves the state forward), but the same code path keeps the
/// two modes structurally identical.
async fn pick_free_suffix_avoiding(
pool: &PgPool,
base: &str,
reserved: &[String],
) -> Result<String, sqlx::Error> {
// Try the shared helper's default candidate first; if it collides
// with a same-run reservation, increment past it and probe again.
let mut n = 2;
loop {
let candidate = format!("{base}-{n}");
let db_taken: (bool,) =
sqlx::query_as("SELECT EXISTS(SELECT 1 FROM auth.users WHERE username = $1)")
.bind(&candidate)
.fetch_one(pool)
.await?;
let locally_taken = reserved.iter().any(|s| s == &candidate);
if !db_taken.0 && !locally_taken {
return Ok(candidate);
}
n += 1;
if n > 10_000 {
// Same cap as the shared helper — a loud panic beats a
// silent truncation for an anomaly this rare.
panic!("pick_free_suffix_avoiding: exhausted 10000 suffix probes for base '{base}'",);
}
}
}
+17 -4
View File
@@ -128,8 +128,15 @@ async fn run_reset(user: Option<String>, all: bool, dry_run: bool) -> u8 {
let rows_result = if all { let rows_result = if all {
sqlx::query(select_sql).fetch_all(&pool).await sqlx::query(select_sql).fetch_all(&pool).await
} else { } else {
let ident = user.as_deref().unwrap(); // Normalise before bind so the CLI accepts any case for
sqlx::query(select_sql).bind(ident).fetch_all(&pool).await // the username branch (email is already case-insensitive
// via a functional index on LOWER(email); lowercasing here
// for both branches is harmless — emails are lowercase
// ASCII in `auth.users.email` too).
// See `docs/plan/username-lowercase.md § 3. Lookup normalization`.
let ident_raw = user.as_deref().unwrap();
let ident = ident_raw.trim().to_ascii_lowercase();
sqlx::query(select_sql).bind(&ident).fetch_all(&pool).await
}; };
let rows = match rows_result { let rows = match rows_result {
Ok(r) => r, Ok(r) => r,
@@ -204,8 +211,14 @@ async fn run_reset(user: Option<String>, all: bool, dry_run: bool) -> u8 {
let write_result = if all { let write_result = if all {
sqlx::query(update_sql_all).execute(&pool).await sqlx::query(update_sql_all).execute(&pool).await
} else { } else {
let ident = user.as_deref().unwrap(); // Same normalisation as the read path above — usernames are
sqlx::query(update_sql_one).bind(ident).execute(&pool).await // canonical lowercase in the DB. See docs/plan/username-lowercase.md.
let ident_raw = user.as_deref().unwrap();
let ident = ident_raw.trim().to_ascii_lowercase();
sqlx::query(update_sql_one)
.bind(&ident)
.execute(&pool)
.await
}; };
let affected = match write_result { let affected = match write_result {
Ok(r) => r.rows_affected(), Ok(r) => r.rows_affected(),
+1
View File
@@ -8,3 +8,4 @@ pub mod mime_detect;
pub mod runtime; pub mod runtime;
pub mod stubs; pub mod stubs;
pub mod text; pub mod text;
pub mod username_migration;
+396
View File
@@ -0,0 +1,396 @@
//! Username-lowercase boot flow: verifier, auto-rename, shared collision helper.
//!
//! The plan (`docs/plan/username-lowercase.md`) makes usernames
//! case-insensitive by canonicalising to lowercase on ingest. Three
//! pieces of infrastructure live here:
//!
//! 1. [`verify_all_usernames_lowercase`] — a **read-only** check that
//! runs after `sqlx::migrate!()` at boot. Classifies every active
//! mixed-case row into one of three outcomes:
//!
//! - [`UsernameCaseCheck::Clean`] — nothing to do.
//! - [`UsernameCaseCheck::AutoRenamable`] — mixed-case rows exist
//! but each `LOWER(username)` form is unique in the active-user
//! set. Safe to lowercase in one atomic transaction; boot proceeds.
//! - [`UsernameCaseCheck::Collisions`] — at least one group has
//! two or more active rows sharing a `LOWER(username)` (e.g.
//! `Alice` + `alice`). Tiebreak requires human judgement; the
//! server refuses to start and prints the CLI command.
//!
//! Follows [[feedback_no_silent_auto_repair]] in spirit: silent
//! action is limited to cases where there is exactly one correct
//! move (rename the sole mixed-case row to its lowercase form).
//! Anywhere ambiguity exists (which of `Alice` and `alice` keeps
//! the canonical name?), boot refuses and defers to `oxicloud
//! migrate lowercase-usernames`.
//!
//! 2. [`apply_auto_renames`] — the one-transaction UPDATE loop that
//! performs the auto-rename path. Emits a structured audit line
//! per row (`user.username_lowercased_on_boot`). All-or-nothing:
//! a mid-tx failure aborts the transaction and boot fails, so the
//! DB is never left in a half-renamed state.
//!
//! 3. [`find_free_username_suffix`] — the shared collision-resolution
//! helper. Called by the migration CLI when it lowercases a name
//! that would clash with an existing row, AND by the un-soft-delete
//! API when it re-normalises a mixed-case account whose lowercase
//! form is now taken by someone else.
//!
//! `NULL` usernames (OPAQUE-migrated accounts) are always skipped — the
//! SQL `WHERE username <> LOWER(username)` predicate is NULL-safe by
//! semantics (`NULL <> anything` yields `NULL`, which `WHERE` excludes).
//! Soft-deleted / disabled accounts (`active = false`) are also skipped:
//! they can't serve traffic anyway.
use sqlx::{PgPool, Row};
/// One mixed-case account row. Used both for the auto-rename list and
/// for reporting collision-group members.
#[derive(Debug, Clone)]
pub struct MixedCaseAccount {
pub id: uuid::Uuid,
pub username: String,
pub last_login_at: Option<chrono::DateTime<chrono::Utc>>,
}
/// A `LOWER(username)` group with two or more active members. At least
/// one member is mixed-case (that's what made the group visible to the
/// verifier); the other member(s) may be already-lowercase (e.g.
/// `Alice` + `alice`) or also mixed-case (`Alice` + `ALICE`).
#[derive(Debug, Clone)]
pub struct CollisionGroup {
/// The lowercase form shared by every member.
pub canonical: String,
/// Members, ordered by the tiebreak that the migration CLI
/// applies: `last_login_at DESC NULLS LAST, created_at ASC`.
pub members: Vec<MixedCaseAccount>,
}
/// The three outcomes of the boot-time verifier.
#[derive(Debug, Clone)]
pub enum UsernameCaseCheck {
/// Every active username is already lowercase (or `NULL`). Boot
/// proceeds unmodified.
Clean,
/// Mixed-case rows exist, but each `LOWER(username)` form is
/// unique among active users. Safe to lowercase atomically at
/// boot; the caller runs [`apply_auto_renames`].
AutoRenamable(Vec<MixedCaseAccount>),
/// At least one `LOWER(username)` group has two or more active
/// members. Tiebreak requires human judgement; the caller formats
/// a refusal message via [`format_refusal_message_collisions`] and
/// aborts boot.
Collisions(Vec<CollisionGroup>),
}
/// Boot-time verifier. Runs AFTER `sqlx::migrate!()` and BEFORE
/// `AppState` is assembled. Read-only: never mutates `auth.users`.
///
/// Returns [`UsernameCaseCheck`] describing what (if anything) the
/// caller should do. Errors are limited to DB failures — semantic
/// outcomes are all `Ok(_)` variants.
pub async fn verify_all_usernames_lowercase(pool: &PgPool) -> Result<UsernameCaseCheck, String> {
// First pass: mixed-case rows that have NO other active row
// sharing their LOWER form. These are safe to auto-rename.
let auto_rows = sqlx::query(
r#"
SELECT u.id, u.username, u.last_login_at
FROM auth.users u
WHERE u.active = true
AND u.username <> LOWER(u.username)
AND NOT EXISTS (
SELECT 1
FROM auth.users u2
WHERE u2.active = true
AND u2.id <> u.id
AND LOWER(u2.username) = LOWER(u.username)
)
ORDER BY LOWER(u.username)
"#,
)
.fetch_all(pool)
.await
.map_err(|e| format!("username lowercase verifier: singleton query failed: {e}"))?;
// Second pass: every active row that belongs to a colliding
// group — a `LOWER(username)` shared by two or more active rows
// where at least one member is mixed-case. Result includes
// already-lowercase members so the refusal report shows the full
// context of each collision.
let collision_rows = sqlx::query(
r#"
WITH colliding_lowers AS (
SELECT LOWER(username) AS canonical
FROM auth.users
WHERE active = true
GROUP BY LOWER(username)
HAVING COUNT(*) > 1
AND SUM(CASE WHEN username <> LOWER(username) THEN 1 ELSE 0 END) >= 1
)
SELECT id, username, last_login_at, LOWER(username) AS canonical
FROM auth.users
WHERE active = true
AND LOWER(username) IN (SELECT canonical FROM colliding_lowers)
ORDER BY LOWER(username),
(last_login_at IS NULL),
last_login_at DESC NULLS LAST,
created_at ASC
"#,
)
.fetch_all(pool)
.await
.map_err(|e| format!("username lowercase verifier: collision query failed: {e}"))?;
if !collision_rows.is_empty() {
// Group by canonical. Rows are already ordered by canonical
// then by tiebreak, so a fold is enough.
let mut groups: Vec<CollisionGroup> = Vec::new();
for r in collision_rows {
let canonical: String = r.get("canonical");
let member = MixedCaseAccount {
id: r.get::<uuid::Uuid, _>("id"),
username: r.get::<String, _>("username"),
last_login_at: r
.try_get::<chrono::DateTime<chrono::Utc>, _>("last_login_at")
.ok(),
};
match groups.last_mut() {
Some(g) if g.canonical == canonical => g.members.push(member),
_ => groups.push(CollisionGroup {
canonical,
members: vec![member],
}),
}
}
return Ok(UsernameCaseCheck::Collisions(groups));
}
if auto_rows.is_empty() {
return Ok(UsernameCaseCheck::Clean);
}
let accounts = auto_rows
.into_iter()
.map(|r| MixedCaseAccount {
id: r.get::<uuid::Uuid, _>("id"),
username: r.get::<String, _>("username"),
last_login_at: r
.try_get::<chrono::DateTime<chrono::Utc>, _>("last_login_at")
.ok(),
})
.collect();
Ok(UsernameCaseCheck::AutoRenamable(accounts))
}
/// Apply the atomic auto-rename transaction. All UPDATEs succeed
/// together or all roll back — the DB is never left in a half-renamed
/// state. Each successful rename emits a structured audit line.
///
/// The `WHERE id = $1 AND username = $3` guard defends against a
/// concurrent rename between the SELECT and this UPDATE. If some
/// other process renamed the row in that window, the UPDATE affects
/// zero rows and we log a warning but do not fail the transaction —
/// the row is already lowercase (that's why the guard didn't match),
/// so the invariant still holds.
pub async fn apply_auto_renames(
pool: &PgPool,
accounts: &[MixedCaseAccount],
) -> Result<(), sqlx::Error> {
let mut tx = pool.begin().await?;
for acc in accounts {
let new_username = acc.username.to_ascii_lowercase();
let res = sqlx::query(
r#"
UPDATE auth.users
SET username = $2
WHERE id = $1
AND username = $3
"#,
)
.bind(acc.id)
.bind(&new_username)
.bind(&acc.username)
.execute(&mut *tx)
.await?;
if res.rows_affected() == 0 {
tracing::warn!(
target: "audit",
event = "user.username_lowercase_skipped_on_boot",
reason = "row_changed_between_verify_and_apply",
user_id = %acc.id,
expected_username = %acc.username,
"👮🏻‍♂️ skipped auto-lowercase: row was modified after verifier ran",
);
continue;
}
tracing::info!(
target: "audit",
event = "user.username_lowercased_on_boot",
reason = "unique_lowercase_group",
user_id = %acc.id,
old_username = %acc.username,
new_username = %new_username,
"👮🏻‍♂️ auto-lowercased username at boot",
);
}
tx.commit().await?;
Ok(())
}
/// Format the FATAL error string shown when boot refuses to proceed
/// because at least one `LOWER(username)` group has multiple active
/// members. Self-sufficient — an operator at 3 AM shouldn't need to
/// consult docs to know what to do.
pub fn format_refusal_message_collisions(groups: &[CollisionGroup]) -> String {
use std::fmt::Write;
let total_members: usize = groups.iter().map(|g| g.members.len()).sum();
let mut out = String::new();
let _ = write!(
&mut out,
"\nFATAL: cannot start — {} colliding username group(s) \
({} affected account(s) in total).\n\n\
Non-colliding mixed-case rows are auto-renamed at boot. \
These groups can't be resolved automatically because two or \
more active accounts share the same lowercase form, and only \
a human can decide who keeps the canonical name.\n\n\
Run the migration:\n\n \
oxicloud migrate lowercase-usernames --dry-run # preview the tiebreak\n \
oxicloud migrate lowercase-usernames # apply\n\n\
The tiebreak rule is `last_login_at DESC NULLS LAST, \
created_at ASC` — the most recently active member keeps the \
canonical lowercase name; the losers get `-2`, `-3`, … as a \
suffix. Sessions and grants survive the rename (they key on \
user_id, not username).\n\n\
Collision groups (up to 10 shown):\n",
groups.len(),
total_members
);
for g in groups.iter().take(10) {
let _ = writeln!(&mut out, "\n Canonical form: {}", g.canonical);
for m in &g.members {
let last = m
.last_login_at
.map(|t| t.format("%Y-%m-%d").to_string())
.unwrap_or_else(|| "never".to_string());
let _ = writeln!(
&mut out,
" {} (id: {} last_login: {})",
m.username, m.id, last
);
}
}
if groups.len() > 10 {
let _ = writeln!(
&mut out,
"\n ... and {} more group(s). Run --dry-run for the full list.",
groups.len() - 10
);
}
out
}
/// Cap on the suffix-probe loop. If we ever need `<base>-10000` there's
/// something very wrong with the account universe — collisions in the
/// wild are 2-3 accounts, not 10 K. The loud abort IS the detection.
/// See [`docs/plan/username-lowercase.md § 3. Suffix-collision robustness`].
const SUFFIX_PROBE_CAP: i32 = 10_000;
/// Find the next free `<base>-<N>` suffix for a colliding username.
///
/// Starts at `<base>-2` and increments until an unused suffix is
/// found. Robust against pre-existing rows already occupying some
/// suffixes (the probe steps past them).
///
/// Called by:
/// - The migration CLI when a `LOWER(username)` group has multiple
/// members and the tiebreak winner keeps the canonical name; the
/// losers get `<base>-2`, `-3`, … from this helper.
/// - The un-soft-delete API when re-normalising a mixed-case
/// account whose lowercase form is now taken by an active row.
///
/// Both callers reach for this single function so the two paths
/// agree by construction — no drift risk between the migration and
/// runtime un-soft-delete.
pub async fn find_free_username_suffix(pool: &PgPool, base: &str) -> Result<String, sqlx::Error> {
for n in 2..=SUFFIX_PROBE_CAP {
let candidate = format!("{base}-{n}");
let exists: (bool,) =
sqlx::query_as("SELECT EXISTS(SELECT 1 FROM auth.users WHERE username = $1)")
.bind(&candidate)
.fetch_one(pool)
.await?;
if !exists.0 {
return Ok(candidate);
}
}
// If we get here, something is very wrong. Loud panic beats
// silent truncation to whatever the caller's fallback is.
panic!(
"find_free_username_suffix: exhausted {SUFFIX_PROBE_CAP} suffix probes for base '{base}'; \
the account universe likely has an anomaly worth investigating"
);
}
#[cfg(test)]
mod tests {
use super::*;
fn acc(name: &str) -> MixedCaseAccount {
MixedCaseAccount {
id: uuid::Uuid::nil(),
username: name.into(),
last_login_at: None,
}
}
#[test]
fn refusal_message_lists_groups_and_cli() {
let groups = vec![CollisionGroup {
canonical: "alice".into(),
members: vec![acc("Alice"), acc("alice")],
}];
let msg = format_refusal_message_collisions(&groups);
assert!(msg.contains("1 colliding username group(s)"));
assert!(msg.contains("2 affected account(s)"));
assert!(msg.contains("oxicloud migrate lowercase-usernames"));
assert!(msg.contains("Canonical form: alice"));
assert!(msg.contains("Alice"));
assert!(msg.contains("last_login: never"));
}
#[test]
fn refusal_message_caps_group_display_and_notes_overflow() {
let groups: Vec<_> = (0..15)
.map(|i| CollisionGroup {
canonical: format!("user{i:02}"),
members: vec![acc(&format!("User{i:02}")), acc(&format!("user{i:02}"))],
})
.collect();
let msg = format_refusal_message_collisions(&groups);
// First 10 groups shown by canonical name.
assert!(msg.contains("Canonical form: user00"));
assert!(msg.contains("Canonical form: user09"));
// Overflow tail names how many are hidden.
assert!(msg.contains("and 5 more group(s)"));
}
#[test]
fn refusal_message_reports_total_across_all_groups() {
// Two groups with different sizes — 2 + 3 = 5 members total.
let groups = vec![
CollisionGroup {
canonical: "alice".into(),
members: vec![acc("Alice"), acc("alice")],
},
CollisionGroup {
canonical: "bob".into(),
members: vec![acc("Bob"), acc("BOB"), acc("bob")],
},
];
let msg = format_refusal_message_collisions(&groups);
assert!(msg.contains("2 colliding username group(s)"));
assert!(msg.contains("5 affected account(s)"));
}
}
+93 -14
View File
@@ -322,9 +322,15 @@ impl User {
is_external: bool, is_external: bool,
) -> UserResult<Self> { ) -> UserResult<Self> {
Self::validate_email(&email)?; Self::validate_email(&email)?;
if let Some(ref u) = username { // Shadow `username` with the canonical (trimmed, lowercased)
Self::validate_username(u)?; // form returned by `validate_username`. Every downstream write
} // consumes the shadowed binding, so the row that lands in the
// DB is always the normalised value. See
// `docs/plan/username-lowercase.md`.
let username = match username {
Some(u) => Some(Self::validate_username(&u)?),
None => None,
};
if let Some(ref h) = password_hash if let Some(ref h) = password_hash
&& h.is_empty() && h.is_empty()
{ {
@@ -809,8 +815,10 @@ impl User {
/// renamed: it was display text at creation; the folder is owned /// renamed: it was display text at creation; the folder is owned
/// by `user_id`. /// by `user_id`.
pub fn set_username(&mut self, new_username: String) -> UserResult<()> { pub fn set_username(&mut self, new_username: String) -> UserResult<()> {
Self::validate_username(&new_username)?; // Canonical form (trim + lowercase) — see
self.username = Some(new_username); // `validate_username`. Callers can pass any case; we store
// the normalised value.
self.username = Some(Self::validate_username(&new_username)?);
self.updated_at = Utc::now(); self.updated_at = Utc::now();
Ok(()) Ok(())
} }
@@ -881,20 +889,41 @@ impl User {
/// a handle that shadows another user's email). No leading/trailing /// a handle that shadows another user's email). No leading/trailing
/// dot or hyphen. The character set also prevents XSS payloads from /// dot or hyphen. The character set also prevents XSS payloads from
/// being stored as usernames. /// being stored as usernames.
fn validate_username(username: &str) -> UserResult<()> { /// Validate AND canonicalise a username.
let len = username.chars().count(); ///
/// Two normalisations run first, before every check:
/// - `trim()` — strip whitespace clients may have added.
/// - `to_ascii_lowercase()` — usernames are case-insensitive
/// identifiers. Users type `Alice`, `ALICE`, `alice` on
/// different clients; all three refer to the same account.
/// ASCII-only by construction (charset check below), so
/// `to_ascii_lowercase` is deterministic and locale-safe —
/// no Unicode case-folding surprises (Turkish dotted-I,
/// German ß, Greek final sigma, NFC vs NFD).
///
/// Returns the canonical form on success. Every entity write
/// site consumes the returned string — because the signature
/// changed from `Result<()>` to `Result<String>`, any caller
/// that ignored the result is now a compile error. That's
/// what forces every write path through the normaliser.
///
/// See `docs/plan/username-lowercase.md` for the full design.
pub fn validate_username(username: &str) -> UserResult<String> {
let normalized = username.trim().to_ascii_lowercase();
let len = normalized.chars().count();
if !(2..=64).contains(&len) { if !(2..=64).contains(&len) {
return Err(UserError::InvalidUsername( return Err(UserError::InvalidUsername(
"Username must be between 2 and 64 characters".to_string(), "Username must be between 2 and 64 characters".to_string(),
)); ));
} }
if username.contains('@') { if normalized.contains('@') {
return Err(UserError::InvalidUsername( return Err(UserError::InvalidUsername(
"Username must not contain '@' — use the email field for email addresses" "Username must not contain '@' — use the email field for email addresses"
.to_string(), .to_string(),
)); ));
} }
if !username if !normalized
.chars() .chars()
.all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_' || c == '.') .all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_' || c == '.')
{ {
@@ -903,16 +932,16 @@ impl User {
.to_string(), .to_string(),
)); ));
} }
if username.starts_with('.') if normalized.starts_with('.')
|| username.starts_with('-') || normalized.starts_with('-')
|| username.ends_with('.') || normalized.ends_with('.')
|| username.ends_with('-') || normalized.ends_with('-')
{ {
return Err(UserError::InvalidUsername( return Err(UserError::InvalidUsername(
"Username must not start or end with a dot or hyphen".to_string(), "Username must not start or end with a dot or hyphen".to_string(),
)); ));
} }
Ok(()) Ok(normalized)
} }
/// Basic but meaningful email validation: /// Basic but meaningful email validation:
@@ -1046,4 +1075,54 @@ mod tests {
assert_eq!(u.display_full(true), "solo@x.com"); assert_eq!(u.display_full(true), "solo@x.com");
assert_eq!(u.display_full(false), "solo@x.com"); assert_eq!(u.display_full(false), "solo@x.com");
} }
// ── validate_username: normalization + rules ─────────────────────────────
//
// Post-lowercase-migration `validate_username` returns the canonical
// (trimmed, lowercased) form on success. Every write-site consumes
// that returned string via the shadow in `User::new` /
// `set_username`, so the invariant "usernames in `auth.users` are
// always canonical" is enforced at the domain boundary.
//
// The rules that DON'T change (charset, length, no leading/trailing
// dot or hyphen, no `@`) get their coverage here too so a future
// rewrite of `validate_username` can't regress them silently.
#[test]
fn validate_username_lowercases_and_trims() {
// Uppercase in the middle → canonical form is lowercase.
assert_eq!(User::validate_username("Alice").unwrap(), "alice");
// All-uppercase.
assert_eq!(User::validate_username("ALICE").unwrap(), "alice");
// Whitespace around a mixed-case name → both stripped.
assert_eq!(User::validate_username(" Alice ").unwrap(), "alice");
// Already-canonical passes through unchanged.
assert_eq!(User::validate_username("alice").unwrap(), "alice");
}
#[test]
fn validate_username_charset_and_boundary_rules_survive_normalization() {
// Trailing hyphen — still rejected after the case-fold.
assert!(User::validate_username("alice-").is_err());
// Leading dot.
assert!(User::validate_username(".alice").is_err());
// Whitespace INSIDE the name (not just around it) — the
// charset check rejects space characters.
assert!(User::validate_username("Al ice").is_err());
// Non-ASCII letter — usernames are ASCII-only.
assert!(User::validate_username("Álice").is_err());
// `@` is forbidden (disjoint namespace with email lookup).
assert!(User::validate_username("alice@example").is_err());
}
#[test]
fn validate_username_length_bounds_apply_after_trim() {
// Two-char minimum satisfied AFTER trim.
assert_eq!(User::validate_username(" ab ").unwrap(), "ab");
// Below the minimum after trim.
assert!(User::validate_username(" a ").is_err());
// Above the maximum after trim.
let too_long = "a".repeat(65);
assert!(User::validate_username(&too_long).is_err());
}
} }
+40
View File
@@ -61,6 +61,46 @@ pub async fn create_database_pools(config: &AppConfig) -> Result<DbPools> {
} }
tracing::info!("Database migrations complete"); tracing::info!("Database migrations complete");
// Username-lowercase verifier — three outcomes:
// * Clean → nothing to do.
// * AutoRenamable → non-colliding mixed-case rows exist; lowercase
// them in one transaction and continue. Silent
// action is bounded to the case where there is
// exactly one correct move ([[feedback_no_silent_auto_repair]]
// in spirit — ambiguity → refusal, unique fix → apply).
// Each rename emits an audit line.
// * Collisions → two or more active rows share a LOWER(username)
// form (e.g. `Alice` + `alice`); tiebreak needs a
// human, refuse to boot and print the CLI command.
// See `common::username_migration::verify_all_usernames_lowercase`.
use crate::common::username_migration::{
UsernameCaseCheck, apply_auto_renames, format_refusal_message_collisions,
verify_all_usernames_lowercase,
};
match verify_all_usernames_lowercase(&primary).await {
Ok(UsernameCaseCheck::Clean) => {}
Ok(UsernameCaseCheck::AutoRenamable(accounts)) => {
let count = accounts.len();
if let Err(e) = apply_auto_renames(&primary, &accounts).await {
return Err(DbError(format!(
"username lowercase auto-rename failed at boot: {e}. \
Run `oxicloud migrate lowercase-usernames --dry-run` to \
inspect the current state, then apply manually."
)));
}
tracing::info!(
target: "audit",
event = "user.usernames_lowercased_on_boot_summary",
renamed = count,
"auto-lowercased {count} non-colliding mixed-case username(s) at boot",
);
}
Ok(UsernameCaseCheck::Collisions(groups)) => {
return Err(DbError(format_refusal_message_collisions(&groups)));
}
Err(msg) => return Err(DbError(msg)),
}
// --- maintenance pool --- // --- maintenance pool ---
let maintenance = create_pool_with_retries( let maintenance = create_pool_with_retries(
&config.database.connection_string, &config.database.connection_string,
@@ -472,8 +472,15 @@ impl UserRepository for UserPgRepository {
Ok((user, flags)) Ok((user, flags))
} }
/// Gets a user by username /// Gets a user by username.
///
/// Lowercases the input before binding: usernames are stored in
/// canonical (lowercase, trimmed) form by `validate_username`
/// (see `docs/plan/username-lowercase.md`), and callers may pass
/// whatever case the user typed at the login form. Normalising
/// here means every caller doesn't have to remember.
async fn get_user_by_username(&self, username: &str) -> UserRepositoryResult<User> { async fn get_user_by_username(&self, username: &str) -> UserRepositoryResult<User> {
let username = username.trim().to_ascii_lowercase();
let row = sqlx::query( let row = sqlx::query(
r#" r#"
SELECT SELECT
@@ -487,7 +494,7 @@ impl UserRepository for UserPgRepository {
WHERE username = $1 WHERE username = $1
"#, "#,
) )
.bind(username) .bind(&username)
.fetch_one(&*self.pool) .fetch_one(&*self.pool)
.await .await
.map_err(Self::map_sqlx_error)?; .map_err(Self::map_sqlx_error)?;
+575 -22
View File
@@ -58,7 +58,7 @@ use crate::application::ports::blob_lifecycle::BlobLifecycleHook;
use crate::application::ports::blob_reference_ports::{BlobReferenceRegistry, RefLevel}; use crate::application::ports::blob_reference_ports::{BlobReferenceRegistry, RefLevel};
use crate::application::ports::blob_storage_ports::BlobStorageBackend; use crate::application::ports::blob_storage_ports::BlobStorageBackend;
use crate::application::ports::dedup_ports::{ use crate::application::ports::dedup_ports::{
BlobMetadataDto, DedupPort, DedupResultDto, DedupStatsDto, BlobMetadataDto, DedupPort, DedupResultDto, DedupStatsDto, DerivedBlobRef,
}; };
use crate::application::services::blob_lifecycle_service::BlobLifecycleService; use crate::application::services::blob_lifecycle_service::BlobLifecycleService;
use crate::domain::errors::{DomainError, ErrorKind}; use crate::domain::errors::{DomainError, ErrorKind};
@@ -72,6 +72,86 @@ pub const CDC_AVG_CHUNK: usize = 262_144;
/// Maximum CDC chunk size (1 MB). /// Maximum CDC chunk size (1 MB).
pub const CDC_MAX_CHUNK: usize = 1_048_576; pub const CDC_MAX_CHUNK: usize = 1_048_576;
// ════════════════════════════════════════════════════════════════════════════
// ref_count audit log — attribution trail for ref_count changes
//
// Emitted at every semantic ref-count mutation site so an unexplained drift
// (`manifests_consistency` / `blobs_consistency` finding) can be traced back
// to its calling function within one log query. See
// `docs/plan/refcount-audit.md` for design rationale, retention model, and
// upgrade path to a DB-backed table if log retention proves insufficient.
//
// Enable at runtime with `RUST_LOG=oxicloud::refcount=info`. Off by default;
// info-level so a healthy prod deployment doesn't spam the log stream.
// ════════════════════════════════════════════════════════════════════════════
/// Table names the audit stream uses. Constants (not free strings) so grep
/// across a log stream matches a fixed vocabulary and a typo in a call site
/// fails to compile instead of silently drifting.
mod refcount_audit_table {
pub const CHUNK_MANIFESTS: &str = "chunk_manifests";
pub const BLOBS: &str = "blobs";
}
/// Source labels for the audit stream — one stable string per Rust function
/// that mutates a ref_count. Kept as a module so `grep source=` on the log
/// stream shows a fixed enumeration; a rename here is intentional, a rename
/// at a call site alone doesn't compile.
///
/// See `docs/plan/refcount-audit.md § Callsites to instrument` for the full
/// list. New callers add a new constant here; adding one string at the call
/// site alone is discouraged (breaks the "closed vocabulary" property).
mod refcount_audit_source {
pub const STORE_FROM_STREAM_NEW_MANIFEST: &str = "store_from_stream.new_manifest";
pub const BUMP_MANIFEST_IF_EXISTS: &str = "bump_manifest_if_exists";
pub const ADD_REFERENCE_MANIFEST: &str = "add_reference.manifest";
pub const ADD_REFERENCE_LEGACY: &str = "add_reference.legacy";
pub const REMOVE_MANIFEST_REFERENCE_DECREMENT: &str = "remove_manifest_reference.decrement";
pub const REMOVE_MANIFEST_REFERENCE_DELETE: &str = "remove_manifest_reference.delete";
pub const REMOVE_LEGACY_REFERENCE: &str = "remove_legacy_reference";
pub const STORE_ATTACHED_BLOB_SAME_CONTENT_BALANCE: &str =
"store_attached_blob.same_content_balance";
pub const STORE_ATTACHED_BLOB_REPLACE_RELEASE: &str = "store_attached_blob.replace_release";
}
/// Outcome of [`DedupService::store_attached_blob_if_absent`]. Split
/// so the caller (`thumb_attached_import_service`) can bump its
/// `imported` vs `already` counters without a second query.
#[derive(Debug, Clone)]
pub enum AttachedBlobInsertOutcome {
/// We won the atomic INSERT — the row now points at `hash`.
Inserted { hash: String },
/// A row already existed when the atomic INSERT ran (someone else
/// won, or the migration was re-triggered). `existing_hash` is
/// the row's current `blob_hash` as read moments after the DO
/// NOTHING resolved — useful for the import service's
/// verify-and-unlink readback.
AlreadyPresent { existing_hash: String },
}
/// Emit a single audit line for a ref_count change. Called AFTER the SQL
/// UPDATE / INSERT / DELETE returns Ok, so a rolled-back transaction won't
/// leave a phantom log line (the SQL error path returns before this call).
///
/// `delta` is the signed change (`+1` on increment, `-1` on decrement,
/// `-old_count` when the row is deleted at its last reference — the
/// convention is "resulting ref_count is 0 for reads afterward").
///
/// The tracing span inherits request-scope context (request_id, caller_id
/// from auth middleware, job run id from scheduler) automatically, so
/// no explicit correlation-id plumbing is needed here.
#[inline]
fn audit_ref_count(table: &'static str, hash: &str, delta: i32, source: &'static str) {
tracing::info!(
target: "oxicloud::refcount",
table,
hash = %hash,
delta,
source,
"ref_count {}", if delta >= 0 { "+" } else { "-" }
);
}
// ── CDC helper types ───────────────────────────────────────────────────────── // ── CDC helper types ─────────────────────────────────────────────────────────
/// Everything a streaming chunk ingest learned about its byte stream. /// Everything a streaming chunk ingest learned about its byte stream.
@@ -566,6 +646,49 @@ fn manifest_reap_sql(registry: &BlobReferenceRegistry) -> String {
) )
} }
// ── Attached-blob lookup cache ───────────────────────────────────────────────
/// Cache size cap for [`DedupService::attached_blob_cache`] — plain entry
/// count (no weigher): an entry is three short strings + two short strings,
/// tens of bytes; 50k entries ≈ a few MB, noise next to the manifest cache.
pub(crate) const ATTACHED_BLOB_CACHE_MAX_ENTRIES: u64 = 50_000;
/// Hard staleness bound for [`DedupService::attached_blob_cache`].
///
/// Deliberately [`moka::future::Cache::builder().time_to_live`] and NOT
/// `time_to_idle`: a hot negative entry under TTI never expires, and TTL must
/// be the last-resort bound for writes this process never saw (bare SQL, a
/// future second instance, the `copy_file_satellites` race window).
pub(crate) const ATTACHED_BLOB_CACHE_TTL_SECS: u64 = 60;
/// Cache key for [`DedupService::attached_blob_cache`] — the
/// `storage.file_attached_blobs` primary key. A struct, not a
/// `(String, String, String)` tuple: three same-typed fields read by position
/// would force every construction site (and the `invalidate_for_file` scan)
/// to guess semantics; self-documenting beats positional here.
#[derive(Clone, PartialEq, Eq, Hash, Debug)]
struct AttachedBlobKey {
file_id: String,
kind: String,
variant: String,
}
impl AttachedBlobKey {
fn new(file_id: &str, kind: &str, variant: &str) -> Self {
Self {
file_id: file_id.to_string(),
kind: kind.to_string(),
variant: variant.to_string(),
}
}
}
/// Loader-error sentinel for the `try_get_with` cache wrapper on
/// [`Self::find_attached_blob`]. The SQL lookup treats a DB fault the same as
/// "no row" only at the very last moment — the cache must never see it, or a
/// transient outage would freeze "no attached blob" into place for a full
/// TTL while rows exist (a read failure is never proof that data is absent).
struct AttachedLookupFault;
pub struct DedupService { pub struct DedupService {
/// Pluggable blob storage backend (local FS, S3, …). /// Pluggable blob storage backend (local FS, S3, …).
backend: Arc<dyn BlobStorageBackend>, backend: Arc<dyn BlobStorageBackend>,
@@ -584,6 +707,28 @@ pub struct DedupService {
/// seen immediately), weight-bounded (a manifest is ~72 B per chunk), /// seen immediately), weight-bounded (a manifest is ~72 B per chunk),
/// short TTL so GC'd manifests age out fast (benches/MANIFEST-CACHE.md). /// short TTL so GC'd manifests age out fast (benches/MANIFEST-CACHE.md).
manifest_cache: moka::future::Cache<String, Arc<ChunkManifest>>, manifest_cache: moka::future::Cache<String, Arc<ChunkManifest>>,
/// `file_id → attached blob` lookup cache (`storage.file_attached_blobs`
/// rows) for the thumbnail hot path — `ThumbnailService::
/// thumbnail_content_id` hits it on EVERY request (including 304
/// revalidations and RAM thumbnail hits, `thumbnail_service.rs` ~:744)
/// and `get_cached_thumbnail` tier 2b hits it again with the same key
/// (~:851); the Nextcloud preview endpoint rides the same lookup.
///
/// Positive AND negative (`Option<DerivedBlobRef>` — most files have no
/// attached preview row, so the negative side is where the win is). The
/// loader NEVER caches a DB error: `find_attached_blob_uncached` returns
/// `Err` and `try_get_with` drops it, so a transient outage cannot freeze
/// "no attached blob" into the cache for a full TTL (a read failure is
/// never proof that data is absent).
///
/// Writes invalidate through the same type: `store_attached_blob` /
/// `store_attached_blob_if_absent` on success, file deletions via the
/// `ThumbnailRefreshHook::on_file_deleted` piggyback. The TTL above
/// remains the bound for anything this process cannot see (bare SQL,
/// `copy_file_satellites` races); invalidate-vs-inflight-REFILL races are
/// narrowed by `try_get_with` but not eliminated, and the residual window
/// is ≤ one TTL.
attached_blob_cache: moka::future::Cache<AttachedBlobKey, Option<DerivedBlobRef>>,
/// Every table that holds blob references, so GC agrees with the /// Every table that holds blob references, so GC agrees with the
/// consistency jobs on what "referenced" means. Defaults to the two /// consistency jobs on what "referenced" means. Defaults to the two
/// built-in sources; DI replaces it once more tables exist. Never /// built-in sources; DI replaces it once more tables exist. Never
@@ -618,6 +763,7 @@ impl DedupService {
maintenance_pool, maintenance_pool,
blob_lifecycle: None, blob_lifecycle: None,
manifest_cache: Self::build_manifest_cache(), manifest_cache: Self::build_manifest_cache(),
attached_blob_cache: Self::build_attached_blob_cache(),
reference_registry: registry.clone(), reference_registry: registry.clone(),
manifest_reap_sql: manifest_reap_sql(&registry), manifest_reap_sql: manifest_reap_sql(&registry),
blob_reap_sql: blob_reap_sql(&registry), blob_reap_sql: blob_reap_sql(&registry),
@@ -649,6 +795,19 @@ impl DedupService {
.build() .build()
} }
/// See the `attached_blob_cache` field docs. Plain entry-count cap (no
/// weigher — an entry is a handful of short strings), TTL as the hard
/// staleness bound; same hard-coded-const treatment as the manifest
/// cache rather than config: an internal accelerator with strict
/// write-side invalidation, where a misconfiguration costs performance,
/// never correctness.
fn build_attached_blob_cache() -> moka::future::Cache<AttachedBlobKey, Option<DerivedBlobRef>> {
moka::future::Cache::builder()
.max_capacity(ATTACHED_BLOB_CACHE_MAX_ENTRIES)
.time_to_live(std::time::Duration::from_secs(ATTACHED_BLOB_CACHE_TTL_SECS))
.build()
}
/// Registers the blob-reference registry used by the manifest reap /// Registers the blob-reference registry used by the manifest reap
/// predicate. Without it `garbage_collect` skips manifest collection /// predicate. Without it `garbage_collect` skips manifest collection
/// entirely — see `docs/plan/derived-blobs.md`. /// entirely — see `docs/plan/derived-blobs.md`.
@@ -751,32 +910,216 @@ impl DedupService {
.await .await
.map_err(|e| DomainError::internal_error("Dedup", format!("record attached blob: {e}")))?; .map_err(|e| DomainError::internal_error("Dedup", format!("record attached blob: {e}")))?;
// A replaced row's old blob loses its only reference from here. Not // The row is replaced — drop any cached (possibly negative) entry so
// releasing it would pin those bytes forever — nothing else points at // the next lookup refills from the new truth. Only on the success
// a superseded preview. // path: if the execute had failed, the row is unchanged and the
if let Some((old_hash,)) = previous // cache is still accurate, so invalidating would just cost a refill.
&& old_hash != attached_hash self.attached_blob_cache
&& let Err(e) = self.remove_reference(&old_hash).await .invalidate(&AttachedBlobKey::new(file_id, kind, variant))
{ .await;
tracing::warn!(
target: "oxicloud::dedup", // Two shapes to balance depending on whether the UPSERT was a
error = %e, // real content replacement or a same-content re-store:
"failed to release replaced attached-blob reference for {}", //
&old_hash[..old_hash.len().min(12)], // - `previous == Some(old) && old != attached_hash` — different
); // content overwritten in the row. Release the old blob's ref
// (its file_attached row is gone; would leak forever otherwise).
//
// - `previous == Some(old) && old == attached_hash` — SAME-content
// re-store. `store_from_stream` above incremented the manifest
// unconditionally, but the row's blob_hash didn't change so no
// logical reference was added. Cancel the phantom increment
// here, or it accumulates one +1 leak per same-content call.
// Mirrors the pattern `store_derived_blob` uses on its
// `ON CONFLICT DO NOTHING` `inserted == 0` branch.
// See `docs/plan/refcount-audit.md` for how the audit stream
// would surface this class of drift if it reappears.
//
// - `previous == None` — brand new (file_id, kind, variant) row.
// `store_from_stream`'s +1 pairs with the new row's implicit
// reference; nothing to release.
if let Some((old_hash,)) = previous {
let source = if old_hash == attached_hash {
refcount_audit_source::STORE_ATTACHED_BLOB_SAME_CONTENT_BALANCE
} else {
refcount_audit_source::STORE_ATTACHED_BLOB_REPLACE_RELEASE
};
if let Err(e) = self.remove_reference(&old_hash).await {
tracing::warn!(
target: "oxicloud::dedup",
error = %e,
kind = source,
"failed to balance attached-blob reference for {}",
&old_hash[..old_hash.len().min(12)],
);
}
} }
Ok(attached_hash) Ok(attached_hash)
} }
/// Atomic never-overwrite variant of [`Self::store_attached_blob`],
/// for migration/import paths whose semantic is "write if absent,
/// leave alone if present" — mirrors the shape
/// [`Self::store_derived_blob`] already uses.
///
/// The plain `store_attached_blob` reads `previous`, then upserts,
/// then decrements — non-transactional. That's correct for the
/// user-driven PUT thumbnail path (a real replacement should
/// release the superseded blob), but it opens a check-then-act
/// race with concurrent writers when the caller's intent is
/// "only import if this file hasn't already got a preview".
/// `thumb_attached_import_service` is exactly that caller.
///
/// This variant uses a single-statement `INSERT ... ON CONFLICT
/// DO NOTHING` — race-free by construction. If the row already
/// exists (any content), the atomic INSERT is a no-op and we
/// release the reference `store_from_stream` just took. If we
/// won the insert, the reference is legitimately held by our
/// new row.
///
/// Return value discriminates the two cases so the caller can
/// track its own `imported` vs `already` counters:
/// - [`AttachedBlobInsertOutcome::Inserted`] — we wrote the row.
/// - [`AttachedBlobInsertOutcome::AlreadyPresent`] — a row was
/// there when we arrived; we made no change and released our
/// ref. `existing_hash` is the concurrent winner's blob hash,
/// returned via a follow-up SELECT (so it's not strictly
/// atomic with the INSERT, but that's fine — the row's shape
/// is stable now that a concurrent writer can no longer
/// collide with us here; any later change goes through the
/// full `store_attached_blob` UPSERT path, which is out of
/// scope for this method's "if absent" contract).
pub async fn store_attached_blob_if_absent(
&self,
file_id: &str,
kind: &str,
variant: &str,
content_type: &str,
bytes: Bytes,
uploaded_by: uuid::Uuid,
) -> Result<AttachedBlobInsertOutcome, DomainError> {
let stored = self
.store_from_stream(
stream::once(async move { Ok::<Bytes, std::io::Error>(bytes) }),
Some(content_type.to_string()),
)
.await?;
let attached_hash = stored.hash().to_string();
// Single-statement atomic INSERT. `ON CONFLICT DO NOTHING`
// means: if another writer got there first, we silently
// yield. Same primitive `store_derived_blob` uses.
let inserted = sqlx::query(
"INSERT INTO storage.file_attached_blobs
(file_id, kind, variant, blob_hash, content_type, uploaded_by)
VALUES ($1::uuid, $2, $3, $4, $5, $6)
ON CONFLICT (file_id, kind, variant) DO NOTHING",
)
.bind(file_id)
.bind(kind)
.bind(variant)
.bind(&attached_hash)
.bind(content_type)
.bind(uploaded_by)
.execute(self.pool.as_ref())
.await
.map_err(|e| DomainError::internal_error("Dedup", format!("record attached blob: {e}")))?
.rows_affected();
if inserted == 0 {
// Row already existed when we arrived. Release the ref
// `store_from_stream` above took — the row that would
// justify it is not ours. Best-effort: leaving a
// dangling ref is worse than a warn log line.
if let Err(e) = self.remove_reference(&attached_hash).await {
tracing::warn!(
target: "oxicloud::dedup",
error = %e,
"failed to release duplicate attached-blob reference for {}",
&attached_hash[..attached_hash.len().min(12)],
);
}
// Fetch the concurrent winner's hash so the import
// service can readback-verify against its sidecar. The
// window between DO NOTHING and this SELECT is narrow;
// if the row gets updated in it, the sidecar delete
// path fails its verify and keeps the sidecar — the
// conservative fallback.
//
// This readback now flows through the `attached_blob_cache`.
// Safe in-process: any write this process made already
// invalidated the key. The only degraded case is a negative
// entry cached before some OTHER process inserted the row —
// nonexistent in a single-instance deployment, and even then
// the consequence is `existing_hash: ""` → the import keeps
// its sidecar, the documented conservative fallback.
let existing = self.find_attached_blob(file_id, kind, variant).await;
return Ok(AttachedBlobInsertOutcome::AlreadyPresent {
existing_hash: existing.map(|r| r.blob_hash).unwrap_or_default(),
});
}
// We wrote a row for a key the cache may hold a negative entry for
// (the common "import backfill" case) — drop it so the new row is
// immediately visible to the thumbnail path.
self.attached_blob_cache
.invalidate(&AttachedBlobKey::new(file_id, kind, variant))
.await;
Ok(AttachedBlobInsertOutcome::Inserted {
hash: attached_hash,
})
}
/// Look up bytes attached to a file. File-keyed counterpart of /// Look up bytes attached to a file. File-keyed counterpart of
/// [`Self::find_derived_blob`]. /// [`Self::find_derived_blob`].
///
/// Cached read-through of [`Self::attached_blob_cache`] (positive AND
/// negative); see the field docs for why. The public signature is
/// unchanged — including the historical "DB fault reads as no row"
/// behaviour — but the fault now dies BEFORE the cache instead of being
/// indistinguishable from an absent row.
pub async fn find_attached_blob( pub async fn find_attached_blob(
&self, &self,
file_id: &str, file_id: &str,
kind: &str, kind: &str,
variant: &str, variant: &str,
) -> Option<crate::application::ports::dedup_ports::DerivedBlobRef> { ) -> Option<DerivedBlobRef> {
match self
.attached_blob_cache
.try_get_with(AttachedBlobKey::new(file_id, kind, variant), async {
self.find_attached_blob_uncached(file_id, kind, variant)
.await
.map_err(|_| AttachedLookupFault) // Err ⇒ never cached
})
.await
{
Ok(attached) => attached,
Err(_) => {
tracing::debug!(
target: "oxicloud::dedup",
"attached-blob lookup failed (not cached): file={} kind={} variant={}",
file_id,
kind,
variant
);
None
}
}
}
/// The uncached lookup — one indexed point query on the
/// `file_attached_blobs` primary key. Unlike the historical inlined
/// body, a DB fault surfaces as `Err` so the cache wrapper can refuse to
/// store it; only a genuine `Ok(None)` means "no row".
async fn find_attached_blob_uncached(
&self,
file_id: &str,
kind: &str,
variant: &str,
) -> sqlx::Result<Option<DerivedBlobRef>> {
sqlx::query_as::<_, (String, String)>( sqlx::query_as::<_, (String, String)>(
"SELECT blob_hash, content_type FROM storage.file_attached_blobs "SELECT blob_hash, content_type FROM storage.file_attached_blobs
WHERE file_id = $1::uuid AND kind = $2 AND variant = $3", WHERE file_id = $1::uuid AND kind = $2 AND variant = $3",
@@ -786,16 +1129,31 @@ impl DedupService {
.bind(variant) .bind(variant)
.fetch_optional(self.pool.as_ref()) .fetch_optional(self.pool.as_ref())
.await .await
.ok() .map(|row| {
.flatten() row.map(|(blob_hash, content_type)| DerivedBlobRef {
.map(|(blob_hash, content_type)| {
crate::application::ports::dedup_ports::DerivedBlobRef {
blob_hash, blob_hash,
content_type, content_type,
} })
}) })
} }
/// Invalidate every `(kind, variant)` entry cached for one file.
///
/// Fired from `ThumbnailRefreshHook::on_file_deleted` so all three
/// production delete paths (single file, folder cascade, trash clear)
/// drop their cached rows after the DELETE commits. A linear scan over
/// the keys is fine here: deletions are rare and the cache is capped at
/// [`ATTACHED_BLOB_CACHE_MAX_ENTRIES`].
pub async fn invalidate_attached_blobs_for_file(&self, file_id: &str) {
// moka's `Iter` yields `(Arc<K>, V)` synchronously — the await lives
// in `invalidate`, not in the scan itself.
for (key, _) in self.attached_blob_cache.iter() {
if key.file_id == file_id {
self.attached_blob_cache.invalidate(&*key).await;
}
}
}
pub async fn store_derived_blob( pub async fn store_derived_blob(
&self, &self,
source_hash: &str, source_hash: &str,
@@ -1105,6 +1463,7 @@ impl DedupService {
maintenance_pool: stub_pool.clone(), maintenance_pool: stub_pool.clone(),
blob_lifecycle: None, blob_lifecycle: None,
manifest_cache: Self::build_manifest_cache(), manifest_cache: Self::build_manifest_cache(),
attached_blob_cache: Self::build_attached_blob_cache(),
reference_registry: stub_registry.clone(), reference_registry: stub_registry.clone(),
manifest_reap_sql: manifest_reap_sql(&stub_registry), manifest_reap_sql: manifest_reap_sql(&stub_registry),
blob_reap_sql: blob_reap_sql(&stub_registry), blob_reap_sql: blob_reap_sql(&stub_registry),
@@ -1316,6 +1675,17 @@ impl DedupService {
total_size, total_size,
chunk_hashes.len(), chunk_hashes.len(),
); );
// The manifest INSERT above set `ref_count = 1` — that's
// the initial reference held by whatever callsite drove
// this ingest (a file's body, a preview attachment, a
// derivation). Audit-log the +1 so drift investigations
// can find where a manifest first came into existence.
audit_ref_count(
refcount_audit_table::CHUNK_MANIFESTS,
file_hash,
1,
refcount_audit_source::STORE_FROM_STREAM_NEW_MANIFEST,
);
self.fire_blob_creation_hooks(file_hash, content_type.as_deref()); self.fire_blob_creation_hooks(file_hash, content_type.as_deref());
return Ok(DedupResultDto::NewBlob { return Ok(DedupResultDto::NewBlob {
hash: file_hash.to_string(), hash: file_hash.to_string(),
@@ -1747,7 +2117,7 @@ impl DedupService {
/// Bump a manifest's ref_count if it exists; returns its total_size. /// Bump a manifest's ref_count if it exists; returns its total_size.
/// Single statement — no window between the existence check and the bump. /// Single statement — no window between the existence check and the bump.
async fn bump_manifest_if_exists(&self, file_hash: &str) -> Result<Option<i64>, DomainError> { async fn bump_manifest_if_exists(&self, file_hash: &str) -> Result<Option<i64>, DomainError> {
sqlx::query_scalar::<_, i64>( let bumped = sqlx::query_scalar::<_, i64>(
"UPDATE storage.chunk_manifests SET ref_count = ref_count + 1 "UPDATE storage.chunk_manifests SET ref_count = ref_count + 1
WHERE file_hash = $1 WHERE file_hash = $1
RETURNING total_size", RETURNING total_size",
@@ -1757,7 +2127,17 @@ impl DedupService {
.await .await
.map_err(|e| { .map_err(|e| {
DomainError::internal_error("Dedup", format!("Failed to bump manifest ref_count: {e}")) DomainError::internal_error("Dedup", format!("Failed to bump manifest ref_count: {e}"))
}) })?;
if bumped.is_some() {
audit_ref_count(
refcount_audit_table::CHUNK_MANIFESTS,
file_hash,
1,
refcount_audit_source::BUMP_MANIFEST_IF_EXISTS,
);
}
Ok(bumped)
} }
/// Stream → chunk store, WITHOUT creating a manifest. /// Stream → chunk store, WITHOUT creating a manifest.
@@ -2219,6 +2599,12 @@ impl DedupService {
.rows_affected(); .rows_affected();
if manifest_affected > 0 { if manifest_affected > 0 {
audit_ref_count(
refcount_audit_table::CHUNK_MANIFESTS,
hash,
1,
refcount_audit_source::ADD_REFERENCE_MANIFEST,
);
return Ok(()); return Ok(());
} }
@@ -2246,6 +2632,12 @@ impl DedupService {
)); ));
} }
audit_ref_count(
refcount_audit_table::BLOBS,
hash,
1,
refcount_audit_source::ADD_REFERENCE_LEGACY,
);
Ok(()) Ok(())
} }
@@ -2355,6 +2747,17 @@ impl DedupService {
&file_hash[..12], &file_hash[..12],
chunk_hashes.len() chunk_hashes.len()
); );
// Emit AFTER the commit so a rolled-back TX doesn't leave a
// phantom audit line — the "delta = -current_rc" reflects
// "the manifest is gone, effective ref_count is 0". Convention
// for the audit stream: use the delta that would produce a
// read-back of 0.
audit_ref_count(
refcount_audit_table::CHUNK_MANIFESTS,
file_hash,
-current_rc,
refcount_audit_source::REMOVE_MANIFEST_REFERENCE_DELETE,
);
Ok(true) Ok(true)
} else { } else {
// Still has references — just decrement // Still has references — just decrement
@@ -2373,6 +2776,12 @@ impl DedupService {
.map_err(|e| DomainError::internal_error("Dedup", format!("Commit: {}", e)))?; .map_err(|e| DomainError::internal_error("Dedup", format!("Commit: {}", e)))?;
tracing::debug!("Reference removed from manifest {}", &file_hash[..12]); tracing::debug!("Reference removed from manifest {}", &file_hash[..12]);
audit_ref_count(
refcount_audit_table::CHUNK_MANIFESTS,
file_hash,
-1,
refcount_audit_source::REMOVE_MANIFEST_REFERENCE_DECREMENT,
);
Ok(false) Ok(false)
} }
} }
@@ -2430,6 +2839,12 @@ impl DedupService {
self.reap_blob(hash).await; self.reap_blob(hash).await;
tracing::info!("BLOB DELETED: {} (no more references)", &hash[..12]); tracing::info!("BLOB DELETED: {} (no more references)", &hash[..12]);
audit_ref_count(
refcount_audit_table::BLOBS,
hash,
-ref_count,
refcount_audit_source::REMOVE_LEGACY_REFERENCE,
);
Ok(true) Ok(true)
} else { } else {
// Still has references — just decrement // Still has references — just decrement
@@ -2450,6 +2865,12 @@ impl DedupService {
})?; })?;
tracing::debug!("Reference removed from blob {}", &hash[..12]); tracing::debug!("Reference removed from blob {}", &hash[..12]);
audit_ref_count(
refcount_audit_table::BLOBS,
hash,
-1,
refcount_audit_source::REMOVE_LEGACY_REFERENCE,
);
Ok(false) Ok(false)
} }
} }
@@ -3920,6 +4341,138 @@ impl crate::infrastructure::scheduler::JobHandler for DedupService {
mod tests { mod tests {
use super::*; use super::*;
// ── attached_blob_cache — find_attached_blob read-through ───────────────
//
// Pure in-memory contract tests: `new_stub()` connects lazily to an
// unreachable pool, so anything that reaches the "DB" fails loudly. That
// is exactly what makes these work — a served `Some` proves the cache was
// consulted, and a missing entry after a fault proves the fault was not
// cached. Same no-SQL style as the hash_cache tests in
// `file_blob_read_repository.rs`.
fn attached_key(file_id: &str, kind: &str, variant: &str) -> AttachedBlobKey {
AttachedBlobKey::new(file_id, kind, variant)
}
fn sample_ref(hash: &str) -> DerivedBlobRef {
DerivedBlobRef {
blob_hash: hash.to_string(),
content_type: "image/jpeg".to_string(),
}
}
/// A seeded entry is served without touching the (unreachable) stub pool
/// — returning `Some` at all proves the read-through hit the cache.
#[tokio::test]
async fn attached_lookup_serves_a_seeded_entry() {
let svc = DedupService::new_stub();
let k = attached_key("0189d1b3-8f2a-7cde-b1ad-000000000001", "preview", "icon");
svc.attached_blob_cache
.insert(k.clone(), Some(sample_ref("abc")))
.await;
assert_eq!(
svc.find_attached_blob(&k.file_id, "preview", "icon").await,
Some(sample_ref("abc"))
);
}
/// Negative entries are where most of the win is (most files have no
/// attached preview). A cached `None` must be served as `None` AND
/// survive the call — not be evicted by the miss path.
#[tokio::test]
async fn attached_lookup_serves_and_keeps_a_negative_entry() {
let svc = DedupService::new_stub();
let k = attached_key("0189d1b3-8f2a-7cde-b1ad-000000000002", "preview", "icon");
svc.attached_blob_cache.insert(k.clone(), None).await;
assert_eq!(
svc.find_attached_blob(&k.file_id, "preview", "icon").await,
None
);
assert!(
svc.attached_blob_cache.get(&k).await.is_some(),
"negative entry was dropped by the lookup"
);
}
/// THE contract this change exists for: a DB fault must not be cached.
/// The stub pool cannot connect, so the uncached lookup errors; the
/// wrapper returns `None` (historical behaviour) and leaves the cache
/// empty — a row that appears after a transient outage must be visible
/// on the very next call, not hidden behind a frozen negative entry.
#[tokio::test]
async fn attached_lookup_does_not_cache_a_db_fault() {
let svc = DedupService::new_stub();
let k = attached_key("0189d1b3-8f2a-7cde-b1ad-000000000003", "preview", "icon");
assert_eq!(
svc.find_attached_blob(&k.file_id, "preview", "icon").await,
None
);
assert!(
svc.attached_blob_cache.get(&k).await.is_none(),
"DB fault was cached as a negative entry"
);
}
/// Per-file invalidation drops every `(kind, variant)` of that file and
/// leaves other files' entries alone.
#[tokio::test]
async fn invalidate_attached_blobs_for_file_is_scoped_to_the_file() {
let svc = DedupService::new_stub();
let k1 = attached_key("0189d1b3-8f2a-7cde-b1ad-000000000004", "preview", "icon");
let k2 = attached_key("0189d1b3-8f2a-7cde-b1ad-000000000004", "preview", "large");
let k3 = attached_key("0189d1b3-8f2a-7cde-b1ad-000000000005", "preview", "icon");
for (k, v) in [
(k1.clone(), Some(sample_ref("a"))),
(k2.clone(), None),
(k3.clone(), Some(sample_ref("c"))),
] {
svc.attached_blob_cache.insert(k, v).await;
}
svc.invalidate_attached_blobs_for_file(&k1.file_id).await;
assert!(svc.attached_blob_cache.get(&k1).await.is_none());
assert!(svc.attached_blob_cache.get(&k2).await.is_none());
assert!(
svc.attached_blob_cache.get(&k3).await.is_some(),
"another file's entry must survive"
);
}
/// Invalidation happens only after a SUCCESSFUL write: the store path
/// fails (unreachable pool) before any row is touched, so the previously
/// cached entry must still be there. Invalidating on failure would be
/// harmless but pointless — the row is unchanged and the cache accurate.
#[tokio::test]
async fn failed_attached_store_leaves_the_cache_alone() {
let svc = DedupService::new_stub();
let k = attached_key("0189d1b3-8f2a-7cde-b1ad-000000000006", "preview", "icon");
svc.attached_blob_cache
.insert(k.clone(), Some(sample_ref("xyz")))
.await;
let result = svc
.store_attached_blob(
&k.file_id,
"preview",
"icon",
"image/png",
Bytes::from_static(b"nope"),
uuid::Uuid::nil(),
)
.await;
assert!(
result.is_err(),
"stub pool is unreachable — store must fail"
);
assert_eq!(
svc.attached_blob_cache.get(&k).await,
Some(Some(sample_ref("xyz"))),
"failed store must not disturb the cache"
);
}
/// Golden test for the statement `garbage_collect` runs against production /// Golden test for the statement `garbage_collect` runs against production
/// data. It is assembled from the registered reference sources rather than /// data. It is assembled from the registered reference sources rather than
/// written as a literal, so this pins the whole thing byte-for-byte — the /// written as a literal, so this pins the whole thing byte-for-byte — the
@@ -102,7 +102,23 @@ async fn fsync_paths_parallel(paths: Vec<PathBuf>, strict: bool) -> Result<(), D
tasks.push(tokio::task::spawn_blocking( tasks.push(tokio::task::spawn_blocking(
move || -> Result<(), (PathBuf, std::io::Error)> { move || -> Result<(), (PathBuf, std::io::Error)> {
for path in &group { for path in &group {
let result = std::fs::File::open(path).and_then(|f| f.sync_all()); // `strict` marks blob *file* fsyncs; best-effort marks
// prefix *directory* fsyncs. That distinction also picks
// the open mode: Windows `FlushFileBuffers` needs a
// GENERIC_WRITE handle and fails with ACCESS_DENIED on
// the read-only handle `File::open` returns (POSIX fsync
// accepts read-only fds, which is why this only surfaced
// on Windows). Directories keep the read-only POSIX
// dirent-sync idiom — they can't be fsync'd on Windows
// at all, and their failures stay best-effort warnings.
let result = if strict {
std::fs::OpenOptions::new()
.write(true)
.open(path)
.and_then(|f| f.sync_all())
} else {
std::fs::File::open(path).and_then(|f| f.sync_all())
};
if let Err(e) = result { if let Err(e) = result {
if strict { if strict {
return Err((path.clone(), e)); return Err((path.clone(), e));
@@ -394,7 +410,11 @@ impl BlobStorageBackend for LocalBlobBackend {
format!("Failed to copy file to blob store: {}", ce), format!("Failed to copy file to blob store: {}", ce),
) )
})?; })?;
if let Ok(f) = fs::File::open(&blob_path).await { // Open for write: Windows `FlushFileBuffers` requires a
// GENERIC_WRITE handle — the read-only handle from
// `File::open` fails with ACCESS_DENIED, silently
// skipping this fsync on every Windows deployment.
if let Ok(f) = fs::OpenOptions::new().write(true).open(&blob_path).await {
let _ = f.sync_all().await; let _ = f.sync_all().await;
} }
let _ = fs::remove_file(&source_path).await; let _ = fs::remove_file(&source_path).await;
@@ -274,49 +274,31 @@ impl RecoverableJobHandler for ThumbAttachedImport {
}; };
let file_id_str = file_id.to_string(); let file_id_str = file_id.to_string();
// Already mapped. Checked BEFORE storing, because // Orphan check first — no atomic-insert exists for a
// `store_attached_blob` is ON CONFLICT DO UPDATE and would // file_id whose FK would reject. Same rationale as before;
// release then retake the reference on every run. // the race window between this check and the INSERT is
if let Some(existing) = self // narrow AND covered by the FK constraint if the file is
.dedup // deleted after we look — the atomic INSERT would then
.find_attached_blob(&file_id_str, "preview", &dir_name) // fail loudly instead of silently drift.
.await //
{ // Everything else — "row present" and "row absent" —
already += 1; // used to be split across two branches with a
// Drains on a later run too: importing first and enabling // non-transactional `find_attached_blob` between the
// deletion afterwards is the expected operator sequence, // check and the write. That opened a check-then-act
// so reaching here is the common path rather than an edge // race window: a concurrent thumbnail writer could
// case. // INSERT the row after the check returned None, and the
if delete_imported { // subsequent `store_attached_blob` UPSERT-UPDATE would
let path = self.thumbnails_root.join(&dir_name).join(&name); // fire with same-or-different content. In the
if ThumbDerivedImport::verify_and_unlink( // same-content case that leaked +1 on the manifest ref
&self.dedup, // (pre-fix; guard branch now cancels).
THUMB_ATTACHED_IMPORT_JOB_NAME, //
&file_id_str, // Merged into ONE atomic call
&existing.blob_hash, // `store_attached_blob_if_absent`: single-statement
&path, // `INSERT ... ON CONFLICT DO NOTHING`, race-free by
) // construction. The outcome enum distinguishes the two
.await // paths so `imported` and `already` counters stay
{ // accurate.
deleted += 1; if !self.file_exists(file_id).await {
} else {
unverified += 1;
record_or_log(
store,
THUMB_ATTACHED_IMPORT_JOB_NAME,
"sidecar_delete_unverified",
"anomaly",
None,
serde_json::json!({
"path": position,
"file_id": file_id_str,
"note": "attached blob did not read back; sidecar kept",
}),
)
.await;
}
}
} else if !self.file_exists(file_id).await {
// The file is gone, so this sidecar is unimportable: the // The file is gone, so this sidecar is unimportable: the
// FK on `file_id` would reject the row. Mirrors the // FK on `file_id` would reject the row. Mirrors the
// dead-source case in thumb_derived_import. // dead-source case in thumb_derived_import.
@@ -389,9 +371,10 @@ impl RecoverableJobHandler for ThumbAttachedImport {
let path = self.thumbnails_root.join(&dir_name).join(&name); let path = self.thumbnails_root.join(&dir_name).join(&name);
match fs::read(&path).await { match fs::read(&path).await {
Ok(data) => { Ok(data) => {
use crate::infrastructure::services::dedup_service::AttachedBlobInsertOutcome;
match self match self
.dedup .dedup
.store_attached_blob( .store_attached_blob_if_absent(
&file_id_str, &file_id_str,
"preview", "preview",
&dir_name, &dir_name,
@@ -404,14 +387,39 @@ impl RecoverableJobHandler for ThumbAttachedImport {
) )
.await .await
{ {
Ok(attached_hash) => { Ok(outcome) => {
imported += 1; // Bump the right counter AND pick the
if delete_imported { // hash we'll verify-and-unlink against:
// Inserted — our new blob
// AlreadyPresent — the concurrent
// winner's blob
// Both drain the sidecar identically
// (verify-then-unlink on repair mode).
let verify_hash = match outcome {
AttachedBlobInsertOutcome::Inserted { hash } => {
imported += 1;
hash
}
AttachedBlobInsertOutcome::AlreadyPresent {
existing_hash,
} => {
already += 1;
existing_hash
}
};
// Empty existing_hash only happens if
// the AlreadyPresent path's follow-up
// SELECT was overtaken by another
// writer. verify_and_unlink would
// refuse the sidecar delete in that
// case anyway, but skipping the call
// saves the pointless readback.
if delete_imported && !verify_hash.is_empty() {
if ThumbDerivedImport::verify_and_unlink( if ThumbDerivedImport::verify_and_unlink(
&self.dedup, &self.dedup,
THUMB_ATTACHED_IMPORT_JOB_NAME, THUMB_ATTACHED_IMPORT_JOB_NAME,
&file_id_str, &file_id_str,
&attached_hash, &verify_hash,
&path, &path,
) )
.await .await
@@ -1874,10 +1874,16 @@ impl crate::application::ports::file_lifecycle::FileLifecycleHook for ThumbnailR
fn on_file_deleted(&self, file_id: &str) { fn on_file_deleted(&self, file_id: &str) {
let thumbnail = self.thumbnail.clone(); let thumbnail = self.thumbnail.clone();
let file_id = file_id.to_string(); let file_id = file_id.to_string();
// The row is gone (CASCADE cleared file_attached_blobs) — drop any
// cached attached-blob lookup for this file too. TTL would bound the
// staleness anyway, but deletes are rare and the cache lookup after a
// delete is pure waste.
let dedup = self.dedup.clone();
tokio::spawn(async move { tokio::spawn(async move {
if let Err(e) = thumbnail.delete_thumbnails(&file_id).await { if let Err(e) = thumbnail.delete_thumbnails(&file_id).await {
tracing::warn!("Failed to delete thumbnails for file {}: {}", file_id, e); tracing::warn!("Failed to delete thumbnails for file {}: {}", file_id, e);
} }
dedup.invalidate_attached_blobs_for_file(&file_id).await;
}); });
} }
} }
+2 -2
View File
@@ -242,7 +242,7 @@ pub async fn access_shared_item(
// every public share landing). // every public share landing).
let (_, item) = tokio::join!( let (_, item) = tokio::join!(
share_use_case.register_shared_link_access(&token), share_use_case.register_shared_link_access(&token),
share_use_case.get_shared_link_with_unlock(&token, unlock_jwt.as_deref()), share_use_case.get_shared_link_meta_with_unlock(&token, unlock_jwt.as_deref()),
); );
match item { match item {
@@ -639,7 +639,7 @@ pub async fn list_share_contents_subfolder(
path = "/api/s/{token}/file/{file_id}", path = "/api/s/{token}/file/{file_id}",
params( params(
("token" = String, Path, description = "Share token"), ("token" = String, Path, description = "Share token"),
("file_id" = String, Path, description = "File ID (must be inside the share)") ("file_id" = String, Path, description = "File ID (the shared item itself, or a file inside the shared folder's subtree)")
), ),
responses( responses(
(status = 200, description = "File content (or 206 for Range request)"), (status = 200, description = "File content (or 206 for Range request)"),
@@ -94,6 +94,19 @@ pub async fn basic_auth_middleware(
let (raw_username, password) = let (raw_username, password) =
parse_basic_auth(auth_header).ok_or(NextcloudAuthError::Unauthorized)?; parse_basic_auth(auth_header).ok_or(NextcloudAuthError::Unauthorized)?;
// Canonicalise the whole Basic-Auth username to lowercase.
//
// Usernames are canonical (lowercase) in the DB post-migration
// (`docs/plan/username-lowercase.md`), and NC / DAVX5 clients that
// cached URLs from before the migration keep sending `Alice:pass`
// — the server continues to accept that indefinitely by
// lowercasing here. Safe for the multi-drive `user~drive_uuid`
// composite because UUID hex `[0-9a-f-]` lowercases to itself.
//
// ASCII-only by `validate_username`'s charset check, so
// `to_ascii_lowercase` is deterministic and locale-safe.
let raw_username = raw_username.to_ascii_lowercase();
// ── Multi-drive composite-username parse ──────────────────────── // ── Multi-drive composite-username parse ────────────────────────
// POC wire shape: `{username}~{drive_marker}` may appear in the // POC wire shape: `{username}~{drive_marker}` may appear in the
// Basic Auth header. `~` was chosen because it needs no URL // Basic Auth header. `~` was chosen because it needs no URL
@@ -319,7 +332,13 @@ pub fn parse_basic_auth(header_value: &str) -> Option<(String, String)> {
let decoded = String::from_utf8(decoded).ok()?; let decoded = String::from_utf8(decoded).ok()?;
let (user, pass) = decoded.split_once(':')?; let (user, pass) = decoded.split_once(':')?;
Some((user.to_string(), pass.to_string())) // Canonicalise the username to lowercase here too, so any caller
// that reaches for `parse_basic_auth` directly (bypassing the
// middleware wrapper) also sees the canonical form. Redundant with
// the middleware's explicit `to_ascii_lowercase` on `raw_username`
// — belt-and-braces to keep the invariant local to the parser too.
// See `docs/plan/username-lowercase.md § 4. NextCloud DAV surface`.
Some((user.to_ascii_lowercase(), pass.to_string()))
} }
#[cfg(test)] #[cfg(test)]
+15 -1
View File
@@ -107,7 +107,21 @@ fn extract_url_user(path: &str) -> Option<std::borrow::Cow<'_, str>> {
// common path allocates nothing; only a percent-encoded username owns. The // common path allocates nothing; only a percent-encoded username owns. The
// old `.into_owned()` forced a `String` on EVERY path-scoped NC DAV request // old `.into_owned()` forced a `String` on EVERY path-scoped NC DAV request
// (benches/ROUND19.md §M7). The caller compares by slice. // (benches/ROUND19.md §M7). The caller compares by slice.
urlencoding::decode(user_seg).ok() //
// Lowercase before returning so cached client URLs like
// `/dav/files/Alice/...` compare equal to the canonical
// (lowercase) `session.raw_username`. See
// `docs/plan/username-lowercase.md § 4. NextCloud DAV surface`.
//
// The lowercase transform always allocates (`to_ascii_lowercase`
// on a `str` returns `String`). Trades the "Cow::Borrowed common
// path" of the ROUND19 optimisation for correctness of the case-
// insensitive comparison at line 157 — a `&str` compare with a
// borrowed segment against a lowercase `session.raw_username`
// would silently mismatch for `Alice`. The alloc is one small
// String per NC DAV request; the correctness win is worth it.
let decoded = urlencoding::decode(user_seg).ok()?;
Some(std::borrow::Cow::Owned(decoded.to_ascii_lowercase()))
} }
/// Axum extractor: the shared handle to the request's [`NcSession`]. /// Axum extractor: the shared handle to the request's [`NcSession`].
+4
View File
@@ -100,6 +100,10 @@ fn make_socket(addr: &SocketAddr, reuse_port: bool) -> std::io::Result<Socket> {
if reuse_port { if reuse_port {
socket.set_reuse_port(true)?; socket.set_reuse_port(true)?;
} }
// SO_REUSEPORT is Unix-only; on Windows the flag is accepted but inert.
// Consume the parameter so `-D warnings` stays clean on Windows builds.
#[cfg(windows)]
let _ = reuse_port;
// Disable Nagle's algorithm — send small responses (JSON, PROPFIND) // Disable Nagle's algorithm — send small responses (JSON, PROPFIND)
// immediately instead of waiting up to 40ms for coalescing. // immediately instead of waiting up to 40ms for coalescing.
socket.set_tcp_nodelay(true)?; socket.set_tcp_nodelay(true)?;
+103
View File
@@ -0,0 +1,103 @@
# status.md — 计划与进度记录
> 本文件由 agent 自动维护,规则见 [AGENTS.md](AGENTS.md) 末尾的"本地 fork 维护规则"一节。
> 新条目加在"进行中"区域顶部;完成后移入"已完成"。
## 进行中
(暂无)
## 已完成
### [2026-09-19] 缩略图路径 DB 点查缓存(find_attached_blob 进程内缓存)
- **状态**: 已完成(commit `d33d1932`;cargo fmt --check ✓;clippy --all-features --all-targets -D warnings 0 警告 ✓;`cargo test --lib` 927 通过 0 失败,含新增 5 个缓存契约测试 ✓)
- **计划**: 给缩略图热路径(ETag 计算 `thumbnail_content_id` + tier 2b)每次请求都要打的 `find_attached_blob` DB 点查加进程内缓存(moka::future + try_get_with,正+负缓存,Err 不入缓存;写路径成功后失效;删除经 `ThumbnailRefreshHook::on_file_deleted` 搭车失效 + 60s TTL 兜底)。缓解"每次进照片墙 = 每张可见图 1-2 次 DB 点查"的负载。
- **改动文件**:
- `src/infrastructure/services/dedup_service.rs` — 模块顶 `ATTACHED_BLOB_CACHE_*` 常量 + `AttachedBlobKey` + `AttachedLookupFault`;`attached_blob_cache` 字段/`build_attached_blob_cache`;`find_attached_blob` 缓存包装(SQL 下移 `find_attached_blob_uncached` 返回 `sqlx::Result`,Err 永不入缓存);`store_attached_blob`/`store_attached_blob_if_absent` 成功路径失效(`Inserted` 臂失效 + AlreadyPresent 读回注释);`invalidate_attached_blobs_for_file`;5 个纯内存单测
- `src/infrastructure/services/thumbnail_service.rs` — `ThumbnailRefreshHook::on_file_deleted` 搭车失效(补 `self.dedup.clone()`)
- `docs/architecture/caching.md` — Layer 1 表新增 Attached blob 行 + "The attached-blob cache" 小节(键/正负缓存/两条诚实规则)
- `docs/architecture/derived-and-attached-blobs.md` — Lifecycle 节新增"Reads are cached"段落(缓存失效协议)
- **仅本地文件**: 无新增(`status.md` 本身)
- **上游冲突风险**: 低 — 两个上游文件均为局部追加,无重排
- **备注**: 本机此前无 Rust 工具链,本次顺带装好 rustup stable 1.98.1(minimal+clippy+rustfmt)、VS Build Tools 2022(MSVC 14.44 + SDK 10.0.26100)、Node 前端构建(`static-dist/` 已生成)——后续 agent 可直接跑 `cargo` 检查;注意 shell 无管理员权限,提权操作需 UAC 确认
### [2026-09-19] 文档更新:公开分享端点与落地页行为
- **状态**: 已完成
- **计划**: 按 AGENTS.md 的文档约定,把本会话的分享相关修复/功能同步进架构文档与用户指南
- **改动文件**:
- `docs/architecture/share-integration.md` — 公开路由表补全(download/contents/file/zip 共 6 条);新增 "/file/{file_id} 文件作用域"(file 分享仅限分享项本身、folder 分享限子树、其余 404 反枚举)与"落地页 meta 富化"(mime_type/size 仅展示、查询失败不失败响应)两小节
- `docs/guide/sharing.md` — 新增 "What recipients see" 用户向小节(单文件内联预览/视频流式拖动、文件夹浏览 + ZIP)
- **上游冲突风险**: 低 — 两文档均属低频改动区
### [2026-09-19] 文件页刷新按钮
- **状态**: 已完成(vitest 467 通过;svelte-check 0 错误;我改的文件 prettier/eslint 全绿)
- **计划**: actions 工具栏加手动刷新,点击重拉当前文件夹列表
- **改动文件**:
- `frontend/src/routes/files/[...path]/+page.svelte` — "New folder" 右侧新增刷新按钮:`load(true)` 重置分页拉第 1 页,`loading` 时禁用,图标 `repeat`(沿用 AdminJobsPanel 先例),testid `files-refresh-btn`
- `frontend/static/locales/{en,zh,zh-TW}.json` — `common.refresh`(刷新 / 重新整理)
- `frontend/src/routes/files/page.test.ts` — 点击刷新 → `fetchFolderPage` 第二次调用
- **上游冲突风险**: 中 — files 页与 locales 是上游活跃区,但改动面小
### [2026-09-19] 文件页文件夹内容统计
- **状态**: 已完成(验证同上)
- **计划**: 进入文件夹时在面包屑旁显示内容统计
- **改动文件**:
- `frontend/src/routes/files/[...path]/+page.svelte` — `folderStat` derived(对 `rlItems` 即点文件过滤后的展示列表计数;分页未完时数字尾随 "+");统计 span 渲染在 breadcrumb snippet 内(`.rl-breadcrumb` 本身是 flex);空文件夹不显示
- `frontend/static/locales/{en,zh,zh-TW}.json` — `files.folder_stat`(其余 13 语言回退英文)
- `frontend/src/routes/files/page.test.ts` — 精确计数 / 分页 "+" 两条测试
- **上游冲突风险**: 中 — 同上
### [2026-09-19] 开发环境修复 + 测试 locale 固定
- **状态**: 已完成
- **计划**: 本机装 Node 跑前端检查;修复暴露出的换行符与 locale 问题
- **改动文件**:
- `frontend/src/lib/utils/time.test.ts` — `vi.spyOn(Intl, 'RelativeTimeFormat')` 固定 `en` locale(time.ts 用运行时默认 locale,中文系统上 `/second/` 等英文断言必挂);注意 mock 需普通函数(time.ts 经 `new` 调用)
- 环境级(无仓库 diff): winget 装 Node 26.7.0;`core.autocrlf=false`(仓库级)+ `git rm --cached -r . && git reset --hard` 全量重写工作区为 LF——修复 276 个文件的 Prettier 假报错
- **仅本地文件**: 无新增
- **上游冲突风险**: 低 — LF 归一化后与上游(CI 全 LF)一致;建议后续给 `.gitattributes` 加 `* text=auto eol=lf`(未做,待定)
### [2026-09-19] 文件页模糊筛选搜索 + 批量操作共享化
- **状态**: 已完成(`npm run check` 0 错误 0 警告;`vitest run` 467 个测试全部通过)
- **计划**:
1. 文件页(`/files/[...path]`)新增搜索过滤栏:关键词(防抖)+ 类型/大小/时间预设 + 递归开关,
筛选生效时列表切换为当前文件夹的递归搜索(`GET /api/search` 的 `folder_id`+`recursive`,
空 `query` 后端视为匹配全部),支持结果多选/全选 + 批量收藏/移动/复制/下载/删除
2. 批量操作提取为共享 composable,`/search` 结果页同步接入选择 + 批量操作(项目去重规范)
3. 两处页内裸 `apiFetch`(favorites/batch、batch/download)改为 endpoint 封装
- **改动文件**:
- 新增(仅本地):
- `frontend/src/lib/components/SearchFilterBar.svelte` — 搜索过滤栏组件(关键词 + 高级筛选 + 递归开关)
- `frontend/src/lib/composables/useResourceActions.svelte.ts` — 共享批量操作(收藏/下载/删除/移动/复制 + MoveDialog 状态)
- `frontend/src/lib/utils/searchFilters.ts` — 类型/大小/时间预设模型 → `SearchOptions` 映射
- `frontend/src/lib/utils/mapLimit.ts` — 有界并发工具(自文件页提取)
- 以上各文件的 Vitest 测试(`*.test.ts`)
- 修改(上游文件):
- `frontend/src/routes/files/[...path]/+page.svelte` — 搜索模式状态/`runSearch`/模式切换/Escape 优先级/深链兼容;批量 handler 换用 composable
- `frontend/src/routes/search/+page.svelte` — 接入选择 + 批量操作;筛选逻辑改用共享 searchFilters;MoveDialog 绑定 composable
- `frontend/src/lib/api/endpoints/batch.ts` — 新增 `downloadBatch()`
- `frontend/src/lib/api/endpoints/favorites.ts` — 新增 `addFavoritesBatch()`
- `frontend/src/routes/files/page.test.ts`、`frontend/src/routes/search/page.test.ts` — mock 补齐 + 新增搜索模式/批量删除测试
- `frontend/static/locales/*.json`(16 个语言文件)— 新增 `filter.*` 6 个 key(zh/zh-TW 为真实翻译)
- **仅本地文件**: `status.md`、上述新增的 4 个源文件及其测试
- **上游冲突风险**: 高 — locales 与两个页面文件是上游活跃区;页面文件改动较大(files 页 ~490 行、search 页 ~170 行 diff),合并时需逐块核对本地意图
- **设计要点**(合并上游时用于核对行为):
- 批量操作参数化接口:`getItems/getSelected/clearSelection/onChanged/afterDelete` 回调注入;
文件页 `getItems` 按模式切换(`searchActive ? searchItems : orderedItems`),`onChanged` 按模式重跑搜索或重载目录
- 删除保持 per-item `mapLimit(ids, 6)` 行为(后端 `POST /api/batch/trash` 批量软删存在,切换记为可选后续)
- 搜索分页 limit 50;"全选"只覆盖已加载页(与 ResourceList 既有语义一致)
- 文件页搜索 wire 层 `type` 排序映射为 `name`(`SortBy` 无 type),`modified_at` 映射为 `updated_at`;始终传 sortBy 不传 relevance
- Escape 优先级:清除选中 → 清除筛选;过滤输入框内 Escape 自行处理并 stopPropagation
### [2026-09-15] 单文件分享无法流式预览修复
- **状态**: 已完成(commit `68e21f4b`;本机无 Rust 环境,fmt/clippy/api-test 未能在本地运行——**push 前需在有 Rust 的环境跑 `just check` + `just api-test`**)
- **计划**: 修复单文件分享落地页预览窗口无内容/按钮失效——根因是 `resolve_folder_share` 对 `item_type != "folder"` 一律拒绝,`/api/s/{token}/file/{id}` 对 file 分享返回 400
- **改动文件**:
- `src/application/services/share_browse_service.rs` — `assert_file_in_share` 改为按 `item_type` 分支:file 分享仅接受 `file_id == share.item_id`;folder 分享维持 ltree 子树校验;其余一律 404(反枚举,保持与"文件不存在"同形)
- `src/interfaces/api/handlers/share_handler.rs` — OpenAPI `file_id` 参数描述同步("the shared item itself, or a file inside the shared folder's subtree")
- `tests/api/public_shares.hurl` — 8b 节扩展:file 分享 token 流式取自身 item(200 + disposition)+ 局外人 id 404 断言
- **仅本地文件**: 无
- **上游冲突风险**: 中 — `share_browse_service.rs` 属上游安全敏感活跃区;改动集中在单个函数,冲突时按"file 分享限自身、folder 限子树、其余 404"核对意图
## 上游合并记录
(暂无。首次合并前先 `git remote add upstream <上游仓库地址>`。)
+160
View File
@@ -0,0 +1,160 @@
# =============================================================
# OxiCloud — usernames are case-insensitive (silent lowercase on ingest)
# =============================================================
# Pin for `docs/plan/username-lowercase.md`. The plan makes usernames
# case-insensitive by canonicalising to lowercase in
# `User::validate_username`. This file covers the wire surface:
#
# 1. Registration with a mixed-case username lands as lowercase.
# 2. Login by the ORIGINAL mixed-case string succeeds (server
# normalises on lookup).
# 3. Login by ALL-CAPS of the same name also succeeds.
# 4. Login by the canonical lowercase form succeeds.
# 5. Profile rename to a mixed-case name lands as lowercase.
# 6. NC Basic Auth accepts every case variant of the same account.
#
# Character-set + boundary rules (no leading dot, no `@`, etc.) are
# NOT re-tested here — that's the Rust unit-test surface. This file
# only pins the end-to-end normalization behaviour.
#
# Runs after `setup.hurl` (admin exists). Uses a self-contained user
# to avoid interfering with other scenarios.
# =============================================================
# ─────────────────────────────────────────────────────────────
# Step 1 — Register with a mixed-case username. Expect the server
# to silently store the lowercase form.
# ─────────────────────────────────────────────────────────────
POST {{base_url}}/api/auth/register
Content-Type: application/json
{
"username": "MixedCaseUser",
"email": "mixedcase@example.com",
"password": "MixedCasePassword1!"
}
HTTP 200
# ─────────────────────────────────────────────────────────────
# Step 2 — Log in with the ORIGINAL mixed-case string. Server
# should normalise on lookup and accept.
# ─────────────────────────────────────────────────────────────
POST {{base_url}}/api/auth/login
Content-Type: application/json
{ "username": "MixedCaseUser", "password": "MixedCasePassword1!" }
HTTP 200
[Asserts]
jsonpath "$.access_token" isString
# The `/auth/me` response inside the login reply exposes the canonical
# stored username. Post-migration, it MUST be lowercase regardless of
# what the caller typed at registration.
jsonpath "$.user.full.user.username" == "mixedcaseuser"
# ─────────────────────────────────────────────────────────────
# Step 3 — Log in with ALL-CAPS. Same account, different casing.
# ─────────────────────────────────────────────────────────────
POST {{base_url}}/api/auth/login
Content-Type: application/json
{ "username": "MIXEDCASEUSER", "password": "MixedCasePassword1!" }
HTTP 200
[Asserts]
jsonpath "$.user.full.user.username" == "mixedcaseuser"
# ─────────────────────────────────────────────────────────────
# Step 4 — Log in with the canonical lowercase form. Same account.
# Capture the token for the profile-rename step below.
# ─────────────────────────────────────────────────────────────
POST {{base_url}}/api/auth/login
Content-Type: application/json
{ "username": "mixedcaseuser", "password": "MixedCasePassword1!" }
HTTP 200
[Captures]
mixed_token: jsonpath "$.access_token"
[Asserts]
jsonpath "$.user.full.user.username" == "mixedcaseuser"
# ─────────────────────────────────────────────────────────────
# Step 5 — Rename via profile PATCH. New name is mixed-case; server
# must store it as lowercase. Same rule as registration, applied on
# the mutation path.
# ─────────────────────────────────────────────────────────────
PATCH {{base_url}}/api/auth/me
Authorization: Bearer {{mixed_token}}
Content-Type: application/json
{ "username": "RenamedTarget" }
HTTP 200
[Asserts]
# The response echoes the stored (canonical) form.
jsonpath "$.full.user.username" == "renamedtarget"
# Old (pre-rename) username no longer resolves — login fails 401.
POST {{base_url}}/api/auth/login
Content-Type: application/json
{ "username": "MixedCaseUser", "password": "MixedCasePassword1!" }
HTTP 401
# New (post-rename) mixed-case login succeeds.
POST {{base_url}}/api/auth/login
Content-Type: application/json
{ "username": "RENAMEDTARGET", "password": "MixedCasePassword1!" }
HTTP 200
[Asserts]
jsonpath "$.user.full.user.username" == "renamedtarget"
# ─────────────────────────────────────────────────────────────
# Step 6 — NextCloud Basic Auth accepts every case variant.
# The middleware lowercases the decoded username on the auth path,
# and `extract_url_user` lowercases the URL segment. A cached
# client URL like `.../dav/files/RenamedTarget` continues to work
# indefinitely across the migration.
#
# Uses PROPFIND `Depth: 0` on `/remote.php/dav/files/{user}/` — a
# well-formed request that touches Basic Auth + URL parse + chroot
# resolve in one hop. 207 Multi-Status is the expected success shape.
# ─────────────────────────────────────────────────────────────
PROPFIND {{base_url}}/remote.php/dav/files/renamedtarget/
Depth: 0
[BasicAuth]
renamedtarget: MixedCasePassword1!
HTTP 207
PROPFIND {{base_url}}/remote.php/dav/files/RenamedTarget/
Depth: 0
[BasicAuth]
RenamedTarget: MixedCasePassword1!
HTTP 207
PROPFIND {{base_url}}/remote.php/dav/files/RENAMEDTARGET/
Depth: 0
[BasicAuth]
RENAMEDTARGET: MixedCasePassword1!
HTTP 207
# Mixed case in URL, lowercase in Basic Auth — still works because
# both surfaces normalise before comparison.
PROPFIND {{base_url}}/remote.php/dav/files/RenamedTarget/
Depth: 0
[BasicAuth]
renamedtarget: MixedCasePassword1!
HTTP 207
+18
View File
@@ -197,6 +197,24 @@ HTTP 200
jsonpath "$.item_type" == "file" jsonpath "$.item_type" == "file"
# The public landing page's inline media preview streams the shared file
# through /api/s/{token}/file/{file_id} — the requested file IS the shared
# item here, so the AuthZ gate must accept it (Range-aware 200, inline
# disposition so <video>/<img> can render it).
GET {{base_url}}/api/s/{{file_share_token}}/file/{{shared_file_id}}
HTTP 200
[Asserts]
header "Content-Disposition" contains "hello.txt"
# A file id that is NOT the shared item must 404 on a file-share token —
# same anti-enumeration shape as the folder-share probe above.
GET {{base_url}}/api/s/{{file_share_token}}/file/{{outsider_file_id}}
HTTP 404
# ───────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────
# 9 — Mint a password-protected share on the same folder. # 9 — Mint a password-protected share on the same folder.
# ───────────────────────────────────────────────────────────── # ─────────────────────────────────────────────────────────────
+65
View File
@@ -254,6 +254,71 @@ ATTACHED_2=$(sql "SELECT count(*) FROM storage.file_attached_blobs WHERE file_id
[[ "$ATTACHED_2" == "$ATTACHED_AFTER" ]] || fail "re-run duplicated attached rows" [[ "$ATTACHED_2" == "$ATTACHED_AFTER" ]] || fail "re-run duplicated attached rows"
log "re-run is a no-op: rows and refcounts unchanged." log "re-run is a no-op: rows and refcounts unchanged."
# ── 5c. store_attached_blob same-content guard (regression test) ─────────────
#
# The PUT /api/files/{id}/thumbnail/{size} endpoint calls
# `dedup_service::store_attached_blob` directly — no pre-check like
# `thumb_attached_import_service` does. A same-content re-PUT is the
# ONLY current public surface that exercises the ref-balance branch
# added to `store_attached_blob` after the Sept-2026 manifest-drift
# investigation:
#
# if let Some((old_hash,)) = previous {
# // same-content: cancel `store_from_stream`'s spurious +1
# // different-hash: release the superseded blob's ref
# remove_reference(&old_hash)
# }
#
# Before the fix, same-content re-PUT would leak +1 on the manifest's
# ref_count every time (store_from_stream incremented, guard skipped
# the decrement when old == new). This test PUTs the same thumbnail
# bytes twice and asserts the manifest's ref_count is unchanged.
#
# Uses variant "icon" so we don't collide with the row the import
# job populated above (variant "preview"), keeping the two flows
# independent. Server re-encodes to JPEG deterministically, so both
# PUTs produce byte-identical manifest content.
log "5c. Same-content re-PUT via API does not churn refcount"
# Round 1: first PUT populates the row (INSERT — previous=None, guard
# doesn't fire). This is the fresh-ingest path; ref_count becomes 1.
curl -sf -X PUT \
-H "$AUTH" \
-H "Content-Type: image/jpeg" \
--data-binary "@$UPLOADED_THUMB" \
"$base_url/api/files/$FILE_ID/thumbnail/icon" \
>/dev/null || fail "5c: first PUT of thumbnail (variant=icon) failed"
GUARD_HASH=$(sql "SELECT blob_hash FROM storage.file_attached_blobs \
WHERE file_id='$FILE_ID' AND kind='preview' AND variant='icon' \
LIMIT 1;")
[[ -n "$GUARD_HASH" ]] || fail "5c: first PUT did not land a file_attached_blobs row"
GUARD_REFS_BEFORE=$(sql "SELECT ref_count FROM storage.chunk_manifests WHERE file_hash='$GUARD_HASH';")
# Legacy blobs (pre-CDC) don't have a chunk_manifests row — skip
# the test in that case rather than fail on an unrelated path.
if [[ -z "$GUARD_REFS_BEFORE" ]]; then
log "5c: attached blob is on the legacy path (no manifest) — guard test skipped (targets CDC path)"
else
# Round 2: second PUT with the SAME bytes. UPSERT-UPDATE fires,
# previous.blob_hash == new attached_hash, and the guard MUST
# cancel `store_from_stream`'s +1. Without the fix, refcount
# would go from 1 → 2 here.
curl -sf -X PUT \
-H "$AUTH" \
-H "Content-Type: image/jpeg" \
--data-binary "@$UPLOADED_THUMB" \
"$base_url/api/files/$FILE_ID/thumbnail/icon" \
>/dev/null || fail "5c: same-content re-PUT of thumbnail failed"
GUARD_REFS_AFTER=$(sql "SELECT ref_count FROM storage.chunk_manifests WHERE file_hash='$GUARD_HASH';")
[[ "$GUARD_REFS_AFTER" == "$GUARD_REFS_BEFORE" ]] \
|| fail "5c: same-content re-PUT churned refcount: $GUARD_REFS_BEFORE → $GUARD_REFS_AFTER (guard regressed?)"
log "5c: same-content re-PUT stable, refcount=$GUARD_REFS_AFTER"
fi
# ── 5b. Deletion: the destructive half, and the only one that can lose data # ── 5b. Deletion: the destructive half, and the only one that can lose data
# #
# Everything above is additive and recoverable. This unlinks files after a # Everything above is additive and recoverable. This unlinks files after a