Compare commits
16 Commits
c4dfa9ccf2
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
 cjw
|
b7640e9be4 | ||
|
cjw
|
d33d1932b6 | ||
|
cjw
|
68e21f4bef | ||
|
cjw
|
d677f92b7f | ||
|
cjw
|
5f9f91ee2f | ||
|
cjw
|
6ee26e46e6 | ||
 Dionisio Pozo
|
4d067b3fc6 | ||
|
Dionisio Pozo
|
fd9e1e669f | ||
|
Dionisio Pozo
|
b5648b82f9 | ||
|
Edouard Vanbelle
|
a95a6b106c | ||
|
Edouard Vanbelle
|
0b9e8bfe23 | ||
|
Edouard Vanbelle
|
03f2956457 | ||
|
Edouard Vanbelle
|
47346b9309 | ||
|
Dionisio Pozo
|
4840930767 | ||
|
Dionisio Pozo
|
69fc88982e | ||
|
Edouard Vanbelle
|
ca85ac7307 |
@@ -2,6 +2,72 @@
|
||||
|
||||
This file provides guidance to coding agents (Claude Code, Codex, Cursor, Aider, …) working with this repository. Claude Code reads it via `@AGENTS.md` in `CLAUDE.md`.
|
||||
|
||||
# Purpose — what OxiCloud is, and what it is not
|
||||
|
||||
Read this before designing anything. Most "should we…" questions are answered by
|
||||
the scale target rather than by taste.
|
||||
|
||||
- **Open source, and it stays that way.** MIT (`LICENSE`). This is a constraint on
|
||||
what you may add, not a footnote: dependencies must be license-compatible — a
|
||||
GPL/AGPL crate or npm package would force the whole project to relicense, and
|
||||
that is not on the table. No feature may be gated behind a licence key or an
|
||||
"enterprise edition", and nothing core may hard-depend on a proprietary service
|
||||
or SDK. Vendored frontend assets (`frontend/static/vendors/`) carry the same
|
||||
rule; record the licence when vendoring.
|
||||
- **Self-hosted**, for an individual or an enterprise. The operator is not an SRE
|
||||
team: defaults must be safe, failures loud, and nothing may silently depend on a
|
||||
cloud service.
|
||||
- **Target scale is up to ~10k users.** Design against that number in both
|
||||
directions. Do not build million-user machinery (sharding, eventual consistency,
|
||||
service decomposition) for a load that will never arrive; equally, never ship
|
||||
anything O(users) per request, or a table scan that is fine at 50 users and
|
||||
fatal at 10k.
|
||||
- **Not a mass hoster.** OxiCloud does not claim to serve millions of users on one
|
||||
deployment, and trade-offs should not pretend otherwise.
|
||||
- **Decentralised by intent.** Many instances federating beats one large instance —
|
||||
OpenCloudMesh is one route. Prefer designs that survive "this is one of many
|
||||
instances" over ones assuming a single authoritative deployment.
|
||||
|
||||
Targets:
|
||||
|
||||
- **Feature ambition: Google Workspace / Office 365.** Breadth of capability is a
|
||||
goal, not scope creep.
|
||||
- **Collaboration is the main feature axis.** OxiCloud is not a personal backup
|
||||
drive that happens to have sharing bolted on — sharing, shared drives, grants,
|
||||
co-editing (WOPI) and live updates are the product. When choosing what to build
|
||||
or how to build it, the multi-user case is the primary one, not the case to
|
||||
generalise to later. A feature that works only for a single owner is unfinished.
|
||||
- **Customer target: NextCloud users.** Hence the NextCloud-compatible API surface
|
||||
(`/remote.php`, `/ocs`, `/status.php`) — compatibility is a feature, and breaking
|
||||
it costs adopters.
|
||||
|
||||
## Design axes
|
||||
|
||||
Four things decide an open design question. **Security and resilience are
|
||||
absolute** — they are not traded against anything. Performance is measured against
|
||||
the 10k target. Privacy is a direction with a stated endpoint.
|
||||
|
||||
- **Resilience.** This is a storage product: **no data loss, no data corruption,
|
||||
ever.** Anything that can silently drop or alter bytes is a top-severity defect,
|
||||
not a trade-off. In practice that means: a job that skips work must never report
|
||||
success (pause at a cursor instead — `docs/plan/jobs-handling-recoverable-error.md`);
|
||||
a read failure is never proof that data is absent; content-addressing and
|
||||
ref-counting are load-bearing, not decoration; and consistency checks are
|
||||
discovery-only unless repair is explicitly requested.
|
||||
- **Security.** Prefer deny-by-default over assert-later; a guarantee enforced by
|
||||
the type system or the router beats one a reviewer must remember. AuthZ lives in
|
||||
the service layer, never in handlers. See `src/AGENTS.md` § AuthZ enforcement
|
||||
points.
|
||||
- **Performance.** Measure against 10k users, not a dev instance. The hot paths are
|
||||
listing, thumbnails and auth — a per-row query or an extra round trip there is a
|
||||
real regression even when it looks harmless.
|
||||
- **Privacy.** When the backend belongs to a third party (S3, Azure), encryption at
|
||||
rest is a *should-have*; **end-to-end encryption is the target.** Designs that
|
||||
assume the server can always read plaintext will have to be undone — the `Vault`
|
||||
drive kind is reserved for the E2E case.
|
||||
|
||||
Where two conflict, resilience and security win, and the cost is documented.
|
||||
|
||||
# Architecture
|
||||
|
||||
This project is split into two parts:
|
||||
@@ -258,3 +324,66 @@ CI runs the same `npm run check` (plus Vitest) — commits that fail will not me
|
||||
- Leave debug `console.log` statements in code
|
||||
- Use raw color values in CSS — always use CSS custom properties
|
||||
- Commit without passing all linters (`npm run check` for the frontend; `cargo fmt` + `cargo clippy` for the backend)
|
||||
|
||||
# 本地 fork 维护规则（Local fork rules）
|
||||
|
||||
> 本节是仅本地追加的内容，不属于上游 OxiCloud。与上游合并时，若本节之外的部分发生冲突，
|
||||
> 以上游为准；本节始终保留在文件末尾以减少冲突面。
|
||||
|
||||
## 背景
|
||||
|
||||
本仓库是开源项目 OxiCloud 的本地副本，上游会持续更新。本地修改必须
|
||||
**可追溯、可合并**：任何时候都要能知道"我们改了什么"，以便与上游主分支合并。
|
||||
|
||||
## 规则 1：计划与进度必须记录在 `status.md`
|
||||
|
||||
- 每次接到非琐碎任务，**开始前**先在 `status.md` 顶部（"进行中"区域）写下计划。
|
||||
- `status.md` 条目格式（每条任务一个区块）：
|
||||
|
||||
```markdown
|
||||
### [YYYY-MM-DD] 任务标题
|
||||
- **状态**: 进行中 / 已完成 / 已放弃（写明原因）
|
||||
- **计划**: 要做什么、分几步
|
||||
- **改动文件**: 列出修改/新增的上游文件（相对路径）+ 一句话说明
|
||||
- **仅本地文件**: 新增的不属于上游的文件（合并时无需处理）
|
||||
- **上游冲突风险**: 高 / 中 / 低，以及可能与上游哪些文件冲突
|
||||
```
|
||||
|
||||
- 状态只允许进行中/已完成/已放弃三种；完成的任务移入"已完成"区域，保留记录不删除。
|
||||
|
||||
## 规则 2：每次修改后立即更新 `status.md`
|
||||
|
||||
- **不需要用户提醒**。任何一次代码/文档修改完成后，agent 必须同步更新
|
||||
`status.md` 中对应条目的状态、改动文件列表和冲突风险。
|
||||
- 即使任务中途被打断，也要把当前进度写清（做到哪一步、剩下什么），保证
|
||||
任何 agent（或人）读了 `status.md` 就能接手。
|
||||
|
||||
## 规则 3：与上游主分支合并
|
||||
|
||||
- **小步提交**：一个任务一个 commit（或少量 commit），commit message 说清楚改了什么。
|
||||
不要把多天的工作堆成一个巨型 commit，否则合并时无法选择性丢弃。
|
||||
- **少改上游文件**：能用新增文件解决的（新组件、新模块、新 endpoint）就不要改上游现有文件；
|
||||
必须改时尽量小而集中，并在 `status.md` 的"上游冲突风险"里注明。
|
||||
**`AGENTS.md` 本身也因此只允许在文件末尾追加内容，不得改动上游已有的章节。**
|
||||
- **不改无关格式**：不要顺手重排上游代码、改无关 import 顺序——纯噪音，制造冲突。
|
||||
- 合并上游的流程：
|
||||
|
||||
```bash
|
||||
git remote add upstream <上游仓库地址> # 只需配置一次
|
||||
git fetch upstream
|
||||
git merge upstream/main # 或 rebase，按团队习惯；首次建议 merge
|
||||
# 解决冲突时：先读 status.md 的"改动文件"列表，逐个文件核对本地意图
|
||||
git status # 确认没有遗漏的冲突标记
|
||||
cargo fmt --all && cargo clippy --all-features --all-targets -- -D warnings
|
||||
just test
|
||||
```
|
||||
|
||||
- 合并完成后，在 `status.md` 新增一条"上游合并"记录：合并到的 upstream commit、
|
||||
解决过的冲突文件、是否有本地修改被上游覆盖/废弃。
|
||||
- 若上游已用别的方式实现了某个本地功能（导致本地补丁不再需要），在 `status.md`
|
||||
把对应条目标为"已放弃（上游已实现）"，并考虑回退本地补丁。
|
||||
|
||||
## 规则 4：其他
|
||||
|
||||
- `status.md` 属于仅本地文件，不向上游提 PR（除非团队明确决定）；
|
||||
`AGENTS.md` 中仅本节（"本地 fork 维护规则"）是本地内容，向上游提 PR 时应剔除。
|
||||
|
||||
@@ -176,6 +176,10 @@ export default defineConfig({
|
||||
{ text: "Authentication model", link: "/architecture/auth-model" },
|
||||
{ text: "Magic-link auth", link: "/architecture/magic-link-auth" },
|
||||
{ text: "Background jobs", link: "/architecture/jobs" },
|
||||
{
|
||||
text: "Message bus & notifications",
|
||||
link: "/architecture/message-bus-and-notifications",
|
||||
},
|
||||
{ text: "UI diagnostics", link: "/architecture/ui-diagnostics" },
|
||||
],
|
||||
},
|
||||
|
||||
@@ -16,8 +16,33 @@ The two layers are orthogonal — the moka caches shave query round-trips regard
|
||||
| Thumbnail cache | configurable | 1 000 | Generated WebP/AVIF thumbnails |
|
||||
| Image transcode | configurable | 500 | On-the-fly image transcoding results |
|
||||
| Blob hash | 30 s TTI | 5 000 | BLAKE3 hashes for dedup lookups |
|
||||
| Attached blob | 60 s TTL | 50 000 | `file_attached_blobs` row lookups on the thumbnail hot path (ETag + tier-2b, also the Nextcloud preview endpoint) |
|
||||
| Audio metadata | — | 2 000 | ID3 tags and duration |
|
||||
|
||||
### The attached-blob cache
|
||||
|
||||
Every thumbnail request pays a `storage.file_attached_blobs` point query
|
||||
before it can even answer "304 Not Modified" — the ETag names the attached
|
||||
blob's hash. A photos grid revalidating 60 thumbnails per visit means
|
||||
60+ point queries per browse. The cache sits in `DedupService` in front of
|
||||
that lookup (`find_attached_blob`), keyed by the row's `(file_id, kind,
|
||||
variant)` primary key, and caches **both directions**: `Some(row)` and
|
||||
`None` (most files have no attached preview, so the negative side is where
|
||||
most of the win is).
|
||||
|
||||
Two rules keep it honest:
|
||||
|
||||
- **DB faults are never cached.** The uncached lookup surfaces errors as
|
||||
`Err`; only a genuine `Ok(None)` fills a negative entry. A transient
|
||||
outage must not freeze "no attached blob" into place for a full TTL —
|
||||
a read failure is never proof that data is absent.
|
||||
- **TTL is the bound, not the invalidation strategy.** Writes invalidate
|
||||
eagerly — `store_attached_blob` / `store_attached_blob_if_absent` on
|
||||
success, deletions via `ThumbnailRefreshHook::on_file_deleted` (which
|
||||
all three production delete paths fire). The 60 s TTL only bounds what
|
||||
the process cannot see: bare SQL, the `copy_file_satellites` race
|
||||
window, a hypothetical second instance.
|
||||
|
||||
### How it works
|
||||
|
||||
1. **Read path:** check cache → if hit, return immediately (sub-ms); if miss, query PostgreSQL, populate cache, return
|
||||
|
||||
@@ -248,6 +248,19 @@ on `DELETE`. The trigger fires on DELETE only; replacing a preview
|
||||
updates `blob_hash` in place and the Rust path handles that reference
|
||||
swap.
|
||||
|
||||
**Reads are cached; the cache never outlives the truth by design.**
|
||||
`DedupService::find_attached_blob` — the lookup the thumbnail ETag path
|
||||
pays on *every* request, 304 or not — reads through an in-process moka
|
||||
cache keyed by the row's PK, positive and negative entries alike. Two
|
||||
properties make that safe rather than merely fast: a DB fault is
|
||||
surfaced as an error and never fills a negative entry (a failed lookup
|
||||
is not a missing row), and every write path that can change an answer
|
||||
invalidates first — the two `store_attached_blob*` variants on success,
|
||||
deletes via `ThumbnailRefreshHook::on_file_deleted` after the CASCADE
|
||||
committed. The 60 s TTL exists for the residual cases the process
|
||||
cannot observe (bare SQL, `copy_file_satellites` racing a concurrent
|
||||
new file), not as the primary coherence mechanism.
|
||||
|
||||
**Writing a derived row requires its source to exist.**
|
||||
`store_derived_blob` guards the insert with an `EXISTS` on
|
||||
`chunk_manifests`/`blobs`. Without it, a row written just after its
|
||||
|
||||
@@ -77,3 +77,4 @@ src/
|
||||
- [Backend Storage →](/architecture/backend-storage)
|
||||
- [Derived and Attached Blobs →](/architecture/derived-and-attached-blobs) — thumbnails, transcodes and uploaded previews: why content-keyed and file-keyed artifacts need separate tables
|
||||
- [Background Jobs →](/architecture/jobs)
|
||||
- [Message Bus & Notifications →](/architecture/message-bus-and-notifications) — real-time WebSocket bus (topics, AuthZ scopes, tab-visibility grace-close), persistent notifications (bell), AsyncAPI vs OpenAPI schema ownership
|
||||
|
||||
@@ -0,0 +1,645 @@
|
||||
# Message Bus & Persistent Notifications
|
||||
|
||||
OxiCloud has two coupled subsystems that together power its real-time
|
||||
UX — a **live message bus** over WebSocket for "something just
|
||||
happened, refresh your view", and a **persistent notifications
|
||||
table** for "you need to know about this even if you weren't
|
||||
online." This document explains how both work, how they authenticate
|
||||
and authorize subscribers, and why the frontend deliberately drops
|
||||
the WebSocket while a tab is hidden.
|
||||
|
||||
Design docs the shipped code implements: [`docs/plan/message-bus.md`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/message-bus.md)
|
||||
+ [`docs/plan/templated-messages.md`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/templated-messages.md).
|
||||
|
||||
---
|
||||
|
||||
## The two channels
|
||||
|
||||
| | **Message bus (WebSocket)** | **Persistent notifications (REST + DB)** |
|
||||
|---|---|---|
|
||||
| Purpose | "Something changed, refresh your view" | "You need to know about this — later is fine" |
|
||||
| Transport | JSON-RPC 2.0 over `/api/rt/ws` | `GET/POST/DELETE /api/notifications/*` + `notif.notifications` table |
|
||||
| Delivery | Best-effort, in-memory, no replay | Durable, per-user rows, survive reboot / offline |
|
||||
| Payload | Thin "poke" facts (id + verb) | Full per-kind DTO with all render data |
|
||||
| Loss on disconnect | Yes (events during outage window are dropped) | No (rows are the source of truth) |
|
||||
| Schema owner | AsyncAPI (`resources/gen/asyncapi.json`) | OpenAPI (`resources/gen/openapi.json`) |
|
||||
|
||||
The two work together: an ingester that wants to notify a user
|
||||
writes **both** — the DB row (for durability + the bell's history)
|
||||
AND publishes a bus event on `user:{u}:notifications` (so online
|
||||
sessions refetch instantly instead of waiting for the next
|
||||
mount). The wire event on that topic is a **pure poke** — empty
|
||||
`data: {}`. The row's real content only ever crosses the REST wire.
|
||||
|
||||
---
|
||||
|
||||
## Schema ownership — AsyncAPI vs OpenAPI
|
||||
|
||||
The bus and the REST endpoints have separate wire specs. The rule
|
||||
the codebase adopts to keep them from drifting:
|
||||
|
||||
> **AsyncAPI defines the envelope + transport for clients.
|
||||
> OpenAPI defines the payload.**
|
||||
|
||||
Concretely:
|
||||
|
||||
| Type | Home | How it stays in sync with Rust |
|
||||
|---|---|---|
|
||||
| **Bus events** (`MessageBusEvent`, subscribe / unsubscribe frames, revoked notifications, envelope shape) | **AsyncAPI** — `resources/gen/asyncapi.json` | Hand-written in `src/bin/generate-asyncapi.rs` via `json!` macros; kept in lockstep with `MessageBusEvent`'s serde shape. Small drift risk — Rust is truth. |
|
||||
| **REST DTOs** (response bodies, request bodies, per-kind notification payloads) | **OpenAPI** — `resources/gen/openapi.json` | `#[derive(utoipa::ToSchema)]` on the Rust struct. Utoipa walks `#[utoipa::path(...)]` handlers + registered schemas. **No drift possible** — projection is derived from Rust. |
|
||||
| **Types on both wires** (rare; none today) | Would live as one Rust struct with both derives, or wait for single-source codegen | — |
|
||||
|
||||
### Why this split, not one unified spec
|
||||
|
||||
The instinct is to put the notification payload schema in AsyncAPI
|
||||
alongside the bus event that triggers a refetch. It looks cleaner
|
||||
until you realize the payload **never travels on the bus wire** —
|
||||
the bus event is `NotificationReceived` with empty `data: {}`, a
|
||||
pure cache-invalidation poke. The FE fetches the payload from
|
||||
`GET /api/notifications`, which is REST → OpenAPI's territory.
|
||||
Putting the payload schema in AsyncAPI would mean "documenting
|
||||
this shape on a transport it doesn't travel on" — a conceptual
|
||||
stretch that adds a drift risk for zero gain.
|
||||
|
||||
### Rejected alternatives
|
||||
|
||||
- **Dual-spec (same type declared in both AsyncAPI + OpenAPI).**
|
||||
Guaranteed drift unless both come from a single codegen. Nothing
|
||||
in the tooling today produces both, so we'd hand-maintain two
|
||||
copies of every shared type. Bug factory.
|
||||
- **Cross-spec `$ref`** — AsyncAPI 3.0 allows `"$ref":
|
||||
"openapi.json#/components/schemas/Foo"`, and utoipa's
|
||||
`components(schemas(...))` can publish orphan types (no
|
||||
`#[utoipa::path]` reference) so OpenAPI advertises "internal"
|
||||
schemas. Technically workable but: Modelina + Swagger UI + Redoc
|
||||
handle external refs inconsistently, OpenAPI stops being "the
|
||||
REST contract" and becomes "a general schema registry",
|
||||
reviewers get confused. Legal, fragile, avoided.
|
||||
- **Bus event carries the full payload** (revert the pure-poke
|
||||
design). Would put per-kind payload schemas in AsyncAPI as
|
||||
`MessageBusEvent::NotificationReceived { granter_id, resource_id,
|
||||
… }`. Rejected because the FE has to REST-fetch anyway (bell
|
||||
reads from DB for history + persistence), so the fields on the
|
||||
wire are dead weight — same-content overlap between the two
|
||||
specs, no consumer benefit.
|
||||
- **Session-resume tokens** (`rt.subscribe { since: N }` +
|
||||
server-side ring buffer). Would let the bus deliver missed rows
|
||||
directly on reconnect, saving one REST round-trip. Rejected for
|
||||
**backward compatibility with `OXICLOUD_MESSAGEBUS_ENABLE=false`**:
|
||||
ops who disable the WS rely on the bell falling back to REST;
|
||||
bus-only replay would leave those deployments with no catch-up
|
||||
path. The REST `?after=` cursor works in every mode (bus on, bus
|
||||
off, network gap); the bus stays purely "instant-poke".
|
||||
|
||||
### What this looks like in the tree
|
||||
|
||||
- `src/application/ports/message_bus_ports.rs` — `MessageBusEvent`
|
||||
enum (Rust source of truth for bus wire shapes).
|
||||
- `src/bin/generate-asyncapi.rs` — projects those Rust variants
|
||||
into `resources/gen/asyncapi.json`.
|
||||
- `src/domain/entities/notification.rs` — `SharegrantedPayload`
|
||||
and its siblings, `#[derive(ToSchema)]`, source of truth for
|
||||
REST payload shapes.
|
||||
- `src/interfaces/api/mod.rs` — utoipa `#[openapi(components(schemas(SharegrantedPayload, ...)))]`
|
||||
registers the payload in OpenAPI even though `NotificationDto.payload`
|
||||
stays `serde_json::Value` on the response type. (FE type-narrows
|
||||
on `row.kind` and casts to the right shape.)
|
||||
- Nothing lives in both specs today.
|
||||
|
||||
### Adding a new bus event
|
||||
|
||||
1. Add a variant to `MessageBusEvent`.
|
||||
2. Add the variant to `generate-asyncapi.rs`'s `event_kind` enum
|
||||
and (if the variant has payload fields) a schema function.
|
||||
3. Regenerate AsyncAPI + FE DTOs via `just asyncapi` +
|
||||
`npm run gen:message-bus`.
|
||||
4. **Do not** add the variant to OpenAPI. Bus events don't
|
||||
travel on REST.
|
||||
|
||||
### Adding a new notification kind's payload
|
||||
|
||||
1. Add a Rust struct in `domain/entities/notification.rs` with
|
||||
`#[derive(Serialize, Deserialize, ToSchema)]`.
|
||||
2. Register it in `src/interfaces/api/mod.rs`'s
|
||||
`components(schemas(...))` list.
|
||||
3. Regenerate OpenAPI via `just openapi`.
|
||||
4. **Do not** add the struct to AsyncAPI. Notification payloads
|
||||
only cross the REST wire.
|
||||
|
||||
---
|
||||
|
||||
## Message bus
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ SERVICE LAYER │
|
||||
│ │
|
||||
│ ShareService.grant() ── after commit ──▶ bus.publish(…) │
|
||||
│ FileMgmtService.…() ── after commit ──▶ bus.publish(…) │
|
||||
│ NotificationService ── after commit ──▶ bus.publish(…) │
|
||||
│ Scheduler engine ── on run start/end ──▶ bus.publish │
|
||||
└──────────────────────────┬──────────────────────────────────┘
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ MessageBus port (application/ports/message_bus_ports.rs) │
|
||||
│ │
|
||||
│ InProcessMessageBus │
|
||||
│ DashMap<Topic, tokio::broadcast::Sender<Event>> │
|
||||
└──────────────────────────┬──────────────────────────────────┘
|
||||
│
|
||||
│ (optional replicator seam)
|
||||
▼
|
||||
NoopReplicator (v1)
|
||||
PgListenReplicator (deferred)
|
||||
BrokerReplicator (RabbitMQ / NATS, deferred)
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ WS handler (interfaces/api/handlers/rt_ws.rs) │
|
||||
│ │
|
||||
│ One session per socket: │
|
||||
│ HashMap<wire_key, Sub> + outbound mpsc │
|
||||
│ │
|
||||
│ rt.subscribe / rt.unsubscribe frames │
|
||||
│ rt.event / rt.revoked / rt.pong notifications │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Topics — a typed enum, not a string
|
||||
|
||||
```rust
|
||||
pub enum Topic {
|
||||
Folder(Uuid), // "folder:{uuid}"
|
||||
UserAuthz(Uuid), // "user:{uuid}:authz"
|
||||
UserNotifications(Uuid), // "user:{uuid}:notifications"
|
||||
Job(String), // "job:{name}"
|
||||
}
|
||||
```
|
||||
|
||||
Defined in `application/ports/message_bus_ports.rs`. Encoded to a
|
||||
stable dotted wire form; parsed back with strict validation. The
|
||||
wire form doubles as a routing key for future broker replicators
|
||||
(RabbitMQ topic exchanges, NATS subjects).
|
||||
|
||||
### Events
|
||||
|
||||
`MessageBusEvent` (same module) is the discriminated union of every
|
||||
payload a publisher can produce — `FileCreated`, `FolderMoved`,
|
||||
`AuthzChanged`, `JobRunStarted / Progress / Ended`,
|
||||
`NotificationReceived`, etc. Serde tags with `#[serde(tag = "event",
|
||||
rename_all = "snake_case")]`, so the wire is
|
||||
`{"event": "file_created", "file_id": "...", "actor": "..."}`.
|
||||
|
||||
Payloads are **thin facts**: the ID of the changed resource + the
|
||||
actor + the verb. Clients refetch details via REST if they need
|
||||
them. Keeps the AuthZ surface small (thin payloads can't leak
|
||||
fields the caller couldn't already read via REST) and keeps events
|
||||
well under any future broker's message-size cap.
|
||||
|
||||
### Wire protocol
|
||||
|
||||
JSON-RPC 2.0 over text frames. Full protocol in [`docs/plan/message-bus.md § Wire protocol`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/message-bus.md).
|
||||
|
||||
```jsonc
|
||||
// Client → server
|
||||
{"jsonrpc":"2.0","id":1,"method":"rt.subscribe","params":{"topic":"folder:abc-…"}}
|
||||
|
||||
// Server → client (ack)
|
||||
{"jsonrpc":"2.0","id":1,"result":{"subscribed":"folder:abc-…"}}
|
||||
|
||||
// Server → client (push, id-less notification)
|
||||
{"jsonrpc":"2.0","method":"rt.event","params":{
|
||||
"topic":"folder:abc-…",
|
||||
"event":"file_created",
|
||||
"data":{"file_id":"…","name":"…","parent_id":"…","actor":"…"}
|
||||
}}
|
||||
```
|
||||
|
||||
### Authentication for the WebSocket upgrade
|
||||
|
||||
Two paths, both accepted by the same handler:
|
||||
|
||||
| Client kind | Path | Why |
|
||||
|---|---|---|
|
||||
| **Programmatic** (CLI, test helper) | `Authorization: Bearer <jwt>` on the upgrade | The `new WebSocket()` API in browsers can attach `Sec-WebSocket-Protocol` but NOT arbitrary headers, so browsers can't do this. |
|
||||
| **Browser** | `POST /api/rt/ticket` (with the full DPoP + CSRF middleware chain) mints a one-shot 30-second opaque UUID; the browser opens the WS with `Sec-WebSocket-Protocol: oxi.ticket.<uuid>` | DPoP-bound sessions cannot attach a `DPoP:` header to `new WebSocket()`. The ticket flow moves the DPoP check to a normal POST that DOES support headers, and the WS upgrade just redeems the opaque token. |
|
||||
|
||||
Tickets are single-use, TTL 30 s, stored in a `RtTicketStore`
|
||||
(in-memory). Redemption removes the entry — replay is impossible.
|
||||
|
||||
The WS route is deliberately mounted **outside** the
|
||||
`protected_api` middleware stack — otherwise the DPoP-required
|
||||
layer would 401 every browser on upgrade before the ticket flow
|
||||
could kick in.
|
||||
|
||||
---
|
||||
|
||||
## The three AuthZ scopes
|
||||
|
||||
`Topic::required_perm(&self) -> AuthzCheck` dispatches every
|
||||
subscribe attempt into exactly one of three classes. This is the
|
||||
authoritative diagram of what the WS handler enforces:
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────┐
|
||||
│ Topic::required_perm() │
|
||||
└─────┬─────────────┬─────────────┬────────┘
|
||||
│ │ │
|
||||
ResourceRead │ IdentityMatch │ RoleAdmin
|
||||
▼ ▼ ▼
|
||||
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
||||
│ Class 1 │ │ Class 2 │ │ Class 3 │
|
||||
│ per-resource│ │ per-user │ │ per-session │
|
||||
│ │ │ │ │ (role) │
|
||||
└─────────────┘ └─────────────┘ └─────────────┘
|
||||
```
|
||||
|
||||
### Class 1 — Per-resource (`AuthzCheck::ResourceRead`)
|
||||
|
||||
**Topics:** `folder:{id}`, and (future) `file:{id}`,
|
||||
`drive:{id}`, `calendar:{id}`, `addressbook:{id}`.
|
||||
|
||||
**Rule:** the caller must hold **`Read`** on the resource via the
|
||||
same `AuthorizationEngine` that guards every REST endpoint. Owner
|
||||
short-circuits pass; direct grants pass; group-mediated grants
|
||||
pass; drive-membership cascades pass. Everything else is denied.
|
||||
|
||||
**Wire response on denial:** JSON-RPC error object with
|
||||
`code = -32001`, `message = "no_read"`. Same shape whether the
|
||||
resource doesn't exist OR the caller lacks the grant — **anti-
|
||||
enumeration invariant**. Audit reason (`no_read` /
|
||||
`no_such_resource`) distinguishes internally.
|
||||
|
||||
**On grant revocation:** the WS handler auto-subscribes each
|
||||
session to `user:{caller}:authz` (Class 2 below). When a
|
||||
`MessageBusEvent::AuthzChanged { affected_folders }` fires, the
|
||||
session's reader translates it to an internal `EvictFolders`
|
||||
signal → the main loop walks the sub set and drops any Class-1
|
||||
subscription whose resource was affected, emitting a client-visible
|
||||
`rt.revoked` notification per evicted topic. Same pattern applies
|
||||
to any Class-1 topic when the AuthZ model widens beyond folders.
|
||||
|
||||
### Class 2 — Per-user, strict privacy (`AuthzCheck::IdentityMatch`)
|
||||
|
||||
**Topics:** `user:{u}:authz`, `user:{u}:notifications`.
|
||||
|
||||
**Rule:** direct UUID equality — `caller_id == u`. **No admin
|
||||
bypass, no group indirection, no owner short-circuit.** Admins
|
||||
cannot subscribe to other users' `:authz` or `:notifications`
|
||||
streams; that's a privacy invariant, not a mere policy choice.
|
||||
|
||||
**Wire response on mismatch:** `topic_forbidden` — the **same
|
||||
wire shape as an unknown topic**. An attacker probing
|
||||
`user:{someone_else_uuid}:authz` cannot distinguish "user exists
|
||||
but not me" from "no such user."
|
||||
|
||||
**Auto-subscription:** the WS handler auto-subscribes every session
|
||||
to its own `user:{caller}:authz` AND `user:{caller}:notifications`
|
||||
at session open. No `rt.subscribe` frame is needed from the client
|
||||
for these — they're always active for the caller's own UUID.
|
||||
|
||||
### Class 3 — Per-session role (`AuthzCheck::RoleAdmin`)
|
||||
|
||||
**Topics:** `job:{name}` today. Future `admin:*` topics land here.
|
||||
|
||||
**Rule:** the session's snapshotted role at open time must be
|
||||
`admin`. The handler resolves `caller_role` once during session
|
||||
setup via `resolve_live_role` and stores it on the session state —
|
||||
no per-subscribe DB round-trip.
|
||||
|
||||
**Wire response on non-admin:** `topic_forbidden` — same anti-enum
|
||||
shape as Class 2. A non-admin probing job topics cannot enumerate
|
||||
which jobs are registered.
|
||||
|
||||
**Why snapshot at session open, not per subscribe:** admin role
|
||||
loss is rare + trivially recoverable (the user closes the tab and
|
||||
reopens, hitting the fresh role check). Per-subscribe checks would
|
||||
be an extra DB round-trip on every frame with no meaningful
|
||||
security gain — the WS session itself was authenticated at upgrade
|
||||
time under the current role.
|
||||
|
||||
### Adding a new topic
|
||||
|
||||
Every new topic variant must decide which class it belongs to at
|
||||
`Topic::required_perm`. The compiler enforces exhaustiveness — a
|
||||
new variant with no branch fails to build, which is deliberate. New
|
||||
topics get audited before shipping precisely because the
|
||||
`required_perm` match forces the author to state the class
|
||||
explicitly.
|
||||
|
||||
---
|
||||
|
||||
## Tab-visibility grace-close — reducing idle connections
|
||||
|
||||
Every open browser tab holds one WebSocket to the server. A user
|
||||
with five tabs open holds five sockets. A user who leaves a tab
|
||||
open all day but only uses one holds five sockets, four of them
|
||||
serving nothing.
|
||||
|
||||
The frontend closes the WebSocket **after 60 seconds of tab
|
||||
hidden** and reopens it when the tab becomes visible again. The
|
||||
subscription state is preserved locally through the outage — every
|
||||
subscriber's release handle stays live, the reactive store still
|
||||
holds the last-known list — but the wire is silent while the tab
|
||||
is hidden.
|
||||
|
||||
### Implementation
|
||||
|
||||
Frontend `MessageBusClient` (`frontend/src/lib/message-bus/client.svelte.ts`)
|
||||
attaches a `visibilitychange` listener on construction:
|
||||
|
||||
- **Tab hidden** → starts a 60-second timer.
|
||||
- **Timer fires while still hidden** → close the WS via a
|
||||
`#closeForHidden` path that sets state to `disconnected` but
|
||||
preserves `#subs` for later replay. The `#onClose` handler is
|
||||
guarded by `#tabIsHidden()` — an auto-reconnect won't fire while
|
||||
the tab remains hidden.
|
||||
- **Tab visible again** → cancels the timer if it hadn't fired
|
||||
yet; if the WS was closed, kicks off a normal reconnect.
|
||||
- **On reopen**, the WS handler auto-subscribes to `:authz` and
|
||||
`:notifications` again, and the client replays every entry in
|
||||
`#subs` as `rt.subscribe` frames. From the user's POV, the state
|
||||
is identical to what they left behind.
|
||||
|
||||
### What this trades
|
||||
|
||||
- **Saved**: N-1 idle sockets per user with N tabs open, over the
|
||||
hidden-tab window. Meaningful at scale (100 users × 5 tabs × 8
|
||||
idle hours = 4000 idle-tab-hours of connection state to keep
|
||||
alive per day).
|
||||
- **Lost**: bus events published during the 60-second delay
|
||||
(transient) + the whole grace-close window (indefinite while
|
||||
hidden) are dropped for that session. **Recovery**: on reopen,
|
||||
every consumer that cares refetches. See "Reconnect catch-up"
|
||||
below.
|
||||
|
||||
### Why 60 seconds
|
||||
|
||||
Short enough that leaving a tab for a coffee break doesn't burn
|
||||
the connection. Long enough that the momentary focus-shifts
|
||||
users do all day (Cmd-Tab to another app, back within seconds)
|
||||
don't churn the socket. Not tunable per-user — the value is
|
||||
hard-coded in `HIDDEN_GRACE_MS`.
|
||||
|
||||
### Reconnect catch-up — the `?after=` cursor
|
||||
|
||||
For consumers whose state can't be reconstructed by refetching a
|
||||
current listing (specifically: **notifications**, whose bell must
|
||||
show rows that landed during the outage), the FE issues a delta
|
||||
fetch:
|
||||
|
||||
- Store tracks `#lastReceivedAt` — the newest `created_at` seen
|
||||
before disconnect.
|
||||
- On `messageBus.onReconnect(...)` fire, calls
|
||||
`GET /api/notifications?after=<lastReceivedAt>&limit=100`.
|
||||
- Merges the returned rows into local state via `mergeById` —
|
||||
duplicates are resolved with **incoming wins** (server value
|
||||
overrides local, so a `read_at` flip on another device shows
|
||||
up correctly).
|
||||
|
||||
The server-side `?after=` predicate is **strict `>`** — a row at
|
||||
exactly `lastReceivedAt` is excluded. This makes the WS push (which
|
||||
delivers a row at time T) and the delta fetch (which asks for
|
||||
"anything after T") non-overlapping by construction. `mergeById`
|
||||
handles the case where the two paths race and both deliver the
|
||||
same row.
|
||||
|
||||
For consumers whose state IS a current listing (folder view: the
|
||||
files/subfolders in a folder), reconnect just refetches the listing
|
||||
via the normal REST endpoint. `useReconnect` composable exposes
|
||||
`onReconnect(cb)` as a one-liner for that pattern.
|
||||
|
||||
---
|
||||
|
||||
## Persistent notifications (the bell)
|
||||
|
||||
### Data model
|
||||
|
||||
```sql
|
||||
notif.notifications (
|
||||
id UUID PRIMARY KEY,
|
||||
user_id UUID NOT NULL REFERENCES auth.users(id) ON DELETE CASCADE,
|
||||
kind TEXT NOT NULL, -- 'share_granted' | 'new_login_from_new_device' | …
|
||||
payload JSONB NOT NULL, -- per-kind shape (see below)
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
read_at TIMESTAMPTZ -- NULL = unread
|
||||
)
|
||||
```
|
||||
|
||||
Two indexes:
|
||||
|
||||
- `(user_id, created_at DESC) INCLUDE (read_at, kind)` covers the
|
||||
bell's list query + the mark-all-read filter.
|
||||
- `(read_at) WHERE read_at IS NOT NULL` — partial, tiny on healthy
|
||||
DBs; feeds the retention job's DELETE.
|
||||
|
||||
### Ingesters
|
||||
|
||||
An ingester is a code path that calls
|
||||
`NotificationApplicationService::create(NewNotification)`. The
|
||||
service atomically:
|
||||
|
||||
1. `INSERT INTO notif.notifications RETURNING …` — durable row.
|
||||
2. `bus.publish(Topic::UserNotifications(user_id), NotificationReceived)` — the fast-path poke.
|
||||
|
||||
Publish happens **after** the DB write succeeds, never inside a
|
||||
transaction — a rolled-back INSERT would otherwise fan out a lie.
|
||||
|
||||
**Currently shipped ingester:** `share_granted` in
|
||||
`interfaces/api/handlers/grant_handler.rs::create_grant`. Fires
|
||||
after `authz.set_role(...)` succeeds. Fans out to every resolved
|
||||
recipient user:
|
||||
|
||||
- `Subject::User(id)` → one row for that user.
|
||||
- `Subject::Group(id)` → one row per transitive member (via
|
||||
`SubjectGroupService::list_transitive_users`).
|
||||
- `Subject::Token(_)` → no row (anonymous share links have no
|
||||
target user).
|
||||
|
||||
Self-shares (owner grants themselves via a group they belong to)
|
||||
skip. Failure to write is best-effort — a warn log; the grant row
|
||||
stays durable in `role_grants`, the recipient can still discover
|
||||
the share via `/api/grants/shared-with-me`.
|
||||
|
||||
**Planned but not wired** (each needs a prerequisite subsystem
|
||||
listed in [`docs/plan/message-bus.md § Deferred`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/message-bus.md)):
|
||||
|
||||
| Kind | Prerequisite |
|
||||
|---|---|
|
||||
| `new_login_from_new_device` | Device-fingerprint tracking table |
|
||||
| `job_completed_for_you` | Scheduler engine threading the trigger caller's `caller_id` through `dispatch()` |
|
||||
| `storage_quota_threshold` | Per-user usage/quota comparator with threshold-crossing detection |
|
||||
|
||||
### Payload shape — typed per kind
|
||||
|
||||
The `payload` column is JSONB (schema-free at the DB layer). Each
|
||||
kind's Rust shape lives in `domain/entities/notification.rs` with
|
||||
`#[derive(Serialize, Deserialize, ToSchema)]`. OpenAPI picks up the
|
||||
struct automatically. Adding a new field is additive on JSONB — no
|
||||
migration.
|
||||
|
||||
Example — `share_granted`:
|
||||
|
||||
```rust
|
||||
pub struct SharegrantedPayload {
|
||||
pub granter_id: Uuid,
|
||||
pub resource_type: String, // 'folder' | 'file' | 'drive' | …
|
||||
pub resource_id: Uuid,
|
||||
pub resource_name: Option<String>, // snapshot at grant time
|
||||
pub resource_path: Option<String>, // storage path snapshot
|
||||
pub navigate_folder_id: Option<Uuid>, // FE routing target for drives
|
||||
pub role: String,
|
||||
pub expires_at: Option<DateTime<Utc>>,
|
||||
}
|
||||
```
|
||||
|
||||
The name/path fields are **snapshotted at grant time**. If the
|
||||
folder is later renamed or moved, the notification still reflects
|
||||
what it was called when the share happened. Same principle as
|
||||
email invitations or activity feeds: the record is a fact about
|
||||
what was true at the moment, not a live pointer.
|
||||
|
||||
### Schema-ownership rule — AsyncAPI vs OpenAPI
|
||||
|
||||
> **AsyncAPI defines the envelope + transport for clients.
|
||||
> OpenAPI defines the payload.**
|
||||
|
||||
The bus event `MessageBusEvent::NotificationReceived` is a **unit
|
||||
variant** — it serializes to `{"event":"notification_received",
|
||||
"data":{}}` with no fields on the wire. The topic identifies the
|
||||
semantic; the FE responds by refetching from REST.
|
||||
|
||||
The payload's shape lives in OpenAPI via `ToSchema` on
|
||||
`SharegrantedPayload` (and its future siblings), auto-derived from
|
||||
Rust. AsyncAPI never sees these types — payloads don't travel on
|
||||
the bus wire.
|
||||
|
||||
**Why this split** — it eliminates schema drift between the two
|
||||
specs. A payload edit changes Rust → OpenAPI updates on regen
|
||||
(mechanical). AsyncAPI stays stable (hand-written, but it never
|
||||
touches payloads). Same rule applies to any future bus consumer
|
||||
that also has a REST DTO — Rust is the source of truth; each spec
|
||||
projects the parts of Rust that travel on its transport.
|
||||
|
||||
Design rationale + rejected alternatives (dual-spec, cross-`$ref`,
|
||||
per-kind Svelte components) in [`docs/plan/templated-messages.md § Schema ownership`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/templated-messages.md).
|
||||
|
||||
### REST surface — `/api/notifications`
|
||||
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| `GET` | `/api/notifications` | List newest-first. Query params: `unread` (bool), `before` / `after` (cursors), `limit` |
|
||||
| `GET` | `/api/notifications/unread` | Badge-only fast path (returns just `unread_count`) |
|
||||
| `POST` | `/api/notifications/{id}/read` | Mark one row read (idempotent, always 204) |
|
||||
| `POST` | `/api/notifications/read-all` | Bulk mark-all-read (returns rows updated) |
|
||||
| `DELETE` | `/api/notifications/{id}` | Hard-delete one row (idempotent, always 204) |
|
||||
|
||||
**Anti-enumeration:** mark-read and delete always respond 204 —
|
||||
whether the row existed and belonged to the caller, or didn't
|
||||
exist at all, or belonged to someone else. Every mutating endpoint
|
||||
scopes on `caller_id` at the SQL layer; the response shape is
|
||||
identical across the three outcomes.
|
||||
|
||||
### Retention
|
||||
|
||||
The `notifications_cleanup` scheduled job (daily, same tier as
|
||||
`trash_cleanup`) DELETEs read rows older than
|
||||
`OXICLOUD_NOTIFICATIONS_RETENTION_DAYS` (default 30). **Unread
|
||||
rows are preserved unconditionally** — a user offline for a month
|
||||
still sees the share-granted notice when they log back in.
|
||||
|
||||
Runtime override via the job's `retention_days` parameter on the
|
||||
admin panel's trigger — the env default seeds it, the panel
|
||||
overrides at trigger time.
|
||||
|
||||
### Frontend rendering
|
||||
|
||||
`frontend/src/lib/composables/useNotifications.svelte.ts` owns the
|
||||
module-scoped store — one instance per SPA session. Exposes:
|
||||
|
||||
- `notifications.items` — reactive list (newest first)
|
||||
- `notifications.unread` — reactive badge count
|
||||
- `notifications.refresh()` / `refreshDelta()` / `markRead(id)` /
|
||||
`markAllRead()` / `delete(id)`
|
||||
|
||||
`NotificationRow.svelte` handles the actual rendering — one file,
|
||||
one `switch` on `row.kind`, one rich template per shipped kind
|
||||
(`share_granted` today; the others fall back to a generic string).
|
||||
Extraction into per-kind components is deferred until a single
|
||||
kind's block exceeds ~30 lines or two kinds start needing the same
|
||||
sub-component (see [`docs/plan/templated-messages.md § Rendering`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/templated-messages.md)).
|
||||
|
||||
### Notification click routing
|
||||
|
||||
| `resource_type` | Route | Data used |
|
||||
|---|---|---|
|
||||
| `folder` | `/files/{resource_id}` | `resource_id` |
|
||||
| `file` | `/shared-with-me?file={resource_id}` | `resource_id` — the `/files/{uuid}` route requires a **folder** id, and a file-scoped grant may not include parent-folder access. `/shared-with-me` is the guaranteed-accessible home for every recipient of a `share_granted`, and its `?file=` deep link opens the inline `FileViewer`. |
|
||||
| `drive` | `/files/{navigate_folder_id}` | Drives have no browsable URL of their own; `navigate_folder_id` is the drive's `root_folder_id`, enriched at ingest via `DriveRepository::get_by_id`. |
|
||||
| `calendar` / `address_book` / `playlist` | no link (bold text) | Not addressable via `/files/*`. |
|
||||
|
||||
The bell also fires a **transient toast** (via the existing
|
||||
`ui.notify(...)` mechanism) on every fresh row that arrives via
|
||||
delta — the toast fades out after ~4 s while the persistent row
|
||||
stays in the bell's history section. Same bell icon, same badge
|
||||
count, no duplicate UX.
|
||||
|
||||
---
|
||||
|
||||
## Feature flags & config
|
||||
|
||||
| Env var | Default | Effect |
|
||||
|---|---|---|
|
||||
| `OXICLOUD_MESSAGEBUS_ENABLE` | `true` | Master switch. When `false`, the `/api/rt/ws` and `/api/rt/ticket` routes are **not registered** at boot (Axum returns 404), and the FE `useNotifications` composable skips the WS setup entirely. The bell falls back to REST-only mode — polling on mount, delta on manual refresh. Zero client-side error spam. |
|
||||
| `OXICLOUD_MESSAGEBUS_KEEPALIVE_SECONDS` | `30` | Server-initiated RFC 6455 Ping interval on each WS connection. Prevents intermediate proxies (nginx, Traefik, Cloudflare) from reaping the TCP session as idle. |
|
||||
| `OXICLOUD_NOTIFICATIONS_RETENTION_DAYS` | `30` | Retention window for read notifications. Unread rows are always preserved. The `notifications_cleanup` job clamps to a minimum of 1 day. |
|
||||
|
||||
Client discovers all of these via `GET /api/config` — no in-band
|
||||
"does the server support the bus?" probe needed. The FE reads
|
||||
`serverConfig.features.message_bus` at boot and skips WS setup
|
||||
entirely when it's false.
|
||||
|
||||
---
|
||||
|
||||
## Failure modes
|
||||
|
||||
| Scenario | Behavior |
|
||||
|---|---|
|
||||
| Bus is disabled server-side (`OXICLOUD_MESSAGEBUS_ENABLE=false`) | `/api/rt/ws` returns 404. `useTopic` in the FE returns early. Bell works via REST only. |
|
||||
| Network drops mid-session | Client-side jittered exponential backoff (250 ms → 30 s cap, 20-failure circuit breaker). On reconnect, WS handler re-auto-subscribes to `:authz` + `:notifications`; `useReconnect` composable fires `onReconnect` callbacks so views refetch. |
|
||||
| Server restart | Same as network drop — the WS breaks, client backs off, reconnects when server is back. Events published during the outage are lost (no persistent event log by design); consumers refetch. |
|
||||
| Tab hidden > 60 s | WS closed via `visibilitychange` grace-close. State preserved locally. On visibility return, reconnect + replay subscriptions. |
|
||||
| Publish before commit | Not allowed. Every publish site is documented as "after commit". A publish inside a transaction that rolls back would fan out a lie. |
|
||||
| Broker replicator failure (future) | The local `InProcessMessageBus` publishes still succeed — the replicator is beside the bus, not in front. Broker-hop failures affect multi-instance fanout but never local delivery. |
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
The api-test suite exercises the full stack end-to-end via `rt-hurl-helper` (a small Rust binary gated on `test_utils`) — Hurl alone can't drive a WebSocket. Sixteen scenarios in `tests/api/rt_bus_check.sh`:
|
||||
|
||||
- Positive delivery, topic isolation
|
||||
- AuthZ denial (Class 1 folder), unknown-topic anti-enum
|
||||
- Server keepalive, delete emits, move fan-out
|
||||
- Grant-revoke eviction (`AuthzChanged` → `rt.revoked`)
|
||||
- Cross-user identity gate (Class 2)
|
||||
- Ticket happy path + single-use replay refused
|
||||
- Admin-only job topic (Class 3)
|
||||
- `notification_received` wire push, DB row via `GET /api/notifications`
|
||||
- Cross-user notifications identity gate (Class 2, notifications topic)
|
||||
- `?after=` cursor with strict-`>` boundary invariant
|
||||
|
||||
`mergeById` — the FE's WS/reconnect race dedup — has its own
|
||||
Vitest with 5 covered cases (empty, non-overlap, exact-dup,
|
||||
stale-read-at overwrite, mixed overlap).
|
||||
|
||||
---
|
||||
|
||||
## Further reading
|
||||
|
||||
- Plan doc: [`docs/plan/message-bus.md`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/message-bus.md) — full design rationale, roadmap, and deferred slices (Yjs collab, broker replicator, SharedWorker, Web Push).
|
||||
- Plan doc: [`docs/plan/templated-messages.md`](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/templated-messages.md) — schema-ownership rule, rendering shape, notification routing decision.
|
||||
- [ReBAC Authorization](/architecture/rebac-authorization) — the engine every Class-1 topic gate calls.
|
||||
- [Background jobs](/architecture/jobs) — `notifications_cleanup` is one of them; `Topic::Job` publishes on job lifecycle.
|
||||
@@ -36,8 +36,43 @@ What used to live on the share row but is now resolved through ReBAC:
|
||||
|
||||
| Method | Path | Description |
|
||||
| --- | --- | --- |
|
||||
| `GET` | `/api/s/{token}` | Access a shared item |
|
||||
| `POST` | `/api/s/{token}/verify` | Verify a password-protected share |
|
||||
| `GET` | `/api/s/{token}` | Share landing metadata (see [landing enrichment](#share-landing-metadata-enrichment)) |
|
||||
| `POST` | `/api/s/{token}/verify` | Verify a password-protected share (sets the unlock-JWT cookie) |
|
||||
| `GET` | `/api/s/{token}/download` | Download a **file share** (Range / 206 / 304 / 416 aware) |
|
||||
| `GET` | `/api/s/{token}/contents` | List a **folder share's** root (folders + files) |
|
||||
| `GET` | `/api/s/{token}/contents/{folder_id}` | List a subfolder inside the shared subtree |
|
||||
| `GET` | `/api/s/{token}/file/{file_id}` | Stream one file — the landing page's inline preview and per-file download path (Range aware) |
|
||||
| `GET` | `/api/s/{token}/zip` | ZIP archive of a **folder share's** root |
|
||||
| `GET` | `/api/s/{token}/zip/{folder_id}` | ZIP archive of a subfolder inside the shared subtree |
|
||||
|
||||
#### File scoping on `/file/{file_id}`
|
||||
|
||||
The AuthZ gate (`ShareBrowseService::assert_file_in_share`) branches on the
|
||||
share's `item_type` — a single-file share and a folder share scope the
|
||||
endpoint differently:
|
||||
|
||||
- **File share** — only the shared item itself may be streamed
|
||||
(`file_id == share.item_id`). This is what renders the public landing
|
||||
page's inline media preview (video player / image) and it is also the
|
||||
NextCloud-desktop-style per-file fetch path.
|
||||
- **Folder share** — the file must live inside the shared subtree
|
||||
(ltree `is_file_in_subtree` against the share's root folder).
|
||||
- Anything else → **404**, the same shape as "file doesn't exist", so the
|
||||
endpoint cannot be used to enumerate ids.
|
||||
|
||||
Password and expiry checks happen inside `get_shared_link_with_unlock`
|
||||
before the scope decision; a password-protected share answers 401 with
|
||||
`requiresPassword: true` until the unlock cookie is presented.
|
||||
|
||||
#### Share landing metadata enrichment
|
||||
|
||||
`GET /api/s/{token}` resolves the shared **file's** `mime_type` + `size`
|
||||
at read time so anonymous viewers get an inline media preview (video
|
||||
player / image) instead of a bare download button. The enrichment is
|
||||
display-only and never fails the response: a failed file lookup (transient
|
||||
DB error, race with a delete) leaves the fields absent and the download
|
||||
endpoints surface the real error — a read failure is never proof that the
|
||||
data is absent. Folder shares pass through unenriched.
|
||||
|
||||
## Service Responsibilities
|
||||
|
||||
|
||||
@@ -123,6 +123,70 @@ rather than as a visible error.
|
||||
| `OXICLOUD_NOTIFICATIONS_RETENTION_DAYS` | `30` | Retention window for **read** notification rows (`notif.notifications`). The `notifications_cleanup` scheduled job runs daily and deletes rows where `read_at IS NOT NULL` and `read_at < now() - retention_days`. Unread rows are preserved unconditionally — the whole point of the durable table is that a user offline for a month still sees the share-granted notice on next login. Clamped to a minimum of 1 (0 would purge every read row on every tick). Adjust down for compliance-sensitive deployments where "cleared once seen" matters; adjust up when operators expect users to reference old notifications for support. |
|
||||
| `OXICLOUD_WEBDAV_DRIVE_LISTING_PREFIX` | `@drive` | Native WebDAV URL segment that renders the caller's drive list. Sanitized by trimming leading/trailing `/`. Three shapes: (1) default `@drive` — `/webdav/…` addresses the caller's default personal drive (back-compat), `/webdav/@drive/` returns the drive listing, `/webdav/@drive/<uuid\|name>/…` targets a specific drive. (2) empty string `""` — `/webdav/` IS the drive listing, `/webdav/<uuid\|name>/…` targets a specific drive, no default-drive shortcut. (3) any other string (e.g. `drives`) — same shape as `@drive` with that segment substituted. Only drives the caller has Read on via `role_grants` resolve. |
|
||||
|
||||
## Boot-time refuse-to-boot checks
|
||||
|
||||
Some upgrades add invariants the running database must satisfy
|
||||
BEFORE the new binary can serve traffic. These are enforced by
|
||||
read-only checks that run after `sqlx::migrate!()` and before the
|
||||
server binds a listen socket. If a check fails, the server exits
|
||||
with a FATAL message spelling out the exact CLI command to run.
|
||||
|
||||
The server **never silently mutates data** at boot — every fix is
|
||||
an explicit `oxicloud migrate <name>` invocation. Follows the
|
||||
"discovery-only by default, mutation opt-in" rule that also
|
||||
governs the consistency-check jobs.
|
||||
|
||||
### `lowercase-usernames`
|
||||
|
||||
Three outcomes at boot, only one of which stops the server:
|
||||
|
||||
- **All lowercase (or `NULL`)** — the check is a no-op, boot
|
||||
proceeds unchanged.
|
||||
- **Mixed-case rows exist, no `LOWER(username)` collision** —
|
||||
boot **auto-lowercases** them in one atomic transaction, emits
|
||||
a structured audit line per rename
|
||||
(`user.username_lowercased_on_boot`, INFO) plus an INFO summary
|
||||
(`user.usernames_lowercased_on_boot_summary` with `renamed=N`),
|
||||
and continues. Silent action is confined to the case with
|
||||
exactly one correct move: `Alice` (with no `alice` row) becomes
|
||||
`alice`.
|
||||
- **`LOWER(username)` collision** — two or more active rows share
|
||||
the same lowercase form (e.g. `Alice` + `alice`). Boot
|
||||
**refuses to start** with a FATAL message spelling out every
|
||||
collision group and the exact CLI command to resolve it.
|
||||
Tiebreak needs a human.
|
||||
|
||||
Soft-deleted / disabled accounts (`active = false`) and `NULL`
|
||||
usernames (OPAQUE-migrated) are skipped in every case.
|
||||
|
||||
**Refusal message pattern:** `FATAL: cannot start — N colliding
|
||||
username group(s) (M affected account(s) in total)`, followed by
|
||||
each group's canonical form and its members with `id` +
|
||||
`last_login`. Up to 10 groups shown; the `--dry-run` CLI reveals
|
||||
the full list.
|
||||
|
||||
**Fix (only required when the server refused):**
|
||||
```
|
||||
oxicloud migrate lowercase-usernames --dry-run # preview the tiebreak
|
||||
oxicloud migrate lowercase-usernames # apply
|
||||
```
|
||||
|
||||
**What the migration does:** lowercases every mixed-case
|
||||
username. On collision (`Alice` + `alice` both exist), the
|
||||
tiebreak `(last_login_at DESC NULLS LAST, created_at ASC)` picks
|
||||
a winner; losers get `alice-2`, `-3`, … as a suffix. Sessions
|
||||
and grants survive the rename — both key on the user's UUID.
|
||||
|
||||
**Client compat:** NextCloud clients that cached URLs like
|
||||
`/remote.php/dav/files/Alice/…` continue to work indefinitely —
|
||||
the Basic Auth middleware and URL parser both lowercase on
|
||||
decode. NC desktop clients will prompt a one-time re-sync on
|
||||
first PROPFIND after upgrade; DAVX5 and NC mobile handle it
|
||||
silently. See `docs/install/binary.md § Upgrading from a
|
||||
case-sensitive-usernames release` for the full upgrade flow.
|
||||
|
||||
**Design:** `docs/plan/username-lowercase.md`.
|
||||
|
||||
## Storage Entries (multi-entry, recommended)
|
||||
|
||||
Declare one or more **named** storage backends. The one the app runs on is picked from the DB (`admin_settings.storage.active_backend_name`); the admin panel's storage tab flips the pointer, and cross-backend migration is a recoverable job that copies blobs between two entries with a read-only safety window. See [Admin Settings — Storage & Migration](/config/admin-settings) for the operator flow and the [multi-entry design doc](https://github.com/oxicloud/oxicloud/blob/main/docs/plan/storage-multi-entry.md) for the full model.
|
||||
|
||||
@@ -47,6 +47,19 @@ If you need to let someone make changes, share with their **email**
|
||||
instead. They'll receive an invitation, and from then on every change
|
||||
they make is recorded under their name.
|
||||
|
||||
## What recipients see
|
||||
|
||||
Opening a **single-file** public link shows the file right on the
|
||||
landing page — images display inline, and videos play in the browser
|
||||
with a working seek bar (streamed, so no full download before
|
||||
playback). A **Download** button always sits below the preview. For
|
||||
files the browser can't display, recipients get the download button as
|
||||
usual.
|
||||
|
||||
Opening a **folder** public link shows a browsable listing — folders
|
||||
and files as cards, with a grid/list toggle and a **Download ZIP**
|
||||
button. Images and videos open in a lightbox viewer.
|
||||
|
||||
## Expiration
|
||||
|
||||
When you share, you can set an **expiration date**. After that date,
|
||||
|
||||
@@ -217,6 +217,75 @@ supported by sqlx's migration model; if you need to roll back, stop
|
||||
the server, roll back your Postgres data directory to a snapshot, and
|
||||
install the previous binary.
|
||||
|
||||
### Upgrading from a case-sensitive-usernames release
|
||||
|
||||
Releases that predate the case-insensitive-usernames change stored
|
||||
`Alice`, `alice`, and `ALICE` as three separate accounts. The
|
||||
current release treats usernames as case-insensitive (canonical
|
||||
lowercase in the database). What happens on the first boot after
|
||||
upgrade depends on your data:
|
||||
|
||||
**No mixed-case usernames.** The check is a no-op; the server
|
||||
starts normally. Nothing to do.
|
||||
|
||||
**Mixed-case usernames with no collision.** The server
|
||||
**auto-lowercases** them in one atomic transaction at boot and
|
||||
continues. Each rename is recorded in the audit log
|
||||
(`user.username_lowercased_on_boot`) and a WARN summary line
|
||||
names the total count. `Alice` (with no `alice` row alongside)
|
||||
becomes `alice`; no ops action needed. This covers the vast
|
||||
majority of single-admin self-hosted deployments.
|
||||
|
||||
**Mixed-case usernames WITH a collision** (`Alice` + `alice`
|
||||
both exist as active accounts). The server **refuses to boot**
|
||||
— tiebreak requires a human. Run the migration:
|
||||
|
||||
```
|
||||
# Preview the tiebreak — no writes.
|
||||
sudo -u oxicloud DATABASE_URL="postgres://..." \
|
||||
/usr/local/bin/oxicloud migrate lowercase-usernames --dry-run
|
||||
|
||||
# Apply. Renames run in a single transaction; safe to re-run if aborted.
|
||||
sudo -u oxicloud DATABASE_URL="postgres://..." \
|
||||
/usr/local/bin/oxicloud migrate lowercase-usernames
|
||||
```
|
||||
|
||||
On collision, the migration picks a winner by `(last_login_at
|
||||
DESC NULLS LAST, created_at ASC)` — most recently active keeps
|
||||
the canonical lowercase name; the loser gets `alice-2`, `-3`, …
|
||||
as a suffix. **Sessions and grants survive the rename** — both
|
||||
key on the user's UUID, not the username.
|
||||
|
||||
Skipped in every path above: soft-deleted / disabled accounts
|
||||
and OPAQUE-migrated users whose `username` column is NULL.
|
||||
Neither blocks boot.
|
||||
|
||||
After the migration completes, restart the service:
|
||||
|
||||
```
|
||||
sudo systemctl start oxicloud
|
||||
```
|
||||
|
||||
**Nextcloud desktop clients** will prompt a one-time re-sync on
|
||||
their first PROPFIND after upgrade — the account URL case
|
||||
changed. Data is safe (files re-verify via ETag, not re-uploaded).
|
||||
DAVX5 (calendars, contacts) and NC mobile handle the URL case
|
||||
change silently. **No client upgrade or reconfiguration is
|
||||
required** — the server accepts uppercase URL segments (`Alice`
|
||||
in `/remote.php/dav/files/Alice/...`) indefinitely.
|
||||
|
||||
The refusal message printed by the server on boot (collision
|
||||
path only) includes the exact CLI command above, so you can't
|
||||
miss it. Full plan and rationale in
|
||||
`docs/plan/username-lowercase.md`.
|
||||
|
||||
**Explicit-preview path.** If you'd rather run the migration
|
||||
before the binary swap — to review renames on your own schedule
|
||||
or to gate a backup step — run `oxicloud migrate
|
||||
lowercase-usernames --dry-run` against the OLD binary's DB
|
||||
first, then apply. On next boot the new binary sees a
|
||||
lowercase-clean DB and the auto-rename path is a no-op.
|
||||
|
||||
## Installing via `cargo binstall`
|
||||
|
||||
If you already have the Rust toolchain and just want the binary
|
||||
|
||||
@@ -0,0 +1,581 @@
|
||||
# Plan — Case-insensitive usernames (lowercase-on-ingest)
|
||||
|
||||
## Context
|
||||
|
||||
Feature ask: [issue #691](https://github.com/AtalayaLabs/OxiCloud/issues/691).
|
||||
Usernames are currently case-sensitive, so `Alice`, `alice`, and `ALICE`
|
||||
refer to three different accounts. Users hit this as a login friction —
|
||||
they type their name with different capitalization on different clients
|
||||
and get "invalid credentials" instead of a successful login.
|
||||
|
||||
## Why this is simpler than it looks in this codebase specifically
|
||||
|
||||
- `validate_username` in `src/domain/entities/user.rs:884` already
|
||||
restricts usernames to ASCII-only `[a-zA-Z0-9._-]{2,64}` with no `@`.
|
||||
The Unicode case-folding minefield (Turkish dotted-I, German ß, Greek
|
||||
final sigma, NFC vs NFD) does not apply — ASCII case-folding is
|
||||
trivial (`to_ascii_lowercase`), deterministic, and locale-independent.
|
||||
- OIDC identity binds via `(iss, sub)` in
|
||||
`get_user_by_federation_subject` at
|
||||
`src/infrastructure/repositories/pg/user_pg_repository.rs:1257-1303` —
|
||||
case-sensitivity of the local username is orthogonal to OIDC identity
|
||||
matching. No OIDC breakage risk.
|
||||
- Password verification runs through Argon2's `verify_password`
|
||||
(constant-time by construction). Not affected.
|
||||
- `@`-forbidden rule in usernames is the disjoint namespace with email
|
||||
lookup (`dispatch_login` at `auth_application_service.rs:1018`).
|
||||
Case-insensitive usernames align semantics with email addresses
|
||||
(already case-insensitive in practice), so any future
|
||||
`groupname@domain` composition stays consistent.
|
||||
- NextCloud URL `/remote.php/dav/files/{user}/…` uses `{user}` as an
|
||||
informational / consistency-check marker, not a security boundary —
|
||||
the chroot ACL is the real authz. Handling case in the URL segment is
|
||||
a small local change (documented in `session.rs:33-34`).
|
||||
|
||||
## Design decisions
|
||||
|
||||
1. **Silently lowercase on ingest** (registration, admin-create, OIDC
|
||||
provisioning, rename). Never reject uppercase input from clients —
|
||||
accept liberally, store strictly (Postel's Law).
|
||||
2. **Explicit migration** (`oxicloud migrate lowercase-usernames
|
||||
[--dry-run]`). The server never mutates `auth.users` at boot. Ops
|
||||
MUST run the migration explicitly. Follows the
|
||||
[[feedback_no_silent_auto_repair]] rule: consistency tenants are
|
||||
discovery-only by default; mutation is opt-in.
|
||||
3. **Refuse-to-boot** if any active-user username is not already
|
||||
lowercase. Boot error message shows the exact CLI command to run.
|
||||
Boot performs a read-only verification only.
|
||||
4. **Collision tiebreak** on migration: `(last_login_at DESC NULLS
|
||||
LAST, created_at ASC)`. Winner keeps the canonical lowercased name.
|
||||
Losers get `-2`, `-3`, … suffix (increment until free), matching the
|
||||
pattern in `oxicloud migrate nfc-filenames`.
|
||||
5. **Active accounts only** in the boot check + migration. Soft-deleted
|
||||
/ disabled rows are skipped (they don't block usable logins). NULL
|
||||
usernames (OPAQUE-migrated accounts) are skipped in every layer —
|
||||
the boot verifier, the migration UPDATE, the CLI report. The
|
||||
`WHERE username <> LOWER(username)` predicate is already NULL-safe
|
||||
by SQL semantics (NULL comparisons yield NULL, filtered out); state
|
||||
it explicitly so a reviewer isn't left wondering.
|
||||
6. **Rename-only** on collision resolution — sessions are not
|
||||
invalidated. Sessions key on `user_id` so they survive the rename.
|
||||
7. **Un-soft-delete of a mixed-case account uses the SAME suffix
|
||||
scheme.** If `Alice` is soft-deleted (skipped by migration) and
|
||||
later un-soft-deleted while `alice` already exists, the un-soft-
|
||||
delete path re-normalizes via `set_username` and, on collision,
|
||||
assigns `alice-2` / `alice-3` / … — the same helper the migration
|
||||
CLI calls. Both callers reach for a shared
|
||||
`find_free_username_suffix(pool, base) -> String` in
|
||||
`src/common/username_migration.rs` so migration + un-soft-delete
|
||||
agree by construction. Without this, an un-soft-delete could
|
||||
create a fresh collision that the next boot's auto-rename
|
||||
couldn't resolve (auto-rename handles singletons only) — the
|
||||
server would then refuse-to-boot until an admin resolves the
|
||||
tiebreak.
|
||||
8. **Boot-time behaviour has three outcomes, not two.** The
|
||||
verifier categorises the DB into: (a) clean — nothing to do;
|
||||
(b) mixed-case rows with no `LOWER(username)` collision — the
|
||||
server **auto-lowercases them in one atomic transaction and
|
||||
continues**, emitting an audit line per rename; (c) at least
|
||||
one `LOWER(username)` collision — the server **refuses to
|
||||
boot** because tiebreak requires human judgement. Silent
|
||||
action is bounded to (b), where there is exactly one correct
|
||||
move. This is a narrower reading of
|
||||
[[feedback_no_silent_auto_repair]] than "no silent action
|
||||
ever": the rule targets consistency-check jobs where drift is
|
||||
a bug signal; a schema-adjacent boot invariant with a
|
||||
unique-correct-fix is a different situation. Making the
|
||||
trivial-case common path a no-op massively lowers upgrade
|
||||
friction for the 90% self-hosted deployment.
|
||||
|
||||
## Not in scope
|
||||
|
||||
- Unicode case-folding (usernames are ASCII-only by validation).
|
||||
- `display_name` split (usernames were already just identifiers;
|
||||
free-form display is a separate future feature if a user asks for
|
||||
it — deferred pending real demand signal).
|
||||
- OIDC provisioning behaviour change beyond the ingest-normalize point.
|
||||
- Case-insensitivity for emails (already achieved in practice; not
|
||||
touched).
|
||||
- Any change to `validate_username`'s character-class rules.
|
||||
- Any change to the WebDAV URL shape `/dav/files/{user}/…` (client
|
||||
compat; drop deferred separately per [[project_nc_multidrive_poc]]).
|
||||
- Group names (`SubjectGroup`). Lowercase by convention today; no
|
||||
runtime enforcement, no migration. If group-name case-insensitivity
|
||||
becomes a real ask, it lands as a sibling plan doc with the same
|
||||
shape.
|
||||
|
||||
## Deliverables
|
||||
|
||||
### 1. Ingest normalization
|
||||
|
||||
Change `validate_username` to return the canonical form instead of
|
||||
`()`:
|
||||
|
||||
```rust
|
||||
// src/domain/entities/user.rs — new signature
|
||||
fn validate_username(username: &str) -> UserResult<String> {
|
||||
let normalized = username.trim().to_ascii_lowercase();
|
||||
// ... existing length + charset + boundary checks apply to `normalized` ...
|
||||
Ok(normalized)
|
||||
}
|
||||
```
|
||||
|
||||
Every caller that today does `Self::validate_username(u)?;` becomes
|
||||
`let u = Self::validate_username(&u)?;` — the returned canonical form
|
||||
is what gets stored. Because the return type changes from `Result<()>`
|
||||
to `Result<String>`, any caller that ignores the result now becomes a
|
||||
compile error — the type system forces every write path through the
|
||||
normalizer.
|
||||
|
||||
Write sites all funnel through `User::new` (`src/domain/entities/user.rs:313`)
|
||||
or `User::set_username` (`:811`), so the signature change catches the
|
||||
entity-write path automatically. Callers to touch:
|
||||
|
||||
- Application services calling `User::new`:
|
||||
- `auth_application_service.rs:840` — `register()` public signup
|
||||
- `auth_application_service.rs:944` — `setup_create_admin()`
|
||||
first-boot admin
|
||||
- `auth_application_service.rs:3520`, `:3532` — `admin_create_user()`
|
||||
external + internal branches
|
||||
- `auth_application_service.rs:4692` — OIDC JIT provisioning
|
||||
- `magic_link_invite_service.rs:233` — magic-link external invite
|
||||
|
||||
- User-driven rename (calls `User::set_username`):
|
||||
- `auth_application_service.rs:2756-2801` — `update_profile()`
|
||||
|
||||
- Repository-write compile-error catches:
|
||||
- `src/infrastructure/repositories/pg/user_pg_repository.rs:281`
|
||||
(`create_user` INSERT) and `:740` (`update_user` UPDATE) — these
|
||||
bind `user_clone.username()`, which is now guaranteed lowercase by
|
||||
the entity constructor.
|
||||
|
||||
### 2. OIDC JIT derivation
|
||||
|
||||
`auth_application_service.rs:4649-4690` derives a local username from
|
||||
the OIDC `preferred_username` / `name` / `sub` claims, filters to
|
||||
`[a-zA-Z0-9._-]`, and truncates. **It does not currently lowercase.**
|
||||
Add `to_ascii_lowercase()` on the derived string before passing to
|
||||
`User::new`. This is beyond what the entity signature change catches —
|
||||
explicit fix required.
|
||||
|
||||
### 3. Lookup normalization
|
||||
|
||||
Repository `find_by_username`-style methods internally lowercase the
|
||||
input before the SQL query, so callers don't have to remember. One-line
|
||||
change per method:
|
||||
|
||||
- `src/infrastructure/repositories/pg/user_pg_repository.rs:487`
|
||||
(`get_user_by_username`) — add `let username = username.trim().
|
||||
to_ascii_lowercase();` before the `.bind(&username)` at line 488.
|
||||
- `src/infrastructure/repositories/pg/user_pg_repository.rs:1043`
|
||||
(`search_users`) — `ILIKE` is already case-insensitive by
|
||||
construction; verify nothing regresses.
|
||||
- `src/infrastructure/repositories/pg/user_pg_repository.rs:1504`
|
||||
(`search_usernames`) — same as above.
|
||||
- `src/application/services/storage_usage_service.rs:145-149`
|
||||
(`update_user_storage_usage_by_username`) — raw SQL bind; normalize
|
||||
before `.bind()`.
|
||||
- `src/cli/opaque.rs:125`, `:202` — `opaque reset` CLI identifier
|
||||
dispatch on `@`; lowercase the username branch input.
|
||||
|
||||
Post-migration, the DB is fully lowercase so `WHERE username = 'alice'`
|
||||
matches. The mixed-case-DB-during-transition state cannot serve
|
||||
traffic because the boot flow either (a) auto-renames the singleton
|
||||
rows before `AppState` assembles, or (b) refuses to boot on collision
|
||||
groups.
|
||||
|
||||
### 4. NextCloud DAV surface
|
||||
|
||||
Two coordinated changes on the NC surface:
|
||||
|
||||
- `src/interfaces/nextcloud/basic_auth_middleware.rs:94-134` — decoded
|
||||
`raw_username` from the Basic Auth header, lowercase the whole
|
||||
string. Safe for the `user~drive_uuid` multi-drive format because
|
||||
UUID hex is `[0-9a-f-]` which lowercases to itself.
|
||||
- `src/interfaces/nextcloud/basic_auth_middleware.rs:307-323`
|
||||
(`parse_basic_auth` helper) — lowercase the username portion before
|
||||
returning.
|
||||
- `src/interfaces/nextcloud/session.rs:90-111`
|
||||
(`extract_url_user`) — lowercase the returned `Cow<'_, str>` value
|
||||
from URL decode. The cross-check comparison at `session.rs:157-161`
|
||||
(`url_user != session.raw_username`) then compares normalized vs
|
||||
normalized — no change needed at the comparison site itself.
|
||||
|
||||
Downstream `session.raw_username` consumers (WebDAV / OCS href
|
||||
builders, MOVE Destination parsers, avatar / trashbin handlers) all
|
||||
pass through and emit lowercase automatically — no per-site change
|
||||
needed.
|
||||
|
||||
**Client compatibility:** NC / DAVX5 clients that cached URLs like
|
||||
`/remote.php/dav/files/Alice/…` continue to work through the migration
|
||||
because the server accepts uppercase URL segments **indefinitely**
|
||||
(the Basic Auth middleware + `extract_url_user` both lowercase on
|
||||
decode). No forced client upgrade or reconfiguration. PROPFIND
|
||||
response bodies emit lowercase hrefs (from canonical
|
||||
`session.raw_username`), which well-behaved clients update on next
|
||||
sync.
|
||||
|
||||
Expected per-client behavior on first PROPFIND after upgrade:
|
||||
|
||||
- **Nextcloud desktop** — prompts a one-time re-sync notification
|
||||
when it notices the account URL case changed. Files re-verify
|
||||
via ETag, so no re-upload; the re-sync completes in
|
||||
seconds-to-minutes depending on file count. Users click through
|
||||
the reconnect dialog.
|
||||
- **DAVX5** (calendars, contacts) — silent update of the internal
|
||||
`principal-URL`; user sees no dialog.
|
||||
- **NC mobile app** — silent refresh of the account tile.
|
||||
- **Older / misbehaving clients** — may create a duplicate account
|
||||
profile (rare, cosmetic, not destructive).
|
||||
|
||||
**Zero data risk in every path.** The chroot ACL keys on
|
||||
`user_id`, not username, so files, calendars, contacts, and
|
||||
grants all follow the user across the rename. The blast radius
|
||||
is a one-time UX notification, not lost bytes.
|
||||
|
||||
**Power-user pre-emption** (worth documenting in CHANGELOG): ops
|
||||
who want to avoid the re-sync prompt entirely can, before
|
||||
upgrading, log into each NC desktop client and manually update
|
||||
the account URL from `.../USERNAME` to lowercase. Cheap
|
||||
prophylactic for organizations rolling out to non-technical
|
||||
users.
|
||||
|
||||
### 5. Chunked-upload directory rename
|
||||
|
||||
`src/infrastructure/services/nextcloud_chunked_upload_service.rs:99-103`
|
||||
uses `user.username` as an on-disk directory name AND as an in-memory
|
||||
cache key. Post-migration, `user.username` becomes lowercase; any
|
||||
in-flight upload for `Alice` at migration time strands the on-disk
|
||||
`base_dir/Alice/upload_xxx/` directory and orphans its cache entry.
|
||||
|
||||
The migration command SHOULD walk `base_dir/*/` and rename any
|
||||
mixed-case subdirectory to its lowercase form. Collision handling
|
||||
(both `Alice/` and `alice/` present) → merge contents; else simple
|
||||
rename. In practice this is likely a no-op — chunked-upload state
|
||||
is ephemeral, and simultaneous mixed-case uploads by the same user
|
||||
are rare.
|
||||
|
||||
**Implementation status:** deferred. Chunked-upload state is
|
||||
ephemeral: any in-flight upload that gets stranded is retryable
|
||||
by the client (the upload session's timeout eventually purges the
|
||||
stale dir; the client retries with a fresh `upload_id`, this time
|
||||
under the lowercase username). Wiring the dir-walk into the CLI
|
||||
adds ~40 lines of async filesystem code (walk, collision merge,
|
||||
mtime-preserving move) and a new `--chunk-dir <path>` arg — the
|
||||
CLI otherwise doesn't need to know about the storage-path
|
||||
config layer. Not worth it for a rare no-op; add if user reports
|
||||
show a real problem.
|
||||
|
||||
**Ops manual step** — if a migration is run WHILE an upload is
|
||||
in flight, ops can either restart the affected client (the
|
||||
upload session is stateful across a `create → chunks → complete`
|
||||
cycle, so the client will retry from scratch) or manually
|
||||
`mv base_dir/Alice base_dir/alice` after the DB migration
|
||||
completes.
|
||||
|
||||
### 6. Boot-time verification
|
||||
|
||||
New module `src/common/username_migration.rs` exposing:
|
||||
|
||||
```rust
|
||||
pub async fn verify_all_usernames_lowercase(pool: &PgPool) -> Result<(), String>
|
||||
```
|
||||
|
||||
Runs after `sqlx::migrate!()` completes, before `AppState` is
|
||||
assembled. Query:
|
||||
|
||||
```sql
|
||||
SELECT id, username, created_at, last_login_at
|
||||
FROM auth.users
|
||||
WHERE username <> LOWER(username)
|
||||
-- NULL usernames (OPAQUE-migrated accounts) are already filtered
|
||||
-- out by SQL semantics: NULL <> anything yields NULL, which
|
||||
-- WHERE excludes. Explicit for the reviewer's benefit.
|
||||
-- add is_deleted / disabled filter if such a flag exists
|
||||
ORDER BY LOWER(username),
|
||||
(last_login_at IS NULL),
|
||||
last_login_at DESC NULLS LAST,
|
||||
created_at ASC
|
||||
LIMIT 200; -- soft cap on error-message size
|
||||
```
|
||||
|
||||
If empty → boot proceeds. If non-empty → format the FATAL error and
|
||||
return `Err(String)`. `main.rs` propagates via `?` to a non-zero
|
||||
process exit.
|
||||
|
||||
Boot only READS `auth.users`; never WRITES. This is the "explicit
|
||||
migration required" enforcement layer.
|
||||
|
||||
**Error message format** (self-sufficient — no docs required at 3 AM):
|
||||
|
||||
```
|
||||
FATAL: cannot start — <N> user account(s) have non-lowercase usernames.
|
||||
|
||||
Before this version can boot, run the migration:
|
||||
|
||||
oxicloud migrate lowercase-usernames --dry-run # preview
|
||||
oxicloud migrate lowercase-usernames # apply
|
||||
|
||||
Affected accounts (up to 20 shown; full list via the dry-run):
|
||||
|
||||
Alice (id: a1b2c3d4-... last_login: 2026-08-01)
|
||||
BOB (id: 9abc0000-... last_login: never)
|
||||
...
|
||||
|
||||
The migration handles case-collisions (Alice + alice → alice keeps
|
||||
the name based on most recent login; the other gets alice-2 suffix).
|
||||
Sessions and grants survive the rename (they key on user_id).
|
||||
```
|
||||
|
||||
### 7. Migration CLI
|
||||
|
||||
New action under `oxicloud migrate`:
|
||||
|
||||
```rust
|
||||
// src/cli/migrate.rs — extend the Action enum
|
||||
Action::LowercaseUsernames { dry_run: bool }
|
||||
```
|
||||
|
||||
Following the shape of `run_nfc_filenames`:
|
||||
|
||||
- Load all active users (skip soft-deleted / disabled AND rows
|
||||
where `username IS NULL` — OPAQUE-migrated accounts have no
|
||||
username string to normalize)
|
||||
- Group by `LOWER(username)`
|
||||
- For each group:
|
||||
- Single-member group with mixed-case name → UPDATE to lowercase
|
||||
- Multi-member group (collision) → apply tiebreak
|
||||
`(last_login_at DESC NULLS LAST, created_at ASC)`, winner UPDATEs
|
||||
to lowercase, losers UPDATE to `<lowercase>-2`, `-3`, … (increment
|
||||
until free)
|
||||
- Per-row `println!` log:
|
||||
`NORMALIZE user=<uuid> '<before>' ({}B) → '<after>' ({}B)`
|
||||
- Summary at end: scanned / already-lowercase / normalized /
|
||||
collision-resolved / renamed-to-suffix
|
||||
- `--dry-run` guards all UPDATEs
|
||||
|
||||
After the DB pass, the chunked-upload directory rename step (see
|
||||
Deliverable 5) is deferred; run manually only if in-flight uploads
|
||||
were live at migration time.
|
||||
|
||||
Suffix search reuses the pattern from
|
||||
`find_free_folder_duplicate_name` in the existing NFC migration —
|
||||
increment-until-free loop, starting at `-2`, probing until an
|
||||
unused suffix is found. Robust against pre-existing rows like
|
||||
`alice-2` already being taken (the probe just steps past them
|
||||
to `-3`, `-4`, …).
|
||||
|
||||
Extracted into a shared public helper in
|
||||
`src/common/username_migration.rs`:
|
||||
|
||||
```rust
|
||||
pub async fn find_free_username_suffix(pool: &PgPool, base: &str) -> Result<String, sqlx::Error>
|
||||
```
|
||||
|
||||
Both the migration CLI AND the un-soft-delete API (Design decision
|
||||
7) call this helper — same collision-resolution behavior by
|
||||
construction, no drift risk between the two paths.
|
||||
|
||||
Bounded at 10,000 as a safety cap. The probability of reaching
|
||||
that in a real deployment is negligible — it would require ~10 K
|
||||
distinct accounts all originally cased differently but sharing
|
||||
the same lowercase form (a normal collision is 2-3 accounts, not
|
||||
10 K). If the cap ever fires, something is very wrong with the
|
||||
account universe and the migration ABORTs with a loud error
|
||||
rather than silently truncating — the loud abort IS the
|
||||
detection mechanism.
|
||||
|
||||
### 8. Test seed audit
|
||||
|
||||
Sweep-verified: existing test seeds all produce lowercase or NULL
|
||||
usernames. Worth one more grep pass to ensure no test fixture INSERTs
|
||||
`INSERT INTO auth.users … 'AliceTest'` — if any exist, lowercase them
|
||||
in the same commit to avoid CI refuse-to-boot regressions.
|
||||
|
||||
Files verified (all safe):
|
||||
- `src/infrastructure/repositories/pg/user_pg_repository.rs:1786`
|
||||
- `src/infrastructure/repositories/pg/opaque_pg_repository.rs:339`
|
||||
(NULL)
|
||||
- `src/application/services/auth_application_service.rs:4982` (NULL)
|
||||
- `src/application/services/subject_group_service.rs:796` (NULL)
|
||||
- `src/bin/load-seed.rs:414`, `:446` (`load_user_XXXX` — lowercase)
|
||||
- `src/mount_it_support.rs:61` (`make_user(name)` — verify callers)
|
||||
- `tests/common/init-test-schema.sh:40` (`ci-admin` — lowercase)
|
||||
|
||||
### 9. Cosmetic side-effects (worth noting in CHANGELOG, non-blocking)
|
||||
|
||||
- `src/interfaces/nextcloud/avatar_handler.rs:283` — `pick_color`
|
||||
derives a deterministic tile color from username bytes. Users whose
|
||||
canonical username had uppercase letters will get a different
|
||||
fallback-avatar tile color after the migration. One-time cosmetic
|
||||
change.
|
||||
- **NC desktop may perform a one-time re-sync** — see Deliverable 4.
|
||||
|
||||
### 10. Documentation
|
||||
|
||||
Release notes / CHANGELOG entry is NOT part of this PR — the
|
||||
canonical repo's maintainer handles release notes at version-bump
|
||||
time. This PR just leaves the notes-worthy items enumerated here
|
||||
so the maintainer has the bullets to pick from when the next
|
||||
version ships:
|
||||
|
||||
- Server auto-lowercases non-colliding mixed-case usernames at
|
||||
first boot. No ops action needed for the common case.
|
||||
- On `LOWER(username)` collision (`Alice` + `alice` both active),
|
||||
the server refuses to boot; ops runs `oxicloud migrate
|
||||
lowercase-usernames`. Exact CLI command shown in the refusal.
|
||||
- Nextcloud desktop clients will prompt for a one-time re-sync on
|
||||
first PROPFIND after upgrade. Files are ETag-verified, not
|
||||
re-uploaded. DAVX5 and NC mobile handle the URL case change
|
||||
silently. **No forced client upgrade or reconfiguration** —
|
||||
server accepts uppercase URL segments indefinitely.
|
||||
- Optional pre-emption for non-technical users: ops can manually
|
||||
update the account URL to lowercase in each NC desktop client
|
||||
before upgrading, avoiding the re-sync prompt entirely.
|
||||
- Usernames become lowercase in ALL UI display surfaces (share
|
||||
dialogs, activity feeds, admin panels, PROPFIND response
|
||||
bodies, notification bell). Login identity unchanged from the
|
||||
user's POV (they can still type any case at the login form).
|
||||
- Avatar fallback color may change for users with previously-
|
||||
uppercase usernames.
|
||||
- Note: `display_name` is a possible follow-up if users miss
|
||||
capitalisation for display — deferred pending demand signal, no
|
||||
compat cost to adding later.
|
||||
|
||||
The two docs that DO ship with this PR:
|
||||
|
||||
- `docs/config/env.md` — note the boot-time check + migration command.
|
||||
- `docs/install/binary.md` — upgrade-from-case-sensitive section.
|
||||
|
||||
### 11. Test coverage
|
||||
|
||||
- **Unit**: `validate_username("Alice")` returns `Ok("alice")`;
|
||||
`validate_username("alice-")` returns `Err(...)` unchanged;
|
||||
`validate_username(" Alice ")` returns `Ok("alice")`.
|
||||
- **Unit**: `format_refusal_message_collisions` — 1 group renders
|
||||
canonical + members + CLI; > 10 groups renders overflow tail;
|
||||
total-affected-count sums across groups.
|
||||
- **Hurl** (`tests/api/lowercase_usernames.hurl`, new): register a
|
||||
user with `MixedCase`, assert DB stores `mixedcase`; log in with
|
||||
`MIXEDCASE` and `mixedcase` — both succeed; rename to `NewName`,
|
||||
assert `newname` stored; NC Basic Auth accepts `MixedCase:pass`,
|
||||
`MIXEDCASE:pass`, `mixedcase:pass`.
|
||||
- **Manual** (against dev DB, not CI):
|
||||
- Auto-rename path: `UPDATE auth.users SET username='Alice' WHERE
|
||||
username='alice'` (no collision); boot server → verify audit log
|
||||
line + WARN summary, row is `alice` after boot, service starts.
|
||||
- Collision path: `INSERT INTO auth.users … 'Alice'` on top of
|
||||
existing `alice`; boot server → verify refusal message names both
|
||||
rows + exact CLI shown, server exits non-zero.
|
||||
- `oxicloud migrate lowercase-usernames --dry-run` → verify report
|
||||
of collision + tiebreak decision
|
||||
- `oxicloud migrate lowercase-usernames` → verify apply, one row
|
||||
keeps `alice`, other gets `alice-2`
|
||||
- Boot again → succeeds (Clean outcome)
|
||||
- `curl -u ALICE:pass https://oxicloud/remote.php/dav/files/ALICE/…`
|
||||
→ succeeds (accepts uppercase input, resolves to lowercase user)
|
||||
|
||||
## Cache-and-consistency observations (informational)
|
||||
|
||||
- `src/infrastructure/services/login_lockout_service.rs:33,68-89` —
|
||||
already lowercases the key at line 58. No code change; comment
|
||||
becomes factual not incidental.
|
||||
- `src/application/services/app_password_service.rs:89,317-323` —
|
||||
BLAKE3-keyed cache using the raw wire username. Post-normalization,
|
||||
both sides normalize consistently → cache stays coherent. 300 s TTL
|
||||
self-heals any transitional window.
|
||||
- `NC_CHROOT_CACHE` in `basic_auth_middleware.rs:34-40` — keyed on
|
||||
`Uuid`, not username. Unaffected.
|
||||
|
||||
## Critical files
|
||||
|
||||
Full enumeration in the Deliverables sections above. Grouped summary:
|
||||
|
||||
**Ingest normalizer:**
|
||||
- `src/domain/entities/user.rs` (signature change + callers)
|
||||
|
||||
**Application services (write callers):**
|
||||
- `src/application/services/auth_application_service.rs`
|
||||
- `src/application/services/magic_link_invite_service.rs`
|
||||
|
||||
**Repositories (lookup normalization):**
|
||||
- `src/infrastructure/repositories/pg/user_pg_repository.rs`
|
||||
- `src/application/services/storage_usage_service.rs`
|
||||
- `src/cli/opaque.rs`
|
||||
|
||||
**NextCloud DAV surface:**
|
||||
- `src/interfaces/nextcloud/basic_auth_middleware.rs`
|
||||
- `src/interfaces/nextcloud/session.rs`
|
||||
- `src/infrastructure/services/nextcloud_chunked_upload_service.rs`
|
||||
|
||||
**New files:**
|
||||
- `src/common/username_migration.rs`
|
||||
- `tests/api/lowercase_usernames.hurl`
|
||||
|
||||
**Main entry:**
|
||||
- `src/main.rs` (call verifier after `sqlx::migrate!()`)
|
||||
|
||||
**Migration CLI:**
|
||||
- `src/cli/migrate.rs`
|
||||
|
||||
**Docs:**
|
||||
- `CHANGELOG.md`
|
||||
- `docs/config/env.md`
|
||||
- `docs/install/binary.md`
|
||||
|
||||
## Delivery order
|
||||
|
||||
1. Change `validate_username` signature to return `Result<String>` —
|
||||
one file.
|
||||
2. Fix OIDC JIT derivation
|
||||
(`auth_application_service.rs:4649-4690`) to lowercase before
|
||||
passing to `User::new` — explicit change beyond the entity
|
||||
normalizer's compile-time catches.
|
||||
3. Iterate on compile errors — the return-type change catches every
|
||||
downstream write-site.
|
||||
4. Update repository lookup methods (`user_pg_repository.rs`,
|
||||
`storage_usage_service.rs`, `cli/opaque.rs`) to internally
|
||||
lowercase input before `.bind()`.
|
||||
5. Update NC `basic_auth_middleware.rs` (lowercase `raw_username` at
|
||||
decode) + `session.rs::extract_url_user` (lowercase return).
|
||||
6. Add the boot-time verification helper (`src/common/username_migration.rs`)
|
||||
+ wire into `main.rs`.
|
||||
7. Extend `oxicloud migrate` with `lowercase-usernames [--dry-run]` —
|
||||
DB pass + chunked-upload directory rename.
|
||||
8. Test seed audit (grep pass).
|
||||
9. Add hurl coverage.
|
||||
10. Admin docs update (`docs/config/env.md` boot-check subsection +
|
||||
`docs/install/binary.md` upgrade section). CHANGELOG is Dio's
|
||||
job at version-bump time — not part of this PR.
|
||||
11. Manual smoke test against dev DB.
|
||||
12. PR to canonical.
|
||||
|
||||
## Total scope estimate
|
||||
|
||||
~6-8 hours of careful work. Larger than the initial estimate because
|
||||
of these sweep-surfaced additions:
|
||||
|
||||
- OIDC JIT explicit fix (small).
|
||||
- Chunked-upload directory rename step in the migration (~30 min).
|
||||
- Test-seed audit (~15 min).
|
||||
- More lookup callsites than initially thought.
|
||||
|
||||
The shape is uniform (`to_ascii_lowercase()` at every touchpoint) and
|
||||
the compiler catches missed entity-write sites via the
|
||||
`Result<String>` signature change. The parts NOT caught by the
|
||||
compiler (OIDC JIT, lookup normalizers, NC URL segment, chunked-upload
|
||||
directory) are the ones needing careful review — enumerated above.
|
||||
|
||||
## References
|
||||
|
||||
- Issue: [#691](https://github.com/AtalayaLabs/OxiCloud/issues/691)
|
||||
- Related feature restrictions today:
|
||||
- `validate_username` at
|
||||
`src/domain/entities/user.rs:884-916`
|
||||
- `@`-disjoint dispatch at
|
||||
`src/application/services/auth_application_service.rs:1018`
|
||||
- Related project docs:
|
||||
- `docs/plan/auth-simplification.md` — the broader auth surface this
|
||||
fits within
|
||||
- Prior similar migration:
|
||||
`oxicloud migrate nfc-filenames` in `src/cli/migrate.rs`
|
||||
@@ -33,3 +33,22 @@ export function copyFolders(folderIds: string[], targetFolderId: string | null):
|
||||
target_folder_id: targetFolderId
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Stream a multi-item selection as a server-built zip (`POST /api/batch/download`
|
||||
* — folders included, unlike the legacy per-item loop). The caller names and
|
||||
* saves the returned blob.
|
||||
*/
|
||||
export async function downloadBatch(fileIds: string[], folderIds: string[]): Promise<Blob> {
|
||||
const res = await apiFetch('/api/batch/download', {
|
||||
method: 'POST',
|
||||
credentials: 'same-origin',
|
||||
headers: { ...JSON_HEADERS, ...getCsrfHeaders() },
|
||||
body: JSON.stringify({ file_ids: fileIds, folder_ids: folderIds })
|
||||
});
|
||||
if (!res.ok) {
|
||||
const e = (await res.json().catch(() => ({}))) as { error?: string; message?: string };
|
||||
throw new Error(e.error || e.message || `batch download failed: ${res.status}`);
|
||||
}
|
||||
return res.blob();
|
||||
}
|
||||
|
||||
@@ -139,3 +139,24 @@ export async function removeFavorite(type: ItemType, id: string): Promise<void>
|
||||
});
|
||||
if (!res.ok) throw new Error(`remove favorite failed: ${res.status}`);
|
||||
}
|
||||
|
||||
/** One item for the batch favorites call. */
|
||||
export interface FavoriteBatchItem {
|
||||
item_id: string;
|
||||
item_type: ItemType;
|
||||
}
|
||||
|
||||
/**
|
||||
* Batch-add favorites via `POST /api/favorites/batch` — a single round trip
|
||||
* for the whole selection (used by the files page and search results batch bar).
|
||||
*/
|
||||
export async function addFavoritesBatch(items: FavoriteBatchItem[]): Promise<void> {
|
||||
if (items.length === 0) return;
|
||||
const res = await apiFetch('/api/favorites/batch', {
|
||||
method: 'POST',
|
||||
credentials: 'same-origin',
|
||||
headers: { 'Content-Type': 'application/json', ...getCsrfHeaders() },
|
||||
body: JSON.stringify({ items })
|
||||
});
|
||||
if (!res.ok) throw new Error(`batch favorites failed: ${res.status}`);
|
||||
}
|
||||
|
||||
@@ -9,6 +9,16 @@ import type { ItemType } from '$lib/api/types';
|
||||
export interface ShareMeta {
|
||||
item_type: ItemType;
|
||||
item_name: string;
|
||||
/** The shared item's id — file shares use it to build the preview src. */
|
||||
item_id: string;
|
||||
/**
|
||||
* File shares only: resolved by the server at read time so the landing
|
||||
* page can inline a media preview (video player / image) instead of a
|
||||
* bare download button. Absent for folder shares.
|
||||
*/
|
||||
mime_type?: string;
|
||||
/** File shares only: the shared file's size in bytes. */
|
||||
size?: number;
|
||||
}
|
||||
|
||||
export interface ShareFolderEntry {
|
||||
|
||||
@@ -0,0 +1,354 @@
|
||||
<script lang="ts">
|
||||
import Icon from '$lib/icons/Icon.svelte';
|
||||
import { t } from '$lib/i18n/index.svelte';
|
||||
import {
|
||||
defaultFilterState,
|
||||
type DateKey,
|
||||
type ResourceFilterState,
|
||||
type SizeKey,
|
||||
type TypeKey
|
||||
} from '$lib/utils/searchFilters';
|
||||
|
||||
interface Props {
|
||||
/** Bindable full filter state (keyword + toggles + presets). */
|
||||
value?: ResourceFilterState;
|
||||
/** Advanced section (type/size/date/recursive) expanded. */
|
||||
expanded?: boolean;
|
||||
placeholder?: string;
|
||||
/** Debounce for the keyword input (ms). */
|
||||
debounceMs?: number;
|
||||
/** Hide the recursive toggle (a surface where scope is owned elsewhere). */
|
||||
hideRecursive?: boolean;
|
||||
}
|
||||
|
||||
let {
|
||||
value = $bindable(defaultFilterState()),
|
||||
expanded = $bindable(false),
|
||||
placeholder = t('filter.placeholder', 'Search this folder and subfolders…'),
|
||||
debounceMs = 300,
|
||||
hideRecursive = false
|
||||
}: Props = $props();
|
||||
|
||||
// Option label lists reuse the /search page's i18n keys — the vocabularies
|
||||
// themselves (extensions, byte bounds) live in the shared searchFilters util.
|
||||
const TYPES: { v: TypeKey; l: string }[] = [
|
||||
{ v: 'all', l: t('search.type.all', 'All types') },
|
||||
{ v: 'image', l: t('search.type.image', 'Images') },
|
||||
{ v: 'video', l: t('search.type.video', 'Videos') },
|
||||
{ v: 'document', l: t('search.type.document', 'Documents') },
|
||||
{ v: 'audio', l: t('search.type.audio', 'Audio') },
|
||||
{ v: 'archive', l: t('search.type.archive', 'Archives') }
|
||||
];
|
||||
const SIZES: { v: SizeKey; l: string }[] = [
|
||||
{ v: 'all', l: t('search.size.all', 'Any size') },
|
||||
{ v: 'small', l: t('search.size.small', '< 1 MB') },
|
||||
{ v: 'medium', l: t('search.size.medium', '1–100 MB') },
|
||||
{ v: 'large', l: t('search.size.large', '> 100 MB') }
|
||||
];
|
||||
const DATES: { v: DateKey; l: string }[] = [
|
||||
{ v: 'all', l: t('search.date.all', 'Any time') },
|
||||
{ v: 'day', l: t('search.date.day', 'Past 24 hours') },
|
||||
{ v: 'week', l: t('search.date.week', 'Past week') },
|
||||
{ v: 'month', l: t('search.date.month', 'Past month') },
|
||||
{ v: 'year', l: t('search.date.year', 'Past year') }
|
||||
];
|
||||
|
||||
// Keyword buffer: typing updates the buffer immediately (responsive input)
|
||||
// and pushes into `value.query` debounced, so a keystroke doesn't fire a
|
||||
// backend search per character. `lastPushed` disambiguates our own pushes
|
||||
// from external writes (e.g. the page's clear-filter Escape path), which
|
||||
// flow back into the buffer via the sync effect below.
|
||||
let keyword = $state(value.query);
|
||||
let lastPushed = value.query;
|
||||
let timer: ReturnType<typeof setTimeout> | null = null;
|
||||
|
||||
$effect(() => {
|
||||
// External `value.query` change → adopt it into the input buffer.
|
||||
const external = value.query;
|
||||
if (external !== lastPushed) {
|
||||
if (timer) {
|
||||
clearTimeout(timer);
|
||||
timer = null;
|
||||
}
|
||||
keyword = external;
|
||||
lastPushed = external;
|
||||
}
|
||||
});
|
||||
|
||||
$effect(() => {
|
||||
// No reactive deps — teardown-only, clearing a pending debounce on destroy.
|
||||
return () => {
|
||||
if (timer) clearTimeout(timer);
|
||||
};
|
||||
});
|
||||
|
||||
function pushKeyword(v: string) {
|
||||
if (timer) {
|
||||
clearTimeout(timer);
|
||||
timer = null;
|
||||
}
|
||||
value.query = v;
|
||||
lastPushed = v;
|
||||
}
|
||||
|
||||
function handleInput(e: Event) {
|
||||
const v = (e.target as HTMLInputElement).value;
|
||||
keyword = v;
|
||||
if (timer) clearTimeout(timer);
|
||||
timer = setTimeout(() => {
|
||||
timer = null;
|
||||
pushKeyword(v);
|
||||
}, debounceMs);
|
||||
}
|
||||
|
||||
function onInputKeydown(e: KeyboardEvent) {
|
||||
// Escape clears the keyword locally and never reaches the page-level
|
||||
// handler (which would otherwise also clear the selection / filters).
|
||||
if (e.key === 'Escape') {
|
||||
e.stopPropagation();
|
||||
clearKeyword();
|
||||
} else if (e.key === 'Enter') {
|
||||
// Enter flushes the debounce for an immediate search.
|
||||
e.preventDefault();
|
||||
pushKeyword(keyword);
|
||||
}
|
||||
}
|
||||
|
||||
function clearKeyword() {
|
||||
keyword = '';
|
||||
pushKeyword('');
|
||||
}
|
||||
|
||||
const activeCount = $derived(
|
||||
(value.type !== 'all' ? 1 : 0) + (value.size !== 'all' ? 1 : 0) + (value.date !== 'all' ? 1 : 0)
|
||||
);
|
||||
</script>
|
||||
|
||||
<div class="sfb" data-testid="search-filter-bar">
|
||||
<div class="sfb__row">
|
||||
<div class="sfb__input-wrap">
|
||||
<span class="sfb__magnifier"><Icon name="search" /></span>
|
||||
<input
|
||||
class="sfb__input"
|
||||
type="search"
|
||||
{placeholder}
|
||||
aria-label={t('filter.keyword', 'Keyword')}
|
||||
data-testid="filter-keyword-input"
|
||||
value={keyword}
|
||||
oninput={handleInput}
|
||||
onkeydown={onInputKeydown}
|
||||
/>
|
||||
{#if keyword.length > 0}
|
||||
<button
|
||||
class="sfb__clear"
|
||||
type="button"
|
||||
aria-label={t('filter.clear_keyword', 'Clear search')}
|
||||
data-testid="filter-clear-keyword-btn"
|
||||
onclick={clearKeyword}
|
||||
>
|
||||
<Icon name="times" />
|
||||
</button>
|
||||
{/if}
|
||||
</div>
|
||||
<button
|
||||
class="sfb__toggle"
|
||||
class:sfb__toggle--active={activeCount > 0}
|
||||
type="button"
|
||||
aria-expanded={expanded}
|
||||
aria-label={t('filter.advanced', 'Filters')}
|
||||
title={t('filter.advanced', 'Filters')}
|
||||
data-testid="filter-advanced-toggle-btn"
|
||||
onclick={() => (expanded = !expanded)}
|
||||
>
|
||||
<Icon name="sliders-h" />
|
||||
{#if activeCount > 0}
|
||||
<span class="sfb__badge">{activeCount}</span>
|
||||
{/if}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{#if expanded}
|
||||
<div class="sfb__advanced" data-testid="filter-advanced-row">
|
||||
<label class="sfb__field">
|
||||
<span class="sfb__label">{t('search.type_label', 'Type')}</span>
|
||||
<select bind:value={value.type}>
|
||||
{#each TYPES as opt (opt.v)}
|
||||
<option value={opt.v}>{opt.l}</option>
|
||||
{/each}
|
||||
</select>
|
||||
</label>
|
||||
<label class="sfb__field">
|
||||
<span class="sfb__label">{t('search.size_label', 'Size')}</span>
|
||||
<select bind:value={value.size}>
|
||||
{#each SIZES as opt (opt.v)}
|
||||
<option value={opt.v}>{opt.l}</option>
|
||||
{/each}
|
||||
</select>
|
||||
</label>
|
||||
<label class="sfb__field">
|
||||
<span class="sfb__label">{t('search.date_label', 'Date')}</span>
|
||||
<select bind:value={value.date}>
|
||||
{#each DATES as opt (opt.v)}
|
||||
<option value={opt.v}>{opt.l}</option>
|
||||
{/each}
|
||||
</select>
|
||||
</label>
|
||||
{#if !hideRecursive}
|
||||
<label class="sfb__check">
|
||||
<input type="checkbox" bind:checked={value.recursive} />
|
||||
<span>{t('filter.recursive', 'Include subfolders')}</span>
|
||||
</label>
|
||||
{/if}
|
||||
</div>
|
||||
{/if}
|
||||
</div>
|
||||
|
||||
<style>
|
||||
.sfb {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: var(--space-1);
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.sfb__row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: var(--space-1);
|
||||
}
|
||||
|
||||
.sfb__input-wrap {
|
||||
position: relative;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
}
|
||||
|
||||
.sfb__magnifier {
|
||||
position: absolute;
|
||||
left: 0.6rem;
|
||||
color: var(--color-text-secondary);
|
||||
pointer-events: none;
|
||||
}
|
||||
|
||||
.sfb__input {
|
||||
width: 100%;
|
||||
padding: 0.45rem 2rem 0.45rem 2.1rem;
|
||||
border: 1px solid var(--color-border);
|
||||
border-radius: var(--radius-md);
|
||||
background: var(--color-bg-surface);
|
||||
color: var(--color-text);
|
||||
font-size: var(--text-sm);
|
||||
}
|
||||
|
||||
.sfb__input:focus {
|
||||
outline: none;
|
||||
border-color: var(--color-accent);
|
||||
}
|
||||
|
||||
.sfb__input::-webkit-search-cancel-button {
|
||||
-webkit-appearance: none;
|
||||
appearance: none;
|
||||
}
|
||||
|
||||
.sfb__clear {
|
||||
position: absolute;
|
||||
right: 0.4rem;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
width: 1.4rem;
|
||||
height: 1.4rem;
|
||||
padding: 0;
|
||||
border: none;
|
||||
border-radius: var(--radius-sm);
|
||||
background: none;
|
||||
color: var(--color-text-secondary);
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.sfb__clear:hover {
|
||||
background: var(--color-bg-hover);
|
||||
color: var(--color-text);
|
||||
}
|
||||
|
||||
.sfb__toggle {
|
||||
position: relative;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
width: 2.2rem;
|
||||
height: 2.2rem;
|
||||
padding: 0;
|
||||
border: 1px solid var(--color-border);
|
||||
border-radius: var(--radius-md);
|
||||
background: var(--color-bg-surface);
|
||||
color: var(--color-text);
|
||||
cursor: pointer;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.sfb__toggle:hover {
|
||||
background: var(--color-bg-hover);
|
||||
}
|
||||
|
||||
.sfb__toggle--active {
|
||||
border-color: var(--color-accent);
|
||||
color: var(--color-accent);
|
||||
}
|
||||
|
||||
.sfb__badge {
|
||||
position: absolute;
|
||||
top: -0.4rem;
|
||||
right: -0.4rem;
|
||||
min-width: 1rem;
|
||||
height: 1rem;
|
||||
padding: 0 0.2rem;
|
||||
border-radius: var(--radius-sm);
|
||||
background: var(--color-accent);
|
||||
color: var(--color-on-accent);
|
||||
font-size: var(--text-xs, 0.7rem);
|
||||
line-height: 1rem;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.sfb__advanced {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
align-items: center;
|
||||
gap: var(--space-2);
|
||||
}
|
||||
|
||||
.sfb__field {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.4rem;
|
||||
}
|
||||
|
||||
.sfb__label {
|
||||
color: var(--color-text-secondary);
|
||||
font-size: var(--text-sm);
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.sfb__field select {
|
||||
padding: 0.3rem 0.4rem;
|
||||
border: 1px solid var(--color-border);
|
||||
border-radius: var(--radius-sm);
|
||||
background: var(--color-bg-surface);
|
||||
color: var(--color-text);
|
||||
font-size: var(--text-sm);
|
||||
max-width: 9rem;
|
||||
}
|
||||
|
||||
.sfb__check {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.4rem;
|
||||
color: var(--color-text);
|
||||
font-size: var(--text-sm);
|
||||
cursor: pointer;
|
||||
white-space: nowrap;
|
||||
}
|
||||
</style>
|
||||
@@ -0,0 +1,241 @@
|
||||
// Shared batch-actions composable for resource list views (files page,
|
||||
// search results, …).
|
||||
//
|
||||
// Extracted verbatim from the files page so every surface that can select
|
||||
// items shares one implementation of the batch favorite / download /
|
||||
// delete / move / copy flows. Surfaces differ in (a) the id→item index,
|
||||
// (b) what "refresh" means (folder reload vs search re-run) and (c) how a
|
||||
// favorites flip reaches the rows — those differences are injected via
|
||||
// the callbacks in `ResourceActionsOptions`.
|
||||
import type { FileItem, FolderItem, ItemType } from '$lib/api/types';
|
||||
import { downloadBatch } from '$lib/api/endpoints/batch';
|
||||
import { deleteFile, fileDownloadUrl } from '$lib/api/endpoints/files';
|
||||
import { deleteFolder } from '$lib/api/endpoints/folders';
|
||||
import { addFavoritesBatch } from '$lib/api/endpoints/favorites';
|
||||
import { t } from '$lib/i18n/index.svelte';
|
||||
import { confirmDialog } from '$lib/stores/dialogs.svelte';
|
||||
import { ui } from '$lib/stores/ui.svelte';
|
||||
import { errorToast } from '$lib/utils/errors';
|
||||
import { mapLimit } from '$lib/utils/mapLimit';
|
||||
|
||||
/** A minimal actionable item reference (dialog props, favorites payload). */
|
||||
export interface ActionTarget {
|
||||
id: string;
|
||||
name: string;
|
||||
kind: ItemType;
|
||||
}
|
||||
|
||||
export interface ResourceActionsOptions {
|
||||
/** Current on-screen rows, read at action time (fresh, never stale). */
|
||||
getItems: () => ReadonlyArray<FileItem | FolderItem>;
|
||||
/** Selected ids — the page's SvelteSet mirror of the list's selection. */
|
||||
getSelected: () => ReadonlySet<string>;
|
||||
clearSelection: () => void;
|
||||
/** After a successful delete: reload the listing or re-run the search. */
|
||||
onChanged: () => void | Promise<void>;
|
||||
/** Extra bookkeeping after a delete (e.g. the session/quota refresh). */
|
||||
afterDelete?: () => void;
|
||||
/**
|
||||
* After favorites succeed, update rows in place (keeps scroll position on
|
||||
* infinite-scroll pages). Defaults to flipping `is_favorite` on the items
|
||||
* returned by `getItems()` — sufficient for plain DTO `$state` arrays.
|
||||
*/
|
||||
onFavoritesApplied?: (ids: ReadonlySet<string>) => void;
|
||||
}
|
||||
|
||||
/** Name for a server-zipped multi-item archive (matches the legacy format). */
|
||||
export function batchZipName(): string {
|
||||
// eslint-disable-next-line svelte/prefer-svelte-reactivity -- filename stamp, never read reactively
|
||||
const stamp = new Date().toISOString().replace('T', ' ').replace(/\..*/, '').replace(/:/g, '-');
|
||||
return `oxicloud ${stamp}.zip`;
|
||||
}
|
||||
|
||||
/** Trigger a browser download of `blob` as `name`. */
|
||||
function saveBlob(blob: Blob, name: string): void {
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = name;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
a.remove();
|
||||
URL.revokeObjectURL(url);
|
||||
}
|
||||
|
||||
export function useResourceActions(opts: ResourceActionsOptions) {
|
||||
// Move/copy dialog state, owned here so both surfaces bind one dialog to
|
||||
// `moveDialog.*` instead of re-implementing the open/mode/items triple.
|
||||
const moveDialog = $state({
|
||||
open: false,
|
||||
mode: 'move' as 'move' | 'copy',
|
||||
item: null as ActionTarget | null,
|
||||
items: null as ActionTarget[] | null
|
||||
});
|
||||
|
||||
function openBatchDialog(mode: 'move' | 'copy', items: ActionTarget[]): void {
|
||||
moveDialog.items = items;
|
||||
moveDialog.item = null;
|
||||
moveDialog.mode = mode;
|
||||
moveDialog.open = true;
|
||||
}
|
||||
|
||||
/** Context-menu / single-row entry points. */
|
||||
function openMove(target: ActionTarget): void {
|
||||
moveDialog.item = target;
|
||||
moveDialog.items = null;
|
||||
moveDialog.mode = 'move';
|
||||
moveDialog.open = true;
|
||||
}
|
||||
function openCopy(target: ActionTarget): void {
|
||||
moveDialog.item = target;
|
||||
moveDialog.items = null;
|
||||
moveDialog.mode = 'copy';
|
||||
moveDialog.open = true;
|
||||
}
|
||||
|
||||
function selectionTargets(): ActionTarget[] {
|
||||
// One O(M) index build instead of an O(N·M) `find` per selected id.
|
||||
// Folders win id collisions, matching the old folder-first probe.
|
||||
// eslint-disable-next-line svelte/prefer-svelte-reactivity -- ephemeral local index, discarded before any reactive read
|
||||
const byId = new Map<string, ActionTarget>();
|
||||
for (const f of opts.getItems()) byId.set(f.id, { id: f.id, name: f.name, kind: kindOf(f) });
|
||||
return [...opts.getSelected()]
|
||||
.map((id) => byId.get(id) ?? null)
|
||||
.filter((x): x is ActionTarget => x !== null);
|
||||
}
|
||||
|
||||
/**
|
||||
* Download the whole selection as a single zip via POST /api/batch/download —
|
||||
* folders are included (the old per-item loop silently skipped them). A lone
|
||||
* file still streams directly so it keeps its original name/extension.
|
||||
*/
|
||||
async function batchDownload(): Promise<void> {
|
||||
const targets = selectionTargets();
|
||||
if (targets.length === 0) return;
|
||||
const fileTargets = targets.filter((it) => it.kind === 'file');
|
||||
const folderTargets = targets.filter((it) => it.kind === 'folder');
|
||||
|
||||
// Single file, no folders → direct download (preserves the real name).
|
||||
if (fileTargets.length === 1 && folderTargets.length === 0) {
|
||||
const file = opts.getItems().find((f) => f.id === fileTargets[0].id);
|
||||
if (file) {
|
||||
const a = document.createElement('a');
|
||||
a.href = fileDownloadUrl(file.id);
|
||||
a.download = file.name;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
a.remove();
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
const blob = await downloadBatch(
|
||||
fileTargets.map((it) => it.id),
|
||||
folderTargets.map((it) => it.id)
|
||||
);
|
||||
saveBlob(blob, batchZipName());
|
||||
} catch (e) {
|
||||
errorToast(e);
|
||||
}
|
||||
}
|
||||
|
||||
/** Batch add the selection to favorites — single /api/favorites/batch call. */
|
||||
async function batchFavorites(): Promise<void> {
|
||||
const items = opts.getItems();
|
||||
// Build an id → item index so the "already favorite" filter is
|
||||
// O(1) per selection member instead of an O(N·M) scan. Reused
|
||||
// after success to flip `is_favorite` in place on each row.
|
||||
// eslint-disable-next-line svelte/prefer-svelte-reactivity -- ephemeral local index, discarded before any reactive read
|
||||
const byId = new Map<string, FileItem | FolderItem>();
|
||||
for (const it of items) byId.set(it.id, it);
|
||||
const targets = selectionTargets().filter((it) => !(byId.get(it.id)?.is_favorite ?? false));
|
||||
if (targets.length === 0) {
|
||||
ui.notify(t('files.already_favorites', 'All selected items are already favorites'), 'info');
|
||||
opts.clearSelection();
|
||||
return;
|
||||
}
|
||||
try {
|
||||
await addFavoritesBatch(targets.map((it) => ({ item_id: it.id, item_type: it.kind })));
|
||||
// eslint-disable-next-line svelte/prefer-svelte-reactivity -- ephemeral local index, discarded before any reactive read
|
||||
const flipped = new Set(targets.map((it) => it.id));
|
||||
if (opts.onFavoritesApplied) opts.onFavoritesApplied(flipped);
|
||||
else
|
||||
for (const id of flipped) {
|
||||
const row = byId.get(id);
|
||||
if (row) row.is_favorite = true;
|
||||
}
|
||||
ui.notify(t('files.added_favorites', 'Added to favorites'), 'success');
|
||||
opts.clearSelection();
|
||||
} catch (e) {
|
||||
errorToast(e);
|
||||
}
|
||||
}
|
||||
|
||||
async function batchDelete(): Promise<void> {
|
||||
const selected = opts.getSelected();
|
||||
const ids = [...selected];
|
||||
const ok = await confirmDialog({
|
||||
title: t('files.batch_delete', 'Delete selected'),
|
||||
message: t('files.confirm_batch_delete', { n: ids.length }, 'Move {{n}} items to trash?'),
|
||||
confirmText: t('common.delete', 'Delete'),
|
||||
danger: true
|
||||
});
|
||||
if (!ok) return;
|
||||
// Bounded fan-out instead of a serial await per item: 100 deletes at
|
||||
// ~30 ms RTT collapse from ~3 s of waterfall to a few round-trip
|
||||
// windows. Failures toast individually and the rest still proceed.
|
||||
const items = opts.getItems();
|
||||
// eslint-disable-next-line svelte/prefer-svelte-reactivity -- ephemeral local index, discarded before any reactive read
|
||||
const folderIdSet = new Set(items.filter((it) => !isFileItem(it)).map((it) => it.id));
|
||||
await mapLimit(ids, 6, async (id) => {
|
||||
try {
|
||||
if (folderIdSet.has(id)) await deleteFolder(id);
|
||||
else await deleteFile(id);
|
||||
} catch (e) {
|
||||
errorToast(e);
|
||||
}
|
||||
});
|
||||
opts.clearSelection();
|
||||
await opts.onChanged();
|
||||
opts.afterDelete?.();
|
||||
}
|
||||
|
||||
function batchMove(): void {
|
||||
const items = selectionTargets();
|
||||
if (items.length) openBatchDialog('move', items);
|
||||
}
|
||||
|
||||
function batchCopy(): void {
|
||||
const items = selectionTargets();
|
||||
if (items.length) openBatchDialog('copy', items);
|
||||
}
|
||||
|
||||
/** Pass as the MoveDialog `onmoved` handler. */
|
||||
async function handleMoved(): Promise<void> {
|
||||
opts.clearSelection();
|
||||
await opts.onChanged();
|
||||
}
|
||||
|
||||
return {
|
||||
selectionTargets,
|
||||
batchFavorites,
|
||||
batchDownload,
|
||||
batchDelete,
|
||||
batchMove,
|
||||
batchCopy,
|
||||
openMove,
|
||||
openCopy,
|
||||
handleMoved,
|
||||
moveDialog
|
||||
};
|
||||
}
|
||||
|
||||
function kindOf(item: FileItem | FolderItem): ItemType {
|
||||
return isFileItem(item) ? 'file' : 'folder';
|
||||
}
|
||||
|
||||
/** `FileItem | FolderItem` uses duck typing (`mime_type`) rather than a tag field. */
|
||||
export function isFileItem(item: FileItem | FolderItem): item is FileItem {
|
||||
return 'mime_type' in item;
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
|
||||
const { confirmDialog, ui } = vi.hoisted(() => ({
|
||||
confirmDialog: vi.fn(),
|
||||
ui: {
|
||||
notify: vi.fn(),
|
||||
startProgress: vi.fn(() => 1),
|
||||
updateProgress: vi.fn(),
|
||||
finishProgress: vi.fn()
|
||||
}
|
||||
}));
|
||||
vi.mock('$lib/stores/dialogs.svelte', () => ({ confirmDialog, promptDialog: vi.fn() }));
|
||||
vi.mock('$lib/stores/ui.svelte', () => ({ ui }));
|
||||
vi.mock('$lib/api/client', () => ({ apiFetch: vi.fn(), apiJson: vi.fn() }));
|
||||
vi.mock('$lib/api/csrf', () => ({ getCsrfHeaders: () => ({}) }));
|
||||
vi.mock('$lib/api/endpoints/batch', () => ({
|
||||
downloadBatch: vi.fn(),
|
||||
copyFiles: vi.fn(),
|
||||
copyFolders: vi.fn()
|
||||
}));
|
||||
vi.mock('$lib/api/endpoints/files', () => ({ deleteFile: vi.fn(), fileDownloadUrl: () => '/dl' }));
|
||||
vi.mock('$lib/api/endpoints/folders', () => ({ deleteFolder: vi.fn() }));
|
||||
vi.mock('$lib/api/endpoints/favorites', () => ({ addFavoritesBatch: vi.fn() }));
|
||||
|
||||
import { downloadBatch } from '$lib/api/endpoints/batch';
|
||||
import { deleteFile } from '$lib/api/endpoints/files';
|
||||
import { deleteFolder } from '$lib/api/endpoints/folders';
|
||||
import { addFavoritesBatch } from '$lib/api/endpoints/favorites';
|
||||
import { useResourceActions, type ActionTarget } from './useResourceActions.svelte';
|
||||
import type { FileItem, FolderItem } from '$lib/api/types';
|
||||
|
||||
const m = (fn: unknown) => fn as ReturnType<typeof vi.fn>;
|
||||
|
||||
// The composable only reads `id` / `name` / `mime_type` / `is_favorite`; the
|
||||
// full DTO shapes are satisfied via casts to keep the fixtures minimal.
|
||||
function fileItem(id: string, overrides: Record<string, unknown> = {}): FileItem {
|
||||
return {
|
||||
id,
|
||||
name: `${id}.txt`,
|
||||
mime_type: 'text/plain',
|
||||
is_favorite: false,
|
||||
...overrides
|
||||
} as unknown as FileItem;
|
||||
}
|
||||
function folderItem(id: string, overrides: Record<string, unknown> = {}): FolderItem {
|
||||
return {
|
||||
id,
|
||||
name: id,
|
||||
is_favorite: false,
|
||||
...overrides
|
||||
} as unknown as FolderItem;
|
||||
}
|
||||
|
||||
function harness(items: Array<FileItem | FolderItem>, selected: string[]) {
|
||||
const selection = new Set(selected);
|
||||
const actions = useResourceActions({
|
||||
getItems: () => items,
|
||||
getSelected: () => selection,
|
||||
clearSelection: () => selection.clear(),
|
||||
onChanged: vi.fn(),
|
||||
afterDelete: vi.fn()
|
||||
});
|
||||
return { actions, selection };
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
});
|
||||
|
||||
describe('selectionTargets', () => {
|
||||
it('maps selected ids to targets with kind', () => {
|
||||
const { actions } = harness([folderItem('d1'), fileItem('f1')], ['f1']);
|
||||
expect(actions.selectionTargets()).toEqual<ActionTarget[]>([
|
||||
{ id: 'f1', name: 'f1.txt', kind: 'file' }
|
||||
]);
|
||||
});
|
||||
|
||||
it('lets folders win id collisions', () => {
|
||||
const { actions } = harness([fileItem('x'), folderItem('x')], ['x']);
|
||||
expect(actions.selectionTargets()[0].kind).toBe('folder');
|
||||
});
|
||||
|
||||
it('drops ids that are no longer on screen', () => {
|
||||
const { actions } = harness([fileItem('f1')], ['gone']);
|
||||
expect(actions.selectionTargets()).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('batchFavorites', () => {
|
||||
it('skips items that are already favorites and flips the rest in place', async () => {
|
||||
const folder = folderItem('d1');
|
||||
const file = fileItem('f1', { is_favorite: true });
|
||||
const { actions, selection } = harness([folder, file], ['d1', 'f1']);
|
||||
m(addFavoritesBatch).mockResolvedValue(undefined);
|
||||
await actions.batchFavorites();
|
||||
expect(addFavoritesBatch).toHaveBeenCalledWith([{ item_id: 'd1', item_type: 'folder' }]);
|
||||
expect(folder.is_favorite).toBe(true);
|
||||
// already-favorite rows keep their state
|
||||
expect(file.is_favorite).toBe(true);
|
||||
expect(selection.size).toBe(0);
|
||||
expect(ui.notify).toHaveBeenCalledWith(expect.anything(), 'success');
|
||||
});
|
||||
|
||||
it('notifies when every selected item is already a favorite', async () => {
|
||||
const file = fileItem('f1', { is_favorite: true });
|
||||
const { actions, selection } = harness([file], ['f1']);
|
||||
await actions.batchFavorites();
|
||||
expect(addFavoritesBatch).not.toHaveBeenCalled();
|
||||
expect(ui.notify).toHaveBeenCalledWith(expect.anything(), 'info');
|
||||
expect(selection.size).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('batchDownload', () => {
|
||||
it('uses the batch zip endpoint for a mixed selection', async () => {
|
||||
const { actions } = harness(
|
||||
[fileItem('f1'), fileItem('f2'), folderItem('d1')],
|
||||
['f1', 'f2', 'd1']
|
||||
);
|
||||
m(downloadBatch).mockResolvedValue(new Blob(['zip']));
|
||||
await actions.batchDownload();
|
||||
expect(downloadBatch).toHaveBeenCalledWith(['f1', 'f2'], ['d1']);
|
||||
});
|
||||
|
||||
it('streams a lone file directly, without the zip endpoint', async () => {
|
||||
const { actions } = harness([fileItem('f1')], ['f1']);
|
||||
await actions.batchDownload();
|
||||
expect(downloadBatch).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('batchDelete', () => {
|
||||
it('fans out per item after confirmation and calls onChanged + afterDelete', async () => {
|
||||
const selection = new Set(['d1', 'f1']);
|
||||
const onChanged = vi.fn();
|
||||
const afterDelete = vi.fn();
|
||||
const actions = useResourceActions({
|
||||
getItems: () => [folderItem('d1'), fileItem('f1')],
|
||||
getSelected: () => selection,
|
||||
clearSelection: () => selection.clear(),
|
||||
onChanged,
|
||||
afterDelete
|
||||
});
|
||||
confirmDialog.mockResolvedValue(true);
|
||||
await actions.batchDelete();
|
||||
expect(deleteFolder).toHaveBeenCalledWith('d1');
|
||||
expect(deleteFile).toHaveBeenCalledWith('f1');
|
||||
expect(onChanged).toHaveBeenCalled();
|
||||
expect(afterDelete).toHaveBeenCalled();
|
||||
expect(selection.size).toBe(0);
|
||||
});
|
||||
|
||||
it('does nothing when the confirm dialog is dismissed', async () => {
|
||||
const { actions } = harness([fileItem('f1')], ['f1']);
|
||||
confirmDialog.mockResolvedValue(false);
|
||||
await actions.batchDelete();
|
||||
expect(deleteFile).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('move/copy dialogs', () => {
|
||||
it('batch move opens the dialog with the selection', () => {
|
||||
const { actions } = harness([fileItem('f1')], ['f1']);
|
||||
actions.batchMove();
|
||||
expect(actions.moveDialog.open).toBe(true);
|
||||
expect(actions.moveDialog.mode).toBe('move');
|
||||
expect(actions.moveDialog.items).toEqual<ActionTarget[]>([
|
||||
{ id: 'f1', name: 'f1.txt', kind: 'file' }
|
||||
]);
|
||||
expect(actions.moveDialog.item).toBeNull();
|
||||
});
|
||||
|
||||
it('batch copy opens the dialog in copy mode', () => {
|
||||
const { actions } = harness([fileItem('f1')], ['f1']);
|
||||
actions.batchCopy();
|
||||
expect(actions.moveDialog.open).toBe(true);
|
||||
expect(actions.moveDialog.mode).toBe('copy');
|
||||
});
|
||||
|
||||
it('openMove sets a single item and handleMoved clears selection', async () => {
|
||||
const selection = new Set(['f1']);
|
||||
const onChanged = vi.fn();
|
||||
const actions = useResourceActions({
|
||||
getItems: () => [fileItem('f1')],
|
||||
getSelected: () => selection,
|
||||
clearSelection: () => selection.clear(),
|
||||
onChanged
|
||||
});
|
||||
actions.openMove({ id: 'f1', name: 'f1.txt', kind: 'file' });
|
||||
expect(actions.moveDialog.open).toBe(true);
|
||||
expect(actions.moveDialog.item).toEqual({ id: 'f1', name: 'f1.txt', kind: 'file' });
|
||||
await actions.handleMoved();
|
||||
expect(selection.size).toBe(0);
|
||||
expect(onChanged).toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,49 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { mapLimit } from './mapLimit';
|
||||
|
||||
describe('mapLimit', () => {
|
||||
it('preserves result order regardless of completion order', async () => {
|
||||
const result = await mapLimit([200, 50, 100, 10], 4, async (ms) => {
|
||||
await new Promise((r) => setTimeout(r, ms));
|
||||
return ms;
|
||||
});
|
||||
expect(result).toEqual([200, 50, 100, 10]);
|
||||
});
|
||||
|
||||
it('never exceeds the concurrency cap', async () => {
|
||||
let active = 0;
|
||||
let peak = 0;
|
||||
await mapLimit([1, 2, 3, 4, 5, 6, 7, 8], 3, async () => {
|
||||
active++;
|
||||
peak = Math.max(peak, active);
|
||||
await new Promise((r) => setTimeout(r, 1));
|
||||
active--;
|
||||
return null;
|
||||
});
|
||||
expect(peak).toBe(3);
|
||||
});
|
||||
|
||||
it('propagates rejections', async () => {
|
||||
await expect(
|
||||
mapLimit([1, 2, 3], 2, async (n) => {
|
||||
if (n === 2) throw new Error('boom');
|
||||
return n;
|
||||
})
|
||||
).rejects.toThrow('boom');
|
||||
});
|
||||
|
||||
it('handles empty input', async () => {
|
||||
const fn = vi.fn(async (n: number) => n);
|
||||
await expect(mapLimit([], 4, fn)).resolves.toEqual([]);
|
||||
expect(fn).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('runs items sequentially when limit is 1', async () => {
|
||||
const calls: number[] = [];
|
||||
await mapLimit([1, 2, 3], 1, async (n) => {
|
||||
calls.push(n);
|
||||
return n;
|
||||
});
|
||||
expect(calls).toEqual([1, 2, 3]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,24 @@
|
||||
/**
|
||||
* Map `fn` over `items` with at most `limit` concurrent calls, preserving
|
||||
* result order regardless of completion order.
|
||||
*
|
||||
* Extracted from the files page so batch fan-out paths (delete, drag-move,
|
||||
* upload probing) and the shared resource-actions composable all use the
|
||||
* same bounded-concurrency primitive.
|
||||
*/
|
||||
export async function mapLimit<T, R>(
|
||||
items: readonly T[],
|
||||
limit: number,
|
||||
fn: (item: T) => Promise<R>
|
||||
): Promise<R[]> {
|
||||
const out = new Array<R>(items.length);
|
||||
let next = 0;
|
||||
const worker = async () => {
|
||||
while (next < items.length) {
|
||||
const i = next++;
|
||||
out[i] = await fn(items[i]);
|
||||
}
|
||||
};
|
||||
await Promise.all(Array.from({ length: Math.max(0, Math.min(limit, items.length)) }, worker));
|
||||
return out;
|
||||
}
|
||||
@@ -0,0 +1,101 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
clearFilterState,
|
||||
dateBound,
|
||||
defaultFilterState,
|
||||
filterToSearchOptions,
|
||||
isFilterActive,
|
||||
sizeBounds,
|
||||
TYPE_EXT
|
||||
} from './searchFilters';
|
||||
|
||||
const MB = 1024 * 1024;
|
||||
|
||||
describe('TYPE_EXT', () => {
|
||||
it('covers the five non-all type keys', () => {
|
||||
expect(Object.keys(TYPE_EXT).sort()).toEqual(
|
||||
['archive', 'audio', 'document', 'image', 'video'].sort()
|
||||
);
|
||||
});
|
||||
|
||||
it('uses bare lowercase extensions', () => {
|
||||
for (const exts of Object.values(TYPE_EXT)) {
|
||||
for (const ext of exts) expect(ext).toBe(ext.toLowerCase());
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('sizeBounds', () => {
|
||||
it('maps the presets to byte ranges', () => {
|
||||
expect(sizeBounds('all')).toEqual({});
|
||||
expect(sizeBounds('small')).toEqual({ maxSize: MB });
|
||||
expect(sizeBounds('medium')).toEqual({ minSize: MB, maxSize: 100 * MB });
|
||||
expect(sizeBounds('large')).toEqual({ minSize: 100 * MB });
|
||||
});
|
||||
});
|
||||
|
||||
describe('dateBound', () => {
|
||||
it('maps day to 24h ago', () => {
|
||||
expect(dateBound('day')).toBe(Math.floor(Date.now() / 1000) - 86400);
|
||||
});
|
||||
|
||||
it('returns undefined for all', () => {
|
||||
expect(dateBound('all')).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('isFilterActive', () => {
|
||||
it('is false for the default state', () => {
|
||||
expect(isFilterActive(defaultFilterState())).toBe(false);
|
||||
});
|
||||
|
||||
it('is true for a non-empty keyword (even whitespace-only counts as empty)', () => {
|
||||
expect(isFilterActive({ ...defaultFilterState(), query: 'x' })).toBe(true);
|
||||
expect(isFilterActive({ ...defaultFilterState(), query: ' ' })).toBe(false);
|
||||
});
|
||||
|
||||
it('is true when any preset differs from all', () => {
|
||||
expect(isFilterActive({ ...defaultFilterState(), type: 'image' })).toBe(true);
|
||||
expect(isFilterActive({ ...defaultFilterState(), size: 'small' })).toBe(true);
|
||||
expect(isFilterActive({ ...defaultFilterState(), date: 'week' })).toBe(true);
|
||||
});
|
||||
|
||||
it('ignores the recursive toggle', () => {
|
||||
expect(isFilterActive({ ...defaultFilterState(), recursive: false })).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('clearFilterState', () => {
|
||||
it('resets every field in place', () => {
|
||||
const f = { ...defaultFilterState(), query: 'a', recursive: false, type: 'video' as const };
|
||||
clearFilterState(f);
|
||||
expect(f).toEqual(defaultFilterState());
|
||||
});
|
||||
});
|
||||
|
||||
describe('filterToSearchOptions', () => {
|
||||
it('omits everything for the default state', () => {
|
||||
expect(filterToSearchOptions(defaultFilterState())).toEqual({
|
||||
fileTypes: undefined,
|
||||
minSize: undefined,
|
||||
maxSize: undefined,
|
||||
modifiedAfter: undefined,
|
||||
recursive: true
|
||||
});
|
||||
});
|
||||
|
||||
it('maps each active dimension onto the wire options', () => {
|
||||
const opts = filterToSearchOptions({
|
||||
query: 'report',
|
||||
recursive: false,
|
||||
type: 'archive',
|
||||
size: 'medium',
|
||||
date: 'month'
|
||||
});
|
||||
expect(opts.fileTypes).toEqual(TYPE_EXT.archive);
|
||||
expect(opts.minSize).toBe(MB);
|
||||
expect(opts.maxSize).toBe(100 * MB);
|
||||
expect(opts.modifiedAfter).toBe(dateBound('month'));
|
||||
expect(opts.recursive).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,96 @@
|
||||
// Shared resource-filter model for search-backed list views.
|
||||
//
|
||||
// Extracted from the /search page so the files page's filter bar and
|
||||
// /search's filter selects share one source of truth for the preset
|
||||
// vocabularies (type / size / date) and their mapping onto
|
||||
// `SearchOptions`. Pure functions only — no runes here, so the module is
|
||||
// unit-testable without component scaffolding.
|
||||
import type { SearchOptions } from '$lib/api/endpoints/search';
|
||||
|
||||
export type TypeKey = 'all' | 'image' | 'video' | 'document' | 'audio' | 'archive';
|
||||
export type SizeKey = 'all' | 'small' | 'medium' | 'large';
|
||||
export type DateKey = 'all' | 'day' | 'week' | 'month' | 'year';
|
||||
|
||||
/** Full filter state for a search-backed resource list. */
|
||||
export interface ResourceFilterState {
|
||||
query: string;
|
||||
/** Search subfolders too (backend default is true; exposed explicitly on /files). */
|
||||
recursive: boolean;
|
||||
type: TypeKey;
|
||||
size: SizeKey;
|
||||
date: DateKey;
|
||||
}
|
||||
|
||||
export function defaultFilterState(): ResourceFilterState {
|
||||
return { query: '', recursive: true, type: 'all', size: 'all', date: 'all' };
|
||||
}
|
||||
|
||||
export const TYPE_EXT: Record<Exclude<TypeKey, 'all'>, string[]> = {
|
||||
image: ['jpg', 'jpeg', 'png', 'gif', 'webp', 'svg', 'bmp', 'heic', 'avif', 'tiff'],
|
||||
video: ['mp4', 'mov', 'mkv', 'avi', 'webm', 'm4v', 'wmv', 'flv'],
|
||||
document: ['pdf', 'doc', 'docx', 'xls', 'xlsx', 'ppt', 'pptx', 'txt', 'md', 'odt', 'rtf', 'csv'],
|
||||
audio: ['mp3', 'wav', 'flac', 'aac', 'ogg', 'm4a', 'opus'],
|
||||
archive: ['zip', 'rar', '7z', 'tar', 'gz', 'bz2', 'xz']
|
||||
};
|
||||
|
||||
const MB = 1024 * 1024;
|
||||
export function sizeBounds(k: SizeKey): { minSize?: number; maxSize?: number } {
|
||||
switch (k) {
|
||||
case 'small':
|
||||
return { maxSize: MB };
|
||||
case 'medium':
|
||||
return { minSize: MB, maxSize: 100 * MB };
|
||||
case 'large':
|
||||
return { minSize: 100 * MB };
|
||||
default:
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
export function dateBound(k: DateKey): number | undefined {
|
||||
const day = 86400;
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
switch (k) {
|
||||
case 'day':
|
||||
return now - day;
|
||||
case 'week':
|
||||
return now - 7 * day;
|
||||
case 'month':
|
||||
return now - 30 * day;
|
||||
case 'year':
|
||||
return now - 365 * day;
|
||||
default:
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
/** True when any filter dimension would change the result set. */
|
||||
export function isFilterActive(f: ResourceFilterState): boolean {
|
||||
return f.query.trim() !== '' || f.type !== 'all' || f.size !== 'all' || f.date !== 'all';
|
||||
}
|
||||
|
||||
/** Reset every dimension in place (runes-friendly — mutates the $state proxy). */
|
||||
export function clearFilterState(f: ResourceFilterState): void {
|
||||
f.query = '';
|
||||
f.recursive = true;
|
||||
f.type = 'all';
|
||||
f.size = 'all';
|
||||
f.date = 'all';
|
||||
}
|
||||
|
||||
/**
|
||||
* Map the filter state onto the search-wire options. Scope (folderId) and
|
||||
* sorting stay the caller's concern — they differ per surface. The date
|
||||
* preset maps to `modifiedAfter` (its labels read "Past N", which matches
|
||||
* modified-time semantics); created-time bounds are a possible follow-up.
|
||||
*/
|
||||
export function filterToSearchOptions(
|
||||
f: ResourceFilterState
|
||||
): Pick<SearchOptions, 'fileTypes' | 'minSize' | 'maxSize' | 'modifiedAfter' | 'recursive'> {
|
||||
return {
|
||||
fileTypes: f.type === 'all' ? undefined : TYPE_EXT[f.type],
|
||||
...sizeBounds(f.size),
|
||||
modifiedAfter: dateBound(f.date),
|
||||
recursive: f.recursive
|
||||
};
|
||||
}
|
||||
@@ -1,7 +1,22 @@
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
|
||||
import { describe, it, expect, vi, beforeEach, afterEach, beforeAll } from 'vitest';
|
||||
import { relativeTimeAgo } from './time';
|
||||
|
||||
describe('relativeTimeAgo', () => {
|
||||
// The formatter resolves the runtime default locale (`undefined`), so on a
|
||||
// non-English dev machine (e.g. zh-CN Windows) the output is localized and
|
||||
// these English-unit regexes fail. Pin English for the tests; vitest's
|
||||
// per-file isolation keeps the module-level formatter cache from leaking.
|
||||
const RealRelativeTimeFormat = Intl.RelativeTimeFormat;
|
||||
beforeAll(() => {
|
||||
// A regular function, not an arrow: time.ts calls the mock via `new`.
|
||||
vi.spyOn(Intl, 'RelativeTimeFormat').mockImplementation(function (
|
||||
locales?: string | string[],
|
||||
options?: Intl.RelativeTimeFormatOptions
|
||||
) {
|
||||
return new RealRelativeTimeFormat('en', options);
|
||||
} as unknown as typeof Intl.RelativeTimeFormat);
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers();
|
||||
vi.setSystemTime(new Date('2024-06-15T12:00:00Z'));
|
||||
|
||||
@@ -40,11 +40,9 @@
|
||||
import { canEditWithWopi, getEditorUrlWithFallback } from '$lib/api/endpoints/wopi';
|
||||
import { addTracks, createPlaylist, listPlaylists } from '$lib/api/endpoints/music';
|
||||
import { copyFiles, copyFolders } from '$lib/api/endpoints/batch';
|
||||
import { apiFetch } from '$lib/api/client';
|
||||
import { getCsrfHeaders } from '$lib/api/csrf';
|
||||
import { countHidden, filterDotfiles } from '$lib/utils/dotfileFilter';
|
||||
import { preferences } from '$lib/stores/preferences.svelte';
|
||||
import type { FileItem, FolderItem, ItemType } from '$lib/api/types';
|
||||
import type { FileItem, FolderItem, ItemType, SortBy } from '$lib/api/types';
|
||||
import ReadOnlyBanner from '$lib/components/ReadOnlyBanner.svelte';
|
||||
import FolderBreadcrumb from '$lib/components/FolderBreadcrumb.svelte';
|
||||
import ResourceList, {
|
||||
@@ -62,6 +60,21 @@
|
||||
import { ui } from '$lib/stores/ui.svelte';
|
||||
import { dateBucket, sizeBucket, typeLabel } from '$lib/stores/files.svelte';
|
||||
import { replaceSet } from '$lib/utils/sets';
|
||||
import { mapLimit } from '$lib/utils/mapLimit';
|
||||
import {
|
||||
defaultFilterState,
|
||||
clearFilterState,
|
||||
isFilterActive,
|
||||
filterToSearchOptions,
|
||||
type ResourceFilterState
|
||||
} from '$lib/utils/searchFilters';
|
||||
import { searchResources } from '$lib/api/endpoints/search';
|
||||
import {
|
||||
useResourceActions,
|
||||
batchZipName,
|
||||
type ActionTarget
|
||||
} from '$lib/composables/useResourceActions.svelte';
|
||||
import SearchFilterBar from '$lib/components/SearchFilterBar.svelte';
|
||||
|
||||
// Message-bus logger. Users can tune with
|
||||
// oxi.setLogLevel('oxi:message-bus', 'debug')
|
||||
@@ -180,16 +193,11 @@
|
||||
let fileInput = $state<HTMLInputElement | null>(null);
|
||||
let uploading = $state(false);
|
||||
|
||||
interface ActionTarget {
|
||||
id: string;
|
||||
name: string;
|
||||
kind: ItemType;
|
||||
}
|
||||
let moveOpen = $state(false);
|
||||
let moveMode = $state<'move' | 'copy'>('move');
|
||||
// Move/copy dialog state lives in the shared `useResourceActions`
|
||||
// composable (`resActions.moveDialog.*`); this page only keeps the share
|
||||
// dialog target, which the composable doesn't own.
|
||||
let shareOpen = $state(false);
|
||||
let actionTarget = $state<ActionTarget | null>(null);
|
||||
let moveItems = $state<ActionTarget[] | null>(null);
|
||||
|
||||
// Favorite / shared state now lives inline on every `FileItem` /
|
||||
// `FolderItem` DTO (`is_favorite`, `is_shared` — see
|
||||
@@ -200,16 +208,10 @@
|
||||
// inside `orderedItems`. No more `SvelteSet` shadowing.
|
||||
|
||||
function openMove(kind: ItemType, id: string, name: string) {
|
||||
actionTarget = { id, name, kind };
|
||||
moveItems = null;
|
||||
moveMode = 'move';
|
||||
moveOpen = true;
|
||||
resActions.openMove({ id, name, kind });
|
||||
}
|
||||
function openCopy(kind: ItemType, id: string, name: string) {
|
||||
actionTarget = { id, name, kind };
|
||||
moveItems = null;
|
||||
moveMode = 'copy';
|
||||
moveOpen = true;
|
||||
resActions.openCopy({ id, name, kind });
|
||||
}
|
||||
function openShare(kind: ItemType, id: string, name: string) {
|
||||
actionTarget = { id, name, kind };
|
||||
@@ -389,9 +391,15 @@
|
||||
* Fetch and append the next page. Invoked by ResourceList's
|
||||
* IntersectionObserver when the bottom sentinel enters the viewport.
|
||||
* The `loadingMore` guard collapses a double-fire (the observer can
|
||||
* tick twice on the same intersection edge).
|
||||
* tick twice on the same intersection edge). Mode-aware: appends to
|
||||
* the search results while the filter bar is active, to the folder
|
||||
* page otherwise.
|
||||
*/
|
||||
async function loadMore() {
|
||||
if (searchActive) {
|
||||
await loadMoreSearch();
|
||||
return;
|
||||
}
|
||||
if (loadingMore || pageCursor === undefined) return;
|
||||
loadingMore = true;
|
||||
try {
|
||||
@@ -437,6 +445,74 @@
|
||||
}
|
||||
}
|
||||
|
||||
// ── Filter / search mode ─────────────────────────────────────────────────
|
||||
// While any SearchFilterBar dimension is active the listing switches from
|
||||
// the folder page (`fetchFolderPage`) to a scoped search (`searchResources`
|
||||
// with folder_id = currentId). The two data paths keep independent cursors
|
||||
// and stale guards; entering/leaving the mode neutralizes the other path's
|
||||
// in-flight response so a slow folder page can never clobber fresh search
|
||||
// rows (and vice versa).
|
||||
//
|
||||
// The backend treats an absent/empty `query` as "match everything"
|
||||
// (`SearchResourcesQuery.query` is `Option<String>`), so filter-only
|
||||
// searches (type/size/date, no keyword) work; its Tantivy content index
|
||||
// additionally requires ≥2 chars before it engages, so empty queries stay
|
||||
// name/filter-driven.
|
||||
let filter = $state<ResourceFilterState>(defaultFilterState());
|
||||
const searchActive = $derived(isFilterActive(filter));
|
||||
let searchItems = $state<Array<FileItem | FolderItem>>([]);
|
||||
let searchCursor = $state<string | undefined>(undefined);
|
||||
let searchSeq = 0;
|
||||
let searchAbort: AbortController | null = null;
|
||||
|
||||
async function runSearch(reset: boolean = true) {
|
||||
const folderId = currentId;
|
||||
if (!searchActive || !folderId) return;
|
||||
error = null;
|
||||
const seq = ++searchSeq;
|
||||
searchAbort?.abort();
|
||||
const ctl = new AbortController();
|
||||
searchAbort = ctl;
|
||||
loading = true;
|
||||
const activeAtStart = searchActive;
|
||||
try {
|
||||
// The search wire has no `type` order (that's a client-side group-by)
|
||||
// and calls modified time `updated_at` — map both before sending.
|
||||
// Relevance is meaningless with a (possibly empty) filter query, so
|
||||
// the current sort field is always sent instead.
|
||||
const sortBy: SortBy =
|
||||
sortField === 'type' ? 'name' : sortField === 'modified_at' ? 'updated_at' : sortField;
|
||||
const res = await searchResources(filter.query.trim(), {
|
||||
folderId,
|
||||
...filterToSearchOptions(filter),
|
||||
sortBy,
|
||||
reverse: reversed,
|
||||
limit: 50,
|
||||
cursor: reset ? undefined : searchCursor,
|
||||
signal: ctl.signal
|
||||
});
|
||||
if (seq !== searchSeq || searchActive !== activeAtStart) return; // superseded
|
||||
// Unwrap the search envelope: each hit's `resource` is already the
|
||||
// shared FileItem | FolderItem shape ResourceList consumes.
|
||||
const hits = res.items.map((it) => it.resource);
|
||||
searchItems = reset ? hits : [...searchItems, ...hits];
|
||||
searchCursor = res.next_cursor;
|
||||
loading = false;
|
||||
} catch (e) {
|
||||
if (seq !== searchSeq || searchActive !== activeAtStart) return;
|
||||
loading = false;
|
||||
if ((e as Error)?.name !== 'AbortError') error = errorMessage(e);
|
||||
} finally {
|
||||
if (searchAbort === ctl) searchAbort = null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Append the next search page — the search-mode twin of `loadMore()`. */
|
||||
async function loadMoreSearch() {
|
||||
if (searchCursor === undefined) return;
|
||||
await runSearch(false);
|
||||
}
|
||||
|
||||
// ── Live folder updates (message bus) ────────────────────────────
|
||||
// Subscribe to `folder:{currentId}` and refresh when THIS session's
|
||||
// tabs, another tab of the same user, or another user with a share
|
||||
@@ -465,7 +541,11 @@
|
||||
// event upload without feeling laggy.
|
||||
setTimeout(() => {
|
||||
reloadScheduled = false;
|
||||
void reload();
|
||||
// In search mode re-run the SEARCH, not the folder page: a
|
||||
// recursive filter covers subfolders, and a mutation in any of
|
||||
// them (or of a matched row itself) can invalidate the results.
|
||||
if (searchActive) void runSearch(true);
|
||||
else void reload();
|
||||
}, 100);
|
||||
}
|
||||
useFolderTopic(() => currentId, {
|
||||
@@ -670,24 +750,6 @@
|
||||
}
|
||||
}
|
||||
|
||||
/** Map `fn` over `items` with at most `limit` concurrent calls, preserving order. */
|
||||
async function mapLimit<T, R>(
|
||||
items: T[],
|
||||
limit: number,
|
||||
fn: (item: T) => Promise<R>
|
||||
): Promise<R[]> {
|
||||
const out = new Array<R>(items.length);
|
||||
let next = 0;
|
||||
const worker = async () => {
|
||||
while (next < items.length) {
|
||||
const i = next++;
|
||||
out[i] = await fn(items[i]);
|
||||
}
|
||||
};
|
||||
await Promise.all(Array.from({ length: Math.min(limit, items.length) }, worker));
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Split items into the readable ones and the unreadable (FIFO/socket/…) ones. */
|
||||
async function partitionReadable<T>(
|
||||
items: T[],
|
||||
@@ -1117,7 +1179,11 @@
|
||||
// viewer-state changes, so a user-initiated close can't be re-opened here.
|
||||
$effect(() => {
|
||||
const fileId = page.url.searchParams.get('file');
|
||||
const files = listing.files;
|
||||
// Deep links must also resolve while the filter bar is active — the
|
||||
// hit may only exist in the search results, not the folder page.
|
||||
const files = searchActive
|
||||
? searchItems.filter((it): it is FileItem => isFile(it))
|
||||
: listing.files;
|
||||
untrack(() => {
|
||||
if (!fileId) {
|
||||
if (viewerOpen) viewerOpen = false;
|
||||
@@ -1167,140 +1233,35 @@
|
||||
selected.clear();
|
||||
}
|
||||
|
||||
/**
|
||||
* Download the whole selection as a single zip via POST /api/batch/download —
|
||||
* folders are included (the old per-item loop silently skipped them). A lone
|
||||
* file still streams directly so it keeps its original name/extension.
|
||||
*/
|
||||
/** Name for a server-zipped multi-item archive (matches the legacy format). */
|
||||
function batchZipName(): string {
|
||||
const stamp = new Date().toISOString().replace('T', ' ').replace(/\..*/, '').replace(/:/g, '-');
|
||||
return `oxicloud ${stamp}.zip`;
|
||||
}
|
||||
|
||||
async function batchDownload() {
|
||||
const fileIds: string[] = [];
|
||||
const folderIds: string[] = [];
|
||||
// One O(M) pass over the listing instead of an O(N·M) `some` per id.
|
||||
const folderIdSet = new Set(listing.folders.map((f) => f.id));
|
||||
const fileIdSet = new Set(listing.files.map((f) => f.id));
|
||||
for (const id of selected) {
|
||||
if (folderIdSet.has(id)) folderIds.push(id);
|
||||
else if (fileIdSet.has(id)) fileIds.push(id);
|
||||
}
|
||||
if (fileIds.length === 0 && folderIds.length === 0) return;
|
||||
|
||||
// Single file, no folders → direct download (preserves the real name).
|
||||
if (fileIds.length === 1 && folderIds.length === 0) {
|
||||
const file = listing.files.find((f) => f.id === fileIds[0]);
|
||||
if (file) {
|
||||
const a = document.createElement('a');
|
||||
a.href = fileDownloadUrl(file.id);
|
||||
a.download = file.name;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
a.remove();
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
const zipName = batchZipName();
|
||||
try {
|
||||
const res = await apiFetch('/api/batch/download', {
|
||||
method: 'POST',
|
||||
credentials: 'same-origin',
|
||||
headers: { 'Content-Type': 'application/json', ...getCsrfHeaders() },
|
||||
body: JSON.stringify({ file_ids: fileIds, folder_ids: folderIds })
|
||||
// Shared batch actions (favorite / download / delete / move / copy),
|
||||
// extracted so this page and the /search results page share one
|
||||
// implementation. `getItems` switches with the view mode: batch
|
||||
// operations act on search hits while the filter is active, on the
|
||||
// folder listing otherwise. (`orderedItems` rather than `rlItems` —
|
||||
// hidden dotfiles can never be selected, and the raw array keeps the
|
||||
// lone-file download name lookup working.)
|
||||
const resActions = useResourceActions({
|
||||
getItems: () => (searchActive ? searchItems : orderedItems),
|
||||
getSelected: () => selected,
|
||||
clearSelection,
|
||||
onChanged: () => (searchActive ? runSearch(true) : reload()),
|
||||
afterDelete: () => void session.refresh()
|
||||
});
|
||||
if (!res.ok) throw new Error(`Server returned ${res.status}`);
|
||||
const blob = await res.blob();
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = zipName;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
a.remove();
|
||||
URL.revokeObjectURL(url);
|
||||
} catch (e) {
|
||||
errorToast(e);
|
||||
}
|
||||
}
|
||||
|
||||
/** Batch add the selection to favorites — single /api/favorites/batch call. */
|
||||
async function batchFavorites() {
|
||||
// Build an id → item index so the "already favorite" filter is
|
||||
// O(1) per selection member instead of an O(N·M) scan. Reused
|
||||
// after success to flip `is_favorite` in place on each row.
|
||||
// eslint-disable-next-line svelte/prefer-svelte-reactivity -- ephemeral local index, discarded before any reactive read
|
||||
const byId = new Map<string, FileItem | FolderItem>();
|
||||
for (const it of orderedItems) byId.set(it.id, it);
|
||||
const items = selectionTargets().filter((it) => !(byId.get(it.id)?.is_favorite ?? false));
|
||||
if (items.length === 0) {
|
||||
ui.notify(t('files.already_favorites', 'All selected items are already favorites'), 'info');
|
||||
clearSelection();
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const res = await apiFetch('/api/favorites/batch', {
|
||||
method: 'POST',
|
||||
credentials: 'same-origin',
|
||||
headers: { 'Content-Type': 'application/json', ...getCsrfHeaders() },
|
||||
body: JSON.stringify({
|
||||
items: items.map((it) => ({ item_id: it.id, item_type: it.kind }))
|
||||
})
|
||||
});
|
||||
if (!res.ok) throw new Error(`Server returned ${res.status}`);
|
||||
for (const it of items) {
|
||||
const row = byId.get(it.id);
|
||||
if (row) row.is_favorite = true;
|
||||
}
|
||||
ui.notify(t('files.added_favorites', 'Added to favorites'), 'success');
|
||||
clearSelection();
|
||||
} catch (e) {
|
||||
errorToast(e);
|
||||
}
|
||||
}
|
||||
|
||||
function selectionTargets(): ActionTarget[] {
|
||||
// One O(M) index build instead of an O(N·M) `find` per selected id.
|
||||
// Folders win id collisions, matching the old folder-first probe.
|
||||
// eslint-disable-next-line svelte/prefer-svelte-reactivity -- ephemeral local index, discarded before any reactive read
|
||||
const byId = new Map<string, ActionTarget>();
|
||||
for (const f of listing.files) byId.set(f.id, { id: f.id, name: f.name, kind: 'file' });
|
||||
for (const f of listing.folders) byId.set(f.id, { id: f.id, name: f.name, kind: 'folder' });
|
||||
return [...selected]
|
||||
.map((id) => byId.get(id) ?? null)
|
||||
.filter((x): x is ActionTarget => x !== null);
|
||||
}
|
||||
|
||||
function batchMove() {
|
||||
const items = selectionTargets();
|
||||
if (items.length) {
|
||||
moveItems = items;
|
||||
moveMode = 'move';
|
||||
moveOpen = true;
|
||||
}
|
||||
}
|
||||
|
||||
function batchCopy() {
|
||||
const items = selectionTargets();
|
||||
if (items.length) {
|
||||
moveItems = items;
|
||||
moveMode = 'copy';
|
||||
moveOpen = true;
|
||||
}
|
||||
}
|
||||
|
||||
function onKeydown(e: KeyboardEvent) {
|
||||
const tag = (e.target as HTMLElement)?.tagName;
|
||||
if (tag === 'INPUT' || tag === 'TEXTAREA' || tag === 'SELECT') return;
|
||||
// Escape priority: selection first, then an active filter. The filter
|
||||
// input handles its own Escape (clears the keyword, stopPropagation)
|
||||
// so reaching here means the focus is outside the search bar.
|
||||
if (e.key === 'Escape' && selected.size) {
|
||||
clearSelection();
|
||||
} else if (e.key === 'Escape' && searchActive) {
|
||||
clearFilterState(filter);
|
||||
} else if (e.key === 'Delete' && selected.size) {
|
||||
// Delete only — Backspace was dropped: it triggered accidental deletes.
|
||||
e.preventDefault();
|
||||
void batchDelete();
|
||||
void resActions.batchDelete();
|
||||
}
|
||||
// Ctrl+A "select all" moved to the list-header checkbox owned by
|
||||
// ResourceList — the row-level selection UX now lives entirely
|
||||
@@ -1309,33 +1270,6 @@
|
||||
// gestures that reference the local `selected` mirror.
|
||||
}
|
||||
|
||||
async function batchDelete() {
|
||||
const ids = [...selected];
|
||||
const ok = await confirmDialog({
|
||||
title: t('files.batch_delete', 'Delete selected'),
|
||||
message: t('files.confirm_batch_delete', { n: ids.length }, 'Move {{n}} items to trash?'),
|
||||
confirmText: t('common.delete', 'Delete'),
|
||||
danger: true
|
||||
});
|
||||
if (!ok) return;
|
||||
// Bounded fan-out instead of a serial await per item: 100 deletes at
|
||||
// ~30 ms RTT collapse from ~3 s of waterfall to a few round-trip
|
||||
// windows. Failures toast individually and the rest still proceed,
|
||||
// exactly like the old serial loop.
|
||||
const folderIdSet = new Set(listing.folders.map((f) => f.id));
|
||||
await mapLimit(ids, 6, async (id) => {
|
||||
try {
|
||||
if (folderIdSet.has(id)) await deleteFolder(id);
|
||||
else await deleteFile(id);
|
||||
} catch (e) {
|
||||
errorToast(e);
|
||||
}
|
||||
});
|
||||
clearSelection();
|
||||
await reload();
|
||||
void session.refresh();
|
||||
}
|
||||
|
||||
// ── Drag-to-move ─────────────────────────────────────────────────────────
|
||||
const DRAG_TYPE = 'application/x-oxi-item';
|
||||
let dropFolderId = $state<string | null>(null);
|
||||
@@ -1380,7 +1314,7 @@
|
||||
*/
|
||||
function onItemDragStart(e: DragEvent, kind: ItemType, id: string, name: string) {
|
||||
const items: ActionTarget[] =
|
||||
selected.has(id) && selected.size > 1 ? selectionTargets() : [{ id, name, kind }];
|
||||
selected.has(id) && selected.size > 1 ? resActions.selectionTargets() : [{ id, name, kind }];
|
||||
e.dataTransfer?.setData(DRAG_TYPE, JSON.stringify(items));
|
||||
if (e.dataTransfer) {
|
||||
// `copyMove` advertises both operations; the drop-target's
|
||||
@@ -1575,7 +1509,7 @@
|
||||
$effect(() => {
|
||||
if (viewerOpen) void fileViewer.load();
|
||||
if (wopiOpen) void wopiEditor.load();
|
||||
if (moveOpen) void moveDialog.load();
|
||||
if (resActions.moveDialog.open) void moveDialog.load();
|
||||
if (shareOpen) void shareDialog.load();
|
||||
});
|
||||
// Editability of the current context-menu target file, resolved async.
|
||||
@@ -1798,6 +1732,10 @@
|
||||
// user has active. First-appearance bucketing in
|
||||
// `buildResourceSections` keys off the item order in the input list.
|
||||
const rlItems = $derived.by<Array<FileItem | FolderItem>>(() => {
|
||||
// Search mode: the rows are the scoped search hits; the swimlane
|
||||
// hoist never applies there (it's cleared on mode entry, see the
|
||||
// filter effect) so the dotfile filter passes straight through.
|
||||
if (searchActive) return filterDotfiles(searchItems, preferences.hideDotfiles);
|
||||
const filtered = filterDotfiles(orderedItems, preferences.hideDotfiles);
|
||||
if (newlyAdded.size === 0) return filtered;
|
||||
const hoisted: Array<FileItem | FolderItem> = [];
|
||||
@@ -1809,6 +1747,23 @@
|
||||
return [...hoisted, ...rest];
|
||||
});
|
||||
|
||||
// ── Folder content stat ──────────────────────────────────────────────────
|
||||
// Counts of what the listing actually renders (post-dotfile-filter, i.e.
|
||||
// exactly the rows on screen), shown next to the breadcrumb. Listing is
|
||||
// cursor-paginated, so while more pages exist (`pageCursor` defined) the
|
||||
// numbers are partial — a "+" suffix says so instead of claiming exactness
|
||||
// for a folder larger than one page.
|
||||
const folderStat = $derived.by(() => {
|
||||
const files = rlItems.filter(isFile).length;
|
||||
// Search mode shows a flat result count instead of the folders/files
|
||||
// split (the recursive result set isn't "this folder's content");
|
||||
// partial/+ semantics carry over via the active mode's cursor.
|
||||
if (searchActive) {
|
||||
return { folders: 0, files: rlItems.length, partial: searchCursor !== undefined };
|
||||
}
|
||||
return { folders: rlItems.length - files, files, partial: pageCursor !== undefined };
|
||||
});
|
||||
|
||||
// Group-by state (bound to <ResourceList>). Kept as a `string` prop
|
||||
// value; the current `sortField` mirrors from the picked group's
|
||||
// `orderBy` so a group-by change also drives the sort.
|
||||
@@ -1965,10 +1920,37 @@
|
||||
// you just added"; carrying it across folders would surface
|
||||
// stale ids that don't belong to the new listing.
|
||||
newlyAdded.clear();
|
||||
// Always re-load the folder page: even in search mode `load()`
|
||||
// resolves the canonical folder id + breadcrumbs that the
|
||||
// scoped search below is anchored to.
|
||||
void load(true);
|
||||
});
|
||||
});
|
||||
|
||||
// Scoped-search driver. Re-runs the search from page 1 whenever its
|
||||
// inputs change: any filter dimension, the folder it's scoped to
|
||||
// (`currentId`, resolved by `load()` above), or the sort dimension.
|
||||
// Inactive (plain folder listing) is the no-op fast path. `filter` is
|
||||
// a `$state` proxy — the field reads are what register the deps.
|
||||
$effect(() => {
|
||||
void filter.query;
|
||||
void filter.type;
|
||||
void filter.size;
|
||||
void filter.date;
|
||||
void filter.recursive;
|
||||
void currentId;
|
||||
void sortField;
|
||||
void reversed;
|
||||
const active = searchActive;
|
||||
untrack(() => {
|
||||
if (!active) return;
|
||||
// Mode entry / re-run → drop the swimlane so its hoisting
|
||||
// never fights the search ordering.
|
||||
newlyAdded.clear();
|
||||
void runSearch(true);
|
||||
});
|
||||
});
|
||||
|
||||
// The command palette's "Upload files" action navigates here then dispatches
|
||||
// this event so the hidden file picker opens (the input lives on this page).
|
||||
$effect(() => {
|
||||
@@ -2001,6 +1983,14 @@
|
||||
<ReadOnlyBanner driveName={currentDrive.name} />
|
||||
{/if}
|
||||
|
||||
<!-- Fuzzy filter bar: scoped keyword + type/size/date presets over the
|
||||
current folder (recursive toggle inside). While any dimension is
|
||||
active the listing below switches from the folder page to the
|
||||
search results; clearing it returns to the plain folder view. -->
|
||||
<div class="files-filter-row">
|
||||
<SearchFilterBar bind:value={filter} />
|
||||
</div>
|
||||
|
||||
<!-- Hidden upload inputs stay mounted even while the batch bar is shown.
|
||||
Kept OUTSIDE ResourceList so the split-button dropdown in the
|
||||
`actions` snippet can click() them without ResourceList's internal
|
||||
@@ -2026,16 +2016,20 @@
|
||||
<ResourceList
|
||||
title={t('nav.files', 'Files')}
|
||||
items={rlItems}
|
||||
emptyText={hiddenCount > 0
|
||||
emptyText={searchActive
|
||||
? t('search.no_results', 'No results found for this search')
|
||||
: hiddenCount > 0
|
||||
? t('files.empty_hidden_title', { n: hiddenCount }, '{{n}} hidden item(s) in this folder')
|
||||
: t('files.empty_title', 'This folder is empty')}
|
||||
emptyHint={hiddenCount > 0
|
||||
emptyHint={searchActive
|
||||
? t('search.prompt', 'Type a query in the search bar above.')
|
||||
: hiddenCount > 0
|
||||
? t(
|
||||
'files.empty_hidden_hint',
|
||||
"Files whose name starts with '.' are hidden. Toggle the setting to see them."
|
||||
)
|
||||
: t('files.empty_hint', 'Drop files here or use the Upload button to add files.')}
|
||||
emptyIcon={hiddenCount > 0 ? 'eye-slash' : undefined}
|
||||
emptyIcon={searchActive ? 'search' : hiddenCount > 0 ? 'eye-slash' : undefined}
|
||||
{loading}
|
||||
error={error ?? undefined}
|
||||
selectable
|
||||
@@ -2052,7 +2046,7 @@
|
||||
groupBys={rlGroupBys}
|
||||
bind:groupBy
|
||||
bind:reversed
|
||||
hasMore={pageCursor !== undefined}
|
||||
hasMore={searchActive ? searchCursor !== undefined : pageCursor !== undefined}
|
||||
onloadmore={loadMore}
|
||||
onreload={(orderBy) => {
|
||||
sortField = orderBy as SortField;
|
||||
@@ -2074,8 +2068,10 @@
|
||||
<!-- Surfaces only when the folder isn't really empty — it's just
|
||||
filtered because the user chose to hide dotfiles. Clicking
|
||||
flips the app-wide `preferences.hideDotfiles` back off,
|
||||
re-populating the list without a hunt through settings. -->
|
||||
{#if hiddenCount > 0}
|
||||
re-populating the list without a hunt through settings.
|
||||
Suppressed in search mode: an empty result there is a real
|
||||
"nothing matched", not a hidden-items artifact. -->
|
||||
{#if !searchActive && hiddenCount > 0}
|
||||
<button
|
||||
class="btn btn-secondary"
|
||||
onclick={() => preferences.setHideDotfiles(false)}
|
||||
@@ -2106,6 +2102,29 @@
|
||||
onDrop={(target, e) => onCrumbDrop(e, target)}
|
||||
dragMime={DRAG_TYPE}
|
||||
/>
|
||||
{#if searchActive}
|
||||
<!-- Filter mode: a flat result count replaces the folders/files
|
||||
split — the recursive result set isn't "this folder's
|
||||
content". "+" keeps the partial-pages meaning. -->
|
||||
{#if rlItems.length > 0}
|
||||
<span class="folder-stat" data-testid="files-folder-stat">
|
||||
{t('filter.results_count', { n: rlItems.length }, '{{n}} results')}{folderStat.partial
|
||||
? '+'
|
||||
: ''}
|
||||
</span>
|
||||
{/if}
|
||||
{:else if folderStat.folders + folderStat.files > 0}
|
||||
<!-- Item count for the folder on screen. Sits in the same sticky
|
||||
strip as the breadcrumb so it stays visible while scrolling.
|
||||
"+" = more pages are still loading via infinite scroll. -->
|
||||
<span class="folder-stat" data-testid="files-folder-stat">
|
||||
{t(
|
||||
'files.folder_stat',
|
||||
{ folders: folderStat.folders, files: folderStat.files },
|
||||
'{{folders}} folders · {{files}} files'
|
||||
)}{folderStat.partial ? '+' : ''}
|
||||
</span>
|
||||
{/if}
|
||||
{/snippet}
|
||||
|
||||
{#snippet actions()}
|
||||
@@ -2155,6 +2174,21 @@
|
||||
<Icon name="folder-plus" class="icon-mr" />
|
||||
<span>{t('actions.new_folder', 'New folder')}</span>
|
||||
</button>
|
||||
<!-- Manual reload of the current folder: resets pagination to page 1
|
||||
and refetches (listing accumulator + folder stat + dotfile
|
||||
filter all recompute). Disabled while a load is already in
|
||||
flight — the button is a convenience, not a hammer. -->
|
||||
<button
|
||||
class="btn btn-secondary"
|
||||
data-testid="files-refresh-btn"
|
||||
title={t('common.refresh', 'Refresh')}
|
||||
aria-label={t('common.refresh', 'Refresh')}
|
||||
disabled={loading}
|
||||
onclick={() => void load(true)}
|
||||
>
|
||||
<Icon name="repeat" class="icon-mr" />
|
||||
<span>{t('common.refresh', 'Refresh')}</span>
|
||||
</button>
|
||||
{/snippet}
|
||||
|
||||
{#snippet batchActions(_sel)}
|
||||
@@ -2162,7 +2196,7 @@
|
||||
class="batch-btn"
|
||||
title={t('files.add_favorites', 'Add to favorites')}
|
||||
data-testid="files-batch-favorite-btn"
|
||||
onclick={() => void batchFavorites()}
|
||||
onclick={() => void resActions.batchFavorites()}
|
||||
>
|
||||
<Icon name="star" />
|
||||
<span>{t('files.add_favorites', 'Add to favorites')}</span>
|
||||
@@ -2171,7 +2205,7 @@
|
||||
class="batch-btn"
|
||||
title={t('files.move', 'Move')}
|
||||
data-testid="files-batch-move-btn"
|
||||
onclick={batchMove}
|
||||
onclick={resActions.batchMove}
|
||||
>
|
||||
<Icon name="arrows-alt" />
|
||||
<span>{t('files.move', 'Move')}</span>
|
||||
@@ -2180,7 +2214,7 @@
|
||||
class="batch-btn"
|
||||
title={t('files.copy', 'Copy')}
|
||||
data-testid="files-batch-copy-btn"
|
||||
onclick={batchCopy}
|
||||
onclick={resActions.batchCopy}
|
||||
>
|
||||
<Icon name="copy" />
|
||||
<span>{t('files.copy', 'Copy')}</span>
|
||||
@@ -2189,7 +2223,7 @@
|
||||
class="batch-btn"
|
||||
title={t('common.download', 'Download')}
|
||||
data-testid="files-batch-download-btn"
|
||||
onclick={() => void batchDownload()}
|
||||
onclick={() => void resActions.batchDownload()}
|
||||
>
|
||||
<Icon name="download" />
|
||||
<span>{t('common.download', 'Download')}</span>
|
||||
@@ -2198,7 +2232,7 @@
|
||||
class="batch-btn batch-btn-danger"
|
||||
title={t('common.delete', 'Delete')}
|
||||
data-testid="files-batch-delete-btn"
|
||||
onclick={batchDelete}
|
||||
onclick={() => void resActions.batchDelete()}
|
||||
>
|
||||
<Icon name="trash" />
|
||||
<span>{t('common.delete', 'Delete')}</span>
|
||||
@@ -2210,14 +2244,11 @@
|
||||
{#if moveDialog.component}
|
||||
{@const MoveDialog = moveDialog.component}
|
||||
<MoveDialog
|
||||
bind:open={moveOpen}
|
||||
item={actionTarget}
|
||||
items={moveItems}
|
||||
mode={moveMode}
|
||||
onmoved={() => {
|
||||
clearSelection();
|
||||
void reload();
|
||||
}}
|
||||
bind:open={resActions.moveDialog.open}
|
||||
item={resActions.moveDialog.item}
|
||||
items={resActions.moveDialog.items}
|
||||
mode={resActions.moveDialog.mode}
|
||||
onmoved={resActions.handleMoved}
|
||||
/>
|
||||
{/if}
|
||||
{#if shareDialog.component}
|
||||
@@ -2434,6 +2465,17 @@
|
||||
z-index: 1000;
|
||||
}
|
||||
|
||||
/* Item count next to the breadcrumb (same sticky strip; `.rl-breadcrumb`
|
||||
is already a flex row, so the span just flows beside the crumbs).
|
||||
`flex-shrink: 0` keeps long crumb trails from squeezing the digits
|
||||
into a vertical stack on narrow viewports — the crumbs wrap instead. */
|
||||
.folder-stat {
|
||||
flex-shrink: 0;
|
||||
color: var(--color-text-muted);
|
||||
font-size: 0.8125rem;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.ctx-menu {
|
||||
position: fixed;
|
||||
z-index: 1001;
|
||||
@@ -2471,4 +2513,9 @@
|
||||
rendered near-invisible here). Mirrors the user-menu logout red. */
|
||||
color: var(--color-danger-alt);
|
||||
}
|
||||
|
||||
.files-filter-row {
|
||||
padding: 0 var(--space-2);
|
||||
margin-bottom: var(--space-1);
|
||||
}
|
||||
</style>
|
||||
|
||||
@@ -27,14 +27,19 @@ vi.mock('$app/state', () => ({ page: pageState }));
|
||||
vi.mock('$lib/stores/session.svelte', () => ({ session }));
|
||||
vi.mock('$lib/stores/ui.svelte', () => ({ ui }));
|
||||
vi.mock('$lib/stores/dialogs.svelte', () => ({ confirmDialog, promptDialog }));
|
||||
vi.mock('$lib/api/client', () => ({ apiFetch: vi.fn() }));
|
||||
vi.mock('$lib/api/client', () => ({ apiFetch: vi.fn(), apiJson: vi.fn() }));
|
||||
vi.mock('$lib/api/csrf', () => ({ getCsrfHeaders: () => ({}) }));
|
||||
vi.mock('$lib/api/endpoints/search', () => ({ searchResources: vi.fn() }));
|
||||
vi.mock('$lib/api/endpoints/deltaUpload', () => ({
|
||||
instantUploadOwned: vi.fn(),
|
||||
resolveOwnedHashes: vi.fn(),
|
||||
tryDeltaUpload: vi.fn()
|
||||
}));
|
||||
vi.mock('$lib/api/endpoints/favorites', () => ({ addFavorite: vi.fn(), removeFavorite: vi.fn() }));
|
||||
vi.mock('$lib/api/endpoints/favorites', () => ({
|
||||
addFavorite: vi.fn(),
|
||||
removeFavorite: vi.fn(),
|
||||
addFavoritesBatch: vi.fn()
|
||||
}));
|
||||
vi.mock('$lib/api/endpoints/wopi', () => ({
|
||||
canEditWithWopi: () => false,
|
||||
getEditorUrlWithFallback: vi.fn()
|
||||
@@ -77,7 +82,8 @@ vi.mock('$lib/api/endpoints/folders', () => ({
|
||||
import { fetchFolderPage, createFolder, deleteFolder } from '$lib/api/endpoints/folders';
|
||||
import { deleteFile, uploadFileWithProgress } from '$lib/api/endpoints/files';
|
||||
import { resolveOwnedHashes, tryDeltaUpload } from '$lib/api/endpoints/deltaUpload';
|
||||
import { apiFetch } from '$lib/api/client';
|
||||
import { addFavoritesBatch } from '$lib/api/endpoints/favorites';
|
||||
import { searchResources } from '$lib/api/endpoints/search';
|
||||
import { files as filesStore } from '$lib/stores/files.svelte';
|
||||
import FilesPage from './[...path]/+page.svelte';
|
||||
|
||||
@@ -196,6 +202,41 @@ it('loads the home folder listing on mount and renders its contents', async () =
|
||||
await screen.findByTestId('files-new-folder-btn');
|
||||
});
|
||||
|
||||
it('shows the folder item count next to the breadcrumb', async () => {
|
||||
withListing(); // 1 folder + 1 file, `nextCursor` undefined → last page
|
||||
render(FilesPage);
|
||||
const stat = await screen.findByTestId('files-folder-stat');
|
||||
expect(stat.textContent).toContain('1 folders · 1 files');
|
||||
// No "+" suffix — the listing is complete, the count is exact.
|
||||
expect(stat.textContent!.trim().endsWith('+')).toBe(false);
|
||||
});
|
||||
|
||||
it('marks the folder count as partial while more pages exist', async () => {
|
||||
const folder = folderItem('sub1', 'Sub');
|
||||
const file = fileItem('f1', 'hello.txt');
|
||||
m(fetchFolderPage).mockResolvedValue({
|
||||
items: [folder, file],
|
||||
folders: [folder],
|
||||
files: [file],
|
||||
nextCursor: 'page-2'
|
||||
});
|
||||
render(FilesPage);
|
||||
const stat = await screen.findByTestId('files-folder-stat');
|
||||
// Counts reflect the pages loaded so far; the trailing "+" says more
|
||||
// are on the way via infinite scroll instead of claiming exactness.
|
||||
expect(stat.textContent).toContain('1 folders · 1 files');
|
||||
expect(stat.textContent!.trim().endsWith('+')).toBe(true);
|
||||
});
|
||||
|
||||
it('reloads the listing when the refresh button is clicked', async () => {
|
||||
withListing();
|
||||
render(FilesPage);
|
||||
await screen.findByTestId('files-refresh-btn');
|
||||
// Initial mount load = 1 call; the click resets pagination and refetches.
|
||||
await fireEvent.click(screen.getByTestId('files-refresh-btn'));
|
||||
await waitFor(() => expect(fetchFolderPage).toHaveBeenCalledTimes(2));
|
||||
});
|
||||
|
||||
it('shows an error when the listing fails with no cache', async () => {
|
||||
m(fetchFolderPage).mockRejectedValue(Object.assign(new Error('nope'), { status: 500 }));
|
||||
render(FilesPage);
|
||||
@@ -233,14 +274,50 @@ it('batch-deletes the whole selection after confirmation', async () => {
|
||||
|
||||
it('batch-favorites the selection via the favorites batch endpoint', async () => {
|
||||
withListing();
|
||||
m(apiFetch).mockResolvedValue({ ok: true });
|
||||
m(addFavoritesBatch).mockResolvedValue(undefined);
|
||||
render(FilesPage);
|
||||
await fireEvent.click(await screen.findByTestId('resource-list-select-all-checkbox'));
|
||||
await fireEvent.click(await screen.findByTestId('files-batch-favorite-btn'));
|
||||
await waitFor(() =>
|
||||
expect(apiFetch).toHaveBeenCalledWith(
|
||||
'/api/favorites/batch',
|
||||
expect.objectContaining({ method: 'POST' })
|
||||
)
|
||||
expect(addFavoritesBatch).toHaveBeenCalledWith([
|
||||
{ item_id: 'sub1', item_type: 'folder' },
|
||||
{ item_id: 'f1', item_type: 'file' }
|
||||
])
|
||||
);
|
||||
});
|
||||
|
||||
it('runs a scoped recursive search when the filter keyword is set', async () => {
|
||||
withListing();
|
||||
m(searchResources).mockResolvedValue({ items: [], query_time_ms: 1 });
|
||||
render(FilesPage);
|
||||
await waitFor(() => expect(fetchFolderPage).toHaveBeenCalled());
|
||||
vi.useFakeTimers();
|
||||
await fireEvent.input(screen.getByTestId('filter-keyword-input'), {
|
||||
target: { value: 'hello' }
|
||||
});
|
||||
await vi.advanceTimersByTimeAsync(400);
|
||||
vi.useRealTimers();
|
||||
expect(searchResources).toHaveBeenCalledWith(
|
||||
'hello',
|
||||
expect.objectContaining({ folderId: 'home', recursive: true })
|
||||
);
|
||||
});
|
||||
|
||||
it('shows the search result count while the filter is active', async () => {
|
||||
withListing();
|
||||
const hit = fileItem('s1', 'found.txt');
|
||||
m(searchResources).mockResolvedValue({
|
||||
items: [{ resource_type: 'file', resource: hit, meta: { score: 50 } }],
|
||||
query_time_ms: 1
|
||||
});
|
||||
render(FilesPage);
|
||||
await waitFor(() => expect(fetchFolderPage).toHaveBeenCalled());
|
||||
vi.useFakeTimers();
|
||||
await fireEvent.input(screen.getByTestId('filter-keyword-input'), {
|
||||
target: { value: 'found' }
|
||||
});
|
||||
await vi.advanceTimersByTimeAsync(400);
|
||||
vi.useRealTimers();
|
||||
const stat = await screen.findByTestId('files-folder-stat');
|
||||
expect(stat.textContent).toContain('1 results');
|
||||
});
|
||||
|
||||
@@ -287,7 +287,29 @@
|
||||
</form>
|
||||
{:else if view === 'file'}
|
||||
<div class="share__center">
|
||||
{#if meta && mediaKind(meta.mime_type) === 'video'}
|
||||
<!-- Inline player: Range-aware endpoint streams the video, so the
|
||||
timeline seeks without downloading the whole file first.
|
||||
lazyVideo defers the load, seeks a few frames in for a poster
|
||||
and retries once on error (same behaviour as the folder grid). -->
|
||||
<video
|
||||
class="share__media"
|
||||
data-testid="public-share-file-video"
|
||||
use:lazyVideo={shareFileUrl(token, meta.item_id)}
|
||||
controls
|
||||
playsinline
|
||||
></video>
|
||||
{:else if meta && mediaKind(meta.mime_type) === 'image'}
|
||||
<img
|
||||
class="share__media"
|
||||
data-testid="public-share-file-image"
|
||||
src={shareFileUrl(token, meta.item_id)}
|
||||
alt={meta.item_name}
|
||||
use:imageRetry
|
||||
/>
|
||||
{:else}
|
||||
<Icon name="file" class="share__big-icon" />
|
||||
{/if}
|
||||
<h1>{meta?.item_name}</h1>
|
||||
<a
|
||||
class="share__btn"
|
||||
@@ -486,6 +508,13 @@
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.share__media {
|
||||
max-width: 100%;
|
||||
max-height: min(70vh, 40rem);
|
||||
border-radius: var(--radius-2xl);
|
||||
border: 1px solid var(--color-border);
|
||||
}
|
||||
|
||||
:global(.share__big-icon) {
|
||||
font-size: 3rem;
|
||||
color: var(--color-text-muted);
|
||||
|
||||
@@ -22,10 +22,18 @@
|
||||
import type { FileItem, FolderItem, SearchResourceItem, SortBy } from '$lib/api/types';
|
||||
import { lazyComponent } from '$lib/composables/lazyComponent.svelte';
|
||||
import { folderAccessCached, probeFolderAccess } from '$lib/utils/folderAccess';
|
||||
import { TYPE_EXT, dateBound, sizeBounds } from '$lib/utils/searchFilters';
|
||||
import { SvelteSet } from 'svelte/reactivity';
|
||||
import { replaceSet } from '$lib/utils/sets';
|
||||
import {
|
||||
useResourceActions,
|
||||
type ActionTarget
|
||||
} from '$lib/composables/useResourceActions.svelte';
|
||||
import Icon from '$lib/icons/Icon.svelte';
|
||||
import { t } from '$lib/i18n/index.svelte';
|
||||
import { confirmDialog, promptDialog } from '$lib/stores/dialogs.svelte';
|
||||
import { files as filesStore } from '$lib/stores/files.svelte';
|
||||
import { session } from '$lib/stores/session.svelte';
|
||||
import { ui } from '$lib/stores/ui.svelte';
|
||||
|
||||
const query = $derived(page.url.searchParams.get('q') ?? '');
|
||||
@@ -145,7 +153,9 @@
|
||||
void goto(target, { replaceState: true, keepFocus: true, noScroll: true });
|
||||
}
|
||||
|
||||
// Filters
|
||||
// Filters — the preset vocabularies and their SearchOptions mapping live
|
||||
// in the shared `searchFilters` util (also consumed by the files page's
|
||||
// filter bar); only the i18n label lists stay local since they need `t()`.
|
||||
type TypeKey = 'all' | 'image' | 'video' | 'document' | 'audio' | 'archive';
|
||||
type SizeKey = 'all' | 'small' | 'medium' | 'large';
|
||||
type DateKey = 'all' | 'day' | 'week' | 'month' | 'year';
|
||||
@@ -153,26 +163,6 @@
|
||||
let sizeFilter = $state<SizeKey>('all');
|
||||
let dateFilter = $state<DateKey>('all');
|
||||
|
||||
const TYPE_EXT: Record<Exclude<TypeKey, 'all'>, string[]> = {
|
||||
image: ['jpg', 'jpeg', 'png', 'gif', 'webp', 'svg', 'bmp', 'heic', 'avif', 'tiff'],
|
||||
video: ['mp4', 'mov', 'mkv', 'avi', 'webm', 'm4v', 'wmv', 'flv'],
|
||||
document: [
|
||||
'pdf',
|
||||
'doc',
|
||||
'docx',
|
||||
'xls',
|
||||
'xlsx',
|
||||
'ppt',
|
||||
'pptx',
|
||||
'txt',
|
||||
'md',
|
||||
'odt',
|
||||
'rtf',
|
||||
'csv'
|
||||
],
|
||||
audio: ['mp3', 'wav', 'flac', 'aac', 'ogg', 'm4a', 'opus'],
|
||||
archive: ['zip', 'rar', '7z', 'tar', 'gz', 'bz2', 'xz']
|
||||
};
|
||||
const TYPES: { v: TypeKey; l: string }[] = [
|
||||
{ v: 'all', l: t('search.type.all', 'All types') },
|
||||
{ v: 'image', l: t('search.type.image', 'Images') },
|
||||
@@ -195,36 +185,6 @@
|
||||
{ v: 'year', l: t('search.date.year', 'Past year') }
|
||||
];
|
||||
|
||||
const MB = 1024 * 1024;
|
||||
function sizeBounds(k: SizeKey): { minSize?: number; maxSize?: number } {
|
||||
switch (k) {
|
||||
case 'small':
|
||||
return { maxSize: MB };
|
||||
case 'medium':
|
||||
return { minSize: MB, maxSize: 100 * MB };
|
||||
case 'large':
|
||||
return { minSize: 100 * MB };
|
||||
default:
|
||||
return {};
|
||||
}
|
||||
}
|
||||
function dateBound(k: DateKey): number | undefined {
|
||||
const day = 86400;
|
||||
const now = Math.floor(Date.now() / 1000);
|
||||
switch (k) {
|
||||
case 'day':
|
||||
return now - day;
|
||||
case 'week':
|
||||
return now - 7 * day;
|
||||
case 'month':
|
||||
return now - 30 * day;
|
||||
case 'year':
|
||||
return now - 365 * day;
|
||||
default:
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
const hasFilters = $derived(typeFilter !== 'all' || sizeFilter !== 'all' || dateFilter !== 'all');
|
||||
function clearFilters() {
|
||||
typeFilter = 'all';
|
||||
@@ -294,6 +254,8 @@
|
||||
scope === 'folder' && filesStore.section !== 'trash'
|
||||
? (effectiveFolder ?? undefined)
|
||||
: undefined;
|
||||
// TYPE_EXT / sizeBounds / dateBound come from `$lib/utils/searchFilters`
|
||||
// (shared with the files-page filter bar).
|
||||
return {
|
||||
recursive: true,
|
||||
sortBy: orderByForGroup() as SortBy,
|
||||
@@ -383,16 +345,46 @@
|
||||
// all reuse the same lazy dialogs.
|
||||
let viewerOpen = $state(false);
|
||||
let viewerFile = $state<FileItem | null>(null);
|
||||
let moveOpen = $state(false);
|
||||
let moveTarget = $state<{ id: string; name: string; kind: 'file' | 'folder' } | null>(null);
|
||||
let shareOpen = $state(false);
|
||||
let shareTarget = $state<{ id: string; name: string; kind: 'file' | 'folder' } | null>(null);
|
||||
const fileViewer = lazyComponent(() => import('$lib/components/FileViewer.svelte'));
|
||||
const moveDialog = lazyComponent(() => import('$lib/components/MoveDialog.svelte'));
|
||||
const shareDialog = lazyComponent(() => import('$lib/components/ShareDialog.svelte'));
|
||||
|
||||
// ── Multi-select + batch actions ─────────────────────────────────────
|
||||
// Same wiring as the files page: ResourceList owns the row-level
|
||||
// selection UX and mirrors it out via `onselectionchange`; the shared
|
||||
// composable owns the batch favorite/download/delete/move/copy flows
|
||||
// and the MoveDialog state. After any mutation the search re-runs —
|
||||
// a move/delete can shift rows in or out of the current scope and
|
||||
// filter set, so patching in place would go stale.
|
||||
const selected = new SvelteSet<string>();
|
||||
function clearSelection() {
|
||||
selected.clear();
|
||||
}
|
||||
const resActions = useResourceActions({
|
||||
getItems: () => items,
|
||||
getSelected: () => selected,
|
||||
clearSelection,
|
||||
onChanged: () => run(query),
|
||||
afterDelete: () => void session.refresh()
|
||||
});
|
||||
|
||||
function onKeydown(e: KeyboardEvent) {
|
||||
const tag = (e.target as HTMLElement)?.tagName;
|
||||
if (tag === 'INPUT' || tag === 'TEXTAREA' || tag === 'SELECT') return;
|
||||
if (e.key === 'Escape' && selected.size) {
|
||||
clearSelection();
|
||||
} else if (e.key === 'Delete' && selected.size) {
|
||||
// Delete only — Backspace was dropped: it triggered accidental deletes.
|
||||
e.preventDefault();
|
||||
void resActions.batchDelete();
|
||||
}
|
||||
}
|
||||
|
||||
$effect(() => {
|
||||
if (viewerOpen) void fileViewer.load();
|
||||
if (moveOpen) void moveDialog.load();
|
||||
if (resActions.moveDialog.open) void moveDialog.load();
|
||||
if (shareOpen) void shareDialog.load();
|
||||
});
|
||||
|
||||
@@ -446,8 +438,8 @@
|
||||
}
|
||||
|
||||
function openMoveDialog(item: FileItem | FolderItem) {
|
||||
moveTarget = { id: item.id, name: item.name, kind: kindOf(item) };
|
||||
moveOpen = true;
|
||||
const target: ActionTarget = { id: item.id, name: item.name, kind: kindOf(item) };
|
||||
resActions.openMove(target);
|
||||
}
|
||||
|
||||
function downloadItem(item: FileItem | FolderItem) {
|
||||
@@ -615,7 +607,7 @@
|
||||
|
||||
<svelte:head><title>{t('search.title', 'Search')} · OxiCloud</title></svelte:head>
|
||||
|
||||
<svelte:window ondragover={onWindowDragOver} ondrop={onWindowDrop} />
|
||||
<svelte:window ondragover={onWindowDragOver} ondrop={onWindowDrop} onkeydown={onKeydown} />
|
||||
|
||||
{#if !query}
|
||||
<EmptyState title={t('search.prompt', 'Type a query in the search bar above.')} />
|
||||
@@ -629,6 +621,9 @@
|
||||
emptyText={t('search.no_results', 'No results found for this search')}
|
||||
hasMore={!!cursor}
|
||||
onloadmore={loadMore}
|
||||
selectable
|
||||
shiftRangeSelect
|
||||
onselectionchange={(ids) => replaceSet(selected, ids)}
|
||||
showPath
|
||||
showViewToggle
|
||||
onopen={open}
|
||||
@@ -750,6 +745,56 @@
|
||||
<FolderBreadcrumb folderId={scopeFolderId} />
|
||||
{/if}
|
||||
{/snippet}
|
||||
{#snippet batchActions(_sel)}
|
||||
<!-- Same five batch buttons as the files page (shared
|
||||
`useResourceActions` composable); testids are search-prefixed
|
||||
so the two pages' tests stay unambiguous. -->
|
||||
<button
|
||||
class="batch-btn"
|
||||
title={t('files.add_favorites', 'Add to favorites')}
|
||||
data-testid="search-batch-favorite-btn"
|
||||
onclick={() => void resActions.batchFavorites()}
|
||||
>
|
||||
<Icon name="star" />
|
||||
<span>{t('files.add_favorites', 'Add to favorites')}</span>
|
||||
</button>
|
||||
<button
|
||||
class="batch-btn"
|
||||
title={t('files.move', 'Move')}
|
||||
data-testid="search-batch-move-btn"
|
||||
onclick={resActions.batchMove}
|
||||
>
|
||||
<Icon name="arrows-alt" />
|
||||
<span>{t('files.move', 'Move')}</span>
|
||||
</button>
|
||||
<button
|
||||
class="batch-btn"
|
||||
title={t('files.copy', 'Copy')}
|
||||
data-testid="search-batch-copy-btn"
|
||||
onclick={resActions.batchCopy}
|
||||
>
|
||||
<Icon name="copy" />
|
||||
<span>{t('files.copy', 'Copy')}</span>
|
||||
</button>
|
||||
<button
|
||||
class="batch-btn"
|
||||
title={t('common.download', 'Download')}
|
||||
data-testid="search-batch-download-btn"
|
||||
onclick={() => void resActions.batchDownload()}
|
||||
>
|
||||
<Icon name="download" />
|
||||
<span>{t('common.download', 'Download')}</span>
|
||||
</button>
|
||||
<button
|
||||
class="batch-btn batch-btn-danger"
|
||||
title={t('common.delete', 'Delete')}
|
||||
data-testid="search-batch-delete-btn"
|
||||
onclick={() => void resActions.batchDelete()}
|
||||
>
|
||||
<Icon name="trash" />
|
||||
<span>{t('common.delete', 'Delete')}</span>
|
||||
</button>
|
||||
{/snippet}
|
||||
{#snippet itemActions(item)}
|
||||
<!--
|
||||
Per-row "Open parent folder" quick-action — search results
|
||||
@@ -787,16 +832,11 @@
|
||||
{#if moveDialog.component}
|
||||
{@const MoveDialog = moveDialog.component}
|
||||
<MoveDialog
|
||||
bind:open={moveOpen}
|
||||
item={moveTarget}
|
||||
onmoved={() => {
|
||||
// A move can shift the row out of the current scope (`?in=<uuid>`)
|
||||
// or into it, and the SQL name-match count may change. Reload
|
||||
// page 1 rather than trying to patch state in place — search
|
||||
// state is already reactive on query/scope so a fresh `run()`
|
||||
// is cheap and correct.
|
||||
void run(query);
|
||||
}}
|
||||
bind:open={resActions.moveDialog.open}
|
||||
item={resActions.moveDialog.item}
|
||||
items={resActions.moveDialog.items}
|
||||
mode={resActions.moveDialog.mode}
|
||||
onmoved={resActions.handleMoved}
|
||||
/>
|
||||
{/if}
|
||||
{#if shareDialog.component}
|
||||
|
||||
@@ -1,24 +1,108 @@
|
||||
import { it, expect, vi, beforeEach } from 'vitest';
|
||||
import { render, screen, waitFor } from '@testing-library/svelte';
|
||||
import { render, screen, waitFor, fireEvent } from '@testing-library/svelte';
|
||||
|
||||
const { goto, pageState } = vi.hoisted(() => ({
|
||||
const { goto, pageState, session, ui, confirmDialog, promptDialog } = vi.hoisted(() => ({
|
||||
goto: vi.fn(),
|
||||
pageState: { url: new URL('http://localhost/search?q=report') }
|
||||
pageState: { url: new URL('http://localhost/search?q=report') },
|
||||
session: {
|
||||
user: { id: 'me', username: 'admin', is_external: false },
|
||||
isExternalUser: false,
|
||||
loadHomeFolder: vi.fn(async () => 'home'),
|
||||
refresh: vi.fn(async () => {})
|
||||
},
|
||||
ui: {
|
||||
notify: vi.fn(),
|
||||
startProgress: vi.fn(() => 1),
|
||||
updateProgress: vi.fn(),
|
||||
finishProgress: vi.fn()
|
||||
},
|
||||
confirmDialog: vi.fn(),
|
||||
promptDialog: vi.fn()
|
||||
}));
|
||||
vi.mock('$app/navigation', () => ({ goto }));
|
||||
vi.mock('$app/state', () => ({ page: pageState }));
|
||||
vi.mock('$lib/api/endpoints/search', () => ({ searchResources: vi.fn() }));
|
||||
vi.mock('$lib/api/endpoints/files', () => ({ fileInlineUrl: () => '/in' }));
|
||||
vi.mock('$lib/api/client', () => ({ apiFetch: vi.fn(), apiJson: vi.fn() }));
|
||||
vi.mock('$lib/api/csrf', () => ({ getCsrfHeaders: () => ({}) }));
|
||||
vi.mock('$lib/stores/session.svelte', () => ({ session }));
|
||||
vi.mock('$lib/stores/ui.svelte', () => ({ ui }));
|
||||
vi.mock('$lib/stores/dialogs.svelte', () => ({ confirmDialog, promptDialog }));
|
||||
vi.mock('$lib/api/endpoints/files', () => ({
|
||||
fileInlineUrl: () => '/in',
|
||||
fileDownloadUrl: () => '/dl',
|
||||
fileThumbnailUrl: () => '/thumb',
|
||||
thumbSizeForView: () => 'preview' as const,
|
||||
deleteFile: vi.fn(),
|
||||
moveFile: vi.fn(),
|
||||
renameFile: vi.fn()
|
||||
}));
|
||||
vi.mock('$lib/api/endpoints/folders', () => ({
|
||||
deleteFolder: vi.fn(),
|
||||
moveFolder: vi.fn(),
|
||||
renameFolder: vi.fn()
|
||||
}));
|
||||
vi.mock('$lib/api/endpoints/favorites', () => ({
|
||||
addFavorite: vi.fn(),
|
||||
removeFavorite: vi.fn(),
|
||||
addFavoritesBatch: vi.fn(),
|
||||
dateBucket: () => 'bucket',
|
||||
sizeBucket: () => 'bucket'
|
||||
}));
|
||||
|
||||
import { searchResources } from '$lib/api/endpoints/search';
|
||||
import { deleteFile } from '$lib/api/endpoints/files';
|
||||
import { deleteFolder } from '$lib/api/endpoints/folders';
|
||||
import { files as filesStore } from '$lib/stores/files.svelte';
|
||||
import SearchPage from './+page.svelte';
|
||||
|
||||
const m = (fn: unknown) => fn as ReturnType<typeof vi.fn>;
|
||||
|
||||
const searchHit = () => ({
|
||||
items: [
|
||||
{
|
||||
resource_type: 'file',
|
||||
resource: {
|
||||
id: 'f1',
|
||||
name: 'a-report.txt',
|
||||
mime_type: 'text/plain',
|
||||
folder_id: 'p',
|
||||
category: 'Document',
|
||||
created_at: 0,
|
||||
modified_at: 0,
|
||||
size: 4,
|
||||
created_by: 'me',
|
||||
updated_by: 'me',
|
||||
path: '/a-report.txt'
|
||||
},
|
||||
meta: { score: 50 }
|
||||
},
|
||||
{
|
||||
resource_type: 'folder',
|
||||
resource: {
|
||||
id: 'd1',
|
||||
name: 'reports',
|
||||
parent_id: 'p',
|
||||
category: 'Folder',
|
||||
created_at: 0,
|
||||
modified_at: 0,
|
||||
is_root: false,
|
||||
created_by: 'me',
|
||||
updated_by: 'me',
|
||||
path: '/reports'
|
||||
},
|
||||
meta: { score: 50 }
|
||||
}
|
||||
],
|
||||
query_time_ms: 1,
|
||||
total: 2
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
pageState.url = new URL('http://localhost/search?q=report');
|
||||
m(searchResources).mockResolvedValue({ items: [], query_time_ms: 0, total: 0 });
|
||||
// List view renders the select-all header + per-row checkboxes; grid hides them.
|
||||
filesStore.viewMode = 'list';
|
||||
});
|
||||
|
||||
it('runs a search from the q query parameter on mount', async () => {
|
||||
@@ -41,3 +125,15 @@ it('surfaces a search error', async () => {
|
||||
await waitFor(() => expect(searchResources).toHaveBeenCalled());
|
||||
await waitFor(() => expect(screen.getByText('search boom')).toBeTruthy());
|
||||
});
|
||||
|
||||
it('batch-deletes the selected search results after confirmation', async () => {
|
||||
m(searchResources).mockResolvedValue(searchHit());
|
||||
confirmDialog.mockResolvedValue(true);
|
||||
render(SearchPage);
|
||||
await waitFor(() => expect(searchResources).toHaveBeenCalled());
|
||||
await fireEvent.click(await screen.findByTestId('resource-list-select-all-checkbox'));
|
||||
await fireEvent.click(await screen.findByTestId('search-batch-delete-btn'));
|
||||
await waitFor(() => expect(deleteFile).toHaveBeenCalledWith('f1'));
|
||||
await waitFor(() => expect(deleteFolder).toHaveBeenCalledWith('d1'));
|
||||
await waitFor(() => expect(session.refresh).toHaveBeenCalled());
|
||||
});
|
||||
|
||||
@@ -416,6 +416,14 @@
|
||||
"shared_notificationSent": "تم إرسال الإشعار بنجاح",
|
||||
"shared_notificationFailed": "فشل إرسال الإشعار"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "ابحث في هذا المجلد والمجلدات الفرعية…",
|
||||
"keyword": "الكلمة المفتاحية",
|
||||
"advanced": "المرشحات",
|
||||
"recursive": "تضمين المجلدات الفرعية",
|
||||
"results_count": "{{n}} نتائج",
|
||||
"clear_keyword": "مسح البحث"
|
||||
},
|
||||
"files": {
|
||||
"name": "الاسم",
|
||||
"type": "النوع",
|
||||
|
||||
@@ -416,6 +416,14 @@
|
||||
"shared_notificationSent": "Benachrichtigung erfolgreich gesendet",
|
||||
"shared_notificationFailed": "Benachrichtigung konnte nicht gesendet werden"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "Diesen Ordner und Unterordner durchsuchen…",
|
||||
"keyword": "Suchbegriff",
|
||||
"advanced": "Filter",
|
||||
"recursive": "Unterordner einbeziehen",
|
||||
"results_count": "{{n}} Ergebnisse",
|
||||
"clear_keyword": "Suche löschen"
|
||||
},
|
||||
"files": {
|
||||
"name": "Name",
|
||||
"type": "Typ",
|
||||
|
||||
@@ -451,11 +451,20 @@
|
||||
"shared_notificationSent": "Notification sent successfully",
|
||||
"shared_notificationFailed": "Failed to send notification"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "Search this folder and subfolders…",
|
||||
"keyword": "Keyword",
|
||||
"advanced": "Filters",
|
||||
"recursive": "Include subfolders",
|
||||
"results_count": "{{n}} results",
|
||||
"clear_keyword": "Clear search"
|
||||
},
|
||||
"files": {
|
||||
"name": "Name",
|
||||
"type": "Type",
|
||||
"size": "Size",
|
||||
"modified": "Modified",
|
||||
"folder_stat": "{{folders}} folders · {{files}} files",
|
||||
"no_files": "No files in this folder",
|
||||
"empty_hint": "Upload files or create folders to get started",
|
||||
"drop_to_upload": "Drop files here to upload",
|
||||
@@ -1645,6 +1654,7 @@
|
||||
"toggle_theme": "Toggle theme"
|
||||
},
|
||||
"common": {
|
||||
"refresh": "Refresh",
|
||||
"add": "Add",
|
||||
"cancel": "Cancel",
|
||||
"clear": "Clear",
|
||||
|
||||
@@ -416,6 +416,14 @@
|
||||
"mit_license": "Licencia MIT",
|
||||
"title": "Menú de usuario"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "Buscar en esta carpeta y subcarpetas…",
|
||||
"keyword": "Palabra clave",
|
||||
"advanced": "Filtros",
|
||||
"recursive": "Incluir subcarpetas",
|
||||
"results_count": "{{n}} resultados",
|
||||
"clear_keyword": "Borrar búsqueda"
|
||||
},
|
||||
"files": {
|
||||
"name": "Nombre",
|
||||
"type": "Tipo",
|
||||
|
||||
@@ -416,6 +416,14 @@
|
||||
"shared_notificationSent": "آگاهسازی با موفقیت ارسال شد",
|
||||
"shared_notificationFailed": "ارسال آگاهسازی ناموفق بود"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "جستجو در این پوشه و زیرپوشهها…",
|
||||
"keyword": "کلیدواژه",
|
||||
"advanced": "فیلترها",
|
||||
"recursive": "شامل زیرپوشهها",
|
||||
"results_count": "{{n}} نتیجه",
|
||||
"clear_keyword": "پاک کردن جستجو"
|
||||
},
|
||||
"files": {
|
||||
"name": "نام",
|
||||
"type": "نوع",
|
||||
|
||||
@@ -416,6 +416,14 @@
|
||||
"shared_notificationSent": "Notification envoyée avec succès",
|
||||
"shared_notificationFailed": "Erreur lors de l'envoi de la notification"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "Rechercher dans ce dossier et ses sous-dossiers…",
|
||||
"keyword": "Mot-clé",
|
||||
"advanced": "Filtres",
|
||||
"recursive": "Inclure les sous-dossiers",
|
||||
"results_count": "{{n}} résultats",
|
||||
"clear_keyword": "Effacer la recherche"
|
||||
},
|
||||
"files": {
|
||||
"name": "Nom",
|
||||
"type": "Type",
|
||||
|
||||
@@ -416,6 +416,14 @@
|
||||
"shared_notificationSent": "सूचना सफलतापूर्वक भेजी गई",
|
||||
"shared_notificationFailed": "सूचना भेजने में विफल"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "इस फ़ोल्डर और सबफ़ोल्डर में खोजें…",
|
||||
"keyword": "कीवर्ड",
|
||||
"advanced": "फ़िल्टर",
|
||||
"recursive": "सबफ़ोल्डर शामिल करें",
|
||||
"results_count": "{{n}} परिणाम",
|
||||
"clear_keyword": "खोज साफ़ करें"
|
||||
},
|
||||
"files": {
|
||||
"name": "नाम",
|
||||
"type": "प्रकार",
|
||||
|
||||
@@ -416,6 +416,14 @@
|
||||
"shared_notificationSent": "Notifica inviata con successo",
|
||||
"shared_notificationFailed": "Impossibile inviare la notifica"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "Cerca in questa cartella e nelle sottocartelle…",
|
||||
"keyword": "Parola chiave",
|
||||
"advanced": "Filtri",
|
||||
"recursive": "Includi sottocartelle",
|
||||
"results_count": "{{n}} risultati",
|
||||
"clear_keyword": "Cancella ricerca"
|
||||
},
|
||||
"files": {
|
||||
"name": "Nome",
|
||||
"type": "Tipo",
|
||||
|
||||
@@ -416,6 +416,14 @@
|
||||
"shared_notificationSent": "通知が正常に送信されました",
|
||||
"shared_notificationFailed": "通知の送信に失敗しました"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "このフォルダとサブフォルダを検索…",
|
||||
"keyword": "キーワード",
|
||||
"advanced": "フィルター",
|
||||
"recursive": "サブフォルダを含める",
|
||||
"results_count": "{{n}} 件の結果",
|
||||
"clear_keyword": "検索をクリア"
|
||||
},
|
||||
"files": {
|
||||
"name": "名前",
|
||||
"type": "種類",
|
||||
|
||||
@@ -416,6 +416,14 @@
|
||||
"shared_notificationSent": "알림이 성공적으로 전송되었습니다",
|
||||
"shared_notificationFailed": "알림 전송에 실패했습니다"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "이 폴더와 하위 폴더 검색…",
|
||||
"keyword": "키워드",
|
||||
"advanced": "필터",
|
||||
"recursive": "하위 폴더 포함",
|
||||
"results_count": "{{n}}개 결과",
|
||||
"clear_keyword": "검색 지우기"
|
||||
},
|
||||
"files": {
|
||||
"name": "이름",
|
||||
"type": "유형",
|
||||
|
||||
@@ -416,6 +416,14 @@
|
||||
"shared_notificationSent": "Notificatie verzonden",
|
||||
"shared_notificationFailed": "Notificatie verzenden mislukt"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "Deze map en submappen doorzoeken…",
|
||||
"keyword": "Trefwoord",
|
||||
"advanced": "Filters",
|
||||
"recursive": "Submappen opnemen",
|
||||
"results_count": "{{n}} resultaten",
|
||||
"clear_keyword": "Zoekopdracht wissen"
|
||||
},
|
||||
"files": {
|
||||
"name": "Naam",
|
||||
"type": "Type",
|
||||
|
||||
@@ -416,6 +416,14 @@
|
||||
"shared_notificationSent": "Powiadomienie wysłane pomyślnie",
|
||||
"shared_notificationFailed": "Nie udało się wysłać powiadomienia"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "Szukaj w tym folderze i podfolderach…",
|
||||
"keyword": "Słowo kluczowe",
|
||||
"advanced": "Filtry",
|
||||
"recursive": "Uwzględnij podfoldery",
|
||||
"results_count": "Wyniki: {{n}}",
|
||||
"clear_keyword": "Wyczyść wyszukiwanie"
|
||||
},
|
||||
"files": {
|
||||
"name": "Nazwa",
|
||||
"type": "Typ",
|
||||
|
||||
@@ -416,6 +416,14 @@
|
||||
"shared_notificationSent": "Notificação enviada com sucesso",
|
||||
"shared_notificationFailed": "Falha ao enviar a notificação"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "Pesquisar nesta pasta e subpastas…",
|
||||
"keyword": "Palavra-chave",
|
||||
"advanced": "Filtros",
|
||||
"recursive": "Incluir subpastas",
|
||||
"results_count": "{{n}} resultados",
|
||||
"clear_keyword": "Limpar pesquisa"
|
||||
},
|
||||
"files": {
|
||||
"name": "Nome",
|
||||
"type": "Tipo",
|
||||
|
||||
@@ -416,6 +416,14 @@
|
||||
"shared_notificationSent": "Уведомление успешно отправлено",
|
||||
"shared_notificationFailed": "Не удалось отправить уведомление"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "Поиск в этой папке и подпапках…",
|
||||
"keyword": "Ключевое слово",
|
||||
"advanced": "Фильтры",
|
||||
"recursive": "Включая подпапки",
|
||||
"results_count": "Результатов: {{n}}",
|
||||
"clear_keyword": "Очистить поиск"
|
||||
},
|
||||
"files": {
|
||||
"name": "Имя",
|
||||
"type": "Тип",
|
||||
|
||||
@@ -416,11 +416,20 @@
|
||||
"shared_typeFile": "檔案",
|
||||
"shared_typeFolder": "資料夾"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "搜尋此資料夾及子資料夾…",
|
||||
"keyword": "關鍵字",
|
||||
"advanced": "篩選",
|
||||
"recursive": "包含子資料夾",
|
||||
"results_count": "{{n}} 個結果",
|
||||
"clear_keyword": "清除搜尋"
|
||||
},
|
||||
"files": {
|
||||
"name": "名稱",
|
||||
"type": "型別",
|
||||
"size": "大小",
|
||||
"modified": "修改日期",
|
||||
"folder_stat": "{{folders}} 個資料夾 · {{files}} 個檔案",
|
||||
"no_files": "此資料夾中沒有檔案",
|
||||
"empty_hint": "上傳檔案或建立資料夾以開始使用",
|
||||
"drop_to_upload": "將檔案拖放到此處上傳",
|
||||
@@ -1598,6 +1607,7 @@
|
||||
"videos": "影片"
|
||||
},
|
||||
"common": {
|
||||
"refresh": "重新整理",
|
||||
"add": "新增",
|
||||
"cancel": "取消",
|
||||
"clear": "清除",
|
||||
|
||||
@@ -416,11 +416,20 @@
|
||||
"shared_typeFile": "文件",
|
||||
"shared_typeFolder": "文件夹"
|
||||
},
|
||||
"filter": {
|
||||
"placeholder": "搜索此文件夹及子文件夹…",
|
||||
"keyword": "关键词",
|
||||
"advanced": "筛选",
|
||||
"recursive": "包含子文件夹",
|
||||
"results_count": "{{n}} 个结果",
|
||||
"clear_keyword": "清除搜索"
|
||||
},
|
||||
"files": {
|
||||
"name": "名称",
|
||||
"type": "类型",
|
||||
"size": "大小",
|
||||
"modified": "修改日期",
|
||||
"folder_stat": "{{folders}} 个文件夹 · {{files}} 个文件",
|
||||
"no_files": "此文件夹中没有文件",
|
||||
"empty_hint": "上传文件或创建文件夹以开始使用",
|
||||
"drop_to_upload": "将文件拖放到此处上传",
|
||||
@@ -1598,6 +1607,7 @@
|
||||
"videos": "视频"
|
||||
},
|
||||
"common": {
|
||||
"refresh": "刷新",
|
||||
"add": "添加",
|
||||
"cancel": "取消",
|
||||
"clear": "清除",
|
||||
|
||||
@@ -15,6 +15,15 @@ Non-obvious rules that trip up new code. Terse on purpose.
|
||||
- Any new endpoint that mints or consumes credentials/tokens must consult one of the `is_*_login_allowed()` helpers, not the raw allowlist.
|
||||
- Any new "policy-disabled" refusal must emit an `audit`-target line before returning — matches `auth.login_rejected`, `magic_link.redemption_rejected` conventions.
|
||||
|
||||
## AuthZ enforcement points
|
||||
|
||||
- **The extractors are NOT a choke point.** Four paths authenticate without ever touching `AuthUser` / `CurrentUserId`: `middleware/admin.rs::require_authenticated` (re-parses the JWT from header *or* cookie itself), the three DAV handlers' hand-rolled `extract_user` (`webdav_handler.rs:154`, `caldav_handler.rs:421`, `carddav_handler.rs:173`), `POST /api/auth/refresh` (mounted outside `auth_middleware`, `main.rs:775`), and `GET /api/rt/ws` (self-auths from a raw Bearer and never reads `claims.role`, `rt_ws.rs:240-255`). A rule added to an extractor does not hold until it is added to these too.
|
||||
- **Never hand-roll principal extraction.** `req.extensions().get::<Arc<CurrentUser>>()` inside a handler is exactly the anti-pattern above — it bypasses every `FromRequestParts` guard. Take the extractor, or call the shared assertion.
|
||||
- **A method check is not an authorization check.** GETs that mint credentials or write exist: `GET /api/wopi/editor-url` returns a WOPI token usable for `POST /wopi/files/{id}/contents`; `GET /api/s/{token}` writes via `register_shared_link_access`; `GET /api/auth/device/verify` is an oracle on live device codes; `GET /api/batch/download` builds an arbitrary ZIP from a querystring. Never gate on verb alone.
|
||||
- **An auth helper's `_ =>` arm must DENY.** Two fail-open gates exist and are bugs, not patterns to copy: `require_internal_user` (`middleware/user.rs:64-71`) admits the caller on *any* `get_user_flags` error, and `decide_live_role` (`:169-176`) resurrects the claim role on a transient DB error. The first is the only middleware guarding all three DAV surfaces.
|
||||
- **Prefer deny-by-default over assert-in-handler.** A restriction enforced inside the extractor covers ~200 call sites with no edits; the same restriction as "handler takes an optional principal and asserts" is one forgotten call away from silently accepting. `OptionalUserId` (`middleware/auth.rs:85-98`) is the cautionary tale — it exists, it is dead code, and nothing ever used it.
|
||||
- Anonymous-session direction (share links as a principal): `docs/plan/rationalize-publicshare.md`.
|
||||
|
||||
## Storage backend access
|
||||
|
||||
- **Read blob content through `Arc<DedupService>`.** It's the ONE canonical read abstraction — CDC-manifest-aware (`file.blob_hash` may reference a chunk manifest, not a blob), backend-agnostic (Local/S3/Azure), wrapper-transparent (encryption/retry/cache). Never take `Arc<dyn BlobStorageBackend>` directly in a service that reads content; you'll silently break on any file ≥ 64 KiB (`CDC_MIN_CHUNK`). Follow `thumbnail_service`, `audio_metadata_service`, `media_metadata_service`, `face_indexing_service`, `search_index::content_index_worker` as reference impls.
|
||||
@@ -30,3 +39,9 @@ Non-obvious rules that trip up new code. Terse on purpose.
|
||||
- **After adding: `cargo run --bin generate-openapi`** to regenerate `resources/gen/openapi.json`, then `git diff resources/gen/openapi.json` — the new path + its request/response schemas must be present. Zero-diff means you missed the registration.
|
||||
- Sanity check for the whole surface: `diff <(grep -oE 'path = "/api[^"]+"' src/interfaces/api/handlers/*.rs | grep -oE '/api[^"]+' | sort -u) <(jq -r '.paths | keys | .[]' resources/gen/openapi.json | sort -u)` — should always be empty. Non-empty diff = drift.
|
||||
- Handlers referenced by the `paths(...)` list MUST be `pub` (module-visible from the paths list). Private `async fn` compiles at the router mount but breaks the paths list with a visibility error — see `get_smtp_info`, `send_smtp_test`, `get_user_profile` for the retrofit.
|
||||
|
||||
### Security / scope in the spec
|
||||
|
||||
- **`security(("bearerAuth" = []))` — the empty array is the SCOPES list**, not decoration. Every route currently declares the same thing, so the spec claims "a session is required" for `GET /api/version` and `PUT /api/admin/users/{id}/role` alike: true, and useless. If a route's gate differs from the default, declare it there.
|
||||
- OpenAPI has **no field for a minimum role** — OAuth2 has no role concept, so the spec has nowhere to put one. Use a pseudo-scope (`["role:admin"]`) rather than a vendor extension no tooling renders.
|
||||
- **Declaring is not enforcing.** utoipa's `security` wires nothing, so it drifts from the real gate silently. Any scope worth declaring is worth a test cross-checking it against the actual mount — otherwise the spec becomes a parallel description of the authorization boundary rather than a picture of it.
|
||||
|
||||
@@ -16,6 +16,15 @@ pub struct ShareDto {
|
||||
pub created_at: u64,
|
||||
pub created_by: String,
|
||||
pub access_count: u64,
|
||||
/// File shares only: the shared file's MIME type, resolved at read time
|
||||
/// so anonymous viewers can render an inline media preview (video player
|
||||
/// / image) instead of a bare download button. Absent for folder shares
|
||||
/// and whenever the file lookup fails (display-only enrichment).
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub mime_type: Option<String>,
|
||||
/// File shares only: the shared file's size in bytes (see `mime_type`).
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub size: Option<u64>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, ToSchema)]
|
||||
@@ -49,6 +58,8 @@ impl ShareDto {
|
||||
created_at: share.created_at(),
|
||||
created_by: share.created_by().to_string(),
|
||||
access_count: share.access_count(),
|
||||
mime_type: None,
|
||||
size: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3425,34 +3425,37 @@ impl AuthApplicationService {
|
||||
&self,
|
||||
dto: crate::application::dtos::settings_dto::AdminCreateUserDto,
|
||||
) -> Result<FullUserDto, DomainError> {
|
||||
// Validate username length
|
||||
if dto.username.len() < 3 || dto.username.len() > 254 {
|
||||
return Err(DomainError::new(
|
||||
ErrorKind::InvalidInput,
|
||||
"User",
|
||||
"Username must be between 3 and 254 characters".to_string(),
|
||||
));
|
||||
}
|
||||
// Normalise the username up-front — trim + lowercase — and use
|
||||
// the canonical form for every downstream check + generated
|
||||
// value below. `User::new` also normalises internally, but the
|
||||
// placeholder-email fallback and the duplicate-check error
|
||||
// message live above that call, so they'd otherwise capture the
|
||||
// raw wire input (e.g. `UpperCase@oxicloud.local`).
|
||||
// See docs/plan/username-lowercase.md § Design decision 5.
|
||||
let username = User::validate_username(&dto.username).map_err(|e| {
|
||||
DomainError::new(ErrorKind::InvalidInput, "User", format!("Username: {e}"))
|
||||
})?;
|
||||
|
||||
// Check for duplicate username
|
||||
if self
|
||||
.user_storage
|
||||
.get_user_by_username(&dto.username)
|
||||
.get_user_by_username(&username)
|
||||
.await
|
||||
.is_ok()
|
||||
{
|
||||
return Err(DomainError::new(
|
||||
ErrorKind::AlreadyExists,
|
||||
"User",
|
||||
format!("User '{}' already exists", dto.username),
|
||||
format!("User '{username}' already exists"),
|
||||
));
|
||||
}
|
||||
|
||||
// Email: use provided or generate placeholder
|
||||
// Email: use provided or generate placeholder from the
|
||||
// canonical (lowercase) username.
|
||||
let email = dto
|
||||
.email
|
||||
.filter(|e| !e.trim().is_empty())
|
||||
.unwrap_or_else(|| format!("{}@oxicloud.local", dto.username));
|
||||
.unwrap_or_else(|| format!("{username}@oxicloud.local"));
|
||||
|
||||
// Check email uniqueness
|
||||
if self.user_storage.get_user_by_email(&email).await.is_ok() {
|
||||
@@ -3519,7 +3522,7 @@ impl AuthApplicationService {
|
||||
let user = if is_external {
|
||||
User::new(
|
||||
email,
|
||||
Some(dto.username.clone()),
|
||||
Some(username.clone()),
|
||||
Some(password_hash),
|
||||
None, // federation_kind: admin-created external, no federation link yet
|
||||
None, // federation_issuer
|
||||
@@ -3531,7 +3534,7 @@ impl AuthApplicationService {
|
||||
} else {
|
||||
User::new(
|
||||
email,
|
||||
Some(dto.username.clone()),
|
||||
Some(username.clone()),
|
||||
Some(password_hash),
|
||||
None, // federation_kind: admin-created local user
|
||||
None, // federation_issuer
|
||||
@@ -3750,8 +3753,33 @@ impl AuthApplicationService {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Activate or deactivate a user (admin only)
|
||||
/// Activate or deactivate a user (admin only).
|
||||
///
|
||||
/// **Reactivation collision handling** — when a deactivated account
|
||||
/// holds a mixed-case username from before the lowercase-usernames
|
||||
/// migration (its row was skipped by that migration precisely
|
||||
/// because it was deactivated), reactivating it can produce a
|
||||
/// username collision if `LOWER(other.username) == LOWER(this.username)`
|
||||
/// for an active row. We resolve the collision by:
|
||||
///
|
||||
/// 1. Re-normalising via `User::set_username` (returns the
|
||||
/// canonical lowercase form on success).
|
||||
/// 2. If the canonical form is already taken by another active
|
||||
/// row, probe `<lower>-2`, `-3`, … via the shared
|
||||
/// `find_free_username_suffix` helper.
|
||||
/// 3. Persist the resolved name BEFORE flipping `active = true`
|
||||
/// so no time window has two-active-users with the same
|
||||
/// lowercase form.
|
||||
///
|
||||
/// See `docs/plan/username-lowercase.md § Design decision 7`.
|
||||
/// Without this, un-soft-deleting the only pre-migration
|
||||
/// mixed-case survivor would refuse-to-boot on the next restart.
|
||||
pub async fn set_user_active(&self, user_id: Uuid, active: bool) -> Result<(), DomainError> {
|
||||
// Only the activate direction needs the collision-resolution
|
||||
// dance — deactivation just flips a bit.
|
||||
if active {
|
||||
self.resolve_reactivation_collision(user_id).await?;
|
||||
}
|
||||
self.user_storage
|
||||
.set_user_active_status(user_id, active)
|
||||
.await?;
|
||||
@@ -3759,6 +3787,93 @@ impl AuthApplicationService {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Pre-flight for `set_user_active(active = true)`: ensures the
|
||||
/// target user's username is canonical (lowercase) AND unique
|
||||
/// against currently-active accounts. Renames the target row if
|
||||
/// either invariant would break.
|
||||
///
|
||||
/// NULL usernames (OPAQUE-migrated accounts) are a no-op — nothing
|
||||
/// to normalise, nothing to collide.
|
||||
async fn resolve_reactivation_collision(&self, user_id: Uuid) -> Result<(), DomainError> {
|
||||
let target = self.user_storage.get_user_by_id(user_id).await?;
|
||||
let Some(current) = target.username().map(str::to_string) else {
|
||||
return Ok(());
|
||||
};
|
||||
|
||||
let canonical = current.to_ascii_lowercase();
|
||||
|
||||
// Look for another ACTIVE user holding the canonical form.
|
||||
// The migration CLI's `find_free_username_suffix` probes
|
||||
// directly via a pool; here we don't have the pool
|
||||
// (`AuthApplicationService` holds a `dyn UserRepository`
|
||||
// trait object). Use `get_user_by_username` — the repo
|
||||
// normalises input to lowercase before bind, so this
|
||||
// resolves against the canonical row.
|
||||
let collision = self
|
||||
.user_storage
|
||||
.get_user_by_username(&canonical)
|
||||
.await
|
||||
.ok()
|
||||
.filter(|other| other.id() != user_id && other.is_active());
|
||||
|
||||
let chosen_name = match collision {
|
||||
None => canonical,
|
||||
Some(_) => {
|
||||
// Collision — probe `<canonical>-2`, `-3`, … via
|
||||
// repository lookups. Same shape as the migration
|
||||
// CLI's `find_free_username_suffix`, just against
|
||||
// the repo trait instead of a raw pool. Both paths
|
||||
// agree by construction on the numbering scheme.
|
||||
//
|
||||
// Cap at 10_000 (matches the shared helper's cap —
|
||||
// see `docs/plan/username-lowercase.md § 3. Suffix-
|
||||
// collision robustness`). Reaching the cap means
|
||||
// the account universe has an anomaly worth
|
||||
// investigating; loud abort beats silent truncation.
|
||||
const SUFFIX_PROBE_CAP: i32 = 10_000;
|
||||
let mut chosen: Option<String> = None;
|
||||
for n in 2..=SUFFIX_PROBE_CAP {
|
||||
let candidate = format!("{canonical}-{n}");
|
||||
match self.user_storage.get_user_by_username(&candidate).await {
|
||||
Ok(_) => continue,
|
||||
Err(_) => {
|
||||
chosen = Some(candidate);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
let suffixed = chosen.ok_or_else(|| {
|
||||
DomainError::internal_error(
|
||||
"User",
|
||||
format!(
|
||||
"reactivation-collision suffix probe exhausted \
|
||||
{SUFFIX_PROBE_CAP} candidates for base '{canonical}'"
|
||||
),
|
||||
)
|
||||
})?;
|
||||
tracing::info!(
|
||||
target: "audit",
|
||||
event = "user.reactivation_renamed",
|
||||
reason = "collision_with_active",
|
||||
target_id = %user_id,
|
||||
from = %current,
|
||||
to = %suffixed,
|
||||
"🔄 user reactivation renamed to avoid username collision",
|
||||
);
|
||||
suffixed
|
||||
}
|
||||
};
|
||||
|
||||
if target.username() != Some(chosen_name.as_str()) {
|
||||
let mut renamed = target;
|
||||
renamed
|
||||
.set_username(chosen_name)
|
||||
.map_err(|e| DomainError::internal_error("User", format!("set_username: {e}")))?;
|
||||
self.user_storage.update_user(renamed).await?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Change user role (admin only).
|
||||
///
|
||||
/// Refuses `role = "admin"` when the target is external (grant-only).
|
||||
@@ -4435,10 +4550,14 @@ impl AuthApplicationService {
|
||||
.clone()
|
||||
.or(claims.name.clone())
|
||||
.unwrap_or_else(|| format!("oidc_{}", &claims.sub[..8.min(claims.sub.len())]));
|
||||
// Placeholder-email fallback when the IdP omits `email` from
|
||||
// the claim set. Lowercase the local-part so the fake address
|
||||
// matches the storage convention for other placeholder-email
|
||||
// paths (see `admin_create_user`'s `<username>@oxicloud.local`).
|
||||
let oidc_email = claims
|
||||
.email
|
||||
.clone()
|
||||
.unwrap_or_else(|| format!("{}@oidc.local", oidc_username));
|
||||
.unwrap_or_else(|| format!("{}@oidc.local", oidc_username.to_ascii_lowercase()));
|
||||
|
||||
// 5. Look up existing user by OIDC subject.
|
||||
//
|
||||
@@ -4652,16 +4771,30 @@ impl AuthApplicationService {
|
||||
&oidc_username
|
||||
};
|
||||
|
||||
// Filter to valid username characters only, then truncate to 32 chars
|
||||
// Lowercase at JIT derivation. `validate_username` in
|
||||
// `User::new` would lowercase too, but the collision
|
||||
// check below (`get_user_by_username`) needs the
|
||||
// canonical form BEFORE `User::new` is called —
|
||||
// otherwise `Alice` from an IdP claim would look
|
||||
// "free" against an `alice` row on the first pass
|
||||
// and fail the DB unique constraint at INSERT time.
|
||||
// See `docs/plan/username-lowercase.md § 2. OIDC JIT
|
||||
// derivation`.
|
||||
//
|
||||
// ASCII-only by the char-filter below, so
|
||||
// `to_ascii_lowercase()` is deterministic and
|
||||
// locale-safe.
|
||||
let mut username = base_username
|
||||
.chars()
|
||||
.filter(|c| {
|
||||
c.is_ascii_alphanumeric() || *c == '-' || *c == '_' || *c == '.'
|
||||
})
|
||||
.take(32)
|
||||
.collect::<String>();
|
||||
.collect::<String>()
|
||||
.to_ascii_lowercase();
|
||||
|
||||
// Filter helper: removes any chars that are not valid in a username
|
||||
// Filter helper: removes any chars that are not valid in a username.
|
||||
// Lowercases too so the collision-suffix path below writes canonical form.
|
||||
let filter_username_chars = |s: &str| {
|
||||
s.chars()
|
||||
.filter(|c| {
|
||||
@@ -4669,6 +4802,7 @@ impl AuthApplicationService {
|
||||
})
|
||||
.take(32)
|
||||
.collect::<String>()
|
||||
.to_ascii_lowercase()
|
||||
};
|
||||
|
||||
// Ensure minimum length (the padding suffix must also be filtered)
|
||||
|
||||
@@ -116,19 +116,35 @@ impl ShareBrowseService {
|
||||
self.list_inner(folder_id, resolved.owner_id).await
|
||||
}
|
||||
|
||||
/// AuthZ gate for `/api/s/{token}/file/{file_id}`: the requested file
|
||||
/// must either BE the shared item (single-file share — the public landing
|
||||
/// page's inline media preview streams through here) or live inside the
|
||||
/// shared folder's subtree (folder share). Anything else is NotFound —
|
||||
/// the same shape as "file doesn't exist", so the endpoint can't be used
|
||||
/// to enumerate file ids.
|
||||
pub async fn assert_file_in_share(
|
||||
&self,
|
||||
token: &str,
|
||||
file_id: &str,
|
||||
unlock_jwt: Option<&str>,
|
||||
) -> Result<(), DomainError> {
|
||||
let resolved = self.resolve_folder_share(token, unlock_jwt).await?;
|
||||
let share = self
|
||||
.share_service
|
||||
.get_shared_link_with_unlock(token, unlock_jwt)
|
||||
.await?;
|
||||
|
||||
if !self
|
||||
.folder_repo
|
||||
.is_file_in_subtree(file_id, &resolved.root_folder_id)
|
||||
let in_scope = match share.item_type.as_str() {
|
||||
// Single-file share: only the shared item itself may be streamed.
|
||||
"file" => file_id == share.item_id,
|
||||
// Folder share: the file must live in the shared subtree.
|
||||
"folder" => {
|
||||
self.folder_repo
|
||||
.is_file_in_subtree(file_id, &share.item_id)
|
||||
.await?
|
||||
{
|
||||
}
|
||||
_ => false,
|
||||
};
|
||||
if !in_scope {
|
||||
return Err(DomainError::not_found("File", file_id));
|
||||
}
|
||||
Ok(())
|
||||
|
||||
@@ -249,6 +249,18 @@ impl ShareService {
|
||||
};
|
||||
self.fetch_share_resolved(token, unlocked).await
|
||||
}
|
||||
|
||||
/// Public share landing payload: share metadata enriched with the shared
|
||||
/// file's `mime_type` + `size` so anonymous viewers get an inline media
|
||||
/// preview (video player / image) instead of a bare download button.
|
||||
pub async fn get_shared_link_meta_with_unlock(
|
||||
&self,
|
||||
token: &str,
|
||||
unlock_jwt: Option<&str>,
|
||||
) -> Result<ShareDto, DomainError> {
|
||||
let dto = self.get_shared_link_with_unlock(token, unlock_jwt).await?;
|
||||
Ok(enrich_share_dto_with_file_info(dto, self.file_repository.as_ref()).await)
|
||||
}
|
||||
}
|
||||
|
||||
impl ShareUseCase for ShareService {
|
||||
@@ -538,7 +550,8 @@ impl ShareUseCase for ShareService {
|
||||
}
|
||||
|
||||
// Password verified (or not required) — return full share metadata
|
||||
Ok(ShareDto::from_entity(&share, &self.base_url))
|
||||
let dto = ShareDto::from_entity(&share, &self.base_url);
|
||||
Ok(enrich_share_dto_with_file_info(dto, self.file_repository.as_ref()).await)
|
||||
}
|
||||
|
||||
async fn register_shared_link_access(&self, token: &str) -> Result<(), DomainError> {
|
||||
@@ -558,6 +571,31 @@ impl ShareUseCase for ShareService {
|
||||
}
|
||||
}
|
||||
|
||||
/// Fill `mime_type`/`size` on file-share DTOs so anonymous viewers get an
|
||||
/// inline media preview (video player / image) on the public share page
|
||||
/// instead of a bare download button.
|
||||
///
|
||||
/// Display-only enrichment that never fails the response: a failed file
|
||||
/// lookup (transient DB error, race with a delete) leaves the fields `None`
|
||||
/// and the download endpoint surfaces the real error — a read failure is
|
||||
/// never proof the data is absent. Folder shares pass through untouched.
|
||||
///
|
||||
/// A free function (not a method) so the `integration_tests` mirror of the
|
||||
/// service exercises the exact same logic instead of re-implementing it.
|
||||
async fn enrich_share_dto_with_file_info<FR: FileReadPort>(
|
||||
mut dto: ShareDto,
|
||||
file_repository: &FR,
|
||||
) -> ShareDto {
|
||||
if dto.item_type != "file" {
|
||||
return dto;
|
||||
}
|
||||
if let Ok(file) = file_repository.get_file(&dto.item_id).await {
|
||||
dto.mime_type = Some(file.mime_type().to_string());
|
||||
dto.size = Some(file.size());
|
||||
}
|
||||
dto
|
||||
}
|
||||
|
||||
#[cfg(feature = "integration_tests")]
|
||||
#[allow(dead_code)]
|
||||
mod tests {
|
||||
@@ -642,6 +680,18 @@ mod tests {
|
||||
})?;
|
||||
self.password_hasher.hash_password(password).await
|
||||
}
|
||||
|
||||
/// Mirror of `ShareService::get_shared_link_meta_with_unlock` —
|
||||
/// fetch by token (the mirror has no unlock-JWT machinery) plus the
|
||||
/// shared file-info enrichment.
|
||||
async fn get_shared_link_meta_with_unlock(
|
||||
&self,
|
||||
token: &str,
|
||||
_unlock_jwt: Option<&str>,
|
||||
) -> Result<ShareDto, DomainError> {
|
||||
let dto = self.get_shared_link_by_token(token).await?;
|
||||
Ok(enrich_share_dto_with_file_info(dto, self.file_repository.as_ref()).await)
|
||||
}
|
||||
}
|
||||
|
||||
impl<SR, FR, FoR, PH> ShareUseCase for ShareServiceForTest<SR, FR, FoR, PH>
|
||||
@@ -1261,4 +1311,75 @@ mod tests {
|
||||
assert!(share_dto.has_password);
|
||||
assert!(share_dto.url.starts_with("http://127.0.0.1:8086/s/"));
|
||||
}
|
||||
|
||||
/// The share-landing meta endpoint enriches file shares with the shared
|
||||
/// file's mime type + size so anonymous viewers can render an inline
|
||||
/// preview (video player / image) instead of a bare download button.
|
||||
#[tokio::test]
|
||||
async fn test_get_shared_link_meta_enriches_file_shares() {
|
||||
let config = Arc::new(AppConfig::default());
|
||||
let service = ShareServiceForTest::new(
|
||||
config,
|
||||
Arc::new(MockShareRepository::new()),
|
||||
Arc::new(MockFileRepository),
|
||||
Arc::new(MockFolderRepository),
|
||||
Arc::new(MockPasswordHasher),
|
||||
);
|
||||
|
||||
let share = service
|
||||
.create_shared_link(
|
||||
Uuid::new_v4(),
|
||||
CreateShareDto {
|
||||
item_id: "test_file_id".to_string(),
|
||||
item_name: Some("movie.mp4".to_string()),
|
||||
item_type: "file".to_string(),
|
||||
password: None,
|
||||
expires_at: None,
|
||||
},
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let meta = service
|
||||
.get_shared_link_meta_with_unlock(&share.token, None)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(meta.mime_type.as_deref(), Some("text/plain"));
|
||||
assert_eq!(meta.size, Some(123));
|
||||
}
|
||||
|
||||
/// Folder shares must NOT gain a bogus mime type — the enrichment is a
|
||||
/// file-share-only passthrough for them.
|
||||
#[tokio::test]
|
||||
async fn test_get_shared_link_meta_leaves_folder_shares_unenriched() {
|
||||
let config = Arc::new(AppConfig::default());
|
||||
let service = ShareServiceForTest::new(
|
||||
config,
|
||||
Arc::new(MockShareRepository::new()),
|
||||
Arc::new(MockFileRepository),
|
||||
Arc::new(MockFolderRepository),
|
||||
Arc::new(MockPasswordHasher),
|
||||
);
|
||||
|
||||
let share = service
|
||||
.create_shared_link(
|
||||
Uuid::new_v4(),
|
||||
CreateShareDto {
|
||||
item_id: "test_folder_id".to_string(),
|
||||
item_name: Some("pictures".to_string()),
|
||||
item_type: "folder".to_string(),
|
||||
password: None,
|
||||
expires_at: None,
|
||||
},
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let meta = service
|
||||
.get_shared_link_meta_with_unlock(&share.token, None)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(meta.mime_type, None);
|
||||
assert_eq!(meta.size, None);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -125,10 +125,16 @@ impl StorageUsageService {
|
||||
}
|
||||
|
||||
/// Same as [`Self::update_user_storage_usage`], keyed by username.
|
||||
///
|
||||
/// Lowercases input before bind — same rule as
|
||||
/// `UserRepository::get_user_by_username`. Usernames are canonical
|
||||
/// (lowercase) in the DB post-migration; callers may pass any case.
|
||||
/// See `docs/plan/username-lowercase.md`.
|
||||
pub async fn update_user_storage_usage_by_username(
|
||||
&self,
|
||||
username: &str,
|
||||
) -> Result<i64, DomainError> {
|
||||
let username = username.trim().to_ascii_lowercase();
|
||||
let total_usage: Option<i64> = sqlx::query_scalar(
|
||||
r#"
|
||||
UPDATE auth.users u
|
||||
@@ -146,7 +152,7 @@ impl StorageUsageService {
|
||||
RETURNING u.storage_used_bytes
|
||||
"#,
|
||||
)
|
||||
.bind(username)
|
||||
.bind(&username)
|
||||
.fetch_optional(self.pool.as_ref())
|
||||
.await
|
||||
.map_err(|e| {
|
||||
|
||||
@@ -82,11 +82,35 @@ pub enum Action {
|
||||
#[arg(long)]
|
||||
dry_run: bool,
|
||||
},
|
||||
|
||||
/// Lowercase every active user's username in `auth.users`.
|
||||
///
|
||||
/// Enforcement-companion for the case-insensitive-usernames
|
||||
/// migration (see `docs/plan/username-lowercase.md`). The server
|
||||
/// refuses to boot after upgrade until this has run. Data touched
|
||||
/// is `auth.users.username` only.
|
||||
///
|
||||
/// Collision handling: when `Alice` and `alice` both exist,
|
||||
/// tiebreak `(last_login_at DESC NULLS LAST, created_at ASC)` —
|
||||
/// the winner keeps the canonical lowercased name, losers get
|
||||
/// `<lowercase>-2`, `-3`, … via the shared
|
||||
/// [`common::username_migration::find_free_username_suffix`]
|
||||
/// probe. Sessions and grants survive the rename (they key on
|
||||
/// `user_id`).
|
||||
///
|
||||
/// Skipped: soft-deleted / disabled rows and rows where
|
||||
/// `username IS NULL` (OPAQUE-migrated accounts).
|
||||
LowercaseUsernames {
|
||||
/// Print what would change without touching the DB.
|
||||
#[arg(long)]
|
||||
dry_run: bool,
|
||||
},
|
||||
}
|
||||
|
||||
pub async fn run(action: Action) -> u8 {
|
||||
match action {
|
||||
Action::NfcFilenames { dry_run } => run_nfc_filenames(dry_run).await,
|
||||
Action::LowercaseUsernames { dry_run } => run_lowercase_usernames(dry_run).await,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -758,3 +782,293 @@ async fn run_folders(pool: &PgPool, dry_run: bool, stats: &mut Stats) -> Result<
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// ════════════════════════════════════════════════════════════════════════════
|
||||
// lowercase-usernames
|
||||
// ════════════════════════════════════════════════════════════════════════════
|
||||
|
||||
#[derive(Default)]
|
||||
struct UsernameStats {
|
||||
scanned: u64,
|
||||
already_lowercase: u64,
|
||||
normalized_in_place: u64,
|
||||
/// Multi-member `LOWER(username)` group where the tiebreak
|
||||
/// winner kept the canonical name.
|
||||
collision_winners: u64,
|
||||
/// Multi-member losers renamed to `<lowercase>-N`.
|
||||
renamed_to_suffix: u64,
|
||||
/// Rows the scan touched but the loop declined to modify. Today
|
||||
/// this is inactive rows (soft-deleted / admin-disabled). NULL
|
||||
/// usernames never enter the scan so they don't contribute here.
|
||||
skipped: u64,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct UsernameRow {
|
||||
id: Uuid,
|
||||
username: String,
|
||||
/// Inactive rows (soft-deleted / admin-disabled) are read but not
|
||||
/// modified — normalising a name we can't reach anyway risks
|
||||
/// creating a `<lower>-N` conflict with a future re-activation of
|
||||
/// the same handle. The loop uses this flag to skip and count.
|
||||
active: bool,
|
||||
// Kept for the SQL row-shape roundtrip (the SELECT ordering
|
||||
// depends on them) even though the Rust-side grouping only
|
||||
// reads `id` and `username`. Marked `#[allow(dead_code)]`
|
||||
// so clippy doesn't nag; renaming to `_last_login_at` would
|
||||
// work too but the SQL column names are load-bearing for the
|
||||
// sqlx `Row::get` calls below.
|
||||
#[allow(dead_code)]
|
||||
last_login_at: Option<DateTime<Utc>>,
|
||||
#[allow(dead_code)]
|
||||
created_at: DateTime<Utc>,
|
||||
}
|
||||
|
||||
async fn run_lowercase_usernames(dry_run: bool) -> u8 {
|
||||
let database_url = match env::var("DATABASE_URL") {
|
||||
Ok(v) => v,
|
||||
Err(_) => {
|
||||
eprintln!("migrate lowercase-usernames: DATABASE_URL not set");
|
||||
return 2;
|
||||
}
|
||||
};
|
||||
|
||||
let pool = match PgPool::connect(&database_url).await {
|
||||
Ok(p) => p,
|
||||
Err(e) => {
|
||||
eprintln!("migrate lowercase-usernames: failed to connect: {e}");
|
||||
return 2;
|
||||
}
|
||||
};
|
||||
|
||||
if dry_run {
|
||||
println!("migrate lowercase-usernames: DRY RUN (no writes)");
|
||||
} else {
|
||||
println!("migrate lowercase-usernames: applying changes");
|
||||
}
|
||||
|
||||
// Load every user with a non-NULL username, active and inactive
|
||||
// alike. Inactive rows are surfaced (not filtered at scan-time)
|
||||
// so the `skipped (inactive)` counter can report them honestly —
|
||||
// an operator reading the summary sees "10 rows scanned, 2
|
||||
// inactive were passed on" instead of a phantom 0.
|
||||
//
|
||||
// NULL usernames stay out of the scan: there's nothing to
|
||||
// normalise for OPAQUE-migrated rows, and pulling them would
|
||||
// inflate `scanned` with rows the migration has no verb for.
|
||||
//
|
||||
// Ordering: alphabetic by `LOWER(username)` groups collisions
|
||||
// together, then the intra-group order is the tiebreak
|
||||
// (`last_login_at DESC NULLS LAST, created_at ASC` — most
|
||||
// recently active wins the canonical name).
|
||||
let rows: Vec<UsernameRow> = match sqlx::query(
|
||||
r#"
|
||||
SELECT id, username, active, last_login_at, created_at
|
||||
FROM auth.users
|
||||
WHERE username IS NOT NULL
|
||||
ORDER BY LOWER(username),
|
||||
(last_login_at IS NULL),
|
||||
last_login_at DESC NULLS LAST,
|
||||
created_at ASC
|
||||
"#,
|
||||
)
|
||||
.fetch_all(&pool)
|
||||
.await
|
||||
{
|
||||
Ok(rs) => rs
|
||||
.into_iter()
|
||||
.map(|r| UsernameRow {
|
||||
id: r.get::<Uuid, _>("id"),
|
||||
username: r.get::<String, _>("username"),
|
||||
active: r.get::<bool, _>("active"),
|
||||
last_login_at: r.try_get::<DateTime<Utc>, _>("last_login_at").ok(),
|
||||
created_at: r.get::<DateTime<Utc>, _>("created_at"),
|
||||
})
|
||||
.collect(),
|
||||
Err(e) => {
|
||||
eprintln!("migrate lowercase-usernames: initial scan failed: {e}");
|
||||
return 2;
|
||||
}
|
||||
};
|
||||
|
||||
let mut stats = UsernameStats::default();
|
||||
|
||||
// Group by `LOWER(username)`. Order preserved from the SQL query
|
||||
// → within a group, the FIRST row is the tiebreak winner.
|
||||
//
|
||||
// Inactive rows are filtered OUT of the grouping (not just
|
||||
// skipped inside the loop) so they can't create a phantom
|
||||
// collision with an active row sharing their lowercase form.
|
||||
// Example: inactive `Alice` + active `alice` would otherwise
|
||||
// look like a two-member group; filtering inactive first leaves
|
||||
// `alice` as a clean singleton no-op. The count goes to
|
||||
// `stats.skipped`, surfaced in the summary as `skipped (inactive)`.
|
||||
let mut groups: Vec<(String, Vec<UsernameRow>)> = Vec::new();
|
||||
for row in rows {
|
||||
stats.scanned += 1;
|
||||
if !row.active {
|
||||
stats.skipped += 1;
|
||||
continue;
|
||||
}
|
||||
let key = row.username.to_ascii_lowercase();
|
||||
match groups.last_mut() {
|
||||
Some((k, v)) if k == &key => v.push(row),
|
||||
_ => groups.push((key, vec![row])),
|
||||
}
|
||||
}
|
||||
|
||||
for (lower, members) in groups {
|
||||
if members.len() == 1 {
|
||||
let row = &members[0];
|
||||
if row.username == lower {
|
||||
stats.already_lowercase += 1;
|
||||
continue;
|
||||
}
|
||||
// Single-member group with a mixed-case name → straight
|
||||
// rename to the lowercase form. No collision.
|
||||
if !dry_run && let Err(e) = update_username(&pool, row.id, &lower).await {
|
||||
eprintln!(
|
||||
"migrate lowercase-usernames: UPDATE failed for {}: {e}",
|
||||
row.id
|
||||
);
|
||||
return 1;
|
||||
}
|
||||
println!(
|
||||
"NORMALIZE user={} '{}' → '{}'",
|
||||
row.id, row.username, lower
|
||||
);
|
||||
stats.normalized_in_place += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Multi-member group → collision. Members are already ordered
|
||||
// by the tiebreak. Winner takes the canonical lowercase name,
|
||||
// losers get `<lower>-2`, `-3`, … via the shared suffix helper.
|
||||
//
|
||||
// ORDER MATTERS: losers must be renamed FIRST. If we renamed
|
||||
// the winner to `<lower>` while a loser still holds that
|
||||
// exact name (the common case where the winner is mixed-case
|
||||
// and the loser is already-lowercase), the UNIQUE constraint
|
||||
// `users_username_key` fires. Freeing the canonical form by
|
||||
// suffixing every non-winner member first eliminates the
|
||||
// race entirely.
|
||||
let (winner, losers) = members.split_first().expect("non-empty by construction");
|
||||
|
||||
// Suffixes assigned inside this group during this run. Used
|
||||
// to keep dry-run consistent (no DB writes → the shared
|
||||
// suffix helper would hand the same probe back for every
|
||||
// loser). During apply, the DB itself deduplicates, but
|
||||
// tracking here keeps the two modes structurally identical.
|
||||
let mut reserved_this_group: Vec<String> = Vec::new();
|
||||
for loser in losers {
|
||||
let suffixed =
|
||||
match pick_free_suffix_avoiding(&pool, &lower, &reserved_this_group).await {
|
||||
Ok(s) => s,
|
||||
Err(e) => {
|
||||
eprintln!(
|
||||
"migrate lowercase-usernames: suffix probe failed for {}: {e}",
|
||||
loser.id
|
||||
);
|
||||
return 1;
|
||||
}
|
||||
};
|
||||
if !dry_run && let Err(e) = update_username(&pool, loser.id, &suffixed).await {
|
||||
eprintln!(
|
||||
"migrate lowercase-usernames: loser UPDATE failed for {}: {e}",
|
||||
loser.id
|
||||
);
|
||||
return 1;
|
||||
}
|
||||
println!(
|
||||
"RENAME user={} '{}' → '{}' (collision suffix)",
|
||||
loser.id, loser.username, suffixed
|
||||
);
|
||||
stats.renamed_to_suffix += 1;
|
||||
reserved_this_group.push(suffixed);
|
||||
}
|
||||
|
||||
if winner.username == lower {
|
||||
// Winner already holds the canonical name (a lowercase
|
||||
// row happened to be the most recently active; other
|
||||
// members are the ones needing renames).
|
||||
stats.already_lowercase += 1;
|
||||
} else {
|
||||
if !dry_run && let Err(e) = update_username(&pool, winner.id, &lower).await {
|
||||
eprintln!(
|
||||
"migrate lowercase-usernames: winner UPDATE failed for {}: {e}",
|
||||
winner.id
|
||||
);
|
||||
return 1;
|
||||
}
|
||||
println!(
|
||||
"NORMALIZE user={} '{}' → '{}' (collision winner)",
|
||||
winner.id, winner.username, lower
|
||||
);
|
||||
stats.collision_winners += 1;
|
||||
}
|
||||
}
|
||||
|
||||
println!();
|
||||
println!("Summary:");
|
||||
println!(" scanned: {}", stats.scanned);
|
||||
println!(" already-lowercase: {}", stats.already_lowercase);
|
||||
println!(" normalized in place: {}", stats.normalized_in_place);
|
||||
println!(" collision winners: {}", stats.collision_winners);
|
||||
println!(" renamed to suffix: {}", stats.renamed_to_suffix);
|
||||
println!(" skipped (inactive): {}", stats.skipped);
|
||||
if dry_run
|
||||
&& (stats.normalized_in_place + stats.collision_winners + stats.renamed_to_suffix) > 0
|
||||
{
|
||||
println!();
|
||||
println!("(dry-run — re-run without --dry-run to apply)");
|
||||
}
|
||||
0
|
||||
}
|
||||
|
||||
async fn update_username(pool: &PgPool, id: Uuid, new_name: &str) -> Result<(), sqlx::Error> {
|
||||
sqlx::query("UPDATE auth.users SET username = $1, updated_at = NOW() WHERE id = $2")
|
||||
.bind(new_name)
|
||||
.bind(id)
|
||||
.execute(pool)
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Pick the next free `<base>-<N>` suffix, skipping any suffix already
|
||||
/// reserved earlier in this migration run.
|
||||
///
|
||||
/// Wraps [`crate::common::username_migration::find_free_username_suffix`]
|
||||
/// with an additional local guard: in dry-run mode, no UPDATEs land so
|
||||
/// the DB probe would return the same suffix for every loser in a
|
||||
/// multi-loser group. The `reserved` slice lets the caller feed back
|
||||
/// the suffixes it has already announced, and the probe steps past
|
||||
/// them. In apply mode the DB probe alone would be enough (each real
|
||||
/// UPDATE moves the state forward), but the same code path keeps the
|
||||
/// two modes structurally identical.
|
||||
async fn pick_free_suffix_avoiding(
|
||||
pool: &PgPool,
|
||||
base: &str,
|
||||
reserved: &[String],
|
||||
) -> Result<String, sqlx::Error> {
|
||||
// Try the shared helper's default candidate first; if it collides
|
||||
// with a same-run reservation, increment past it and probe again.
|
||||
let mut n = 2;
|
||||
loop {
|
||||
let candidate = format!("{base}-{n}");
|
||||
let db_taken: (bool,) =
|
||||
sqlx::query_as("SELECT EXISTS(SELECT 1 FROM auth.users WHERE username = $1)")
|
||||
.bind(&candidate)
|
||||
.fetch_one(pool)
|
||||
.await?;
|
||||
let locally_taken = reserved.iter().any(|s| s == &candidate);
|
||||
if !db_taken.0 && !locally_taken {
|
||||
return Ok(candidate);
|
||||
}
|
||||
n += 1;
|
||||
if n > 10_000 {
|
||||
// Same cap as the shared helper — a loud panic beats a
|
||||
// silent truncation for an anomaly this rare.
|
||||
panic!("pick_free_suffix_avoiding: exhausted 10000 suffix probes for base '{base}'",);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+17
-4
@@ -128,8 +128,15 @@ async fn run_reset(user: Option<String>, all: bool, dry_run: bool) -> u8 {
|
||||
let rows_result = if all {
|
||||
sqlx::query(select_sql).fetch_all(&pool).await
|
||||
} else {
|
||||
let ident = user.as_deref().unwrap();
|
||||
sqlx::query(select_sql).bind(ident).fetch_all(&pool).await
|
||||
// Normalise before bind so the CLI accepts any case for
|
||||
// the username branch (email is already case-insensitive
|
||||
// via a functional index on LOWER(email); lowercasing here
|
||||
// for both branches is harmless — emails are lowercase
|
||||
// ASCII in `auth.users.email` too).
|
||||
// See `docs/plan/username-lowercase.md § 3. Lookup normalization`.
|
||||
let ident_raw = user.as_deref().unwrap();
|
||||
let ident = ident_raw.trim().to_ascii_lowercase();
|
||||
sqlx::query(select_sql).bind(&ident).fetch_all(&pool).await
|
||||
};
|
||||
let rows = match rows_result {
|
||||
Ok(r) => r,
|
||||
@@ -204,8 +211,14 @@ async fn run_reset(user: Option<String>, all: bool, dry_run: bool) -> u8 {
|
||||
let write_result = if all {
|
||||
sqlx::query(update_sql_all).execute(&pool).await
|
||||
} else {
|
||||
let ident = user.as_deref().unwrap();
|
||||
sqlx::query(update_sql_one).bind(ident).execute(&pool).await
|
||||
// Same normalisation as the read path above — usernames are
|
||||
// canonical lowercase in the DB. See docs/plan/username-lowercase.md.
|
||||
let ident_raw = user.as_deref().unwrap();
|
||||
let ident = ident_raw.trim().to_ascii_lowercase();
|
||||
sqlx::query(update_sql_one)
|
||||
.bind(&ident)
|
||||
.execute(&pool)
|
||||
.await
|
||||
};
|
||||
let affected = match write_result {
|
||||
Ok(r) => r.rows_affected(),
|
||||
|
||||
@@ -8,3 +8,4 @@ pub mod mime_detect;
|
||||
pub mod runtime;
|
||||
pub mod stubs;
|
||||
pub mod text;
|
||||
pub mod username_migration;
|
||||
|
||||
@@ -0,0 +1,396 @@
|
||||
//! Username-lowercase boot flow: verifier, auto-rename, shared collision helper.
|
||||
//!
|
||||
//! The plan (`docs/plan/username-lowercase.md`) makes usernames
|
||||
//! case-insensitive by canonicalising to lowercase on ingest. Three
|
||||
//! pieces of infrastructure live here:
|
||||
//!
|
||||
//! 1. [`verify_all_usernames_lowercase`] — a **read-only** check that
|
||||
//! runs after `sqlx::migrate!()` at boot. Classifies every active
|
||||
//! mixed-case row into one of three outcomes:
|
||||
//!
|
||||
//! - [`UsernameCaseCheck::Clean`] — nothing to do.
|
||||
//! - [`UsernameCaseCheck::AutoRenamable`] — mixed-case rows exist
|
||||
//! but each `LOWER(username)` form is unique in the active-user
|
||||
//! set. Safe to lowercase in one atomic transaction; boot proceeds.
|
||||
//! - [`UsernameCaseCheck::Collisions`] — at least one group has
|
||||
//! two or more active rows sharing a `LOWER(username)` (e.g.
|
||||
//! `Alice` + `alice`). Tiebreak requires human judgement; the
|
||||
//! server refuses to start and prints the CLI command.
|
||||
//!
|
||||
//! Follows [[feedback_no_silent_auto_repair]] in spirit: silent
|
||||
//! action is limited to cases where there is exactly one correct
|
||||
//! move (rename the sole mixed-case row to its lowercase form).
|
||||
//! Anywhere ambiguity exists (which of `Alice` and `alice` keeps
|
||||
//! the canonical name?), boot refuses and defers to `oxicloud
|
||||
//! migrate lowercase-usernames`.
|
||||
//!
|
||||
//! 2. [`apply_auto_renames`] — the one-transaction UPDATE loop that
|
||||
//! performs the auto-rename path. Emits a structured audit line
|
||||
//! per row (`user.username_lowercased_on_boot`). All-or-nothing:
|
||||
//! a mid-tx failure aborts the transaction and boot fails, so the
|
||||
//! DB is never left in a half-renamed state.
|
||||
//!
|
||||
//! 3. [`find_free_username_suffix`] — the shared collision-resolution
|
||||
//! helper. Called by the migration CLI when it lowercases a name
|
||||
//! that would clash with an existing row, AND by the un-soft-delete
|
||||
//! API when it re-normalises a mixed-case account whose lowercase
|
||||
//! form is now taken by someone else.
|
||||
//!
|
||||
//! `NULL` usernames (OPAQUE-migrated accounts) are always skipped — the
|
||||
//! SQL `WHERE username <> LOWER(username)` predicate is NULL-safe by
|
||||
//! semantics (`NULL <> anything` yields `NULL`, which `WHERE` excludes).
|
||||
//! Soft-deleted / disabled accounts (`active = false`) are also skipped:
|
||||
//! they can't serve traffic anyway.
|
||||
|
||||
use sqlx::{PgPool, Row};
|
||||
|
||||
/// One mixed-case account row. Used both for the auto-rename list and
|
||||
/// for reporting collision-group members.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct MixedCaseAccount {
|
||||
pub id: uuid::Uuid,
|
||||
pub username: String,
|
||||
pub last_login_at: Option<chrono::DateTime<chrono::Utc>>,
|
||||
}
|
||||
|
||||
/// A `LOWER(username)` group with two or more active members. At least
|
||||
/// one member is mixed-case (that's what made the group visible to the
|
||||
/// verifier); the other member(s) may be already-lowercase (e.g.
|
||||
/// `Alice` + `alice`) or also mixed-case (`Alice` + `ALICE`).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CollisionGroup {
|
||||
/// The lowercase form shared by every member.
|
||||
pub canonical: String,
|
||||
/// Members, ordered by the tiebreak that the migration CLI
|
||||
/// applies: `last_login_at DESC NULLS LAST, created_at ASC`.
|
||||
pub members: Vec<MixedCaseAccount>,
|
||||
}
|
||||
|
||||
/// The three outcomes of the boot-time verifier.
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum UsernameCaseCheck {
|
||||
/// Every active username is already lowercase (or `NULL`). Boot
|
||||
/// proceeds unmodified.
|
||||
Clean,
|
||||
/// Mixed-case rows exist, but each `LOWER(username)` form is
|
||||
/// unique among active users. Safe to lowercase atomically at
|
||||
/// boot; the caller runs [`apply_auto_renames`].
|
||||
AutoRenamable(Vec<MixedCaseAccount>),
|
||||
/// At least one `LOWER(username)` group has two or more active
|
||||
/// members. Tiebreak requires human judgement; the caller formats
|
||||
/// a refusal message via [`format_refusal_message_collisions`] and
|
||||
/// aborts boot.
|
||||
Collisions(Vec<CollisionGroup>),
|
||||
}
|
||||
|
||||
/// Boot-time verifier. Runs AFTER `sqlx::migrate!()` and BEFORE
|
||||
/// `AppState` is assembled. Read-only: never mutates `auth.users`.
|
||||
///
|
||||
/// Returns [`UsernameCaseCheck`] describing what (if anything) the
|
||||
/// caller should do. Errors are limited to DB failures — semantic
|
||||
/// outcomes are all `Ok(_)` variants.
|
||||
pub async fn verify_all_usernames_lowercase(pool: &PgPool) -> Result<UsernameCaseCheck, String> {
|
||||
// First pass: mixed-case rows that have NO other active row
|
||||
// sharing their LOWER form. These are safe to auto-rename.
|
||||
let auto_rows = sqlx::query(
|
||||
r#"
|
||||
SELECT u.id, u.username, u.last_login_at
|
||||
FROM auth.users u
|
||||
WHERE u.active = true
|
||||
AND u.username <> LOWER(u.username)
|
||||
AND NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM auth.users u2
|
||||
WHERE u2.active = true
|
||||
AND u2.id <> u.id
|
||||
AND LOWER(u2.username) = LOWER(u.username)
|
||||
)
|
||||
ORDER BY LOWER(u.username)
|
||||
"#,
|
||||
)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.map_err(|e| format!("username lowercase verifier: singleton query failed: {e}"))?;
|
||||
|
||||
// Second pass: every active row that belongs to a colliding
|
||||
// group — a `LOWER(username)` shared by two or more active rows
|
||||
// where at least one member is mixed-case. Result includes
|
||||
// already-lowercase members so the refusal report shows the full
|
||||
// context of each collision.
|
||||
let collision_rows = sqlx::query(
|
||||
r#"
|
||||
WITH colliding_lowers AS (
|
||||
SELECT LOWER(username) AS canonical
|
||||
FROM auth.users
|
||||
WHERE active = true
|
||||
GROUP BY LOWER(username)
|
||||
HAVING COUNT(*) > 1
|
||||
AND SUM(CASE WHEN username <> LOWER(username) THEN 1 ELSE 0 END) >= 1
|
||||
)
|
||||
SELECT id, username, last_login_at, LOWER(username) AS canonical
|
||||
FROM auth.users
|
||||
WHERE active = true
|
||||
AND LOWER(username) IN (SELECT canonical FROM colliding_lowers)
|
||||
ORDER BY LOWER(username),
|
||||
(last_login_at IS NULL),
|
||||
last_login_at DESC NULLS LAST,
|
||||
created_at ASC
|
||||
"#,
|
||||
)
|
||||
.fetch_all(pool)
|
||||
.await
|
||||
.map_err(|e| format!("username lowercase verifier: collision query failed: {e}"))?;
|
||||
|
||||
if !collision_rows.is_empty() {
|
||||
// Group by canonical. Rows are already ordered by canonical
|
||||
// then by tiebreak, so a fold is enough.
|
||||
let mut groups: Vec<CollisionGroup> = Vec::new();
|
||||
for r in collision_rows {
|
||||
let canonical: String = r.get("canonical");
|
||||
let member = MixedCaseAccount {
|
||||
id: r.get::<uuid::Uuid, _>("id"),
|
||||
username: r.get::<String, _>("username"),
|
||||
last_login_at: r
|
||||
.try_get::<chrono::DateTime<chrono::Utc>, _>("last_login_at")
|
||||
.ok(),
|
||||
};
|
||||
match groups.last_mut() {
|
||||
Some(g) if g.canonical == canonical => g.members.push(member),
|
||||
_ => groups.push(CollisionGroup {
|
||||
canonical,
|
||||
members: vec![member],
|
||||
}),
|
||||
}
|
||||
}
|
||||
return Ok(UsernameCaseCheck::Collisions(groups));
|
||||
}
|
||||
|
||||
if auto_rows.is_empty() {
|
||||
return Ok(UsernameCaseCheck::Clean);
|
||||
}
|
||||
|
||||
let accounts = auto_rows
|
||||
.into_iter()
|
||||
.map(|r| MixedCaseAccount {
|
||||
id: r.get::<uuid::Uuid, _>("id"),
|
||||
username: r.get::<String, _>("username"),
|
||||
last_login_at: r
|
||||
.try_get::<chrono::DateTime<chrono::Utc>, _>("last_login_at")
|
||||
.ok(),
|
||||
})
|
||||
.collect();
|
||||
Ok(UsernameCaseCheck::AutoRenamable(accounts))
|
||||
}
|
||||
|
||||
/// Apply the atomic auto-rename transaction. All UPDATEs succeed
|
||||
/// together or all roll back — the DB is never left in a half-renamed
|
||||
/// state. Each successful rename emits a structured audit line.
|
||||
///
|
||||
/// The `WHERE id = $1 AND username = $3` guard defends against a
|
||||
/// concurrent rename between the SELECT and this UPDATE. If some
|
||||
/// other process renamed the row in that window, the UPDATE affects
|
||||
/// zero rows and we log a warning but do not fail the transaction —
|
||||
/// the row is already lowercase (that's why the guard didn't match),
|
||||
/// so the invariant still holds.
|
||||
pub async fn apply_auto_renames(
|
||||
pool: &PgPool,
|
||||
accounts: &[MixedCaseAccount],
|
||||
) -> Result<(), sqlx::Error> {
|
||||
let mut tx = pool.begin().await?;
|
||||
for acc in accounts {
|
||||
let new_username = acc.username.to_ascii_lowercase();
|
||||
let res = sqlx::query(
|
||||
r#"
|
||||
UPDATE auth.users
|
||||
SET username = $2
|
||||
WHERE id = $1
|
||||
AND username = $3
|
||||
"#,
|
||||
)
|
||||
.bind(acc.id)
|
||||
.bind(&new_username)
|
||||
.bind(&acc.username)
|
||||
.execute(&mut *tx)
|
||||
.await?;
|
||||
|
||||
if res.rows_affected() == 0 {
|
||||
tracing::warn!(
|
||||
target: "audit",
|
||||
event = "user.username_lowercase_skipped_on_boot",
|
||||
reason = "row_changed_between_verify_and_apply",
|
||||
user_id = %acc.id,
|
||||
expected_username = %acc.username,
|
||||
"👮🏻♂️ skipped auto-lowercase: row was modified after verifier ran",
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
tracing::info!(
|
||||
target: "audit",
|
||||
event = "user.username_lowercased_on_boot",
|
||||
reason = "unique_lowercase_group",
|
||||
user_id = %acc.id,
|
||||
old_username = %acc.username,
|
||||
new_username = %new_username,
|
||||
"👮🏻♂️ auto-lowercased username at boot",
|
||||
);
|
||||
}
|
||||
tx.commit().await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Format the FATAL error string shown when boot refuses to proceed
|
||||
/// because at least one `LOWER(username)` group has multiple active
|
||||
/// members. Self-sufficient — an operator at 3 AM shouldn't need to
|
||||
/// consult docs to know what to do.
|
||||
pub fn format_refusal_message_collisions(groups: &[CollisionGroup]) -> String {
|
||||
use std::fmt::Write;
|
||||
let total_members: usize = groups.iter().map(|g| g.members.len()).sum();
|
||||
let mut out = String::new();
|
||||
let _ = write!(
|
||||
&mut out,
|
||||
"\nFATAL: cannot start — {} colliding username group(s) \
|
||||
({} affected account(s) in total).\n\n\
|
||||
Non-colliding mixed-case rows are auto-renamed at boot. \
|
||||
These groups can't be resolved automatically because two or \
|
||||
more active accounts share the same lowercase form, and only \
|
||||
a human can decide who keeps the canonical name.\n\n\
|
||||
Run the migration:\n\n \
|
||||
oxicloud migrate lowercase-usernames --dry-run # preview the tiebreak\n \
|
||||
oxicloud migrate lowercase-usernames # apply\n\n\
|
||||
The tiebreak rule is `last_login_at DESC NULLS LAST, \
|
||||
created_at ASC` — the most recently active member keeps the \
|
||||
canonical lowercase name; the losers get `-2`, `-3`, … as a \
|
||||
suffix. Sessions and grants survive the rename (they key on \
|
||||
user_id, not username).\n\n\
|
||||
Collision groups (up to 10 shown):\n",
|
||||
groups.len(),
|
||||
total_members
|
||||
);
|
||||
for g in groups.iter().take(10) {
|
||||
let _ = writeln!(&mut out, "\n Canonical form: {}", g.canonical);
|
||||
for m in &g.members {
|
||||
let last = m
|
||||
.last_login_at
|
||||
.map(|t| t.format("%Y-%m-%d").to_string())
|
||||
.unwrap_or_else(|| "never".to_string());
|
||||
let _ = writeln!(
|
||||
&mut out,
|
||||
" {} (id: {} last_login: {})",
|
||||
m.username, m.id, last
|
||||
);
|
||||
}
|
||||
}
|
||||
if groups.len() > 10 {
|
||||
let _ = writeln!(
|
||||
&mut out,
|
||||
"\n ... and {} more group(s). Run --dry-run for the full list.",
|
||||
groups.len() - 10
|
||||
);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Cap on the suffix-probe loop. If we ever need `<base>-10000` there's
|
||||
/// something very wrong with the account universe — collisions in the
|
||||
/// wild are 2-3 accounts, not 10 K. The loud abort IS the detection.
|
||||
/// See [`docs/plan/username-lowercase.md § 3. Suffix-collision robustness`].
|
||||
const SUFFIX_PROBE_CAP: i32 = 10_000;
|
||||
|
||||
/// Find the next free `<base>-<N>` suffix for a colliding username.
|
||||
///
|
||||
/// Starts at `<base>-2` and increments until an unused suffix is
|
||||
/// found. Robust against pre-existing rows already occupying some
|
||||
/// suffixes (the probe steps past them).
|
||||
///
|
||||
/// Called by:
|
||||
/// - The migration CLI when a `LOWER(username)` group has multiple
|
||||
/// members and the tiebreak winner keeps the canonical name; the
|
||||
/// losers get `<base>-2`, `-3`, … from this helper.
|
||||
/// - The un-soft-delete API when re-normalising a mixed-case
|
||||
/// account whose lowercase form is now taken by an active row.
|
||||
///
|
||||
/// Both callers reach for this single function so the two paths
|
||||
/// agree by construction — no drift risk between the migration and
|
||||
/// runtime un-soft-delete.
|
||||
pub async fn find_free_username_suffix(pool: &PgPool, base: &str) -> Result<String, sqlx::Error> {
|
||||
for n in 2..=SUFFIX_PROBE_CAP {
|
||||
let candidate = format!("{base}-{n}");
|
||||
let exists: (bool,) =
|
||||
sqlx::query_as("SELECT EXISTS(SELECT 1 FROM auth.users WHERE username = $1)")
|
||||
.bind(&candidate)
|
||||
.fetch_one(pool)
|
||||
.await?;
|
||||
if !exists.0 {
|
||||
return Ok(candidate);
|
||||
}
|
||||
}
|
||||
// If we get here, something is very wrong. Loud panic beats
|
||||
// silent truncation to whatever the caller's fallback is.
|
||||
panic!(
|
||||
"find_free_username_suffix: exhausted {SUFFIX_PROBE_CAP} suffix probes for base '{base}'; \
|
||||
the account universe likely has an anomaly worth investigating"
|
||||
);
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn acc(name: &str) -> MixedCaseAccount {
|
||||
MixedCaseAccount {
|
||||
id: uuid::Uuid::nil(),
|
||||
username: name.into(),
|
||||
last_login_at: None,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn refusal_message_lists_groups_and_cli() {
|
||||
let groups = vec![CollisionGroup {
|
||||
canonical: "alice".into(),
|
||||
members: vec![acc("Alice"), acc("alice")],
|
||||
}];
|
||||
let msg = format_refusal_message_collisions(&groups);
|
||||
assert!(msg.contains("1 colliding username group(s)"));
|
||||
assert!(msg.contains("2 affected account(s)"));
|
||||
assert!(msg.contains("oxicloud migrate lowercase-usernames"));
|
||||
assert!(msg.contains("Canonical form: alice"));
|
||||
assert!(msg.contains("Alice"));
|
||||
assert!(msg.contains("last_login: never"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn refusal_message_caps_group_display_and_notes_overflow() {
|
||||
let groups: Vec<_> = (0..15)
|
||||
.map(|i| CollisionGroup {
|
||||
canonical: format!("user{i:02}"),
|
||||
members: vec![acc(&format!("User{i:02}")), acc(&format!("user{i:02}"))],
|
||||
})
|
||||
.collect();
|
||||
let msg = format_refusal_message_collisions(&groups);
|
||||
// First 10 groups shown by canonical name.
|
||||
assert!(msg.contains("Canonical form: user00"));
|
||||
assert!(msg.contains("Canonical form: user09"));
|
||||
// Overflow tail names how many are hidden.
|
||||
assert!(msg.contains("and 5 more group(s)"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn refusal_message_reports_total_across_all_groups() {
|
||||
// Two groups with different sizes — 2 + 3 = 5 members total.
|
||||
let groups = vec![
|
||||
CollisionGroup {
|
||||
canonical: "alice".into(),
|
||||
members: vec![acc("Alice"), acc("alice")],
|
||||
},
|
||||
CollisionGroup {
|
||||
canonical: "bob".into(),
|
||||
members: vec![acc("Bob"), acc("BOB"), acc("bob")],
|
||||
},
|
||||
];
|
||||
let msg = format_refusal_message_collisions(&groups);
|
||||
assert!(msg.contains("2 colliding username group(s)"));
|
||||
assert!(msg.contains("5 affected account(s)"));
|
||||
}
|
||||
}
|
||||
+93
-14
@@ -322,9 +322,15 @@ impl User {
|
||||
is_external: bool,
|
||||
) -> UserResult<Self> {
|
||||
Self::validate_email(&email)?;
|
||||
if let Some(ref u) = username {
|
||||
Self::validate_username(u)?;
|
||||
}
|
||||
// Shadow `username` with the canonical (trimmed, lowercased)
|
||||
// form returned by `validate_username`. Every downstream write
|
||||
// consumes the shadowed binding, so the row that lands in the
|
||||
// DB is always the normalised value. See
|
||||
// `docs/plan/username-lowercase.md`.
|
||||
let username = match username {
|
||||
Some(u) => Some(Self::validate_username(&u)?),
|
||||
None => None,
|
||||
};
|
||||
if let Some(ref h) = password_hash
|
||||
&& h.is_empty()
|
||||
{
|
||||
@@ -809,8 +815,10 @@ impl User {
|
||||
/// renamed: it was display text at creation; the folder is owned
|
||||
/// by `user_id`.
|
||||
pub fn set_username(&mut self, new_username: String) -> UserResult<()> {
|
||||
Self::validate_username(&new_username)?;
|
||||
self.username = Some(new_username);
|
||||
// Canonical form (trim + lowercase) — see
|
||||
// `validate_username`. Callers can pass any case; we store
|
||||
// the normalised value.
|
||||
self.username = Some(Self::validate_username(&new_username)?);
|
||||
self.updated_at = Utc::now();
|
||||
Ok(())
|
||||
}
|
||||
@@ -881,20 +889,41 @@ impl User {
|
||||
/// a handle that shadows another user's email). No leading/trailing
|
||||
/// dot or hyphen. The character set also prevents XSS payloads from
|
||||
/// being stored as usernames.
|
||||
fn validate_username(username: &str) -> UserResult<()> {
|
||||
let len = username.chars().count();
|
||||
/// Validate AND canonicalise a username.
|
||||
///
|
||||
/// Two normalisations run first, before every check:
|
||||
/// - `trim()` — strip whitespace clients may have added.
|
||||
/// - `to_ascii_lowercase()` — usernames are case-insensitive
|
||||
/// identifiers. Users type `Alice`, `ALICE`, `alice` on
|
||||
/// different clients; all three refer to the same account.
|
||||
/// ASCII-only by construction (charset check below), so
|
||||
/// `to_ascii_lowercase` is deterministic and locale-safe —
|
||||
/// no Unicode case-folding surprises (Turkish dotted-I,
|
||||
/// German ß, Greek final sigma, NFC vs NFD).
|
||||
///
|
||||
/// Returns the canonical form on success. Every entity write
|
||||
/// site consumes the returned string — because the signature
|
||||
/// changed from `Result<()>` to `Result<String>`, any caller
|
||||
/// that ignored the result is now a compile error. That's
|
||||
/// what forces every write path through the normaliser.
|
||||
///
|
||||
/// See `docs/plan/username-lowercase.md` for the full design.
|
||||
pub fn validate_username(username: &str) -> UserResult<String> {
|
||||
let normalized = username.trim().to_ascii_lowercase();
|
||||
|
||||
let len = normalized.chars().count();
|
||||
if !(2..=64).contains(&len) {
|
||||
return Err(UserError::InvalidUsername(
|
||||
"Username must be between 2 and 64 characters".to_string(),
|
||||
));
|
||||
}
|
||||
if username.contains('@') {
|
||||
if normalized.contains('@') {
|
||||
return Err(UserError::InvalidUsername(
|
||||
"Username must not contain '@' — use the email field for email addresses"
|
||||
.to_string(),
|
||||
));
|
||||
}
|
||||
if !username
|
||||
if !normalized
|
||||
.chars()
|
||||
.all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_' || c == '.')
|
||||
{
|
||||
@@ -903,16 +932,16 @@ impl User {
|
||||
.to_string(),
|
||||
));
|
||||
}
|
||||
if username.starts_with('.')
|
||||
|| username.starts_with('-')
|
||||
|| username.ends_with('.')
|
||||
|| username.ends_with('-')
|
||||
if normalized.starts_with('.')
|
||||
|| normalized.starts_with('-')
|
||||
|| normalized.ends_with('.')
|
||||
|| normalized.ends_with('-')
|
||||
{
|
||||
return Err(UserError::InvalidUsername(
|
||||
"Username must not start or end with a dot or hyphen".to_string(),
|
||||
));
|
||||
}
|
||||
Ok(())
|
||||
Ok(normalized)
|
||||
}
|
||||
|
||||
/// Basic but meaningful email validation:
|
||||
@@ -1046,4 +1075,54 @@ mod tests {
|
||||
assert_eq!(u.display_full(true), "solo@x.com");
|
||||
assert_eq!(u.display_full(false), "solo@x.com");
|
||||
}
|
||||
|
||||
// ── validate_username: normalization + rules ─────────────────────────────
|
||||
//
|
||||
// Post-lowercase-migration `validate_username` returns the canonical
|
||||
// (trimmed, lowercased) form on success. Every write-site consumes
|
||||
// that returned string via the shadow in `User::new` /
|
||||
// `set_username`, so the invariant "usernames in `auth.users` are
|
||||
// always canonical" is enforced at the domain boundary.
|
||||
//
|
||||
// The rules that DON'T change (charset, length, no leading/trailing
|
||||
// dot or hyphen, no `@`) get their coverage here too so a future
|
||||
// rewrite of `validate_username` can't regress them silently.
|
||||
|
||||
#[test]
|
||||
fn validate_username_lowercases_and_trims() {
|
||||
// Uppercase in the middle → canonical form is lowercase.
|
||||
assert_eq!(User::validate_username("Alice").unwrap(), "alice");
|
||||
// All-uppercase.
|
||||
assert_eq!(User::validate_username("ALICE").unwrap(), "alice");
|
||||
// Whitespace around a mixed-case name → both stripped.
|
||||
assert_eq!(User::validate_username(" Alice ").unwrap(), "alice");
|
||||
// Already-canonical passes through unchanged.
|
||||
assert_eq!(User::validate_username("alice").unwrap(), "alice");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validate_username_charset_and_boundary_rules_survive_normalization() {
|
||||
// Trailing hyphen — still rejected after the case-fold.
|
||||
assert!(User::validate_username("alice-").is_err());
|
||||
// Leading dot.
|
||||
assert!(User::validate_username(".alice").is_err());
|
||||
// Whitespace INSIDE the name (not just around it) — the
|
||||
// charset check rejects space characters.
|
||||
assert!(User::validate_username("Al ice").is_err());
|
||||
// Non-ASCII letter — usernames are ASCII-only.
|
||||
assert!(User::validate_username("Álice").is_err());
|
||||
// `@` is forbidden (disjoint namespace with email lookup).
|
||||
assert!(User::validate_username("alice@example").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validate_username_length_bounds_apply_after_trim() {
|
||||
// Two-char minimum satisfied AFTER trim.
|
||||
assert_eq!(User::validate_username(" ab ").unwrap(), "ab");
|
||||
// Below the minimum after trim.
|
||||
assert!(User::validate_username(" a ").is_err());
|
||||
// Above the maximum after trim.
|
||||
let too_long = "a".repeat(65);
|
||||
assert!(User::validate_username(&too_long).is_err());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -61,6 +61,46 @@ pub async fn create_database_pools(config: &AppConfig) -> Result<DbPools> {
|
||||
}
|
||||
tracing::info!("Database migrations complete");
|
||||
|
||||
// Username-lowercase verifier — three outcomes:
|
||||
// * Clean → nothing to do.
|
||||
// * AutoRenamable → non-colliding mixed-case rows exist; lowercase
|
||||
// them in one transaction and continue. Silent
|
||||
// action is bounded to the case where there is
|
||||
// exactly one correct move ([[feedback_no_silent_auto_repair]]
|
||||
// in spirit — ambiguity → refusal, unique fix → apply).
|
||||
// Each rename emits an audit line.
|
||||
// * Collisions → two or more active rows share a LOWER(username)
|
||||
// form (e.g. `Alice` + `alice`); tiebreak needs a
|
||||
// human, refuse to boot and print the CLI command.
|
||||
// See `common::username_migration::verify_all_usernames_lowercase`.
|
||||
use crate::common::username_migration::{
|
||||
UsernameCaseCheck, apply_auto_renames, format_refusal_message_collisions,
|
||||
verify_all_usernames_lowercase,
|
||||
};
|
||||
match verify_all_usernames_lowercase(&primary).await {
|
||||
Ok(UsernameCaseCheck::Clean) => {}
|
||||
Ok(UsernameCaseCheck::AutoRenamable(accounts)) => {
|
||||
let count = accounts.len();
|
||||
if let Err(e) = apply_auto_renames(&primary, &accounts).await {
|
||||
return Err(DbError(format!(
|
||||
"username lowercase auto-rename failed at boot: {e}. \
|
||||
Run `oxicloud migrate lowercase-usernames --dry-run` to \
|
||||
inspect the current state, then apply manually."
|
||||
)));
|
||||
}
|
||||
tracing::info!(
|
||||
target: "audit",
|
||||
event = "user.usernames_lowercased_on_boot_summary",
|
||||
renamed = count,
|
||||
"auto-lowercased {count} non-colliding mixed-case username(s) at boot",
|
||||
);
|
||||
}
|
||||
Ok(UsernameCaseCheck::Collisions(groups)) => {
|
||||
return Err(DbError(format_refusal_message_collisions(&groups)));
|
||||
}
|
||||
Err(msg) => return Err(DbError(msg)),
|
||||
}
|
||||
|
||||
// --- maintenance pool ---
|
||||
let maintenance = create_pool_with_retries(
|
||||
&config.database.connection_string,
|
||||
|
||||
@@ -472,8 +472,15 @@ impl UserRepository for UserPgRepository {
|
||||
Ok((user, flags))
|
||||
}
|
||||
|
||||
/// Gets a user by username
|
||||
/// Gets a user by username.
|
||||
///
|
||||
/// Lowercases the input before binding: usernames are stored in
|
||||
/// canonical (lowercase, trimmed) form by `validate_username`
|
||||
/// (see `docs/plan/username-lowercase.md`), and callers may pass
|
||||
/// whatever case the user typed at the login form. Normalising
|
||||
/// here means every caller doesn't have to remember.
|
||||
async fn get_user_by_username(&self, username: &str) -> UserRepositoryResult<User> {
|
||||
let username = username.trim().to_ascii_lowercase();
|
||||
let row = sqlx::query(
|
||||
r#"
|
||||
SELECT
|
||||
@@ -487,7 +494,7 @@ impl UserRepository for UserPgRepository {
|
||||
WHERE username = $1
|
||||
"#,
|
||||
)
|
||||
.bind(username)
|
||||
.bind(&username)
|
||||
.fetch_one(&*self.pool)
|
||||
.await
|
||||
.map_err(Self::map_sqlx_error)?;
|
||||
|
||||
@@ -58,7 +58,7 @@ use crate::application::ports::blob_lifecycle::BlobLifecycleHook;
|
||||
use crate::application::ports::blob_reference_ports::{BlobReferenceRegistry, RefLevel};
|
||||
use crate::application::ports::blob_storage_ports::BlobStorageBackend;
|
||||
use crate::application::ports::dedup_ports::{
|
||||
BlobMetadataDto, DedupPort, DedupResultDto, DedupStatsDto,
|
||||
BlobMetadataDto, DedupPort, DedupResultDto, DedupStatsDto, DerivedBlobRef,
|
||||
};
|
||||
use crate::application::services::blob_lifecycle_service::BlobLifecycleService;
|
||||
use crate::domain::errors::{DomainError, ErrorKind};
|
||||
@@ -72,6 +72,86 @@ pub const CDC_AVG_CHUNK: usize = 262_144;
|
||||
/// Maximum CDC chunk size (1 MB).
|
||||
pub const CDC_MAX_CHUNK: usize = 1_048_576;
|
||||
|
||||
// ════════════════════════════════════════════════════════════════════════════
|
||||
// ref_count audit log — attribution trail for ref_count changes
|
||||
//
|
||||
// Emitted at every semantic ref-count mutation site so an unexplained drift
|
||||
// (`manifests_consistency` / `blobs_consistency` finding) can be traced back
|
||||
// to its calling function within one log query. See
|
||||
// `docs/plan/refcount-audit.md` for design rationale, retention model, and
|
||||
// upgrade path to a DB-backed table if log retention proves insufficient.
|
||||
//
|
||||
// Enable at runtime with `RUST_LOG=oxicloud::refcount=info`. Off by default;
|
||||
// info-level so a healthy prod deployment doesn't spam the log stream.
|
||||
// ════════════════════════════════════════════════════════════════════════════
|
||||
|
||||
/// Table names the audit stream uses. Constants (not free strings) so grep
|
||||
/// across a log stream matches a fixed vocabulary and a typo in a call site
|
||||
/// fails to compile instead of silently drifting.
|
||||
mod refcount_audit_table {
|
||||
pub const CHUNK_MANIFESTS: &str = "chunk_manifests";
|
||||
pub const BLOBS: &str = "blobs";
|
||||
}
|
||||
|
||||
/// Source labels for the audit stream — one stable string per Rust function
|
||||
/// that mutates a ref_count. Kept as a module so `grep source=` on the log
|
||||
/// stream shows a fixed enumeration; a rename here is intentional, a rename
|
||||
/// at a call site alone doesn't compile.
|
||||
///
|
||||
/// See `docs/plan/refcount-audit.md § Callsites to instrument` for the full
|
||||
/// list. New callers add a new constant here; adding one string at the call
|
||||
/// site alone is discouraged (breaks the "closed vocabulary" property).
|
||||
mod refcount_audit_source {
|
||||
pub const STORE_FROM_STREAM_NEW_MANIFEST: &str = "store_from_stream.new_manifest";
|
||||
pub const BUMP_MANIFEST_IF_EXISTS: &str = "bump_manifest_if_exists";
|
||||
pub const ADD_REFERENCE_MANIFEST: &str = "add_reference.manifest";
|
||||
pub const ADD_REFERENCE_LEGACY: &str = "add_reference.legacy";
|
||||
pub const REMOVE_MANIFEST_REFERENCE_DECREMENT: &str = "remove_manifest_reference.decrement";
|
||||
pub const REMOVE_MANIFEST_REFERENCE_DELETE: &str = "remove_manifest_reference.delete";
|
||||
pub const REMOVE_LEGACY_REFERENCE: &str = "remove_legacy_reference";
|
||||
pub const STORE_ATTACHED_BLOB_SAME_CONTENT_BALANCE: &str =
|
||||
"store_attached_blob.same_content_balance";
|
||||
pub const STORE_ATTACHED_BLOB_REPLACE_RELEASE: &str = "store_attached_blob.replace_release";
|
||||
}
|
||||
|
||||
/// Outcome of [`DedupService::store_attached_blob_if_absent`]. Split
|
||||
/// so the caller (`thumb_attached_import_service`) can bump its
|
||||
/// `imported` vs `already` counters without a second query.
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum AttachedBlobInsertOutcome {
|
||||
/// We won the atomic INSERT — the row now points at `hash`.
|
||||
Inserted { hash: String },
|
||||
/// A row already existed when the atomic INSERT ran (someone else
|
||||
/// won, or the migration was re-triggered). `existing_hash` is
|
||||
/// the row's current `blob_hash` as read moments after the DO
|
||||
/// NOTHING resolved — useful for the import service's
|
||||
/// verify-and-unlink readback.
|
||||
AlreadyPresent { existing_hash: String },
|
||||
}
|
||||
|
||||
/// Emit a single audit line for a ref_count change. Called AFTER the SQL
|
||||
/// UPDATE / INSERT / DELETE returns Ok, so a rolled-back transaction won't
|
||||
/// leave a phantom log line (the SQL error path returns before this call).
|
||||
///
|
||||
/// `delta` is the signed change (`+1` on increment, `-1` on decrement,
|
||||
/// `-old_count` when the row is deleted at its last reference — the
|
||||
/// convention is "resulting ref_count is 0 for reads afterward").
|
||||
///
|
||||
/// The tracing span inherits request-scope context (request_id, caller_id
|
||||
/// from auth middleware, job run id from scheduler) automatically, so
|
||||
/// no explicit correlation-id plumbing is needed here.
|
||||
#[inline]
|
||||
fn audit_ref_count(table: &'static str, hash: &str, delta: i32, source: &'static str) {
|
||||
tracing::info!(
|
||||
target: "oxicloud::refcount",
|
||||
table,
|
||||
hash = %hash,
|
||||
delta,
|
||||
source,
|
||||
"ref_count {}", if delta >= 0 { "+" } else { "-" }
|
||||
);
|
||||
}
|
||||
|
||||
// ── CDC helper types ─────────────────────────────────────────────────────────
|
||||
|
||||
/// Everything a streaming chunk ingest learned about its byte stream.
|
||||
@@ -566,6 +646,49 @@ fn manifest_reap_sql(registry: &BlobReferenceRegistry) -> String {
|
||||
)
|
||||
}
|
||||
|
||||
// ── Attached-blob lookup cache ───────────────────────────────────────────────
|
||||
|
||||
/// Cache size cap for [`DedupService::attached_blob_cache`] — plain entry
|
||||
/// count (no weigher): an entry is three short strings + two short strings,
|
||||
/// tens of bytes; 50k entries ≈ a few MB, noise next to the manifest cache.
|
||||
pub(crate) const ATTACHED_BLOB_CACHE_MAX_ENTRIES: u64 = 50_000;
|
||||
/// Hard staleness bound for [`DedupService::attached_blob_cache`].
|
||||
///
|
||||
/// Deliberately [`moka::future::Cache::builder().time_to_live`] and NOT
|
||||
/// `time_to_idle`: a hot negative entry under TTI never expires, and TTL must
|
||||
/// be the last-resort bound for writes this process never saw (bare SQL, a
|
||||
/// future second instance, the `copy_file_satellites` race window).
|
||||
pub(crate) const ATTACHED_BLOB_CACHE_TTL_SECS: u64 = 60;
|
||||
|
||||
/// Cache key for [`DedupService::attached_blob_cache`] — the
|
||||
/// `storage.file_attached_blobs` primary key. A struct, not a
|
||||
/// `(String, String, String)` tuple: three same-typed fields read by position
|
||||
/// would force every construction site (and the `invalidate_for_file` scan)
|
||||
/// to guess semantics; self-documenting beats positional here.
|
||||
#[derive(Clone, PartialEq, Eq, Hash, Debug)]
|
||||
struct AttachedBlobKey {
|
||||
file_id: String,
|
||||
kind: String,
|
||||
variant: String,
|
||||
}
|
||||
|
||||
impl AttachedBlobKey {
|
||||
fn new(file_id: &str, kind: &str, variant: &str) -> Self {
|
||||
Self {
|
||||
file_id: file_id.to_string(),
|
||||
kind: kind.to_string(),
|
||||
variant: variant.to_string(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Loader-error sentinel for the `try_get_with` cache wrapper on
|
||||
/// [`Self::find_attached_blob`]. The SQL lookup treats a DB fault the same as
|
||||
/// "no row" only at the very last moment — the cache must never see it, or a
|
||||
/// transient outage would freeze "no attached blob" into place for a full
|
||||
/// TTL while rows exist (a read failure is never proof that data is absent).
|
||||
struct AttachedLookupFault;
|
||||
|
||||
pub struct DedupService {
|
||||
/// Pluggable blob storage backend (local FS, S3, …).
|
||||
backend: Arc<dyn BlobStorageBackend>,
|
||||
@@ -584,6 +707,28 @@ pub struct DedupService {
|
||||
/// seen immediately), weight-bounded (a manifest is ~72 B per chunk),
|
||||
/// short TTL so GC'd manifests age out fast (benches/MANIFEST-CACHE.md).
|
||||
manifest_cache: moka::future::Cache<String, Arc<ChunkManifest>>,
|
||||
/// `file_id → attached blob` lookup cache (`storage.file_attached_blobs`
|
||||
/// rows) for the thumbnail hot path — `ThumbnailService::
|
||||
/// thumbnail_content_id` hits it on EVERY request (including 304
|
||||
/// revalidations and RAM thumbnail hits, `thumbnail_service.rs` ~:744)
|
||||
/// and `get_cached_thumbnail` tier 2b hits it again with the same key
|
||||
/// (~:851); the Nextcloud preview endpoint rides the same lookup.
|
||||
///
|
||||
/// Positive AND negative (`Option<DerivedBlobRef>` — most files have no
|
||||
/// attached preview row, so the negative side is where the win is). The
|
||||
/// loader NEVER caches a DB error: `find_attached_blob_uncached` returns
|
||||
/// `Err` and `try_get_with` drops it, so a transient outage cannot freeze
|
||||
/// "no attached blob" into the cache for a full TTL (a read failure is
|
||||
/// never proof that data is absent).
|
||||
///
|
||||
/// Writes invalidate through the same type: `store_attached_blob` /
|
||||
/// `store_attached_blob_if_absent` on success, file deletions via the
|
||||
/// `ThumbnailRefreshHook::on_file_deleted` piggyback. The TTL above
|
||||
/// remains the bound for anything this process cannot see (bare SQL,
|
||||
/// `copy_file_satellites` races); invalidate-vs-inflight-REFILL races are
|
||||
/// narrowed by `try_get_with` but not eliminated, and the residual window
|
||||
/// is ≤ one TTL.
|
||||
attached_blob_cache: moka::future::Cache<AttachedBlobKey, Option<DerivedBlobRef>>,
|
||||
/// Every table that holds blob references, so GC agrees with the
|
||||
/// consistency jobs on what "referenced" means. Defaults to the two
|
||||
/// built-in sources; DI replaces it once more tables exist. Never
|
||||
@@ -618,6 +763,7 @@ impl DedupService {
|
||||
maintenance_pool,
|
||||
blob_lifecycle: None,
|
||||
manifest_cache: Self::build_manifest_cache(),
|
||||
attached_blob_cache: Self::build_attached_blob_cache(),
|
||||
reference_registry: registry.clone(),
|
||||
manifest_reap_sql: manifest_reap_sql(®istry),
|
||||
blob_reap_sql: blob_reap_sql(®istry),
|
||||
@@ -649,6 +795,19 @@ impl DedupService {
|
||||
.build()
|
||||
}
|
||||
|
||||
/// See the `attached_blob_cache` field docs. Plain entry-count cap (no
|
||||
/// weigher — an entry is a handful of short strings), TTL as the hard
|
||||
/// staleness bound; same hard-coded-const treatment as the manifest
|
||||
/// cache rather than config: an internal accelerator with strict
|
||||
/// write-side invalidation, where a misconfiguration costs performance,
|
||||
/// never correctness.
|
||||
fn build_attached_blob_cache() -> moka::future::Cache<AttachedBlobKey, Option<DerivedBlobRef>> {
|
||||
moka::future::Cache::builder()
|
||||
.max_capacity(ATTACHED_BLOB_CACHE_MAX_ENTRIES)
|
||||
.time_to_live(std::time::Duration::from_secs(ATTACHED_BLOB_CACHE_TTL_SECS))
|
||||
.build()
|
||||
}
|
||||
|
||||
/// Registers the blob-reference registry used by the manifest reap
|
||||
/// predicate. Without it `garbage_collect` skips manifest collection
|
||||
/// entirely — see `docs/plan/derived-blobs.md`.
|
||||
@@ -751,32 +910,216 @@ impl DedupService {
|
||||
.await
|
||||
.map_err(|e| DomainError::internal_error("Dedup", format!("record attached blob: {e}")))?;
|
||||
|
||||
// A replaced row's old blob loses its only reference from here. Not
|
||||
// releasing it would pin those bytes forever — nothing else points at
|
||||
// a superseded preview.
|
||||
if let Some((old_hash,)) = previous
|
||||
&& old_hash != attached_hash
|
||||
&& let Err(e) = self.remove_reference(&old_hash).await
|
||||
{
|
||||
// The row is replaced — drop any cached (possibly negative) entry so
|
||||
// the next lookup refills from the new truth. Only on the success
|
||||
// path: if the execute had failed, the row is unchanged and the
|
||||
// cache is still accurate, so invalidating would just cost a refill.
|
||||
self.attached_blob_cache
|
||||
.invalidate(&AttachedBlobKey::new(file_id, kind, variant))
|
||||
.await;
|
||||
|
||||
// Two shapes to balance depending on whether the UPSERT was a
|
||||
// real content replacement or a same-content re-store:
|
||||
//
|
||||
// - `previous == Some(old) && old != attached_hash` — different
|
||||
// content overwritten in the row. Release the old blob's ref
|
||||
// (its file_attached row is gone; would leak forever otherwise).
|
||||
//
|
||||
// - `previous == Some(old) && old == attached_hash` — SAME-content
|
||||
// re-store. `store_from_stream` above incremented the manifest
|
||||
// unconditionally, but the row's blob_hash didn't change so no
|
||||
// logical reference was added. Cancel the phantom increment
|
||||
// here, or it accumulates one +1 leak per same-content call.
|
||||
// Mirrors the pattern `store_derived_blob` uses on its
|
||||
// `ON CONFLICT DO NOTHING` `inserted == 0` branch.
|
||||
// See `docs/plan/refcount-audit.md` for how the audit stream
|
||||
// would surface this class of drift if it reappears.
|
||||
//
|
||||
// - `previous == None` — brand new (file_id, kind, variant) row.
|
||||
// `store_from_stream`'s +1 pairs with the new row's implicit
|
||||
// reference; nothing to release.
|
||||
if let Some((old_hash,)) = previous {
|
||||
let source = if old_hash == attached_hash {
|
||||
refcount_audit_source::STORE_ATTACHED_BLOB_SAME_CONTENT_BALANCE
|
||||
} else {
|
||||
refcount_audit_source::STORE_ATTACHED_BLOB_REPLACE_RELEASE
|
||||
};
|
||||
if let Err(e) = self.remove_reference(&old_hash).await {
|
||||
tracing::warn!(
|
||||
target: "oxicloud::dedup",
|
||||
error = %e,
|
||||
"failed to release replaced attached-blob reference for {}",
|
||||
kind = source,
|
||||
"failed to balance attached-blob reference for {}",
|
||||
&old_hash[..old_hash.len().min(12)],
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
Ok(attached_hash)
|
||||
}
|
||||
|
||||
/// Atomic never-overwrite variant of [`Self::store_attached_blob`],
|
||||
/// for migration/import paths whose semantic is "write if absent,
|
||||
/// leave alone if present" — mirrors the shape
|
||||
/// [`Self::store_derived_blob`] already uses.
|
||||
///
|
||||
/// The plain `store_attached_blob` reads `previous`, then upserts,
|
||||
/// then decrements — non-transactional. That's correct for the
|
||||
/// user-driven PUT thumbnail path (a real replacement should
|
||||
/// release the superseded blob), but it opens a check-then-act
|
||||
/// race with concurrent writers when the caller's intent is
|
||||
/// "only import if this file hasn't already got a preview".
|
||||
/// `thumb_attached_import_service` is exactly that caller.
|
||||
///
|
||||
/// This variant uses a single-statement `INSERT ... ON CONFLICT
|
||||
/// DO NOTHING` — race-free by construction. If the row already
|
||||
/// exists (any content), the atomic INSERT is a no-op and we
|
||||
/// release the reference `store_from_stream` just took. If we
|
||||
/// won the insert, the reference is legitimately held by our
|
||||
/// new row.
|
||||
///
|
||||
/// Return value discriminates the two cases so the caller can
|
||||
/// track its own `imported` vs `already` counters:
|
||||
/// - [`AttachedBlobInsertOutcome::Inserted`] — we wrote the row.
|
||||
/// - [`AttachedBlobInsertOutcome::AlreadyPresent`] — a row was
|
||||
/// there when we arrived; we made no change and released our
|
||||
/// ref. `existing_hash` is the concurrent winner's blob hash,
|
||||
/// returned via a follow-up SELECT (so it's not strictly
|
||||
/// atomic with the INSERT, but that's fine — the row's shape
|
||||
/// is stable now that a concurrent writer can no longer
|
||||
/// collide with us here; any later change goes through the
|
||||
/// full `store_attached_blob` UPSERT path, which is out of
|
||||
/// scope for this method's "if absent" contract).
|
||||
pub async fn store_attached_blob_if_absent(
|
||||
&self,
|
||||
file_id: &str,
|
||||
kind: &str,
|
||||
variant: &str,
|
||||
content_type: &str,
|
||||
bytes: Bytes,
|
||||
uploaded_by: uuid::Uuid,
|
||||
) -> Result<AttachedBlobInsertOutcome, DomainError> {
|
||||
let stored = self
|
||||
.store_from_stream(
|
||||
stream::once(async move { Ok::<Bytes, std::io::Error>(bytes) }),
|
||||
Some(content_type.to_string()),
|
||||
)
|
||||
.await?;
|
||||
let attached_hash = stored.hash().to_string();
|
||||
|
||||
// Single-statement atomic INSERT. `ON CONFLICT DO NOTHING`
|
||||
// means: if another writer got there first, we silently
|
||||
// yield. Same primitive `store_derived_blob` uses.
|
||||
let inserted = sqlx::query(
|
||||
"INSERT INTO storage.file_attached_blobs
|
||||
(file_id, kind, variant, blob_hash, content_type, uploaded_by)
|
||||
VALUES ($1::uuid, $2, $3, $4, $5, $6)
|
||||
ON CONFLICT (file_id, kind, variant) DO NOTHING",
|
||||
)
|
||||
.bind(file_id)
|
||||
.bind(kind)
|
||||
.bind(variant)
|
||||
.bind(&attached_hash)
|
||||
.bind(content_type)
|
||||
.bind(uploaded_by)
|
||||
.execute(self.pool.as_ref())
|
||||
.await
|
||||
.map_err(|e| DomainError::internal_error("Dedup", format!("record attached blob: {e}")))?
|
||||
.rows_affected();
|
||||
|
||||
if inserted == 0 {
|
||||
// Row already existed when we arrived. Release the ref
|
||||
// `store_from_stream` above took — the row that would
|
||||
// justify it is not ours. Best-effort: leaving a
|
||||
// dangling ref is worse than a warn log line.
|
||||
if let Err(e) = self.remove_reference(&attached_hash).await {
|
||||
tracing::warn!(
|
||||
target: "oxicloud::dedup",
|
||||
error = %e,
|
||||
"failed to release duplicate attached-blob reference for {}",
|
||||
&attached_hash[..attached_hash.len().min(12)],
|
||||
);
|
||||
}
|
||||
|
||||
// Fetch the concurrent winner's hash so the import
|
||||
// service can readback-verify against its sidecar. The
|
||||
// window between DO NOTHING and this SELECT is narrow;
|
||||
// if the row gets updated in it, the sidecar delete
|
||||
// path fails its verify and keeps the sidecar — the
|
||||
// conservative fallback.
|
||||
//
|
||||
// This readback now flows through the `attached_blob_cache`.
|
||||
// Safe in-process: any write this process made already
|
||||
// invalidated the key. The only degraded case is a negative
|
||||
// entry cached before some OTHER process inserted the row —
|
||||
// nonexistent in a single-instance deployment, and even then
|
||||
// the consequence is `existing_hash: ""` → the import keeps
|
||||
// its sidecar, the documented conservative fallback.
|
||||
let existing = self.find_attached_blob(file_id, kind, variant).await;
|
||||
return Ok(AttachedBlobInsertOutcome::AlreadyPresent {
|
||||
existing_hash: existing.map(|r| r.blob_hash).unwrap_or_default(),
|
||||
});
|
||||
}
|
||||
|
||||
// We wrote a row for a key the cache may hold a negative entry for
|
||||
// (the common "import backfill" case) — drop it so the new row is
|
||||
// immediately visible to the thumbnail path.
|
||||
self.attached_blob_cache
|
||||
.invalidate(&AttachedBlobKey::new(file_id, kind, variant))
|
||||
.await;
|
||||
|
||||
Ok(AttachedBlobInsertOutcome::Inserted {
|
||||
hash: attached_hash,
|
||||
})
|
||||
}
|
||||
|
||||
/// Look up bytes attached to a file. File-keyed counterpart of
|
||||
/// [`Self::find_derived_blob`].
|
||||
///
|
||||
/// Cached read-through of [`Self::attached_blob_cache`] (positive AND
|
||||
/// negative); see the field docs for why. The public signature is
|
||||
/// unchanged — including the historical "DB fault reads as no row"
|
||||
/// behaviour — but the fault now dies BEFORE the cache instead of being
|
||||
/// indistinguishable from an absent row.
|
||||
pub async fn find_attached_blob(
|
||||
&self,
|
||||
file_id: &str,
|
||||
kind: &str,
|
||||
variant: &str,
|
||||
) -> Option<crate::application::ports::dedup_ports::DerivedBlobRef> {
|
||||
) -> Option<DerivedBlobRef> {
|
||||
match self
|
||||
.attached_blob_cache
|
||||
.try_get_with(AttachedBlobKey::new(file_id, kind, variant), async {
|
||||
self.find_attached_blob_uncached(file_id, kind, variant)
|
||||
.await
|
||||
.map_err(|_| AttachedLookupFault) // Err ⇒ never cached
|
||||
})
|
||||
.await
|
||||
{
|
||||
Ok(attached) => attached,
|
||||
Err(_) => {
|
||||
tracing::debug!(
|
||||
target: "oxicloud::dedup",
|
||||
"attached-blob lookup failed (not cached): file={} kind={} variant={}",
|
||||
file_id,
|
||||
kind,
|
||||
variant
|
||||
);
|
||||
None
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The uncached lookup — one indexed point query on the
|
||||
/// `file_attached_blobs` primary key. Unlike the historical inlined
|
||||
/// body, a DB fault surfaces as `Err` so the cache wrapper can refuse to
|
||||
/// store it; only a genuine `Ok(None)` means "no row".
|
||||
async fn find_attached_blob_uncached(
|
||||
&self,
|
||||
file_id: &str,
|
||||
kind: &str,
|
||||
variant: &str,
|
||||
) -> sqlx::Result<Option<DerivedBlobRef>> {
|
||||
sqlx::query_as::<_, (String, String)>(
|
||||
"SELECT blob_hash, content_type FROM storage.file_attached_blobs
|
||||
WHERE file_id = $1::uuid AND kind = $2 AND variant = $3",
|
||||
@@ -786,14 +1129,29 @@ impl DedupService {
|
||||
.bind(variant)
|
||||
.fetch_optional(self.pool.as_ref())
|
||||
.await
|
||||
.ok()
|
||||
.flatten()
|
||||
.map(|(blob_hash, content_type)| {
|
||||
crate::application::ports::dedup_ports::DerivedBlobRef {
|
||||
.map(|row| {
|
||||
row.map(|(blob_hash, content_type)| DerivedBlobRef {
|
||||
blob_hash,
|
||||
content_type,
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
/// Invalidate every `(kind, variant)` entry cached for one file.
|
||||
///
|
||||
/// Fired from `ThumbnailRefreshHook::on_file_deleted` so all three
|
||||
/// production delete paths (single file, folder cascade, trash clear)
|
||||
/// drop their cached rows after the DELETE commits. A linear scan over
|
||||
/// the keys is fine here: deletions are rare and the cache is capped at
|
||||
/// [`ATTACHED_BLOB_CACHE_MAX_ENTRIES`].
|
||||
pub async fn invalidate_attached_blobs_for_file(&self, file_id: &str) {
|
||||
// moka's `Iter` yields `(Arc<K>, V)` synchronously — the await lives
|
||||
// in `invalidate`, not in the scan itself.
|
||||
for (key, _) in self.attached_blob_cache.iter() {
|
||||
if key.file_id == file_id {
|
||||
self.attached_blob_cache.invalidate(&*key).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn store_derived_blob(
|
||||
@@ -1105,6 +1463,7 @@ impl DedupService {
|
||||
maintenance_pool: stub_pool.clone(),
|
||||
blob_lifecycle: None,
|
||||
manifest_cache: Self::build_manifest_cache(),
|
||||
attached_blob_cache: Self::build_attached_blob_cache(),
|
||||
reference_registry: stub_registry.clone(),
|
||||
manifest_reap_sql: manifest_reap_sql(&stub_registry),
|
||||
blob_reap_sql: blob_reap_sql(&stub_registry),
|
||||
@@ -1316,6 +1675,17 @@ impl DedupService {
|
||||
total_size,
|
||||
chunk_hashes.len(),
|
||||
);
|
||||
// The manifest INSERT above set `ref_count = 1` — that's
|
||||
// the initial reference held by whatever callsite drove
|
||||
// this ingest (a file's body, a preview attachment, a
|
||||
// derivation). Audit-log the +1 so drift investigations
|
||||
// can find where a manifest first came into existence.
|
||||
audit_ref_count(
|
||||
refcount_audit_table::CHUNK_MANIFESTS,
|
||||
file_hash,
|
||||
1,
|
||||
refcount_audit_source::STORE_FROM_STREAM_NEW_MANIFEST,
|
||||
);
|
||||
self.fire_blob_creation_hooks(file_hash, content_type.as_deref());
|
||||
return Ok(DedupResultDto::NewBlob {
|
||||
hash: file_hash.to_string(),
|
||||
@@ -1747,7 +2117,7 @@ impl DedupService {
|
||||
/// Bump a manifest's ref_count if it exists; returns its total_size.
|
||||
/// Single statement — no window between the existence check and the bump.
|
||||
async fn bump_manifest_if_exists(&self, file_hash: &str) -> Result<Option<i64>, DomainError> {
|
||||
sqlx::query_scalar::<_, i64>(
|
||||
let bumped = sqlx::query_scalar::<_, i64>(
|
||||
"UPDATE storage.chunk_manifests SET ref_count = ref_count + 1
|
||||
WHERE file_hash = $1
|
||||
RETURNING total_size",
|
||||
@@ -1757,7 +2127,17 @@ impl DedupService {
|
||||
.await
|
||||
.map_err(|e| {
|
||||
DomainError::internal_error("Dedup", format!("Failed to bump manifest ref_count: {e}"))
|
||||
})
|
||||
})?;
|
||||
|
||||
if bumped.is_some() {
|
||||
audit_ref_count(
|
||||
refcount_audit_table::CHUNK_MANIFESTS,
|
||||
file_hash,
|
||||
1,
|
||||
refcount_audit_source::BUMP_MANIFEST_IF_EXISTS,
|
||||
);
|
||||
}
|
||||
Ok(bumped)
|
||||
}
|
||||
|
||||
/// Stream → chunk store, WITHOUT creating a manifest.
|
||||
@@ -2219,6 +2599,12 @@ impl DedupService {
|
||||
.rows_affected();
|
||||
|
||||
if manifest_affected > 0 {
|
||||
audit_ref_count(
|
||||
refcount_audit_table::CHUNK_MANIFESTS,
|
||||
hash,
|
||||
1,
|
||||
refcount_audit_source::ADD_REFERENCE_MANIFEST,
|
||||
);
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
@@ -2246,6 +2632,12 @@ impl DedupService {
|
||||
));
|
||||
}
|
||||
|
||||
audit_ref_count(
|
||||
refcount_audit_table::BLOBS,
|
||||
hash,
|
||||
1,
|
||||
refcount_audit_source::ADD_REFERENCE_LEGACY,
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -2355,6 +2747,17 @@ impl DedupService {
|
||||
&file_hash[..12],
|
||||
chunk_hashes.len()
|
||||
);
|
||||
// Emit AFTER the commit so a rolled-back TX doesn't leave a
|
||||
// phantom audit line — the "delta = -current_rc" reflects
|
||||
// "the manifest is gone, effective ref_count is 0". Convention
|
||||
// for the audit stream: use the delta that would produce a
|
||||
// read-back of 0.
|
||||
audit_ref_count(
|
||||
refcount_audit_table::CHUNK_MANIFESTS,
|
||||
file_hash,
|
||||
-current_rc,
|
||||
refcount_audit_source::REMOVE_MANIFEST_REFERENCE_DELETE,
|
||||
);
|
||||
Ok(true)
|
||||
} else {
|
||||
// Still has references — just decrement
|
||||
@@ -2373,6 +2776,12 @@ impl DedupService {
|
||||
.map_err(|e| DomainError::internal_error("Dedup", format!("Commit: {}", e)))?;
|
||||
|
||||
tracing::debug!("Reference removed from manifest {}", &file_hash[..12]);
|
||||
audit_ref_count(
|
||||
refcount_audit_table::CHUNK_MANIFESTS,
|
||||
file_hash,
|
||||
-1,
|
||||
refcount_audit_source::REMOVE_MANIFEST_REFERENCE_DECREMENT,
|
||||
);
|
||||
Ok(false)
|
||||
}
|
||||
}
|
||||
@@ -2430,6 +2839,12 @@ impl DedupService {
|
||||
self.reap_blob(hash).await;
|
||||
|
||||
tracing::info!("BLOB DELETED: {} (no more references)", &hash[..12]);
|
||||
audit_ref_count(
|
||||
refcount_audit_table::BLOBS,
|
||||
hash,
|
||||
-ref_count,
|
||||
refcount_audit_source::REMOVE_LEGACY_REFERENCE,
|
||||
);
|
||||
Ok(true)
|
||||
} else {
|
||||
// Still has references — just decrement
|
||||
@@ -2450,6 +2865,12 @@ impl DedupService {
|
||||
})?;
|
||||
|
||||
tracing::debug!("Reference removed from blob {}", &hash[..12]);
|
||||
audit_ref_count(
|
||||
refcount_audit_table::BLOBS,
|
||||
hash,
|
||||
-1,
|
||||
refcount_audit_source::REMOVE_LEGACY_REFERENCE,
|
||||
);
|
||||
Ok(false)
|
||||
}
|
||||
}
|
||||
@@ -3920,6 +4341,138 @@ impl crate::infrastructure::scheduler::JobHandler for DedupService {
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
// ── attached_blob_cache — find_attached_blob read-through ───────────────
|
||||
//
|
||||
// Pure in-memory contract tests: `new_stub()` connects lazily to an
|
||||
// unreachable pool, so anything that reaches the "DB" fails loudly. That
|
||||
// is exactly what makes these work — a served `Some` proves the cache was
|
||||
// consulted, and a missing entry after a fault proves the fault was not
|
||||
// cached. Same no-SQL style as the hash_cache tests in
|
||||
// `file_blob_read_repository.rs`.
|
||||
|
||||
fn attached_key(file_id: &str, kind: &str, variant: &str) -> AttachedBlobKey {
|
||||
AttachedBlobKey::new(file_id, kind, variant)
|
||||
}
|
||||
|
||||
fn sample_ref(hash: &str) -> DerivedBlobRef {
|
||||
DerivedBlobRef {
|
||||
blob_hash: hash.to_string(),
|
||||
content_type: "image/jpeg".to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
/// A seeded entry is served without touching the (unreachable) stub pool
|
||||
/// — returning `Some` at all proves the read-through hit the cache.
|
||||
#[tokio::test]
|
||||
async fn attached_lookup_serves_a_seeded_entry() {
|
||||
let svc = DedupService::new_stub();
|
||||
let k = attached_key("0189d1b3-8f2a-7cde-b1ad-000000000001", "preview", "icon");
|
||||
svc.attached_blob_cache
|
||||
.insert(k.clone(), Some(sample_ref("abc")))
|
||||
.await;
|
||||
assert_eq!(
|
||||
svc.find_attached_blob(&k.file_id, "preview", "icon").await,
|
||||
Some(sample_ref("abc"))
|
||||
);
|
||||
}
|
||||
|
||||
/// Negative entries are where most of the win is (most files have no
|
||||
/// attached preview). A cached `None` must be served as `None` AND
|
||||
/// survive the call — not be evicted by the miss path.
|
||||
#[tokio::test]
|
||||
async fn attached_lookup_serves_and_keeps_a_negative_entry() {
|
||||
let svc = DedupService::new_stub();
|
||||
let k = attached_key("0189d1b3-8f2a-7cde-b1ad-000000000002", "preview", "icon");
|
||||
svc.attached_blob_cache.insert(k.clone(), None).await;
|
||||
assert_eq!(
|
||||
svc.find_attached_blob(&k.file_id, "preview", "icon").await,
|
||||
None
|
||||
);
|
||||
assert!(
|
||||
svc.attached_blob_cache.get(&k).await.is_some(),
|
||||
"negative entry was dropped by the lookup"
|
||||
);
|
||||
}
|
||||
|
||||
/// THE contract this change exists for: a DB fault must not be cached.
|
||||
/// The stub pool cannot connect, so the uncached lookup errors; the
|
||||
/// wrapper returns `None` (historical behaviour) and leaves the cache
|
||||
/// empty — a row that appears after a transient outage must be visible
|
||||
/// on the very next call, not hidden behind a frozen negative entry.
|
||||
#[tokio::test]
|
||||
async fn attached_lookup_does_not_cache_a_db_fault() {
|
||||
let svc = DedupService::new_stub();
|
||||
let k = attached_key("0189d1b3-8f2a-7cde-b1ad-000000000003", "preview", "icon");
|
||||
assert_eq!(
|
||||
svc.find_attached_blob(&k.file_id, "preview", "icon").await,
|
||||
None
|
||||
);
|
||||
assert!(
|
||||
svc.attached_blob_cache.get(&k).await.is_none(),
|
||||
"DB fault was cached as a negative entry"
|
||||
);
|
||||
}
|
||||
|
||||
/// Per-file invalidation drops every `(kind, variant)` of that file and
|
||||
/// leaves other files' entries alone.
|
||||
#[tokio::test]
|
||||
async fn invalidate_attached_blobs_for_file_is_scoped_to_the_file() {
|
||||
let svc = DedupService::new_stub();
|
||||
let k1 = attached_key("0189d1b3-8f2a-7cde-b1ad-000000000004", "preview", "icon");
|
||||
let k2 = attached_key("0189d1b3-8f2a-7cde-b1ad-000000000004", "preview", "large");
|
||||
let k3 = attached_key("0189d1b3-8f2a-7cde-b1ad-000000000005", "preview", "icon");
|
||||
for (k, v) in [
|
||||
(k1.clone(), Some(sample_ref("a"))),
|
||||
(k2.clone(), None),
|
||||
(k3.clone(), Some(sample_ref("c"))),
|
||||
] {
|
||||
svc.attached_blob_cache.insert(k, v).await;
|
||||
}
|
||||
|
||||
svc.invalidate_attached_blobs_for_file(&k1.file_id).await;
|
||||
|
||||
assert!(svc.attached_blob_cache.get(&k1).await.is_none());
|
||||
assert!(svc.attached_blob_cache.get(&k2).await.is_none());
|
||||
assert!(
|
||||
svc.attached_blob_cache.get(&k3).await.is_some(),
|
||||
"another file's entry must survive"
|
||||
);
|
||||
}
|
||||
|
||||
/// Invalidation happens only after a SUCCESSFUL write: the store path
|
||||
/// fails (unreachable pool) before any row is touched, so the previously
|
||||
/// cached entry must still be there. Invalidating on failure would be
|
||||
/// harmless but pointless — the row is unchanged and the cache accurate.
|
||||
#[tokio::test]
|
||||
async fn failed_attached_store_leaves_the_cache_alone() {
|
||||
let svc = DedupService::new_stub();
|
||||
let k = attached_key("0189d1b3-8f2a-7cde-b1ad-000000000006", "preview", "icon");
|
||||
svc.attached_blob_cache
|
||||
.insert(k.clone(), Some(sample_ref("xyz")))
|
||||
.await;
|
||||
|
||||
let result = svc
|
||||
.store_attached_blob(
|
||||
&k.file_id,
|
||||
"preview",
|
||||
"icon",
|
||||
"image/png",
|
||||
Bytes::from_static(b"nope"),
|
||||
uuid::Uuid::nil(),
|
||||
)
|
||||
.await;
|
||||
assert!(
|
||||
result.is_err(),
|
||||
"stub pool is unreachable — store must fail"
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
svc.attached_blob_cache.get(&k).await,
|
||||
Some(Some(sample_ref("xyz"))),
|
||||
"failed store must not disturb the cache"
|
||||
);
|
||||
}
|
||||
|
||||
/// Golden test for the statement `garbage_collect` runs against production
|
||||
/// data. It is assembled from the registered reference sources rather than
|
||||
/// written as a literal, so this pins the whole thing byte-for-byte — the
|
||||
|
||||
@@ -102,7 +102,23 @@ async fn fsync_paths_parallel(paths: Vec<PathBuf>, strict: bool) -> Result<(), D
|
||||
tasks.push(tokio::task::spawn_blocking(
|
||||
move || -> Result<(), (PathBuf, std::io::Error)> {
|
||||
for path in &group {
|
||||
let result = std::fs::File::open(path).and_then(|f| f.sync_all());
|
||||
// `strict` marks blob *file* fsyncs; best-effort marks
|
||||
// prefix *directory* fsyncs. That distinction also picks
|
||||
// the open mode: Windows `FlushFileBuffers` needs a
|
||||
// GENERIC_WRITE handle and fails with ACCESS_DENIED on
|
||||
// the read-only handle `File::open` returns (POSIX fsync
|
||||
// accepts read-only fds, which is why this only surfaced
|
||||
// on Windows). Directories keep the read-only POSIX
|
||||
// dirent-sync idiom — they can't be fsync'd on Windows
|
||||
// at all, and their failures stay best-effort warnings.
|
||||
let result = if strict {
|
||||
std::fs::OpenOptions::new()
|
||||
.write(true)
|
||||
.open(path)
|
||||
.and_then(|f| f.sync_all())
|
||||
} else {
|
||||
std::fs::File::open(path).and_then(|f| f.sync_all())
|
||||
};
|
||||
if let Err(e) = result {
|
||||
if strict {
|
||||
return Err((path.clone(), e));
|
||||
@@ -394,7 +410,11 @@ impl BlobStorageBackend for LocalBlobBackend {
|
||||
format!("Failed to copy file to blob store: {}", ce),
|
||||
)
|
||||
})?;
|
||||
if let Ok(f) = fs::File::open(&blob_path).await {
|
||||
// Open for write: Windows `FlushFileBuffers` requires a
|
||||
// GENERIC_WRITE handle — the read-only handle from
|
||||
// `File::open` fails with ACCESS_DENIED, silently
|
||||
// skipping this fsync on every Windows deployment.
|
||||
if let Ok(f) = fs::OpenOptions::new().write(true).open(&blob_path).await {
|
||||
let _ = f.sync_all().await;
|
||||
}
|
||||
let _ = fs::remove_file(&source_path).await;
|
||||
|
||||
@@ -274,49 +274,31 @@ impl RecoverableJobHandler for ThumbAttachedImport {
|
||||
};
|
||||
let file_id_str = file_id.to_string();
|
||||
|
||||
// Already mapped. Checked BEFORE storing, because
|
||||
// `store_attached_blob` is ON CONFLICT DO UPDATE and would
|
||||
// release then retake the reference on every run.
|
||||
if let Some(existing) = self
|
||||
.dedup
|
||||
.find_attached_blob(&file_id_str, "preview", &dir_name)
|
||||
.await
|
||||
{
|
||||
already += 1;
|
||||
// Drains on a later run too: importing first and enabling
|
||||
// deletion afterwards is the expected operator sequence,
|
||||
// so reaching here is the common path rather than an edge
|
||||
// case.
|
||||
if delete_imported {
|
||||
let path = self.thumbnails_root.join(&dir_name).join(&name);
|
||||
if ThumbDerivedImport::verify_and_unlink(
|
||||
&self.dedup,
|
||||
THUMB_ATTACHED_IMPORT_JOB_NAME,
|
||||
&file_id_str,
|
||||
&existing.blob_hash,
|
||||
&path,
|
||||
)
|
||||
.await
|
||||
{
|
||||
deleted += 1;
|
||||
} else {
|
||||
unverified += 1;
|
||||
record_or_log(
|
||||
store,
|
||||
THUMB_ATTACHED_IMPORT_JOB_NAME,
|
||||
"sidecar_delete_unverified",
|
||||
"anomaly",
|
||||
None,
|
||||
serde_json::json!({
|
||||
"path": position,
|
||||
"file_id": file_id_str,
|
||||
"note": "attached blob did not read back; sidecar kept",
|
||||
}),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
}
|
||||
} else if !self.file_exists(file_id).await {
|
||||
// Orphan check first — no atomic-insert exists for a
|
||||
// file_id whose FK would reject. Same rationale as before;
|
||||
// the race window between this check and the INSERT is
|
||||
// narrow AND covered by the FK constraint if the file is
|
||||
// deleted after we look — the atomic INSERT would then
|
||||
// fail loudly instead of silently drift.
|
||||
//
|
||||
// Everything else — "row present" and "row absent" —
|
||||
// used to be split across two branches with a
|
||||
// non-transactional `find_attached_blob` between the
|
||||
// check and the write. That opened a check-then-act
|
||||
// race window: a concurrent thumbnail writer could
|
||||
// INSERT the row after the check returned None, and the
|
||||
// subsequent `store_attached_blob` UPSERT-UPDATE would
|
||||
// fire with same-or-different content. In the
|
||||
// same-content case that leaked +1 on the manifest ref
|
||||
// (pre-fix; guard branch now cancels).
|
||||
//
|
||||
// Merged into ONE atomic call
|
||||
// `store_attached_blob_if_absent`: single-statement
|
||||
// `INSERT ... ON CONFLICT DO NOTHING`, race-free by
|
||||
// construction. The outcome enum distinguishes the two
|
||||
// paths so `imported` and `already` counters stay
|
||||
// accurate.
|
||||
if !self.file_exists(file_id).await {
|
||||
// The file is gone, so this sidecar is unimportable: the
|
||||
// FK on `file_id` would reject the row. Mirrors the
|
||||
// dead-source case in thumb_derived_import.
|
||||
@@ -389,9 +371,10 @@ impl RecoverableJobHandler for ThumbAttachedImport {
|
||||
let path = self.thumbnails_root.join(&dir_name).join(&name);
|
||||
match fs::read(&path).await {
|
||||
Ok(data) => {
|
||||
use crate::infrastructure::services::dedup_service::AttachedBlobInsertOutcome;
|
||||
match self
|
||||
.dedup
|
||||
.store_attached_blob(
|
||||
.store_attached_blob_if_absent(
|
||||
&file_id_str,
|
||||
"preview",
|
||||
&dir_name,
|
||||
@@ -404,14 +387,39 @@ impl RecoverableJobHandler for ThumbAttachedImport {
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(attached_hash) => {
|
||||
Ok(outcome) => {
|
||||
// Bump the right counter AND pick the
|
||||
// hash we'll verify-and-unlink against:
|
||||
// Inserted — our new blob
|
||||
// AlreadyPresent — the concurrent
|
||||
// winner's blob
|
||||
// Both drain the sidecar identically
|
||||
// (verify-then-unlink on repair mode).
|
||||
let verify_hash = match outcome {
|
||||
AttachedBlobInsertOutcome::Inserted { hash } => {
|
||||
imported += 1;
|
||||
if delete_imported {
|
||||
hash
|
||||
}
|
||||
AttachedBlobInsertOutcome::AlreadyPresent {
|
||||
existing_hash,
|
||||
} => {
|
||||
already += 1;
|
||||
existing_hash
|
||||
}
|
||||
};
|
||||
// Empty existing_hash only happens if
|
||||
// the AlreadyPresent path's follow-up
|
||||
// SELECT was overtaken by another
|
||||
// writer. verify_and_unlink would
|
||||
// refuse the sidecar delete in that
|
||||
// case anyway, but skipping the call
|
||||
// saves the pointless readback.
|
||||
if delete_imported && !verify_hash.is_empty() {
|
||||
if ThumbDerivedImport::verify_and_unlink(
|
||||
&self.dedup,
|
||||
THUMB_ATTACHED_IMPORT_JOB_NAME,
|
||||
&file_id_str,
|
||||
&attached_hash,
|
||||
&verify_hash,
|
||||
&path,
|
||||
)
|
||||
.await
|
||||
|
||||
@@ -1874,10 +1874,16 @@ impl crate::application::ports::file_lifecycle::FileLifecycleHook for ThumbnailR
|
||||
fn on_file_deleted(&self, file_id: &str) {
|
||||
let thumbnail = self.thumbnail.clone();
|
||||
let file_id = file_id.to_string();
|
||||
// The row is gone (CASCADE cleared file_attached_blobs) — drop any
|
||||
// cached attached-blob lookup for this file too. TTL would bound the
|
||||
// staleness anyway, but deletes are rare and the cache lookup after a
|
||||
// delete is pure waste.
|
||||
let dedup = self.dedup.clone();
|
||||
tokio::spawn(async move {
|
||||
if let Err(e) = thumbnail.delete_thumbnails(&file_id).await {
|
||||
tracing::warn!("Failed to delete thumbnails for file {}: {}", file_id, e);
|
||||
}
|
||||
dedup.invalidate_attached_blobs_for_file(&file_id).await;
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -242,7 +242,7 @@ pub async fn access_shared_item(
|
||||
// every public share landing).
|
||||
let (_, item) = tokio::join!(
|
||||
share_use_case.register_shared_link_access(&token),
|
||||
share_use_case.get_shared_link_with_unlock(&token, unlock_jwt.as_deref()),
|
||||
share_use_case.get_shared_link_meta_with_unlock(&token, unlock_jwt.as_deref()),
|
||||
);
|
||||
|
||||
match item {
|
||||
@@ -639,7 +639,7 @@ pub async fn list_share_contents_subfolder(
|
||||
path = "/api/s/{token}/file/{file_id}",
|
||||
params(
|
||||
("token" = String, Path, description = "Share token"),
|
||||
("file_id" = String, Path, description = "File ID (must be inside the share)")
|
||||
("file_id" = String, Path, description = "File ID (the shared item itself, or a file inside the shared folder's subtree)")
|
||||
),
|
||||
responses(
|
||||
(status = 200, description = "File content (or 206 for Range request)"),
|
||||
|
||||
@@ -94,6 +94,19 @@ pub async fn basic_auth_middleware(
|
||||
let (raw_username, password) =
|
||||
parse_basic_auth(auth_header).ok_or(NextcloudAuthError::Unauthorized)?;
|
||||
|
||||
// Canonicalise the whole Basic-Auth username to lowercase.
|
||||
//
|
||||
// Usernames are canonical (lowercase) in the DB post-migration
|
||||
// (`docs/plan/username-lowercase.md`), and NC / DAVX5 clients that
|
||||
// cached URLs from before the migration keep sending `Alice:pass`
|
||||
// — the server continues to accept that indefinitely by
|
||||
// lowercasing here. Safe for the multi-drive `user~drive_uuid`
|
||||
// composite because UUID hex `[0-9a-f-]` lowercases to itself.
|
||||
//
|
||||
// ASCII-only by `validate_username`'s charset check, so
|
||||
// `to_ascii_lowercase` is deterministic and locale-safe.
|
||||
let raw_username = raw_username.to_ascii_lowercase();
|
||||
|
||||
// ── Multi-drive composite-username parse ────────────────────────
|
||||
// POC wire shape: `{username}~{drive_marker}` may appear in the
|
||||
// Basic Auth header. `~` was chosen because it needs no URL
|
||||
@@ -319,7 +332,13 @@ pub fn parse_basic_auth(header_value: &str) -> Option<(String, String)> {
|
||||
let decoded = String::from_utf8(decoded).ok()?;
|
||||
let (user, pass) = decoded.split_once(':')?;
|
||||
|
||||
Some((user.to_string(), pass.to_string()))
|
||||
// Canonicalise the username to lowercase here too, so any caller
|
||||
// that reaches for `parse_basic_auth` directly (bypassing the
|
||||
// middleware wrapper) also sees the canonical form. Redundant with
|
||||
// the middleware's explicit `to_ascii_lowercase` on `raw_username`
|
||||
// — belt-and-braces to keep the invariant local to the parser too.
|
||||
// See `docs/plan/username-lowercase.md § 4. NextCloud DAV surface`.
|
||||
Some((user.to_ascii_lowercase(), pass.to_string()))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
||||
@@ -107,7 +107,21 @@ fn extract_url_user(path: &str) -> Option<std::borrow::Cow<'_, str>> {
|
||||
// common path allocates nothing; only a percent-encoded username owns. The
|
||||
// old `.into_owned()` forced a `String` on EVERY path-scoped NC DAV request
|
||||
// (benches/ROUND19.md §M7). The caller compares by slice.
|
||||
urlencoding::decode(user_seg).ok()
|
||||
//
|
||||
// Lowercase before returning so cached client URLs like
|
||||
// `/dav/files/Alice/...` compare equal to the canonical
|
||||
// (lowercase) `session.raw_username`. See
|
||||
// `docs/plan/username-lowercase.md § 4. NextCloud DAV surface`.
|
||||
//
|
||||
// The lowercase transform always allocates (`to_ascii_lowercase`
|
||||
// on a `str` returns `String`). Trades the "Cow::Borrowed common
|
||||
// path" of the ROUND19 optimisation for correctness of the case-
|
||||
// insensitive comparison at line 157 — a `&str` compare with a
|
||||
// borrowed segment against a lowercase `session.raw_username`
|
||||
// would silently mismatch for `Alice`. The alloc is one small
|
||||
// String per NC DAV request; the correctness win is worth it.
|
||||
let decoded = urlencoding::decode(user_seg).ok()?;
|
||||
Some(std::borrow::Cow::Owned(decoded.to_ascii_lowercase()))
|
||||
}
|
||||
|
||||
/// Axum extractor: the shared handle to the request's [`NcSession`].
|
||||
|
||||
@@ -100,6 +100,10 @@ fn make_socket(addr: &SocketAddr, reuse_port: bool) -> std::io::Result<Socket> {
|
||||
if reuse_port {
|
||||
socket.set_reuse_port(true)?;
|
||||
}
|
||||
// SO_REUSEPORT is Unix-only; on Windows the flag is accepted but inert.
|
||||
// Consume the parameter so `-D warnings` stays clean on Windows builds.
|
||||
#[cfg(windows)]
|
||||
let _ = reuse_port;
|
||||
// Disable Nagle's algorithm — send small responses (JSON, PROPFIND)
|
||||
// immediately instead of waiting up to 40ms for coalescing.
|
||||
socket.set_tcp_nodelay(true)?;
|
||||
|
||||
@@ -0,0 +1,103 @@
|
||||
# status.md — 计划与进度记录
|
||||
|
||||
> 本文件由 agent 自动维护，规则见 [AGENTS.md](AGENTS.md) 末尾的"本地 fork 维护规则"一节。
|
||||
> 新条目加在"进行中"区域顶部；完成后移入"已完成"。
|
||||
|
||||
## 进行中
|
||||
|
||||
（暂无）
|
||||
|
||||
## 已完成
|
||||
|
||||
### [2026-09-19] 缩略图路径 DB 点查缓存（find_attached_blob 进程内缓存）
|
||||
- **状态**: 已完成（commit `d33d1932`；cargo fmt --check ✓；clippy --all-features --all-targets -D warnings 0 警告 ✓；`cargo test --lib` 927 通过 0 失败，含新增 5 个缓存契约测试 ✓）
|
||||
- **计划**: 给缩略图热路径（ETag 计算 `thumbnail_content_id` + tier 2b）每次请求都要打的 `find_attached_blob` DB 点查加进程内缓存（moka::future + try_get_with，正+负缓存，Err 不入缓存；写路径成功后失效；删除经 `ThumbnailRefreshHook::on_file_deleted` 搭车失效 + 60s TTL 兜底）。缓解"每次进照片墙 = 每张可见图 1-2 次 DB 点查"的负载。
|
||||
- **改动文件**:
|
||||
- `src/infrastructure/services/dedup_service.rs` — 模块顶 `ATTACHED_BLOB_CACHE_*` 常量 + `AttachedBlobKey` + `AttachedLookupFault`；`attached_blob_cache` 字段/`build_attached_blob_cache`；`find_attached_blob` 缓存包装（SQL 下移 `find_attached_blob_uncached` 返回 `sqlx::Result`，Err 永不入缓存）；`store_attached_blob`/`store_attached_blob_if_absent` 成功路径失效（`Inserted` 臂失效 + AlreadyPresent 读回注释）；`invalidate_attached_blobs_for_file`；5 个纯内存单测
|
||||
- `src/infrastructure/services/thumbnail_service.rs` — `ThumbnailRefreshHook::on_file_deleted` 搭车失效（补 `self.dedup.clone()`）
|
||||
- `docs/architecture/caching.md` — Layer 1 表新增 Attached blob 行 + "The attached-blob cache" 小节（键/正负缓存/两条诚实规则）
|
||||
- `docs/architecture/derived-and-attached-blobs.md` — Lifecycle 节新增"Reads are cached"段落（缓存失效协议）
|
||||
- **仅本地文件**: 无新增（`status.md` 本身）
|
||||
- **上游冲突风险**: 低 — 两个上游文件均为局部追加，无重排
|
||||
- **备注**: 本机此前无 Rust 工具链，本次顺带装好 rustup stable 1.98.1（minimal+clippy+rustfmt）、VS Build Tools 2022（MSVC 14.44 + SDK 10.0.26100）、Node 前端构建（`static-dist/` 已生成）——后续 agent 可直接跑 `cargo` 检查；注意 shell 无管理员权限，提权操作需 UAC 确认
|
||||
|
||||
### [2026-09-19] 文档更新：公开分享端点与落地页行为
|
||||
- **状态**: 已完成
|
||||
- **计划**: 按 AGENTS.md 的文档约定，把本会话的分享相关修复/功能同步进架构文档与用户指南
|
||||
- **改动文件**:
|
||||
- `docs/architecture/share-integration.md` — 公开路由表补全（download/contents/file/zip 共 6 条）；新增 "/file/{file_id} 文件作用域"（file 分享仅限分享项本身、folder 分享限子树、其余 404 反枚举）与"落地页 meta 富化"（mime_type/size 仅展示、查询失败不失败响应）两小节
|
||||
- `docs/guide/sharing.md` — 新增 "What recipients see" 用户向小节（单文件内联预览/视频流式拖动、文件夹浏览 + ZIP）
|
||||
- **上游冲突风险**: 低 — 两文档均属低频改动区
|
||||
|
||||
### [2026-09-19] 文件页刷新按钮
|
||||
- **状态**: 已完成（vitest 467 通过；svelte-check 0 错误；我改的文件 prettier/eslint 全绿）
|
||||
- **计划**: actions 工具栏加手动刷新，点击重拉当前文件夹列表
|
||||
- **改动文件**:
|
||||
- `frontend/src/routes/files/[...path]/+page.svelte` — "New folder" 右侧新增刷新按钮：`load(true)` 重置分页拉第 1 页，`loading` 时禁用，图标 `repeat`（沿用 AdminJobsPanel 先例），testid `files-refresh-btn`
|
||||
- `frontend/static/locales/{en,zh,zh-TW}.json` — `common.refresh`（刷新 / 重新整理）
|
||||
- `frontend/src/routes/files/page.test.ts` — 点击刷新 → `fetchFolderPage` 第二次调用
|
||||
- **上游冲突风险**: 中 — files 页与 locales 是上游活跃区，但改动面小
|
||||
|
||||
### [2026-09-19] 文件页文件夹内容统计
|
||||
- **状态**: 已完成（验证同上）
|
||||
- **计划**: 进入文件夹时在面包屑旁显示内容统计
|
||||
- **改动文件**:
|
||||
- `frontend/src/routes/files/[...path]/+page.svelte` — `folderStat` derived（对 `rlItems` 即点文件过滤后的展示列表计数；分页未完时数字尾随 "+"）；统计 span 渲染在 breadcrumb snippet 内（`.rl-breadcrumb` 本身是 flex）；空文件夹不显示
|
||||
- `frontend/static/locales/{en,zh,zh-TW}.json` — `files.folder_stat`（其余 13 语言回退英文）
|
||||
- `frontend/src/routes/files/page.test.ts` — 精确计数 / 分页 "+" 两条测试
|
||||
- **上游冲突风险**: 中 — 同上
|
||||
|
||||
### [2026-09-19] 开发环境修复 + 测试 locale 固定
|
||||
- **状态**: 已完成
|
||||
- **计划**: 本机装 Node 跑前端检查；修复暴露出的换行符与 locale 问题
|
||||
- **改动文件**:
|
||||
- `frontend/src/lib/utils/time.test.ts` — `vi.spyOn(Intl, 'RelativeTimeFormat')` 固定 `en` locale（time.ts 用运行时默认 locale，中文系统上 `/second/` 等英文断言必挂）；注意 mock 需普通函数（time.ts 经 `new` 调用）
|
||||
- 环境级（无仓库 diff）: winget 装 Node 26.7.0；`core.autocrlf=false`（仓库级）+ `git rm --cached -r . && git reset --hard` 全量重写工作区为 LF——修复 276 个文件的 Prettier 假报错
|
||||
- **仅本地文件**: 无新增
|
||||
- **上游冲突风险**: 低 — LF 归一化后与上游（CI 全 LF）一致；建议后续给 `.gitattributes` 加 `* text=auto eol=lf`（未做，待定）
|
||||
|
||||
### [2026-09-19] 文件页模糊筛选搜索 + 批量操作共享化
|
||||
- **状态**: 已完成（`npm run check` 0 错误 0 警告；`vitest run` 467 个测试全部通过）
|
||||
- **计划**:
|
||||
1. 文件页（`/files/[...path]`）新增搜索过滤栏：关键词（防抖）+ 类型/大小/时间预设 + 递归开关，
|
||||
筛选生效时列表切换为当前文件夹的递归搜索（`GET /api/search` 的 `folder_id`+`recursive`，
|
||||
空 `query` 后端视为匹配全部），支持结果多选/全选 + 批量收藏/移动/复制/下载/删除
|
||||
2. 批量操作提取为共享 composable，`/search` 结果页同步接入选择 + 批量操作（项目去重规范）
|
||||
3. 两处页内裸 `apiFetch`（favorites/batch、batch/download）改为 endpoint 封装
|
||||
- **改动文件**:
|
||||
- 新增（仅本地）:
|
||||
- `frontend/src/lib/components/SearchFilterBar.svelte` — 搜索过滤栏组件（关键词 + 高级筛选 + 递归开关）
|
||||
- `frontend/src/lib/composables/useResourceActions.svelte.ts` — 共享批量操作（收藏/下载/删除/移动/复制 + MoveDialog 状态）
|
||||
- `frontend/src/lib/utils/searchFilters.ts` — 类型/大小/时间预设模型 → `SearchOptions` 映射
|
||||
- `frontend/src/lib/utils/mapLimit.ts` — 有界并发工具（自文件页提取）
|
||||
- 以上各文件的 Vitest 测试（`*.test.ts`）
|
||||
- 修改（上游文件）:
|
||||
- `frontend/src/routes/files/[...path]/+page.svelte` — 搜索模式状态/`runSearch`/模式切换/Escape 优先级/深链兼容；批量 handler 换用 composable
|
||||
- `frontend/src/routes/search/+page.svelte` — 接入选择 + 批量操作；筛选逻辑改用共享 searchFilters；MoveDialog 绑定 composable
|
||||
- `frontend/src/lib/api/endpoints/batch.ts` — 新增 `downloadBatch()`
|
||||
- `frontend/src/lib/api/endpoints/favorites.ts` — 新增 `addFavoritesBatch()`
|
||||
- `frontend/src/routes/files/page.test.ts`、`frontend/src/routes/search/page.test.ts` — mock 补齐 + 新增搜索模式/批量删除测试
|
||||
- `frontend/static/locales/*.json`（16 个语言文件）— 新增 `filter.*` 6 个 key（zh/zh-TW 为真实翻译）
|
||||
- **仅本地文件**: `status.md`、上述新增的 4 个源文件及其测试
|
||||
- **上游冲突风险**: 高 — locales 与两个页面文件是上游活跃区；页面文件改动较大（files 页 ~490 行、search 页 ~170 行 diff），合并时需逐块核对本地意图
|
||||
- **设计要点**（合并上游时用于核对行为）:
|
||||
- 批量操作参数化接口：`getItems/getSelected/clearSelection/onChanged/afterDelete` 回调注入；
|
||||
文件页 `getItems` 按模式切换（`searchActive ? searchItems : orderedItems`），`onChanged` 按模式重跑搜索或重载目录
|
||||
- 删除保持 per-item `mapLimit(ids, 6)` 行为（后端 `POST /api/batch/trash` 批量软删存在，切换记为可选后续）
|
||||
- 搜索分页 limit 50；"全选"只覆盖已加载页（与 ResourceList 既有语义一致）
|
||||
- 文件页搜索 wire 层 `type` 排序映射为 `name`（`SortBy` 无 type），`modified_at` 映射为 `updated_at`；始终传 sortBy 不传 relevance
|
||||
- Escape 优先级：清除选中 → 清除筛选；过滤输入框内 Escape 自行处理并 stopPropagation
|
||||
|
||||
### [2026-09-15] 单文件分享无法流式预览修复
|
||||
- **状态**: 已完成（commit `68e21f4b`；本机无 Rust 环境，fmt/clippy/api-test 未能在本地运行——**push 前需在有 Rust 的环境跑 `just check` + `just api-test`**）
|
||||
- **计划**: 修复单文件分享落地页预览窗口无内容/按钮失效——根因是 `resolve_folder_share` 对 `item_type != "folder"` 一律拒绝，`/api/s/{token}/file/{id}` 对 file 分享返回 400
|
||||
- **改动文件**:
|
||||
- `src/application/services/share_browse_service.rs` — `assert_file_in_share` 改为按 `item_type` 分支：file 分享仅接受 `file_id == share.item_id`；folder 分享维持 ltree 子树校验；其余一律 404（反枚举，保持与"文件不存在"同形）
|
||||
- `src/interfaces/api/handlers/share_handler.rs` — OpenAPI `file_id` 参数描述同步（"the shared item itself, or a file inside the shared folder's subtree"）
|
||||
- `tests/api/public_shares.hurl` — 8b 节扩展：file 分享 token 流式取自身 item（200 + disposition）+ 局外人 id 404 断言
|
||||
- **仅本地文件**: 无
|
||||
- **上游冲突风险**: 中 — `share_browse_service.rs` 属上游安全敏感活跃区；改动集中在单个函数，冲突时按"file 分享限自身、folder 限子树、其余 404"核对意图
|
||||
|
||||
## 上游合并记录
|
||||
|
||||
（暂无。首次合并前先 `git remote add upstream <上游仓库地址>`。）
|
||||
@@ -0,0 +1,160 @@
|
||||
# =============================================================
|
||||
# OxiCloud — usernames are case-insensitive (silent lowercase on ingest)
|
||||
# =============================================================
|
||||
# Pin for `docs/plan/username-lowercase.md`. The plan makes usernames
|
||||
# case-insensitive by canonicalising to lowercase in
|
||||
# `User::validate_username`. This file covers the wire surface:
|
||||
#
|
||||
# 1. Registration with a mixed-case username lands as lowercase.
|
||||
# 2. Login by the ORIGINAL mixed-case string succeeds (server
|
||||
# normalises on lookup).
|
||||
# 3. Login by ALL-CAPS of the same name also succeeds.
|
||||
# 4. Login by the canonical lowercase form succeeds.
|
||||
# 5. Profile rename to a mixed-case name lands as lowercase.
|
||||
# 6. NC Basic Auth accepts every case variant of the same account.
|
||||
#
|
||||
# Character-set + boundary rules (no leading dot, no `@`, etc.) are
|
||||
# NOT re-tested here — that's the Rust unit-test surface. This file
|
||||
# only pins the end-to-end normalization behaviour.
|
||||
#
|
||||
# Runs after `setup.hurl` (admin exists). Uses a self-contained user
|
||||
# to avoid interfering with other scenarios.
|
||||
# =============================================================
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# Step 1 — Register with a mixed-case username. Expect the server
|
||||
# to silently store the lowercase form.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
POST {{base_url}}/api/auth/register
|
||||
Content-Type: application/json
|
||||
{
|
||||
"username": "MixedCaseUser",
|
||||
"email": "mixedcase@example.com",
|
||||
"password": "MixedCasePassword1!"
|
||||
}
|
||||
|
||||
HTTP 200
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# Step 2 — Log in with the ORIGINAL mixed-case string. Server
|
||||
# should normalise on lookup and accept.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
POST {{base_url}}/api/auth/login
|
||||
Content-Type: application/json
|
||||
{ "username": "MixedCaseUser", "password": "MixedCasePassword1!" }
|
||||
|
||||
HTTP 200
|
||||
[Asserts]
|
||||
jsonpath "$.access_token" isString
|
||||
# The `/auth/me` response inside the login reply exposes the canonical
|
||||
# stored username. Post-migration, it MUST be lowercase regardless of
|
||||
# what the caller typed at registration.
|
||||
jsonpath "$.user.full.user.username" == "mixedcaseuser"
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# Step 3 — Log in with ALL-CAPS. Same account, different casing.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
POST {{base_url}}/api/auth/login
|
||||
Content-Type: application/json
|
||||
{ "username": "MIXEDCASEUSER", "password": "MixedCasePassword1!" }
|
||||
|
||||
HTTP 200
|
||||
[Asserts]
|
||||
jsonpath "$.user.full.user.username" == "mixedcaseuser"
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# Step 4 — Log in with the canonical lowercase form. Same account.
|
||||
# Capture the token for the profile-rename step below.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
POST {{base_url}}/api/auth/login
|
||||
Content-Type: application/json
|
||||
{ "username": "mixedcaseuser", "password": "MixedCasePassword1!" }
|
||||
|
||||
HTTP 200
|
||||
[Captures]
|
||||
mixed_token: jsonpath "$.access_token"
|
||||
[Asserts]
|
||||
jsonpath "$.user.full.user.username" == "mixedcaseuser"
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# Step 5 — Rename via profile PATCH. New name is mixed-case; server
|
||||
# must store it as lowercase. Same rule as registration, applied on
|
||||
# the mutation path.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
PATCH {{base_url}}/api/auth/me
|
||||
Authorization: Bearer {{mixed_token}}
|
||||
Content-Type: application/json
|
||||
{ "username": "RenamedTarget" }
|
||||
|
||||
HTTP 200
|
||||
[Asserts]
|
||||
# The response echoes the stored (canonical) form.
|
||||
jsonpath "$.full.user.username" == "renamedtarget"
|
||||
|
||||
|
||||
# Old (pre-rename) username no longer resolves — login fails 401.
|
||||
POST {{base_url}}/api/auth/login
|
||||
Content-Type: application/json
|
||||
{ "username": "MixedCaseUser", "password": "MixedCasePassword1!" }
|
||||
|
||||
HTTP 401
|
||||
|
||||
|
||||
# New (post-rename) mixed-case login succeeds.
|
||||
POST {{base_url}}/api/auth/login
|
||||
Content-Type: application/json
|
||||
{ "username": "RENAMEDTARGET", "password": "MixedCasePassword1!" }
|
||||
|
||||
HTTP 200
|
||||
[Asserts]
|
||||
jsonpath "$.user.full.user.username" == "renamedtarget"
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# Step 6 — NextCloud Basic Auth accepts every case variant.
|
||||
# The middleware lowercases the decoded username on the auth path,
|
||||
# and `extract_url_user` lowercases the URL segment. A cached
|
||||
# client URL like `.../dav/files/RenamedTarget` continues to work
|
||||
# indefinitely across the migration.
|
||||
#
|
||||
# Uses PROPFIND `Depth: 0` on `/remote.php/dav/files/{user}/` — a
|
||||
# well-formed request that touches Basic Auth + URL parse + chroot
|
||||
# resolve in one hop. 207 Multi-Status is the expected success shape.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
PROPFIND {{base_url}}/remote.php/dav/files/renamedtarget/
|
||||
Depth: 0
|
||||
[BasicAuth]
|
||||
renamedtarget: MixedCasePassword1!
|
||||
|
||||
HTTP 207
|
||||
|
||||
|
||||
PROPFIND {{base_url}}/remote.php/dav/files/RenamedTarget/
|
||||
Depth: 0
|
||||
[BasicAuth]
|
||||
RenamedTarget: MixedCasePassword1!
|
||||
|
||||
HTTP 207
|
||||
|
||||
|
||||
PROPFIND {{base_url}}/remote.php/dav/files/RENAMEDTARGET/
|
||||
Depth: 0
|
||||
[BasicAuth]
|
||||
RENAMEDTARGET: MixedCasePassword1!
|
||||
|
||||
HTTP 207
|
||||
|
||||
|
||||
# Mixed case in URL, lowercase in Basic Auth — still works because
|
||||
# both surfaces normalise before comparison.
|
||||
PROPFIND {{base_url}}/remote.php/dav/files/RenamedTarget/
|
||||
Depth: 0
|
||||
[BasicAuth]
|
||||
renamedtarget: MixedCasePassword1!
|
||||
|
||||
HTTP 207
|
||||
@@ -197,6 +197,24 @@ HTTP 200
|
||||
jsonpath "$.item_type" == "file"
|
||||
|
||||
|
||||
# The public landing page's inline media preview streams the shared file
|
||||
# through /api/s/{token}/file/{file_id} — the requested file IS the shared
|
||||
# item here, so the AuthZ gate must accept it (Range-aware 200, inline
|
||||
# disposition so <video>/<img> can render it).
|
||||
GET {{base_url}}/api/s/{{file_share_token}}/file/{{shared_file_id}}
|
||||
|
||||
HTTP 200
|
||||
[Asserts]
|
||||
header "Content-Disposition" contains "hello.txt"
|
||||
|
||||
|
||||
# A file id that is NOT the shared item must 404 on a file-share token —
|
||||
# same anti-enumeration shape as the folder-share probe above.
|
||||
GET {{base_url}}/api/s/{{file_share_token}}/file/{{outsider_file_id}}
|
||||
|
||||
HTTP 404
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# 9 — Mint a password-protected share on the same folder.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -254,6 +254,71 @@ ATTACHED_2=$(sql "SELECT count(*) FROM storage.file_attached_blobs WHERE file_id
|
||||
[[ "$ATTACHED_2" == "$ATTACHED_AFTER" ]] || fail "re-run duplicated attached rows"
|
||||
log "re-run is a no-op: rows and refcounts unchanged."
|
||||
|
||||
# ── 5c. store_attached_blob same-content guard (regression test) ─────────────
|
||||
#
|
||||
# The PUT /api/files/{id}/thumbnail/{size} endpoint calls
|
||||
# `dedup_service::store_attached_blob` directly — no pre-check like
|
||||
# `thumb_attached_import_service` does. A same-content re-PUT is the
|
||||
# ONLY current public surface that exercises the ref-balance branch
|
||||
# added to `store_attached_blob` after the Sept-2026 manifest-drift
|
||||
# investigation:
|
||||
#
|
||||
# if let Some((old_hash,)) = previous {
|
||||
# // same-content: cancel `store_from_stream`'s spurious +1
|
||||
# // different-hash: release the superseded blob's ref
|
||||
# remove_reference(&old_hash)
|
||||
# }
|
||||
#
|
||||
# Before the fix, same-content re-PUT would leak +1 on the manifest's
|
||||
# ref_count every time (store_from_stream incremented, guard skipped
|
||||
# the decrement when old == new). This test PUTs the same thumbnail
|
||||
# bytes twice and asserts the manifest's ref_count is unchanged.
|
||||
#
|
||||
# Uses variant "icon" so we don't collide with the row the import
|
||||
# job populated above (variant "preview"), keeping the two flows
|
||||
# independent. Server re-encodes to JPEG deterministically, so both
|
||||
# PUTs produce byte-identical manifest content.
|
||||
|
||||
log "5c. Same-content re-PUT via API does not churn refcount"
|
||||
|
||||
# Round 1: first PUT populates the row (INSERT — previous=None, guard
|
||||
# doesn't fire). This is the fresh-ingest path; ref_count becomes 1.
|
||||
curl -sf -X PUT \
|
||||
-H "$AUTH" \
|
||||
-H "Content-Type: image/jpeg" \
|
||||
--data-binary "@$UPLOADED_THUMB" \
|
||||
"$base_url/api/files/$FILE_ID/thumbnail/icon" \
|
||||
>/dev/null || fail "5c: first PUT of thumbnail (variant=icon) failed"
|
||||
|
||||
GUARD_HASH=$(sql "SELECT blob_hash FROM storage.file_attached_blobs \
|
||||
WHERE file_id='$FILE_ID' AND kind='preview' AND variant='icon' \
|
||||
LIMIT 1;")
|
||||
[[ -n "$GUARD_HASH" ]] || fail "5c: first PUT did not land a file_attached_blobs row"
|
||||
|
||||
GUARD_REFS_BEFORE=$(sql "SELECT ref_count FROM storage.chunk_manifests WHERE file_hash='$GUARD_HASH';")
|
||||
# Legacy blobs (pre-CDC) don't have a chunk_manifests row — skip
|
||||
# the test in that case rather than fail on an unrelated path.
|
||||
if [[ -z "$GUARD_REFS_BEFORE" ]]; then
|
||||
log "5c: attached blob is on the legacy path (no manifest) — guard test skipped (targets CDC path)"
|
||||
else
|
||||
# Round 2: second PUT with the SAME bytes. UPSERT-UPDATE fires,
|
||||
# previous.blob_hash == new attached_hash, and the guard MUST
|
||||
# cancel `store_from_stream`'s +1. Without the fix, refcount
|
||||
# would go from 1 → 2 here.
|
||||
curl -sf -X PUT \
|
||||
-H "$AUTH" \
|
||||
-H "Content-Type: image/jpeg" \
|
||||
--data-binary "@$UPLOADED_THUMB" \
|
||||
"$base_url/api/files/$FILE_ID/thumbnail/icon" \
|
||||
>/dev/null || fail "5c: same-content re-PUT of thumbnail failed"
|
||||
|
||||
GUARD_REFS_AFTER=$(sql "SELECT ref_count FROM storage.chunk_manifests WHERE file_hash='$GUARD_HASH';")
|
||||
[[ "$GUARD_REFS_AFTER" == "$GUARD_REFS_BEFORE" ]] \
|
||||
|| fail "5c: same-content re-PUT churned refcount: $GUARD_REFS_BEFORE → $GUARD_REFS_AFTER (guard regressed?)"
|
||||
|
||||
log "5c: same-content re-PUT stable, refcount=$GUARD_REFS_AFTER"
|
||||
fi
|
||||
|
||||
# ── 5b. Deletion: the destructive half, and the only one that can lose data
|
||||
#
|
||||
# Everything above is additive and recoverable. This unlinks files after a
|
||||
|
||||
Reference in New Issue
Block a user