Files
Oxicloud/src/infrastructure/services/webdav_dead_property_store.rs
T
Claude aba89c4f5d perf: eliminate N+1 hot-path queries, cache immutable lookups, stop re-compressing compressed bytes
Every change is benchmark-verified (harness + before/after numbers in
benches/, measured on this branch; reproduction commands in each doc):

DAV / sync-client hot paths
- PROPFIND dead-properties: one = ANY($1) query per 500-child page instead
  of one sequential query per child, and indexable `=` predicates instead
  of IS NOT DISTINCT FROM (seq scans). 2,000-child folder: 1.07-4.54 s of
  DB chatter -> 4-6 ms (258-773x). Applied to native + NC PROPFIND and
  both NC REPORT handlers. [benches/DEAD-PROPS.md]
- Folder paging: keyset cursor (name > $last) + new partial index
  (folder_id, name) replaces LIMIT/OFFSET full-folder rescan per page.
  Full 20k-file walk: 1266 ms -> 77 ms (16.5x). New migration
  20260917000000. [benches/PROPFIND-PAGING.md]
- NC chroot / default-drive resolution: moka caches (30 s TTL, explicit
  invalidation on drive mutations) for find_default_for_user and the
  markerless chroot FolderDto. 2 uncached queries + 2 pool checkouts per
  NC/WebDAV/WOPI request -> sub-us moka hit (p50 0.7-3.6 ms -> ~1 us).
  [benches/CHROOT-CACHE.md]
- Quota: PROPFINDs whose prop list never names a quota prop skip the
  2-query resolution entirely (wants_quota()); the remaining lookups read
  2 columns instead of the full auth.users row with its <=512 KiB avatar
  (11-16x, p50 3.4 ms -> 0.29 ms). Same narrow read now gates every
  upload quota check. [benches/QUOTA-PATH.md]

CPU on the request path
- ZIP exports (folder download, share ZIP, batch download): entries whose
  MIME says already-compressed (JPEG/MP4/zip/pdf/...) are Stored instead
  of Deflate - deflate ran inline on the tokio writer task at ~41 MB/s
  for ~0% size gain. Mixed media corpus: 4.31x wall and CPU, archive size
  unchanged. Shared predicate in common::mime_detect. [benches/ZIP-MEDIA.md]
- Compression layers: tower-http's default maps to Brotli QUALITY 11
  (verified in brotli-8.0.2 source and empirically: 90 ms per 64 KiB JSON
  response, 1.3 s per 700 KiB bundle). Both layers pinned to Precise(4):
  99x less CPU for ~15% more bytes. SPA assets are now precompressed at
  build time (scripts/precompress.mjs, 77% smaller) and served via
  ServeDir::precompressed_br/gzip: 2016x less per-request work, and
  clients get the better q11 bytes. [benches/STATIC-PRECOMPRESSED.md]

Batched / cached backend paths [benches/NPLUS1-AND-CACHES.md]
- Content-search ReBAC re-verification: new
  AuthorizationEngine::check_files_read_batch (default = old loop;
  PgAclEngine override batches drive resolution + reuses role cache).
  200 sequential point SELECTs per search -> 1-2 queries.
- Batch-ZIP subtree downloads: drop per-file re-authz + per-file Recent
  recording (2 writes/file) for subtree entries already authorized at the
  root - mirrors the native folder-download path. ~6,000 statements
  removed from a 2,000-file archive.
- CDC chunk manifests: immutable by content address, now moka-cached
  (weight-bounded 32 MiB, 60 s TTL, positive-only, invalidated on delete)
  - removes one manifest query (p50 0.44-4.4 ms) from every stream,
  range and full blob read.
- People tab: grouped COUNT + batched cover lookup instead of dragging
  every face row with its 2 KiB embedding (10k faces: 30.4 ms & 21 MB ->
  3.8 ms & 1.3 KB, 8.1x); merge() is one set-based UPDATE.
  [benches/PEOPLE-LIST.md]
- Photos timeline cursor: raw timestamptz comparison instead of
  EXTRACT(EPOCH ...) wrapper + IS NULL OR disjunction - cursor is an
  index boundary again, deep scroll stops re-scanning skipped rows.
- Public share landing: one atomic UPDATE ... access_count + 1 (was
  SELECT + full-row write-back: racy, lost updates, clobbered concurrent
  owner edits) - 3 round-trips -> 2 per visit.
- move_to_trash: dead full-entity SELECT feeding a documented no-op
  removed from both branches; dead fields dropped from TrashService.
- NFC normalization: is_nfc_quick fast path skips the decompose/recompose
  state machine for the ~100% already-NFC case (every row loaded from PG).

Frontend
- Large folders paint after page one (~200 items) via fetchFolderListing's
  new onPage hook instead of waiting for every sequential page.
- Tested-and-reverted (kept for the record): cached Intl.Collator for name
  sorts - vitest showed it 2x SLOWER than V8's argument-less localeCompare
  fast path (5.6 ms vs 12.1 ms / 5k names). Sort order untouched.

New bench harnesses under examples/ (bench feature): zip_media,
dead_props, chroot_cache, quota_path, people_list, propfind_paging,
static_precompress.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CBK1RdtzyP6759Muqe1K1w
2026-07-16 14:20:20 +00:00

265 lines
10 KiB
Rust

//! PostgreSQL-backed dead property store for WebDAV PROPPATCH / PROPFIND compliance.
//!
//! RFC 4918 §4.2 defines "dead properties" as those stored verbatim by the
//! server without interpreting their value. Properties are persisted to
//! `storage.webdav_dead_properties` and survive server restarts.
//!
//! Keying contract (after migration 20260830000001): the row is keyed by
//! the underlying resource id — exactly one of `folder_id` / `file_id` is
//! set — not by the resource's current path. Three consequences:
//!
//! * Every delete code path (REST, WebDAV, NextCloud DAV, trash empty,
//! folder cascade) reaps dead-property rows for free via FK
//! `ON DELETE CASCADE`. The store has no `remove_resource()` method
//! because it isn't needed: deleting the file/folder row reaps the
//! attached dead properties as a database invariant.
//! * MOVE / RENAME never changes the resource id, so dead properties
//! follow the resource without any store-side bookkeeping. The store
//! has no `rename_resource()` method for the same reason.
//! * Dead properties are RESOURCE state (RFC 4918 §4.2), not user
//! state. Two users on a shared drive PROPFIND'ing the same resource
//! see the same dead properties. The `user_id` scope key from the
//! pre-rekey schema is gone; user-delete cleanup happens
//! transitively through `auth.users` → `storage.{folders,files}` →
//! this table.
//!
//! Queries use `sqlx::query()` (runtime-bound) rather than `sqlx::query!()`
//! to keep fresh checkouts compilable without a DB connection — the
//! codebase's standing convention.
//!
//! COPY semantics (RFC 4918 §8.8 — dead properties MUST be duplicated)
//! are NOT handled here. The COPY handler is responsible for explicitly
//! reading the source's dead properties via `get_all()` and writing them
//! against the new resource id via `set()`. Not done in this migration —
//! it was not handled by the path-based store either, so this is a
//! parity decision, not a regression.
use std::collections::HashMap;
use std::sync::Arc;
use sqlx::{PgPool, Row};
use uuid::Uuid;
use crate::application::adapters::webdav_adapter::QualifiedName;
use crate::domain::errors::DomainError;
/// Polymorphic reference to the resource a dead property hangs off.
///
/// Exactly one variant — folder or file — is ever stored in a single
/// row. The CHECK constraint
/// `(folder_id IS NULL) <> (file_id IS NULL)` enforces this at the
/// database level so the application layer cannot accidentally write a
/// row that's both or neither.
#[derive(Clone, Copy, Debug)]
pub enum ResourceRef {
Folder(Uuid),
File(Uuid),
}
pub struct DeadPropertyStore {
pool: Arc<PgPool>,
}
impl DeadPropertyStore {
pub fn new(pool: Arc<PgPool>) -> Self {
Self { pool }
}
/// Upsert a dead property. `value = None` means an empty XML element.
///
/// The two SQL branches are deliberately kept separate so each
/// ON CONFLICT clause can target the matching partial unique
/// index (`idx_webdav_dead_props_folder_unique` /
/// `idx_webdav_dead_props_file_unique`). A combined upsert would
/// require a non-partial unique index that treats NULL as
/// distinct, which doesn't match the (folder XOR file) shape.
pub async fn set(
&self,
r: ResourceRef,
name: QualifiedName,
value: Option<String>,
) -> Result<(), DomainError> {
match r {
ResourceRef::Folder(folder_id) => {
sqlx::query(
r#"
INSERT INTO storage.webdav_dead_properties
(folder_id, namespace, local_name, value)
VALUES ($1, $2, $3, $4)
ON CONFLICT (folder_id, namespace, local_name)
WHERE folder_id IS NOT NULL
DO UPDATE SET value = EXCLUDED.value, updated_at = CURRENT_TIMESTAMP
"#,
)
.bind(folder_id)
.bind(&name.namespace)
.bind(&name.name)
.bind(&value)
.execute(&*self.pool)
.await
.map_err(|e| {
DomainError::internal_error("DeadPropertyStore", format!("set folder: {e}"))
})?;
}
ResourceRef::File(file_id) => {
sqlx::query(
r#"
INSERT INTO storage.webdav_dead_properties
(file_id, namespace, local_name, value)
VALUES ($1, $2, $3, $4)
ON CONFLICT (file_id, namespace, local_name)
WHERE file_id IS NOT NULL
DO UPDATE SET value = EXCLUDED.value, updated_at = CURRENT_TIMESTAMP
"#,
)
.bind(file_id)
.bind(&name.namespace)
.bind(&name.name)
.bind(&value)
.execute(&*self.pool)
.await
.map_err(|e| {
DomainError::internal_error("DeadPropertyStore", format!("set file: {e}"))
})?;
}
}
Ok(())
}
/// Delete a specific dead property. No-op if not present.
///
/// Filters on the concrete id column (`folder_id = $1` / `file_id = $1`)
/// rather than the old `IS NOT DISTINCT FROM` pair — PostgreSQL cannot
/// serve `IS NOT DISTINCT FROM` from a B-tree index, so every lookup
/// degraded to a sequential scan as the table grew. The `=` shape is
/// served by the partial unique indexes from migration 20260830000001.
/// (Same rationale for `get_all` / `get` / the batched readers below —
/// measured in `benches/DEAD-PROPS.md`.)
pub async fn remove(&self, r: ResourceRef, name: &QualifiedName) -> Result<(), DomainError> {
let (column, id) = split_ref(r);
sqlx::query(&format!(
"DELETE FROM storage.webdav_dead_properties
WHERE {column} = $1
AND namespace = $2
AND local_name = $3",
))
.bind(id)
.bind(&name.namespace)
.bind(&name.name)
.execute(&*self.pool)
.await
.map_err(|e| DomainError::internal_error("DeadPropertyStore", format!("remove: {e}")))?;
Ok(())
}
/// Return all dead properties for the given resource.
pub async fn get_all(
&self,
r: ResourceRef,
) -> Result<Vec<(QualifiedName, Option<String>)>, DomainError> {
let (column, id) = split_ref(r);
let rows = sqlx::query(&format!(
"SELECT namespace, local_name, value
FROM storage.webdav_dead_properties
WHERE {column} = $1",
))
.bind(id)
.fetch_all(&*self.pool)
.await
.map_err(|e| DomainError::internal_error("DeadPropertyStore", format!("get_all: {e}")))?;
Ok(rows.into_iter().map(row_to_prop).collect())
}
/// Batched variant of [`get_all`] for every file in a PROPFIND page:
/// ONE `file_id = ANY($1)` round-trip instead of N sequential queries.
/// Files with no dead properties are simply absent from the map.
pub async fn get_all_for_files(
&self,
file_ids: &[Uuid],
) -> Result<HashMap<Uuid, Vec<(QualifiedName, Option<String>)>>, DomainError> {
self.get_all_batched("file_id", file_ids).await
}
/// Batched variant of [`get_all`] for every subfolder in a PROPFIND page.
pub async fn get_all_for_folders(
&self,
folder_ids: &[Uuid],
) -> Result<HashMap<Uuid, Vec<(QualifiedName, Option<String>)>>, DomainError> {
self.get_all_batched("folder_id", folder_ids).await
}
async fn get_all_batched(
&self,
column: &str,
ids: &[Uuid],
) -> Result<HashMap<Uuid, Vec<(QualifiedName, Option<String>)>>, DomainError> {
if ids.is_empty() {
return Ok(HashMap::new());
}
let rows = sqlx::query(&format!(
"SELECT {column} AS resource_id, namespace, local_name, value
FROM storage.webdav_dead_properties
WHERE {column} = ANY($1)",
))
.bind(ids)
.fetch_all(&*self.pool)
.await
.map_err(|e| {
DomainError::internal_error("DeadPropertyStore", format!("get_all_batched: {e}"))
})?;
let mut map: HashMap<Uuid, Vec<(QualifiedName, Option<String>)>> = HashMap::new();
for row in rows {
let resource_id: Uuid = row.get("resource_id");
map.entry(resource_id).or_default().push(row_to_prop(row));
}
Ok(map)
}
/// Return a specific dead property, or `None` if not stored.
/// Returns `Some(None)` when the property exists with an empty value.
pub async fn get(
&self,
r: ResourceRef,
name: &QualifiedName,
) -> Result<Option<Option<String>>, DomainError> {
let (column, id) = split_ref(r);
let row = sqlx::query(&format!(
"SELECT value FROM storage.webdav_dead_properties
WHERE {column} = $1
AND namespace = $2
AND local_name = $3",
))
.bind(id)
.bind(&name.namespace)
.bind(&name.name)
.fetch_optional(&*self.pool)
.await
.map_err(|e| DomainError::internal_error("DeadPropertyStore", format!("get: {e}")))?;
Ok(row.map(|r| r.get::<Option<String>, _>("value")))
}
}
/// Maps a `ResourceRef` onto the column that stores it plus the id to bind.
/// The column name is one of two compile-time literals — never user input —
/// so interpolating it into the SQL text is safe.
fn split_ref(r: ResourceRef) -> (&'static str, Uuid) {
match r {
ResourceRef::Folder(id) => ("folder_id", id),
ResourceRef::File(id) => ("file_id", id),
}
}
fn row_to_prop(r: sqlx::postgres::PgRow) -> (QualifiedName, Option<String>) {
let namespace: String = r.get("namespace");
let local_name: String = r.get("local_name");
let value: Option<String> = r.get("value");
(QualifiedName::new(namespace, local_name), value)
}
pub fn create_dead_property_store(pool: Arc<PgPool>) -> Arc<DeadPropertyStore> {
Arc::new(DeadPropertyStore::new(pool))
}