Files
Oxicloud/src/application/services/people_service.rs
T
Claude aba89c4f5d perf: eliminate N+1 hot-path queries, cache immutable lookups, stop re-compressing compressed bytes
Every change is benchmark-verified (harness + before/after numbers in
benches/, measured on this branch; reproduction commands in each doc):

DAV / sync-client hot paths
- PROPFIND dead-properties: one = ANY($1) query per 500-child page instead
  of one sequential query per child, and indexable `=` predicates instead
  of IS NOT DISTINCT FROM (seq scans). 2,000-child folder: 1.07-4.54 s of
  DB chatter -> 4-6 ms (258-773x). Applied to native + NC PROPFIND and
  both NC REPORT handlers. [benches/DEAD-PROPS.md]
- Folder paging: keyset cursor (name > $last) + new partial index
  (folder_id, name) replaces LIMIT/OFFSET full-folder rescan per page.
  Full 20k-file walk: 1266 ms -> 77 ms (16.5x). New migration
  20260917000000. [benches/PROPFIND-PAGING.md]
- NC chroot / default-drive resolution: moka caches (30 s TTL, explicit
  invalidation on drive mutations) for find_default_for_user and the
  markerless chroot FolderDto. 2 uncached queries + 2 pool checkouts per
  NC/WebDAV/WOPI request -> sub-us moka hit (p50 0.7-3.6 ms -> ~1 us).
  [benches/CHROOT-CACHE.md]
- Quota: PROPFINDs whose prop list never names a quota prop skip the
  2-query resolution entirely (wants_quota()); the remaining lookups read
  2 columns instead of the full auth.users row with its <=512 KiB avatar
  (11-16x, p50 3.4 ms -> 0.29 ms). Same narrow read now gates every
  upload quota check. [benches/QUOTA-PATH.md]

CPU on the request path
- ZIP exports (folder download, share ZIP, batch download): entries whose
  MIME says already-compressed (JPEG/MP4/zip/pdf/...) are Stored instead
  of Deflate - deflate ran inline on the tokio writer task at ~41 MB/s
  for ~0% size gain. Mixed media corpus: 4.31x wall and CPU, archive size
  unchanged. Shared predicate in common::mime_detect. [benches/ZIP-MEDIA.md]
- Compression layers: tower-http's default maps to Brotli QUALITY 11
  (verified in brotli-8.0.2 source and empirically: 90 ms per 64 KiB JSON
  response, 1.3 s per 700 KiB bundle). Both layers pinned to Precise(4):
  99x less CPU for ~15% more bytes. SPA assets are now precompressed at
  build time (scripts/precompress.mjs, 77% smaller) and served via
  ServeDir::precompressed_br/gzip: 2016x less per-request work, and
  clients get the better q11 bytes. [benches/STATIC-PRECOMPRESSED.md]

Batched / cached backend paths [benches/NPLUS1-AND-CACHES.md]
- Content-search ReBAC re-verification: new
  AuthorizationEngine::check_files_read_batch (default = old loop;
  PgAclEngine override batches drive resolution + reuses role cache).
  200 sequential point SELECTs per search -> 1-2 queries.
- Batch-ZIP subtree downloads: drop per-file re-authz + per-file Recent
  recording (2 writes/file) for subtree entries already authorized at the
  root - mirrors the native folder-download path. ~6,000 statements
  removed from a 2,000-file archive.
- CDC chunk manifests: immutable by content address, now moka-cached
  (weight-bounded 32 MiB, 60 s TTL, positive-only, invalidated on delete)
  - removes one manifest query (p50 0.44-4.4 ms) from every stream,
  range and full blob read.
- People tab: grouped COUNT + batched cover lookup instead of dragging
  every face row with its 2 KiB embedding (10k faces: 30.4 ms & 21 MB ->
  3.8 ms & 1.3 KB, 8.1x); merge() is one set-based UPDATE.
  [benches/PEOPLE-LIST.md]
- Photos timeline cursor: raw timestamptz comparison instead of
  EXTRACT(EPOCH ...) wrapper + IS NULL OR disjunction - cursor is an
  index boundary again, deep scroll stops re-scanning skipped rows.
- Public share landing: one atomic UPDATE ... access_count + 1 (was
  SELECT + full-row write-back: racy, lost updates, clobbered concurrent
  owner edits) - 3 round-trips -> 2 per visit.
- move_to_trash: dead full-entity SELECT feeding a documented no-op
  removed from both branches; dead fields dropped from TrashService.
- NFC normalization: is_nfc_quick fast path skips the decompose/recompose
  state machine for the ~100% already-NFC case (every row loaded from PG).

Frontend
- Large folders paint after page one (~200 items) via fetchFolderListing's
  new onPage hook instead of waiting for every sequential page.
- Tested-and-reverted (kept for the record): cached Intl.Collator for name
  sorts - vitest showed it 2x SLOWER than V8's argument-less localeCompare
  fast path (5.6 ms vs 12.1 ms / 5k names). Sort order untouched.

New bench harnesses under examples/ (bench feature): zip_media,
dead_props, chroot_cache, quota_path, people_list, propfind_paging,
static_precompress.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CBK1RdtzyP6759Muqe1K1w
2026-07-16 14:20:20 +00:00

268 lines
9.0 KiB
Rust

//! People (faces) use cases: identity clustering + the read/mutation methods
//! the HTTP layer calls.
//!
//! Clustering is a full re-cluster over the user's faces: a union-find groups
//! faces whose embeddings are within a cosine threshold (connected
//! components), and groups of at least `min_faces` become a "person". This is
//! O(n²) in the user's face count — fine for moderate libraries; an ANN index
//! (pgvector/VectorChord) is the documented scale-up.
//!
//! Strictly user-scoped (the repository filters by user), so — like
//! `RecentService` / `PlacesService` — no `AuthorizationEngine` check is
//! needed: the `caller_id` parameter is the access scope.
use std::collections::HashMap;
use std::sync::Arc;
use chrono::Utc;
use uuid::Uuid;
use crate::application::dtos::people_dto::{FaceBoxDto, PersonDto};
use crate::application::ports::face_ports::FaceRepository;
use crate::common::errors::DomainError;
use crate::domain::entities::face::Person;
use crate::infrastructure::repositories::pg::FacePgRepository;
/// Cosine similarity of two equal-length vectors. Embeddings are produced
/// L2-normalized, so this is ~a dot product; we normalize anyway for safety.
fn cosine(a: &[f32], b: &[f32]) -> f32 {
if a.len() != b.len() || a.is_empty() {
return 0.0;
}
let (mut dot, mut na, mut nb) = (0.0f32, 0.0f32, 0.0f32);
for (&x, &y) in a.iter().zip(b.iter()) {
dot += x * y;
na += x * x;
nb += y * y;
}
if na == 0.0 || nb == 0.0 {
return 0.0;
}
dot / (na.sqrt() * nb.sqrt())
}
/// Disjoint-set with path-halving + union by rank.
struct UnionFind {
parent: Vec<usize>,
rank: Vec<usize>,
}
impl UnionFind {
fn new(n: usize) -> Self {
Self {
parent: (0..n).collect(),
rank: vec![0; n],
}
}
fn find(&mut self, mut x: usize) -> usize {
while self.parent[x] != x {
self.parent[x] = self.parent[self.parent[x]];
x = self.parent[x];
}
x
}
fn union(&mut self, a: usize, b: usize) {
let (ra, rb) = (self.find(a), self.find(b));
if ra == rb {
return;
}
match self.rank[ra].cmp(&self.rank[rb]) {
std::cmp::Ordering::Less => self.parent[ra] = rb,
std::cmp::Ordering::Greater => self.parent[rb] = ra,
std::cmp::Ordering::Equal => {
self.parent[rb] = ra;
self.rank[ra] += 1;
}
}
}
}
pub struct PeopleService {
repo: Arc<FacePgRepository>,
/// Min cosine similarity to link two faces into the same identity.
cluster_threshold: f32,
/// Min faces in a cluster before it becomes a named-able "person".
min_faces: usize,
}
impl PeopleService {
pub fn new(repo: Arc<FacePgRepository>) -> Self {
Self {
repo,
cluster_threshold: 0.5,
min_faces: 3,
}
}
/// Re-cluster a user's faces. Returns the number of new persons created.
pub async fn recluster(&self, user_id: Uuid) -> Result<usize, DomainError> {
let faces = self.repo.faces_for_user(user_id).await?;
let n = faces.len();
if n == 0 {
return Ok(0);
}
let mut uf = UnionFind::new(n);
for i in 0..n {
for j in (i + 1)..n {
if cosine(&faces[i].embedding, &faces[j].embedding) >= self.cluster_threshold {
uf.union(i, j);
}
}
}
let mut groups: HashMap<usize, Vec<usize>> = HashMap::new();
for i in 0..n {
let root = uf.find(i);
groups.entry(root).or_default().push(i);
}
let mut created = 0usize;
for idxs in groups.into_values() {
if idxs.len() < self.min_faces {
// Too small to be a person — leave/reset these faces unassigned.
for &i in &idxs {
if faces[i].person_id.is_some() {
self.repo.assign_person(faces[i].id, None).await?;
}
}
continue;
}
// Reuse an existing person on this cluster (preserves a user's name)
// or mint a new one.
let existing = idxs.iter().find_map(|&i| faces[i].person_id);
let person_id = match existing {
Some(pid) => pid,
None => {
let pid = Uuid::new_v4();
let person = Person {
id: pid,
user_id,
display_name: None,
cover_face_id: Some(faces[idxs[0]].id),
is_hidden: false,
created_at: Utc::now(),
};
self.repo.create_person(&person).await?;
created += 1;
pid
}
};
for &i in &idxs {
if faces[i].person_id != Some(person_id) {
self.repo
.assign_person(faces[i].id, Some(person_id))
.await?;
}
}
let _ = self
.repo
.set_person_cover(person_id, faces[idxs[0]].id)
.await;
}
Ok(created)
}
/// People (non-empty clusters), most-photographed first.
///
/// Counts come from a grouped-COUNT query and cover photos from one
/// batched lookup of just the cover face ids — the previous
/// `faces_for_user` shipped every face row (2 KiB embedding included)
/// only to count them: ~20 MB of BYTEA per request on a 10k-face
/// library (benches/PEOPLE-LIST.md).
pub async fn list_people(&self, caller_id: Uuid) -> Result<Vec<PersonDto>, DomainError> {
let persons = self.repo.persons_for_user(caller_id).await?;
let count: HashMap<Uuid, i64> = self
.repo
.person_face_stats(caller_id)
.await?
.into_iter()
.collect();
let cover_ids: Vec<Uuid> = persons.iter().filter_map(|p| p.cover_face_id).collect();
let face_file: HashMap<Uuid, Uuid> =
self.repo.file_ids_for_faces(caller_id, &cover_ids).await?;
let mut out: Vec<PersonDto> = persons
.into_iter()
.filter_map(|p| {
let c = count.get(&p.id).copied().unwrap_or(0);
if c == 0 {
return None; // hide empty clusters (e.g. after a merge)
}
let cover_file_id = p
.cover_face_id
.and_then(|fid| face_file.get(&fid).copied())
.map(|u| u.to_string());
Some(PersonDto {
id: p.id.to_string(),
name: p.display_name,
cover_file_id,
face_count: c,
is_hidden: p.is_hidden,
})
})
.collect();
out.sort_by_key(|p| std::cmp::Reverse(p.face_count));
Ok(out)
}
/// File ids of a person's photos (most recent first).
pub async fn person_photos(
&self,
caller_id: Uuid,
person_id: Uuid,
) -> Result<Vec<String>, DomainError> {
let files = self.repo.files_for_person(caller_id, person_id).await?;
Ok(files.into_iter().map(|u| u.to_string()).collect())
}
/// Face boxes within a photo (for lightbox tagging), caller-scoped.
pub async fn faces_for_file(
&self,
caller_id: Uuid,
file_id: Uuid,
) -> Result<Vec<FaceBoxDto>, DomainError> {
let faces = self.repo.faces_for_file(file_id).await?;
Ok(faces
.into_iter()
.filter(|f| f.user_id == caller_id)
.map(|f| FaceBoxDto {
id: f.id.to_string(),
person_id: f.person_id.map(|u| u.to_string()),
x: f.bbox.x,
y: f.bbox.y,
w: f.bbox.w,
h: f.bbox.h,
})
.collect())
}
pub async fn rename_person(
&self,
caller_id: Uuid,
person_id: Uuid,
name: Option<String>,
) -> Result<(), DomainError> {
self.repo.rename_person(caller_id, person_id, name).await
}
/// Merge `from` into `into` by reassigning all of `from`'s faces. The
/// now-empty `from` person is hidden by `list_people`.
///
/// One set-based UPDATE — the previous shape loaded every face row
/// (embeddings included) and issued one UPDATE per matching face.
pub async fn merge(&self, caller_id: Uuid, into: Uuid, from: Uuid) -> Result<(), DomainError> {
self.repo
.reassign_person_faces(caller_id, from, into)
.await?;
Ok(())
}
/// Erase all of the caller's face data (right to erasure / opt-out).
pub async fn delete_all(&self, caller_id: Uuid) -> Result<(), DomainError> {
self.repo.delete_all_for_user(caller_id).await
}
}