cjw/Oxicloud
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

perf: round 12 — auth write-path narrowing, fused quota gate, moka blob-cache index, media single-read, sized listing JSON

Browse Source 
Benchmark-gated round (benches/ROUND12.md; every change ships with a
BEFORE/AFTER harness + equivalence gates, one candidate rejected by its
own bench):

DB / query shapes (bench_round12_queries):
- NC sharee search: username-only projection instead of the 21-column row
  (incl. the <=512 KiB avatar) per match, + gin_trgm_ops indexes on
  auth.users for the leading-wildcard ILIKE (4.98x; 54.7x with index).
- Password login: delete the redundant full-row update_user — create_session
  already stamps last_login_at in its own txn (4.45x per login).
- Email-verified stamp: narrow conditional UPDATE (8.9x); OIDC repeat login
  now compares profile state in memory and issues ZERO queries when nothing
  changed (was: full 17-column rewrite per login).
- Refresh rotation: revoke+insert+stamp fused into one transaction via new
  rotate_session port method (1.18x).
- WOPI CheckFileInfo / authorize_wopi_access: require(Read) + get_file +
  check(Update) overlapped with tokio::join!, original result precedence
  (cold 1.34x).
- Upload quota gate: user-envelope + drive-cap checks fused into ONE
  round-trip (check_upload_quotas) — the NC chunked PUT pays this per
  chunk (1.81x, 2 -> 1 queries/chunk); shared verdict evaluators keep
  error shapes byte-identical.

CPU / allocs (bench_round12_micro):
- sized_json: pre-sized listing serialization replacing axum Json's 128 B
  seed + doubling-realloc chain on files/folder-resources/photos/search
  responses (1.40x, 13 -> 2 allocs per 500-row page; byte-identical).
- Security headers: 4 SetResponseHeaderLayer folded into the CSP middleware
  pass (5 layers -> 1; 1.43x per request, -26 allocs; header set gated
  byte-identical incl. 304s).
- Media capture-metadata: single-read extraction — nom-exif now parses the
  buffer kamadak already read (zero-copy Bytes) and videos open once with a
  kind() dispatch; per-image opens 2-3 -> 1 (1.44x warm geomean, 1.6-3.2x
  cold cache; extraction outputs gated identical incl. the MIME-mislabel
  track fallback).
- Chunked-upload session ops: owner gate folded into the operation's own
  DashMap lookup + stack-encoded uuid compare (5 -> 3 lookups, -2 allocs,
  1.28x per chunk).

Blob cache (bench_blob_cache_index + round-3 regression guard):
- CachedBlobBackend index: tokio::sync::Mutex<LruCache> -> moka::sync::Cache
  with byte weigher. The mutex serialized every cached chunk read and scaled
  NEGATIVELY (2.08 -> 1.07 Mops/s from 1 -> 2 readers); moka probes are
  lock-free (2.17x at K=2). Byte budget now enforced by moka (manual
  current_size + collect_evictions machinery deleted); eviction listener
  unlinks size-evicted files only (Replaced entries keep their file —
  gated). Single-flight miss gate unchanged (16 concurrent misses -> 1
  fetch re-verified via the round-3 harness).
- put_blob now populates the cache BEFORE the inner backend consumes the
  source file (the old order failed 100% of the time — local renames,
  S3/Azure delete the source — so the first read after a whole-file put
  re-downloaded from the remote); inner-put failure invalidates the entry.

Frontend (vitest gates):
- List-view thumbnails request the 150px icon rendition instead of 400px
  preview into a 40px slot (~7.1x fewer pixels, ~4-5x fewer bytes per
  thumbnail across list views); grid keeps preview.

Rejected by its own bench (kept as evidence in bench_round12_micro §2):
- Single-pass compression predicate: the monomorphized And-chain already
  costs ~4.6 ns / 0 allocs total; the fused node measured within noise.

New migration: 20260719000000_users_search_trgm.sql (trgm indexes).
Deferred with prepared design: grouped file/grid view virtualization
(single-VirtualRows flatten, the photos pattern) — next round's headline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfidAJD5AHw23jtvBUNamB
This commit is contained in:
Claude
2026-07-19 01:32:00 +00:00
parent a793cd62eb
commit 50eca0627f
33 changed files with 3989 additions and 410 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/interfaces/api/handlers/file_handler.rs
+7 -1
View File
@@ -864,7 +864,13 @@ impl FileHandler {
}
tracing::info!("Found {} files", files.len());
let mut resp = (StatusCode::OK, Json(files)).into_response();
// Pre-sized serialization — this listing is unbounded (no
// page cap), the axum Json 128-byte seed reallocs ~11 times
// on a big folder (benches/ROUND12.md §M1).
let mut resp = crate::interfaces::api::sized_json::sized_json(
64 + files.len() * crate::interfaces::api::sized_json::EST_ROW_BYTES,
&files,
);
resp.headers_mut()
.insert(header::ETAG, header::HeaderValue::from_str(&etag).unwrap());
resp
src/interfaces/api/handlers/folder_handler.rs
+9 -5
View File
@@ -551,11 +551,15 @@ pub async fn list_folder_resources(
})
.collect();
(
StatusCode::OK,
Json(FolderResourcesDto::with_cursor(items, next_cursor)),
)
.into_response()
{
// Pre-sized serialization (benches/ROUND12.md §M1).
let body = FolderResourcesDto::with_cursor(items, next_cursor);
crate::interfaces::api::sized_json::sized_json(
128 + body.items.len()
* crate::interfaces::api::sized_json::EST_WRAPPED_ROW_BYTES,
&body,
)
}
}
Err(e) => AppError::from(e).into_response(),
}
src/interfaces/api/handlers/photos_handler.rs
+5 -1
View File
@@ -119,7 +119,11 @@ pub async fn list_photos(
})
.collect();
let mut response = Json(&dtos).into_response();
// Pre-sized serialization (benches/ROUND12.md §M1).
let mut response = crate::interfaces::api::sized_json::sized_json(
64 + dtos.len() * crate::interfaces::api::sized_json::EST_WRAPPED_ROW_BYTES,
&dtos,
);
{
let h = response.headers_mut();
h.insert(header::ETAG, header::HeaderValue::from_str(&etag).unwrap());
src/interfaces/api/handlers/search_handler.rs
+16 -2
View File
@@ -84,7 +84,14 @@ impl SearchHandler {
results.files.len(),
results.folders.len()
);
(StatusCode::OK, Json(&*results)).into_response()
{
// Pre-sized serialization (benches/ROUND12.md §M1).
let rows = results.files.len() + results.folders.len();
crate::interfaces::api::sized_json::sized_json(
256 + rows * crate::interfaces::api::sized_json::EST_WRAPPED_ROW_BYTES,
&*results,
)
}
}
Err(err) => {
error!("Search error: {}", err);
@@ -125,7 +132,14 @@ impl SearchHandler {
results.files.len(),
results.folders.len()
);
(StatusCode::OK, Json(&*results)).into_response()
{
// Pre-sized serialization (benches/ROUND12.md §M1).
let rows = results.files.len() + results.folders.len();
crate::interfaces::api::sized_json::sized_json(
256 + rows * crate::interfaces::api::sized_json::EST_WRAPPED_ROW_BYTES,
&*results,
)
}
}
Err(err) => {
error!("Search error: {}", err);
src/interfaces/api/handlers/wopi_handler.rs
+72 -59
View File
@@ -83,14 +83,21 @@ pub struct CheckFileInfoResponse {
/// structured `audit` line on denial internally, so ops sees the real
/// reason without the attacker being able to distinguish "gone" from
/// "revoked".
/// Shared id parsing for the WOPI authz paths: a malformed caller sub is a
/// bad token (401), a malformed file id can't exist (404, anti-enum).
fn parse_wopi_ids(caller_sub: &str, file_id: &str) -> Result<(uuid::Uuid, uuid::Uuid), StatusCode> {
let caller_uuid = uuid::Uuid::parse_str(caller_sub).map_err(|_| StatusCode::UNAUTHORIZED)?;
let file_uuid = uuid::Uuid::parse_str(file_id).map_err(|_| StatusCode::NOT_FOUND)?;
Ok((caller_uuid, file_uuid))
}
async fn require_wopi_perm(
authz: &PgAclEngine,
caller_sub: &str,
file_id: &str,
perm: Permission,
) -> Result<(uuid::Uuid, uuid::Uuid), StatusCode> {
let caller_uuid = uuid::Uuid::parse_str(caller_sub).map_err(|_| StatusCode::UNAUTHORIZED)?;
let file_uuid = uuid::Uuid::parse_str(file_id).map_err(|_| StatusCode::NOT_FOUND)?;
let (caller_uuid, file_uuid) = parse_wopi_ids(caller_sub, file_id)?;
authz
.require(Subject::User(caller_uuid), perm, Resource::File(file_uuid))
.await
@@ -118,25 +125,52 @@ async fn check_file_info(
// Redemption-time authz: even with a valid token, the caller must
// still hold Read on this file. Catches revoked-grant-mid-session.
if let Err(status) = require_wopi_perm(
state.app_state.authorization.as_ref(),
&claims.sub,
&file_id,
Permission::Read,
)
.await
{
return status.into_response();
//
// The Read gate, the metadata fetch and the Update probe are three
// independent lookups keyed only off (caller, file) — overlapped with
// `tokio::join!` (benches/ROUND12.md §5). Results are evaluated in the
// original precedence: Read gate first, then file existence.
let (caller_uuid, file_uuid) = match parse_wopi_ids(&claims.sub, &file_id) {
Ok(ids) => ids,
Err(status) => return status.into_response(),
};
let authz = state.app_state.authorization.as_ref();
let (read_gate, file, can_write_now) = tokio::join!(
authz.require(
Subject::User(caller_uuid),
Permission::Read,
Resource::File(file_uuid)
),
state
.app_state
.applications
.file_retrieval_service
.get_file(&file_id),
// `user_can_write` = actual current Update permission ∧ token's
// can_write flag. If the caller's Update was revoked since the
// token was minted (e.g. their grant was downgraded from Editor
// to Viewer), the editor sees the file as read-only and won't
// even attempt PutFile. The stricter `require_wopi_perm(Update)`
// in put_file is the actual gate; this field is a UI hint.
async {
if claims.can_write {
authz
.check(
Subject::User(caller_uuid),
Permission::Update,
Resource::File(file_uuid),
)
.await
.unwrap_or(false)
} else {
false
}
}
);
if read_gate.is_err() {
return StatusCode::NOT_FOUND.into_response();
}
// Fetch file metadata
let file = match state
.app_state
.applications
.file_retrieval_service
.get_file(&file_id)
.await
{
let file = match file {
Ok(f) => f,
Err(_) => return StatusCode::NOT_FOUND.into_response(),
};
@@ -146,24 +180,6 @@ async fn check_file_info(
.map(|dt| dt.to_rfc3339())
.unwrap_or_default();
// `user_can_write` = actual current Update permission ∧ token's
// can_write flag. If the caller's Update was revoked since the
// token was minted (e.g. their grant was downgraded from Editor
// to Viewer), the editor sees the file as read-only and won't
// even attempt PutFile. The stricter `require_wopi_perm(Update)`
// in put_file is the actual gate; this field is a UI hint.
let can_write_now = claims.can_write
&& state
.app_state
.authorization
.check(
Subject::User(uuid::Uuid::parse_str(&claims.sub).unwrap_or(uuid::Uuid::nil())),
Permission::Update,
Resource::File(uuid::Uuid::parse_str(&file_id).unwrap_or(uuid::Uuid::nil())),
)
.await
.unwrap_or(false);
let response = CheckFileInfoResponse {
base_file_name: file.name.clone(),
// WOPI's `OwnerId` field is required. Post-D7 the DTO no
@@ -550,34 +566,31 @@ async fn authorize_wopi_access<S: FileRetrievalUseCase>(
) -> Result<(crate::application::dtos::file_dto::FileDto, bool), StatusCode> {
let file_uuid = uuid::Uuid::parse_str(file_id).map_err(|_| StatusCode::NOT_FOUND)?;
// Step 1 — Read is required to even open the file.
authz
.require(
// The Read gate (step 1), the metadata fetch and the Update probe
// (step 2) are independent — overlapped with `tokio::join!`
// (benches/ROUND12.md §5); results evaluated in the original order.
//
// Step 2 rationale — can_write reflects real Update, not the client's
// action-string. `check` returns bool without throwing; failure
// just means the caller lacks Update, so we degrade the token to
// read-only. Deliberately no `require` there — a Viewer opening
// the file is legitimate; only the write claim is suppressed.
let (read_gate, file, has_update) = tokio::join!(
authz.require(
Subject::User(caller_id),
Permission::Read,
Resource::File(file_uuid),
)
.await
.map_err(|_| StatusCode::NOT_FOUND)?;
let file = file_retrieval
.get_file(file_id)
.await
.map_err(|_| StatusCode::NOT_FOUND)?;
// Step 2 — can_write reflects real Update, not the client's
// action-string. `check` returns bool without throwing; failure
// just means the caller lacks Update, so we degrade the token to
// read-only. Deliberately no `require` here — a Viewer opening
// the file is legitimate; only the write claim is suppressed.
let has_update = authz
.check(
),
file_retrieval.get_file(file_id),
authz.check(
Subject::User(caller_id),
Permission::Update,
Resource::File(file_uuid),
)
.await
.unwrap_or(false);
);
read_gate.map_err(|_| StatusCode::NOT_FOUND)?;
let file = file.map_err(|_| StatusCode::NOT_FOUND)?;
let has_update = has_update.unwrap_or(false);
// Step 3 — allow explicit view-mode downgrade for Editors.
let can_write = has_update && requested_action != "view";
src/interfaces/api/mod.rs
+1
View File
@@ -2,6 +2,7 @@ pub mod cookie_auth;
pub mod deserializer;
pub mod handlers;
pub mod routes;
pub mod sized_json;
pub use routes::create_api_routes;
pub use routes::create_health_routes;
src/interfaces/api/sized_json.rs
+54
View File
@@ -0,0 +1,54 @@
//! Pre-sized JSON responses for listing endpoints.
//!
//! `axum::Json` serializes into a `BytesMut::with_capacity(128)` — a 500-row
//! listing grows that seed through ~11 doubling reallocations, memcpy-ing
//! ~1.3× the payload on every hot listing response (files, folder
//! resources, photos timeline, search). `sized_json` serializes into one
//! right-sized `Vec` instead: 2 allocations total and no copy chain
//! (benches/ROUND12.md §M1, 1.40x / −11 allocs on a 500-row page).
//!
//! The per-row estimates are calibrated against the serialized DTOs (a
//! realistic `FileDto` row measures ~380 B). Underestimates cost one extra
//! doubling — still far better than the 128-byte seed; overestimates waste
//! transient capacity only (the buffer is freed after the response).
use axum::http::{HeaderValue, StatusCode, header};
use axum::response::{IntoResponse, Response};
use bytes::Bytes;
use serde::Serialize;
/// Serialized size estimate for one file/folder row (FileDto ≈ 380 B).
pub const EST_ROW_BYTES: usize = 384;
/// Serialized size estimate for one wrapped resource row (PhotoDto /
/// FolderResourcesDto items carry a FileDto plus wrapper fields).
pub const EST_WRAPPED_ROW_BYTES: usize = 448;
/// Serialize `value` into a single pre-sized buffer and wrap it as an
/// `application/json` response — drop-in for `Json(value).into_response()`
/// (byte-identical body, gated in `bench_round12_micro` §1), minus the
/// doubling-realloc chain.
pub fn sized_json<T: Serialize>(estimated_bytes: usize, value: &T) -> Response {
let mut buf = Vec::with_capacity(estimated_bytes.max(128));
match serde_json::to_writer(&mut buf, value) {
Ok(()) => (
StatusCode::OK,
[(
header::CONTENT_TYPE,
HeaderValue::from_static("application/json"),
)],
Bytes::from(buf),
)
.into_response(),
// Mirror axum's Json error arm: 500 + plain-text serializer error.
Err(err) => (
StatusCode::INTERNAL_SERVER_ERROR,
[(
header::CONTENT_TYPE,
HeaderValue::from_static("text/plain; charset=utf-8"),
)],
err.to_string(),
)
.into_response(),
}
}