Commit Graph

20 Commits

Author SHA1 Message Date
DioCrafts b14c4dc911 perf: cache-stampede coalescing + DB safeguards; ui/i18n fixes
Backend — tail latency & throughput:
- FileContentCache, image transcode, and search now use moka single-flight
  (try_get_with / get_or_load) so N concurrent misses for the same key
  collapse to one disk read / transcode / query instead of a thundering herd.
  Microbenchmark (128 concurrent on one hot key): 128 loads / p99 ~1023ms
  before vs 1 load / p99 ~32ms after.
- DB: configurable per-statement timeout on the primary pool
  (OXICLOUD_DB_STATEMENT_TIMEOUT_SECS, default 30; maintenance pool exempt) so
  a runaway query can't pin a connection and starve the pool.
- DB: background pool-saturation monitor
  (OXICLOUD_DB_POOL_MONITOR_INTERVAL_SECS) that WARNs as the primary pool nears
  exhaustion — the early signal before tail latency cliffs.
- mimalloc: set MIMALLOC_PURGE_DELAY=0 (Dockerfile + compose) so freed pages
  return to the OS and RSS tracks the live working set; benchmarked on
  musl/aarch64 at ~400MB reclaimed vs 0MB with the default.

Frontend — UI / i18n fixes:
- i18n: fix literal "{{count}}" and "{{percentage}}/{{used}}/{{total}}" in the
  selection toolbar and storage line — the call sites passed param names that
  didn't match the locale placeholders; unify on `count` and pass the storage
  template its params. Add es files.selected_count.
- sidebar: hide the drive picker when there's only one drive (the redundant
  "Personal" row); remove the coloured left accent on the active nav item.
- logo: stop clipping the cloud's left bulge — viewBox recentred on the cloud's
  true bbox with proportional SVG size so it keeps the same rendered scale.
- user menu: drop the default <a> underline on the link rows.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 14:42:10 +02:00
Edouard Vanbelle 55487187eb chore(ci): improve db migratiion to unserstand clash in CI 2026-06-19 16:06:38 +02:00
Claude fc55e92299 perf(blob-cache,db): release index mutex before disk I/O; skip per-acquire DB ping
CachedBlobBackend held its single tokio::Mutex<LruCache> across filesystem
syscalls, serializing every concurrent cache operation behind one lock:
- get_blob_stream / get_blob_range_stream: held across File::open()/seek()
- delete_blob: held across remove_file()
- initialize: held across the full cache-dir walk
- eviction (insert + fetch paths): held across remove_file() loops

Now the lock only guards the in-memory LRU. Presence checks bump recency
and release the guard before touching the filesystem (a vanished file falls
through to the existing fetch-and-cache path, covering the race), and
eviction selects victims under the lock then unlinks them after releasing
it. The duplicated eviction loop is extracted into
CachedRef::collect_evictions.

db: set test_before_acquire(false). With warm min_connections and a bounded
max_lifetime, the liveness ping sqlx issues on every acquire() costs more
than the rare dead connection it catches; stale sockets surface as a query
error and the pool recycles them either way.

https://claude.ai/code/session_01UtfkS3nZF1vrF5jNAps6wV
2026-06-09 13:18:09 +00:00
Edouard Vanbelle 4531ee9f15 fix(db-migration): fix 2 changes with same ID
This solve issue with 2 migrations made the same day, due to merge on pull request, DB migration is blocking

    the same version prefix:
     - 20260625000000_files_user_size_index.sql    (Dio)
     - 20260625000000_folder_tree_modified_at.sql  (Ed)
    They were renamed to ...0001 and ...0002 (disjoint versions) + protection like "IF NOT EXISTS"

    I have opt for an automated clean up of old entry:
        `DELETE FROM _sqlx_migrations WHERE version = 20260625000000;`
    runned on startup

    affected users: Dio, myself and any dev that wanted to work on this project since eb0ba58158
2026-06-07 16:32:42 +02:00
Edouard Vanbelle badf35f08f chore: remove all executable attributes on non bash files 2026-04-01 23:14:42 +02:00
Jared Wolff 036390a242 feat: add SQL migration system using sqlx::migrate!()
Replace manual schema.sql application with sqlx's built-in migration
system. Migrations are embedded at compile time and tracked in the
_sqlx_migrations table. Pending migrations run automatically on startup.

- Move db/schema.sql → migrations/20260307000000_initial_schema.sql
- Remove apply_schema() and split_sql_statements() from db.rs
- Add run_migrations() using sqlx::migrate!() macro
- Remove docker-compose schema.sql mount (app handles it now)
- Enable sqlx "migrate" feature in Cargo.toml

Future schema changes: add a new timestamped .sql in migrations/.

Closes #190

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 15:30:52 -04:00
zjean 18518bedaf fix: resolve clippy warnings and rustfmt issues for CI compliance
Fix all clippy lints (collapsible if, clone on Copy, needless borrow,
redundant bindings, unused params) and apply rustfmt across the codebase.
Update test mocks to match Uuid-based trait signatures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 14:34:07 +01:00
Dionisio fc78dcd35d perf: eliminate Vec<char> allocation in split_sql_statements
- Replace sql.chars().collect::<Vec<char>>() with direct byte-level
  iteration over sql.as_bytes()
- Saves ~140 KB heap allocation for 35 KB schema (4× input size)
- Remove unused _start variable
- SQL is ASCII-safe so byte comparison is sufficient for all delimiters
2026-03-06 22:44:04 +01:00
Dionisio f2d35ca792 feat: auto-persist JWT secret, remove setup token requirement
- JWT secret auto-generates and persists to <STORAGE_PATH>/.jwt_secret
- Remove setup token: first admin setup is open until system initialized
- Fix schema.sql: move CREATE EXTENSION pg_trgm/ltree to top
- Update login UI and auth.js to remove setup token fields
2026-03-05 22:12:53 +01:00
Diocrafts ee86c3a128 fix: resolve all clippy warnings and convert integration_tests to custom cfg
- Add type aliases (FileRow, FolderRow, FolderRowPaginated, FolderRowOptUser) to reduce type complexity
- Simplify redundant closures in app_password_handler and webdav_handler
- Remove needless borrow in auth_handler
- Collapse nested if/let chains in login_lockout, webdav_lock, auth, rate_limit
- Box LockEntry in acquire() Err variant to fix large enum variant warning
- Rename DeviceCodeStatus::from_str to parse to avoid should_implement_trait lint
- Add #[allow(clippy::too_many_arguments)] and #[allow(clippy::result_unit_err)] where appropriate
- Convert integration_tests from cargo feature to custom cfg attribute
- Add check-cfg lint config in Cargo.toml for integration_tests cfg
2026-03-04 23:55:08 +01:00
Dionisio 77fc9bba47 refactor: remove anyhow dependency, replace with thiserror in db.rs and auth_factory.rs 2026-03-03 11:11:39 +01:00
Dionisio 97cf6402e2 fix: resolve all clippy warnings and enforce cargo fmt
- display_helpers: convert module doc-comments to regular comments,
  merge identical text/markdown + text/ branches
- search_service: replace needless range loops with slice-based pagination
- folder_repository, folder_db_repository: collapse nested if statements
- favorites_pg_repository: remove unnecessary borrow on generic arg
- file_blob_read_repository: collapse 6 nested if-let blocks
- file_blob_write_repository: collapse nested if for dedup ref decrement
- chunked_upload_service: use div_ceil(), collapse 2 nested if blocks
- folder_handler: collapse nested if-let for owner check
- webdav_handler: replace 7x io::Error::new(ErrorKind::Other, ..) with
  io::Error::other(..)
- cargo fmt applied to all files

Passes: cargo clippy --all-targets --all-features -- -D warnings
2026-02-25 10:28:34 +01:00
Dionisio 9f692f03c3 Implement dual DB pools (primary + maintenance) and wire services 2026-02-24 19:28:00 +01:00
George Wu d782d8469c fix: always apply database schema on startup
Previously, the application only applied the schema if auth.users table
didn't exist. This caused issues when upgrading - new schema elements
(like storage.trash_items view) weren't created on existing databases.

Now the schema is applied on every startup. Since schema.sql uses
'IF NOT EXISTS' and 'CREATE OR REPLACE', it's safe to run repeatedly.
2026-02-18 17:56:06 -08:00
Dionisio 4c98c5a657 style: apply cargo fmt to entire codebase
Standardize code formatting across all 173 Rust source files
using rustfmt. No functional changes - purely cosmetic.

This establishes a consistent code style baseline for the
project going forward.
2026-02-14 01:29:34 +01:00
Dionisio 67137a3ef2 refactor: apply clippy auto-fixes (162 warnings resolved)
- Fix needless borrows and references
- Collapse nested if statements
- Replace manual strip_prefix with str::strip_prefix()
- Remove redundant closures in map/unwrap_or_else
- Use Iterator::next_back() instead of rev().next()
- Simplify map_or patterns
- Use std::io::Error::other() instead of new(ErrorKind::Other, ..)
- Use div_ceil() instead of manual ceiling division
- Consolidate format! string arguments
- Various other idiomatic Rust improvements

39 files changed, 220 insertions(+), 320 deletions(-)
2026-02-14 01:26:02 +01:00
Dionisio e2297d276a fix: resolve admin registration failure on fresh Docker installs (#81)
Three bugs caused 403 errors when creating the first admin on fresh
Docker deployments (Unraid, Komodo):

1. db.rs: Schema application failures were silently swallowed. The app
   started with no tables, causing all auth queries to fail. Now the
   startup aborts if schema cannot be applied, with a fallback
   statement-by-statement executor that handles dollar-quoted blocks.
   Retries increased to 5 with 2s intervals.

2. auth_application_service.rs: count_admin_users() used fragile string
   matching (contains "does not exist")) on multi-layer wrapped errors.
   count_all_users() rejected admin creation on any DB error. Both now
   allow admin creation on any error for bootstrap scenarios.

3. auth_handler.rs: Redundant 60-line handler-level admin detection
   duplicated service-layer logic and generated noisy ERROR logs on
   fresh installs. Removed entirely - service layer handles it all.

Closes #81
2026-02-12 23:20:46 +01:00
Dionisio ad07a5abda fix: auto-apply DB schema on fresh install and fix admin registration (#81)
- Auto-apply schema.sql when database tables don't exist (embedded in binary)
- Handle missing tables gracefully during admin registration (treat as fresh install)
- Fix docker-compose depends_on to wait for postgres healthcheck
- Rename personal folder from 'Mi Carpeta' to 'My Folder' with backward compat
- Translate remaining Spanish messages to English
2026-02-12 14:45:52 +01:00
Dionisio d31a413e57 chore: translate all Spanish comments and log messages to English 2026-02-12 09:41:25 +01:00
Diocrafts a82faa5eaf refactoring hexagonal and clean architecture 2026-02-08 13:40:23 +01:00