Commit Graph

295 Commits

Author SHA1 Message Date
Diocrafts c0cb86c273 docs: migrate legacy docs to official site 2026-04-22 07:50:41 +02:00
BillionClaw 0c4290261e fix(auth): sanitize claims.sub when padding/truncating OIDC usernames
When OIDC providers (e.g. Keycloak) use email addresses as usernames or
when claims.sub contains @ or other invalid characters, the username
padding and collision-suffix logic could introduce invalid characters.

The fix filters claims.sub through the same allowed-character filter
before using it in username construction.

Fixes DioCrafts/OxiCloud#259
2026-04-18 07:26:36 +00:00
BillionClaw b2f0cec6bc fix(auth): handle email-style usernames in OIDC JIT provisioning 2026-04-17 18:10:17 +00:00
Diocrafts 761d159a92 feat(dedup): CDC sub-file deduplication with FastCDC + parallel chunk storage + dedup skip
- Replace whole-file SHA-256 dedup with FastCDC 2020 content-defined chunking
  (min 64KB, avg 256KB, max 1MB) + BLAKE3 hashing
- Add chunk_manifests table (file_hash → chunk_hashes[] + chunk_sizes[])
- Add put_blob_from_bytes to BlobStorageBackend trait (all 7 backends)
- 3-phase store_chunks pipeline:
  Phase 0: batch-check existing chunks (single PG query)
  Phase 1: selective disk read (skip existing chunks entirely)
  Phase 2: parallel upload with buffer_unordered(8)
- CDC-aware read_blob_stream and read_blob_range_stream with legacy fallback
- Transactional manifest + chunk ref-count cascade on remove_reference
- 12 CDC tests (determinism, reassembly, contiguity, sub-file dedup, etc.)
- Update deduplication.md to reflect new architecture
2026-04-14 23:17:39 +02:00
Diocrafts cd3733b459 feat: pluggable storage backends (S3, Azure, local) with admin UI
Implement 4-phase external storage backends architecture:

Phase 1 - Foundation:
- BlobStorageBackend trait (application/ports/blob_storage_ports.rs)
- LocalBlobBackend: extracted all tokio::fs ops from DedupService
- S3BlobBackend: AWS SDK with custom endpoint support (MinIO, R2, B2)
- DedupService refactored to use Arc<dyn BlobStorageBackend>

Phase 2 - Admin Panel:
- StorageSettingsService with DB persistence + env override
- Storage tab in admin panel (backend selector, S3 form, provider presets)
- GET/PUT/POST endpoints for storage settings + connection test
- i18n keys (en/es) and BEM CSS

Phase 3 - Migration:
- MigrationBlobBackend decorator (dual-read: target-first + source fallback)
- Background migration job with parallel transfers + progress tracking
- Migration UI (progress bar, ETA, pause/resume/verify/complete)
- 6 admin API endpoints for migration lifecycle

Phase 4 - Enterprise Extras:
- CachedBlobBackend: LRU disk cache for remote backends
- EncryptedBlobBackend: AES-256-GCM at-rest encryption
- AzureBlobBackend: Azure Blob Storage support
- RetryBlobBackend: exponential backoff for transient errors
- Decorator composition in DI: retry → encryption → cache

All 223 tests passing, clippy clean, fmt verified.
2026-04-14 21:33:38 +02:00
Edouard Vanbelle 62a7713af5 refactor(server): apply fmt + lint recos 2026-04-14 19:05:01 +02:00
su77ungr 1b14475d05 add WWW-Authenticate handshake for spec-compliancy 2026-04-13 02:12:17 +02:00
Diocrafts edf1e66989 feat: add public share page with download support (#253)
Share links now point to /s/{token} (was /api/s/{token}) and render a
proper HTML page instead of raw JSON.

Changes:
- static/share.html: standalone public share page
- static/css/views/share-public.css: share page styles
- static/js/views/public/publicShare.js: client-side logic that fetches
  share metadata via /api/s/{token}, handles password-protected shares,
  and renders file download / folder info
- build.rs: include share.html in the HTML embed pipeline
- web/mod.rs: serve /s/{token} route (unauthenticated)
- share_dto.rs: generate URLs as /s/{token} instead of /api/s/{token}
- share_handler.rs: new download_shared_file() handler that validates the
  share token and streams file content without requiring authentication
- routes.rs: mount GET /api/s/{token}/download (public, uses AppState)
2026-04-12 01:51:30 +02:00
Diocrafts c512534bfa fix: session_expired after login on HTTP deployments (#241)
Three changes to fix the immediate-logout issue reported by multiple
Docker users:

1. Add explicit `credentials: 'same-origin'` to the login fetch call.
   This was the only fetch in the entire codebase missing it. While
   modern browsers default to 'same-origin', some privacy configs or
   older engines may default to 'omit', silently dropping Set-Cookie
   headers from the login response.

2. Post-login cookie verification: after a successful login, the
   frontend now checks that the CSRF cookie (non-HttpOnly, readable
   by JS) was actually stored before redirecting. If the browser
   rejected the cookies, a clear error message is shown explaining
   the OXICLOUD_COOKIE_SECURE / HTTP mismatch.

3. Server-side diagnostic: the login handler now warns in logs when
   Secure cookies are set on a request that didn't arrive via HTTPS
   (no X-Forwarded-Proto: https header), pointing admins to the
   OXICLOUD_COOKIE_SECURE=false fix.

Root cause: users who set OXICLOUD_BASE_URL=https://... (or have
OXICLOUD_COOKIE_SECURE=true) but access via plain HTTP get cookies
with the Secure flag, which browsers silently reject over HTTP.
2026-04-12 01:38:19 +02:00
Diocrafts 5be035a172 fix: restore ext-thumbnail disk lookup in get_cached_thumbnail
After the blob_hash dedup change, get_cached_thumbnail with
blob_hash=None (cache-first fast path) only checked moka. External
video thumbnails stored as ext-{file_id}.jpg on disk were not found
after a server restart when moka is empty.

Now checks ext-{file_id}.jpg on disk before falling through to the
blob-hash disk lookup, so video thumbnails survive restarts.
2026-04-12 00:56:58 +02:00
Diocrafts 2dde4da5cf feat: thumbnail dedup — store thumbnails by blob_hash instead of file_id (#233)
Thumbnails are now keyed by blob_hash on disk so identical files share
a single set of thumbnails (icon/preview/large). For 4000 duplicate
files with the same content, this reduces thumbnail storage from 12,000
files to just 3.

Changes:
- get_thumbnail_path() keys by blob_hash instead of file_id
- get_thumbnail(), get_cached_thumbnail(), generate_all_sizes_background()
  accept blob_hash parameter for disk dedup
- generate_all_sizes_background() fast path: if blob-hash thumbnails
  already exist on disk, skip image processing entirely and just
  populate moka cache for the new file_id
- delete_thumbnails() only invalidates moka cache (shared disk
  thumbnails must not be deleted when one file is removed)
- delete_blob_thumbnails() added for GC; garbage_collect() now cleans
  up orphaned thumbnail files alongside blob files
- External thumbnails (video frames) stored as ext-{file_id}.jpg
  since they are client-generated and not dedup-able
- ThumbnailPort trait updated with blob_hash parameters
- All handler call sites updated (file_handler, preview_handler)
- Tests updated for new signatures
2026-04-12 00:50:10 +02:00
Diocrafts 78fcf5f08f perf: move audio metadata I/O to spawn_blocking + stream reextract_all
- Move all sync I/O (id3::Tag, mp3_duration) into spawn_blocking via
  extract_metadata_blocking() to avoid stalling Tokio worker threads
- Replace fetch_all with streaming .fetch() in reextract_all_audio_metadata
  for O(1) memory usage regardless of audio file count
- Consolidate get_duration_secs into extract_metadata_blocking, eliminating
  redundant file open (tag was read twice before)
2026-04-11 19:36:34 +02:00
Diocrafts 63bcd0ffe7 perf: non-blocking hash + parallel thumbnail generation
- Replace blocking blake3::hash() with DedupService::hash_file() in
  file_upload_service (create_file, update_file). Uses spawn_blocking +
  mmap_rayon instead of blocking the Tokio async worker.
- Parallelize thumbnail resize+encode with rayon par_iter() for the 3
  sizes (Icon, Preview, Large) inside spawn_blocking.
2026-04-11 19:13:48 +02:00
Diocrafts a2aece0752 perf: scale thumbnail decode semaphore with available CPUs
Replace hardcoded DEFAULT_MAX_CONCURRENT_DECODES=4 with adaptive
max_concurrent_decodes() that uses available_parallelism()/2 (min 2).
Matches the pattern already used in image_transcode_service.

Improves thumbnail throughput on 16+ core servers by 2-4x.
2026-04-11 18:14:33 +02:00
Diocrafts d32a32c30a perf: add TTL/TTI to content cache and invalidate on file mutations 2026-04-11 17:39:17 +02:00
Diocrafts 725f661f72 style: fix cargo fmt formatting in dedup_handler 2026-04-11 16:19:49 +02:00
Diocrafts 81f33458e0 perf: streaming hash-on-write dedup upload, remove dead code 2026-04-11 16:17:47 +02:00
Diocrafts 3ce8ec25d6 feat(music): complete music UI overhaul - bugs, UX, missing features & styling
- Fix dead buttons (fa-edit/fa-share → fa-pen/fa-share-alt matching icon registry)
- Fix volume icon, shuffle bias, queue removal, repeat-one CSS
- Replace native prompt/confirm/alert with Modal system
- Add click-to-select, dblclick-to-play, playback error notifications
- Add loading indicators, success toasts, close player button
- Implement remove track from playlist (DELETE endpoint)
- Implement drag & drop track reorder (PUT reorder endpoint)
- Implement manage shares dialog (GET/DELETE shares endpoints)
- Implement cover art upload & display (cover_file_id in UpdatePlaylistDto)
- Implement public/private toggle (is_public in UpdatePlaylistDto)
- Mount get_audio_metadata route in backend routes.rs
- Redesign empty state: full-width centered onboarding when 0 playlists
- Move create button into sidebar header as compact + button
- Add music.css to build.rs production CSS bundle
- Add 16+ i18n keys in en.json and es.json
- Add CSS for drag handles, track remove, cover overlay, shares dialog, public badge
2026-04-11 10:59:01 +02:00
Andrey Tkachenko da066f47fa Music Player & Playlist Manager 2026-04-08 15:14:03 +03:00
titagass 534f278d1c fix(config): use OXICLOUD_SERVER_PORT env var instead of hardcoded 8086 2026-04-03 05:38:44 +00:00
Edouard Vanbelle badf35f08f chore: remove all executable attributes on non bash files 2026-04-01 23:14:42 +02:00
iltumio 9c009aecd1 chore: remove generated openapi.json from tracking, add test and docs
- Remove resources/gen/openapi.json from git (served dynamically at /api/openapi.json)
- Add resources/gen/ to .gitignore
- Add OpenAPI spec validation test (paths, schemas, serialization)
- Restore removed doc-comment on get_version
- Fix cargo fmt violation in mod.rs import
- Update CLAUDE.md: test count (~208), generate-openapi command, justfile reference
2026-04-01 12:25:10 +02:00
iltumio bf7e030cd6 feat: add OpenAPI spec generation with utoipa and justfile
- Add utoipa v5 dependency with ToSchema derives on all REST API DTOs
- Annotate free-function handlers with #[utoipa::path] (trash, share, favorites, recent)
- Create ApiDoc struct with OpenApi derive registering 37 schemas across 7 tags
- Add generate-openapi binary outputting resources/gen/openapi.json
- Serve OpenAPI spec at GET /api/openapi.json (public, no auth)
- Add justfile with common dev commands (build, test, lint, check, openapi, db)
2026-03-29 18:49:10 +02:00
Claude dacb8f388e fix(thumbnail): add timeout protection for large image processing
Resolves conflict between main and PR #229 by applying timeout support
on top of main's drop(data) memory optimization. Changes:
- Add thumbnail_generation_ms to TimeoutConfig (default 30s)
- Add generation_timeout field to ThumbnailService
- Wrap spawn_blocking in tokio::time::timeout to prevent hanging
- Update DI to pass timeout from config
- Update tests to provide timeout parameter

https://claude.ai/code/session_015FD62aAoMYx1XBZbzPvUu8
2026-03-28 18:40:34 +00:00
Edouard Vanbelle d78b279ec9 style: apply rust format + fix clippy warning 2026-03-26 10:05:44 +01:00
Dionisio Pozo 69555d1cd2 Merge pull request #236 from EdouardVanbelle/chore/no-cache-in-dev
chore: static: if PROFILE=dev assets are directly /static without any…
2026-03-25 18:18:03 +01:00
Edouard Vanbelle 8194b01299 chore: static: if PROFILE=dev assets are directly /static without any cache 2026-03-25 16:33:04 +01:00
BillionClaw 7c76b1f04d fix(wopi): support OXICLOUD_WOPI_PUBLIC_BASE_URL env var for Docker deployments
The WOPI integration was not loading documents because the code only read
OXICLOUD_WOPI_BASE_URL, but the documentation and docker-compose examples
used OXICLOUD_WOPI_PUBLIC_BASE_URL. When only WOPI_PUBLIC_BASE_URL was set,
the wopi_base_url defaulted to config.base_url() which resolved to the
internal Docker hostname instead of the public URL.

Fixes #230.
2026-03-24 06:43:43 +08:00
Dionisio Pozo 628707e884 Merge pull request #228 from BillionClaw/clawoss/fix/thumbnail-memory-optimization
fix(thumbnails): drop encoded image data after decoding to reduce memory
2026-03-18 09:39:20 +01:00
BillionClaw 9794bba975 fix(thumbnails): drop encoded image data after decoding to reduce memory usage
Explicitly drop the encoded image buffer after decoding and extracting
EXIF orientation data. This reduces peak memory consumption during
thumbnail generation by the size of the original file.

The encoded data is no longer needed once the image is decoded into
a DynamicImage, but it was being held in memory until the end of the
spawn_blocking scope.

Fixes excessive memory consumption in thumbnail generation.
2026-03-18 11:51:39 +08:00
BillionClaw 5cf0266d24 fix(auth): cap admin initial quota to available disk space
The admin_create_user method was using hardcoded quota values (100GB for
admin, 1GB for user) instead of the capped_quota method that checks
available disk space. This could result in setting a quota higher than
the actual available disk space.

Fixes #92
2026-03-18 11:06:39 +08:00
Dionisio Pozo 4b62499b8f Merge pull request #225 from BillionClaw/clawoss/fix/copy-error-messages
fix: improve error messages for file/folder already exists
2026-03-17 10:07:41 +01:00
BillionClaw aebd51788b fix: improve error messages for file/folder already exists 2026-03-17 17:05:52 +08:00
Dionisio Pozo 775abab8e1 Merge pull request #224 from BillionClaw/clawoss/fix/file-move-folder-ownership
fix(file): verify target folder ownership on move
2026-03-17 09:57:28 +01:00
BillionClaw 30d6e53fca fix(file): verify target folder ownership on move
When moving a file to a folder, verify that the caller owns the target
folder. Without this check, a file could be moved to another user's
folder, causing the file to "disappear" from the original user's view
since file listings filter by user_id.

- Add folder_repo to FileManagementService
- Add verify_target_folder_owner() method
- Call it in move_file_owned() before moving
2026-03-17 16:53:55 +08:00
Dionisio Pozo 79902bf727 Merge pull request #223 from BillionClaw/clawoss/fix/shared-link-404
fix(share): correct shared link URL to include /api prefix
2026-03-17 09:50:15 +01:00
BillionClaw 852db02cd0 fix(share): correct shared link URL to include /api prefix
The shared link URL was generated as {base_url}/s/{token} but the
API endpoint is actually at /api/s/{token}. This caused 404 errors
when users accessed shared links.

Fixes #101
2026-03-17 16:48:11 +08:00
BillionClaw 3c48aa1e6f fix(webdav): preserve correct status codes for file rename/move failures
When file or folder rename/move operations failed in WebDAV handlers,
all errors were incorrectly converted to HTTP 500 Internal Server Error
using AppError::internal_error(). This masked specific error types:

- AlreadyExists errors should return 409 CONFLICT
- NotFound errors should return 404 NOT FOUND
- AccessDenied errors should return 403 FORBIDDEN

Changed error handling to use AppError::from() which preserves the
original DomainError type and maps to appropriate HTTP status codes.

Fixes handling of incorrect status code for file rename failure.
2026-03-17 16:29:21 +08:00
Dionisio Pozo d1699357d8 Merge pull request #221 from BillionClaw/clawoss/fix/trash-view-rendering
fix(trash): add missing display fields to TrashedItemDto
2026-03-17 09:12:28 +01:00
BillionClaw 32c5630a5f fix(trash): add missing display fields to TrashedItemDto
Add category, icon_class, and icon_special_class fields to TrashedItemDto
to fix trash view rendering issues. The frontend expects these fields to
properly display file type information, icons, and deletion dates.

- category: Human-readable file category (e.g., "Image", "Document")
- icon_class: FontAwesome icon class for the file type
- icon_special_class: Special CSS class for icon styling

Fixes #107
2026-03-17 16:08:12 +08:00
BillionClaw ec09136836 fix(webdav): enforce storage quota on WebDAV PUT uploads
Adds storage quota checking to the WebDAV PUT handler, which was missing
while present in other upload handlers (regular upload and chunked upload).

The quota check happens after the file is spooled to a temp file (so we
know the exact size) but before it's moved to permanent storage. If the
quota is exceeded, the temp file is cleaned up and a 507 Insufficient
Storage error is returned.

Fixes #104
2026-03-17 16:04:20 +08:00
BillionClaw 665220dd0b fix(files): batch folder deletion fails
Add debug logging to diagnose batch trash failures for folders.
When batch trash operations fail, the specific error was not being
logged, making it difficult to diagnose issues like #124.

- Add tracing::debug! logs for failed file and folder trash operations
  in batch_operations.rs
- Add unit tests for batch operation result handling to verify
  correct success/failure counting

Fixes #124
2026-03-17 11:21:16 +08:00
BillionClaw 2daee68d20 fix(auth): resolve CSP blocking and session refresh loop
Fix two issues causing login loop after successful admin setup:

1. CSP blocking inline styles: The frontend JavaScript dynamically sets
   inline styles (e.g., element.style.display = 'none') for UI state
   management. The CSP header only allowed 'self' for style-src, blocking
   these dynamic styles. Added 'unsafe-inline' to style-src directive.

2. Session refresh 401 errors: The cookie Secure flag defaulted to true
   when OXICLOUD_BASE_URL was not set, causing cookies to not be sent
   over HTTP in Docker deployments. Changed the default to false when
   the base URL is not explicitly set to HTTPS, with clear logging to
   guide users to set OXICLOUD_COOKIE_SECURE=true for production.

Fixes #203
2026-03-17 06:14:02 +08:00
BillionClaw 0cbbb6b7ac fix: ARMv7 32-bit compilation overflow
The constant 10 * 1024 * 1024 * 1024 (10 GB) overflows on 32-bit systems
where usize is 32-bit (max ~4GB). This caused compilation failures on
ARMv7 architecture.

Fix by using architecture-appropriate limits:
- 64-bit: 10 GB (unchanged)
- 32-bit: 1 GB (safe maximum for 32-bit usize)

Fixes #206
2026-03-17 04:39:18 +08:00
BillionClaw 7645792d5a fix(calendar): change owner_id from String to Uuid type
The database schema defines owner_id as UUID, but the Calendar entity
stored it as String. This caused a type mismatch error when creating
calendars via CalDAV clients like DAVx:

  column "owner_id" is of type uuid but expression is of type text

Changes:
- Change Calendar.owner_id from String to Uuid
- Update new() and with_id() to accept Uuid
- Update owner_id() getter to return &Uuid
- Update belongs_to() to accept &Uuid
- Update calendar_storage_adapter to pass Uuid directly
- Update tests to use Uuid::new_v4()

Fixes #200
2026-03-17 02:58:34 +08:00
Jared Wolff 1fcd02a519 Fix Nextcloud sync conflict by using content-hash ETags
The Nextcloud Android client compares ETags before and after upload to
verify its write landed. OxiCloud was returning the stable file UUID as
the ETag, which never changed on content updates, causing false
SYNC_CONFLICT errors on every upload.

Five fixes applied:
1. Thread blob_hash (SHA-256) through File entity, FileDto, all read/write
   queries, and all WebDAV/PROPFIND responses as the ETag — changes on
   every content update, no DB migration needed.
2. Honor X-OC-Mtime header: parse the client-supplied mtime and use it
   for updated_at via COALESCE(to_timestamp($n), NOW()).
3. Disable phantom checksum capability (preferredUploadType/supportedTypes)
   that the server never actually implemented, stopping retry loops.
4. Add nc:creation_time and nc:upload_time to PROPFIND responses.
5. Return oc-etag header in chunked upload MOVE (assemble) responses.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 14:14:24 -04:00
Julian Merkle 85663ef050 calendar.rs: Allow RGBA colors
The android app "DAVx" creates calendars with RGBA color codes.

fixes #199
2026-03-14 15:48:11 +01:00
Jared Wolff 036390a242 feat: add SQL migration system using sqlx::migrate!()
Replace manual schema.sql application with sqlx's built-in migration
system. Migrations are embedded at compile time and tracked in the
_sqlx_migrations table. Pending migrations run automatically on startup.

- Move db/schema.sql → migrations/20260307000000_initial_schema.sql
- Remove apply_schema() and split_sql_statements() from db.rs
- Add run_migrations() using sqlx::migrate!() macro
- Remove docker-compose schema.sql mount (app handles it now)
- Enable sqlx "migrate" feature in Cargo.toml

Future schema changes: add a new timestamped .sql in migrations/.

Closes #190

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 15:30:52 -04:00
zjean 18518bedaf fix: resolve clippy warnings and rustfmt issues for CI compliance
Fix all clippy lints (collapsible if, clone on Copy, needless borrow,
redundant bindings, unused params) and apply rustfmt across the codebase.
Update test mocks to match Uuid-based trait signatures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 14:34:07 +01:00
Diocrafts df336da679 feat(frontend): i18n expansion, admin/profile i18n, grid/list view fix, empty state
- Add 5 new locales (hi, ar, ru, ja, ko) — now 14 total
- Admin panel: 117 i18n keys, confirm modal, animated tabs, no inline handlers
- Profile page: 58 i18n keys with data-i18n attributes
- Fix i18n safeT() shadowing bug and translationsLoaded timing
- Fix grid/list view: list header no longer shows in grid mode on login
- Fix classList.toggle hidden sync for view switching across all nav functions
- Revert .hidden important that broke login page rendering
- Add files empty state (no_files + empty_hint) with translations
- Fix language selector dropdown scroll and styling
- Fix admin panel scroll with sticky tabs
2026-03-09 00:08:34 +01:00