Commit Graph

1809 Commits

Author SHA1 Message Date
Edouard Vanbelle ddede168f9 feat(job): delete temp files at boot time (only possible case: crash) 2026-08-02 02:40:33 +02:00
Edouard Vanbelle bc481bdd99 feat(job): fix key rotation on local storage (replace blob) 2026-08-02 02:40:33 +02:00
Edouard Vanbelle e12d8bcb90 feat(job): show if a job is recverable 2026-08-02 02:40:33 +02:00
Edouard Vanbelle a58351b7ad feat(storage key rot): add admin panel 2026-08-02 02:40:33 +02:00
Edouard Vanbelle a9d5aae781 feat(storage key rot): add rotate services 2026-08-02 02:40:33 +02:00
Edouard Vanbelle 30b0000c26 feat(storage key rot): add blob/chunk creation with the header 2026-08-02 02:40:33 +02:00
Edouard Vanbelle 9485ee5540 feat(storage key rot): add blob header engine 2026-08-02 02:40:33 +02:00
Edouard Vanbelle e164689771 feat(storage key rot): remove dead born OXICLOUD_STORAGE_<N>_ENCRYPTION_CIPHER
+ alway ovewrite on storage migration (got issue when migrating with blob already existing and a key change)
2026-08-02 02:40:33 +02:00
Edouard Vanbelle 03c8f87f1f feat(storage key rot): prepare format <cipher1>:<key2>,<cipher2>:<key2>,... 2026-08-02 02:40:33 +02:00
Edouard Vanbelle 4cb73eaf39 plan(storage-key-rotation): add a key rotation + header version blob 2026-08-02 02:40:33 +02:00
Dionisio Pozo 0660ac1e6c Merge pull request #654 from EdouardVanbelle/feat/storage-migration 2026-08-02 00:15:53 +02:00
Edouard Vanbelle f1c72f8837 test(storage): adapt playwright admin tests 2026-08-01 20:35:04 +02:00
Edouard Vanbelle 836c7a57c1 audit(RUSTSEC-2026-0222): inhibit alert, wasmtime plugin are not used per today 2026-08-01 20:30:33 +02:00
Edouard Vanbelle 88921c975a fix(hurl test): add new job 2026-08-01 20:08:17 +02:00
Edouard Vanbelle c31b8b814d fix(oidc): change the test 2026-08-01 19:58:38 +02:00
Edouard Vanbelle bbfb106a32 feat(maintenance): add a maintenance notification during backend migration 2026-08-01 18:46:17 +02:00
Edouard Vanbelle 142afecbbf feat(storage): add a guide on storage 2026-08-01 17:50:35 +02:00
Edouard Vanbelle f409658c96 feat(storage): hot swap backend on migrate 2026-08-01 17:11:07 +02:00
Edouard Vanbelle 8329b4aa56 feat(storage): improve admin panel 2026-08-01 17:10:06 +02:00
Edouard Vanbelle cc439aaff9 fix(dedup): fix informations 2026-08-01 16:20:49 +02:00
Edouard Vanbelle f949939508 feat(storage): add cmd option --select-storage 2026-08-01 14:36:24 +02:00
Edouard Vanbelle d7c19570a5 feat(storage): wire choice of storage 2026-08-01 14:19:14 +02:00
Edouard Vanbelle 2de71b6d9a feat(storage): add readonly during storage migration 2026-08-01 13:43:04 +02:00
Edouard Vanbelle 6b7bb67500 feat(storage): jobs can choose storage to migrate/scan 2026-08-01 13:06:09 +02:00
Edouard Vanbelle 354e058114 feat(storage): add multi entry in config 2026-08-01 12:58:43 +02:00
Edouard Vanbelle 7534427dc2 fix(oidc): apply clippy recos on PR 652 2026-08-01 12:27:20 +02:00
Edouard Vanbelle 2de5abc6ca plan(storage-multi-entry): simplify the storage migration
Two chronic problems fall out:

1. **Split-brain config.** Admin edits DB via the panel; app boot ignores DB.
   Migration completes; live backend hasn't moved. Admin has to remember to
   copy env vars into `.env` and restart. Two sources of truth for the same
   setting. Cutover is a manual multi-step flow; users routinely get it wrong.

2. **Migration data-loss window on concurrent writes.** The copy walks
   `storage.blobs` in hash order. A blob whose hash is lex-lower than the
   current cursor, written to source AFTER migration passed it, is never
   copied to target. `passed=true, findings=0` completion does NOT guarantee
   target has every blob. Silent.

3. **Migration target selection is fragile.** DTO passes the whole S3 config
   at trigger time; secrets sit plaintext in `admin_settings`. Any future
   pluggable-storage story compounds this (Azure, GCS, WebDAV-as-source, …).

This plan replaces the split-brain model with a single-source-of-truth
architecture:

- `.env` declares **N named storage entries** (immutable per-deploy).
- `admin_settings.storage.active_backend_name` holds ONE row — which named
  entry the app currently runs on. That's the whole runtime config.
- Migration is the atomic transition from one active entry to another. Server
  is put in read-only mode for the copy window; on completion, the active
  pointer flips; a restart cuts over.
2026-08-01 12:05:01 +02:00
Edouard Vanbelle 82a50936d4 feat(storage-migration): move storage mig. to recoverable job 2026-08-01 11:53:51 +02:00
Dionisio Pozo 0c6a33c721 Merge pull request #652 from DragonDuck24/oidc-check-require-email-verification
fix: Make oidc check if email verification is required
2026-08-01 00:12:33 +02:00
Drake 1801150ad4 make oidc check if email verification is required 2026-07-31 11:09:33 -05:00
Dionisio Pozo 8ba2d81297 Merge pull request #651 from EdouardVanbelle/feat/recoverable-jobs
feat(recoverable jobs): add engine + consistency jobs
2026-07-30 10:30:39 +02:00
Dionisio Pozo 53a80cf181 Merge pull request #650 from EdouardVanbelle/chore/db-migration-check
ci: add protection on merge and db migration
2026-07-30 10:30:03 +02:00
Edouard Vanbelle d3c2fc3e94 fix(job): correct amount of jobs + remove cound due to grace window 2026-07-30 02:00:39 +02:00
Edouard Vanbelle 5a87999949 fix(integration test): fix drive test used bytes with grace period 2026-07-30 01:29:09 +02:00
Edouard Vanbelle 48cbec8fae feat(admin ui): too many tabs, change display 2026-07-30 01:14:35 +02:00
Edouard Vanbelle 9fe47a53ea feat(jobs): add admin call to purge jubs result 2026-07-30 00:39:11 +02:00
Edouard Vanbelle 507bc2e98d feat(recoverable-job): add backend_consistency (storage) 2026-07-30 00:15:20 +02:00
Edouard Vanbelle 07beb461d6 feat(jobs/ui): hide consistency_batch, the main button is here 2026-07-29 23:22:36 +02:00
Edouard Vanbelle 8de50ec404 feat(jobs/ui): blobs_consistency add b3sum check 2026-07-29 23:15:45 +02:00
Edouard Vanbelle e822714b50 feat(jobs/ui): reorder job names + fix(blobs_consistency) 2026-07-29 23:01:20 +02:00
Edouard Vanbelle 5527d09618 feat(recoverable-job): check if old blob (no cdc) still remains: notice only 2026-07-29 22:47:07 +02:00
Edouard Vanbelle 61b8571a48 feat(recoverable-job): add grace period for drives, files, folders consistency 2026-07-29 22:29:20 +02:00
Edouard Vanbelle 44356b14f2 feat(recoverable-job): add blobs_consistency job 2026-07-29 22:25:05 +02:00
Edouard Vanbelle 0f12399a48 feat(recoverable-job): fix files_consistency to check blob chunk consistency 2026-07-29 22:17:15 +02:00
Edouard Vanbelle 5881968f50 feat(recoverable-job): add progress view 2026-07-29 08:41:19 +02:00
Edouard Vanbelle 4336eca4d1 feat(recoverable-job): add admin page 2026-07-29 08:03:43 +02:00
Edouard Vanbelle e1556e3d36 feat(recoverable-job): add findings 2026-07-29 07:57:06 +02:00
Edouard Vanbelle 41d83b3053 feat(recoverable-job): add consistency_batch (runs all consistency check) 2026-07-29 01:44:52 +02:00
Edouard Vanbelle 782a5c99bd feat(recoverable-job): add folder_consistency 2026-07-29 01:39:58 +02:00
Edouard Vanbelle 394708c9a1 feat(recov. job): add drive consistency service 2026-07-29 01:25:15 +02:00