d7c19570a5
feat(storage): wire choice of storage
2026-08-01 14:19:14 +02:00
2de71b6d9a
feat(storage): add readonly during storage migration
2026-08-01 13:43:04 +02:00
6b7bb67500
feat(storage): jobs can choose storage to migrate/scan
2026-08-01 13:06:09 +02:00
354e058114
feat(storage): add multi entry in config
2026-08-01 12:58:43 +02:00
7534427dc2
fix(oidc): apply clippy recos on PR 652
2026-08-01 12:27:20 +02:00
2de5abc6ca
plan(storage-multi-entry): simplify the storage migration
...
Two chronic problems fall out:
1. **Split-brain config.** Admin edits DB via the panel; app boot ignores DB.
Migration completes; live backend hasn't moved. Admin has to remember to
copy env vars into `.env` and restart. Two sources of truth for the same
setting. Cutover is a manual multi-step flow; users routinely get it wrong.
2. **Migration data-loss window on concurrent writes.** The copy walks
`storage.blobs` in hash order. A blob whose hash is lex-lower than the
current cursor, written to source AFTER migration passed it, is never
copied to target. `passed=true, findings=0` completion does NOT guarantee
target has every blob. Silent.
3. **Migration target selection is fragile.** DTO passes the whole S3 config
at trigger time; secrets sit plaintext in `admin_settings`. Any future
pluggable-storage story compounds this (Azure, GCS, WebDAV-as-source, …).
This plan replaces the split-brain model with a single-source-of-truth
architecture:
- `.env` declares **N named storage entries** (immutable per-deploy).
- `admin_settings.storage.active_backend_name` holds ONE row — which named
entry the app currently runs on. That's the whole runtime config.
- Migration is the atomic transition from one active entry to another. Server
is put in read-only mode for the copy window; on completion, the active
pointer flips; a restart cuts over.
2026-08-01 12:05:01 +02:00
82a50936d4
feat(storage-migration): move storage mig. to recoverable job
2026-08-01 11:53:51 +02:00
0c6a33c721
Merge pull request #652 from DragonDuck24/oidc-check-require-email-verification
...
fix: Make oidc check if email verification is required
2026-08-01 00:12:33 +02:00
1801150ad4
make oidc check if email verification is required
2026-07-31 11:09:33 -05:00
8ba2d81297
Merge pull request #651 from EdouardVanbelle/feat/recoverable-jobs
...
feat(recoverable jobs): add engine + consistency jobs
2026-07-30 10:30:39 +02:00
53a80cf181
Merge pull request #650 from EdouardVanbelle/chore/db-migration-check
...
ci: add protection on merge and db migration
2026-07-30 10:30:03 +02:00
d3c2fc3e94
fix(job): correct amount of jobs + remove cound due to grace window
2026-07-30 02:00:39 +02:00
5a87999949
fix(integration test): fix drive test used bytes with grace period
2026-07-30 01:29:09 +02:00
48cbec8fae
feat(admin ui): too many tabs, change display
2026-07-30 01:14:35 +02:00
9fe47a53ea
feat(jobs): add admin call to purge jubs result
2026-07-30 00:39:11 +02:00
507bc2e98d
feat(recoverable-job): add backend_consistency (storage)
2026-07-30 00:15:20 +02:00
07beb461d6
feat(jobs/ui): hide consistency_batch, the main button is here
2026-07-29 23:22:36 +02:00
8de50ec404
feat(jobs/ui): blobs_consistency add b3sum check
2026-07-29 23:15:45 +02:00
e822714b50
feat(jobs/ui): reorder job names + fix(blobs_consistency)
2026-07-29 23:01:20 +02:00
5527d09618
feat(recoverable-job): check if old blob (no cdc) still remains: notice only
2026-07-29 22:47:07 +02:00
61b8571a48
feat(recoverable-job): add grace period for drives, files, folders consistency
2026-07-29 22:29:20 +02:00
44356b14f2
feat(recoverable-job): add blobs_consistency job
2026-07-29 22:25:05 +02:00
0f12399a48
feat(recoverable-job): fix files_consistency to check blob chunk consistency
2026-07-29 22:17:15 +02:00
5881968f50
feat(recoverable-job): add progress view
2026-07-29 08:41:19 +02:00
4336eca4d1
feat(recoverable-job): add admin page
2026-07-29 08:03:43 +02:00
e1556e3d36
feat(recoverable-job): add findings
2026-07-29 07:57:06 +02:00
41d83b3053
feat(recoverable-job): add consistency_batch (runs all consistency check)
2026-07-29 01:44:52 +02:00
782a5c99bd
feat(recoverable-job): add folder_consistency
2026-07-29 01:39:58 +02:00
394708c9a1
feat(recov. job): add drive consistency service
2026-07-29 01:25:15 +02:00
b343ab5e0e
plan(job): clarify way to split consistency job
2026-07-29 01:22:12 +02:00
8177b04786
feat(recoverable-job): wire API (cancel, view run, ...)
2026-07-28 23:25:37 +02:00
302d2ff80f
feat(recoverable-job): add core engine
2026-07-28 22:39:07 +02:00
2315a54154
ci: add protection on merge and db migration
...
this is a protection to prevent the merge of any branch adding a database
migration with an timestamp earlier than the current targetted branch
2026-07-28 22:07:45 +02:00
10ae6c3e44
doc(job-registry): add SQL entry for recoverable jobs
2026-07-28 22:01:11 +02:00
0b7618d858
clarify naming conventions
2026-07-28 22:00:28 +02:00
996cb98a6d
doc(job-registry): add doc for implementors
2026-07-28 21:24:09 +02:00
0e8b1fbbeb
refactor(job-registry): simplify the job registering*
2026-07-28 21:13:09 +02:00
4d64603233
Merge pull request #573 from EdouardVanbelle/plan/consistency-check+job-registry
...
feat(job registry): normalize engine to handle periodic or event's triggered job
2026-07-28 17:53:25 +02:00
4ab57d0d8e
test(job-registry): additional hurl coverage
2026-07-28 00:35:34 +02:00
24c514fe59
fix(job-registry): fix error handling*
2026-07-28 00:35:31 +02:00
f66f7fa31f
feat(job-registry): remplace /api/admin/internal/trigger-*
...
remplace /api/admin/internal/trigger-* to /api/admin/jobs/{...}/trigger
remove OXICLOUD_ENABLE_ADMIN_INTERNAL_ENDPOINTS
2026-07-27 23:54:43 +02:00
dfedde54a4
feat(job-registry): wire /api/admin/jobs/*
2026-07-27 23:38:12 +02:00
72b99f5b9a
refactor: apply clippy
2026-07-27 23:06:02 +02:00
1190dfef87
feat(job-registry): wire grant cleaner
2026-07-27 23:03:22 +02:00
f740de3c10
feat(job-registry): wire dedub_gc
2026-07-27 22:56:02 +02:00
f5f794fde5
feat(job-registry): handle job without periodicity but with trigger
2026-07-27 22:53:13 +02:00
e29f3ba067
feat(job-registry): wire storage usage update
2026-07-27 22:33:04 +02:00
d0f4e1ef5a
feat(job-registry): wire engine + wire trash cleaner
2026-07-27 22:24:15 +02:00
8bb62affc8
feat(job-registry): add the job engine
2026-07-27 22:05:08 +02:00
5e8d894a0c
chore(plan): add job-registry + consistency check
...
purpose is to design a job registry with a scheduler
thi aim to drive in the same way any services requiring execution of periodic background tasks
plugins could benefeciate it
purpose is mostly to add a normative way to implement consistency check per services
this is to edutcate implementors adding any new services
goal is to ensure data quality with oxicloud and resumable jobs by default
2026-07-27 21:53:00 +02:00
Edouard Vanbelle