refactor(backend): normalize naming convention to backend rather storage
no ambiguity with the backend rather storage
This commit is contained in:
Edouard Vanbelle
@@ -159,7 +159,7 @@ Properties this buys us:
|
||||
format-conversion work.
|
||||
* **Lazy conversion on hot paths.** Any COW overwrite (WebDAV MOVE, PUT-over,
|
||||
content-hash re-upload) naturally lands as v1 at the same object key.
|
||||
* **Explicit conversion via `storage_rotate`.** The rotate job walks
|
||||
* **Explicit conversion via `backend_rotate`.** The rotate job walks
|
||||
`storage.blobs`, reads each blob via the magic-byte dispatch, and if the
|
||||
blob is not already v1 with the head-pair key, PUTs it back as v1 with
|
||||
the head pair — in place, same object key.
|
||||
@@ -186,7 +186,7 @@ Guardrail:
|
||||
whatever consistency scan cadence the deployment has (weekly by default; on
|
||||
demand from the admin panel).
|
||||
* The *"Rotation complete — safe to remove the old key"* hint appears in the
|
||||
entry card only when the last `storage_rotate` run completed with zero
|
||||
entry card only when the last `backend_rotate` run completed with zero
|
||||
findings AND the most recent consistency scan reported zero legacy blobs.
|
||||
* Nothing enforces removal at code level — the admin is trusted, given a
|
||||
clear signal, and warned.
|
||||
@@ -226,11 +226,11 @@ disk pre-date the v1-code deployment.
|
||||
|
||||
### The rotation job
|
||||
|
||||
New `RecoverableJobHandler` tenant, `storage_rotate`. Mirrors
|
||||
`storage_migration`'s shape:
|
||||
New `RecoverableJobHandler` tenant, `backend_rotate`. Mirrors
|
||||
`backend_migration`'s shape:
|
||||
|
||||
* **Iterates `storage.blobs`** in hash-lex order. Cursor is the last-processed
|
||||
hash (64 hex chars). Same cursor encoding as `storage_migration` and
|
||||
hash (64 hex chars). Same cursor encoding as `backend_migration` and
|
||||
`blobs_consistency`.
|
||||
* **Per blob:**
|
||||
1. Fetch `<hash>.blob` and dispatch via the standard read path.
|
||||
@@ -256,7 +256,7 @@ New `RecoverableJobHandler` tenant, `storage_rotate`. Mirrors
|
||||
* The v1 write is atomic at object-storage level (S3 replace, Local
|
||||
rename-into-place). A concurrent reader sees either state.
|
||||
* No readonly mode. This is a critical improvement over
|
||||
`storage_migration`: rotation is per-blob idempotent, so we don't need
|
||||
`backend_migration`: rotation is per-blob idempotent, so we don't need
|
||||
to freeze writes.
|
||||
* **Restart-survivable** — same boot-time sweep as every other recoverable
|
||||
handler.
|
||||
@@ -274,7 +274,7 @@ Preconditions:
|
||||
`OXICLOUD_STORAGE_<name>_ENCRYPTION_KEY` first, or wait for legacy blobs to
|
||||
accumulate — nothing to rotate right now."*
|
||||
|
||||
Clicking the button dispatches the `storage_rotate` job for that entry. The
|
||||
Clicking the button dispatches the `backend_rotate` job for that entry. The
|
||||
job's progress rides on the same `X-Server-Status` header infrastructure the
|
||||
maintenance banner uses — but this time WITHOUT engaging read-only mode.
|
||||
Banner variant reads *"Rotating encryption key on `<entry>` — X% (Y / Z
|
||||
@@ -285,7 +285,7 @@ The entry card shows a **legacy-blob counter** sourced from the most recent
|
||||
next rotation)"*. Refresh-on-demand button next to it triggers a targeted
|
||||
`blobs_consistency` scan (already available via the admin surface). The
|
||||
*"Rotation complete — safe to remove the old key"* hint appears only when
|
||||
N = 0 and the last `storage_rotate` run completed with zero findings.
|
||||
N = 0 and the last `backend_rotate` run completed with zero findings.
|
||||
|
||||
### Removing the old pair
|
||||
|
||||
@@ -307,7 +307,7 @@ Zero admin work required. On upgrade:
|
||||
key.
|
||||
* Existing legacy blobs stay readable via the magic-byte dispatch — the
|
||||
legacy read path is preserved verbatim.
|
||||
* Admin can optionally trigger a `storage_rotate` run to consolidate every
|
||||
* Admin can optionally trigger a `backend_rotate` run to consolidate every
|
||||
legacy blob into v1 format. Not required — legacy blobs migrate
|
||||
opportunistically via COW overwrites and stay readable indefinitely
|
||||
otherwise.
|
||||
@@ -347,7 +347,7 @@ The user-facing recipe (goes verbatim into `docs/guide/backend-storage.md`):
|
||||
2. Add it AFTER `none`:
|
||||
OXICLOUD_STORAGE_local_main_ENCRYPTION_KEY=none:,aes-256-gcm:<K>
|
||||
3. Restart. New uploads are encrypted; existing plaintext blobs stay readable.
|
||||
4. Run `storage_rotate` to encrypt existing blobs in place.
|
||||
4. Run `backend_rotate` to encrypt existing blobs in place.
|
||||
5. Remove `none:` from the list; restart.
|
||||
```
|
||||
|
||||
@@ -357,7 +357,7 @@ The user-facing recipe (goes verbatim into `docs/guide/backend-storage.md`):
|
||||
1. Add `none:` AFTER the current encryption key:
|
||||
OXICLOUD_STORAGE_local_main_ENCRYPTION_KEY=aes-256-gcm:<K>,none:
|
||||
2. Restart. New uploads are plaintext; existing encrypted blobs stay readable.
|
||||
3. Run `storage_rotate` to decrypt existing blobs in place.
|
||||
3. Run `backend_rotate` to decrypt existing blobs in place.
|
||||
4. Remove the key pair, keep `none:` only (or drop `_ENCRYPTION_KEY` entirely);
|
||||
restart.
|
||||
```
|
||||
@@ -395,7 +395,7 @@ modules within source files").
|
||||
never silent misread. Keeps the "collisions can't silently corrupt" claim
|
||||
in the plan honest.
|
||||
* **Rotation decision tree.** Unit tests on the per-blob `decide()` helper of
|
||||
`storage_rotate_service.rs`. All six cases from *The rotation job* section
|
||||
`backend_rotate_service.rs`. All six cases from *The rotation job* section
|
||||
as separate tests with clear names (`legacy_upgrades_to_v1`,
|
||||
`v1_encrypted_under_head_skips`,
|
||||
`v1_encrypted_under_older_pair_rewrites`,
|
||||
@@ -403,13 +403,13 @@ modules within source files").
|
||||
`v1_plaintext_encrypts_when_head_is_cipher`,
|
||||
`v1_encrypted_decrypts_when_head_is_none`).
|
||||
* **Recoverable-job round-trip.** Integration test in
|
||||
`storage_rotate_service::tests` using the existing recoverable-run harness:
|
||||
`backend_rotate_service::tests` using the existing recoverable-run harness:
|
||||
seed N blobs (mix of legacy + v1-under-old-key), trigger rotation, assert
|
||||
every blob ends v1-with-head, `format_generation` (if we add it later) or
|
||||
the consistency-scan count reports zero legacy remaining, findings=0.
|
||||
* **Crash recovery.** Same harness: interrupt mid-run at cursor position K,
|
||||
restart, assert resume from K and eventual completion with correct final
|
||||
state. Same discipline `storage_migration` already uses.
|
||||
state. Same discipline `backend_migration` already uses.
|
||||
* **Concurrency safety.** Test that a `put_blob` call during a rotation
|
||||
targeting the same hash produces exactly one v1 blob at end-state (either
|
||||
the rotate's or the concurrent write's — both are head-format so the final
|
||||
@@ -496,19 +496,19 @@ the legacy-blob count.
|
||||
deployment reads existing legacy blobs and writes new v1 blobs at the same
|
||||
object-key. `blobs_consistency` reports a legacy-blob count in its run stats.
|
||||
|
||||
### Slice K3 — The `storage_rotate` recoverable job
|
||||
### Slice K3 — The `backend_rotate` recoverable job
|
||||
|
||||
**Scope.** New handler, admin-triggered, iterates blobs, per-blob decision
|
||||
tree (legacy → v1 upgrade, v1 with old key → v1 with head key, plaintext ↔
|
||||
encrypted where applicable), records findings.
|
||||
|
||||
* New file `src/infrastructure/services/storage_rotate_service.rs`.
|
||||
* Registered in `JobRegistry` as `storage_rotate`. Runs on the same
|
||||
* New file `src/infrastructure/services/backend_rotate_service.rs`.
|
||||
* Registered in `JobRegistry` as `backend_rotate`. Runs on the same
|
||||
recoverable-runs engine (crash recovery, cursor persistence, pause/resume).
|
||||
* Trigger endpoint: `POST /api/admin/storage/entries/{name}/rotate`.
|
||||
Requires admin. Refuses if no work would happen (all blobs already at
|
||||
head format + head key). Refuses if a `storage_rotate` or
|
||||
`storage_migration` run is already Active for any entry.
|
||||
head format + head key). Refuses if a `backend_rotate` or
|
||||
`backend_migration` run is already Active for any entry.
|
||||
* Per-blob decision tree per *The rotation job* section above. In-place
|
||||
atomic replace at the same `<hash>.blob` object key.
|
||||
* No readonly mode engaged. `X-Server-Status` header payload gains a
|
||||
@@ -566,7 +566,7 @@ changes. Reserved slots:
|
||||
|
||||
v1 and v2 coexist in the same storage indefinitely — the magic-byte read
|
||||
dispatch handles arbitrary versions at position 5-6. Migration between
|
||||
generations reuses `storage_rotate`'s pattern: rewrite each blob with the
|
||||
generations reuses `backend_rotate`'s pattern: rewrite each blob with the
|
||||
new-generation writer, in-place at the same object key.
|
||||
|
||||
## Non-goals
|
||||
@@ -592,7 +592,7 @@ new-generation writer, in-place at the same object key.
|
||||
|
||||
## Open questions
|
||||
|
||||
* **Should we throttle the rotate job?** Same question `storage_migration` had.
|
||||
* **Should we throttle the rotate job?** Same question `backend_migration` had.
|
||||
Answer: not in v1. If throughput bites, add a `_ROTATE_MAX_MB_PER_SEC` on
|
||||
the entry later.
|
||||
* **Should rotation be idempotent under repeat trigger?** Yes. Running it a
|
||||
|
||||
Reference in New Issue
Block a user