Commit Graph

6 Commits

Author SHA1 Message Date
Edouard Vanbelle 4154019603 feat(ui:delta): add heartbeat on worker 2026-08-14 01:52:53 +02:00
Edouard Vanbelle 24da7443de feat(ui delta upload): user can change oxi.UPLOAD_BATCH_BYTES
```
oxi.UPLOAD_BATCH_BYTES // get current value
oxi.UPLOAD_BATCH_BYTES=1024*1024 // change values
```

values are stored in localstorage, default is 8*1024*1024
2026-08-14 01:52:14 +02:00
Edouard Vanbelle e357caf43b feat(ui): add a logger to change log level and permit better diagnostics
in this case logs are added in the delta upload worker

you can increase verbosity from console via:

```javascript
// Usage:
oxi.setLogLevel('oxi:upload', 'debug')    // deep dive
oxi.setLogLevel('oxi:upload', 'warn')     // quiet
oxi.log.setLevel('debug')                  // everything to debug
```

values are stored in localstorage

example:

```
oxi.setLogLevel('oxi:upload', 'debug');
'oxi:upload → debug'
deltaUpload.ts:120 [f32ce1] delta start {file: 'Revue de presseg.odp', size: 23658927}
deltaUpload.ts:185 [f32ce1] worker: worker start {file: 'Revue de presseg.odp', size: 23658927}
deltaUpload.ts:186 [f32ce1] worker: wasm loaded
deltaUpload.ts:186 [f32ce1] worker: hashed — blake3=150edbc35f3475a0bdeecd15fe5278dda305ce3b27e53e9c40515a69f0b883f3 (76 chunks)
deltaUpload.ts:186 [f32ce1] worker: negotiate: 76 hashes → 76 missing, 0 dedup'd
deltaUpload.ts:186 [f32ce1] worker: chunk PUT: 28 chunks, 8825338 bytes
deltaUpload.ts:186 [f32ce1] worker: chunk PUT: 29 chunks, 8950230 bytes
deltaUpload.ts:186 [f32ce1] worker: chunk PUT: 19 chunks, 5883663 bytes
deltaUpload.ts:186 [f32ce1] worker: ✅ committed — uploaded 23 658 927 B (no dedup, blake3=150edbc35f3475a0bdeecd15fe5278dda305ce3b27e53e9c40515a69f0b883f3)
deltaUpload.ts:185 [f32ce1] worker: commit HTTP 201 {blake3: '150edbc35f3475a0bdeecd15fe5278dda305ce3b27e53e9c40515a69f0b883f3', uploadedBytes: 23658927, reusedBytes: 0, totalBytes: 23658927, attempt: 0}
deltaUpload.ts:213 [f32ce1] delta done {file: 'Revue de presseg.odp', blake3: '150edbc35f3475a0bdeecd15fe5278dda305ce3b27e53e9c40515a69f0b883f3', savedBytes: 0, uploadedBytes: 23658927}
```
2026-08-14 01:52:03 +02:00
DioCrafts d66956824c Optimize storage, GC, and upload hot paths 2026-07-22 02:06:29 +02:00
DioCrafts 6be3c99580 fix(upload): bound delta-worker connections instead of disabling delta
Follow-up to the connection-exhaustion fix. Rather than routing large files to
plain uploads (which kept STORAGE dedup but gave up delta's re-upload bandwidth
savings), keep delta for every file >= 8 MB and instead cap each worker's
concurrent connections so a few large files uploading at once can't blow past
the browser's ~6-per-host budget and starve the small-file plain uploads.

- deltaWorker.js: serialize negotiate (at most one in flight per worker) and
  drop chunk-PUT concurrency 2 -> 1, so each worker holds ~2 connections max.
- deltaUpload.ts: revert the 64 MB threshold back to 8 MB — every large file
  gets sub-file dedup again. (Storage dedup was never affected: BLAKE3 + CDC +
  ref-counting run server-side for plain and delta uploads alike.)

With main upload concurrency at 2, total in-flight upload connections stay <= ~4.

npm run check: clean, 58 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 03:09:45 +02:00
Bradley Nelson 89e14f8f9e visual continunity 2026-06-17 22:07:18 -06:00