fix(thumbnails): key the ETag on content hash, not file id
The thumbnail ETag was "thumb-{file_id}-{size}-{format}", sent with
Cache-Control: public, max-age=31536000, immutable. Replacing a file's
content preserves its id — file_upload_service rebuilds the entity with
parts.id and a new hash, then fires on_file_updated, which deletes and
regenerates the thumbnails — so the server produced a new thumbnail while
still advertising the old ETag. Because `immutable` tells a conforming
browser not to revalidate at all inside the freshness window, clients kept
rendering the previous image for up to a year, unfixably.
Keyed on the content hash the directive becomes honest: a thumbnail is a
pure function of (source bytes, size, format), so that triple identifies
the response. New content yields a new ETag.
The same change fixes the opposite direction. A copy, or any dedup twin,
had a different id and therefore a different ETag, so clients refetched
bytes they already held even though both are served from the same derived
blob. Now identical content agrees on an ETag and revalidates to 304
across files, users and copies.
Both thumbnail endpoints were affected: the REST handler and the
NextCloud preview handler.
Cost is one PK lookup ahead of the 304 decision, where the id-keyed
version needed none — paid for by no longer serving stale images. It is
partly recovered: both handlers already resolved the same hash further
down for the render path, and that second lookup is now gone, so the
cache-miss path is unchanged and only the 304 path pays. The resolved
hash is also handed to get_cached_thumbnail instead of None, saving the
service its own lookup.
No new disclosure: content_hash is already on FileDto and returned by
GET /api/files/{id}.
Tests: thumbnail_etag_content_keyed.hurl covers invalidation — overwrite
in place via WebDAV PUT, assert the ETag changed, assert a client holding
the stale one gets 200 rather than 304. derived_blob_copy.hurl gains the
sharing direction: a copy answers with the SAME ETag and revalidates to
304, which is the one externally observable consequence of content-keying
and was not previously testable.
This commit is contained in:
@@ -140,6 +140,7 @@ Authorization: Bearer {{token}}
|
||||
HTTP 200
|
||||
[Captures]
|
||||
thumb_bytes: bytes
|
||||
thumb_etag: header "ETag"
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
@@ -173,10 +174,14 @@ jsonpath "$.ref_count" == 2
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# Step 7 – The copy is readable and renders the same bytes.
|
||||
# Step 7 – The copy is readable, renders the same bytes, and carries the
|
||||
# SAME ETag as the original.
|
||||
#
|
||||
# NOT a proof of derived-blob sharing — see the header. This catches the
|
||||
# copy being unreadable or resolving to different content.
|
||||
# The ETag is keyed on the content hash, which the copy shares. Two
|
||||
# different files agreeing on an ETag is the one externally visible
|
||||
# consequence of content-keying — a file-id-keyed ETag could not produce
|
||||
# it. The 304 below is the payoff: a client that already holds the
|
||||
# original's thumbnail does not refetch it for the copy.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
GET {{base_url}}/api/files/{{file_copy_id}}/thumbnail/preview
|
||||
Authorization: Bearer {{token}}
|
||||
@@ -184,6 +189,16 @@ Authorization: Bearer {{token}}
|
||||
HTTP 200
|
||||
[Asserts]
|
||||
bytes == {{thumb_bytes}}
|
||||
header "ETag" == "{{thumb_etag}}"
|
||||
|
||||
|
||||
GET {{base_url}}/api/files/{{file_copy_id}}/thumbnail/preview
|
||||
Authorization: Bearer {{token}}
|
||||
If-None-Match: {{thumb_etag}}
|
||||
|
||||
HTTP 304
|
||||
[Asserts]
|
||||
header "ETag" == "{{thumb_etag}}"
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -167,6 +167,7 @@ hurl --variables-file "$API_DIR/test.env" --file-root "$REPO_ROOT/tests" --test
|
||||
"$API_DIR/batch_folder_copy.hurl" \
|
||||
"$API_DIR/dedup_blob_cleanup.hurl" \
|
||||
"$API_DIR/derived_blob_copy.hurl" \
|
||||
"$API_DIR/thumbnail_etag_content_keyed.hurl" \
|
||||
"$API_DIR/dedup_admin_gate.hurl" \
|
||||
"$API_DIR/admin_jobs.hurl" \
|
||||
"$API_DIR/recoverable_jobs.hurl" \
|
||||
|
||||
@@ -0,0 +1,161 @@
|
||||
# =============================================================
|
||||
# OxiCloud – Thumbnail ETag is keyed on CONTENT, not on file id
|
||||
# =============================================================
|
||||
# Regression guard for a stale-cache bug.
|
||||
#
|
||||
# The thumbnail ETag used to be `"thumb-{file_id}-{size}-{format}"`, sent
|
||||
# with `Cache-Control: public, max-age=31536000, immutable`. Replacing a
|
||||
# file's content preserves its id — the upload service rebuilds the entity
|
||||
# with `parts.id` and a new hash, then fires `on_file_updated`, which
|
||||
# regenerates the thumbnails — so the server produced a NEW thumbnail while
|
||||
# advertising the OLD ETag. And `immutable` tells a conforming browser not
|
||||
# to revalidate at all inside the freshness window, so clients kept showing
|
||||
# the previous image for up to a year with no way to invalidate it.
|
||||
#
|
||||
# Keying on the content hash fixes it: new bytes → new hash → new ETag.
|
||||
#
|
||||
# This file asserts the invalidation direction. The sharing direction (two
|
||||
# distinct files with identical content answering with the SAME ETag, so a
|
||||
# copy revalidates to 304) is covered in `derived_blob_copy.hurl`.
|
||||
#
|
||||
# Overwrite goes through WebDAV PUT because that is the path that replaces
|
||||
# content in place; the REST upload endpoint creates a new file instead.
|
||||
#
|
||||
# Prerequisites: setup.hurl must have run (admin user exists).
|
||||
# =============================================================
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# Step 1 – Login
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
POST {{base_url}}/api/auth/login
|
||||
Content-Type: application/json
|
||||
{
|
||||
"username": "{{username}}",
|
||||
"password": "{{password}}"
|
||||
}
|
||||
|
||||
HTTP 200
|
||||
[Captures]
|
||||
token: jsonpath "$.access_token"
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# Step 2 – Upload the first image
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
POST {{base_url}}/api/files/upload
|
||||
Authorization: Bearer {{token}}
|
||||
[MultipartFormData]
|
||||
file: file,fixtures/red-image.png; image/png
|
||||
|
||||
HTTP 201
|
||||
[Captures]
|
||||
file_id: jsonpath "$.id"
|
||||
file_name: jsonpath "$.name"
|
||||
hash_before: jsonpath "$.content_hash"
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# Step 3 – Its thumbnail, and the ETag that goes with it
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
GET {{base_url}}/api/files/{{file_id}}/thumbnail/preview
|
||||
Authorization: Bearer {{token}}
|
||||
|
||||
HTTP 200
|
||||
[Captures]
|
||||
etag_before: header "ETag"
|
||||
thumb_before: bytes
|
||||
[Asserts]
|
||||
header "Cache-Control" contains "immutable"
|
||||
|
||||
|
||||
# Unchanged content revalidates to 304 — the caching path works.
|
||||
GET {{base_url}}/api/files/{{file_id}}/thumbnail/preview
|
||||
Authorization: Bearer {{token}}
|
||||
If-None-Match: {{etag_before}}
|
||||
|
||||
HTTP 304
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# Step 4 – Replace the content in place, keeping the same file id.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
PUT {{base_url}}/webdav/{{file_name}}
|
||||
Authorization: Bearer {{token}}
|
||||
Content-Type: image/png
|
||||
file,fixtures/green-image.png;
|
||||
|
||||
HTTP *
|
||||
[Asserts]
|
||||
status >= 200
|
||||
status < 300
|
||||
|
||||
|
||||
# Same file row, different content.
|
||||
GET {{base_url}}/api/files/{{file_id}}
|
||||
Authorization: Bearer {{token}}
|
||||
|
||||
HTTP 200
|
||||
[Captures]
|
||||
hash_after: jsonpath "$.content_hash"
|
||||
[Asserts]
|
||||
jsonpath "$.id" == "{{file_id}}"
|
||||
jsonpath "$.content_hash" != "{{hash_before}}"
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# Step 5 – The ETag must have changed with the content.
|
||||
#
|
||||
# This is the assertion the file exists for. With the id-keyed ETag it was
|
||||
# byte-identical to `etag_before`, and the next request would have been
|
||||
# answered 304 from cache — serving the OLD image indefinitely.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
GET {{base_url}}/api/files/{{file_id}}/thumbnail/preview
|
||||
Authorization: Bearer {{token}}
|
||||
|
||||
HTTP 200
|
||||
[Captures]
|
||||
etag_after: header "ETag"
|
||||
[Asserts]
|
||||
header "ETag" != "{{etag_before}}"
|
||||
|
||||
|
||||
# A client holding the stale ETag must be told to refetch, not given a 304.
|
||||
GET {{base_url}}/api/files/{{file_id}}/thumbnail/preview
|
||||
Authorization: Bearer {{token}}
|
||||
If-None-Match: {{etag_before}}
|
||||
|
||||
HTTP 200
|
||||
[Asserts]
|
||||
header "ETag" == "{{etag_after}}"
|
||||
|
||||
|
||||
# ...and the new ETag revalidates normally.
|
||||
GET {{base_url}}/api/files/{{file_id}}/thumbnail/preview
|
||||
Authorization: Bearer {{token}}
|
||||
If-None-Match: {{etag_after}}
|
||||
|
||||
HTTP 304
|
||||
|
||||
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
# Step 6 – Teardown. Hurl files share one database within run.sh.
|
||||
# ─────────────────────────────────────────────────────────────
|
||||
DELETE {{base_url}}/api/files/{{file_id}}
|
||||
Authorization: Bearer {{token}}
|
||||
|
||||
HTTP 204
|
||||
|
||||
|
||||
GET {{base_url}}/api/trash/resources
|
||||
Authorization: Bearer {{token}}
|
||||
|
||||
HTTP 200
|
||||
[Captures]
|
||||
trash_id: jsonpath "$.items[?(@.resource.id == '{{file_id}}')].resource.id"
|
||||
|
||||
|
||||
DELETE {{base_url}}/api/trash/{{trash_id}}
|
||||
Authorization: Bearer {{token}}
|
||||
|
||||
HTTP 200
|
||||
Reference in New Issue
Block a user