perf(thumbnails): SIMD resize via fast_image_resize (PNG 2.6x, RAM 2.5x lower)
Replace the image crate's scalar resampler with fast_image_resize (AVX2/SSE4.1/NEON) in a shared encode_thumbnail() helper. render_all now converts to RGB8 once and SIMD-resizes the shared buffer per size. Lanczos3 for downscaling, CatmullRom when upscaling (Lanczos rings on enlargement). Also folds the duplicated path-variant generate_all_sizes_background into the shared render path -- it had missed BOTH shrink-on-load and SIMD resizing -- so every thumbnail path now goes through one optimised routine (no duplication). Measured on 14 cores vs the post-1.5 state (benches/BASELINE.md): - PNG 2.60x faster (33.6->12.9ms), GIF/WebP 1.25-1.6x: full-resolution decode paths where the resize dominates, so SIMD helps most - JPEG only ~7% (shrink-on-load already shrank the bitmap) but peak heap fell another ~2.5x (17.6->7.1MB): tight RGB buffers, RGB conversion once - quality SSIM 0.986-0.994 at identical dims (>=0.98 gate) Thumbnails are now exactly max_dim on the long side (e.g. 400x266) vs the old fit-within 399x266 -- a <=1px change, invisible under object-fit: cover. Bench example gains an exact-dims quality reference + semaphore throughput table. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
DioCrafts
Generated
+28
@@ -2012,6 +2012,15 @@ dependencies = [
|
||||
"syn",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "document-features"
|
||||
version = "0.2.12"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d4b8a88685455ed29a21542a33abd9cb6510b6b129abadabdcef0f4c55bc8f61"
|
||||
dependencies = [
|
||||
"litrs",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "dotenvy"
|
||||
version = "0.15.7"
|
||||
@@ -2297,6 +2306,18 @@ dependencies = [
|
||||
"serde_json",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "fast_image_resize"
|
||||
version = "5.5.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "fbc7fe45cf92b43817ff62a3723e862b85bd1d06288f63007f7645d1d2f7a060"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"document-features",
|
||||
"num-traits",
|
||||
"thiserror 2.0.18",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "fastcdc"
|
||||
version = "4.0.1"
|
||||
@@ -3626,6 +3647,12 @@ version = "0.8.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "92daf443525c4cce67b150400bc2316076100ce0b3686209eb8cf3c31612e6f0"
|
||||
|
||||
[[package]]
|
||||
name = "litrs"
|
||||
version = "1.0.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "11d3d7f243d5c5a8b9bb5d6dd2b1602c0cb0b9db1621bafc7ed66e35ff9fe092"
|
||||
|
||||
[[package]]
|
||||
name = "lock_api"
|
||||
version = "0.4.14"
|
||||
@@ -4204,6 +4231,7 @@ dependencies = [
|
||||
"dashmap",
|
||||
"dotenvy",
|
||||
"extism",
|
||||
"fast_image_resize",
|
||||
"fastcdc",
|
||||
"file-rotate",
|
||||
"flate2",
|
||||
|
||||
@@ -43,6 +43,9 @@ image = { version = "0.25.10", default-features = false, features = ["jpeg", "pn
|
||||
# Rust, no C toolchain. The `image` crate's zune-jpeg backend can't scale during
|
||||
# decode; this can, cutting decode time/RAM ~order-of-magnitude on large photos.
|
||||
jpeg-decoder = "0.3"
|
||||
# SIMD (AVX2/SSE4.1/NEON) image resizing for thumbnails — far faster than the
|
||||
# `image` crate's scalar resampler, and it speeds every format (incl. PNG/WebP).
|
||||
fast_image_resize = "5"
|
||||
id3 = "1.17"
|
||||
mp3-duration = "0.1"
|
||||
kamadak-exif = "0.6.1"
|
||||
|
||||
@@ -175,3 +175,55 @@ Measured with a harness that mirrors the **real service path** (Table D:
|
||||
already partly fill all cores — the remaining headroom is **Task 1.7**
|
||||
(rayon oversubscription).
|
||||
|
||||
---
|
||||
|
||||
# Phase 1.2 results — SIMD resize (`fast_image_resize`, Lanczos3)
|
||||
|
||||
Replaced the `image` crate's scalar resampler with `fast_image_resize`
|
||||
(AVX2/SSE4.1/NEON) in the shared `encode_thumbnail` helper; `render_all` now
|
||||
converts to RGB8 once and SIMD-resizes the shared buffer per size. Lanczos3 for
|
||||
downscaling, CatmullRom when upscaling (avoids Lanczos ringing). Also folded the
|
||||
duplicated path-variant `generate_all_sizes_background` into the shared render
|
||||
path, so it too gets shrink-on-load + SIMD. "before" = post-1.5 state.
|
||||
|
||||
### Single-thread latency `render_all` (ms) and peak heap (MB)
|
||||
|
||||
| case | ms before | ms after | speedup | heap before | heap after |
|
||||
|-----------|----------:|---------:|--------:|------------:|-----------:|
|
||||
| jpeg_12mp | 60.89 | 56.46 | 1.08× | 17.6 | **7.1** |
|
||||
| jpeg_24mp | 113.98 | 106.76 | 1.07× | 24.9 | **10.1** |
|
||||
| jpeg_48mp | 203.29 | 198.75 | 1.02× | 17.6 | **7.1** |
|
||||
| **png_large** | 33.63 | **12.92** | **2.60×** | 58.4 | **27.1** |
|
||||
| gif_large | 12.93 | 7.99 | 1.62× | 15.4 | 5.9 |
|
||||
| webp_large| 21.15 | 16.89 | 1.25× | 20.7 | 8.3 |
|
||||
| small_300 | 10.64 | 6.37 | 1.67× | 8.0 | 3.9 |
|
||||
|
||||
- **JPEG: only ~6–8 %** — shrink-on-load already shrank the decoded bitmap, so
|
||||
the resize was a small slice of the time. But **peak heap fell another ~2.5×**
|
||||
(7 MB): fir works on tight RGB buffers with no intermediate `DynamicImage`,
|
||||
and RGB conversion now happens once instead of per size.
|
||||
- **PNG: 2.6×** (and GIF/WebP 1.25–1.6×) — exactly as predicted: these decode at
|
||||
full resolution (no DCT shrink), so the SIMD resize dominates the win.
|
||||
|
||||
### Throughput (≈ +10–15 %, run-to-run noisy)
|
||||
|
||||
Saturated 12 MP ≈ 142→157 photos/s; semaphore-bounded 12 MP @ 14 permits
|
||||
≈ 134→148. Directionally up; treat as noise-bounded.
|
||||
|
||||
### Quality gate (vs full-decode CatmullRom at identical dims)
|
||||
|
||||
| case | SSIM | PSNR dB |
|
||||
|-----------|-------:|--------:|
|
||||
| jpeg_12mp | 0.9865 | 47.21 |
|
||||
| jpeg_24mp | 0.9923 | 48.73 |
|
||||
| jpeg_48mp | 0.9938 | 49.33 |
|
||||
| small_300 | 0.9921 | 42.59 |
|
||||
|
||||
All **≥ 0.98**. (The upscale case `small_300` needed the Lanczos3→CatmullRom
|
||||
upscale rule — Lanczos rings when enlarging; it was 0.954 before that fix.)
|
||||
|
||||
### Note
|
||||
Output thumbnails are now exactly `max_dim` on the long side (e.g. 400×266),
|
||||
vs the old `image::resize` fit-within which produced 399×266 — a ≤1 px change,
|
||||
invisible under the frontend's `object-fit: cover`.
|
||||
|
||||
|
||||
@@ -453,20 +453,29 @@ fn measure_semaphore_throughput(case: &CorpusCase, permits: usize, window: Durat
|
||||
// Quality verification helpers (Table C)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Reference thumbnail: identical resample + q80 JPEG encode as production, but
|
||||
/// forced through a **full decode** (no shrink-on-load). Comparing against this
|
||||
/// isolates exactly the quality impact of DCT scale-on-decode. EXIF orientation
|
||||
/// is not applied here, so only run it on orientation=1 corpus cases.
|
||||
/// Reference thumbnail: a **full-resolution decode** + high-quality CatmullRom
|
||||
/// resample + q80 JPEG encode — the original (pre-optimisation) quality target.
|
||||
/// Comparing the optimised output against this gauges whether shrink-on-load +
|
||||
/// SIMD resizing degrades quality. Uses the same exact target dims as production
|
||||
/// (`resize_exact`) so the comparison is apples-to-apples, never a dim mismatch.
|
||||
/// EXIF orientation is not applied, so only run it on orientation=1 cases.
|
||||
fn reference_render_full_decode(bytes: &[u8], max_dim: u32) -> Vec<u8> {
|
||||
let img = image::load_from_memory(bytes).expect("ref full decode");
|
||||
let (ow, oh) = (img.width(), img.height());
|
||||
// Same fit-to-longest-side dims production computes (see fit_dims()).
|
||||
let (nw, nh) = if ow > oh {
|
||||
(max_dim, (oh as f32 * (max_dim as f32 / ow as f32)) as u32)
|
||||
(
|
||||
max_dim,
|
||||
((oh as f32 * (max_dim as f32 / ow as f32)) as u32).max(1),
|
||||
)
|
||||
} else {
|
||||
((ow as f32 * (max_dim as f32 / oh as f32)) as u32, max_dim)
|
||||
(
|
||||
((ow as f32 * (max_dim as f32 / oh as f32)) as u32).max(1),
|
||||
max_dim,
|
||||
)
|
||||
};
|
||||
let rgb = img
|
||||
.resize(nw, nh, image::imageops::FilterType::CatmullRom)
|
||||
.resize_exact(nw, nh, image::imageops::FilterType::CatmullRom)
|
||||
.to_rgb8();
|
||||
let mut buf = Vec::new();
|
||||
let enc = image::codecs::jpeg::JpegEncoder::new_with_quality(&mut buf, 80);
|
||||
|
||||
@@ -665,74 +665,93 @@ impl ThumbnailService {
|
||||
apply_orientation(img, orientation)
|
||||
}
|
||||
|
||||
/// Aspect-ratio-preserving target dimensions so the longest side equals
|
||||
/// `max_dim` (clamped to ≥1 to keep the SIMD resizer happy on extreme ratios).
|
||||
fn fit_dims(src_w: u32, src_h: u32, max_dim: u32) -> (u32, u32) {
|
||||
if src_w > src_h {
|
||||
let ratio = max_dim as f32 / src_w as f32;
|
||||
(max_dim, ((src_h as f32 * ratio) as u32).max(1))
|
||||
} else {
|
||||
let ratio = max_dim as f32 / src_h as f32;
|
||||
(((src_w as f32 * ratio) as u32).max(1), max_dim)
|
||||
}
|
||||
}
|
||||
|
||||
/// Resampling filter per size: Bilinear for the tiny icon (cheap, output is
|
||||
/// 150 px), Lanczos3 for preview/large (highest quality, SIMD-fast here).
|
||||
fn filter_for(size: ThumbnailSize) -> fast_image_resize::FilterType {
|
||||
use fast_image_resize::FilterType;
|
||||
match size {
|
||||
ThumbnailSize::Icon => FilterType::Bilinear,
|
||||
ThumbnailSize::Preview | ThumbnailSize::Large => FilterType::Lanczos3,
|
||||
}
|
||||
}
|
||||
|
||||
/// SIMD-resize an RGB8 source to `dst_w×dst_h` (via `fast_image_resize`,
|
||||
/// AVX2/SSE4.1/NEON) and encode the result as a q80 JPEG.
|
||||
fn encode_thumbnail(
|
||||
src_rgb: &[u8],
|
||||
src_w: u32,
|
||||
src_h: u32,
|
||||
dst_w: u32,
|
||||
dst_h: u32,
|
||||
filter: fast_image_resize::FilterType,
|
||||
) -> Result<Vec<u8>, ThumbnailError> {
|
||||
use fast_image_resize::images::{Image, ImageRef};
|
||||
use fast_image_resize::{FilterType, PixelType, ResizeAlg, ResizeOptions, Resizer};
|
||||
|
||||
// Lanczos3 is ideal for downscaling but rings on edges when upscaling;
|
||||
// fall back to a smooth bicubic (CatmullRom) whenever the target is
|
||||
// larger than the source on either axis.
|
||||
let filter = if dst_w > src_w || dst_h > src_h {
|
||||
FilterType::CatmullRom
|
||||
} else {
|
||||
filter
|
||||
};
|
||||
|
||||
let src = ImageRef::new(src_w, src_h, src_rgb, PixelType::U8x3)
|
||||
.map_err(|e| ThumbnailError::ImageError(e.to_string()))?;
|
||||
let mut dst = Image::new(dst_w, dst_h, PixelType::U8x3);
|
||||
let opts = ResizeOptions::new().resize_alg(ResizeAlg::Convolution(filter));
|
||||
Resizer::new()
|
||||
.resize(&src, &mut dst, &opts)
|
||||
.map_err(|e| ThumbnailError::ImageError(e.to_string()))?;
|
||||
|
||||
let rgb = image::RgbImage::from_raw(dst_w, dst_h, dst.into_vec())
|
||||
.ok_or_else(|| ThumbnailError::ImageError("resize buffer size mismatch".into()))?;
|
||||
let mut buffer = Vec::new();
|
||||
let encoder = JpegEncoder::new_with_quality(&mut buffer, 80);
|
||||
rgb.write_with_encoder(encoder)
|
||||
.map_err(|e| ThumbnailError::ImageError(e.to_string()))?;
|
||||
Ok(buffer)
|
||||
}
|
||||
|
||||
fn render_thumbnail_from_data(
|
||||
data: &[u8],
|
||||
size: ThumbnailSize,
|
||||
) -> Result<Vec<u8>, ThumbnailError> {
|
||||
let max_dim = size.max_dimension();
|
||||
let img = Self::decode_oriented(data, max_dim)?;
|
||||
|
||||
let (orig_width, orig_height) = (img.width(), img.height());
|
||||
let (new_width, new_height) = if orig_width > orig_height {
|
||||
let ratio = max_dim as f32 / orig_width as f32;
|
||||
(max_dim, (orig_height as f32 * ratio) as u32)
|
||||
} else {
|
||||
let ratio = max_dim as f32 / orig_height as f32;
|
||||
((orig_width as f32 * ratio) as u32, max_dim)
|
||||
};
|
||||
|
||||
let filter = match size {
|
||||
ThumbnailSize::Icon => FilterType::Triangle,
|
||||
ThumbnailSize::Preview => FilterType::CatmullRom,
|
||||
ThumbnailSize::Large => FilterType::CatmullRom,
|
||||
};
|
||||
let thumbnail = img.resize(new_width, new_height, filter);
|
||||
|
||||
let rgb = thumbnail.to_rgb8();
|
||||
let mut buffer = Vec::new();
|
||||
let encoder = JpegEncoder::new_with_quality(&mut buffer, 80);
|
||||
rgb.write_with_encoder(encoder)
|
||||
.map_err(|e| ThumbnailError::ImageError(e.to_string()))?;
|
||||
|
||||
Ok(buffer)
|
||||
let rgb = Self::decode_oriented(data, max_dim)?.into_rgb8();
|
||||
let (sw, sh) = (rgb.width(), rgb.height());
|
||||
let (nw, nh) = Self::fit_dims(sw, sh, max_dim);
|
||||
Self::encode_thumbnail(rgb.as_raw(), sw, sh, nw, nh, Self::filter_for(size))
|
||||
}
|
||||
|
||||
fn render_all_thumbnails_from_data(
|
||||
data: &[u8],
|
||||
) -> Result<Vec<(ThumbnailSize, Bytes)>, ThumbnailError> {
|
||||
// Decode once, shrunk-on-load for the largest size (800 px); all three
|
||||
// sizes are then resampled from this single shared bitmap. Sizing the
|
||||
// shared decode to Large keeps quality for every size while paying the
|
||||
// (now much smaller) decode cost only once.
|
||||
let img = Self::decode_oriented(data, ThumbnailSize::Large.max_dimension())?;
|
||||
let (orig_w, orig_h) = (img.width(), img.height());
|
||||
// Decode once, shrunk-on-load for the largest size (800 px), and convert
|
||||
// to RGB8 once; all three sizes are then SIMD-resampled from this single
|
||||
// shared bitmap (the RGB conversion is no longer repeated per size).
|
||||
let rgb = Self::decode_oriented(data, ThumbnailSize::Large.max_dimension())?.into_rgb8();
|
||||
let (sw, sh) = (rgb.width(), rgb.height());
|
||||
let src = rgb.as_raw().as_slice();
|
||||
|
||||
ThumbnailSize::all()
|
||||
.par_iter()
|
||||
.map(|&size| {
|
||||
let max_dim = size.max_dimension();
|
||||
|
||||
let (new_w, new_h) = if orig_w > orig_h {
|
||||
let ratio = max_dim as f32 / orig_w as f32;
|
||||
(max_dim, (orig_h as f32 * ratio) as u32)
|
||||
} else {
|
||||
let ratio = max_dim as f32 / orig_h as f32;
|
||||
((orig_w as f32 * ratio) as u32, max_dim)
|
||||
};
|
||||
|
||||
let filter = match size {
|
||||
ThumbnailSize::Icon => FilterType::Triangle,
|
||||
ThumbnailSize::Preview => FilterType::CatmullRom,
|
||||
ThumbnailSize::Large => FilterType::CatmullRom,
|
||||
};
|
||||
let thumb = img.resize(new_w, new_h, filter);
|
||||
|
||||
let rgb = thumb.to_rgb8();
|
||||
let mut buf = Vec::new();
|
||||
let encoder = JpegEncoder::new_with_quality(&mut buf, 80);
|
||||
rgb.write_with_encoder(encoder)
|
||||
.map_err(|e| ThumbnailError::ImageError(e.to_string()))?;
|
||||
|
||||
let (nw, nh) = Self::fit_dims(sw, sh, size.max_dimension());
|
||||
let buf = Self::encode_thumbnail(src, sw, sh, nw, nh, Self::filter_for(size))?;
|
||||
Ok((size, Bytes::from(buf)))
|
||||
})
|
||||
.collect::<Result<Vec<_>, ThumbnailError>>()
|
||||
@@ -875,73 +894,14 @@ impl ThumbnailService {
|
||||
|
||||
let path = original_path.clone();
|
||||
|
||||
// Single spawn_blocking: 1 read + 1 decode + 3 resize + 3 encode
|
||||
// Single spawn_blocking: read the file once, then run the shared
|
||||
// render path (shrink-on-load decode + SIMD resize) — identical to
|
||||
// the blob variant, so this path gets both optimisations and there
|
||||
// is no duplicated decode/resize logic.
|
||||
let results = tokio::task::spawn_blocking(move || {
|
||||
// Single read: load file once into memory
|
||||
let data =
|
||||
std::fs::read(&path).map_err(|e| ThumbnailError::ImageError(e.to_string()))?;
|
||||
|
||||
// Safety check: read dimensions from in-memory buffer (no 2nd I/O)
|
||||
let (w, h) = image::ImageReader::new(std::io::Cursor::new(&data))
|
||||
.with_guessed_format()
|
||||
.map_err(|e| ThumbnailError::ImageError(e.to_string()))?
|
||||
.into_dimensions()
|
||||
.map_err(|e| ThumbnailError::ImageError(e.to_string()))?;
|
||||
if (w as u64) * (h as u64) > MAX_DECODE_PIXELS {
|
||||
return Err(ThumbnailError::ImageError(format!(
|
||||
"Image too large for thumbnail: {w}×{h} ({} MP, max {MAX_DECODE_PIXELS})",
|
||||
w as u64 * h as u64 / 1_000_000
|
||||
)));
|
||||
}
|
||||
|
||||
// Full decode from the same in-memory buffer (no 2nd disk read)
|
||||
let img = image::load_from_memory(&data)
|
||||
.map_err(|e| ThumbnailError::ImageError(e.to_string()))?;
|
||||
|
||||
// Apply EXIF orientation so thumbnails display correctly
|
||||
let img = {
|
||||
use crate::infrastructure::services::exif_service::{
|
||||
ExifService, apply_orientation,
|
||||
};
|
||||
let orientation = ExifService::extract(&data)
|
||||
.and_then(|m| m.orientation)
|
||||
.unwrap_or(1);
|
||||
// Free the encoded image data now that image is decoded and EXIF extracted
|
||||
drop(data);
|
||||
apply_orientation(img, orientation)
|
||||
};
|
||||
|
||||
let (orig_w, orig_h) = (img.width(), img.height());
|
||||
|
||||
ThumbnailSize::all()
|
||||
.par_iter()
|
||||
.map(|&size| {
|
||||
let max_dim = size.max_dimension();
|
||||
|
||||
let (new_w, new_h) = if orig_w > orig_h {
|
||||
let ratio = max_dim as f32 / orig_w as f32;
|
||||
(max_dim, (orig_h as f32 * ratio) as u32)
|
||||
} else {
|
||||
let ratio = max_dim as f32 / orig_h as f32;
|
||||
((orig_w as f32 * ratio) as u32, max_dim)
|
||||
};
|
||||
|
||||
let filter = match size {
|
||||
ThumbnailSize::Icon => FilterType::Triangle,
|
||||
ThumbnailSize::Preview => FilterType::CatmullRom,
|
||||
ThumbnailSize::Large => FilterType::CatmullRom,
|
||||
};
|
||||
let thumb = img.resize(new_w, new_h, filter);
|
||||
|
||||
let rgb = thumb.to_rgb8();
|
||||
let mut buf = Vec::new();
|
||||
let encoder = JpegEncoder::new_with_quality(&mut buf, 80);
|
||||
rgb.write_with_encoder(encoder)
|
||||
.map_err(|e| ThumbnailError::ImageError(e.to_string()))?;
|
||||
|
||||
Ok((size, Bytes::from(buf)))
|
||||
})
|
||||
.collect::<Result<Vec<_>, ThumbnailError>>()
|
||||
Self::render_all_thumbnails_from_data(&data)
|
||||
})
|
||||
.await;
|
||||
|
||||
|
||||
Reference in New Issue
Block a user