perf(zip): eliminate N+1 queries with ltree bulk subtree fetch

Replace BFS traversal that issued 2 SQL queries per folder (list_files +
list_folders) with 2 total queries using PostgreSQL ltree <@ operator:

1. list_subtree_folders: single GiST-indexed scan for all folders
2. list_files_in_subtree: single GiST-indexed join for all files

Files are grouped by folder_id in a HashMap, then iterated in directory
order (folders pre-sorted by path from SQL).

Changes across 4 architecture layers:
- Domain: FolderRepository::list_subtree_folders (default impl)
- Application ports: FolderUseCase, FileRetrievalUseCase, FileReadPort
- Application services: FolderService, FileRetrievalService passthroughs
- Infrastructure: PG implementations + ZipService rewrite

Query count: O(N) → O(1). Latency for 100-folder tree: ~200 round-trips → 3.
This commit is contained in:
Dionisio
2026-02-24 12:18:38 +01:00
parent a79700b11c
commit ed433df2af
9 changed files with 224 additions and 99 deletions
+10
View File
@@ -160,6 +160,16 @@ pub trait FileRetrievalUseCase: Send + Sync + 'static {
start: u64,
end: Option<u64>,
) -> Result<Box<dyn Stream<Item = Result<Bytes, std::io::Error>> + Send>, DomainError>;
/// Lists every file in the subtree rooted at `folder_id`.
///
/// Default: falls back to `list_files(Some(folder_id))` (one level).
async fn list_files_in_subtree(
&self,
folder_id: &str,
) -> Result<Vec<FileDto>, DomainError> {
self.list_files(Some(folder_id)).await
}
}
// ─────────────────────────────────────────────────────
+12
View File
@@ -71,6 +71,18 @@ pub trait FolderUseCase: Send + Sync + 'static {
user_id: &str,
name: String,
) -> Result<FolderDto, DomainError>;
/// Lists every folder in a subtree rooted at `folder_id` (inclusive),
/// ordered by path. Uses ltree `<@` — single GiST-indexed query.
///
/// Default: returns an empty vec (stubs / mocks).
async fn list_subtree_folders(
&self,
folder_id: &str,
) -> Result<Vec<FolderDto>, DomainError> {
let _ = folder_id;
Ok(Vec::new())
}
}
/**
+13
View File
@@ -78,6 +78,19 @@ pub trait FileReadPort: Send + Sync + 'static {
Ok(None)
}
/// Lists every file in the subtree rooted at `folder_id`.
///
/// Uses an ltree `<@` join against `storage.folders` so the entire
/// subtree is fetched in a single GiST-indexed query.
///
/// Default: falls back to `list_files(Some(folder_id))` (one level).
async fn list_files_in_subtree(
&self,
folder_id: &str,
) -> Result<Vec<File>, DomainError> {
self.list_files(Some(folder_id)).await
}
/// Search files with pagination and filtering at database level.
///
/// This is more efficient than loading all files and filtering in memory,
@@ -308,4 +308,12 @@ impl FileRetrievalUseCase for FileRetrievalService {
) -> Result<Box<dyn Stream<Item = Result<Bytes, std::io::Error>> + Send>, DomainError> {
self.file_read.get_file_range_stream(id, start, end).await
}
async fn list_files_in_subtree(
&self,
folder_id: &str,
) -> Result<Vec<FileDto>, DomainError> {
let files = self.file_read.list_files_in_subtree(folder_id).await?;
Ok(files.into_iter().map(FileDto::from).collect())
}
}
@@ -182,6 +182,14 @@ impl FolderUseCase for FolderService {
Ok(FolderDto::from(folder))
}
async fn list_subtree_folders(
&self,
folder_id: &str,
) -> Result<Vec<FolderDto>, DomainError> {
let folders = self.folder_storage.list_subtree_folders(folder_id).await?;
Ok(folders.into_iter().map(FolderDto::from).collect())
}
/// Gets a folder by its ID
async fn get_folder(&self, id: &str) -> Result<FolderDto, DomainError> {
let folder = self.folder_storage.get_folder(id).await.map_err(|e| {