Edouard Vanbelle
9a83f8c0d1
An e2e run emitted 81 × 429 in 763 log lines. The env already set
LOGIN/REGISTER/REFRESH to 36000/hour, and that changed nothing, because
those three are the only rate limiters with env vars — and they are the
wrong ones. They key on the client IP and guard the unauthenticated
front door. The limiters that fired key on the CALLER ID.
The log distinguishes them: all 81 landed on target `http::api`, never
`http::api::auth`, where login/register/refresh live.
The likely culprit is `user_profile_rate_limiter`, 60 lookups/min/caller,
guarding the visibility query behind GET /api/users/{id}. The whole
suite runs as a single `admin`, so every test shares one bucket; admin
views resolve an owner name per row and the run creates 34 users, so a
minute of tests clears 60 easily. Nothing failed, because the SPA
degrades to an unresolved name — which is exactly the problem, since
that noise would hide a real rate-limit regression.
Adds OXICLOUD_RATE_LIMIT_USER_PROFILE_MAX / _WINDOW_SECS and
OXICLOUD_RATE_LIMIT_DELTA_UPLOAD_MAX / _WINDOW_SECS, following the
existing three exactly. Defaults are the literals they replaced (60/60
and 240/60), so an operator who sets nothing sees no change; a unit test
pins that, because the failure is silent in both directions — too low
and real users get 429s on listings, too high and the `access_grants`
query loses the guard that stops an attacker exhausting it with random
UUIDs.
`tests/common/server.env` (shared by the e2e AND hurl suites) sets both
to a 1-hour budget, matching the posture already used for the other
three rather than a raised per-minute rate that would still burst-trip.
The docs now state the IP-vs-caller split, since that is what decides
which knob to reach for — and note that several actors sharing one
identity (CI, a bot, a kiosk) share one caller bucket.
Left alone: the four narrower env files (OIDC, webdav-drive-root) keep
their existing MAX=3600 with default windows. No evidence they trip the
per-caller limits, and adding config on speculation is how these files
drift.
Not fixed here: rate-limit rejections emit NO audit line, which is why
the attribution above reads "likely" rather than "confirmed" — nothing
in the log names the limiter. AGENTS.md requires one for every
rejection; that is a separate change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
213 lines
11 KiB
Bash
213 lines
11 KiB
Bash
# Shared test-server environment variables.
|
||
# Sourced by tests/api/run.sh (shell) and read by tests/e2e/playwright.config.ts (Node).
|
||
# Do NOT include OXICLOUD_SERVER_PORT or OXICLOUD_STORAGE_PATH here —
|
||
# each test suite sets those to avoid port/directory conflicts.
|
||
|
||
DATABASE_URL=postgres://oxicloud_test:oxicloud_test@localhost:5433/oxicloud_test
|
||
OXICLOUD_DB_CONNECTION_STRING=postgres://oxicloud_test:oxicloud_test@localhost:5433/oxicloud_test
|
||
# `OXICLOUD_STATIC_PATH` intentionally NOT set — every remaining
|
||
# test suite either serves the built SvelteKit SPA from
|
||
# `static-dist/` (coverage suite via `start-server-spa.sh`) or
|
||
# doesn't need static assets at all (Hurl API / WebDAV / CalDAV
|
||
# suites). The legacy `./static` vanilla frontend was retired.
|
||
OXICLOUD_JWT_SECRET=test-secret-do-not-use-in-prod-minimum-32-chars
|
||
OXICLOUD_ENABLE_AUTH=true
|
||
OXICLOUD_ENABLE_TRASH=true
|
||
OXICLOUD_ENABLE_SEARCH=true
|
||
OXICLOUD_ENABLE_FILE_SHARING=true
|
||
OXICLOUD_ENABLE_MUSIC=true
|
||
# External file mounts — enabled so external_mounts.hurl can exercise the
|
||
# admin CRUD endpoints against a live mount registry.
|
||
OXICLOUD_ENABLE_EXTERNAL_MOUNTS=true
|
||
OXICLOUD_EXPOSE_SYSTEM_USERS=true
|
||
OXICLOUD_WOPI_ENABLED=true
|
||
# Fixed secret so the Hurl WOPI test can hand-craft valid access
|
||
# tokens with a known signing key. Prod deployments MUST override
|
||
# this to a random per-deployment value.
|
||
OXICLOUD_WOPI_SECRET=test-wopi-secret-do-not-use-in-prod-do-not-use-in-prod
|
||
# Discovery URL points at a black hole — VERB endpoints don't need
|
||
# discovery, and the WOPI Hurl suite deliberately does NOT touch
|
||
# `/api/wopi/editor-url` (the only path that would fetch it), so
|
||
# an unreachable URL keeps startup fast and hermetic.
|
||
OXICLOUD_WOPI_DISCOVERY_URL=http://127.0.0.1:9100/discovery.xml
|
||
OXICLOUD_WOPI_TOKEN_TTL_SECS=3600
|
||
OXICLOUD_OIDC_ENABLED=false
|
||
|
||
OXICLOUD_NEXTCLOUD_ENABLED=true
|
||
|
||
# Multi-entry storage config — see docs/plan/storage-multi-entry.md.
|
||
# `local_main` is FIRST so the boot fallback picks it when no active
|
||
# pointer is set in the DB yet (fresh test DB). Its root_dir falls
|
||
# back to $OXICLOUD_STORAGE_PATH (set in each test suite's run.sh).
|
||
# `s3_stub` is declared but never activated — it lets storage_config.hurl
|
||
# assert the entries table has more than one row without needing a
|
||
# real S3 backend.
|
||
OXICLOUD_STORAGE_ENTRIES=local_main,s3_stub,azurite
|
||
OXICLOUD_STORAGE_local_main_BACKEND=local
|
||
OXICLOUD_STORAGE_s3_stub_BACKEND=s3
|
||
OXICLOUD_STORAGE_s3_stub_S3_BUCKET=oxicloud-test-stub
|
||
OXICLOUD_STORAGE_s3_stub_S3_REGION=us-east-1
|
||
OXICLOUD_STORAGE_s3_stub_S3_ENDPOINT_URL=http://127.0.0.1:9999
|
||
OXICLOUD_STORAGE_s3_stub_S3_ACCESS_KEY=stub
|
||
OXICLOUD_STORAGE_s3_stub_S3_SECRET_KEY=stub
|
||
|
||
# `azurite` — a REAL, reachable Azure backend, unlike `s3_stub` above.
|
||
# It points at the Azurite emulator started by spawn-db.sh, which speaks
|
||
# the actual Blob REST API, so this is the only way to exercise the Azure
|
||
# path without an account.
|
||
#
|
||
# Declared but NOT activated: the active backend stays local_main, so the
|
||
# rest of the suite is untouched. Tests reach it explicitly through
|
||
# `?storage=azurite` on the jobs that accept an entry name — which is
|
||
# exactly what that parameter was added for.
|
||
#
|
||
# The credentials are Azurite's fixed, publicly-documented development
|
||
# pair. They are in every Microsoft quickstart; nothing here is secret.
|
||
OXICLOUD_STORAGE_azurite_BACKEND=azure
|
||
OXICLOUD_STORAGE_azurite_AZURE_ACCOUNT_NAME=devstoreaccount1
|
||
OXICLOUD_STORAGE_azurite_AZURE_ACCOUNT_KEY=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==
|
||
OXICLOUD_STORAGE_azurite_AZURE_CONTAINER=oxicloud-test
|
||
OXICLOUD_STORAGE_azurite_AZURE_ENDPOINT_URL=http://127.0.0.1:10000/devstoreaccount1
|
||
|
||
RUST_LOG="warn,audit=info,sqlx::migrate=info"
|
||
#RUST_LOG="warn,audit=info,oxicloud::quota=debug"
|
||
#RUST_LOG=debug
|
||
#RUST_LOG=info
|
||
|
||
# Per-chunk upload cap, exercised by chunked_upload_cap.hurl.
|
||
# 4 MiB: lets the existing grants.hurl single-chunk test (2.76 MB) pass
|
||
# under the cap, while the cap test sends a 5 MiB fixture to trigger 413.
|
||
OXICLOUD_CHUNK_MAX_BYTES=4194304
|
||
|
||
# Direct-PUT (non-chunked) cap, exercised by chunked_upload_cap.hurl.
|
||
# 4 MiB: same threshold as the chunked cap so the existing 5 MiB
|
||
# fixture (chunk-over-cap-5mb.bin) can prove BOTH caps with one
|
||
# generated file. All existing direct-PUT tests
|
||
# (test_dedup_webdav_multichunk.sh = 2.76 MB, _ref_count = ~66 KB,
|
||
# _nextcloud_put_blake3 = 32 B) stay safely under this cap.
|
||
OXICLOUD_DIRECT_PUT_MAX_BYTES=4194304
|
||
|
||
# Rate-limit ceiling for tests. `_MAX` alone isn't enough because
|
||
# the default `_WINDOW_SECS = 60` gives an effective rate of
|
||
# `MAX / 60` req/sec — a bursty Playwright suite (many workers,
|
||
# parallel logins + auto-refresh churn under DPoP required mode)
|
||
# overflows even at MAX=3600 (60 req/sec shared across the whole
|
||
# runner IP). Widening the window to 1 h means the same MAX is a
|
||
# 1-hour budget, well above what any single CI run consumes
|
||
# (< 5 min end-to-end). Applied to all three buckets so login /
|
||
# refresh / register all share the same generous test posture.
|
||
OXICLOUD_RATE_LIMIT_REFRESH_MAX=36000
|
||
OXICLOUD_RATE_LIMIT_REFRESH_WINDOW_SECS=3600
|
||
OXICLOUD_RATE_LIMIT_LOGIN_MAX=36000
|
||
OXICLOUD_RATE_LIMIT_LOGIN_WINDOW_SECS=3600
|
||
OXICLOUD_RATE_LIMIT_REGISTER_MAX=36000
|
||
OXICLOUD_RATE_LIMIT_REGISTER_WINDOW_SECS=3600
|
||
|
||
# The three above are keyed on the client IP. These two are keyed on the
|
||
# CALLER ID, which is why raising the three did nothing: the whole suite
|
||
# runs as a single `admin`, so every test shares one bucket.
|
||
#
|
||
# A 763-line e2e server log showed 81 × 429 — all on `http::api`, never
|
||
# `http::api::auth` — against the 60/min default for user-profile
|
||
# lookups. Admin views resolve an owner name per row, and the run
|
||
# creates 34 users, so a minute of tests clears 60 easily. Nothing
|
||
# failed, because the SPA degrades to an unresolved name, which is
|
||
# precisely why it went unnoticed: the noise would hide a real
|
||
# rate-limit regression.
|
||
#
|
||
# Same posture as above — a 1-hour window with a budget far beyond what
|
||
# one run consumes, rather than a raised per-minute rate that would
|
||
# still burst-trip.
|
||
OXICLOUD_RATE_LIMIT_USER_PROFILE_MAX=36000
|
||
OXICLOUD_RATE_LIMIT_USER_PROFILE_WINDOW_SECS=3600
|
||
OXICLOUD_RATE_LIMIT_DELTA_UPLOAD_MAX=36000
|
||
OXICLOUD_RATE_LIMIT_DELTA_UPLOAD_WINDOW_SECS=3600
|
||
|
||
# Magic-link / external-users flow (PR 9). The mock SMTP captures every
|
||
# outbound message in-process so external_users.hurl can retrieve the
|
||
# invitation body and follow the magic-link URL. The `SMTP_FROM` value
|
||
# is required so the mock can build a valid Message; host/port are
|
||
# irrelevant in mock mode but kept set for completeness.
|
||
OXICLOUD_SMTP_MOCK=true
|
||
OXICLOUD_SMTP_HOST=localhost
|
||
OXICLOUD_SMTP_PORT=25
|
||
OXICLOUD_SMTP_FROM='OxiCloud Tests <test@oxicloud.local>'
|
||
OXICLOUD_SMTP_TLS=none
|
||
OXICLOUD_ALLOW_EXTERNAL_USERS=true
|
||
|
||
# Public-registration email-domain allowlist. Exercised by
|
||
# `registration.hurl` step "off-domain rejection" (attempts to
|
||
# register with @nowhere.invalid and asserts 403
|
||
# `RegistrationDomainNotAllowed`). Contains BOTH `example.com` (Hurl
|
||
# fixtures use it — charlie@example.com etc.) AND `example.test` (E2E
|
||
# login.spec uses it — reg-*@example.test). Every legitimate test
|
||
# path stays inside the allowlist; the rejection test picks a domain
|
||
# outside it deliberately.
|
||
OXICLOUD_REGISTRATION_ALLOWED_EMAIL_DOMAINS=example.com,example.test
|
||
|
||
# Auth-policy vector. Enables the "magic-link login is allowed for
|
||
# accounts that also have a password" branch — required by
|
||
# auth_magic_link_login.hurl (alice has a password AND requests a
|
||
# magic-link login). Inert for every other test: `has_password`
|
||
# refusal only fires when the endpoint is called, and no other file
|
||
# calls `magic-link/send` for a password-holding account.
|
||
OXICLOUD_AUTH_POLICIES=permit_magic_link_for_password_users
|
||
OXICLOUD_AUTH_METHODS=password,magic_link
|
||
# Explicit pin — `--config` now overrides shell env (main.rs uses
|
||
# `from_filename_override`), but pinning here documents the intended
|
||
# test-env state. Flip to true for the deferred `tests/verify_email/`
|
||
# suite; leaving false here keeps every other suite on the "verified
|
||
# email not required" path (charlie's classic register+login etc.).
|
||
OXICLOUD_REQUIRE_VERIFIED_EMAIL=false
|
||
|
||
# PR 12 — magic-link rate-limit caps lowered so external_users.hurl can
|
||
# exercise the cap behaviour with a small, deterministic request count.
|
||
# Production defaults are 50 / 5 / 200 respectively (see example.env).
|
||
OXICLOUD_MAGIC_LINK_INVITE_PER_CALLER_PER_HOUR=3
|
||
OXICLOUD_MAGIC_LINK_SEND_PER_EMAIL_PER_HOUR=2
|
||
OXICLOUD_MAGIC_LINK_SEND_PER_IP_PER_HOUR=50
|
||
|
||
# permits IP spoofing for tests
|
||
OXICLOUD_TRUST_PROXY_CIDR=0.0.0.0/0
|
||
|
||
# ── OPAQUE aPAKE (Phase 0 substrate) ────────────────────────────────
|
||
# Boot the OPAQUE service in Migrate mode so every Hurl run exercises:
|
||
# 1. OpaqueConfig::from_env parsing all five OPAQUE env vars.
|
||
# 2. effective_mode(&auth) permitting Migrate because password IS in
|
||
# OXICLOUD_AUTH_METHODS above (would auto-downgrade to Off if we
|
||
# had disabled it — that path is unit-tested in
|
||
# common::config::tests::effective_mode_downgrades_to_off_...).
|
||
# 3. OpaqueService::from_config accepting a valid serialised setup.
|
||
# 4. AppState wiring populating `opaque_service = Some(_)`.
|
||
#
|
||
# Phase 0 has no wire-facing endpoints, so this only proves the
|
||
# substrate loads cleanly; opaque_substrate.hurl asserts the legacy
|
||
# /api/auth/login path remains intact and the future OPAQUE endpoints
|
||
# still 404 (they flip to 200 when Phase 1 lands).
|
||
#
|
||
# The SERVER_SETUP below is a throwaway keypair generated once for the
|
||
# test env — real deployments call `oxicloud opaque setup` and paste
|
||
# the output. Never reuse this value outside CI. Regenerate any time
|
||
# with:
|
||
# cargo run --bin oxicloud -- opaque setup
|
||
OXICLOUD_AUTH_OPAQUE_MODE=migrate
|
||
OXICLOUD_AUTH_OPAQUE_SERVER_SETUP="ZY4hAGa1MNyE7Ht+8ksLcyMmi/K2iJvxQly+DdfllUxjiH0+CjCt4hG6+9Y68jGet2L213dV0hajCbr4fXnekkWtUxqLr+butVHEksZ9NJRuZTvS6SMC73yf/yku4WUHT1NSRB2yHurAFmYn75D9wdA1VaXTuwgO/u5i1pvcsQs="
|
||
# Fast Argon2id — CI machines are underpowered vs production (256 MiB
|
||
# default would drag every test-scaffold future OPAQUE handshake in
|
||
# Phase 1+). Matches the params used in the round-trip unit test.
|
||
OXICLOUD_AUTH_OPAQUE_KSF_MEMORY_KIB=8
|
||
OXICLOUD_AUTH_OPAQUE_KSF_ITERATIONS=1
|
||
OXICLOUD_AUTH_OPAQUE_KSF_PARALLELISM=1
|
||
|
||
# DPoP (RFC 9449) — required mode surfaces every verifier /
|
||
# nonce / replay bug as a hard 401 (opportunistic would swallow
|
||
# some cases as warnings). Safe today because unbound sessions —
|
||
# `session.dpop_jkt IS NULL`, which is where the test admin's
|
||
# legacy-login lands — are still exempted at the middleware; the
|
||
# session-level enforcement flip is Gate 9. Once that arrives,
|
||
# every bind-time downgrade path also gets caught here without
|
||
# a config change. `dpop-hurl-helper` needs this on or the
|
||
# middleware is a pass-through and its scenarios all silently 200.
|
||
OXICLOUD_DPOP_MODE=required
|
||
|