From 10MB to 500MB: How Direct-to-Bucket Uploads Scaled With Us
How we moved Cereby's file ingestion off the Next.js request path and onto a direct browser-to-bucket flow, while keeping the server as the source of truth for validation, quota, and lifecycle.
Update: 2026-04-26. When this post first shipped, the ceiling was 50MB. That is what Supabase's free tier allowed per object, and the architecture below is what unlocked it. We have since moved to Supabase Pro, which raises the per-object limit by orders of magnitude, and we used that headroom to lift the in-app cap to 500MB. The architecture did not change. The design we shipped at 50MB is exactly what made the 500MB jump a one-line edit. The numbers and tables below reflect the current state; the original 10MB to 50MB framing is preserved because the shape of the problem has not changed.
The wall every file hits
Cereby is a study workspace. Students drop in lecture slides, textbook PDFs, recorded audio, and screen-recorded videos so the app can parse them, chunk them, and feed them into chat, quizzes, and focus mode. The whole product assumes you can get your materials in. The moment a 14MB slide deck or a 30-minute MP4 hits a wall, the rest of the experience is irrelevant.
For a long stretch we had exactly that wall. Anything past about 10MB failed at the request-body layer before our handler even ran. The student saw no friendly "file too large" message. They saw a 413 from the platform, which our error UI could not parse, which meant a generic failure state with no actionable guidance.
Three things compounded to produce that wall.
First, the body parser is the bottleneck, not storage. Next.js caps API route bodies well below what you would guess from serverActions.bodySizeLimit in next.config. That config knob applies to Server Actions, not to app/api/* route handlers. Multipart uploads through an API route hit the parser ceiling first, regardless of what the config says.
Second, the whole file was in server memory before storage ever saw it. A 12MB upload meant a 12MB ArrayBuffer allocation in the function's heap before a single byte left for Supabase.
Third, failure modes were hostile. The 413 happened above our code, so we had no way to shape the error into something useful.
The framing shift came from staring at the memory problem: the server had no business touching the bytes at all. Its job is to authorize the user, validate the metadata, and write a database row. Carrying the file through the request was something we had inherited from the obvious shape of an upload form, not something we needed.
The shape we landed on
We split the upload into two phases. The browser does the byte transfer directly to Supabase Storage. The server does the bookkeeping. Neither one tries to do the other's job.
The Next.js body limit is not a storage limit. Once we routed bytes around the API request entirely, the limit stopped applying. The server never has to trust the client's declared size or MIME type either: it re-reads both from the bucket using a list() on the exact storagepath the client claims to have written. That verification step is what closes the quota-bypass vector, because the declared size is what drives the quota check.
Direct uploads can fail halfway. Phase 1 can succeed and Phase 2 can error, leaving a blob with no database row. The architecture has to account for that explicitly, not as an afterthought.
We also kept the legacy FormData path alive for callers sending small files. The route detects intake mode by Content-Type and runs the right reader. We did not need a coordinated switchover.
How we built it
The client helper
The browser-side helper works in three steps. It generates a storage path client-side (a timestamp prefix plus a short random suffix plus a sanitized filename), uploads the file directly to the bucket using the storage SDK, and then POSTs a JSON record to the metadata endpoint with the path, declared size, declared MIME type, and any caller-supplied tags or folder.
On a non-2xx response from the metadata endpoint, the helper fires a best-effort remove on the path it just wrote. The failure case does not leak an orphan blob in the common case. If the cleanup itself fails, a server-side reaper catches it later.
The path is generated client-side to avoid a round-trip asking the server for a name. The server retains authority: it can reject any storagepath that does not pass its checks. The client choosing the name is harmless.
The intake-mode switch
The route handler reads Content-Type and dispatches to one of two readers. The JSON reader verifies the blob actually exists by listing the bucket prefix, then reconciles the declared filesize against the real object size. If they disagree, the request is rejected and the orphan is deleted. Without that check, a client could under-report bytes to bypass tier limits. The FormData reader (the legacy fallback) is still bound by the 10MB ceiling, kept alive because rolling every small-file caller over was not worth a coordinated push.
Both readers normalize to the same internal shape, so validation, quota enforcement, the database insert, and the parse-job enqueue are written once.
The validation gauntlet
After the reader returns, every upload flows through the same gates:
| Gate | What it checks | On failure |
|---|---|---|
| Size cap | filesize <= 500MB | 400, orphan deleted |
| MIME allowlist | PDF, DOCX, PPTX, XLSX, TXT, MD, images, MP3, WAV, MP4 (roughly 16 types) | 400, orphan deleted |
| Per-user quota | Tier-aware bytes-remaining check | 403, orphan deleted |
| Auth | Clerk session, with a JWT-decode fallback for clock skew | 401, no row, no blob mutation |
The orphan-deletion step is the architectural cost of direct uploads. Every validation-failure branch fires a best-effort delete on the storage path before returning the error.
Operational details worth naming
Clock skew in production
Clerk session validation occasionally fails on first request after a deploy because of clock drift between the platform and Clerk's servers. We added a fallback that decodes the session JWT directly to extract the user ID when the verified path returns null. It eliminated a class of phantom 401s that disproportionately affected uploads, which run after a user has been idle long enough to pick a file.
"Uploaded but not visible"
The most common direct-upload failure mode is Phase 1 succeeding and Phase 2 not firing, usually because the tab closed or the network blipped between steps. The blob exists, no row exists, the user sees nothing in their materials list.
We handle this two ways. The upload helper attempts a cleanup remove on any non-2xx response from the metadata endpoint. A periodic reaper sweeps the uploads/ prefix for objects whose names predate any matching database row by a wide margin and removes them. Neither layer alone is sufficient: the client cleanup misses tab-close cases, the reaper alone leaves zombie blobs sitting around for a sweep cycle. Together they keep the bucket clean.
Before and after
| Dimension | Before (FormData via API) | After (direct-to-bucket) |
|---|---|---|
| Per-file ceiling | ~10MB (Next.js body parser) | 500MB (app-enforced; was 50MB before Supabase Pro) |
| API request body size | Up to the file size | A few hundred bytes of JSON |
| Server memory per upload | One full ArrayBuffer of the file | None (server never holds bytes) |
| Failure shape on oversize | Platform 413 with no JSON body | App-shaped 400 with parseable error |
| MIME and size validation | After parsing the body | After listing the actual bucket object |
| Orphan-blob risk | None (single phase) | Yes, handled by client cleanup and server reaper |
Qualitative outcomes worth naming: error toasts in the upload modal now reflect what actually went wrong (size, MIME, quota), because the failure happens inside our code instead of above it. Cold-start memory on the upload route dropped meaningfully because we stopped allocating multi-megabyte buffers per request. And the modal can show real upload progress on Phase 1 because it is a fetch directly to storage that emits progress events, which the API-route path never could.
What this taught us
Find the actual layer. "Next.js has a 10MB limit" was true but underspecified. The body parser is the layer. Server Action body limits and storage limits are separate. Routing around the right layer was the unlock.
Do not trust the client's metadata, even when the client uploaded the bytes. Re-reading size and MIME from storage costs almost nothing and closes a real quota-bypass vector.
Two-phase flows mean orphan handling is part of the architecture. Build it in from day one. Client-side cleanup plus a server-side reaper is the durable pair.
Keep the legacy path during the transition. Detecting intake mode by Content-Type let us ship the new path without touching every caller. The cost of two readers is much smaller than the cost of a coordinated rewrite.
Decouple parse from upload. If a 500MB PDF had to finish OCR before the upload returned, we would have re-introduced a different latency cliff. Async parse plus lazy backfill on first read is what makes the larger ceiling actually feel responsive.
What's next
Resumable uploads for flaky connections. Supabase Storage supports tus; we do not use it yet, but at 500MB the case is much stronger than it was at 50MB. A network blip 80% through a half-gigabyte upload is a real failure mode, and "start over" is not a good answer for a student on cellular.
Client-side hashing so re-uploads of the same file dedupe at the bucket level instead of producing a second blob.
The remaining items (tighter signed-URL TTLs, antivirus scanning before parse, a scheduled reaper with metrics) are on the list but not yet started.