Node.js file upload libraries for large files and streams

Why large uploads break apps

Large uploads fail for boring reasons, not exotic ones. A single 3 GB video can push a weak upload flow into trouble fast, especially if the app reads too much of the file into RAM before it writes anything out. If two or three people do that at once, memory usage jumps, garbage collection gets busy, and the server starts to feel slow long before it crashes.

This catches teams off guard because normal testing often uses small files on fast office internet. Real users do the opposite. They upload from phones, home Wi-Fi, or a train connection that drops for a few seconds. That means the request stays open for a long time, sometimes for minutes, and your Node.js process keeps sockets, buffers, and temp work alive the whole time.

Retries make the mess worse. A user sees the progress bar freeze, refreshes the page, and starts again. The browser may retry too. If your app already wrote part of the file or created a database row, you can end up with duplicate objects in storage and duplicate records in the app. That is not just wasted space. It also creates cleanup work and confusing support tickets.

Limits often clash in places people forget to check. The browser sends a big multipart request, a proxy allows 100 MB, your app allows 500 MB, and object storage would have accepted 5 GB. The request dies at the proxy, while the app logs look harmless. In other setups the proxy timeout is shorter than the app timeout, so slow uploads fail even though the server code looks fine.

Good Node.js file upload libraries help, but they do not fix a bad path on their own. Large file uploads in Node.js stay stable when the whole chain agrees on size limits, timeouts, retries, and where bytes should go. If that chain is sloppy, one upload is annoying. Ten uploads can take the server down.

What happens between the browser and storage

A large upload starts before your Node.js code sees a single byte. The browser opens a connection and sends either multipart/form-data or a raw stream. Multipart wraps the file together with normal form fields like "title" or "projectId". A raw stream sends the file body by itself, which is simpler when you only need the file.

If you run a proxy in front of your app, that proxy accepts the connection first. nginx, a cloud load balancer, or another edge service can buffer some data, reject requests that are too large, or pass the stream through. This step matters because upload limits and timeouts often live there, not in your app.

Once the request reaches Node.js, a parser reads the incoming bytes. With multipart upload handling, the parser has two jobs at the same time: it extracts form fields and it emits file chunks as they arrive. Good parsers do this incrementally. Bad setups read too much into memory and turn one 2 GB video into a server problem.

Your app then decides where those bytes go. It might write them to local disk, push them to object storage, or pipe them to another internal service for scanning or processing. For large file uploads in Node.js, the safest path is usually a stream from request to destination, with as little buffering as possible in between.

A simple example helps. A user uploads a 4 GB video. The browser sends multipart data. The proxy accepts the connection. The parser reads the "title" field, then starts emitting video chunks. Your app pipes those chunks straight to storage instead of building a giant Buffer. If storage slows down, the stream should slow down too. That backpressure keeps memory from climbing.

The response should wait until the write really finishes. Do not reply with 200 OK when the request ends if the file is still flushing to disk or storage. Reply only after the destination confirms success. If the write fails halfway through, the client needs an error, not a false success.

Which library fits which job

Some Node.js file upload libraries make small forms easy. Others help you move big files without filling memory or blocking the app. The best choice depends less on popularity and more on how much control you need over the upload stream.

Multer fits simple Express apps. If users upload profile photos, PDFs, or a few small attachments, Multer keeps the code short and easy to read. It starts to feel cramped when files get large or when you want tight control over where bytes go while they arrive.

Busboy is a better match for large file uploads in Node.js. It gives you direct access to multipart upload handling at the stream level, so you can pipe data to disk or object storage as it comes in. That usually means lower memory use and fewer nasty surprises when someone uploads a 4 GB video instead of a 4 MB image.

Formidable sits in the middle. It parses form fields and files in one place, which helps when a request includes both metadata and uploads. It is often easier to work with than raw streaming code, but you still need to check its temp file behavior and cleanup rules.

If you use Fastify, @fastify/multipart is usually the cleanest option. It fits the framework well and supports streaming file uploads without forcing Express-style middleware into the app.

A quick way to sort the options:

• Choose Multer when the app is simple, the files are modest, and Express already handles your forms.
• Choose Busboy when performance matters and you want direct control over streams, limits, and storage.
• Choose Formidable when one parser for fields and files makes request handling easier.
• Choose @fastify/multipart when your app already runs on Fastify.

Before you pick, check three boring details. First, set file size and field count limits. Second, learn whether the library stores temp files and where it puts them. Third, test error handling: canceled uploads, half-open connections, full disks, and storage failures. Those cases decide whether your server stays calm or falls over under load.

If you expect steady traffic or very large media files, low-level streaming or direct-to-storage uploads usually age better than convenience middleware.

When multipart parsing still makes sense

Multipart parsing is still the right choice when a file arrives with form fields that your server needs right away. A common case is an admin page that uploads a CSV, image, or PDF along with a user ID, project ID, tags, or a moderation flag. The server can read the fields, check permissions, and decide where the file should go before it stores anything.

This pattern also fits internal dashboards better than direct-to-storage uploads in many cases. Internal tools often have fewer users, smaller traffic spikes, and simpler security rules. A team usually wants one request, one endpoint, and clear validation. That is easier to build and maintain than signed upload flows when the audience is a few staff members rather than thousands of public users.

Server-side parsing also helps when you must inspect metadata first. Maybe you need to reject files with the wrong MIME type, match the upload to an existing record, scan the filename, or enforce a rule like "only accountants can upload month-end reports." If your app needs that decision before storage, multipart handling stays useful.

That does not mean you should buffer the whole file in memory. Good Node.js file upload libraries let you parse fields and stream the file onward at the same time. You keep the simple form flow, but you avoid the memory spike that breaks apps under load.

Set limits before you accept real traffic. Start with a few hard rules:

• cap file size per upload
• cap the number of files per request
• cap the number of text fields
• cap field size for long notes or JSON blobs
• reject unexpected field names

Those limits matter even for a private admin tool. One mistaken export, one dragged folder, or one huge text field can tie up your Node process for no good reason.

A simple example: an operations team uploads a product video and enters a product SKU, language, and publish date in the same form. Multipart parsing makes sense because the server must verify the SKU, check who is logged in, and attach the upload to the right record. That is much simpler than sending the video to storage first and fixing the metadata later.

How to stream uploads step by step

Most Node.js file upload libraries can either buffer or stream. For large file uploads in Node.js, buffering is where trouble starts. A 4 GB video can fill memory fast, stall other requests, and make a small server fall over.

Treat the incoming upload like a moving data pipe. Read each chunk as it arrives, inspect what you can early, and send it straight to its destination.

1. Start with the request stream, not req.body. If your framework tries to parse the whole body first, turn that off for this route.
2. Check access right away. Verify the user, reject blocked file types, and stop uploads that exceed your size limit before you write too much data.
3. Open the destination stream early. That destination might be a temp file, object storage, or another process that handles media work.
4. Count bytes while data flows. If the upload grows past the allowed limit, destroy the stream and clean up the partial file.
5. Wait for the destination to confirm the write. Only send a success response after the file is fully stored and the final stream event fires.

A few details matter more than people expect. Set a request timeout that fits slow but normal connections. Listen for client disconnects, because users close tabs and mobile networks drop. When that happens, stop writing and delete incomplete output.

Content type checks help, but do not trust them alone. A browser can claim a file is video/mp4 when it is not. For anything sensitive, inspect the file signature after the first chunk or pass the upload to a worker that can verify it safely.

Streaming file uploads also make costs easier to control. If you run lean infrastructure, you do not want uploads sitting in app memory while your server waits. Oleg Sotnikov often pushes teams toward this kind of design for exactly that reason: the app stays responsive, and you do not need oversized machines just to survive upload traffic.

If the destination write fails, return an error and remove any partial data. A half-saved file is not success.

When direct-to-storage works better

If your app only needs to approve the upload and remember what the file is, sending the file through your Node.js server is often the wrong move. Let the browser upload it straight to object storage instead. Your app stays light, and your server does not spend minutes babysitting a 5 GB request.

This pattern works well for large file uploads in Node.js because your app does three small jobs instead of one heavy one. It checks who the user is, decides whether that user can upload the file, and creates a short-lived upload permission. After that, the browser sends the file to storage on its own.

Your app can also create a record before the upload starts. That record might include the file name, size, content type, project ID, and the storage path you expect. This gives you a clean way to track uploads without turning your API into a file pipe.

A simple flow looks like this:

• The user asks to upload a file.
• Your app returns a signed upload URL or temporary storage credentials.
• The browser uploads the file directly to storage.
• The browser calls your app again to confirm completion, or storage sends an event.
• Your app verifies the object exists and marks the upload as ready.

That follow-up step matters. Files fail halfway, users close tabs, and networks drop. If you never confirm that the object arrived, your database fills with uploads that do not exist.

Direct-to-storage uploads fit apps that do not need to inspect every byte on the way in. Media libraries, user document portals, backups, product asset managers, and training data uploads often fit this model well. Your Node app handles rules and metadata, while storage handles the heavy transfer.

If you must scan the file, rewrite it, or reject content based on deep inspection before storage, keep the upload in your own pipeline. Otherwise, this approach usually beats most Node.js file upload libraries for very large files because it cuts memory pressure, lowers timeout risk, and keeps your app free for actual app work.

A simple example with a large video upload

A customer needs to upload a 4 GB training video from weak hotel Wi-Fi. If your Node.js app tries to receive that whole file, keep it in memory, and then forward it to storage, you invite trouble fast. One shaky connection can tie up your server for minutes, or much longer.

A safer flow keeps the app out of the data path. Your server creates a short-lived upload permission, often as a signed request or temporary token. The browser uses that permission to send the video straight to object storage.

That small change matters. The app handles a tiny auth request, not a 4 GB stream. If the hotel Wi-Fi drops for a moment, the upload client can retry or resume without forcing your Node process to juggle a giant request body.

A practical flow looks like this:

• The user clicks Upload and the app asks your Node backend for temporary upload permission.
• The backend checks who the user is, which file they want to send, and how large it can be.
• The browser uploads the video directly to storage and shows progress in the UI.
• Storage confirms the upload, and only then does your app create the final database record.

While the file moves, your app can still track progress. The browser reports bytes sent, and the UI shows percent complete. For long uploads, that feedback matters. Users stay calmer when they can see movement instead of a frozen spinner.

The background work should wait. Do not start transcoding, virus scanning, or thumbnail generation when the first chunk arrives. Start that job only after storage says the full file landed. That rule prevents half-finished uploads from kicking off expensive work.

The database model stays cleaner too. Instead of saving partial state for every chunk, store one final record when the upload succeeds. You can keep a temporary upload session in cache if needed, but the permanent row should appear only after the file exists in storage.

This is why direct-to-storage uploads beat many default setups for large file uploads in Node.js. Your app stays responsive, memory stays flat, and a bad network hurts the upload less than it hurts your server.

Mistakes that cause memory spikes

A lot of upload problems start before the file reaches storage. The server reads too much into memory, waits too long to reject bad input, or keeps half-finished files around after the client disappears. With large file uploads in Node.js, small choices add up fast.

The most common mistake is buffering the whole file in middleware before you validate anything. A 2 GB video should never sit in RAM just so the app can check its type, size, or user token. If your upload path turns each file into a Buffer first, memory usage can jump hard under even light traffic.

Writing every upload to local disk can go wrong too, especially in small containers. Many teams treat container storage like a normal hard drive, then wonder why uploads fail after a few retries. Local temp files fill fast, and ephemeral storage often disappears when the container restarts. That leaves broken uploads, wasted disk, and extra load when users try again.

Backpressure is another place where apps break. If the app reads incoming data faster than it can write to disk or object storage, chunks pile up in memory. If you ignore stream errors or partial writes, the app may keep holding data for a file that will never finish.

A few warning signs show up early:

• Upload handlers create Buffers for full files
• Temp directories grow and never shrink
• Reverse proxy limits are higher than app limits
• Canceled uploads leave files behind
• Memory climbs during slow network uploads

Proxy settings often hide the problem. If nginx or another proxy accepts a larger file than your app allows, the request can travel deep into the stack before the app rejects it. That wastes bandwidth, CPU, and memory. Set limits in the proxy and the app, and keep them aligned.

Canceled uploads need cleanup too. If the browser closes the tab halfway through a 4 GB upload, your app should stop writing and delete the temp file right away. Teams that skip this end up with orphaned files and full disks. On systems like the ones Oleg builds for lean production stacks, that kind of cleanup is not optional. It keeps costs low and keeps the server calm under load.

Quick checks before you ship

Large uploads usually fail in boring places: size limits, cleanup, timeout settings, and retry confusion. A feature can look fine with a 20 MB test file, then fall apart when someone sends a 4 GB video over weak Wi-Fi.

Before release, test the unhappy path on purpose. If your app says it allows files up to a certain size, send one that goes past that limit and watch the full response. The server should stop work early, return a clear error, and avoid half-written files or stuck workers.

A short pre-launch pass catches most of the expensive mistakes:

• Throttle the network and upload a large file slowly. Watch for timeouts, hanging requests, and memory growth that never drops.
• Start an upload, then kill the browser tab or cut the connection. Check whether your app closes streams and frees disk space right away.
• Force a parser or storage error in the middle of the upload. Temp files should disappear, not pile up on the server.
• Record basic facts for every attempt: how long it ran, how many bytes arrived, where the write stopped, and the error that ended it.
• Pick one place for retries. If the browser retries, the app should not also retry to storage behind the scenes.

This last point saves a lot of pain. When retries happen in two places, one failed upload can turn into duplicate writes, extra storage costs, or users seeing the same file twice. For large file uploads in Node.js, simple rules beat clever behavior.

If you use Node.js file upload libraries that buffer too much, these tests expose that fast. Memory climbs, temp folders grow, and stalled uploads stay open far too long. Good upload code fails fast, cleans up after itself, and leaves a trail in the logs that a human can read in ten seconds.

Run these checks with the same limits and infrastructure settings you plan to use in production. That is where the ugly bugs show up.

What to do next

Pick one upload path and commit to it. Teams get into trouble when they mix parsed multipart, ad hoc buffering, and half-finished streaming in the same app.

For most projects, the choice is simple:

• Use parsed multipart when files are small, forms are simple, and your server needs the whole payload at once.
• Use server-side streaming when the app must inspect, transform, or route the file while it arrives.
• Use direct-to-storage uploads when the app should stay out of the data path and you want the lowest memory pressure.

That decision matters more than the specific Node.js file upload libraries you compare. A good library cannot save a design that forces your app server to hold giant uploads in memory.

Next, write down hard limits before you ship. Set a maximum file size, a maximum file count per request, and a maximum request time. Put those numbers in code, in your reverse proxy, and in your storage rules. If your product allows 2 GB videos, say so clearly. If it only allows five images under 20 MB each, enforce that early and return a plain error message.

Then add monitoring. Do it before real users find the weak spot for you. Track memory use, request duration, upload failures, aborted connections, storage errors, and queue buildup if you process files after upload. One bad upload path can hide for weeks and then crash the app during a busy hour.

A small test plan helps too. Upload one normal file, one file near the limit, one file far above the limit, and one upload over a slow connection. Watch what your server does, not what you hope it does.

If uploads start affecting product rules, infrastructure choices, and cloud spend, get an outside review. Oleg Sotnikov helps startups and small teams sort out architecture, infrastructure, and AI-driven development as a Fractional CTO advisor. This kind of review usually pays off when uploads touch storage cost, uptime, and developer time all at once.

Make one decision, set the limits, and test the worst case. That is where broken servers usually show up.