S3 for Developers: Presigned URLs and Multipart Uploads

Priya's app let users upload videos up to 4 GB. The first version sent each file to her API, which then uploaded it to S3. It worked for small files. For larger ones, it ran out of Lambda memory, hit API Gateway's 30-second timeout, and cost twice as much as it should have.

The fix is the pattern every AWS developer eventually learns: let clients talk to S3 directly. Two features make this possible — presigned URLs and multipart uploads.

Presigned URLs

A presigned URL is a time-limited URL that grants temporary permission to GET or PUT a specific S3 object, without the user needing AWS credentials. Your backend generates the URL using its own credentials, sends it to the client, and the client uploads or downloads directly.

The flow:

• Client requests an upload URL from your API;
• Your API calls getSignedUrl from the AWS SDK, specifying the bucket, key, and expiry;
• API returns the URL to the client;
• Client PUTs the file directly to S3 using that URL;
• S3 verifies the signature and accepts the upload. The backend never touches the bytes. Your Lambda finishes in 50 milliseconds. Your bandwidth bill drops.

A minimal SDK call in Node.js:

const url = await s3.getSignedUrlPromise('putObject', {
  Bucket: 'acme-uploads',
  Key: `videos/${userId}/${videoId}.mp4`,
  Expires: 900,
  ContentType: 'video/mp4'
});

The URL is valid for 900 seconds (15 minutes). Maximum expiry is 7 days for signature version 4, or 12 hours when the URL is generated by an IAM role using temporary credentials.

When to Use Presigned URLs

The pattern fits whenever a client needs direct access to a single object:

• User uploads (profile pictures, videos, attachments);
• Temporary download links sent in email;
• Cross-tenant sharing in SaaS apps where you do not want to give the user IAM access;
• Pre-authorized downloads that expire after checkout. A common mistake: generating a presigned GET URL with a very long expiry (say, 7 days) and emailing it to a user. If the email leaks, anyone can download the object until the URL expires. Keep expiries short and regenerate on demand.

Multipart Uploads

Files larger than 5 GB cannot be uploaded in a single PUT — AWS requires multipart upload. Files smaller than 5 GB can use multipart anyway, and for files over 100 MB AWS recommends it.

The flow:

• Call CreateMultipartUpload — S3 returns an upload ID;

• Split the file into parts, each between 5 MB and 5 GB (except the last);

• Upload each part with UploadPart in parallel — each call returns an ETag;

• Call CompleteMultipartUpload with the list of parts and ETags — S3 assembles the file;

• Or call AbortMultipartUpload to discard everything if it fails. Multipart gives three big wins:

• Parallelism — upload 10 parts at once, finish the upload in a tenth of the wall time;

• Resilience — if one part fails, retry just that part instead of the whole file;

• Resumability — pause an upload overnight, resume in the morning with the upload ID.

Combining the Two

The pattern Priya ended up with handles videos of any size:

• Client requests a multipart upload, your API calls CreateMultipartUpload;
• API returns the upload ID to the client;
• For each part, client requests a presigned URL for UploadPart with that ID and part number;
• Client uploads each part directly to S3 via its presigned URL, in parallel;
• Once all parts succeed, client calls your API, which calls CompleteMultipartUpload. The backend stays small. The upload survives network blips. Lambda timeouts stop mattering.

Cleaning Up After Failure

Multipart uploads that never complete leave parts in the bucket — invisible in the Console, but accruing storage costs. Every production bucket should have a lifecycle rule that aborts incomplete multipart uploads after 7 days:

Rules:
  - Id: CleanUpIncompleteUploads
    Status: Enabled
    AbortIncompleteMultipartUpload:
      DaysAfterInitiation: 7

Five lines. Saves teams from a slow-growing storage bill they cannot otherwise explain.