Double URL Encoding Bug In Presigned URLs: A Deep Dive

by Rajiv Sharma 55 views

Hey everyone, let's dive into a tricky bug we've found in how presigned URLs handle filenames with special characters. This is especially important for those of you using the parse-server-s3-adapter, so pay close attention! We're going to break down the problem, why it's happening, and how it impacts you.

Understanding the Issue

Presigned URLs are super handy for granting temporary access to files stored in S3. However, a bug is causing double URL encoding for filenames with special characters when using presigned URLs, making them inaccessible. This is a high-severity issue because it directly breaks file access for many users. When using presignedUrl: true, filenames with special characters such as brackets, spaces, or other URL-sensitive characters end up getting double-encoded. The result? Malformed URLs that just won't work. This is a major headache, especially when you rely on presigned URLs for secure and time-limited access to your files.

What's Affected?

  • Affected Version: Current version
  • Component: getFileLocation() method in index.js
  • Severity: High – Breaks file access for filenames with special characters when using presigned URLs

Diving into the Problem Description

The heart of the issue lies within the getFileLocation() method. Let's walk through what's happening step by step. The issue stems from the getFileLocation() method where the filename encoding happens twice. First, on Line 242, the filename is pre-encoded using filename.split('/').map(encodeURIComponent).join('/'). This initial encoding is intended to handle special characters. Then, on Line 247, this pre-encoded filename is then used to construct the S3 key for generating the presigned URL. Finally, on Line 255, the AWS SDK's getSignedUrl() function takes over and, under the hood, encodes the key again. This second encoding is where the problem arises, leading to the dreaded double encoding.

Example Scenario

Let's say you have a filename like doc[123].pdf. Here's what should happen and what's actually going wrong:

  • Expected Behavior: The filename should be encoded to doc%5B123%5D.pdf in the presigned URL. This is the correct, single-encoded format that S3 can understand.
  • Actual Behavior: Instead, it gets double-encoded to doc%255B123%255D.pdf. This is a double-encoded mess that S3 can't interpret, resulting in access errors.

Here's a breakdown of the encoding process:

  1. First Encoding: [123] → %5B123%5D (This is the initial, correct encoding.)
  2. Second Encoding (by AWS SDK): %5B123%5D → %255B123%255D (This is the problematic double encoding.)

Unpacking the Root Cause

To really understand this, let's look at the code snippet where this double encoding is happening:

// Line 242: Pre-encodes the filename
const fileName = filename.split('/').map(encodeURIComponent).join('/');

// Line 247: Uses pre-encoded filename for S3 key
const fileKey = `${this._bucketPrefix}${fileName}`;

// Line 251: AWS SDK encodes the already-encoded key again
const params = { Bucket: this._bucket, Key: fileKey };
presignedUrl = await this.getFileSignedUrl(this._s3Client, command, options);

The key takeaway here is that the filename is being encoded before it's passed to the AWS SDK. The AWS SDK then helpfully (but incorrectly, in this case) encodes it again, leading to the double encoding. This redundant encoding process is the core of the problem, and it's what's causing those inaccessible URLs.

Impact on Your Files

So, what's the real-world impact of this bug? It's pretty significant if you're using presigned URLs.

  • Presigned URLs become inaccessible for files with special characters. This is the most direct and frustrating consequence.
  • It affects any filename containing: [], (), spaces, and other URL-sensitive characters. These characters are commonly used in filenames, so this bug has a wide reach.
  • Interestingly, it works fine for regular S3 URLs (non-presigned) because they don't go through AWS SDK's internal encoding. This can lead to confusion because some URLs work while others don't.

Reproducing the Bug: A Step-by-Step Guide

Want to see this bug in action? Here's how you can reproduce it:

  1. Configure the S3 adapter with presignedUrl: true. This is the crucial setting that triggers the bug.
  2. Upload a file with special characters in the name (e.g., `