Toolium

URL Encoding: encodeURI vs encodeURIComponent, and Which You Want

Hemanth Gedda7 min read

Once, debugging a flaky integration test, I lost three days to a URL that looked correct in the logs but kept returning 404. The path contained %2B instead of +, which in turn was supposed to represent a literal + character in the user's email address, which in the database was stored with the + intact. The test was checking that I could fetch the user's profile by URL-encoded email. The encoding was double-applied somewhere in the test harness: the + got encoded to %2B, then the % in %2B got encoded to %25, producing %252B in the URL. The server decoded once, saw %2B, looked up the user with email username%2B@gmail.com, found nothing, returned 404.

That kind of bug is the reason URL encoding deserves a serious explanation rather than the hand-wave most documentation gives it. This post is the explanation.

The basic mechanism

URLs can only contain a limited character set: letters, digits, and a handful of safe symbols (-, _, ., ~). Everything else has to be "percent-encoded" - represented as a % followed by two hex digits giving the byte value. Space becomes %20. The @ symbol becomes %40. A literal % becomes %25.

For ASCII characters this is straightforward: each character is one byte, encoded as % followed by the hex byte value. For non-ASCII characters, things are more interesting. The character is first UTF-8 encoded into bytes (most non-ASCII characters are multiple bytes in UTF-8), then each byte is percent-encoded. A Japanese character that is three bytes in UTF-8 becomes three consecutive %XX sequences in the URL.

encodeURI vs encodeURIComponent

JavaScript has two URL-encoder functions, and the difference between them is the source of approximately 80% of URL bugs:

  • encodeURI assumes you have an entire URL and leaves URL-structural characters alone. It encodes spaces and other unsafe characters but does NOT encode characters like :, /, ?, #, =, or &, because those have special meaning in the URL structure and encoding them would break the URL.
  • encodeURIComponent assumes you have a piece of data that goes inside a URL and encodes everything that could have any URL meaning. The same characters that encodeURI leaves alone, encodeURIComponent encodes. ? becomes %3F, & becomes %26, = becomes %3D.

The 90%-of-the-time answer: use encodeURIComponent. encodeURI is for the rare case where you have a string that is already a valid URL with maybe one or two unsafe characters in it (a space, an accented letter). encodeURIComponent is for the much more common case of taking a user-supplied value and putting it into a URL.

The classic mistake: building a URL like "/search?q=" + encodeURI(query). If the query contains an &, encodeURI leaves it alone, the URL ends up with an unexpected extra parameter boundary, and the server parses the query wrong. Use encodeURIComponent for query parameter values.

The + sign trap inside query strings

Historically, the application/x-www-form-urlencoded spec (used by HTML form submissions) encoded spaces as + rather than %20. This convention spread to URL query strings, and most server frameworks accept + as an alternate encoding of space inside the query string portion of a URL.

The consequence: a literal + in a query value (e.g., in an email address like user+tag@example.com) has to be encoded as %2B to avoid being interpreted as a space. encodeURIComponent does this correctly (it always encodes + as %2B). But if you concatenate encoded values into a URL manually, and one of those values contains a literal +, the server might decode it as a space, breaking your lookup.

The fix: trust encodeURIComponent for any value you put into a query string. Do not try to be clever about which characters are "safe."

Double-encoding (and how to spot it)

Double-encoding is what bit me in the opening story. It happens when a string gets URL-encoded twice, usually because two layers of code are each trying to ensure safety without checking what the previous layer did.

The signature of double-encoded data is the appearance of %25 followed by what looks like other hex codes. %252B is double-encoded +. %2520 is double-encoded space. If you decode the string once and it still looks like it has percent escapes in it, you have double-encoding.

The Toolium URL Encoder/Decoder does not auto-detect double-encoding - you have to manually run decode twice. This is intentional because automatic double-decoding could mangle strings that legitimately contain %25 sequences.

What about encodeURI's intended use case?

There is a legitimate use for encodeURI: when you have a string that is a complete URL (with scheme, host, path, query) but contains a few unsafe characters that need encoding. For example, a URL with a space in the path: https://example.com/my file.pdf. Running this through encodeURI produces https://example.com/my%20file.pdf, leaving the structural characters alone.

This case is rare enough that I almost never use encodeURI. When I have a URL with structural characters and value characters mixed together, I usually build it from pieces using encodeURIComponent on the value parts.

Reserved vs unreserved characters

The RFC 3986 spec divides URL characters into:

  • Unreserved (always safe, never need encoding): A-Z, a-z, 0-9, -, _, ., ~.
  • Reserved (have special meaning, must be encoded when used as data): :, /, ?, #, [, ], @, !, $, &, ', (, ), *, +, ,, ;, =.
  • Unsafe (must always be encoded): space, control characters, non-ASCII.

encodeURIComponent encodes both reserved and unsafe. encodeURI encodes only unsafe.

Common encoded values you should recognize

  • %20 - space
  • %2B - plus sign
  • %2F - forward slash
  • %3F - question mark
  • %23 - hash
  • %25 - literal percent
  • %26 - ampersand
  • %3D - equals sign
  • %40 - at sign

The Toolium tool handles all of these correctly, with full Unicode support so non-ASCII characters round-trip without corruption.

Try the tool mentioned in this article

Open tool