Base64, Explained Like You Are Going to Implement It

The thing nobody told me about base64 when I was learning it was that the browser's built-in btoa() function does not handle Unicode. The function is named like it converts "binary to ASCII," which sounds general-purpose, but it is in fact strictly limited to single-byte values (Latin-1 codepoints, 0-255). If you call btoa("é"), you get an exception. btoa("hello world") works. btoa("héllo world") throws "InvalidCharacterError: String contains an invalid character."

This is the bug that ate the first version of Toolium's Base64 Encoder/Decoder. A user pasted in a Japanese product name and got an error message that did not explain what was wrong. Why does this happen? Because btoa was designed in the 1990s when ASCII was the working assumption, and JavaScript strings since then have moved on to being UTF-16, but the btoa API was never updated. The fix is to UTF-8 encode the string into bytes first, then base64-encode the bytes. Modern code uses TextEncoder for that conversion. Once I added the TextEncoder pre-pass, every Unicode input started working.

That story is a useful starting point for explaining what base64 actually does, because it forces you to think about the layers: there is the encoding from text into bytes (UTF-8 in modern systems), and then there is the encoding from bytes into ASCII (base64). Those are two different operations, and conflating them is the source of most base64 bugs.

What base64 actually does, at the byte level

Base64 takes a sequence of bytes (each byte is 8 bits, 0-255) and produces a sequence of characters from a fixed 64-character alphabet. The alphabet, by convention, is A-Z (26) + a-z (26) + 0-9 (10) + + + / = 64 characters. Each character in the output represents exactly 6 bits of the input.

The mechanics: take 3 bytes from the input (24 bits total). Split those 24 bits into 4 groups of 6 bits. Look up each 6-bit value in the 64-character alphabet. Emit those 4 characters. Repeat until you have processed every input byte.

What happens if the input is not a multiple of 3 bytes? Pad it with zero bits, but mark the padding in the output with = characters so the decoder knows which bits are real. An input of 1 byte produces 4 output characters with 2 padding =s. An input of 2 bytes produces 4 output characters with 1 padding =. An input of 3 bytes (or any multiple of 3) produces output with no padding.

That is the whole algorithm. There is no compression, no encryption, no checksum. It is purely a re-encoding from one alphabet to another, fully reversible, completely deterministic.

Where the 33% overhead comes from

Every 3 input bytes become 4 output characters. 4/3 is 1.333. So base64 output is exactly 33% larger than the input it represents. There is no way around this; it is a property of the encoding.

This overhead is sometimes a real cost. If you base64-encode a 100 MB binary file to put it in a JSON field, you are now shipping 133 MB across the wire. For small assets - tiny icons inlined in CSS, short binary tokens in JWT - the overhead is negligible. For large assets, base64 is the wrong tool and you should use multipart upload, a separate binary endpoint, or a presigned URL instead.

Why base64 exists in the first place

The history is worth knowing because it explains why base64 looks the way it does. Early email systems (SMTP, defined in the 1980s) could only transmit 7-bit ASCII text. Sending a binary attachment - an image, a PDF, anything - required some way to represent those binary bytes inside ASCII text. The MIME standard solved this by defining content-transfer-encodings, of which base64 is one. The choice of 64 characters from the ASCII set was a careful trade-off: too many characters and you would include ones that some mail relays interpreted as control codes; too few and the overhead would be unbearable. 64 turned out to be the sweet spot.

Since then, base64 has been pulled into a much wider range of uses. JSON cannot contain raw binary bytes (it is a text format), so any system that needs to embed binary in JSON uses base64. URLs cannot contain certain characters, so URL-safe variants emerged (more on this below). Data URIs use base64 to embed binary inside CSS or HTML. JWT tokens use base64 to encode their three sections.

Base64 vs base64url

The standard base64 alphabet includes + and /, both of which have meanings inside URLs (+ is sometimes interpreted as a space, / is a path separator). To allow base64 strings to be put in URLs without escaping, the base64url variant replaces those two characters with - and _ respectively, and omits the = padding altogether.

JWT tokens use base64url because they often live in URLs (authorization redirects, magic links). The Toolium decoder auto-detects which variant is being used: if the input contains - or _ but not + or /, it assumes base64url; otherwise standard base64.

The bytes you get back are identical between the two variants. The difference is only in the encoded form.

The UTF-8 trap, in more detail

This deserves its own section because it bites everyone the first time.

JavaScript strings are sequences of UTF-16 code units, not bytes. When you have a string like "héllo," the actual data in memory is 5 code units: h, é, l, l, o. The character "é" is a single code unit in UTF-16.

btoa expects single-byte values. It iterates the input string and demands that every code unit fit in 8 bits. "é" has the codepoint U+00E9, which fits, so technically btoa("héllo") could work; but the resulting bytes would be Latin-1 bytes, not UTF-8. Now an emoji like 🎉 has codepoint U+1F389, which does not fit in 8 bits at all, so btoa throws an error.

The fix, and the architecture every modern base64 encoder uses, is to do the UTF-8 conversion first:

const bytes = new TextEncoder().encode(text);
const base64 = btoa(String.fromCharCode(...bytes));

Now the string passed to btoa is guaranteed to be in the single-byte range (because UTF-8 bytes are always 0-255 by definition), and the encoding is well-defined. On the decode side, you do the reverse: base64 decode to bytes, then UTF-8 decode the bytes to text via TextDecoder.

If you ever see a base64 decoder that mangles non-ASCII characters into question marks or garbled symbols, it is almost certainly skipping the UTF-8 step.

The "is this encryption?" question

People sometimes use base64 thinking it obfuscates data. It does not. Base64 is fully reversible by anyone who has the encoded string. There is no key, no secret, no algorithm to keep hidden. A casual observer cannot read a base64 string at a glance, but a slightly less casual observer can paste it into any decoder and read the contents.

If you actually need to hide data, use real encryption (AES, ChaCha20, libsodium, whatever your language ecosystem recommends). The base64 step happens after the encryption, only if you need the ciphertext to be ASCII-safe for transport.

Concrete uses I have seen base64 used for, well and badly

• Inlining small icons in CSS or HTML. Good use. Saves an HTTP request for icons under ~5 KB, where the 33% size overhead is less than the latency cost of a separate fetch.
• Sending file uploads in JSON. Acceptable for small files, bad for large ones. Multipart form upload is more efficient for anything over a few hundred KB.
• JWT tokens. Good use. The three sections need to be URL-safe and to survive transport through HTTP headers and query strings without escaping.
• "Encrypting" passwords in a config file. Bad use, and dangerously misleading. Base64 is not encryption. Anyone with the config file can read the password.
• Image data URIs in emails. Acceptable but heavy. Modern email clients support cid: references to attached images, which is more efficient.
• API keys with characters like + or /. Usually fine, but base64url is friendlier for keys that get pasted into URLs.

How the Toolium encoder is built

The encoder runs in your browser. The encode flow is: TextEncoder converts your input string to UTF-8 bytes, then btoa converts the bytes to a base64 string. The decode flow is the reverse: atob converts the base64 string to a string of single-byte values, then we pull the byte values out and run TextDecoder on them.

The auto-detection between base64 and base64url is a check on the input characters. If the input contains + or /, it is standard base64. If it contains - or _, it is base64url. If it contains neither (only A-Z, a-z, 0-9), both variants are valid and the result is the same either way.

Nothing leaves your browser. If you are base64-encoding a credential or a token, the data is not transmitted to my server (because there is no server). Open the network tab while you use the tool; you will see no requests carrying the input.