Hex Is Twice the Size of Your Data, Base64 Is a Third More
Hex and base64 solve the same problem — get binary through a channel that only carries text — and one of them costs six times as much as the other. Hex still turns up constantly, and usually for a reason that has nothing to do with size.
Sixty-four kilobytes of data, written three ways:
| size | overhead | |
|---|---|---|
| raw bytes | 65,536 B | — |
| base64 | 87,384 B | +33.3% |
| hex | 131,072 B | +100% |
Hex doubles. Exactly, with no rounding: every byte becomes two characters, so the output is always precisely twice the input.
Why hex is so wasteful, and why it survives anyway
A byte holds 256 possible values. Hex spends two characters on it, and each of those characters is drawn from an alphabet of sixteen — so the pair carries 16 × 16 = 256 values. Perfectly efficient use of the alphabet, and a terrible use of the space, because eight bits of information are being written in sixteen bits of text.
Base64 does better by using a bigger alphabet. Sixty-four symbols is six bits each, so four symbols carry 24 bits — three whole bytes. Four characters for three bytes instead of two characters for one.
Hex survives because it is readable, and that is worth a great deal in the places it is used:
- You can find a byte. The fifth byte of a hex string is at position ten, always. In base64 the boundaries do not line up with byte boundaries at all, so no character corresponds to a byte you can point at.
- You can compare it by eye. Two hex checksums that differ are visibly different in the same position. Two base64 strings that differ by one bit look different everywhere after that point.
- It is case-insensitive and has no punctuation. Base64 uses
+,/and=, all of which mean something in a URL, which is why URL-safe base64 is a separate variant.
Where you meet each one
Hex is the format of hashes and fingerprints. A SHA-256 digest is 32 bytes and you always see it as 64 hex characters. That is a doubling nobody minds, because 64 characters is small and being able to read it matters — you compare the first and last few characters when checking a download.
Base64 is the format of payloads. Anything where the size is the point and nobody will read it: email attachments, data URIs, files inside JSON, binary in a config file.
The rough rule: hex when a human will look at it and it is short, base64 when a machine will read it and it is long.
The one that catches people
Storing a file as hex in a database column because it "looks cleaner" than base64 costs you the whole file again. On a table of a million small blobs that is not a rounding error, and it does not compress away either — hex has less structure for a compressor to work with than you might hope, because the alphabet is small but the byte pattern is not repetitive.
If the reason for hex is readability and the field is never read, base64 is a third instead of double. If the reason is that something downstream expects hex, that is a real constraint and the size is the cost of it.
Check the overhead yourself
head -c 65536 /dev/urandom > sample.bin
base64 -w0 sample.bin | wc -c
xxd -p sample.bin | tr -d '\n' | wc -c
The second number will be roughly a third above 65,536 and the third will be exactly twice it. Repeat with a text file, a JPEG or anything else and the ratios do not move — which is the point of the article, and takes ten seconds to confirm on your own machine.
The variant that catches people out
Standard base64 uses +and /, and both mean something in a URL: +decodes
as a space in a query string, and /is a path separator. A base64 value pasted
into a URL therefore arrives corrupted, intermittently, depending on the value.
URL-safe base64 swaps them for -and _and usually drops the =padding.
Same size, different alphabet, and the two are not interchangeable — a decoder
expecting one will fail on the other.
Hex has no such problem, because its alphabet is sixteen characters that are safe everywhere. That is another reason it survives in places where the doubling is affordable: a hash in a URL, a token in a filename, an identifier in a log line.
The practical version
Base64 encode/decodeis the one to reach for when the job is moving data through a text channel. For hashes, SHA-256 hash generatorgives you the hex digest that everything else expects, and file checksumdoes the same for a file you want to verify — both cases where the doubling is exactly the right trade.