The most effective way to keep base64 data in Python?


Suppose you have a MD5 hash encoded in base64. Then each character needs only 6 bits to store each character in the resultant 22-byte string (excluding the ending '=='). Thus, each base64 md5 hash can shrink down to 6*22 = 132 bits, which requires 25% less memory space compared to the original 8*22=176 bits string.

Is there any Python module or function that lets you store base64 data in the way described above?

The most efficient way to store base64 encoded data is to decode it and store it as binary. base64 is a transport encoding - there's no sense in storing data in it, especially in memory, unless you have a compelling reason otherwise.

Also, nitpick: The output of a hash function is not a hex string - that's just a common representation. The output of a hash function is some number of bytes of binary data. If you're using the md5, sha, or hashlib modules, for example, you don't need to encode it as anything in the first place - just call .digest() instead of .hexdigest() on the hash object.