I'm having an issue parsing data after reading a file. What I'm doing is reading a binary file in and need to create a list of attributes from the read file all of the data in the file is terminated with a null byte. What I'm trying to do is find every instance of a null byte terminated attribute.

Essentially taking a string like:

Health\x00experience\x00charactername\x00

and storing it in a list.

The real issue is I need to keep the null bytes in tact, I just need to be able to find each instance of a null byte and store the data that precedes it.

While it boils down to using split('\x00') a convenience wrapper might be nice.

def readlines(f, bufsize):
    buf = ""
    data = True
    while data:
        data = f.read(bufsize)
        buf += data
        lines = buf.split('\x00')
        buf = lines.pop()
        for line in lines:
            yield line + '\x00'
    yield buf + '\x00'

then you can do something like

with open('myfile', 'rb') as f:
    mylist = [item for item in readlines(f, 524288)]

This has the added benefit of not needing to load the entire contents into memory before splitting the text.

more