3 ms·
Show HN: bstruct – Declarative and efficient binary parsing in Python
- flxbe 4y agoI was unhappy with the performance, rough type support and general usability of construct[0], as well as the large amount of glue-code necessary when working with Python's built-in struct library directly. This project is a re-implementation of a minimal prototype I already used in production for over a year. My goal is to find a more balanced approach to translate between binary data and complex data types. The design works well for me so far, retaining enough performance while still being pleasant to work with. The information for encoding/decoding is derived only from type annotations on (possibly nested) dataclasses. The annotations are flattened in order to create a single struct.Struct instance, which is used to efficiently pack/unpack the binary data. The project is still in a fairly rough state, but I am very happy to get some feedback. Cheers! [0]: https://pypi.org/project/construct/ https://pypi.org/project/construct/
- mdaniel 4y agoCongratulations on releasing it! How would one encode something like the Java class file, which has "pascal strings," tagged structs, and repetition? e.g. https://github.com/kaitai-io/kaitai_struct_formats/blob/c61bec5f60f04059636c3b5c8777a3995b5bb103/executable/java_class.ksy#L23-L26 https://github.com/kaitai-io/kaitai_struct_formats/blob/c61b... And, my other favorite thing to pick on: file formats with offset references
- flxbe 4y agoThanks! For reading an arbitrarily sized sequence, you could use the "decode_all" function (Sorry, this one is not documented yet). Besides that, the library only supports parsing data with a statically known layout. If your data contains any self-describing sections, you would have to parse in multiple steps. Not including these features natively was a deliberate decision, as my personal preference in those cases is to write the outer glue code myself.