4 ms·

Any other lib that also generates Encoder/Writer code?

comex 6y ago
The Construct library for Python can do it: https://construct.readthedocs.io/en/latest/intro.html#example https://construct.readthedocs.io/en/latest/intro.html#exampl... I’ve long searched for something better than Construct, but so far I have yet to find even an equal.
- wpietri 6y ago
  Ooh, exciting! I built a parser [1] for AIS messages [2], a quirky ship-to-ship protocol. My lower-level stuff always felt clumsy to me. I'll have to see if this cleans it up. [1] https://github.com/wpietri/simpleais https://github.com/wpietri/simpleais [2] https://gpsd.gitlab.io/gpsd/AIVDM.html https://gpsd.gitlab.io/gpsd/AIVDM.html
- attheicearcade 6y ago
  It can, but it can get incredibly slow for large formats. I was using it to reverse engineer some binary game formats but the parser would take a couple of minutes to complete. I rewrote it using struct and that time dropped to a few seconds. Useful for probing an unknown format, but I prefer the 010 editor since it’s more interactive.
armsforsharks 6y ago
I have been working on Deku: a declarative binary reading and writing: bit-level, symmetric, serialization/deserialization library. https://github.com/sharksforarms/deku https://github.com/sharksforarms/deku
jchw 6y ago
I wrote a sort-of adjacent library for Go at one point. I’m a bit stuck trying to figure out exactly what to do on 2.0 but it has a lot of Kaitai like features including an expression language for transforming things (on master version) and it supports writing structures out. https://github.com/go-restruct/restruct https://github.com/go-restruct/restruct
tprynn 6y ago
BinData (https://github.com/dmendel/bindata https://github.com/dmendel/bindata) is a Ruby gem for this, basically using a DSL in Ruby to declaratively define binary data formats that can be both read and written.
touisteur 6y ago
Recordflux?
floppy123 6y ago
and still there is no generator that creates efficient and partialy streamed readers/writers for high performance protocols or resource constraint environments (as less pre allocations as possible, zero copy concepts,..., streamed reading/writing, good inline possibility, ...) 100% fix formats with no self reference or for example checksums that sits in front of the checksumd data (no streaming possible)...

The Construct library for Python can do it: https://construct.readthedocs.io/en/latest/intro.html#example https://construct.readthedocs.io/en/latest/intro.html#exampl... I’ve long searched for something better than Construct, but so far I have yet to find even an equal.

wpietri 6y ago
Ooh, exciting! I built a parser [1] for AIS messages [2], a quirky ship-to-ship protocol. My lower-level stuff always felt clumsy to me. I'll have to see if this cleans it up. [1] https://github.com/wpietri/simpleais https://github.com/wpietri/simpleais [2] https://gpsd.gitlab.io/gpsd/AIVDM.html https://gpsd.gitlab.io/gpsd/AIVDM.html
attheicearcade 6y ago
It can, but it can get incredibly slow for large formats. I was using it to reverse engineer some binary game formats but the parser would take a couple of minutes to complete. I rewrote it using struct and that time dropped to a few seconds. Useful for probing an unknown format, but I prefer the 010 editor since it’s more interactive.

armsforsharks 6y ago

I have been working on Deku: a declarative binary reading and writing: bit-level, symmetric, serialization/deserialization library. https://github.com/sharksforarms/deku https://github.com/sharksforarms/deku

jchw 6y ago

I wrote a sort-of adjacent library for Go at one point. I’m a bit stuck trying to figure out exactly what to do on 2.0 but it has a lot of Kaitai like features including an expression language for transforming things (on master version) and it supports writing structures out. https://github.com/go-restruct/restruct https://github.com/go-restruct/restruct

tprynn 6y ago

BinData (https://github.com/dmendel/bindata https://github.com/dmendel/bindata) is a Ruby gem for this, basically using a DSL in Ruby to declaratively define binary data formats that can be both read and written.

touisteur 6y ago

Recordflux?

floppy123 6y ago

and still there is no generator that creates efficient and partialy streamed readers/writers for high performance protocols or resource constraint environments (as less pre allocations as possible, zero copy concepts,..., streamed reading/writing, good inline possibility, ...) 100% fix formats with no self reference or for example checksums that sits in front of the checksumd data (no streaming possible)...