4 ms·
Honestly, I dislike plaintext formats a lot. It is more accessible because it’s human readable. But, this only extends to humans who happen to speak the languag
by jchw 6y ago
Honestly, I dislike plaintext formats a lot. It is more accessible because it’s human readable. But, this only extends to humans who happen to speak the language the protocol uses for keywords. While it’s not a huge ask, I still suggest this is mostly not that interesting of a benefit.
Parsing and emitting plaintext formats, meanwhile, is a rabbit hole. It’s human readable which makes you tempted to make it human writable. Should you accept extraneous whitespace? Tabs vs spaces? Terminating new line? Unix or DOS line endings? Case sensitivity? Unicode normalized forms? Etc.
Binary data may seem less accessible, but I blame the libraries. There’s tons of easy (if inefficient) ways to parse text. You can use string.split, atoi and scanf in your language of choice. What is there for binary?
In Go, the encoding/binary package actually implements something really cool. A simple reflection-based mechanism that can read and write binary data into a structure in a defined and simple way.
lunixbochs extended this to struc[1], which adds additional tags for advanced reading and writing of binary structures, including variable length structures. I went further and maybe a bit off into the deep end with Restruct[2], a similar concept but with a lot more features, designed specifically so I could handle advanced structures quickly.
The end result is that I can define some Go structs with integers, strings, byte arrays and corresponding tags, and be able to serialize and deserialize from those structures to their corresponding binary representation. For an overdone demo of what you could do with Restruct for example, see this (incomplete) PNG demo: https://github.com/go-restruct/restruct/blob/master/formats/png/png.go https://github.com/go-restruct/restruct/blob/master/formats/... (It is mainly incomplete because I had moved focus to develop a codegen for restruct, to improve runtime performance, although such work has since stalled.)
Most of my inspiration for newer advanced features like expressions comes from Kaitai Struct[3], an excellent project I have also contributed to a bit. I learned about Kaitai after having written Restruct thanks to a friend, though its lack of write capabilities has lead me to continue using Restruct for many hacks, such as some tools I wrote that modified FL Studio Project files.
[1]: https://pkg.go.dev/github.com/lunixbochs/struc https://pkg.go.dev/github.com/lunixbochs/struc
[2]: https://pkg.go.dev/github.com/go-restruct/restruct https://pkg.go.dev/github.com/go-restruct/restruct
[3]: https://kaitai.io https://kaitai.io