5 ms·

I’m a big fan of Kaitai Struct, to the point where I’ve even contributed a small bit of improvements to its Go support, and I use it in a handful of small proje

by jchw 5y ago

I’m a big fan of Kaitai Struct, to the point where I’ve even contributed a small bit of improvements to its Go support, and I use it in a handful of small projects. It’s indispensable for spelunking blobs of binary data. I’ve also taken some inspiration with a Go library I wrote, restruct: https://github.com/go-restruct/restruct https://github.com/go-restruct/restruct … which is a bit like Go’s JSON encoding/decoding library, but with kaitai-like annotations for binary encoding. Check the PNG example to see some of what can be done with it: https://github.com/go-restruct/restruct/blob/master/formats/png/png.go https://github.com/go-restruct/restruct/blob/master/formats/... The main advantage it has over Kaitai at the moment is that it can serialize, which is still not a feature of mainline Kaitai Struct, though it will hopefully be some day.

billfruit 5y ago
I do find the documentation/packaging rather unclear. For example the website mentions an IDE, but when you download it, there is no IDE in the downloaded package(thats what I remember). Exactly details I'm not remembering, but the Java version documentation was rather unclear, had to work with a bit before understanding how it worked. Actually could not figure out how to parse a binary file/stream containing heterogeneous binary structures from reading the docs. I suppose it is possible?
tgv 5y ago
It's some time ago I had to parse a binary file, but your go struct annotations look great. Adding streaming (un)marshalling could be a killer feature for some.
c7DJTLrn 5y ago
I wrote a similar solution based on reflection for a game server I wrote in Go. It's probably too early to be worrying this about but reflection is apparently slow. I also found it difficult to write reflection code that was robust. The problem I'm stuck on now is that since my "Unmarshal" function returns interface{}, I have to type assert for every type of packet in the protocol. There's probably thousands in total. I need some magic that can automatically Unmarshal to a concrete type and pass it to an appropriate handler. gRPC's codegen does this for you. Still working out how best to do this in the constraints of Go.
- majewsky 5y ago
  You can type-cast from interface{} into any other interface, so something like this might work: https://go.dev/play/p/bhrXr8-V7Gc https://go.dev/play/p/bhrXr8-V7Gc

by jchw 5y ago

billfruit 5y ago

I do find the documentation/packaging rather unclear. For example the website mentions an IDE, but when you download it, there is no IDE in the downloaded package(thats what I remember). Exactly details I'm not remembering, but the Java version documentation was rather unclear, had to work with a bit before understanding how it worked. Actually could not figure out how to parse a binary file/stream containing heterogeneous binary structures from reading the docs. I suppose it is possible?

tgv 5y ago

It's some time ago I had to parse a binary file, but your go struct annotations look great. Adding streaming (un)marshalling could be a killer feature for some.

c7DJTLrn 5y ago

I wrote a similar solution based on reflection for a game server I wrote in Go. It's probably too early to be worrying this about but reflection is apparently slow. I also found it difficult to write reflection code that was robust. The problem I'm stuck on now is that since my "Unmarshal" function returns interface{}, I have to type assert for every type of packet in the protocol. There's probably thousands in total. I need some magic that can automatically Unmarshal to a concrete type and pass it to an appropriate handler. gRPC's codegen does this for you. Still working out how best to do this in the constraints of Go.

majewsky 5y ago
You can type-cast from interface{} into any other interface, so something like this might work: https://go.dev/play/p/bhrXr8-V7Gc https://go.dev/play/p/bhrXr8-V7Gc