4 ms·

There's also Wuffs, a safe and fast programming language made by Google specifically for decoding and encoding file formats https://github.com/google/wuffs http

There's also Wuffs, a safe and fast programming language made by Google specifically for decoding and encoding file formats https://github.com/google/wuffs https://github.com/google/wuffs Paired with C FFI available in most languages, this seems like the nicer solution. It's simpler than generating code for a bunch of high level languages, and more performant

layer8 5y ago
Not for managed environments like client-side JS, JVM, .NET, …
- jmgao 5y ago
  This appears to just allow you to parse binary formats to the represented fields. (Not that that's not extremely useful, doing this in managed languages is generally a giant pain in the ass!) wuffs is much more powerful: it's essentially a safe C dialect that compiles to C, that lets you write an entire codec and know that there aren't any overflows.
eesmith 5y ago
How much of a future should I expect for Wuffs? The linked-to page says: "Version 0.2. The API and ABI aren't stabilized yet. The compiler undoubtedly has bugs." There are not many recent commits, and mostly by one developer.
felixr 5y ago
This is "made by a Googler"; that is not the same as "Made by Google". This is not an official Google product, it is just code that happens to be owned by Google
- jmgao 5y ago
  That's probably just an artifact of how the author initially went through the open sourcing process. It is code that is written and maintained by a Googler as his day job, that's used in skia (i.e. chromium, android, etc.) today for GIF decoding: https://source.chromium.org/chromium/chromium/src/+/main:third_party/skia/src/codec/SkWuffsCodec.cpp https://source.chromium.org/chromium/chromium/src/+/main:thi...
saagarjha 5y ago
They’re designed to solve different problems, really. Kaitai is often used when trying to represent a binary format from a reverse engineer’s perspective, to allow for sharing this work and using it from whatever other tooling you have. Wuffs is more “production grade” and is intended to provide performant, safe parsing of known formats where one would typically use a language like C or C++.
- nigeltao 5y ago
  I agree that it's similar but ultimately different problems. Copy/pasting from https://github.com/google/wuffs/blob/main/doc/related-work.md https://github.com/google/wuffs/blob/main/doc/related-work.m... gives: > Kaitai Struct is in a similar space, generating safe parsers for multiple target programming languages from one declarative specification. Again, Wuffs differs in that it is a complete (and performant) end to end implementation, not just for the structured parts of a file format. Repeating a point in the previous paragraph, the difficulty in decoding the GIF format isn't in the regularly-expressible part of the format, it's in the LZW compression. Kaitai's GIF parser returns the compressed LZW data as an opaque blob.
MaxBarraclough 5y ago
> Wuffs the Library is available as transpiled C code > However, unlike hand-written C, Wuffs the Language is safe with respect to buffer overflows, integer arithmetic overflows and null pointer dereferences. A key difference between Wuffs and other memory-safe languages is that all such checks are done at compile time, not at run time. If it compiles, it is safe, with respect to those three bug classes. Sounds a lot like ZZ. [0] It also sounds much like SPARK Ada, except for transpiling to C. [0] https://github.com/zetzit/zz https://github.com/zetzit/zz

layer8 5y ago

Not for managed environments like client-side JS, JVM, .NET, …

jmgao 5y ago
This appears to just allow you to parse binary formats to the represented fields. (Not that that's not extremely useful, doing this in managed languages is generally a giant pain in the ass!) wuffs is much more powerful: it's essentially a safe C dialect that compiles to C, that lets you write an entire codec and know that there aren't any overflows.

eesmith 5y ago

How much of a future should I expect for Wuffs? The linked-to page says: "Version 0.2. The API and ABI aren't stabilized yet. The compiler undoubtedly has bugs." There are not many recent commits, and mostly by one developer.

felixr 5y ago

This is "made by a Googler"; that is not the same as "Made by Google". This is not an official Google product, it is just code that happens to be owned by Google

jmgao 5y ago
That's probably just an artifact of how the author initially went through the open sourcing process. It is code that is written and maintained by a Googler as his day job, that's used in skia (i.e. chromium, android, etc.) today for GIF decoding: https://source.chromium.org/chromium/chromium/src/+/main:third_party/skia/src/codec/SkWuffsCodec.cpp https://source.chromium.org/chromium/chromium/src/+/main:thi...

saagarjha 5y ago

They’re designed to solve different problems, really. Kaitai is often used when trying to represent a binary format from a reverse engineer’s perspective, to allow for sharing this work and using it from whatever other tooling you have. Wuffs is more “production grade” and is intended to provide performant, safe parsing of known formats where one would typically use a language like C or C++.

nigeltao 5y ago
I agree that it's similar but ultimately different problems. Copy/pasting from https://github.com/google/wuffs/blob/main/doc/related-work.md https://github.com/google/wuffs/blob/main/doc/related-work.m... gives: > Kaitai Struct is in a similar space, generating safe parsers for multiple target programming languages from one declarative specification. Again, Wuffs differs in that it is a complete (and performant) end to end implementation, not just for the structured parts of a file format. Repeating a point in the previous paragraph, the difficulty in decoding the GIF format isn't in the regularly-expressible part of the format, it's in the LZW compression. Kaitai's GIF parser returns the compressed LZW data as an opaque blob.

MaxBarraclough 5y ago

> Wuffs the Library is available as transpiled C code > However, unlike hand-written C, Wuffs the Language is safe with respect to buffer overflows, integer arithmetic overflows and null pointer dereferences. A key difference between Wuffs and other memory-safe languages is that all such checks are done at compile time, not at run time. If it compiles, it is safe, with respect to those three bug classes. Sounds a lot like ZZ. [0] It also sounds much like SPARK Ada, except for transpiling to C. [0] https://github.com/zetzit/zz https://github.com/zetzit/zz