3 ms·

Lua does these very well too in my opinion. No messing around with bit-wise operators or masks - it's just an array of bits!

by rusk 8y ago

masklinn 8y ago
Could you give an example? E.g. if you have a 3 bits signed integer V followed by two 1-bit flags (F1 and F2), followed by a 3-bits unsigned integer L, followed by binary data D of length L bytes, followed by a little-endian signed integer I over two bytes, how do you parse it? IIRC in Erlang it's something along the lines of <<V:3/signed, F1:1, F2:1, L:3, D:L/binary, I:16/signed-little>>
- rusk 8y ago
  I went to look this up... I think actually the features I describe are more features of Wireshark which provides a Lua API, which is what confused me. That Erlang syntax reminds of the python 'struct' [0], except it's generating the unpacking logic at compile time, am I right? Yes that is indeed very cool - it doesn't even seem that hard to do I'd wonder why other languages don't have a similar facility! [0] https://docs.python.org/2/library/struct.html https://docs.python.org/2/library/struct.html
  - rusk 8y ago
    > I'd wonder why other languages don't have a similar facility Actually this is the kind of thing lex/yacc/Bison, javacc etc. do ... except I think only for text ...
  - rwmj 8y ago
    I wrote ocaml-bitstring (based on Erlang) and it also works by generating the code to handle the bits at compile time. It's fully general too - eg. if you describe a structure of 1 bit followed by an array of bytes then it will add the necessary bit shifts automatically when reading the bytes. Or if you describe a struct where everything is aligned then at compile time it will generate the most efficient code. You can even have variable length bitfields (a huge headache when implementing, but it does deliver the best possible code based on what it can prove at compile time). [1] https://rwmj.wordpress.com/2010/02/03/on-the-awesomeness-of-ocaml-bitstring/ https://rwmj.wordpress.com/2010/02/03/on-the-awesomeness-of-...
  - masklinn 8y ago
    > That Erlang syntax reminds of the python 'struct' [0], except it's generating the unpacking logic at compile time, am I right? Kinda, however there are a few superior items to the bit syntax: * In Erlang, the bindings are in the expression itself (whether packing or unpacking) which makes the correspondance easier. * Because the bindings are embedded and set left to right, it becomes possible to use one binding as parameter to the next pattern e.g. `L:3, D:L/binary` will first parse 3 bits as an unsigned integer L, then extract the next L bytes as a binary substring. * Also Erlang's bit syntax can do sub-byte matching, I don't think python's struct can. * And Erlang's bit syntax is much clearer and orthogonal (modulo knowing the — convenient — defaults) e.g. where extracting a little-endian 32-bit signed integer with struct is "<i" with Erlang it's :32/integer-signed-little (32 units, integer, signed, little-endian; the default unit size for integers is the bit). More verbose, but also completely explicit and more flexible, and very easy to change.
    - rusk 8y ago
      That is wonderful ... I’ve been looking for something like this for years. Don’t suppose you know off the top of your head are there any Erlang compilers for java and if they’re any good?

Lua does these very well too in my opinion. No messing around with bit-wise operators or masks - it's just an array of bits!

by rusk 8y ago

Lua does these very well too in my opinion. No messing around with bit-wise operators or masks - it's just an array of bits!

masklinn 8y ago

Could you give an example? E.g. if you have a 3 bits signed integer V followed by two 1-bit flags (F1 and F2), followed by a 3-bits unsigned integer L, followed by binary data D of length L bytes, followed by a little-endian signed integer I over two bytes, how do you parse it? IIRC in Erlang it's something along the lines of <<V:3/signed, F1:1, F2:1, L:3, D:L/binary, I:16/signed-little>>

rusk 8y ago
I went to look this up... I think actually the features I describe are more features of Wireshark which provides a Lua API, which is what confused me. That Erlang syntax reminds of the python 'struct' [0], except it's generating the unpacking logic at compile time, am I right? Yes that is indeed very cool - it doesn't even seem that hard to do I'd wonder why other languages don't have a similar facility! [0] https://docs.python.org/2/library/struct.html https://docs.python.org/2/library/struct.html
- rusk 8y ago
  > I'd wonder why other languages don't have a similar facility Actually this is the kind of thing lex/yacc/Bison, javacc etc. do ... except I think only for text ...
- rwmj 8y ago
  I wrote ocaml-bitstring (based on Erlang) and it also works by generating the code to handle the bits at compile time. It's fully general too - eg. if you describe a structure of 1 bit followed by an array of bytes then it will add the necessary bit shifts automatically when reading the bytes. Or if you describe a struct where everything is aligned then at compile time it will generate the most efficient code. You can even have variable length bitfields (a huge headache when implementing, but it does deliver the best possible code based on what it can prove at compile time). [1] https://rwmj.wordpress.com/2010/02/03/on-the-awesomeness-of-ocaml-bitstring/ https://rwmj.wordpress.com/2010/02/03/on-the-awesomeness-of-...
- masklinn 8y ago
  > That Erlang syntax reminds of the python 'struct' [0], except it's generating the unpacking logic at compile time, am I right? Kinda, however there are a few superior items to the bit syntax: * In Erlang, the bindings are in the expression itself (whether packing or unpacking) which makes the correspondance easier. * Because the bindings are embedded and set left to right, it becomes possible to use one binding as parameter to the next pattern e.g. `L:3, D:L/binary` will first parse 3 bits as an unsigned integer L, then extract the next L bytes as a binary substring. * Also Erlang's bit syntax can do sub-byte matching, I don't think python's struct can. * And Erlang's bit syntax is much clearer and orthogonal (modulo knowing the — convenient — defaults) e.g. where extracting a little-endian 32-bit signed integer with struct is "<i" with Erlang it's :32/integer-signed-little (32 units, integer, signed, little-endian; the default unit size for integers is the bit). More verbose, but also completely explicit and more flexible, and very easy to change.
  - rusk 8y ago
    That is wonderful ... I’ve been looking for something like this for years. Don’t suppose you know off the top of your head are there any Erlang compilers for java and if they’re any good?