6 ms·

Here are a few examples: - Inferred types: you're constantly wondering "what is the type of this variable?". Type errors occur far from where the real type err

Here are a few examples: - Inferred types: you're constantly wondering "what is the type of this variable?". Type errors occur far from where the real type error is. Annotating is complicated because you're often dealing with complicated types (with nested tuples for instance). - Using higher-order functions: often you'll see a function with nested functions in it [1], or some functional list processing function like "unzip" being used [2]. This increases the code complexity a lot ("what does unzip do again?"). - Using monads: yet another distraction from what's really happening. It forces you to write nested functions and doesn't play well with conditions [3]. Also various people use monadic operators way weirder than >>= and all of this becomes really hard to read. - Using functional-friendly data structures: the linker is using immutable lists of optional bytes to represent memory images, because this is idiomatic in OCaml. However this is slow and inefficient as hell. Really, I can't load a 60MiB binary because the linker runs out of memory and is killed by the kernel on machine with 6GiB of RAM. Also, just linking hello-world takes ages. - Designing functional loops: when the loop you want isn't in your stdlib, you first need to create a recursive function, and then make sure all recursive calls are tail calls otherwise you blow up your stack. The result is usually not very fun to read. tl;dr: the C language is small, the OCaml language is not, and when you're dealing with complicated stuff you really don't _need_ more complexity from the language itself. James Mickens explains this (among other things) in his essay. [1]: https://github.com/rems-project/linksem/blob/master/src/link.lem#L820 https://github.com/rems-project/linksem/blob/master/src/link... [2]: https://github.com/rems-project/linksem/blob/master/src/link.lem#L341 https://github.com/rems-project/linksem/blob/master/src/link... [3]: https://github.com/emersion/linksem/blob/54c3a8430e621198653744738933cdbea3151866/src/main_load.lem#L230 https://github.com/emersion/linksem/blob/54c3a8430e621198653... (Note: set the syntax highlighting to "OCaml" when trying to read this source code) EDIT: list formatting EDIT 2: grammar

ernst_klim 8y ago
>the linker is using immutable lists of optional bytes to represent memory images Why would you use linked list instead of Bytes, Bigarrays or Arrays? You are doing it wrong, take a look at BAP's memory representation: https://github.com/BinaryAnalysisPlatform/bap/tree/master/lib/bap_image https://github.com/BinaryAnalysisPlatform/bap/tree/master/li... https://binaryanalysisplatform.github.io/bap/api/master/argot_index.html https://binaryanalysisplatform.github.io/bap/api/master/argo... Looked at your code, damn it's bad, I'd really suggest to dive into BAP sources and documentation a bit to understand how to write OCaml. It feels like you are trying to write in OCaml as in some different language, Haskell maybe, not using modules and functors. Reminds me how J people write C. You could write `fun x y z -> ...` instead of `fun x -> fun y -> fun z -> ...`, it's also better to write signatures (with docstrings) in separate .mli files (much easier to read and generate docs), if you use monads like maybe (why not result) and option, it would be much more convenient to use bind operators instead of `match ... | None -> None`
- emersion 8y ago
  We need optional values for bytes, as a way to say "this byte is undefined", so we can't use Bytes directly. But yes, we're doing it wrong for sure. EDIT: seems like you edited your reply, so let me add this to mine :) >Looked at your code, damn it's bad, I'd really suggest to dive into BAP sources and documentation a bit to understand how to write OCaml. It feels like you are trying to write in OCaml as in some different language, Haskell maybe, not using modules and functors. Reminds me how J people write C. Oh, I also think it's bad don't worry. However we're using a subset of OCaml (Lem) so we don't have things like modules for instance. And no, I really don't want to start using complicated module features and functors, for the reasons I explained above.
  - ernst_klim 8y ago
    Well, you could use arrays of optional bytes, but are use sure it's clever? Maybe it's much better to define an abstract module of type module type Mem = sig type t val get_byte : t -> ~offset:int -> char option end And invent some clever (maybe lazy) implementation using just Bigarray? BAP also has something like this in https://github.com/BinaryAnalysisPlatform/bap/blob/master/lib/bap_image/bap_memory.mli#L22 https://github.com/BinaryAnalysisPlatform/bap/blob/master/li... You could ask about BAP in general on https://discuss.ocaml.org/ https://discuss.ocaml.org/ especially https://discuss.ocaml.org/u/ivg/summary https://discuss.ocaml.org/u/ivg/summary (one of the guys behind BAP, very clever and talented in explaining stuff)
  - dpwm 8y ago
    You can always do something like: type optional_bytes = (Bytes.t * Bytes.t) let get i (data, mask) = match mask.[i] with | '\x00' -> None | _ -> Some data.[i] If you need immutability and you're not changing often, you can use strings. If you're using lists because you're doing all manner of comprehensions, you'd still be better iterating over the Bytes and rebuilding using a Buffer.
  - ernst_klim 8y ago
    >I really don't want to start using complicated module features and functors, for the reasons I explained above. And why is that? Because you really don't want to write ML without modules (and you are using them anyway). Edit: Ah, I see, you have an auto-generated ocaml code, which is used along with the Isabelle. That makes sense (although your code generator is rather strange and there is still no need to use lists for memory representations since Isabelle reasoning about memory chunks shouldn't be that hard). Anyway I wouldn't claim that OCaml is that bad for low level programming basing on such a subset of a language.
dpwm 8y ago
The code you link to is pretty bad and I really cannot understand why. [1] OCaml does the currying for you. Why would this ever need to be done? [2] I have never even thought of putting a let statement in a list before let alone seen one in the wild. Also that looks like a list comprehension, so there is some kind of non-standard syntax going on there. [3] Monads are a pretty good fit for some things, like parsing. > the linker is using immutable lists of optional bytes to represent memory images OCaml has support for bytes in the Bytes module. How do you deal with optional bytes in C? Well, one way is to have a mask array. You could also have a bitmask. You can do the same in OCaml and memory usage will be similar. I've never worked on anything OCaml that I didn't start myself. However I've dived into mature code-bases including the OCaml compiler and they are generally readable and understandable. The examples you highlight, even if unrepresentative of the code-base, are very difficult to comprehend. I think a lot of your problems do seem to stem from this code-base and some of the mysterious things look to me like Haskellisms and attempts to make OCaml into something it's not. Sadly there is a lot of OCaml like this out there. Batteries and Core really haven't helped in this regard. There is definitely an element of understanding OCaml. There isn't much of it to learn, but you need to approach it from the right direction and that can take time. Learn it without Core or Batteries. Even so, I would be very strongly tempted to clean up that code if at all possible.
- emersion 8y ago
  I didn't link the code to say "hey look this code is bad", my intent was just to illustrate that functions inside functions indeed do happen, and unzip is also used, etc. >OCaml has support for bytes in the Bytes module. How do you deal with optional bytes in C? Well, one way is to have a mask array. You could also have a bitmask. You can do the same in OCaml and memory usage will be similar. Yeah, that one is probably more because the original authors have written this code in their perfect mathematical world. But this is also one thing I don't like about systems functional programming: it forces you to use abstractions that don't always make sense.
  - ernst_klim 8y ago
    It was more like they are from a strange haskell realm. Take a look at the ideal mathematical (verified) C compiler written in Coq. I find it neat: https://github.com/AbsInt/CompCert https://github.com/AbsInt/CompCert
lmm 8y ago
> Inferred types: you're constantly wondering "what is the type of this variable?". Your tools should be able to tell you that. Can't you just ask your IDE? In cases where it really is confusing you should probably annotate it. > Annotating is complicated because you're often dealing with complicated types (with nested tuples for instance). Having types for nested tuples doesn't make them any more complicated than they were already. Sometimes people overuse complex tuple structures when they should just declare a record, but that can happen in any language. > Using higher-order functions: often you'll see a function with nested functions in it [1], or some functional list processing function like "unzip" being used [2]. This increases the code complexity a lot ("what does unzip do again?"). In any language you would need to do what unzip does. Having a common version of the function that can be reused in a lot of different places is much better for understandability than writing it out longhand every time. > Using monads: yet another distraction from what's really happening. They should do just the opposite: let you shift the distractions in the monad and keep the main code expressing the essence of the problem. Some things are better expressed in straight-through code that just does the things, but you can do that in OCaml too. > It forces you to write nested functions and doesn't play well with conditions [3]. It plays well with conditions that are already expressed as functions/expressions, which are easier to think about in the first place. In good functional style you can completely forget about "control flow" because things like "if" become just an ordinary case of ordinary functions. > Also various people use monadic operators way weirder than >>= and all of this becomes really hard to read. Naming things is hard, but at least an OCaml "operator" is always just a normal function, so you can always just click through to the definition of it and read exactly what it does. > Using functional-friendly data structures: the linker is using immutable lists of optional bytes to represent memory images, because this is idiomatic in OCaml. However this is slow and inefficient as hell. This sounds like a specific bug rather than a general problem. Certainly there are plenty of datastructures that manage to be both functional-friendly and efficient. > Designing functional loops: when the loop you want isn't in your stdlib, you first need to create a recursive function, and then make sure all recursive calls are tail calls otherwise you blow up your stack. The result is usually not very fun to read. Does OCaml not have a recursion-schemes-like library? Then you can just fold down your datastructure (because a "loop" is never really about looping, it's about handling a datastructure) and you already have your intermediate representation and you can just write the essence of what to do at each step and it's really clean and lovely. > tl;dr: the C language is small, the OCaml language is not Disagree with that. C has all these different kinds of control flow and all these built-in operators, whereas everything you would want to do in OCaml is just plain old functions and expressions.

ernst_klim 8y ago

>the linker is using immutable lists of optional bytes to represent memory images Why would you use linked list instead of Bytes, Bigarrays or Arrays? You are doing it wrong, take a look at BAP's memory representation: https://github.com/BinaryAnalysisPlatform/bap/tree/master/lib/bap_image https://github.com/BinaryAnalysisPlatform/bap/tree/master/li... https://binaryanalysisplatform.github.io/bap/api/master/argot_index.html https://binaryanalysisplatform.github.io/bap/api/master/argo... Looked at your code, damn it's bad, I'd really suggest to dive into BAP sources and documentation a bit to understand how to write OCaml. It feels like you are trying to write in OCaml as in some different language, Haskell maybe, not using modules and functors. Reminds me how J people write C. You could write `fun x y z -> ...` instead of `fun x -> fun y -> fun z -> ...`, it's also better to write signatures (with docstrings) in separate .mli files (much easier to read and generate docs), if you use monads like maybe (why not result) and option, it would be much more convenient to use bind operators instead of `match ... | None -> None`

emersion 8y ago
We need optional values for bytes, as a way to say "this byte is undefined", so we can't use Bytes directly. But yes, we're doing it wrong for sure. EDIT: seems like you edited your reply, so let me add this to mine :) >Looked at your code, damn it's bad, I'd really suggest to dive into BAP sources and documentation a bit to understand how to write OCaml. It feels like you are trying to write in OCaml as in some different language, Haskell maybe, not using modules and functors. Reminds me how J people write C. Oh, I also think it's bad don't worry. However we're using a subset of OCaml (Lem) so we don't have things like modules for instance. And no, I really don't want to start using complicated module features and functors, for the reasons I explained above.
- ernst_klim 8y ago
  Well, you could use arrays of optional bytes, but are use sure it's clever? Maybe it's much better to define an abstract module of type module type Mem = sig type t val get_byte : t -> ~offset:int -> char option end And invent some clever (maybe lazy) implementation using just Bigarray? BAP also has something like this in https://github.com/BinaryAnalysisPlatform/bap/blob/master/lib/bap_image/bap_memory.mli#L22 https://github.com/BinaryAnalysisPlatform/bap/blob/master/li... You could ask about BAP in general on https://discuss.ocaml.org/ https://discuss.ocaml.org/ especially https://discuss.ocaml.org/u/ivg/summary https://discuss.ocaml.org/u/ivg/summary (one of the guys behind BAP, very clever and talented in explaining stuff)
- dpwm 8y ago
  You can always do something like: type optional_bytes = (Bytes.t * Bytes.t) let get i (data, mask) = match mask.[i] with | '\x00' -> None | _ -> Some data.[i] If you need immutability and you're not changing often, you can use strings. If you're using lists because you're doing all manner of comprehensions, you'd still be better iterating over the Bytes and rebuilding using a Buffer.
- ernst_klim 8y ago
  >I really don't want to start using complicated module features and functors, for the reasons I explained above. And why is that? Because you really don't want to write ML without modules (and you are using them anyway). Edit: Ah, I see, you have an auto-generated ocaml code, which is used along with the Isabelle. That makes sense (although your code generator is rather strange and there is still no need to use lists for memory representations since Isabelle reasoning about memory chunks shouldn't be that hard). Anyway I wouldn't claim that OCaml is that bad for low level programming basing on such a subset of a language.

dpwm 8y ago

The code you link to is pretty bad and I really cannot understand why. [1] OCaml does the currying for you. Why would this ever need to be done? [2] I have never even thought of putting a let statement in a list before let alone seen one in the wild. Also that looks like a list comprehension, so there is some kind of non-standard syntax going on there. [3] Monads are a pretty good fit for some things, like parsing. > the linker is using immutable lists of optional bytes to represent memory images OCaml has support for bytes in the Bytes module. How do you deal with optional bytes in C? Well, one way is to have a mask array. You could also have a bitmask. You can do the same in OCaml and memory usage will be similar. I've never worked on anything OCaml that I didn't start myself. However I've dived into mature code-bases including the OCaml compiler and they are generally readable and understandable. The examples you highlight, even if unrepresentative of the code-base, are very difficult to comprehend. I think a lot of your problems do seem to stem from this code-base and some of the mysterious things look to me like Haskellisms and attempts to make OCaml into something it's not. Sadly there is a lot of OCaml like this out there. Batteries and Core really haven't helped in this regard. There is definitely an element of understanding OCaml. There isn't much of it to learn, but you need to approach it from the right direction and that can take time. Learn it without Core or Batteries. Even so, I would be very strongly tempted to clean up that code if at all possible.

emersion 8y ago
I didn't link the code to say "hey look this code is bad", my intent was just to illustrate that functions inside functions indeed do happen, and unzip is also used, etc. >OCaml has support for bytes in the Bytes module. How do you deal with optional bytes in C? Well, one way is to have a mask array. You could also have a bitmask. You can do the same in OCaml and memory usage will be similar. Yeah, that one is probably more because the original authors have written this code in their perfect mathematical world. But this is also one thing I don't like about systems functional programming: it forces you to use abstractions that don't always make sense.
- ernst_klim 8y ago
  It was more like they are from a strange haskell realm. Take a look at the ideal mathematical (verified) C compiler written in Coq. I find it neat: https://github.com/AbsInt/CompCert https://github.com/AbsInt/CompCert

lmm 8y ago

> Inferred types: you're constantly wondering "what is the type of this variable?". Your tools should be able to tell you that. Can't you just ask your IDE? In cases where it really is confusing you should probably annotate it. > Annotating is complicated because you're often dealing with complicated types (with nested tuples for instance). Having types for nested tuples doesn't make them any more complicated than they were already. Sometimes people overuse complex tuple structures when they should just declare a record, but that can happen in any language. > Using higher-order functions: often you'll see a function with nested functions in it [1], or some functional list processing function like "unzip" being used [2]. This increases the code complexity a lot ("what does unzip do again?"). In any language you would need to do what unzip does. Having a common version of the function that can be reused in a lot of different places is much better for understandability than writing it out longhand every time. > Using monads: yet another distraction from what's really happening. They should do just the opposite: let you shift the distractions in the monad and keep the main code expressing the essence of the problem. Some things are better expressed in straight-through code that just does the things, but you can do that in OCaml too. > It forces you to write nested functions and doesn't play well with conditions [3]. It plays well with conditions that are already expressed as functions/expressions, which are easier to think about in the first place. In good functional style you can completely forget about "control flow" because things like "if" become just an ordinary case of ordinary functions. > Also various people use monadic operators way weirder than >>= and all of this becomes really hard to read. Naming things is hard, but at least an OCaml "operator" is always just a normal function, so you can always just click through to the definition of it and read exactly what it does. > Using functional-friendly data structures: the linker is using immutable lists of optional bytes to represent memory images, because this is idiomatic in OCaml. However this is slow and inefficient as hell. This sounds like a specific bug rather than a general problem. Certainly there are plenty of datastructures that manage to be both functional-friendly and efficient. > Designing functional loops: when the loop you want isn't in your stdlib, you first need to create a recursive function, and then make sure all recursive calls are tail calls otherwise you blow up your stack. The result is usually not very fun to read. Does OCaml not have a recursion-schemes-like library? Then you can just fold down your datastructure (because a "loop" is never really about looping, it's about handling a datastructure) and you already have your intermediate representation and you can just write the essence of what to do at each step and it's really clean and lovely. > tl;dr: the C language is small, the OCaml language is not Disagree with that. C has all these different kinds of control flow and all these built-in operators, whereas everything you would want to do in OCaml is just plain old functions and expressions.