6 ms·

OCaml: The Bugs So Far

76 points by ctoth 12y ago

amirmc 12y ago
Also worth reading the follow up - OCaml: What You Gain [1] and the overall retrospective [2]. If you search HN for the titles you can find discussions about those. The author will be also be speaking at OCaml 2014 this year [3]. [1] http://roscidus.com/blog/blog/2014/02/13/ocaml-what-you-gain/ http://roscidus.com/blog/blog/2014/02/13/ocaml-what-you-gain... [2] http://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-retrospective/ http://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-ret... [3] http://ocaml.org/meetings/ocaml/2014/program.html http://ocaml.org/meetings/ocaml/2014/program.html
moron4hire 12y ago
"Fails to Start on Windows" sounds more like your lack of experience with Windows than a 3rd party issue.
DanielBMarkham 12y ago
Amazing that the bug count was so small. The only one that caught my attention was garbage collecting your functional reactive code. The solution was to use global variables. That's a huge red flag. Overall though, this is a pretty positive report for committing some 30KLOC. Amazing, really.
illumen 12y ago
Quite an interesting series of articles :) The Unix.waitpid failing because the author ported the python code, and did not realise that the python version did something nice which avoided one of the types of bugs. Quite a number of the bugs appear to be because of things that the python code did nicely without the author realising it. This shows how the comparison between a second system, and the first system is not really fair. This is because much of the code is already tested in the first system. It also seems to suggest that prototyping in a language like python as a first system is a great way to improve the quality of the second system. Finish it first quickly, and then improve the quality. It also shows the danger of not using the protective parts of the first system in the second system. For example, the advanced integers in python avoided some bugs for the author, which manifested in the second system. The python integers automatically handle overflow, whereas the OCaml ones are 31bit on 32bit systems. OCaml uses non-deterministic garbage collection, whereas python mostly uses reference counting (in CPython). This is another case where the first system was safer in one aspect. Perhaps OCaml could consider taking on some of these features from python to improve safety. Glue code when libraries are wrapped is another area where these bugs seem to happen. Some of the GTK bugs, and system call bugs are ones caused by badly wrapped functions. With more stable and well tested wrapped code, the smaller the likelihood of bugs there will be. Whilst automating wrapper generation might stop some bugs, I am not convinced, after having used buggy automatically generated wrappers many times where the manually written ones are far superior. The post also proves that a number of bugs were not caught by testing or static type checking. This makes me wonder if the author has 100% unit test coverage or not? Some of the bugs should have been caught by unit testing, and others could only have been caught by functional testing. Some of the platform bugs could have been caught with continuous integration testing on those platforms.
- fzltrp 12y ago
  > OCaml uses non-deterministic garbage collection, whereas python mostly uses reference counting (in CPython). This is another case where the first system was safer in one aspect. How is reference counting safer?
  - illumen 12y ago
    It is deterministic. Meaning you know how it will behave each time the program is run. Whereas garbage collection can behave differently each time the program is run.
    - avsm 12y ago
      GC vs reference counting has nothing to do with determinism. For the same inputs, the same allocation profiles and same GC events will happen. Note that CPython's reference counting is by means automatically simpler than OCaml's incremental GC. CPython has heuristics to detect cycles (which normally cause memory leaks in a pure reference counting system), as well as a number of heuristics to optimise performance based on locality and temporal properties of object allocation. All of these contribute to making the CPython runtime less predictable (but faster). The OCaml GC is extremely predictable and has very few tricks. It's explained in one chapter of Real World OCaml: https://realworldocaml.org/v1/en/html/understanding-the-garbage-collector.html https://realworldocaml.org/v1/en/html/understanding-the-garb...
      - illumen 12y ago
        When is the GC activated exactly? When there is not enough memory available. This is the problem, because depending on the data the program is processing this changes. So if you run your program with different inputs (why would you run it if the inputs are always the same?) then the GC will be activated in entirely different parts of your program! This makes it non deterministic because objects in OCaml can destruct at different times in your program depending on the inputs. With reference counting, they are destructed at exactly the time when they have no references. With OCaml GC, the code can run in a different order depending on the GC variables and depending on the program inputs. The bug in the original article was caused because the GC was finalising it at a time the author did not consider. Probably because the code assumed reference counting behaviour of finalising when there are no references. "OCaml's automatic memory management guarantees that a value will eventually be freed when it's no longer in use, either via the GC sweeping it or the program terminating" So, it can guarantee your object will be freed... but only when the program terminates? That's not a very strong promise.

76 points by ctoth 12y ago

Also worth reading the follow up - OCaml: What You Gain [1] and the overall retrospective [2]. If you search HN for the titles you can find discussions about those. The author will be also be speaking at OCaml 2014 this year [3]. [1] http://roscidus.com/blog/blog/2014/02/13/ocaml-what-you-gain/ http://roscidus.com/blog/blog/2014/02/13/ocaml-what-you-gain... [2] http://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-retrospective/ http://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-ret... [3] http://ocaml.org/meetings/ocaml/2014/program.html http://ocaml.org/meetings/ocaml/2014/program.html

moron4hire 12y ago

"Fails to Start on Windows" sounds more like your lack of experience with Windows than a 3rd party issue.

DanielBMarkham 12y ago

Amazing that the bug count was so small. The only one that caught my attention was garbage collecting your functional reactive code. The solution was to use global variables. That's a huge red flag. Overall though, this is a pretty positive report for committing some 30KLOC. Amazing, really.

illumen 12y ago

Quite an interesting series of articles :) The Unix.waitpid failing because the author ported the python code, and did not realise that the python version did something nice which avoided one of the types of bugs. Quite a number of the bugs appear to be because of things that the python code did nicely without the author realising it. This shows how the comparison between a second system, and the first system is not really fair. This is because much of the code is already tested in the first system. It also seems to suggest that prototyping in a language like python as a first system is a great way to improve the quality of the second system. Finish it first quickly, and then improve the quality. It also shows the danger of not using the protective parts of the first system in the second system. For example, the advanced integers in python avoided some bugs for the author, which manifested in the second system. The python integers automatically handle overflow, whereas the OCaml ones are 31bit on 32bit systems. OCaml uses non-deterministic garbage collection, whereas python mostly uses reference counting (in CPython). This is another case where the first system was safer in one aspect. Perhaps OCaml could consider taking on some of these features from python to improve safety. Glue code when libraries are wrapped is another area where these bugs seem to happen. Some of the GTK bugs, and system call bugs are ones caused by badly wrapped functions. With more stable and well tested wrapped code, the smaller the likelihood of bugs there will be. Whilst automating wrapper generation might stop some bugs, I am not convinced, after having used buggy automatically generated wrappers many times where the manually written ones are far superior. The post also proves that a number of bugs were not caught by testing or static type checking. This makes me wonder if the author has 100% unit test coverage or not? Some of the bugs should have been caught by unit testing, and others could only have been caught by functional testing. Some of the platform bugs could have been caught with continuous integration testing on those platforms.

fzltrp 12y ago
> OCaml uses non-deterministic garbage collection, whereas python mostly uses reference counting (in CPython). This is another case where the first system was safer in one aspect. How is reference counting safer?
- illumen 12y ago
  It is deterministic. Meaning you know how it will behave each time the program is run. Whereas garbage collection can behave differently each time the program is run.
  - avsm 12y ago
    GC vs reference counting has nothing to do with determinism. For the same inputs, the same allocation profiles and same GC events will happen. Note that CPython's reference counting is by means automatically simpler than OCaml's incremental GC. CPython has heuristics to detect cycles (which normally cause memory leaks in a pure reference counting system), as well as a number of heuristics to optimise performance based on locality and temporal properties of object allocation. All of these contribute to making the CPython runtime less predictable (but faster). The OCaml GC is extremely predictable and has very few tricks. It's explained in one chapter of Real World OCaml: https://realworldocaml.org/v1/en/html/understanding-the-garbage-collector.html https://realworldocaml.org/v1/en/html/understanding-the-garb...
    - illumen 12y ago
      When is the GC activated exactly? When there is not enough memory available. This is the problem, because depending on the data the program is processing this changes. So if you run your program with different inputs (why would you run it if the inputs are always the same?) then the GC will be activated in entirely different parts of your program! This makes it non deterministic because objects in OCaml can destruct at different times in your program depending on the inputs. With reference counting, they are destructed at exactly the time when they have no references. With OCaml GC, the code can run in a different order depending on the GC variables and depending on the program inputs. The bug in the original article was caused because the GC was finalising it at a time the author did not consider. Probably because the code assumed reference counting behaviour of finalising when there are no references. "OCaml's automatic memory management guarantees that a value will eventually be freed when it's no longer in use, either via the GC sweeping it or the program terminating" So, it can guarantee your object will be freed... but only when the program terminates? That's not a very strong promise.