24 ms·

Replacing Python

173 points by antics 13y ago

hartror 13y ago
> A major benefit of OCaml and Haskell is the ease of refactoring. In Python, once the code is working it’s best not to change things, in case you break something that isn’t covered by the unit-tests. In OCaml and Haskell, you can rename a function, delete old code or change a data structure and rely on the compiler to check that everything’s still OK. These sort of statement always bothers me when I encounter it. If you're not testing the code you are refactoring how do you know it still works or even worked in the first place? Static typing gives you useful things (with a trade off), not having to write tests isn't one of them.
- davidp 13y ago
  Static typing gives you useful things (with a trade off), not having to write tests isn't one of them. I disagree; static typing is, effectively, having an automatic set of tests automatically run for (by the compiler) you that you don't have to run/create/maintain yourself. As someone who's moved from statically-typed languages to Python (2.x), I can't count how many times I've made errors that "should/could have been caught for me by the compiler", if there were such a thing in Python. It slows me down in nontrivial ways. It would be good if we could use 3.x-series annotations to achieve much of the same thing, but the world hasn't gone 3.x yet. Your point about the value of unit testing in general is well-taken, however.
  - hartror 13y ago
    I'm not saying static typing/compilation doesn't give you anything or that it doesn't catch errors for you. My point is that static typing/compilation means you don't have to worry about code coverage with your tests is a false assertion.
- tikhonj 13y ago
  He never said anything about not writing unit tests. Rather, his claim is that--even with unit tests--refactoring is not safe in Python et al; in Haskell and OCaml, by contrast, many of the same actions are guaranteed safe by the type system. Static types do not mean you don't have to write unit tests. But they do mean you can write fewer. (And, looking at libraries like QuickCheck, a static type system can make writing tests easier.) Quite a bit of the safety you get in Haskell especially comes down to controlling effects. Mutable state implicitly couples all the code in a given scope--unless you've read and understood all of it, distinct parts might have effects on each other that you're unaware of. In Haskell, on the other hand, this is impossible, so refactoring like extracting variables and reordering your code can be entirely safe even if you haven't looked at exactly what the code does. The refactoring actions are guaranteed not to change the semantics of the code by the language itself. The best way to think about it is that refactoring becomes a purely syntactic action. You just remember a few rules akin to algebra, and you can rewrite Haskell code in a bunch of different ways all preserving the original meaning. Regardless of what that meaning really is. This also extends to library code--most good libraries come with algebraic laws, which ensure you can rewrite their code in logical ways. Once you learn these laws, you can start doing things like changing multiple passes over a datastructure into one with the same confidence. If you make a change that could break things, the type system will often help you find every place that does break. This extends beyond just type mismatches: Haskell also ensures that you consider every possible case in your functions; if you forget an option, it will give you a warning. This means it's safe to add a new alternative to an existing type: you will get a warning everywhere you haven't considered this new option. I've found this to have a profound effect on how I program. With Haskell, I actually follow the rule of making any code I visit look better than before, simply because refactoring has very little mental overhead. I can move things around liberally, break them into multiple modules, condense them into fewer functions and even change the types, knowing that any mistakes I make will be caught by the type system.
  - njharman 13y ago
    > his claim is that--even with unit tests--refactoring is not safe in Python et al; Well that claim is False. I and thousands of other Python developers disprove it daily. Just because they can't refactor dynamic languages doesn't me we can't. Something not-dynamic is probably the language for them. And Python is the language for us. In other words there is far more variability between developers than there is between languages. Find the language(s) that work for you and quit believing they are the languages for everyone.
- opminion 13y ago
  If you're not testing the code you are refactoring how do you know it still works or even worked in the first place? There are other ways, different than testing, to convince yourself and others that a program does what it is meant to do. They complement testing. You use them already while constructing the deterministic parts of your program: much of what the machine does for you is predictable, and good languages and libraries are designed to make it so. In fact, tests won't tell you that the program works, only that it doesn't fail the test cases. You then use the predictable aspects of your domain to convince yourself that the program works if it doesn't fail those test cases. In the same way that programmatically renaming a variable does not usually warrant the writing of a test case, many forms of automatic refactoring are theoretically guaranteed to not break your program. If they involve type renames, you might be able to deduce that any resulting errors will be caught by the type checker. (Please don't take this as an argument against testing, but against always requiring test coverage)
  - deleted 13y ago
    [deleted]
- lmm 13y ago
  Of course it's possible to write non-working code in a typed language. But with a good type system you don't have to write the code in a way that can be wrong (or at least, that can be wrong in a way that unit tests would help with - if you've misunderstood the requirements then nothing can save you).
nothingspecial 13y ago
The Xen hypervisor relies on a curious mix of Python and OCaml.
jzwinck 13y ago
Perhaps the title should be "Why Python is good enough." Between this post and the preceding one (http://roscidus.com/blog/blog/2013/06/09/choosing-a-python-replacement-for-0install/ http://roscidus.com/blog/blog/2013/06/09/choosing-a-python-r...), the takeaway for me is that the author didn't find sufficient benefit in any of the candidate languages to justify switching an existing product--perhaps not even a new product. It's true that Python can leave you with hidden crash bugs in unusual code paths, but the language brings so many benefits that it's easy to forgive this. And there seemed to be a vague concern about performance at the beginning of the article, but I didn't see anything substantive in that regard. So yay, let's all just keep using Python. Back to work.
- tikhonj 13y ago
  Except in the end, the author switches to OCaml[1]. So Python might be good enough, but he certainly found OCaml better. And since when are we willing to settle for good enough? [1]: http://roscidus.com/blog/blog/2013/09/28/ocaml-objects/ http://roscidus.com/blog/blog/2013/09/28/ocaml-objects/ (In fact, the author writes that he's basically willing to move to OCaml in the conclusion of this post.)
  - jzwinck 13y ago
    "Good enough" the way I'm using it is nothing to be ashamed of. It is very difficult to find computing systems that are "good enough." Here are some of the motivating factors identified by the author at the outset of the search for a better language: - Canonical’s Colin Watson is worried about Python’s performance on mobile phones. - Marco Jez and Dave Abrahams proposed a C++ version. - Bastian Eicher would like a .NET version (though IronPython might work here). It doesn't seem like OCaml effectively addresses the above very well. On the other hand, here are some items the author liked about Python which may now be lost: - Widely known and easy to learn. - A large standard library. - You only need to ship source code (interpreted). - Can run inside a Java or .NET VM (using Jython/IronPython). - All current 0install contributors know it. - The current code is all Python and is well-tested. Of course the last item is the most alarming: see http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html (2000), Brooks' Mythical Man Month (1975) for the Second System Effect, etc.
tghw 13y ago
The first Python example throws up a big red flag: manually parsing command line arguments instead of using argparse. Why reinvent the wheel when there is a standard library to handle it? Likewise, asserting that "For storing general records, Python provides a choice of classes and tuples" completely ignores one of Python's fastest and most powerful data types: dictionaries. Lastly, in get_value, the nest of ifs is unnecessary and makes the code seem more complex than it really is.
- Luyt 13y ago
  In his defense, the author makes the following disclaimer: "As before, note that I’m a beginner in these languages."
  - avalaunch 13y ago
    He was referring to all of the other languages. In his previous article, he writes: "I’m not an expert in these languages (except Python)."
- Erwin 13y ago
  Over the last decade, there's probably been half a dozen "standard" command line parsing tools. getopt being the classic, then we had optparse and that got deprecated, then we got argparse. And there there's e.g. twisted.usage and the pretty cool docopt. In this case the author wants to parse "program <foo | bar | baz> [optional args]" and does that in a few lines of code. You seriously think that's a "big red flag" ? Do you also believe the author does not (despite having created the zero install tool 10 years ago) know Python has dictionaries? This is a blog post where the author evaluates a number of languages/environment to replace his particular Python needs -- do you think that him not using a new standard library module invalidates his finding?
  - mjn 13y ago
    I like this article on that subject, which as it happens uses argument-parsing as the running example: http://www.yosefk.com/blog/redundancy-vs-dependencies-which-is-worse.html http://www.yosefk.com/blog/redundancy-vs-dependencies-which-...
tikhonj 13y ago
I think the important insight from these articles is not which language the author ends up using (OCaml[1]), but rather which languages he's managed to rule out by now. In particular, I think it's good advice to avoid both ATS (however much I like dependent types) and Go, both for completely different reasons. You probably wouldn't want to use Rust in the short term either. [1]: http://roscidus.com/blog/blog/2013/09/28/ocaml-objects/ http://roscidus.com/blog/blog/2013/09/28/ocaml-objects/ ATS makes everything more difficult than it's worth except for some very specific use cases. Unlike some of the other languages like Haskell and OCaml, I don't think this is just because it's different; rather, it's because ATS is simply so much more demanding. It gives you extremely solid static guarantees and high performance, but it's a tool that really sacrifices expressiveness and programmability to get there. Haskell and OCaml may be difficult to learn, but ATS is actually difficult to use--a very different concept. Go, on the other hand, is neither difficult to learn nor difficult to use. But, more generally, Go has very little to commend itself and--compared to the other options--quite a few shortcomings. If you're willing to put in a little bit of effort to learn something new, there are a whole bunch of options which are simply better. And you should be willing to put in the effort: your programming language is your single most important tool; it affects not only how you write your code and how you maintain it but even how you think. So it seems extremely shortsighted to choose a language because it's easy to learn and similar to what you already know! Rust is awesome, but it's simply not ready yet. This is widely acknowledged by everyone in the project, and is part of their policy of open development. Once it is ready, it will be a very good choice for certain domains.
- pron 13y ago
  I think you may be overestimating the importance of the programming language over the whole programming environment. I think it is the programming environment that is the most important tool – not just the language. There are often many requirements that might lead you to choose a possibly inferior language because it lives in an altogether superior environment for your needs. I have been writing large, server side software for many years. These are long-running applications that require superb performance as well as first-class monitoring tools. So, for me, one requirement for a language is that it runs on the JVM. The JVM gives you superb performance as well as unparalleled monitoring tools (made even better by the inclusion of the flight-recorder and Java Mission Control in the latest version of the JVM); you also get a vast ecosystem with a huge selection of very high quality libraries (and some other important features like dynamic linking) – all that before you even choose a particular language. You can also buy commercial support for every single component you're using if you need it. On the other hand, if you're writing a command-line tool that needs a very fast startup time, or if your RAM is very constrained, then the JVM might be a bad choice for you, which would rule out the JVM languages no matter how good they are. My point is that there are more important issues to consider than the mere merits of the language itself. Also, you're putting a lot of emphasis on expressivity, while downplaying the importance of a short learning curve. I think the two are of similar importance. It's been my experience that programs written in some of the more expressive languages are actually harder to maintain than code written in the easy-to-learn ones. The reason seems to be that programmers use the expressivity of the former (which seems to be of a particular nature or natures: functional construct, higher-order types, meta-programming) to model their own thought-process, which may be hard to replicate for someone else maintaining the code. The easier-to-learn languages usually work at a lower level of abstraction, which can sometimes be easier for others to follow because its a well-understood common denominator. Now, I'm not saying that worse is better, or that more "primitive" languages are better, or even that one should pick them over more expressive ones. I'm just saying that, especially when working in a large team or writing code that would need to be maintained for a long time, there are other issues to consider.
gmantastic 13y ago
Cython might also be worth considering. It lets you add static type declarations to Python and produces compiled code. You could avoid a big-bang rewrite, migrating the slower operations first. I don't know whether it would meet all your cross-platform requirements though. http://cython.org/ http://cython.org/
- wikiburner 13y ago
  Isn't Cython a solution that frequently requires significant code changes (refactoring to C-like code) beyond adding typing?
  - gmantastic 13y ago
    It's selectively adding C-like type declarations to Python code speed up the slow bits. C code is then generated and compiled into a Python extension module. This is similar to adding type information to Common Lisp code to allow the compiler to optimise it. http://docs.cython.org/src/quickstart/cythonize.html http://docs.cython.org/src/quickstart/cythonize.html
    - wikiburner 13y ago
      My impression was that you sometimes have to refactor your Python to make the code more C-like. From what I have heard generators aren't supported, and there are other situations where you need to make your loops (and other elements of your code) more C-like. That's not the case? You just add typing to your python and you're good to go?
      - gmantastic 13y ago
        There's some info on semantic differences here: http://docs.cython.org/src/userguide/limitations.html http://docs.cython.org/src/userguide/limitations.html - it doesn't bring up much, and the design goal is full language compatibility. I've only played with it so far, but it's case of optionally adding type info, and also writing some distutils scaffolding to say how to build the module as a cython extension. This is then callable from Python via the regular import mechanism.
dchichkov 13y ago
> You still have classes, objects, functions, mutable data and low-level access to the OS, all with an easy and concise syntax, but you gain type checking, much better data structures and a huge amount of speed for no additional effort. Why aren’t more people using it? I think it is a community issue actually. Not the language. It is just not converging. Just think of how many distinct OCaml code styles you have seen. Some use hardcore math notation, some not; some use recursions everywhere; some use imperative syntax; some use pseudo-OOP approach. Some even try to fork the language syntax. And a result. Well. Even with the language itself providing easy and concise syntax, ocaml code is not very readable. And not very popular.
dancecodes 13y ago
python is nice
dscrd 13y ago
I like articles like this one, but this doesn't really bode well for 0install.
- talex5 13y ago
  Any particular reason why? We just released 0install 2.4, which has around 10,000 lines of OCaml. It seems to be working well so far, but there are bound to be a few bugs... we can always use more testers!
ssadler 13y ago
I find it suspicious that Java or Scala didn't make the list. Java may not be sexy but it checks many boxes... And I suspect that Scala would have been a serious contender in brevity too.
- mercurial 13y ago
  C# was excluded for being too slow to start. JVM-based languages would not fare any better.
  - spongle 13y ago
    I can't say I've had that problem myself. They are quite startup heavy but typical c# console apps start up on an ssd based system in ~150ms and 700ms on rust disks. If you're calling something thousands of times, fast process startup times are good. This isn't the model windows uses though which is the primary target of c#.
    - steveklabnik 13y ago
      In the tests, his OCaML implementation takes 7ms, and his Python one takes 64 or 109 ms. So even on your SSD, that's 50% to an order of magnitude too slow, just for startup.
      - spongle 13y ago
        Do you really notice that though?
        steveklabnik 13y ago
        I cannot remember exactly, but in the discussions about page load times translating into revenue, 100ms was the number being tossed around, IIRC. I certainly notice any time I boot something up that requires the JVM, I refuse to use the CLR, so I can't tell you much about my own experience with that.
hershel 13y ago
Anybody knows or have any reasonable guesses when Rust will be ready?
- chrismorgan 13y ago
  I hope some time next year, but see recent discussion of it at https://news.ycombinator.com/item?id=6454455 https://news.ycombinator.com/item?id=6454455.
_random_ 13y ago
Plus he will be able to use F# easily since it is derived from OCaml.
deleted 13y ago
[deleted]
njharman 13y ago
I do not understand at all author's statement that OCaml datastructures are big win over Python's let {cache; config} = b in print_endline (String.concat "," cache); print_endline (String.concat "," config) ;; vs, actually I'm not even sure what the ocaml is attempting, some variation of print '%s,' % b.cache print '%s,' % b.config Which, in Python, if your printing more than a few should be print ',\n'.join((b.config, b.cache, b.data)) Or if want all fields and they are in correct order print ',\n'.join(b) >> The syntax [of NamedTuples] isn’t great, though, and you can’t do pattern matching on the names, only by remembering the order: Other than misuse of term "pattern matching", that statement is true and is trivially overcome with two line function, one line lambda, or once and for all by subclassing NamedTuple. Here's the function variant: def GimmieThing(keyword args in any order) return ThingNamedTuple(args in correct order)
- anaphor 13y ago
  How is it a misuse of the term "pattern matching" ?
  - njharman 13y ago
    Generically http://en.wikipedia.org/wiki/Pattern_matching http://en.wikipedia.org/wiki/Pattern_matching And as for a language / call semantic look at Erlang for what actual pattern matching looks like.
    - anaphor 13y ago
      That doesn't contradict the use of pattern matching in OCaml.
      - njharman 13y ago
        Both the OA and myself are talking about Python.
brandonbloom 13y ago
It seems pretty clear to me that Julia is gunning for Python's sweet spot. Worth checking out. http://julialang.org/ http://julialang.org/
abraxasz 13y ago
A quick comment concerning Haskell. I've been learning it on and off, with some help from my roommate who's a Haskell genius. The way he writes Haskell always amazes me. He starts with a very straightforward verbose version, then constantly refactors it (on the fly, it's not a separate step), abstracting stuff out in typeclasses and monads until it seems that most of the code is just monads and typeclasses definitions, and only a couple lines that seem to actually do something. I always joke around saying that if your Haskell code doesn't have typeclasses and monads, you're doing something wrong. By which I mean that unlike other languages, there's a huge, huge, huge gap between writing great haskell, and regular haskell, with a very steep learning curve.
- ijl 13y ago
  Interesting. I wonder about maintainability if the writer's "obvious" version is only there to be refactored out. Can you still read the intent when he's done?
  - abraxasz 13y ago
    I can't, by a long shot: I don't know Haskell deeply enough. But I think he can read his code without problem.
deleted 13y ago
[deleted]
IgorPartola 13y ago
Wonder if he looked at Nimrod. On paper that is a language that would both fit his needs and would not be a far leap from his current Python implementation.
- dmytrish 13y ago
  When I looked at Nimrod it seemed very raw to me: REPL malfunctions and crashes, the whole language seems to be more an enthusiast effort than a production-quality tool. Rust is very raw too, but it already looks pretty solid.
  - dom96 13y ago
    Nimrod by default compiles to C. Its REPL is just an extra feature which as of right now is not stable because few people use it. It for example doesn't even support the FFI currently. As for whether Nimrod is production quality, I would say yes. The compiler is written in Nimrod which is impressive in itself. I have written many projects in it already and even though the compiler is still not at 1.0 the projects continue to build with no (or very little) errors between new compiler versions. Examples of my nimrod projects: an IDE (https://github.com/nimrod-code/aporia https://github.com/nimrod-code/aporia), a build farm (https://github.com/nimrod-code/nimbuild https://github.com/nimrod-code/nimbuild) and a web framework (https://github.com/dom96/jester https://github.com/dom96/jester). The Nimrod forum (http://forum.nimrod-code.org http://forum.nimrod-code.org) is also written in Nimrod.
anaphor 13y ago
You could've avoided a lot of the "case of blah" nonsense in the Haskell code by using Data.Maybe and Data.Either, just sayin'.
vph 13y ago
Why is it necessary to replace Python for this project? And what are the objectives for the replacement? It's not clear what the author's problems and objectives are.
- __float 13y ago
  This is all explained in the first part he so kindly links to in the third word ;) http://roscidus.com/blog/blog/2013/06/09/choosing-a-python-replacement-for-0install/ http://roscidus.com/blog/blog/2013/06/09/choosing-a-python-r...

173 points by antics 13y ago

> A major benefit of OCaml and Haskell is the ease of refactoring. In Python, once the code is working it’s best not to change things, in case you break something that isn’t covered by the unit-tests. In OCaml and Haskell, you can rename a function, delete old code or change a data structure and rely on the compiler to check that everything’s still OK. These sort of statement always bothers me when I encounter it. If you're not testing the code you are refactoring how do you know it still works or even worked in the first place? Static typing gives you useful things (with a trade off), not having to write tests isn't one of them.

davidp 13y ago
Static typing gives you useful things (with a trade off), not having to write tests isn't one of them. I disagree; static typing is, effectively, having an automatic set of tests automatically run for (by the compiler) you that you don't have to run/create/maintain yourself. As someone who's moved from statically-typed languages to Python (2.x), I can't count how many times I've made errors that "should/could have been caught for me by the compiler", if there were such a thing in Python. It slows me down in nontrivial ways. It would be good if we could use 3.x-series annotations to achieve much of the same thing, but the world hasn't gone 3.x yet. Your point about the value of unit testing in general is well-taken, however.
- hartror 13y ago
  I'm not saying static typing/compilation doesn't give you anything or that it doesn't catch errors for you. My point is that static typing/compilation means you don't have to worry about code coverage with your tests is a false assertion.
tikhonj 13y ago
He never said anything about not writing unit tests. Rather, his claim is that--even with unit tests--refactoring is not safe in Python et al; in Haskell and OCaml, by contrast, many of the same actions are guaranteed safe by the type system. Static types do not mean you don't have to write unit tests. But they do mean you can write fewer. (And, looking at libraries like QuickCheck, a static type system can make writing tests easier.) Quite a bit of the safety you get in Haskell especially comes down to controlling effects. Mutable state implicitly couples all the code in a given scope--unless you've read and understood all of it, distinct parts might have effects on each other that you're unaware of. In Haskell, on the other hand, this is impossible, so refactoring like extracting variables and reordering your code can be entirely safe even if you haven't looked at exactly what the code does. The refactoring actions are guaranteed not to change the semantics of the code by the language itself. The best way to think about it is that refactoring becomes a purely syntactic action. You just remember a few rules akin to algebra, and you can rewrite Haskell code in a bunch of different ways all preserving the original meaning. Regardless of what that meaning really is. This also extends to library code--most good libraries come with algebraic laws, which ensure you can rewrite their code in logical ways. Once you learn these laws, you can start doing things like changing multiple passes over a datastructure into one with the same confidence. If you make a change that could break things, the type system will often help you find every place that does break. This extends beyond just type mismatches: Haskell also ensures that you consider every possible case in your functions; if you forget an option, it will give you a warning. This means it's safe to add a new alternative to an existing type: you will get a warning everywhere you haven't considered this new option. I've found this to have a profound effect on how I program. With Haskell, I actually follow the rule of making any code I visit look better than before, simply because refactoring has very little mental overhead. I can move things around liberally, break them into multiple modules, condense them into fewer functions and even change the types, knowing that any mistakes I make will be caught by the type system.
- njharman 13y ago
  > his claim is that--even with unit tests--refactoring is not safe in Python et al; Well that claim is False. I and thousands of other Python developers disprove it daily. Just because they can't refactor dynamic languages doesn't me we can't. Something not-dynamic is probably the language for them. And Python is the language for us. In other words there is far more variability between developers than there is between languages. Find the language(s) that work for you and quit believing they are the languages for everyone.
opminion 13y ago
If you're not testing the code you are refactoring how do you know it still works or even worked in the first place? There are other ways, different than testing, to convince yourself and others that a program does what it is meant to do. They complement testing. You use them already while constructing the deterministic parts of your program: much of what the machine does for you is predictable, and good languages and libraries are designed to make it so. In fact, tests won't tell you that the program works, only that it doesn't fail the test cases. You then use the predictable aspects of your domain to convince yourself that the program works if it doesn't fail those test cases. In the same way that programmatically renaming a variable does not usually warrant the writing of a test case, many forms of automatic refactoring are theoretically guaranteed to not break your program. If they involve type renames, you might be able to deduce that any resulting errors will be caught by the type checker. (Please don't take this as an argument against testing, but against always requiring test coverage)
- deleted 13y ago
  [deleted]
lmm 13y ago
Of course it's possible to write non-working code in a typed language. But with a good type system you don't have to write the code in a way that can be wrong (or at least, that can be wrong in a way that unit tests would help with - if you've misunderstood the requirements then nothing can save you).

nothingspecial 13y ago

The Xen hypervisor relies on a curious mix of Python and OCaml.

jzwinck 13y ago

Perhaps the title should be "Why Python is good enough." Between this post and the preceding one (http://roscidus.com/blog/blog/2013/06/09/choosing-a-python-replacement-for-0install/ http://roscidus.com/blog/blog/2013/06/09/choosing-a-python-r...), the takeaway for me is that the author didn't find sufficient benefit in any of the candidate languages to justify switching an existing product--perhaps not even a new product. It's true that Python can leave you with hidden crash bugs in unusual code paths, but the language brings so many benefits that it's easy to forgive this. And there seemed to be a vague concern about performance at the beginning of the article, but I didn't see anything substantive in that regard. So yay, let's all just keep using Python. Back to work.

tikhonj 13y ago
Except in the end, the author switches to OCaml[1]. So Python might be good enough, but he certainly found OCaml better. And since when are we willing to settle for good enough? [1]: http://roscidus.com/blog/blog/2013/09/28/ocaml-objects/ http://roscidus.com/blog/blog/2013/09/28/ocaml-objects/ (In fact, the author writes that he's basically willing to move to OCaml in the conclusion of this post.)
- jzwinck 13y ago
  "Good enough" the way I'm using it is nothing to be ashamed of. It is very difficult to find computing systems that are "good enough." Here are some of the motivating factors identified by the author at the outset of the search for a better language: - Canonical’s Colin Watson is worried about Python’s performance on mobile phones. - Marco Jez and Dave Abrahams proposed a C++ version. - Bastian Eicher would like a .NET version (though IronPython might work here). It doesn't seem like OCaml effectively addresses the above very well. On the other hand, here are some items the author liked about Python which may now be lost: - Widely known and easy to learn. - A large standard library. - You only need to ship source code (interpreted). - Can run inside a Java or .NET VM (using Jython/IronPython). - All current 0install contributors know it. - The current code is all Python and is well-tested. Of course the last item is the most alarming: see http://www.joelonsoftware.com/articles/fog0000000069.html http://www.joelonsoftware.com/articles/fog0000000069.html (2000), Brooks' Mythical Man Month (1975) for the Second System Effect, etc.

tghw 13y ago

The first Python example throws up a big red flag: manually parsing command line arguments instead of using argparse. Why reinvent the wheel when there is a standard library to handle it? Likewise, asserting that "For storing general records, Python provides a choice of classes and tuples" completely ignores one of Python's fastest and most powerful data types: dictionaries. Lastly, in get_value, the nest of ifs is unnecessary and makes the code seem more complex than it really is.

Luyt 13y ago
In his defense, the author makes the following disclaimer: "As before, note that I’m a beginner in these languages."
- avalaunch 13y ago
  He was referring to all of the other languages. In his previous article, he writes: "I’m not an expert in these languages (except Python)."
Erwin 13y ago
Over the last decade, there's probably been half a dozen "standard" command line parsing tools. getopt being the classic, then we had optparse and that got deprecated, then we got argparse. And there there's e.g. twisted.usage and the pretty cool docopt. In this case the author wants to parse "program <foo | bar | baz> [optional args]" and does that in a few lines of code. You seriously think that's a "big red flag" ? Do you also believe the author does not (despite having created the zero install tool 10 years ago) know Python has dictionaries? This is a blog post where the author evaluates a number of languages/environment to replace his particular Python needs -- do you think that him not using a new standard library module invalidates his finding?
- mjn 13y ago
  I like this article on that subject, which as it happens uses argument-parsing as the running example: http://www.yosefk.com/blog/redundancy-vs-dependencies-which-is-worse.html http://www.yosefk.com/blog/redundancy-vs-dependencies-which-...

tikhonj 13y ago

I think the important insight from these articles is not which language the author ends up using (OCaml[1]), but rather which languages he's managed to rule out by now. In particular, I think it's good advice to avoid both ATS (however much I like dependent types) and Go, both for completely different reasons. You probably wouldn't want to use Rust in the short term either. [1]: http://roscidus.com/blog/blog/2013/09/28/ocaml-objects/ http://roscidus.com/blog/blog/2013/09/28/ocaml-objects/ ATS makes everything more difficult than it's worth except for some very specific use cases. Unlike some of the other languages like Haskell and OCaml, I don't think this is just because it's different; rather, it's because ATS is simply so much more demanding. It gives you extremely solid static guarantees and high performance, but it's a tool that really sacrifices expressiveness and programmability to get there. Haskell and OCaml may be difficult to learn, but ATS is actually difficult to use--a very different concept. Go, on the other hand, is neither difficult to learn nor difficult to use. But, more generally, Go has very little to commend itself and--compared to the other options--quite a few shortcomings. If you're willing to put in a little bit of effort to learn something new, there are a whole bunch of options which are simply better. And you should be willing to put in the effort: your programming language is your single most important tool; it affects not only how you write your code and how you maintain it but even how you think. So it seems extremely shortsighted to choose a language because it's easy to learn and similar to what you already know! Rust is awesome, but it's simply not ready yet. This is widely acknowledged by everyone in the project, and is part of their policy of open development. Once it is ready, it will be a very good choice for certain domains.

pron 13y ago
I think you may be overestimating the importance of the programming language over the whole programming environment. I think it is the programming environment that is the most important tool – not just the language. There are often many requirements that might lead you to choose a possibly inferior language because it lives in an altogether superior environment for your needs. I have been writing large, server side software for many years. These are long-running applications that require superb performance as well as first-class monitoring tools. So, for me, one requirement for a language is that it runs on the JVM. The JVM gives you superb performance as well as unparalleled monitoring tools (made even better by the inclusion of the flight-recorder and Java Mission Control in the latest version of the JVM); you also get a vast ecosystem with a huge selection of very high quality libraries (and some other important features like dynamic linking) – all that before you even choose a particular language. You can also buy commercial support for every single component you're using if you need it. On the other hand, if you're writing a command-line tool that needs a very fast startup time, or if your RAM is very constrained, then the JVM might be a bad choice for you, which would rule out the JVM languages no matter how good they are. My point is that there are more important issues to consider than the mere merits of the language itself. Also, you're putting a lot of emphasis on expressivity, while downplaying the importance of a short learning curve. I think the two are of similar importance. It's been my experience that programs written in some of the more expressive languages are actually harder to maintain than code written in the easy-to-learn ones. The reason seems to be that programmers use the expressivity of the former (which seems to be of a particular nature or natures: functional construct, higher-order types, meta-programming) to model their own thought-process, which may be hard to replicate for someone else maintaining the code. The easier-to-learn languages usually work at a lower level of abstraction, which can sometimes be easier for others to follow because its a well-understood common denominator. Now, I'm not saying that worse is better, or that more "primitive" languages are better, or even that one should pick them over more expressive ones. I'm just saying that, especially when working in a large team or writing code that would need to be maintained for a long time, there are other issues to consider.

gmantastic 13y ago

Cython might also be worth considering. It lets you add static type declarations to Python and produces compiled code. You could avoid a big-bang rewrite, migrating the slower operations first. I don't know whether it would meet all your cross-platform requirements though. http://cython.org/ http://cython.org/

wikiburner 13y ago
Isn't Cython a solution that frequently requires significant code changes (refactoring to C-like code) beyond adding typing?
- gmantastic 13y ago
  It's selectively adding C-like type declarations to Python code speed up the slow bits. C code is then generated and compiled into a Python extension module. This is similar to adding type information to Common Lisp code to allow the compiler to optimise it. http://docs.cython.org/src/quickstart/cythonize.html http://docs.cython.org/src/quickstart/cythonize.html
  - wikiburner 13y ago
    My impression was that you sometimes have to refactor your Python to make the code more C-like. From what I have heard generators aren't supported, and there are other situations where you need to make your loops (and other elements of your code) more C-like. That's not the case? You just add typing to your python and you're good to go?
    - gmantastic 13y ago
      There's some info on semantic differences here: http://docs.cython.org/src/userguide/limitations.html http://docs.cython.org/src/userguide/limitations.html - it doesn't bring up much, and the design goal is full language compatibility. I've only played with it so far, but it's case of optionally adding type info, and also writing some distutils scaffolding to say how to build the module as a cython extension. This is then callable from Python via the regular import mechanism.

dchichkov 13y ago

> You still have classes, objects, functions, mutable data and low-level access to the OS, all with an easy and concise syntax, but you gain type checking, much better data structures and a huge amount of speed for no additional effort. Why aren’t more people using it? I think it is a community issue actually. Not the language. It is just not converging. Just think of how many distinct OCaml code styles you have seen. Some use hardcore math notation, some not; some use recursions everywhere; some use imperative syntax; some use pseudo-OOP approach. Some even try to fork the language syntax. And a result. Well. Even with the language itself providing easy and concise syntax, ocaml code is not very readable. And not very popular.

dancecodes 13y ago

python is nice

dscrd 13y ago

I like articles like this one, but this doesn't really bode well for 0install.

talex5 13y ago
Any particular reason why? We just released 0install 2.4, which has around 10,000 lines of OCaml. It seems to be working well so far, but there are bound to be a few bugs... we can always use more testers!

ssadler 13y ago

I find it suspicious that Java or Scala didn't make the list. Java may not be sexy but it checks many boxes... And I suspect that Scala would have been a serious contender in brevity too.

mercurial 13y ago
C# was excluded for being too slow to start. JVM-based languages would not fare any better.
- spongle 13y ago
  I can't say I've had that problem myself. They are quite startup heavy but typical c# console apps start up on an ssd based system in ~150ms and 700ms on rust disks. If you're calling something thousands of times, fast process startup times are good. This isn't the model windows uses though which is the primary target of c#.
  - steveklabnik 13y ago
    In the tests, his OCaML implementation takes 7ms, and his Python one takes 64 or 109 ms. So even on your SSD, that's 50% to an order of magnitude too slow, just for startup.
    - spongle 13y ago
      Do you really notice that though?
      - steveklabnik 13y ago
        I cannot remember exactly, but in the discussions about page load times translating into revenue, 100ms was the number being tossed around, IIRC. I certainly notice any time I boot something up that requires the JVM, I refuse to use the CLR, so I can't tell you much about my own experience with that.

hershel 13y ago

Anybody knows or have any reasonable guesses when Rust will be ready?

chrismorgan 13y ago
I hope some time next year, but see recent discussion of it at https://news.ycombinator.com/item?id=6454455 https://news.ycombinator.com/item?id=6454455.

_random_ 13y ago

Plus he will be able to use F# easily since it is derived from OCaml.

deleted 13y ago

[deleted]

njharman 13y ago

I do not understand at all author's statement that OCaml datastructures are big win over Python's let {cache; config} = b in print_endline (String.concat "," cache); print_endline (String.concat "," config) ;; vs, actually I'm not even sure what the ocaml is attempting, some variation of print '%s,' % b.cache print '%s,' % b.config Which, in Python, if your printing more than a few should be print ',\n'.join((b.config, b.cache, b.data)) Or if want all fields and they are in correct order print ',\n'.join(b) >> The syntax [of NamedTuples] isn’t great, though, and you can’t do pattern matching on the names, only by remembering the order: Other than misuse of term "pattern matching", that statement is true and is trivially overcome with two line function, one line lambda, or once and for all by subclassing NamedTuple. Here's the function variant: def GimmieThing(keyword args in any order) return ThingNamedTuple(args in correct order)

anaphor 13y ago
How is it a misuse of the term "pattern matching" ?
- njharman 13y ago
  Generically http://en.wikipedia.org/wiki/Pattern_matching http://en.wikipedia.org/wiki/Pattern_matching And as for a language / call semantic look at Erlang for what actual pattern matching looks like.
  - anaphor 13y ago
    That doesn't contradict the use of pattern matching in OCaml.
    - njharman 13y ago
      Both the OA and myself are talking about Python.

brandonbloom 13y ago

It seems pretty clear to me that Julia is gunning for Python's sweet spot. Worth checking out. http://julialang.org/ http://julialang.org/

abraxasz 13y ago

A quick comment concerning Haskell. I've been learning it on and off, with some help from my roommate who's a Haskell genius. The way he writes Haskell always amazes me. He starts with a very straightforward verbose version, then constantly refactors it (on the fly, it's not a separate step), abstracting stuff out in typeclasses and monads until it seems that most of the code is just monads and typeclasses definitions, and only a couple lines that seem to actually do something. I always joke around saying that if your Haskell code doesn't have typeclasses and monads, you're doing something wrong. By which I mean that unlike other languages, there's a huge, huge, huge gap between writing great haskell, and regular haskell, with a very steep learning curve.

ijl 13y ago
Interesting. I wonder about maintainability if the writer's "obvious" version is only there to be refactored out. Can you still read the intent when he's done?
- abraxasz 13y ago
  I can't, by a long shot: I don't know Haskell deeply enough. But I think he can read his code without problem.

deleted 13y ago

[deleted]

IgorPartola 13y ago

Wonder if he looked at Nimrod. On paper that is a language that would both fit his needs and would not be a far leap from his current Python implementation.

dmytrish 13y ago
When I looked at Nimrod it seemed very raw to me: REPL malfunctions and crashes, the whole language seems to be more an enthusiast effort than a production-quality tool. Rust is very raw too, but it already looks pretty solid.
- dom96 13y ago
  Nimrod by default compiles to C. Its REPL is just an extra feature which as of right now is not stable because few people use it. It for example doesn't even support the FFI currently. As for whether Nimrod is production quality, I would say yes. The compiler is written in Nimrod which is impressive in itself. I have written many projects in it already and even though the compiler is still not at 1.0 the projects continue to build with no (or very little) errors between new compiler versions. Examples of my nimrod projects: an IDE (https://github.com/nimrod-code/aporia https://github.com/nimrod-code/aporia), a build farm (https://github.com/nimrod-code/nimbuild https://github.com/nimrod-code/nimbuild) and a web framework (https://github.com/dom96/jester https://github.com/dom96/jester). The Nimrod forum (http://forum.nimrod-code.org http://forum.nimrod-code.org) is also written in Nimrod.

anaphor 13y ago

You could've avoided a lot of the "case of blah" nonsense in the Haskell code by using Data.Maybe and Data.Either, just sayin'.

vph 13y ago

Why is it necessary to replace Python for this project? And what are the objectives for the replacement? It's not clear what the author's problems and objectives are.

__float 13y ago
This is all explained in the first part he so kindly links to in the third word ;) http://roscidus.com/blog/blog/2013/06/09/choosing-a-python-replacement-for-0install/ http://roscidus.com/blog/blog/2013/06/09/choosing-a-python-r...