6 ms·

The article mentions that Go, OCaml, or a compiled lisp were considered for this project. I wonder why Rust wasn't on that list. It seems to cover every concern

The article mentions that Go, OCaml, or a compiled lisp were considered for this project. I wonder why Rust wasn't on that list. It seems to cover every concern that is raised here except for keyword arguments (which are high on my list of desired Rust features too). I guess maybe they were worried about the lack of GC, but my experience has been that's it's generally quite easy to port code from dynamic languages like Python/JS/PHP to Rust. So I suspect that may have gone into their "Expected problems that weren’t" section if they'd tried it.

mwcampbell 5y ago
Here is what esr wrote about Rust in early 2017: http://esr.ibiblio.org/?p=7294 http://esr.ibiblio.org/?p=7294 http://esr.ibiblio.org/?p=7303 http://esr.ibiblio.org/?p=7303
- kubb 5y ago
  It sounds that it was hard to learn and use for him, that's why he preferred Go and Python. It's a frequent complaint and the step from Python to Go is indeed easier than from Python to Rust. He also had some philosophical concerns about the crate system.
- brandmeyer 5y ago
  > A lot of Rustaceans don’t seem to grasp why, when the question is “where do I get feature X?” the answer “oh, there are 23 crates for that” is objectively terrifying. This is exactly the impression I have gotten from the crates system every time I've looked. It happens for things as foundational as mmap. Also terrifying: The plethora of highly-recommended crates that have not yet committed to a stable API (ie, are still semantically on version 0.X).
  - throwaway894345 5y ago
    I agree, but at the same time things tend to stabilize over time as the community settles around the best packages. The real thing to fear is the relative instability of Rust at the present moment (for certain things, like async), but the overall trajectory seems promising.
  - db48x 5y ago
    I’m not sure that’s a great example. mmap(2) is a swiss army knife of a function. There are many crates that build all kinds of things on top of mmap, and they’re not all interchangeable. Allocators, file io, gpio, actual virtual memory mappings. My view is that when you develop a product, you have to own everything. The users don’t care if the bug comes from code you wrote, or code in a library, or the language’s standard library, or the OS. You have to fix the bug no matter what caused it. It doesn’t matter if the code came from a vendor or a language designer or stack overflow; you are the one responsible for fixing it if something goes wrong. Non est salvatori salvator, etc. With that perspective, I don’t think that 23 crates is terrifying. I’m going to look them all over and either pick one, or write the 24th crate myself. The result is the same either way. I do agree with you about APIs though. It is nice to find a crate where the author has had the confidence to stabilize their API and declare the version to be 1.x instead of 0.x. But at the same time I recognize that getting to that point requires some real software to use the crate, to create the feedback loop. If nobody ever used a 0.x crate, no crate would ever get that feedback.
    - simion314 5y ago
      >My view is that when you develop a product, you have to own everything. The users don’t care if the bug comes from code you wrote, or code in a library, or the language’s standard library, or the OS. You have to fix the bug no matter what caused it. It doesn’t matter if the code came from a vendor or a language designer or stack overflow; you are the one responsible for fixing it if something goes wrong. So then using a programing language that has a big standard library and a more rich and stable ecosystem is a big advantage. If I want to do a network request, parse a json file, parse or format a date or some other trivial thing I prefer to have a good enough built in way to do it then evaluate 15 packages or write it myself. Otherwise you get in a shitty situation where you inherit some projects and it has 20 dependencies with vulnerabilities, 20 dependencies abandoned, some dependencies that are incompatible with some new version of the language/ecosystem causing issue if you would like to upgrade.
      - db48x 5y ago
        Sometimes. Using someone else’s code is only a benefit if it saves time overall, compared to writing it yourself. You have to count both the time to write it and the time to maintain it over however long your product is active. You have to assume that you’ll be maintaining it yourself either way. Also, don’t forget that languages with big standard libraries, like Python or C# or Java, always end up with oodles of stuff in there that is deprecated or incompatible. That’s as bad a sign as anything. But I do agree with you, a lot of the time you have inherited the mess rather than writing it yourself. Not much you can do about it, except to clean it up and then dodge better next time. Going back to Rust specifically, Cargo gives you some tools that make the mess easier to clean. The Rust language also helps, because you can safely use multiple versions of your dependencies, when it turns out to be necessary.
db48x 5y ago
Async wasn’t in yet at the time. He did investigate Rust to a certain extent, but bounced hard. His chosen program didn’t really show off Rust’s strengths, because he started by trying to call select and write essentially the same program that he would have written in C. That’s a pretty painful way to go. Incidentally, I helped with the port to Go, and I spent some time polishing the code, and finding and fixing performance problems. When we were helping GCC convert their SVN repository to Git (a repository with 287k commits, btw), we reduced the memory usage by 50% (from over 250GB to under 128GB), and the run time by quite a lot as well (down to just around 2 hours to read in the SVN repository and convert it to a basic Git repository). Now that we’ve done that work, Reposurgeon spends 50–60% of its cpu time scanning the heap for garbage. There is often garbage to find, but just as often there is not. GC is useful, but for Reposurgeon it has become a bottleneck. My preliminary work on a Rust port shows that it is around 4× faster than the Go version. I personally think that Rust is the future, but I haven’t been able to put as much effort into the port as I would like.
- stouset 5y ago
  Appreciate your thoughtful reply! One question. Regarding this bit: > He did investigate Rust to a certain extent, but bounced hard. His chosen program didn’t really show off Rust’s strengths, because he started by trying to call select and write essentially the same program that he would have written in C. That’s a pretty painful way to go. Is this stating that he was writing the Rust version similarly to how he would have written it in C because that was the most natural way to do it in Rust or simply because it was the first path he went down? In other words, was your point that Rust is a fundamentally poor fit for this problem, or was it that his lack of experience with the language took him down the wrong path? Thanks!
  - db48x 5y ago
    He went down the wrong path, but at the same time Rust didn’t do anything to make async code easier to write. You just had to call libc::select and std::thread::spawn and whatnot, with all of the unsafe blocks that implies. These days there are about 47 different crates that can help you do it, all of them a lot nicer than that.
steveklabnik 5y ago
Two threads from back in the day about this: https://news.ycombinator.com/item?id=13385530 https://news.ycombinator.com/item?id=13385530 https://news.ycombinator.com/item?id=13398068 https://news.ycombinator.com/item?id=13398068
eikenberry 5y ago
Go, OCaml and Lisp (depending on the Lisp) all have one other thing in common.. a fast compile/feedback loop. Rust doesn't have a good story here. This would be my guess as to why (at least in part). Particularly coming from a dynamic languages which have super fast compile times if they compile at all.

mwcampbell 5y ago

Here is what esr wrote about Rust in early 2017: http://esr.ibiblio.org/?p=7294 http://esr.ibiblio.org/?p=7294 http://esr.ibiblio.org/?p=7303 http://esr.ibiblio.org/?p=7303

kubb 5y ago
It sounds that it was hard to learn and use for him, that's why he preferred Go and Python. It's a frequent complaint and the step from Python to Go is indeed easier than from Python to Rust. He also had some philosophical concerns about the crate system.
brandmeyer 5y ago
> A lot of Rustaceans don’t seem to grasp why, when the question is “where do I get feature X?” the answer “oh, there are 23 crates for that” is objectively terrifying. This is exactly the impression I have gotten from the crates system every time I've looked. It happens for things as foundational as mmap. Also terrifying: The plethora of highly-recommended crates that have not yet committed to a stable API (ie, are still semantically on version 0.X).
- throwaway894345 5y ago
  I agree, but at the same time things tend to stabilize over time as the community settles around the best packages. The real thing to fear is the relative instability of Rust at the present moment (for certain things, like async), but the overall trajectory seems promising.
- db48x 5y ago
  I’m not sure that’s a great example. mmap(2) is a swiss army knife of a function. There are many crates that build all kinds of things on top of mmap, and they’re not all interchangeable. Allocators, file io, gpio, actual virtual memory mappings. My view is that when you develop a product, you have to own everything. The users don’t care if the bug comes from code you wrote, or code in a library, or the language’s standard library, or the OS. You have to fix the bug no matter what caused it. It doesn’t matter if the code came from a vendor or a language designer or stack overflow; you are the one responsible for fixing it if something goes wrong. Non est salvatori salvator, etc. With that perspective, I don’t think that 23 crates is terrifying. I’m going to look them all over and either pick one, or write the 24th crate myself. The result is the same either way. I do agree with you about APIs though. It is nice to find a crate where the author has had the confidence to stabilize their API and declare the version to be 1.x instead of 0.x. But at the same time I recognize that getting to that point requires some real software to use the crate, to create the feedback loop. If nobody ever used a 0.x crate, no crate would ever get that feedback.
  - simion314 5y ago
    >My view is that when you develop a product, you have to own everything. The users don’t care if the bug comes from code you wrote, or code in a library, or the language’s standard library, or the OS. You have to fix the bug no matter what caused it. It doesn’t matter if the code came from a vendor or a language designer or stack overflow; you are the one responsible for fixing it if something goes wrong. So then using a programing language that has a big standard library and a more rich and stable ecosystem is a big advantage. If I want to do a network request, parse a json file, parse or format a date or some other trivial thing I prefer to have a good enough built in way to do it then evaluate 15 packages or write it myself. Otherwise you get in a shitty situation where you inherit some projects and it has 20 dependencies with vulnerabilities, 20 dependencies abandoned, some dependencies that are incompatible with some new version of the language/ecosystem causing issue if you would like to upgrade.
    - db48x 5y ago
      Sometimes. Using someone else’s code is only a benefit if it saves time overall, compared to writing it yourself. You have to count both the time to write it and the time to maintain it over however long your product is active. You have to assume that you’ll be maintaining it yourself either way. Also, don’t forget that languages with big standard libraries, like Python or C# or Java, always end up with oodles of stuff in there that is deprecated or incompatible. That’s as bad a sign as anything. But I do agree with you, a lot of the time you have inherited the mess rather than writing it yourself. Not much you can do about it, except to clean it up and then dodge better next time. Going back to Rust specifically, Cargo gives you some tools that make the mess easier to clean. The Rust language also helps, because you can safely use multiple versions of your dependencies, when it turns out to be necessary.

db48x 5y ago

Async wasn’t in yet at the time. He did investigate Rust to a certain extent, but bounced hard. His chosen program didn’t really show off Rust’s strengths, because he started by trying to call select and write essentially the same program that he would have written in C. That’s a pretty painful way to go. Incidentally, I helped with the port to Go, and I spent some time polishing the code, and finding and fixing performance problems. When we were helping GCC convert their SVN repository to Git (a repository with 287k commits, btw), we reduced the memory usage by 50% (from over 250GB to under 128GB), and the run time by quite a lot as well (down to just around 2 hours to read in the SVN repository and convert it to a basic Git repository). Now that we’ve done that work, Reposurgeon spends 50–60% of its cpu time scanning the heap for garbage. There is often garbage to find, but just as often there is not. GC is useful, but for Reposurgeon it has become a bottleneck. My preliminary work on a Rust port shows that it is around 4× faster than the Go version. I personally think that Rust is the future, but I haven’t been able to put as much effort into the port as I would like.

stouset 5y ago
Appreciate your thoughtful reply! One question. Regarding this bit: > He did investigate Rust to a certain extent, but bounced hard. His chosen program didn’t really show off Rust’s strengths, because he started by trying to call select and write essentially the same program that he would have written in C. That’s a pretty painful way to go. Is this stating that he was writing the Rust version similarly to how he would have written it in C because that was the most natural way to do it in Rust or simply because it was the first path he went down? In other words, was your point that Rust is a fundamentally poor fit for this problem, or was it that his lack of experience with the language took him down the wrong path? Thanks!
- db48x 5y ago
  He went down the wrong path, but at the same time Rust didn’t do anything to make async code easier to write. You just had to call libc::select and std::thread::spawn and whatnot, with all of the unsafe blocks that implies. These days there are about 47 different crates that can help you do it, all of them a lot nicer than that.

steveklabnik 5y ago

Two threads from back in the day about this: https://news.ycombinator.com/item?id=13385530 https://news.ycombinator.com/item?id=13385530 https://news.ycombinator.com/item?id=13398068 https://news.ycombinator.com/item?id=13398068

eikenberry 5y ago

Go, OCaml and Lisp (depending on the Lisp) all have one other thing in common.. a fast compile/feedback loop. Rust doesn't have a good story here. This would be my guess as to why (at least in part). Particularly coming from a dynamic languages which have super fast compile times if they compile at all.