7 ms·

Also Rust is risky to parallelize: you can get deadlocks. I don't get the obsession of parallel code in low level languages by the way. If you have an architec

Also Rust is risky to parallelize: you can get deadlocks. I don't get the obsession of parallel code in low level languages by the way. If you have an architecture where you can afford real parallelism you can afford higher level languages anyway. In embedded applications you don't usually have the possibility to have parallel code, and even in low level software (for example the classical UNIX utilities), for simplicity and solidity using a single thread is really fine. Threads also are not really as portable as they seem, different operating systems have different way to manage threads, or even don't supports thread at all.

yholio 6y ago
You would go to parallelism precisely on those platforms where simpler performance fixes (changing some data structures or implementing limited sections in a fast language) are insuficient. Eficient parallelization of an existing algorithm is a major undertaking.
eru 6y ago
> In embedded applications you don't usually have the possibility to have parallel code, and even in low level software (for example the classical UNIX utilities), for simplicity and solidity using a single thread is really fine. Depends on which of the classic utilities you are talking about. Many of them are typically IO bound. You might not get much out of throwing more CPU at them.
- ben0x539 6y ago
  ripgrep? :)
  - burntsushi 6y ago
    Indeed. Many of the optimizations ripgrep (and the underlying regex engine) does only show benefits if the data you're searching is already in memory.[1] The same is true of GNU grep. This is because searching data that's in your OS's file cache is an exceptionally common case. [1] - I'm assuming commodity SSD in the range of a few hundred MB/s read speed. This will likely become less true as the prevalence of faster SSDs increases (low single digit GB/s).
  - tsegratis 6y ago
    ripgrep is based on re2, a c library I would guess it contains more c than rust code... But what I love about this article is its lack of hype. It makes clear arguments both ways and all of them I can get behind Hype doesn't help Edit: To all my downvoters; I anticipated you :) With love and best wishes
    - burntsushi 6y ago
      No it's not. Its regex library is written in Rust, but was inspired by RE2. It shares no code with RE2. (And RE2 is a C++ library, not C.) Off the top of my head, the only C code in ripgrep is optional integration with PCRE2. In addition to whatever libc is being used on POSIX platforms. Everything else is pure Rust.
      - tsegratis 6y ago
        Ah, thanks burntsushi, I believe you are the ripgrep author even? Great work btw. Ripgrep is the best ... I will have to restrict my comment to just LLVM being a larger, c++, dependency ... Just angling for more downvotes ;) Thanks for the reply
        burntsushi 6y ago
        To be clear, ripgrep has no runtime dependency on any LLVM or C++ library. rustc does.
        tsegratis 6y ago
        The interesting thing here is that rust has good threading and fantastic crates I played with making a regex library in rust. Which, as per RE2 design involves constructing graphs and glueing them together as the regex is traversed This requires a cycle catching gc, or, just a preallocated arena... It was my first foray into rust and felt I would need to be hitting into unsafe, which I wasn't ready for. Array indexing might decompose into an arena, but syntactically just a bit messier (imho) Would be interesting to see how the RE2 does it in rust (didn't know that) I like how the article shows both sides of the fence, it makes me realize: I get a lot of optimizations from ptr stuffing in c. But sometimes we should lay down the good, for the better
fulafel 6y ago
A lot of modern embedded hw are running operating systems providing threads (such as Linux) and multi-core CPUs.
ReactiveJelly 6y ago
Deadlocks are unique to Rust, eh?
burntsushi 6y ago
This is a bad take. ripgrep, to my knowledge, cannot be written in a higher level language without becoming a lot slower.[1] And yet, if I removed its use of parallelism by default, there will be a significantly degraded user experience by virtue of it being a lot slower. This isn't an "obsession." It's engineering. [1] - I make this claim loosely. Absence of evidence isn't evidence of absence and all that. But if I saw ripgrep implemented in, say, Python and it matched speed in the majority of cases, I would learn something.
- pjmlp 6y ago
  Python isn't really something I would even think as possible example, Common Lisp, D, Nim, Swift, most likely.
  - cb321 6y ago
    Cython with all the appropriate cdef type declarations can match C and so might also do it. Not sure Cython exactly counts as "Python"..it's more a superset/dialect { and I also doubt such a port would hold many lessons for @burntsushi, but it bore noting. }
  - burntsushi 6y ago
    So? I said, "higher level language." I didn't say, "Python specifically." I would guess D could do it. I don't know enough about Nim or Swift. I would learn something if Common Lisp did it. I'd also learn something if Haskell or Go did it.
    - cb321 6y ago
      I am not trying to contradict anyone here, but any language mature enough to have an impl/way to not have arbitrary performance ceilings needs access to inline assembly/SIMD. Cython/Nim/SBCL can all do that..probably Haskell..Not so sure about Go or Swift. Anyway, many languages can respond well to optimization effort. I doubt anyone disagrees. At the point of realizing the above no ceiling bit, the argument devolves to more one about (fairly subjective) high/low levelness of the code itself/the effort applied to optimizing, not about the language the code is written in. So, it's not very informative and tends to go nowhere (EDIT: especially when the focus is on a single, highly optimized tool like `rg` as opposed to "broad demographic traits" of pools of developers, and "levelness" is often somewhat subjective, too).

yholio 6y ago

You would go to parallelism precisely on those platforms where simpler performance fixes (changing some data structures or implementing limited sections in a fast language) are insuficient. Eficient parallelization of an existing algorithm is a major undertaking.

eru 6y ago

> In embedded applications you don't usually have the possibility to have parallel code, and even in low level software (for example the classical UNIX utilities), for simplicity and solidity using a single thread is really fine. Depends on which of the classic utilities you are talking about. Many of them are typically IO bound. You might not get much out of throwing more CPU at them.

ben0x539 6y ago
ripgrep? :)
- burntsushi 6y ago
  Indeed. Many of the optimizations ripgrep (and the underlying regex engine) does only show benefits if the data you're searching is already in memory.[1] The same is true of GNU grep. This is because searching data that's in your OS's file cache is an exceptionally common case. [1] - I'm assuming commodity SSD in the range of a few hundred MB/s read speed. This will likely become less true as the prevalence of faster SSDs increases (low single digit GB/s).
- tsegratis 6y ago
  ripgrep is based on re2, a c library I would guess it contains more c than rust code... But what I love about this article is its lack of hype. It makes clear arguments both ways and all of them I can get behind Hype doesn't help Edit: To all my downvoters; I anticipated you :) With love and best wishes
  - burntsushi 6y ago
    No it's not. Its regex library is written in Rust, but was inspired by RE2. It shares no code with RE2. (And RE2 is a C++ library, not C.) Off the top of my head, the only C code in ripgrep is optional integration with PCRE2. In addition to whatever libc is being used on POSIX platforms. Everything else is pure Rust.
    - tsegratis 6y ago
      Ah, thanks burntsushi, I believe you are the ripgrep author even? Great work btw. Ripgrep is the best ... I will have to restrict my comment to just LLVM being a larger, c++, dependency ... Just angling for more downvotes ;) Thanks for the reply
      - burntsushi 6y ago
        To be clear, ripgrep has no runtime dependency on any LLVM or C++ library. rustc does.
        tsegratis 6y ago
        The interesting thing here is that rust has good threading and fantastic crates I played with making a regex library in rust. Which, as per RE2 design involves constructing graphs and glueing them together as the regex is traversed This requires a cycle catching gc, or, just a preallocated arena... It was my first foray into rust and felt I would need to be hitting into unsafe, which I wasn't ready for. Array indexing might decompose into an arena, but syntactically just a bit messier (imho) Would be interesting to see how the RE2 does it in rust (didn't know that) I like how the article shows both sides of the fence, it makes me realize: I get a lot of optimizations from ptr stuffing in c. But sometimes we should lay down the good, for the better

fulafel 6y ago

A lot of modern embedded hw are running operating systems providing threads (such as Linux) and multi-core CPUs.

ReactiveJelly 6y ago

Deadlocks are unique to Rust, eh?

burntsushi 6y ago

This is a bad take. ripgrep, to my knowledge, cannot be written in a higher level language without becoming a lot slower.[1] And yet, if I removed its use of parallelism by default, there will be a significantly degraded user experience by virtue of it being a lot slower. This isn't an "obsession." It's engineering. [1] - I make this claim loosely. Absence of evidence isn't evidence of absence and all that. But if I saw ripgrep implemented in, say, Python and it matched speed in the majority of cases, I would learn something.

pjmlp 6y ago
Python isn't really something I would even think as possible example, Common Lisp, D, Nim, Swift, most likely.
- cb321 6y ago
  Cython with all the appropriate cdef type declarations can match C and so might also do it. Not sure Cython exactly counts as "Python"..it's more a superset/dialect { and I also doubt such a port would hold many lessons for @burntsushi, but it bore noting. }
- burntsushi 6y ago
  So? I said, "higher level language." I didn't say, "Python specifically." I would guess D could do it. I don't know enough about Nim or Swift. I would learn something if Common Lisp did it. I'd also learn something if Haskell or Go did it.
  - cb321 6y ago
    I am not trying to contradict anyone here, but any language mature enough to have an impl/way to not have arbitrary performance ceilings needs access to inline assembly/SIMD. Cython/Nim/SBCL can all do that..probably Haskell..Not so sure about Go or Swift. Anyway, many languages can respond well to optimization effort. I doubt anyone disagrees. At the point of realizing the above no ceiling bit, the argument devolves to more one about (fairly subjective) high/low levelness of the code itself/the effort applied to optimizing, not about the language the code is written in. So, it's not very informative and tends to go nowhere (EDIT: especially when the focus is on a single, highly optimized tool like `rg` as opposed to "broad demographic traits" of pools of developers, and "levelness" is often somewhat subjective, too).