7 ms·

Out of curiosity (I've never done systems programming), what are some of the things that make a language more suited to systems programming rather than web (oth

Out of curiosity (I've never done systems programming), what are some of the things that make a language more suited to systems programming rather than web (other than libraries and community support - I'm more interested in the intrinsic qualities of the language)?

Thaxll 11y ago
Standard library, look at PHP it's all about web!
supster 11y ago
Super fast, type safe (and thread safe in Rust), compiled, low/no cost abstractions
- pecan_xt 11y ago
  Super fast? More like Java ballpark: http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=rust&lang2=java http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan...
  - steveklabnik 11y ago
    We've been focused more on language semantics than the Benchmarks Game for a while now. A few months ago, iirc, we were faster than C on some of them.
    - jamii 11y ago
      It looks like the worst offenders (vs C) are regex-dna (which is a young regex + unicode vs pcre + ascii) and fasta (where the rust version is still single-core). The others are at 1-2x cpu and on par for memory. I suspect most of them could be on par or better with a little effort - only regex looks hard.
      - burntsushi 11y ago
        > only regex looks hard. I'm working on improving the regex engine now. Hopefully I can submit some improvements soon. :-) It will be hard to get near the top, but I think there's some low hanging fruit I can tackle first.
    - igouy 11y ago
      Let's not make this about the benchmarks game. A couple of weeks ago you said -- "… optimizing performance hasn't been a focus, shipping good language semantics has been." https://news.ycombinator.com/item?id=9565405 https://news.ycombinator.com/item?id=9565405 edit: Please provide a better response than downmods.
      - pcwalton 11y ago
        Optimizing language performance has been a focus. Optimizing the workloads tested on the benchmarks game has not been a focus.
        igouy 11y ago
        You seem to have a disagreement with what steveklabnik said, so please take that up with him. https://news.ycombinator.com/item?id=9554676 https://news.ycombinator.com/item?id=9554676 edit: Please provide a better response than downmods.
        ben0x539 11y ago
        Please provide better manners than those in your comments. :)
        igouy 11y ago
        It isn't bad manners to ask Rusky or pcwalton to show measurements in support of their claims. It isn't bad manners to trust that when steveklabnik answers "No, as optimizing performance hasn't been a focus, shipping good language semantics has been." that's what he means. https://news.ycombinator.com/item?id=9554676 https://news.ycombinator.com/item?id=9554676
  - supster 11y ago
    > what are some of the things that make a language more suited to systems programming rather than web the answer was to what qualities make a lang suited for systems programming not just specific to Rust
  - Rusky 11y ago
    The benchmarks game's results are not very correlated with real-world performance. In this context, Rust's semantics do allow C-level performance- they make it straightforward to describe the same machine code that C would. In some cases, they even enable more compiler optimizations than C does. Java, on the other hand, makes it hard or impossible to do that (straightforwardly at least). A more mature compiler will make a lot of difference- compare early Java performance to where it is now, for example.
    - igouy 11y ago
      >>not very correlated with real-world performance<< Please share your measurements of "real-world performance" that are "not very correlated" with the benchmarks game's results. edit: Please provide a better response than downmods.
      - pcwalton 11y ago
        Servo layout, painting, DOM performance. JSON serialization in serde.
        igouy 11y ago
        Obviously not in any way an answer to the question asked. edit: Please provide a better response than downmods.
        Daishiman 11y ago
        Go actually check the things pcwalton mentioned instead of complaining.
        igouy 11y ago
        Please provide comparison measurements of the things pcwalton mentioned. Just a URL to published comparison measurements would be so much more interesting for anyone interested in Rust, than downmods.
  - adwn 11y ago
    Okay, so I just tested the n-body benchmark [1] for Rust and C++ (program #8), and where they list runtimes of 24.62/9.4 (a factor of 2.62), I actually got a factor of 1.25 (5.45/4.35) on my machine [2]. I used the same version of Rust and nearly the same version of gcc (4.9.1 vs 4.9.2). Conclusion: Don't draw conclusions from the benchmarks game. [1] http://benchmarksgame.alioth.debian.org/u64q/performance.php?test=nbody http://benchmarksgame.alioth.debian.org/u64q/performance.php... [2] i5-3470 @ 3.20GHz
    - igouy 11y ago
      Conclusion: "Measurement is highly specific … Measurement is not prophesy." (benchmarks game homepage.)
bbrazil 11y ago
For Prometheus's console templates (basically offering a html templating engine to end users to create monitoring consoles with), I used Go's own templating language as nothing else seemed mature at the time. Go is strongly typed, and so is it's templating language. This means that you need to take a much more rigorous approach to using them, as a single unhandled edge case value can break the entire page. Contrast this with something like Jinja2 where if you mess up it'll only break a small bit of the page. I managed to produce a usable template system, but it's still a tad more verbose than I'd like. It'd have been nice if a more scripting-language style templating language had been an option.
- S4M 11y ago
  > Contrast this with something like Jinja2 where if you mess up it'll only break a small bit of the page. Do you have some examples of error that are easy to make with Jinja2? I personally find it to be a thing of beauty, but it's possible that I don't try to make very elaborate things with templating engines and thus might be missing out.
pjmlp 11y ago
Some form of controlling the memory management. In GC enabled systems programming languages like e.g. Oberon and Modula-3, there there will be APIs to have control about memory. Ability to convert between language types and raw addresses, in the form of pointers. Toolchain should also provide the ability for aot compilation both to static and dynamic executables/libraries.
kybernetyk 11y ago
For me it's strong static typing, control about memory layout of you data, memory locality (arrays are arrays and not linked lists in disguise, you know what goes on the stack and what's on the heap, etc), support for bit based operations (which Rust lacks a little as there's not bitfields), deterministic memory management and a few things I didn't think of.
OneMoreIdol 11y ago
Outside of userland tasks like file manipulation, it is impractical to use the popular web application languages like Java or scripting languages such as Javascript or PHP or Python for systems programming without employing interfaces or hooks to libraries or programs written in languages that are compiled to machine code, simply because programs in those languages are commonly deployed using virtual machines or interpreters that have only userland privileges. So good luck writing a driver. And I don't know ... if your vm has heap allocated to it, are you able to address system memory out side of that to address hardware? Also, the language runtimes usually add a lot of overhead as well and employ automatic garbage collection, which is generally something you don't want in performance-critical systems.
- justincormack 11y ago
  That is not strictly true. Look at Snabb switch, written almost entirely in LuaJIT for example https://github.com/SnabbCo/snabbswitch https://github.com/SnabbCo/snabbswitch - you need some kind of ffi to mmap device memory, but it is quite possible. (LuaJIT is fast of course which helps for 10Gb ethernet).
  - OneMoreIdol 11y ago
    That's kinda what I was thinking when I commented, and there are sure to be exceptions. But I don't see anything different with your Snabb switch example. You can probably use Java and JNI for that matter to call mmap, but would you be able to write an implementation of mmap in LuaJIT ?
    - justincormack 11y ago
      Yes you can call mmap with the ffi in LuaJIT just fine. It is not very complicated, it just asks the kernel to do it. Implementing what the kernel does, well that does need some assembly.
      - OneMoreIdol 11y ago
        Indeed we agree. Using any ffi for that matter.
Gibbon1 11y ago
I think seriously the intrinsic thing is that the language itself doesn't marshal (right word, not sure) anything that's particularly fat. The classic is if allocating memory require dealing with a general purpose garbage collector. If that's the case you're pretty much dead as far as systems stuff goes because now you can't predict how long anything will take to run. And the minimal libraries usually need to be light weight as well. Or at least such that you can strip out what's not wanted. Here again garbage collection becomes an issue. If the language supports it natively then it's going to be impossible in practice to prevent the libraries from using it for something unexpected. The last bit is escape. There are always some cases where the language doesn't have the ability to implement a critical bit of stuff and you need a way to seamlessly mess with that something. In simple you need to be able to insert a bit of arbitrary code and have it work properly.
- derefr 11y ago
  > If that's the case you're pretty much dead as far as systems stuff goes because now you can't predict how long anything will take to run. Erlang would argue with you. GC works fine as long as you have a bunch of really tiny heaps (attached to a bunch of really tiny processes) which can be collected in parallel. It just sucks for Unix-process-sized heaps.

Thaxll 11y ago

Standard library, look at PHP it's all about web!

supster 11y ago

Super fast, type safe (and thread safe in Rust), compiled, low/no cost abstractions

pecan_xt 11y ago
Super fast? More like Java ballpark: http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=rust&lang2=java http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan...
- steveklabnik 11y ago
  We've been focused more on language semantics than the Benchmarks Game for a while now. A few months ago, iirc, we were faster than C on some of them.
  - jamii 11y ago
    It looks like the worst offenders (vs C) are regex-dna (which is a young regex + unicode vs pcre + ascii) and fasta (where the rust version is still single-core). The others are at 1-2x cpu and on par for memory. I suspect most of them could be on par or better with a little effort - only regex looks hard.
    - burntsushi 11y ago
      > only regex looks hard. I'm working on improving the regex engine now. Hopefully I can submit some improvements soon. :-) It will be hard to get near the top, but I think there's some low hanging fruit I can tackle first.
  - igouy 11y ago
    Let's not make this about the benchmarks game. A couple of weeks ago you said -- "… optimizing performance hasn't been a focus, shipping good language semantics has been." https://news.ycombinator.com/item?id=9565405 https://news.ycombinator.com/item?id=9565405 edit: Please provide a better response than downmods.
    - pcwalton 11y ago
      Optimizing language performance has been a focus. Optimizing the workloads tested on the benchmarks game has not been a focus.
      - igouy 11y ago
        You seem to have a disagreement with what steveklabnik said, so please take that up with him. https://news.ycombinator.com/item?id=9554676 https://news.ycombinator.com/item?id=9554676 edit: Please provide a better response than downmods.
        ben0x539 11y ago
        Please provide better manners than those in your comments. :)
        igouy 11y ago
        It isn't bad manners to ask Rusky or pcwalton to show measurements in support of their claims. It isn't bad manners to trust that when steveklabnik answers "No, as optimizing performance hasn't been a focus, shipping good language semantics has been." that's what he means. https://news.ycombinator.com/item?id=9554676 https://news.ycombinator.com/item?id=9554676
- supster 11y ago
  > what are some of the things that make a language more suited to systems programming rather than web the answer was to what qualities make a lang suited for systems programming not just specific to Rust
- Rusky 11y ago
  The benchmarks game's results are not very correlated with real-world performance. In this context, Rust's semantics do allow C-level performance- they make it straightforward to describe the same machine code that C would. In some cases, they even enable more compiler optimizations than C does. Java, on the other hand, makes it hard or impossible to do that (straightforwardly at least). A more mature compiler will make a lot of difference- compare early Java performance to where it is now, for example.
  - igouy 11y ago
    >>not very correlated with real-world performance<< Please share your measurements of "real-world performance" that are "not very correlated" with the benchmarks game's results. edit: Please provide a better response than downmods.
    - pcwalton 11y ago
      Servo layout, painting, DOM performance. JSON serialization in serde.
      - igouy 11y ago
        Obviously not in any way an answer to the question asked. edit: Please provide a better response than downmods.
        Daishiman 11y ago
        Go actually check the things pcwalton mentioned instead of complaining.
        igouy 11y ago
        Please provide comparison measurements of the things pcwalton mentioned. Just a URL to published comparison measurements would be so much more interesting for anyone interested in Rust, than downmods.
- adwn 11y ago
  Okay, so I just tested the n-body benchmark [1] for Rust and C++ (program #8), and where they list runtimes of 24.62/9.4 (a factor of 2.62), I actually got a factor of 1.25 (5.45/4.35) on my machine [2]. I used the same version of Rust and nearly the same version of gcc (4.9.1 vs 4.9.2). Conclusion: Don't draw conclusions from the benchmarks game. [1] http://benchmarksgame.alioth.debian.org/u64q/performance.php?test=nbody http://benchmarksgame.alioth.debian.org/u64q/performance.php... [2] i5-3470 @ 3.20GHz
  - igouy 11y ago
    Conclusion: "Measurement is highly specific … Measurement is not prophesy." (benchmarks game homepage.)

bbrazil 11y ago

For Prometheus's console templates (basically offering a html templating engine to end users to create monitoring consoles with), I used Go's own templating language as nothing else seemed mature at the time. Go is strongly typed, and so is it's templating language. This means that you need to take a much more rigorous approach to using them, as a single unhandled edge case value can break the entire page. Contrast this with something like Jinja2 where if you mess up it'll only break a small bit of the page. I managed to produce a usable template system, but it's still a tad more verbose than I'd like. It'd have been nice if a more scripting-language style templating language had been an option.

S4M 11y ago
> Contrast this with something like Jinja2 where if you mess up it'll only break a small bit of the page. Do you have some examples of error that are easy to make with Jinja2? I personally find it to be a thing of beauty, but it's possible that I don't try to make very elaborate things with templating engines and thus might be missing out.

pjmlp 11y ago

Some form of controlling the memory management. In GC enabled systems programming languages like e.g. Oberon and Modula-3, there there will be APIs to have control about memory. Ability to convert between language types and raw addresses, in the form of pointers. Toolchain should also provide the ability for aot compilation both to static and dynamic executables/libraries.

kybernetyk 11y ago

For me it's strong static typing, control about memory layout of you data, memory locality (arrays are arrays and not linked lists in disguise, you know what goes on the stack and what's on the heap, etc), support for bit based operations (which Rust lacks a little as there's not bitfields), deterministic memory management and a few things I didn't think of.

OneMoreIdol 11y ago

Outside of userland tasks like file manipulation, it is impractical to use the popular web application languages like Java or scripting languages such as Javascript or PHP or Python for systems programming without employing interfaces or hooks to libraries or programs written in languages that are compiled to machine code, simply because programs in those languages are commonly deployed using virtual machines or interpreters that have only userland privileges. So good luck writing a driver. And I don't know ... if your vm has heap allocated to it, are you able to address system memory out side of that to address hardware? Also, the language runtimes usually add a lot of overhead as well and employ automatic garbage collection, which is generally something you don't want in performance-critical systems.

justincormack 11y ago
That is not strictly true. Look at Snabb switch, written almost entirely in LuaJIT for example https://github.com/SnabbCo/snabbswitch https://github.com/SnabbCo/snabbswitch - you need some kind of ffi to mmap device memory, but it is quite possible. (LuaJIT is fast of course which helps for 10Gb ethernet).
- OneMoreIdol 11y ago
  That's kinda what I was thinking when I commented, and there are sure to be exceptions. But I don't see anything different with your Snabb switch example. You can probably use Java and JNI for that matter to call mmap, but would you be able to write an implementation of mmap in LuaJIT ?
  - justincormack 11y ago
    Yes you can call mmap with the ffi in LuaJIT just fine. It is not very complicated, it just asks the kernel to do it. Implementing what the kernel does, well that does need some assembly.
    - OneMoreIdol 11y ago
      Indeed we agree. Using any ffi for that matter.

Gibbon1 11y ago

I think seriously the intrinsic thing is that the language itself doesn't marshal (right word, not sure) anything that's particularly fat. The classic is if allocating memory require dealing with a general purpose garbage collector. If that's the case you're pretty much dead as far as systems stuff goes because now you can't predict how long anything will take to run. And the minimal libraries usually need to be light weight as well. Or at least such that you can strip out what's not wanted. Here again garbage collection becomes an issue. If the language supports it natively then it's going to be impossible in practice to prevent the libraries from using it for something unexpected. The last bit is escape. There are always some cases where the language doesn't have the ability to implement a critical bit of stuff and you need a way to seamlessly mess with that something. In simple you need to be able to insert a bit of arbitrary code and have it work properly.

derefr 11y ago
> If that's the case you're pretty much dead as far as systems stuff goes because now you can't predict how long anything will take to run. Erlang would argue with you. GC works fine as long as you have a bunch of really tiny heaps (attached to a bunch of really tiny processes) which can be collected in parallel. It just sucks for Unix-process-sized heaps.