9 ms·

Memory-safe, clean implementation of classic Posix "BC" calculator

33 points by jgarzik 2y ago

scrubs 2y ago
The title reminds me of the adds for "Shen Yun China Before Communism" which begs the question: ok what about life after communism ... where's the song and dance show about that? Similarly, if this is the memsafe bc, where's the wow it leaks memory like a 1980s rock ballad bc code? Hint: one has an answer, and the other has really never been on the radar safe or no.
downvotetruth 2y ago
Lost as to why there is a tests folder yet the test function marked with the #[test] attribute is being added to /src: https://github.com/rustcoreutils/posixutils-rs/pull/132/commits/dd9f9531b48171f9417685b8a9440180fd39a985#diff-399ef6729038be6ed4b917820c57c591b487d20593f93b70428976bd0072dc67 https://github.com/rustcoreutils/posixutils-rs/pull/132/comm... edit: Since the change was to the grammar saved in a separate .pest file, which is separated from the procedural code, then would that make the test an integration rather than a unit test as the "parse_program" function is public?
- Cyph0n 2y ago
  In Rust, unit tests are marked using the #[test] attribute and typically live alongside the code under test. Integration tests live under a top-level “tests” folder - in this case, you’ll see that the “tests” directory contains test that call the CLI command and verify its output. See: https://doc.rust-lang.org/book/ch11-03-test-organization.html https://doc.rust-lang.org/book/ch11-03-test-organization.htm...
  - deleted 2y ago
    [deleted]
- boustrophedon 2y ago
  The `parse_program` function is public inside the `bc_util::parser` module, and the parser module is marked public inside `bc_util`, but in the `calc/src/bc.rs` file the `bc_util` mod isn't public, and can't be accessed from a test inside the `tests/` folder, which only has access to the public API exported by the library.
gavinhoward 2y ago
As the author of the default bc in macOS and FreeBSD, good luck. Also, while mine is in C, I have tried very hard to eliminate all memory safety bugs, and I think I have done pretty well. https://git.gavinhoward.com/gavin/bc/ https://git.gavinhoward.com/gavin/bc/
- c0wb0yc0d3r 2y ago
  > good luck. Do you see this as an unexpectedly challenging task? If so, what do you think makes it challenging?
  - gavinhoward 2y ago
    Not terribly challenging, but harder than expected. The reason: parsing. The bc language is poorly designed for a REPL, which it needs to handle. I got sick of fixing parse bugs, though, and my source code got a sarcastic comment: https://git.gavinhoward.com/gavin/bc/src/commit/4b83bf07b97f65b53ffa53f37737b1ddce42dcde/src/bc_parse.c#L49 https://git.gavinhoward.com/gavin/bc/src/commit/4b83bf07b97f...
- jgarzik 2y ago
  Amazing work. Thanks and kudos. I definitely tested some early stuff vs your bc.
082349872349872 2y ago
bc(1) is nice to have, but —for my purposes— less useful than dc(1), because it doesn't generate arbitrary binary output.
- gavinhoward 2y ago
  If you're talking about the `P` command, I added an extension to my bc to do the same thing. It's the `stream` statement. But it's probably still more convenient in dc. https://git.gavinhoward.com/gavin/bc/ https://git.gavinhoward.com/gavin/bc/
  - 082349872349872 2y ago
    Exactly; also nice to see https://git.gavinhoward.com/gavin/bc/#ai-free https://git.gavinhoward.com/gavin/bc/#ai-free . Lagniappe: 6581840dnP
    - gavinhoward 2y ago
      Is that a dc quine? Cool! Thank you! And yes, I hate AI.
- HobbitJack 2y ago
  I prefer dc because I'm an HP calculator user... we're not the same ^-^
teo_zero 2y ago
Am I the only one who thinks we need less short-lived tools written in memory-safe languages an more system infrastructure written in said languages? Don't take me wrong, I like when someone creates a better version of something existing. For example, ripgrep is fantastic. But seeing the recent trend of rewriting whatever in Rust just because, I can't help wondering why I should bother whether bc leaks or not...
- dijit 2y ago
  Those will take time, nobody will re-invent Linux, likely it will come with the OS systems knowledge advancements from the last 30 years. Redox-OS is one such effort.
- uecker 2y ago
  Rust does not guarantee absence of memory leaks, does it? I think this is the general "rewriting solves all problems" fallacy, which is rarely true, but who wants to maintain existing code, if there is something new and cool? So we get rewrites which largely just add new projects that then have to be maintained in parallel, because they rarely are to completely replace the old (if at all) and at some point the new is not cool anymore and that it is just another thing. I like some key ideas of Rust, but overall I still like many things about C much more (stability, fast compile times, simplicity), so I would much prefer people improving memory safety in C projects.
  - ilius2 2y ago
    I like Zig's approach much more than Rust. When you get fixated on one aspect and sacrifice everything for that, you get things like Perl, Java or Rust.
    - tialaramex 2y ago
      So, massive success ?
    - pjmlp 2y ago
      Major adoption across the industry,being taught at university courses, bootcamps, shipping in commercial OSes?
uecker 2y ago
I am still waiting for Rust people to do something new and innovative. I acknowledge that Rust itself is innovative, don't get me wrong, But where are the true proof-of-concept projects such as the Linux kernel, qemu, git, postgresql, vim, etc. of the Rust world? Rewriting code is a waste of time of time, fragments the eco system, and introduces new bugs. And with a new focus on memory safety and analysis tools and sanitizers evolving, a lot of C code will be made memory safe in the future without needing a rewrite in Rust.
- jonjojojon 2y ago
  Aren't most of the standalone wasm runtimes written in rust?
  - uecker 2y ago
    No idea, but according to this site: https://github.com/appcypher/awesome-wasm-runtimes https://github.com/appcypher/awesome-wasm-runtimes there a wasm runtimes written in all kinds of languages. This does not appear to be a very challenging thing to do, but I haven't look at this closely.
    - pjmlp 2y ago
      Writing bytecode runtimes is a common thing since 1958 (see UNCOL), only a novelty for those selling WASM as some great invention never done before. Only this year I started seeing WebAssembly conference talks actually acknowledging the history of bytecode based distribution.
- maxbond 2y ago
  There's tons of stuff of the kind you're describing in Rust, more than I care to list but I'll offer you some [1-3] that I think would interest you (based on this comment alone). If your perception of what people are doing in Rust is driven by what people post to HN, it may not reflect everything that is going on in that ecosystem. [1] https://os.phil-opp.com/ https://os.phil-opp.com/ [2] https://gitlab.redox-os.org/redox-os/redox https://gitlab.redox-os.org/redox-os/redox [3] https://oxide.computer/ https://oxide.computer/
mongol 2y ago
Seems there are multiple Rust coreutils implementation projects? I have heard about uutils, but this is another one?
- rfl890 2y ago
  Quoting the repo: > The goal is to create clean, race-free userland utilities that are POSIX compliant, maximizing compatibility with existing shell scripts while minimizing bloat. > It is not a goal to be compatible with GNU utilities, which are sometimes viewed as bloated and overloaded with rarely-used options. > A similar project with the aim of GNU compatibility is https://github.com/uutils/coreutils https://github.com/uutils/coreutils
- jgarzik 2y ago
  Just added this FAQ section to the README: Because it is a FAQ, the major differences between this project and uutils are: 1. Wider scope: posixutils is far more ambitious than uutils from a breadth standpoint: posixutils will include bc, m4, c99 compiler, fort77 compiler, a cron daemon etc. uutils is far more limited in the scope of programs covered, mimicing GNU coreutils. 2. More minimalist: Each posixutils utility implementation is intentionally more minimalist, intending to avoid the bloat of supporting rarely-used, non-POSIX features. Our common denominator and baseline is the POSIX spec, then add non-POSIX features that users cannot live without. 3. Transportable: Each posixutils utility should look like normal Rust code, easily stand alone with little-or-no deps, and be used in another project. This project is MIT-licensed, not GPL licensed, to aid in that transportability goal.
tiffanyh 2y ago
If you want core utilities rewritten in Rust … ‘uutils’ is exactly that. https://github.com/uutils/coreutils https://github.com/uutils/coreutils
- jgarzik 2y ago
  posixutils is something more. (see README.md)
anthk 2y ago
- DC to convert between bases and if you like RPN. - Calc it's better if you want to cover complex numbers: https://github.com/lcn2/calc https://github.com/lcn2/calc - Qalc from libqalculate in order to solve some equations in legacy machines. It's lighter and smaller than Maxima, but not as complete. - - - Maxima for anything else, the biggie one. This will cover calculus. Gnuplot will work fine as a plotter for the previous tools.
mstef 2y ago
this must be parody. what exactly is the threatmodel where memory-safety matters for a calculator? did these devs miss the point of popping a calc.exe? surely no bc has ever been used for LPE or RCE.
- jgarzik 2y ago
  The entire repo is written in Rust. bc is one of 145 utils specified by POSIX.
mstef 2y ago
what's next that urgently needs mem-safety? /bin/true?

33 points by jgarzik 2y ago

The title reminds me of the adds for "Shen Yun China Before Communism" which begs the question: ok what about life after communism ... where's the song and dance show about that? Similarly, if this is the memsafe bc, where's the wow it leaks memory like a 1980s rock ballad bc code? Hint: one has an answer, and the other has really never been on the radar safe or no.

downvotetruth 2y ago

Lost as to why there is a tests folder yet the test function marked with the #[test] attribute is being added to /src: https://github.com/rustcoreutils/posixutils-rs/pull/132/commits/dd9f9531b48171f9417685b8a9440180fd39a985#diff-399ef6729038be6ed4b917820c57c591b487d20593f93b70428976bd0072dc67 https://github.com/rustcoreutils/posixutils-rs/pull/132/comm... edit: Since the change was to the grammar saved in a separate .pest file, which is separated from the procedural code, then would that make the test an integration rather than a unit test as the "parse_program" function is public?

Cyph0n 2y ago
In Rust, unit tests are marked using the #[test] attribute and typically live alongside the code under test. Integration tests live under a top-level “tests” folder - in this case, you’ll see that the “tests” directory contains test that call the CLI command and verify its output. See: https://doc.rust-lang.org/book/ch11-03-test-organization.html https://doc.rust-lang.org/book/ch11-03-test-organization.htm...
- deleted 2y ago
  [deleted]
boustrophedon 2y ago
The `parse_program` function is public inside the `bc_util::parser` module, and the parser module is marked public inside `bc_util`, but in the `calc/src/bc.rs` file the `bc_util` mod isn't public, and can't be accessed from a test inside the `tests/` folder, which only has access to the public API exported by the library.

gavinhoward 2y ago

As the author of the default bc in macOS and FreeBSD, good luck. Also, while mine is in C, I have tried very hard to eliminate all memory safety bugs, and I think I have done pretty well. https://git.gavinhoward.com/gavin/bc/ https://git.gavinhoward.com/gavin/bc/

c0wb0yc0d3r 2y ago
> good luck. Do you see this as an unexpectedly challenging task? If so, what do you think makes it challenging?
- gavinhoward 2y ago
  Not terribly challenging, but harder than expected. The reason: parsing. The bc language is poorly designed for a REPL, which it needs to handle. I got sick of fixing parse bugs, though, and my source code got a sarcastic comment: https://git.gavinhoward.com/gavin/bc/src/commit/4b83bf07b97f65b53ffa53f37737b1ddce42dcde/src/bc_parse.c#L49 https://git.gavinhoward.com/gavin/bc/src/commit/4b83bf07b97f...
jgarzik 2y ago
Amazing work. Thanks and kudos. I definitely tested some early stuff vs your bc.

082349872349872 2y ago

bc(1) is nice to have, but —for my purposes— less useful than dc(1), because it doesn't generate arbitrary binary output.

gavinhoward 2y ago
If you're talking about the `P` command, I added an extension to my bc to do the same thing. It's the `stream` statement. But it's probably still more convenient in dc. https://git.gavinhoward.com/gavin/bc/ https://git.gavinhoward.com/gavin/bc/
- 082349872349872 2y ago
  Exactly; also nice to see https://git.gavinhoward.com/gavin/bc/#ai-free https://git.gavinhoward.com/gavin/bc/#ai-free . Lagniappe: 6581840dnP
  - gavinhoward 2y ago
    Is that a dc quine? Cool! Thank you! And yes, I hate AI.
HobbitJack 2y ago
I prefer dc because I'm an HP calculator user... we're not the same ^-^

teo_zero 2y ago

Am I the only one who thinks we need less short-lived tools written in memory-safe languages an more system infrastructure written in said languages? Don't take me wrong, I like when someone creates a better version of something existing. For example, ripgrep is fantastic. But seeing the recent trend of rewriting whatever in Rust just because, I can't help wondering why I should bother whether bc leaks or not...

dijit 2y ago
Those will take time, nobody will re-invent Linux, likely it will come with the OS systems knowledge advancements from the last 30 years. Redox-OS is one such effort.
uecker 2y ago
Rust does not guarantee absence of memory leaks, does it? I think this is the general "rewriting solves all problems" fallacy, which is rarely true, but who wants to maintain existing code, if there is something new and cool? So we get rewrites which largely just add new projects that then have to be maintained in parallel, because they rarely are to completely replace the old (if at all) and at some point the new is not cool anymore and that it is just another thing. I like some key ideas of Rust, but overall I still like many things about C much more (stability, fast compile times, simplicity), so I would much prefer people improving memory safety in C projects.
- ilius2 2y ago
  I like Zig's approach much more than Rust. When you get fixated on one aspect and sacrifice everything for that, you get things like Perl, Java or Rust.
  - tialaramex 2y ago
    So, massive success ?
  - pjmlp 2y ago
    Major adoption across the industry,being taught at university courses, bootcamps, shipping in commercial OSes?

uecker 2y ago

I am still waiting for Rust people to do something new and innovative. I acknowledge that Rust itself is innovative, don't get me wrong, But where are the true proof-of-concept projects such as the Linux kernel, qemu, git, postgresql, vim, etc. of the Rust world? Rewriting code is a waste of time of time, fragments the eco system, and introduces new bugs. And with a new focus on memory safety and analysis tools and sanitizers evolving, a lot of C code will be made memory safe in the future without needing a rewrite in Rust.

jonjojojon 2y ago
Aren't most of the standalone wasm runtimes written in rust?
- uecker 2y ago
  No idea, but according to this site: https://github.com/appcypher/awesome-wasm-runtimes https://github.com/appcypher/awesome-wasm-runtimes there a wasm runtimes written in all kinds of languages. This does not appear to be a very challenging thing to do, but I haven't look at this closely.
  - pjmlp 2y ago
    Writing bytecode runtimes is a common thing since 1958 (see UNCOL), only a novelty for those selling WASM as some great invention never done before. Only this year I started seeing WebAssembly conference talks actually acknowledging the history of bytecode based distribution.
maxbond 2y ago
There's tons of stuff of the kind you're describing in Rust, more than I care to list but I'll offer you some [1-3] that I think would interest you (based on this comment alone). If your perception of what people are doing in Rust is driven by what people post to HN, it may not reflect everything that is going on in that ecosystem. [1] https://os.phil-opp.com/ https://os.phil-opp.com/ [2] https://gitlab.redox-os.org/redox-os/redox https://gitlab.redox-os.org/redox-os/redox [3] https://oxide.computer/ https://oxide.computer/

mongol 2y ago

Seems there are multiple Rust coreutils implementation projects? I have heard about uutils, but this is another one?

rfl890 2y ago
Quoting the repo: > The goal is to create clean, race-free userland utilities that are POSIX compliant, maximizing compatibility with existing shell scripts while minimizing bloat. > It is not a goal to be compatible with GNU utilities, which are sometimes viewed as bloated and overloaded with rarely-used options. > A similar project with the aim of GNU compatibility is https://github.com/uutils/coreutils https://github.com/uutils/coreutils
jgarzik 2y ago
Just added this FAQ section to the README: Because it is a FAQ, the major differences between this project and uutils are: 1. Wider scope: posixutils is far more ambitious than uutils from a breadth standpoint: posixutils will include bc, m4, c99 compiler, fort77 compiler, a cron daemon etc. uutils is far more limited in the scope of programs covered, mimicing GNU coreutils. 2. More minimalist: Each posixutils utility implementation is intentionally more minimalist, intending to avoid the bloat of supporting rarely-used, non-POSIX features. Our common denominator and baseline is the POSIX spec, then add non-POSIX features that users cannot live without. 3. Transportable: Each posixutils utility should look like normal Rust code, easily stand alone with little-or-no deps, and be used in another project. This project is MIT-licensed, not GPL licensed, to aid in that transportability goal.

tiffanyh 2y ago

If you want core utilities rewritten in Rust … ‘uutils’ is exactly that. https://github.com/uutils/coreutils https://github.com/uutils/coreutils

jgarzik 2y ago
posixutils is something more. (see README.md)

anthk 2y ago

- DC to convert between bases and if you like RPN. - Calc it's better if you want to cover complex numbers: https://github.com/lcn2/calc https://github.com/lcn2/calc - Qalc from libqalculate in order to solve some equations in legacy machines. It's lighter and smaller than Maxima, but not as complete. - - - Maxima for anything else, the biggie one. This will cover calculus. Gnuplot will work fine as a plotter for the previous tools.

mstef 2y ago

this must be parody. what exactly is the threatmodel where memory-safety matters for a calculator? did these devs miss the point of popping a calc.exe? surely no bc has ever been used for LPE or RCE.

jgarzik 2y ago
The entire repo is written in Rust. bc is one of 145 utils specified by POSIX.

mstef 2y ago

what's next that urgently needs mem-safety? /bin/true?