4 ms·

Wuffs touts its lack of memory allocation as a safety feature [0]. Rust could add a `#![forbid(alloc)]` declaration and achieve the same effect. I think this

Wuffs touts its lack of memory allocation as a safety feature [0]. Rust could add a `#![forbid(alloc)]` declaration and achieve the same effect. I think this would be a good idea. Wuffs types are not parameterized with lifetimes, so they may contain references only to items with static lifetime. This means that Wuffs cannot implement linked lists, hash tables, or other common data structures. To support those structures, Wuffs will need to either add lifetime parameters and a borrow-checker (like Rust) or add reference counting (like Rust `Arc`, C++ `shared_ptr`, Golang, Java, etc). [0] https://github.com/google/wuffs/blob/v0.3.0-beta.1/doc/note/memory-safety.md https://github.com/google/wuffs/blob/v0.3.0-beta.1/doc/note/...

andrepd 6y ago
Isn't that stupidly limiting?
- danielheath 6y ago
  Depends. AFAICT it’s designed to be embedded in a c project; in that context, it’s merely saying “the wuffs part does not allocate; do that in the c code”.
  - andrepd 6y ago
    Makes sense.
- amelius 6y ago
  After NoSQL, now we also have NoAlloc :)
- dahart 6y ago
  Some people consider avoiding heap allocations to be smartly limiting. ;) It’s not really that limiting either, when you learn the techniques for it. There’s a famous list of NASA’s rules for safety-critical code that you’ll see pop up now and then, that subscribes to the “avoid heap memory allocation” philosophy. https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Developing_Safety-Critical_Code https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Dev... It’s not nearly as true anymore today, but working in console games 10-20 years ago, it was very common to try to avoid dynamic memory allocation because it was viewed as slow, unsafe, and unnecessary. A common strategy was to statically pre-allocate a single block of the worst-case memory needed for a given feature, because you need to be prepared to hit the worst case without crashing, so you might as well just reserve the max and focus on reducing what the maximum possible usage is. I once debugged a crash in a console game that was caused by someone trying to be too clever by half with their allocation and copy constructors. We had a team of a dozen people working on the crash through a weekend during crunch because the crash only occurred in a release build on the console dev kit. The cost of this one stupid bug due to using memory allocation could literally accounted for in the thousands of dollars. I basically had to write a mini one-off release-build stack-tracing sampling debugger in order to trap it. So, anyway, long story short, I’m old enough to appreciate why some people suggest that avoiding allocations is a good thing.
- zamalek 6y ago
  Seems like an honest question. "Stupidly" is a key word there because humans are, without exception, vastly too stupid to deal with the combinatorial explosion that the heap presents. This is a key goal of languages like Rust: allow the compiler to deal with the complexity for you. There are vastly fewer interacting elements when you restrict yourself to the stack: basically logic errors and pointer arithmetic become the key concerns (and Wuffs supposedly deals with the latter). Really smart people may be able to completely reason about this space. Developers are known for imposter syndrome, but we have an unhealthy amount of overconfidence at the same time.
- phire 6y ago
  Yes. I'm not so sure about the usefulness for an image decoder. But if you are designing safety critical realtime software for controlling a car, then you kind of want the extra reliability guarantees that avoiding the heap gives you.
marius_k 6y ago
As I understand wuffs goal is to be good enough for parsers coders decoders. It looks like they can still achieve that by sacrificing some of the nice features of other languages.
pipeep 6y ago
> Rust could add a `#![forbid(alloc)]` declaration and achieve the same effect. Isn't that pretty much what `#![no_std]` is for? You can still use an allocator with no_std, but you have to explicitly import it. https://docs.rust-embedded.org/book/intro/no-std.html https://docs.rust-embedded.org/book/intro/no-std.html
- saagarjha 6y ago
  It seeems like Wuffs is not really designed to support dynamic memory allocation at all, which is typical for usecases like oneshot parsers.
nynx 6y ago
Rust already has a "mode" that forbids allocations: no-std without the alloc crate.
- deleted 6y ago
  [deleted]
deleted 6y ago
[deleted]
amenghra 6y ago
Go & Java don't do reference counting. They have a GC instead.
- pjmlp 6y ago
  Wrong, reference counting is a GC algorithm, and both language specifications don't state what exact algorithm is to be implemented. In fact there are tons of GC implementations across all Java implementations, including reference counting. For example, "An on-the-fly reference-counting garbage collector for java" https://dl.acm.org/doi/10.1145/1111596.1111597 https://dl.acm.org/doi/10.1145/1111596.1111597 And here is one for Go. https://github.com/sendilkumarn/gopurerc https://github.com/sendilkumarn/gopurerc
Jweb_Guru 6y ago
That is true, but on the other hand, Wuffles allows safe unchecked array indexing, which is a very hard problem that Rust doesn't attempt to tackle--in part because it spends most of its complexity budget on the borrow checker! I for one would be extremely interested in using Wuffles in the inner loop of many routines I currently write in Rust which are performance critical, and for which I know bounds checking is hurting performance (sometimes in ways it's hard to convince LLVM to remove), but for which I can't justify the use of `unsafe`. I would particularly love it as an embedded language, rather than one needing a full-blown application.
- Gaelan 6y ago
  > I would particularly love it as an embedded language, rather than one needing a full-blown application. Given the existence of [0], I imagine this is doable. [0]: https://docs.rs/inline-python/0.6.0/inline_python/ https://docs.rs/inline-python/0.6.0/inline_python/
  - mckirk 6y ago
    That inline Python is the closest thing to black magic I've seen in a while. I guess I have to find an excuse to play around with Rust some time soon, it seems like such a cool ecosystem.
    - iudqnolq 6y ago
      If you like that sort of thing you'll enjoy following the creater on twitter. Here's another of her creations > Did you know you can just fork() the @rustlang compiler from your proc_macro? > Unfortunately rustc is multithreaded and fork() only forks one thread. We'll just assume the other threads weren't doing anything important. > In today's rustc, this means we're only missing the main thread in the fork. But that one was only going to call exit() after joining our thread anyway, so not very important. It just means the process always exits with a success status. So instead we grep the output for "error". https://mobile.twitter.com/m_ou_se/status/1368844186607357957 https://mobile.twitter.com/m_ou_se/status/136884418660735795...
- 6y ago
Karliss 6y ago
Goals and Non-Goals explain that wuffs is not a general purpose programming language. If all the allocations are done by code invoking the function implemented in wuffs there is no point analyzing lifetimes. For the intended usecase lifetime of the objects and their lifetimes fall into following categories: * input buffer -> prepared by caller, must live as long as you are decoding it * output buffer -> prepared by caller, must live as long decoder outputs to it * fixed size decoder state -> allocated during initialization * decoder state with size depending on input -> various compression formats formats typically include information about expected sizes, used dictionary sizes and similar so that required memory for it can be allocated ahead of time. You don't want memory allocations during the hot loop anyway. And if it turns out that content doesn't match sizes from header it's an invalid file.
pjmlp 6y ago
MISRA-C(++), Ada/SPARK also forbid it, yet they are there powering lots of embedded systems, there is a lot of stuff that can still be implemented with such restrictions.
flohofwoe 6y ago
IMHO if Rust had a "proper" way to transpile to C, then a "C-style subset" of Rust would be a perfect library-implementation language and something like Wuff wouldn't be needed.

andrepd 6y ago

Isn't that stupidly limiting?

danielheath 6y ago
Depends. AFAICT it’s designed to be embedded in a c project; in that context, it’s merely saying “the wuffs part does not allocate; do that in the c code”.
- andrepd 6y ago
  Makes sense.
amelius 6y ago
After NoSQL, now we also have NoAlloc :)
dahart 6y ago
Some people consider avoiding heap allocations to be smartly limiting. ;) It’s not really that limiting either, when you learn the techniques for it. There’s a famous list of NASA’s rules for safety-critical code that you’ll see pop up now and then, that subscribes to the “avoid heap memory allocation” philosophy. https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Developing_Safety-Critical_Code https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Dev... It’s not nearly as true anymore today, but working in console games 10-20 years ago, it was very common to try to avoid dynamic memory allocation because it was viewed as slow, unsafe, and unnecessary. A common strategy was to statically pre-allocate a single block of the worst-case memory needed for a given feature, because you need to be prepared to hit the worst case without crashing, so you might as well just reserve the max and focus on reducing what the maximum possible usage is. I once debugged a crash in a console game that was caused by someone trying to be too clever by half with their allocation and copy constructors. We had a team of a dozen people working on the crash through a weekend during crunch because the crash only occurred in a release build on the console dev kit. The cost of this one stupid bug due to using memory allocation could literally accounted for in the thousands of dollars. I basically had to write a mini one-off release-build stack-tracing sampling debugger in order to trap it. So, anyway, long story short, I’m old enough to appreciate why some people suggest that avoiding allocations is a good thing.
zamalek 6y ago
Seems like an honest question. "Stupidly" is a key word there because humans are, without exception, vastly too stupid to deal with the combinatorial explosion that the heap presents. This is a key goal of languages like Rust: allow the compiler to deal with the complexity for you. There are vastly fewer interacting elements when you restrict yourself to the stack: basically logic errors and pointer arithmetic become the key concerns (and Wuffs supposedly deals with the latter). Really smart people may be able to completely reason about this space. Developers are known for imposter syndrome, but we have an unhealthy amount of overconfidence at the same time.
phire 6y ago
Yes. I'm not so sure about the usefulness for an image decoder. But if you are designing safety critical realtime software for controlling a car, then you kind of want the extra reliability guarantees that avoiding the heap gives you.

marius_k 6y ago

As I understand wuffs goal is to be good enough for parsers coders decoders. It looks like they can still achieve that by sacrificing some of the nice features of other languages.

pipeep 6y ago

> Rust could add a `#![forbid(alloc)]` declaration and achieve the same effect. Isn't that pretty much what `#![no_std]` is for? You can still use an allocator with no_std, but you have to explicitly import it. https://docs.rust-embedded.org/book/intro/no-std.html https://docs.rust-embedded.org/book/intro/no-std.html

saagarjha 6y ago
It seeems like Wuffs is not really designed to support dynamic memory allocation at all, which is typical for usecases like oneshot parsers.

nynx 6y ago

Rust already has a "mode" that forbids allocations: no-std without the alloc crate.

deleted 6y ago
[deleted]

deleted 6y ago

[deleted]

amenghra 6y ago

Go & Java don't do reference counting. They have a GC instead.

pjmlp 6y ago
Wrong, reference counting is a GC algorithm, and both language specifications don't state what exact algorithm is to be implemented. In fact there are tons of GC implementations across all Java implementations, including reference counting. For example, "An on-the-fly reference-counting garbage collector for java" https://dl.acm.org/doi/10.1145/1111596.1111597 https://dl.acm.org/doi/10.1145/1111596.1111597 And here is one for Go. https://github.com/sendilkumarn/gopurerc https://github.com/sendilkumarn/gopurerc

Jweb_Guru 6y ago

That is true, but on the other hand, Wuffles allows safe unchecked array indexing, which is a very hard problem that Rust doesn't attempt to tackle--in part because it spends most of its complexity budget on the borrow checker! I for one would be extremely interested in using Wuffles in the inner loop of many routines I currently write in Rust which are performance critical, and for which I know bounds checking is hurting performance (sometimes in ways it's hard to convince LLVM to remove), but for which I can't justify the use of `unsafe`. I would particularly love it as an embedded language, rather than one needing a full-blown application.

Gaelan 6y ago
> I would particularly love it as an embedded language, rather than one needing a full-blown application. Given the existence of [0], I imagine this is doable. [0]: https://docs.rs/inline-python/0.6.0/inline_python/ https://docs.rs/inline-python/0.6.0/inline_python/
- mckirk 6y ago
  That inline Python is the closest thing to black magic I've seen in a while. I guess I have to find an excuse to play around with Rust some time soon, it seems like such a cool ecosystem.
  - iudqnolq 6y ago
    If you like that sort of thing you'll enjoy following the creater on twitter. Here's another of her creations > Did you know you can just fork() the @rustlang compiler from your proc_macro? > Unfortunately rustc is multithreaded and fork() only forks one thread. We'll just assume the other threads weren't doing anything important. > In today's rustc, this means we're only missing the main thread in the fork. But that one was only going to call exit() after joining our thread anyway, so not very important. It just means the process always exits with a success status. So instead we grep the output for "error". https://mobile.twitter.com/m_ou_se/status/1368844186607357957 https://mobile.twitter.com/m_ou_se/status/136884418660735795...
6y ago

Karliss 6y ago

Goals and Non-Goals explain that wuffs is not a general purpose programming language. If all the allocations are done by code invoking the function implemented in wuffs there is no point analyzing lifetimes. For the intended usecase lifetime of the objects and their lifetimes fall into following categories: * input buffer -> prepared by caller, must live as long as you are decoding it * output buffer -> prepared by caller, must live as long decoder outputs to it * fixed size decoder state -> allocated during initialization * decoder state with size depending on input -> various compression formats formats typically include information about expected sizes, used dictionary sizes and similar so that required memory for it can be allocated ahead of time. You don't want memory allocations during the hot loop anyway. And if it turns out that content doesn't match sizes from header it's an invalid file.

pjmlp 6y ago

MISRA-C(++), Ada/SPARK also forbid it, yet they are there powering lots of embedded systems, there is a lot of stuff that can still be implemented with such restrictions.

flohofwoe 6y ago

IMHO if Rust had a "proper" way to transpile to C, then a "C-style subset" of Rust would be a perfect library-implementation language and something like Wuff wouldn't be needed.