4 ms·

Safety features of the Hare programming language

62 points by ddevault 4y ago

epage 4y ago
> Hare gets around this by simply requiring all variables to be initialized as soon as they are declared. Since Hare is an expression-based language, variables which require logic to initialize correctly can be initialized from the result of any arbitrary expression, such as an if or match statement. > > “trust the programmer, but not by default”. Every one of the features I have explained now can be circumvented, by design, if the programmer tells the compiler that they know better. I wonder what the circumvention is for uninitialized memory, which is important for large data structures in high performance code. > The “!” operator shown here is an error assertion, which promises that the error cannot occur. The compiler will check your work at runtime, and terminate the program if the error does indeed occur — you can simulate this via hare run main.ha >/dev/full. If we removed this operator, we would get a compiler error, because you cannot ignore errors. This is similar to Rust's `.unwrap()` but even more invisible. I already wish `.unwrap()` was `.unwrap_or_panic()` to both make it more cumbersome, discouraging its use, and to make it easier to search for compared to the other `unwrap_or`s that exist. > This one might raise eyebrows, but I consider Hare’s lack of a package manager to be an important security feature. There have been thousands of packages compromised on countless language-specific package managers: npm, PyPI, RubyGems, Cargo, and more — none of them have escaped this. This comes down to a fundamental problem in their design: arbitrary people on the internet cannot be trusted to publish packages directly for user consumption. This also has to be offset with the risks that come from people writing their own code or copy/pasting it and losing track of where it came from for when a CVE is released. This also doesn't deal with the other challenges: - You can't predict what your code is going to build against when relying on Linux distributions - You are stuck with the lowest common denominator of the distributions you support (which implies you are having to track what distribution versions are compatible with your project) - Care is needed to ensure you don't end up in dependency purgatory where two programs need incompatible versions of a dependency - Bootstrapping a new dependency or developing features in tandem has a high cost Overall, if the friction is too high, people will go with suboptimal solutions, whether just not creating as feature-rich projects or creating ad-hoc "dependency management solutions" that are even worse than the problem being avoided. This is reminding me of a recent /r/cpp / /r/rust thread where someone was amazed at how fast the standard recursive directory walker was in Rust compared to C++. The Rust one is an algorithm from ripgrep split into its own library for others to use it, so you get the benefit of all of the research that was done to make ripgrep fast. Without cargo, we'd have a lot more siloing of code where you couldn't take advantage of the code people wrote for their program in your own. It'd be hard to be aware the code existed in the first place and it'd be more entangled, harder to extract it for your own use.
- ddevault 4y ago
  >I wonder what the circumvention is for uninitialized memory, which is important for large data structures in high performance code. Actually there's not really much of a means around this. You could use static memory, which is only initialized once, and you can mmap pages directly from the kernel without initializing them, but allocating uninitialized data on the stack is not possible afaik.
  - tialaramex 4y ago
    So the answer (this is Drew's language) is "We don't offer one"?
- steeleduncan 4y ago
  > This one might raise eyebrows, but I consider Hare’s lack of a package manager to be an important security feature Hare looks really interesting to me, and I want to give it a try, but this is frustrating. Whatever the author's intentions are, if there is no built-in package manager, 3rd party package managers will pop up as soon as Hare catches on. All the problems he mentions will arise in those, and any development effort to tackle those problems will be split across multiple competing package managers.
  - ddevault 4y ago
    I sincerely hope that this does not happen, but it's a matter of shaping the culture appropriately. We explain our motivations for not having one and re-enforce that through the community and hope that this prevents people from building or adopting third-party package managers. Should the situation arise regardless, I will make some attempt to address it by defining upstream standards and perhaps blessing an implementation, but I hope very much that we won't have to. I will also point out that Hare's approach is similar to C, and no attempts to make a C package manager have caught on. It's also helped by the fact that Hare does not support proprietary operating systems IMO.
    - steeleduncan 4y ago
seanw444 4y ago
Drew is getting a lot of flak for this language for some reason. I think it's interesting, and it's obviously still brand-new. Problems will get worked out with time. I, for one, am quite excited to give it a try. I like the "minimalism" (in a sense) of C, so Hare seems to be right up my alley. Just judging based off of syntax, I'm already a fan. Side note - Drew, your Gemini capsule was one of the few I regularly check! I thought my internet was down when it wouldn't connect. Hope to see you back on sometime.
- duped 4y ago
  I think it's because the Hare authors seem to willingly ignore decades of PL research and practice. In this post the section on garbage collection and the notion of adding a borrow checker without changing the language seem a little misguided or even naive.
  - ddevault 4y ago
    A lot of languages have been accused of "ignoring decades of PL research", an almost word-for-word criticism I've heard of a dozen languages or more. In fact, Hare includes a lot of innovations from newer PL research, but indeed not all of them, because different designers can approach a problem with different values and opinions. It's a deliberate design choice, not ignorance.
    - duped 4y ago
      I can respect intentional and principled design choices, but some of the comments you've made on things like garbage collection, borrow checking, generic programming, compiler optimizations, etc don't come off as such. It would be interesting to read some of the modern design choices that Hare is making, for me most of what I understand about Hare are the things you've chosen not to do. Which is interesting, but not necessarily the most compelling all the time. I also think two of the principles you laid out - ease of implementation and transparency/explicitness of code are the hottest takes in Hare. I can see why people like or dislike those choices, and it's compelling to think about, but I'm not sure that the explicit choices to add or not add features from PL research reflect those. For example the paragraph on GC - I think there's a ton of reasons not to use a garbage collector in a language like Hare (for example, you have to write a GC in some language and not the one that is GC'd) but I'm not sure the reasoning you laid out is among them (concurrent and hard-realtime GCs are thing, whether removing the burden of manual memory management is more/less transparent to the intent of a programmer is very debatable).
cpurdy 4y ago
Wonderfully written and explained. Kudos!
xigoi 4y ago
There's a typo: > This plays into Hare’s goals for explicitness and transparency, and improves the saftey and predictability of the language.
- ddevault 4y ago
  Thanks!
Zababa 4y ago
Interesting and well written, as always. I might be missing something, but I think that one approach that hasn't been discussed is reference counting. The explanation for why garbage collection is: > Garbage collection magically stops your code (sometimes at predictable points, but it’s generally not very explicit), which is not very transparent and is definitely off the table for use-cases like kernels, video games, real-time applications, and so on. The behavior of a Hare program should be easy to predict, and a garbage collector interferes with that goal. Thus, despite the fact that a garbage collector would improve safety and ease-of-use, it’s not a good fit for Hare. From what I understand, reference counting doesn't stop the code. However, it might not meet the goal of "Transparency and explicitness in code". Maybe it's too much for kernels, video games, real-time applications too.
- ddevault 4y ago
  Reference counting is still pretty high level, given that you have to find somewhere to put the count. It's not really suitable for kernels or some embedded applications, for instance. That makes it less desirable as a language feature. However, you can definitely implement reference counting in your own programs and libraries.
  - Zababa 4y ago
    Thank you for the explanation, that makes sense.
lifthrasiir 4y ago
I'm happy to see this article, it does look like a much clearer explanation of Hare's pros and cons in terms of safety. I might not agree to every point but acknowledging and documenting trade-offs is surely a big plus. On borrow checkers: There is a big dichitomy between type systems and static analyses, where type systems aim to be sound---never fails to catch certain bugs, at expense of false positives---while static analyses tend not. But Rust's borrow checker is a sound static analysis combined with a type system. In fact it is a kind of optimal design when you are only allowed to add additional kind of types, and this restriction made the borrow checker a very impressive achievement. But there are other possibilities if you loosen a bit---either slightly unsound analysis [1] or annotation methods other than types. I believe Hare should better explore those possibilities instead of having a borrow checker. [1] In fact Rust has already compromised as well; for example its notion of memory safety doesn't ensure no memory leak. It is still unlikely but not impossible to unintentionally leak memory, so it is deemed acceptable. You can use a similar compromise in Hare.
- tialaramex 4y ago
  The existence of features like Box::leak() doesn't compromise Rust's memory safety which is why that's not marked unsafe. The reference counted and garbage collected languages likewise can leak, and if Box::leak() didn't exist, people would just roll their own by the same means. It's not even as though their hand-rolled mechanism would be syntactically heavier, you can't thing.leak() because leak intentionally isn't a method, it's just an associated function on Box and so you will need to explicitly Box::leak(thing) I don't think Hare's design is at all conducive to trying to retrofit a borrow checker, and so I would agree with you that it should not attempt such a thing.
  - lifthrasiir 4y ago
    I mean, the memory safety is not an exactly defined term and it took a compromise to pick the exact definition of memory safety for Rust. I fully agree that the memory leak is far less risky than this notion of memory safety, but it is still a (slight) compromise especially when you are new to Rust. As a new language Hare doesn't have to take this granted.
goodpoint 4y ago
The interesting aspect is not having a language-specific package manager.
pwdisswordfish9 4y ago
> I consider Hare’s lack of a package manager to be an important security feature. Hare’s package manager is curl|bash? How is that a security feature?
- deleted 4y ago
  [deleted]
- ddevault 4y ago
  No. The article details that Hare's package manager is your system package manager, e.g. aptitude. Such package managers have independent signatures, features against partial updates, reproducible builds, and independent accountability, as well as many other advantages besides.
  - pwdisswordfish9 4y ago
    People offering Hare packages will not bother to wait for distro repositories to graciously package them six months later. They will just tell their users to copy-paste curl-to-bash one-liners. And users will follow, because why would they want to use stale or outright non-existent distro packages? ‘But I am going to tell people not to do that!’ As if people are going to bow down before the unstoppable appeal of your charisma instead of following the path of least resistance. Of course, this is all assuming there will be meaningful external Hare packages in the first place.
    - ddevault 4y ago
      Use a distribution whose release cadence matches your needs. I use Alpine Linux, which has frequent updates on edge and biannual stable releases, which allows me to enjoy both approaches. Debian is designed for the longer term, Arch is designed to be bleeding edge. The idea that Linux distros have outdated software is a symptom of the fact that people are not choosing the right tool for the job, something that I am particularly unsympathetic to when thinking of an audience of software engineers. Hare is a conservative language which strongly encourages a conservative approach. We are not here to move fast and break things. Culture comes from the top-down, and while we may not succeed for all users, we can certainly set the standard of quality for the language. C has no successful independent package manager, yet we rarely if ever see people distributing C libraries with curl|bash.
mamcx 4y ago
In relation with the borrow checker, I think the complications on Rust are overblown at the end: Most of the complexity of Rust is in the interaction of all the others features AND when sometimes it interfere with the borrow checker. The borrow/mutability checker are fairly easy to use (as long you stay on simple Rust: Only enum, structs, functions and not get with traits + generics + lifetimes annotations, and today: async). Playing with the simple subset of Rust the borrow checker is fairly easy to get. I don't think nobody get stuck at this stage, only when start gettin ambitious and is not clear how ALL the features must intermix. If the language is truly simple, I think it will work wonders.
- turminal 4y ago
  I'm one of the Hare contributors researching the possibility of adding a borrow checker. I'm still at the start of that journey, and I'm looking at the problem implementation-wise rather than user-wise, but my impression so far regarding simplicity and issues that arise in interaction with other complex language features matches what you said.

62 points by ddevault 4y ago

> Hare gets around this by simply requiring all variables to be initialized as soon as they are declared. Since Hare is an expression-based language, variables which require logic to initialize correctly can be initialized from the result of any arbitrary expression, such as an if or match statement. > > “trust the programmer, but not by default”. Every one of the features I have explained now can be circumvented, by design, if the programmer tells the compiler that they know better. I wonder what the circumvention is for uninitialized memory, which is important for large data structures in high performance code. > The “!” operator shown here is an error assertion, which promises that the error cannot occur. The compiler will check your work at runtime, and terminate the program if the error does indeed occur — you can simulate this via hare run main.ha >/dev/full. If we removed this operator, we would get a compiler error, because you cannot ignore errors. This is similar to Rust's `.unwrap()` but even more invisible. I already wish `.unwrap()` was `.unwrap_or_panic()` to both make it more cumbersome, discouraging its use, and to make it easier to search for compared to the other `unwrap_or`s that exist. > This one might raise eyebrows, but I consider Hare’s lack of a package manager to be an important security feature. There have been thousands of packages compromised on countless language-specific package managers: npm, PyPI, RubyGems, Cargo, and more — none of them have escaped this. This comes down to a fundamental problem in their design: arbitrary people on the internet cannot be trusted to publish packages directly for user consumption. This also has to be offset with the risks that come from people writing their own code or copy/pasting it and losing track of where it came from for when a CVE is released. This also doesn't deal with the other challenges: - You can't predict what your code is going to build against when relying on Linux distributions - You are stuck with the lowest common denominator of the distributions you support (which implies you are having to track what distribution versions are compatible with your project) - Care is needed to ensure you don't end up in dependency purgatory where two programs need incompatible versions of a dependency - Bootstrapping a new dependency or developing features in tandem has a high cost Overall, if the friction is too high, people will go with suboptimal solutions, whether just not creating as feature-rich projects or creating ad-hoc "dependency management solutions" that are even worse than the problem being avoided. This is reminding me of a recent /r/cpp / /r/rust thread where someone was amazed at how fast the standard recursive directory walker was in Rust compared to C++. The Rust one is an algorithm from ripgrep split into its own library for others to use it, so you get the benefit of all of the research that was done to make ripgrep fast. Without cargo, we'd have a lot more siloing of code where you couldn't take advantage of the code people wrote for their program in your own. It'd be hard to be aware the code existed in the first place and it'd be more entangled, harder to extract it for your own use.

ddevault 4y ago
>I wonder what the circumvention is for uninitialized memory, which is important for large data structures in high performance code. Actually there's not really much of a means around this. You could use static memory, which is only initialized once, and you can mmap pages directly from the kernel without initializing them, but allocating uninitialized data on the stack is not possible afaik.
- tialaramex 4y ago
  So the answer (this is Drew's language) is "We don't offer one"?
steeleduncan 4y ago
> This one might raise eyebrows, but I consider Hare’s lack of a package manager to be an important security feature Hare looks really interesting to me, and I want to give it a try, but this is frustrating. Whatever the author's intentions are, if there is no built-in package manager, 3rd party package managers will pop up as soon as Hare catches on. All the problems he mentions will arise in those, and any development effort to tackle those problems will be split across multiple competing package managers.
- ddevault 4y ago
  I sincerely hope that this does not happen, but it's a matter of shaping the culture appropriately. We explain our motivations for not having one and re-enforce that through the community and hope that this prevents people from building or adopting third-party package managers. Should the situation arise regardless, I will make some attempt to address it by defining upstream standards and perhaps blessing an implementation, but I hope very much that we won't have to. I will also point out that Hare's approach is similar to C, and no attempts to make a C package manager have caught on. It's also helped by the fact that Hare does not support proprietary operating systems IMO.
  - steeleduncan 4y ago

seanw444 4y ago

Drew is getting a lot of flak for this language for some reason. I think it's interesting, and it's obviously still brand-new. Problems will get worked out with time. I, for one, am quite excited to give it a try. I like the "minimalism" (in a sense) of C, so Hare seems to be right up my alley. Just judging based off of syntax, I'm already a fan. Side note - Drew, your Gemini capsule was one of the few I regularly check! I thought my internet was down when it wouldn't connect. Hope to see you back on sometime.

duped 4y ago
I think it's because the Hare authors seem to willingly ignore decades of PL research and practice. In this post the section on garbage collection and the notion of adding a borrow checker without changing the language seem a little misguided or even naive.
- ddevault 4y ago
  A lot of languages have been accused of "ignoring decades of PL research", an almost word-for-word criticism I've heard of a dozen languages or more. In fact, Hare includes a lot of innovations from newer PL research, but indeed not all of them, because different designers can approach a problem with different values and opinions. It's a deliberate design choice, not ignorance.
  - duped 4y ago
    I can respect intentional and principled design choices, but some of the comments you've made on things like garbage collection, borrow checking, generic programming, compiler optimizations, etc don't come off as such. It would be interesting to read some of the modern design choices that Hare is making, for me most of what I understand about Hare are the things you've chosen not to do. Which is interesting, but not necessarily the most compelling all the time. I also think two of the principles you laid out - ease of implementation and transparency/explicitness of code are the hottest takes in Hare. I can see why people like or dislike those choices, and it's compelling to think about, but I'm not sure that the explicit choices to add or not add features from PL research reflect those. For example the paragraph on GC - I think there's a ton of reasons not to use a garbage collector in a language like Hare (for example, you have to write a GC in some language and not the one that is GC'd) but I'm not sure the reasoning you laid out is among them (concurrent and hard-realtime GCs are thing, whether removing the burden of manual memory management is more/less transparent to the intent of a programmer is very debatable).

cpurdy 4y ago

Wonderfully written and explained. Kudos!

xigoi 4y ago

There's a typo: > This plays into Hare’s goals for explicitness and transparency, and improves the saftey and predictability of the language.

ddevault 4y ago
Thanks!

Zababa 4y ago

Interesting and well written, as always. I might be missing something, but I think that one approach that hasn't been discussed is reference counting. The explanation for why garbage collection is: > Garbage collection magically stops your code (sometimes at predictable points, but it’s generally not very explicit), which is not very transparent and is definitely off the table for use-cases like kernels, video games, real-time applications, and so on. The behavior of a Hare program should be easy to predict, and a garbage collector interferes with that goal. Thus, despite the fact that a garbage collector would improve safety and ease-of-use, it’s not a good fit for Hare. From what I understand, reference counting doesn't stop the code. However, it might not meet the goal of "Transparency and explicitness in code". Maybe it's too much for kernels, video games, real-time applications too.

ddevault 4y ago
Reference counting is still pretty high level, given that you have to find somewhere to put the count. It's not really suitable for kernels or some embedded applications, for instance. That makes it less desirable as a language feature. However, you can definitely implement reference counting in your own programs and libraries.
- Zababa 4y ago
  Thank you for the explanation, that makes sense.

lifthrasiir 4y ago

I'm happy to see this article, it does look like a much clearer explanation of Hare's pros and cons in terms of safety. I might not agree to every point but acknowledging and documenting trade-offs is surely a big plus. On borrow checkers: There is a big dichitomy between type systems and static analyses, where type systems aim to be sound---never fails to catch certain bugs, at expense of false positives---while static analyses tend not. But Rust's borrow checker is a sound static analysis combined with a type system. In fact it is a kind of optimal design when you are only allowed to add additional kind of types, and this restriction made the borrow checker a very impressive achievement. But there are other possibilities if you loosen a bit---either slightly unsound analysis [1] or annotation methods other than types. I believe Hare should better explore those possibilities instead of having a borrow checker. [1] In fact Rust has already compromised as well; for example its notion of memory safety doesn't ensure no memory leak. It is still unlikely but not impossible to unintentionally leak memory, so it is deemed acceptable. You can use a similar compromise in Hare.

tialaramex 4y ago
The existence of features like Box::leak() doesn't compromise Rust's memory safety which is why that's not marked unsafe. The reference counted and garbage collected languages likewise can leak, and if Box::leak() didn't exist, people would just roll their own by the same means. It's not even as though their hand-rolled mechanism would be syntactically heavier, you can't thing.leak() because leak intentionally isn't a method, it's just an associated function on Box and so you will need to explicitly Box::leak(thing) I don't think Hare's design is at all conducive to trying to retrofit a borrow checker, and so I would agree with you that it should not attempt such a thing.
- lifthrasiir 4y ago
  I mean, the memory safety is not an exactly defined term and it took a compromise to pick the exact definition of memory safety for Rust. I fully agree that the memory leak is far less risky than this notion of memory safety, but it is still a (slight) compromise especially when you are new to Rust. As a new language Hare doesn't have to take this granted.

goodpoint 4y ago

The interesting aspect is not having a language-specific package manager.

pwdisswordfish9 4y ago

> I consider Hare’s lack of a package manager to be an important security feature. Hare’s package manager is curl|bash? How is that a security feature?

deleted 4y ago
[deleted]
ddevault 4y ago
No. The article details that Hare's package manager is your system package manager, e.g. aptitude. Such package managers have independent signatures, features against partial updates, reproducible builds, and independent accountability, as well as many other advantages besides.
- pwdisswordfish9 4y ago
  People offering Hare packages will not bother to wait for distro repositories to graciously package them six months later. They will just tell their users to copy-paste curl-to-bash one-liners. And users will follow, because why would they want to use stale or outright non-existent distro packages? ‘But I am going to tell people not to do that!’ As if people are going to bow down before the unstoppable appeal of your charisma instead of following the path of least resistance. Of course, this is all assuming there will be meaningful external Hare packages in the first place.
  - ddevault 4y ago
    Use a distribution whose release cadence matches your needs. I use Alpine Linux, which has frequent updates on edge and biannual stable releases, which allows me to enjoy both approaches. Debian is designed for the longer term, Arch is designed to be bleeding edge. The idea that Linux distros have outdated software is a symptom of the fact that people are not choosing the right tool for the job, something that I am particularly unsympathetic to when thinking of an audience of software engineers. Hare is a conservative language which strongly encourages a conservative approach. We are not here to move fast and break things. Culture comes from the top-down, and while we may not succeed for all users, we can certainly set the standard of quality for the language. C has no successful independent package manager, yet we rarely if ever see people distributing C libraries with curl|bash.

mamcx 4y ago

In relation with the borrow checker, I think the complications on Rust are overblown at the end: Most of the complexity of Rust is in the interaction of all the others features AND when sometimes it interfere with the borrow checker. The borrow/mutability checker are fairly easy to use (as long you stay on simple Rust: Only enum, structs, functions and not get with traits + generics + lifetimes annotations, and today: async). Playing with the simple subset of Rust the borrow checker is fairly easy to get. I don't think nobody get stuck at this stage, only when start gettin ambitious and is not clear how ALL the features must intermix. If the language is truly simple, I think it will work wonders.

turminal 4y ago
I'm one of the Hare contributors researching the possibility of adding a borrow checker. I'm still at the start of that journey, and I'm looking at the problem implementation-wise rather than user-wise, but my impression so far regarding simplicity and issues that arise in interaction with other complex language features matches what you said.