4 ms·

I hear you, but I think the problem is that you're framing this as "I, the developer, don't want to accept these costs". And that's fine when the software doesn

I hear you, but I think the problem is that you're framing this as "I, the developer, don't want to accept these costs". And that's fine when the software doesn't leave your system. The problem is that you're them pushing other costs onto your users ie: exploitable software. So from the developer perspective, great, it works for you, but the cost is there. I'm sympathetic to not wanting to use the other languages available, I'm not saying that any other language is doing things the "right" way, there's room for a lot of improvement. But I personally think that setting out to build new systems software in a memory unsafe language is setting users up for very serious harm.

vladharbuz 4y ago
I think I understand your view better now. Are you aware of any current memory management strategies (implemented as part of a language or otherwise) that perform well in situations with high performance requirements? For example, as someone who works on video games and real-time audio, most options seem non-starters to me aside from Rust, even if I decided to make sacrifices for the sake of security, and I at least have the impression I've explored this space quite a bit. Anyway, I would be happy to learn more about minimal memory safety strategies that don't require massive scaffolding and also allow for high-performance situations.
- staticassertion 4y ago
  Not in mainstream languages. There's a lot of ongoing research in the space. Otherwise, Rust is probably the most mainstream language that achieves your goals. Games are a bit different imo. While they're often networked they tend to not get attacked the same way as other software for a variety of reasons (though some games become so popular that it becomes worthwhile, like Minecraft). If a language set out to be "safer" (ie: improve temporal safety) but still prioritized performance, and emphasized its use case as being gaming, or explicitly for non-security-sensitive use cases, I'd be a lot more onboard with that. Jai seems to be driving towards that. My issue with Hare is that it's presented (both on its page and in this HN thread) as being a language for general systems work.
  - vladharbuz 4y ago
    Thank you for all of your feedback, I hope you end up at least trying Hare for the use cases that feel right to you! :)
    - staticassertion 4y ago
      I appreciate that. I really do hate to be critical of open source work. I just feel this is an important issue.
      - mrtweetyhack 4y ago
- nyanpasu64 4y ago
  Not staticassertion, but I'm a hobbyist in real-time audio. I like Rust as a vocabulary for describing/teaching safe programming (&/&mut/Send/Sync). I find that multithreaded programs written in Rust are usually correct while multithreaded programs written in C++ are usually wrong, because Rust encodes the rules of shared-memory threading in its type system (&T: Sync objects are thread-shared, but are either immutable or atomic or requires locking to acquire a &mut T). I also appreciate guiding users towards exclusive references (&mut) to make it easier to reason about code. However I find it makes it too difficult to mutate through shared references or write unsafe code (passing Stacked Borrows while lending out &mut is more like solving puzzles than writing imperative code, and writing code that never touches & and &mut is a quagmire of addr_of_mut!() and unsafe blocks on every pointer dereference), and the Rust community appears uninterested in making unsafe programming more ergonomic. Personally I'm a fan of C++'s unique_ptr/& as an unsafe escape hatch from Rust's single ownership or runtime refcounting overhead. It's at least as safe as Rust's unsafe pointers, and far more pleasant to use. Qt's QObject ownership system is reasonably ergonomic and QPointer is fun (though dangerous since it can turn into null unexpectedly), but Qt uses it pervasively (rather than only when safe memory management fails), relies on prose documentation to describe which pointers transfer ownership or not (resulting in memory management bugs), and QObject child destruction and nulling-out QPointers relies on runtime overhead. I haven't tried ECS or generational indexes yet, but those are popular in games, and Vale has its own ideas in this field (https://verdagon.dev/blog/generational-references https://verdagon.dev/blog/generational-references). On an aesthetic/principled level, I'd rather punt alias analysis to the programmer (pointer/restrict or &/&mut) rather than compiler complexity/magic (TBAA and provenance checking). Glancing at https://harelang.org/specification/ https://harelang.org/specification/, it seems Hare lacks an equivalent of restrict/&mut, and I wonder if that prevents the compiler from ever adding support for removing redundant loads/stores through pointers.
  - zozbot234 4y ago
    > However I find it makes it too difficult to mutate through shared references It's not that difficult, you just need to use UnsafeCell<…> or one of its safe derivatives (each of which has some potential runtime overhead) to keep the semantics tractable.
- kaba0 4y ago
  I’m sure it is not the answer you want to hear, but partial use of GCs seems to be exactly that. Modern GCs have insanely good throughput OR latency. Quite a few languages have value types now, with that you can restrict your usage to stack allocations for the critical hot loops, while low-latency GCs promise less pauses than the OS itself, which should be plenty good for even the most demanding games.
  - vladharbuz 4y ago
    Hey, I'm open to any answer that helps me write better programs. :) Which languages do you have experience working with in high-performance situations? I, for one, had high hopes for using Go for video game development, but it turns out that even in highly-tuned Go code with the latest GC optimisations, there are still significant GC pauses that cannot be overcome [0]. However, perhaps you're referring to other types of GCs I'm not aware of? [0]: https://ebiten.org/blog/native_compiling_for_nintendo_switch.html https://ebiten.org/blog/native_compiling_for_nintendo_switch...
    - kaba0 4y ago
      I don’t have much experience with C#, but currently that seems to have the best balance of control over allocations and a performant GC due to having value types (and pointers as well if I’m not mistaken?) But regarding GC, Java is unquestionably the king in that aspect, throughput-wise G1 is unbeatable and its relatively new ZGC might be of interest to use. It is the one I thought about previously, it currently promises sub-millisecond max pause times and this pause time doesn’t grow with heap size. Unfortunately Java doesn’t have value types yet, so you either write your hot loops with only primitives and allocations you make sure gets optimized by the escape-analyser, or do some manual memory management with the new Panama APIs, which are quite friendly in my opinion. EDIT: Just read your link, while Java can be AOT-compiled with GraalVM, only the enterprise version supports some of the more exotic GC-variants (though not sure about ZGC). It should be free for personal use, but do have a look at it. Though what I wrote concern mostly running code with the JVM.
      - 4y ago

vladharbuz 4y ago

I think I understand your view better now. Are you aware of any current memory management strategies (implemented as part of a language or otherwise) that perform well in situations with high performance requirements? For example, as someone who works on video games and real-time audio, most options seem non-starters to me aside from Rust, even if I decided to make sacrifices for the sake of security, and I at least have the impression I've explored this space quite a bit. Anyway, I would be happy to learn more about minimal memory safety strategies that don't require massive scaffolding and also allow for high-performance situations.

staticassertion 4y ago
Not in mainstream languages. There's a lot of ongoing research in the space. Otherwise, Rust is probably the most mainstream language that achieves your goals. Games are a bit different imo. While they're often networked they tend to not get attacked the same way as other software for a variety of reasons (though some games become so popular that it becomes worthwhile, like Minecraft). If a language set out to be "safer" (ie: improve temporal safety) but still prioritized performance, and emphasized its use case as being gaming, or explicitly for non-security-sensitive use cases, I'd be a lot more onboard with that. Jai seems to be driving towards that. My issue with Hare is that it's presented (both on its page and in this HN thread) as being a language for general systems work.
- vladharbuz 4y ago
  Thank you for all of your feedback, I hope you end up at least trying Hare for the use cases that feel right to you! :)
  - staticassertion 4y ago
    I appreciate that. I really do hate to be critical of open source work. I just feel this is an important issue.
    - mrtweetyhack 4y ago
nyanpasu64 4y ago
Not staticassertion, but I'm a hobbyist in real-time audio. I like Rust as a vocabulary for describing/teaching safe programming (&/&mut/Send/Sync). I find that multithreaded programs written in Rust are usually correct while multithreaded programs written in C++ are usually wrong, because Rust encodes the rules of shared-memory threading in its type system (&T: Sync objects are thread-shared, but are either immutable or atomic or requires locking to acquire a &mut T). I also appreciate guiding users towards exclusive references (&mut) to make it easier to reason about code. However I find it makes it too difficult to mutate through shared references or write unsafe code (passing Stacked Borrows while lending out &mut is more like solving puzzles than writing imperative code, and writing code that never touches & and &mut is a quagmire of addr_of_mut!() and unsafe blocks on every pointer dereference), and the Rust community appears uninterested in making unsafe programming more ergonomic. Personally I'm a fan of C++'s unique_ptr/& as an unsafe escape hatch from Rust's single ownership or runtime refcounting overhead. It's at least as safe as Rust's unsafe pointers, and far more pleasant to use. Qt's QObject ownership system is reasonably ergonomic and QPointer is fun (though dangerous since it can turn into null unexpectedly), but Qt uses it pervasively (rather than only when safe memory management fails), relies on prose documentation to describe which pointers transfer ownership or not (resulting in memory management bugs), and QObject child destruction and nulling-out QPointers relies on runtime overhead. I haven't tried ECS or generational indexes yet, but those are popular in games, and Vale has its own ideas in this field (https://verdagon.dev/blog/generational-references https://verdagon.dev/blog/generational-references). On an aesthetic/principled level, I'd rather punt alias analysis to the programmer (pointer/restrict or &/&mut) rather than compiler complexity/magic (TBAA and provenance checking). Glancing at https://harelang.org/specification/ https://harelang.org/specification/, it seems Hare lacks an equivalent of restrict/&mut, and I wonder if that prevents the compiler from ever adding support for removing redundant loads/stores through pointers.
- zozbot234 4y ago
  > However I find it makes it too difficult to mutate through shared references It's not that difficult, you just need to use UnsafeCell<…> or one of its safe derivatives (each of which has some potential runtime overhead) to keep the semantics tractable.
kaba0 4y ago
I’m sure it is not the answer you want to hear, but partial use of GCs seems to be exactly that. Modern GCs have insanely good throughput OR latency. Quite a few languages have value types now, with that you can restrict your usage to stack allocations for the critical hot loops, while low-latency GCs promise less pauses than the OS itself, which should be plenty good for even the most demanding games.
- vladharbuz 4y ago
  Hey, I'm open to any answer that helps me write better programs. :) Which languages do you have experience working with in high-performance situations? I, for one, had high hopes for using Go for video game development, but it turns out that even in highly-tuned Go code with the latest GC optimisations, there are still significant GC pauses that cannot be overcome [0]. However, perhaps you're referring to other types of GCs I'm not aware of? [0]: https://ebiten.org/blog/native_compiling_for_nintendo_switch.html https://ebiten.org/blog/native_compiling_for_nintendo_switch...
  - kaba0 4y ago
    I don’t have much experience with C#, but currently that seems to have the best balance of control over allocations and a performant GC due to having value types (and pointers as well if I’m not mistaken?) But regarding GC, Java is unquestionably the king in that aspect, throughput-wise G1 is unbeatable and its relatively new ZGC might be of interest to use. It is the one I thought about previously, it currently promises sub-millisecond max pause times and this pause time doesn’t grow with heap size. Unfortunately Java doesn’t have value types yet, so you either write your hot loops with only primitives and allocations you make sure gets optimized by the escape-analyser, or do some manual memory management with the new Panama APIs, which are quite friendly in my opinion. EDIT: Just read your link, while Java can be AOT-compiled with GraalVM, only the enterprise version supports some of the more exotic GC-variants (though not sure about ZGC). It should be free for personal use, but do have a look at it. Though what I wrote concern mostly running code with the JVM.
    - 4y ago