3 ms·

That's always been true in all the safe languages with unsafe escape hatches, except here these "primitives" are the main reason to reach for a low-level langua

by pron 7d ago

That's always been true in all the safe languages with unsafe escape hatches, except here these "primitives" are the main reason to reach for a low-level language in the first place - because they presumably require the control that low-level languages offer. Combining them in the same language might appeal to some and not to others who think that the high-level, safe parts are unnecessarily complicated because it needs to integrate with the low-level parts, and the low-level parts are unnecessarily complicated because they need to integrate with the safe parts. Anyway, some like this and some don't, but my point is that it's not "mostly solved". > What are you talking about? I've never encountered this and I've been using Rust for 10 years. Okay, but I've been doing low-level programming professionally for 25 years, and have encountered this over and over in large programs (over 500KLOC) as they evolve.

treyd 7d ago
That's just not an accurate description of how you write Rust in practice. There's no separate "high level" and "low level" parts/forms of the language any more than the software development process already is all about building abstractions. You should be doing this in Zig, too. It's just that sometimes some of the abstractions you need to build go outside what the ownership and borrowing system can model. And when you don't need to do that (which is 99% of the time) you also get all the benefits of the ownership/borrow system for free.
- pron 7d ago
  They're not separate forms but they are separate modes, and it is precisely because the language tries to fit both these modes into the same language that both suffer. I fully understand the goal of trying to unify these modes into the same language (C++ does the same thing), but there have always been very experienced people who like this approach and those who dislike it, hence it's not "solved". Something is solved when there's a broad consensus it's solved, and there isn't one here. I mean, someone can think it's solved for them, but if they're asking why others don't see it the same way and why many expert low-level programmers are at least intrigued by Zig, this is why. I prefer a simpler high-performance high-level language for high-level things, and a simpler low-level language for low-level things, and I dislike the C++/Rust approach of combining them into one complicated language. Some may think you get the best of both worlds; others, like me, think you get the worst of both worlds.
  - treyd 7d ago
    Can you point to a specific example that ends up being a "worst of both worlds" in your perspective?
    - pron 7d ago
      I don't know exactly how specific you want to be, but sure, because we've come across this countless times in C++, which suffers from the exact same problem. Suppose you're writing a program that's mostly high-level, say some kind of concurrent server, and it's large-ish, say around 1MLOC (most C++ programs I've worked on were significantly larger). Because the language is also a low-level language, it has low-level constraints, so: 1. It needs to use an AOT compiler, and consequently to get good performance you need to use less general mechanisms, such as direct (as opposed to dynamic) dispatch and even manual monorphisation (with generics/templates). These are viral, so they have to be carefully chosen (you can't monomorphise everything or you'll get machine code explosion). Five years later you need to make a big change that requires more generality, and then you either have to reconsider all of your manual optimisations, which is expensive, or go for more general constructs (dynamic dispatch) and the program gets slower. 2. It needs to use machine pointers (i.e. you can't enjoy a moving GC), and so you try to use the stack as much as possible (which you can't really do for anything dynamic), or suffer the high cost of malloc/free on individual objects. As the program evolves, you need to make things more general, and objects that could live on the stack now need to go on the heap, and objects that lived on the heap now may need to be shared among threads, in which case you often add the additional cost of refcounting GC. Of course, you want to use arenas in many cases, but they're very, very hard to use in C++ and Rust. You'd be better off - performance-wise and maintenance-wise - with a good optimising JIT and a moving GC. This was exactly a problem with many C++ programs that didn't really need a lot of direct hardware interaction - everything worked great for a few years, and then the evolution and maintenance costs became really high (or the programs became slow). Now suppose you're writing something low-level, i.e. you really need to interact with the hardware and/or OS directly a lot, and want to control everything - where everything is in memory, exactly when it's initialised, exactly when it's freed, exactly which operations are executed and when. But now you have a language that's also high-level, so it has a lot of implicitness that hides from you the things you want to see (and in Rust's case, you lose the safety). Best case scenario, you rely on disciplne and avoid implicit features, but then you also need to avoid much of the standard library. Anyway, combining high and low level in the same language was C++'s dream: one language for everything. Of course, for a while we didn't know about the maintenance problems, as those appear only years down the line, but more importantly, there weren't really high-performance high-level languages back then. These days, with lessons learnt and with more options, I prefer a language that focuses on being high-level for high-level stuff, and a language that focuses on low-level for low-level stuff. If you really need both kinds, use two languages.
brabel 6d ago
> I've been doing low-level programming professionally for 25 years You haven't been doing any Rust though. You seem to think you can extrapolate your C++ experience to Rust. That's preposterous. The actual Rust programmers can't recognize this theoretical problem in their Rust programs.
- Pannoniae 6d ago
  Could that be because the language is fairly young? You don't see the "20-year-old legacy system" in Rust because it doesn't exist yet ;) And if you look at other comments in this thread, many engineers have this mentality of "just use a crate, it's probably optimised already". They might not have performance problems immediately or obviously but it's more like ten thousand papercuts - a few allocations here and there, a few extra copies here and there and you've got a way slower program than it should have been.
  - pron 6d ago
    Well, the problems don't start after 20 years but after 5 or so (depending on the size of the codebase and the rate of the application's evolution), and the reason there aren't many large and oldish Rust codebases isn't because the language is too young for that (work on it began twenty years ago, and it's been stable for over a decade); that's middle-aged for a programming language. When C++ was of a similar age, there were thousands of >1MLOC programs written in it. One reason is obviously because when C++ was of the same age, there weren't as many suitable high-level alternatives, and people just don't pick a low-level language for most large applications anymore. But most Rust fans at least on social media, have not actually had much experience with it or with low-level programming in general; I'm guessing most haven't worked on Rust projects with more than 10 full-time people on them (this isn't normal in the industry, BTW, as a lot of software lives in large programs). And again, there are people who can certainly live with these issues, but they are real, and many certainly find them troubling.
- pron 6d ago
  It's not theoretical, it's one of the main reasons many large applications abandoned C++, and there's absolutely no reason for it to not exist in Rust. All low-level languages suffer from expensive evolution for fundamental reasons - the reliance on an AOT compiler and the lack of movable pointers impose serious performance tradeoffs in large programs. Optimising JITs and moving GCs were invented, in large part, to address this very real problem, familiar to many low-level programmers who have maintained large codebases for a long time. It's also why large runtimes like TCMalloc were invented to assist as much as they can. Most actual Rust programmers haven't maintained a large Rust program for a long time. Now, don't get me wrong - there are many C++ programmers who are fine with it, but many who aren't. What I find annoying is people without much experience in Rust assuming that everyone or almost everyone should like it, even though that's never been true for any language. I'm not saying Rust is bad by any means; in fact, I think it's better than C++ in a few ways. I'm explaining why I don't like it.

by pron 7d ago

treyd 7d ago

That's just not an accurate description of how you write Rust in practice. There's no separate "high level" and "low level" parts/forms of the language any more than the software development process already is all about building abstractions. You should be doing this in Zig, too. It's just that sometimes some of the abstractions you need to build go outside what the ownership and borrowing system can model. And when you don't need to do that (which is 99% of the time) you also get all the benefits of the ownership/borrow system for free.

pron 7d ago
They're not separate forms but they are separate modes, and it is precisely because the language tries to fit both these modes into the same language that both suffer. I fully understand the goal of trying to unify these modes into the same language (C++ does the same thing), but there have always been very experienced people who like this approach and those who dislike it, hence it's not "solved". Something is solved when there's a broad consensus it's solved, and there isn't one here. I mean, someone can think it's solved for them, but if they're asking why others don't see it the same way and why many expert low-level programmers are at least intrigued by Zig, this is why. I prefer a simpler high-performance high-level language for high-level things, and a simpler low-level language for low-level things, and I dislike the C++/Rust approach of combining them into one complicated language. Some may think you get the best of both worlds; others, like me, think you get the worst of both worlds.
- treyd 7d ago
  Can you point to a specific example that ends up being a "worst of both worlds" in your perspective?
  - pron 7d ago
    I don't know exactly how specific you want to be, but sure, because we've come across this countless times in C++, which suffers from the exact same problem. Suppose you're writing a program that's mostly high-level, say some kind of concurrent server, and it's large-ish, say around 1MLOC (most C++ programs I've worked on were significantly larger). Because the language is also a low-level language, it has low-level constraints, so: 1. It needs to use an AOT compiler, and consequently to get good performance you need to use less general mechanisms, such as direct (as opposed to dynamic) dispatch and even manual monorphisation (with generics/templates). These are viral, so they have to be carefully chosen (you can't monomorphise everything or you'll get machine code explosion). Five years later you need to make a big change that requires more generality, and then you either have to reconsider all of your manual optimisations, which is expensive, or go for more general constructs (dynamic dispatch) and the program gets slower. 2. It needs to use machine pointers (i.e. you can't enjoy a moving GC), and so you try to use the stack as much as possible (which you can't really do for anything dynamic), or suffer the high cost of malloc/free on individual objects. As the program evolves, you need to make things more general, and objects that could live on the stack now need to go on the heap, and objects that lived on the heap now may need to be shared among threads, in which case you often add the additional cost of refcounting GC. Of course, you want to use arenas in many cases, but they're very, very hard to use in C++ and Rust. You'd be better off - performance-wise and maintenance-wise - with a good optimising JIT and a moving GC. This was exactly a problem with many C++ programs that didn't really need a lot of direct hardware interaction - everything worked great for a few years, and then the evolution and maintenance costs became really high (or the programs became slow). Now suppose you're writing something low-level, i.e. you really need to interact with the hardware and/or OS directly a lot, and want to control everything - where everything is in memory, exactly when it's initialised, exactly when it's freed, exactly which operations are executed and when. But now you have a language that's also high-level, so it has a lot of implicitness that hides from you the things you want to see (and in Rust's case, you lose the safety). Best case scenario, you rely on disciplne and avoid implicit features, but then you also need to avoid much of the standard library. Anyway, combining high and low level in the same language was C++'s dream: one language for everything. Of course, for a while we didn't know about the maintenance problems, as those appear only years down the line, but more importantly, there weren't really high-performance high-level languages back then. These days, with lessons learnt and with more options, I prefer a language that focuses on being high-level for high-level stuff, and a language that focuses on low-level for low-level stuff. If you really need both kinds, use two languages.

brabel 6d ago

> I've been doing low-level programming professionally for 25 years You haven't been doing any Rust though. You seem to think you can extrapolate your C++ experience to Rust. That's preposterous. The actual Rust programmers can't recognize this theoretical problem in their Rust programs.

Pannoniae 6d ago
Could that be because the language is fairly young? You don't see the "20-year-old legacy system" in Rust because it doesn't exist yet ;) And if you look at other comments in this thread, many engineers have this mentality of "just use a crate, it's probably optimised already". They might not have performance problems immediately or obviously but it's more like ten thousand papercuts - a few allocations here and there, a few extra copies here and there and you've got a way slower program than it should have been.
- pron 6d ago
  Well, the problems don't start after 20 years but after 5 or so (depending on the size of the codebase and the rate of the application's evolution), and the reason there aren't many large and oldish Rust codebases isn't because the language is too young for that (work on it began twenty years ago, and it's been stable for over a decade); that's middle-aged for a programming language. When C++ was of a similar age, there were thousands of >1MLOC programs written in it. One reason is obviously because when C++ was of the same age, there weren't as many suitable high-level alternatives, and people just don't pick a low-level language for most large applications anymore. But most Rust fans at least on social media, have not actually had much experience with it or with low-level programming in general; I'm guessing most haven't worked on Rust projects with more than 10 full-time people on them (this isn't normal in the industry, BTW, as a lot of software lives in large programs). And again, there are people who can certainly live with these issues, but they are real, and many certainly find them troubling.
pron 6d ago
It's not theoretical, it's one of the main reasons many large applications abandoned C++, and there's absolutely no reason for it to not exist in Rust. All low-level languages suffer from expensive evolution for fundamental reasons - the reliance on an AOT compiler and the lack of movable pointers impose serious performance tradeoffs in large programs. Optimising JITs and moving GCs were invented, in large part, to address this very real problem, familiar to many low-level programmers who have maintained large codebases for a long time. It's also why large runtimes like TCMalloc were invented to assist as much as they can. Most actual Rust programmers haven't maintained a large Rust program for a long time. Now, don't get me wrong - there are many C++ programmers who are fine with it, but many who aren't. What I find annoying is people without much experience in Rust assuming that everyone or almost everyone should like it, even though that's never been true for any language. I'm not saying Rust is bad by any means; in fact, I think it's better than C++ in a few ways. I'm explaining why I don't like it.