3 ms·

Thank you for the reply. Do you mind a reasoned discussion? > A JIT with speculative optimisation and a moving GC. Idiomatic Rust, through its concepts of ow

Thank you for the reply. Do you mind a reasoned discussion? > A JIT with speculative optimisation and a moving GC. Idiomatic Rust, through its concepts of ownership and borrowing, encourages a pattern where you receive data as an argument or create it directly, perform operations on it, and then discard it via RAII. This bears some resemblance to functional programming. This approach does not apply to buffers of unknown size, which still require heap allocation; unfortunately, Rust lacks automatic buffer reuse. However, such optimization is theoretically possible. The stack is definitely faster than anything else. > This means that you need to translate data storage or code storage to hardware addresses, and that interferes with both moving collection and with JIT compilation. You don't need GC if you allocate data on stack. You also do not need to dereference the pointer. > dynamic dispatch You mentioned templates. In Rust, traits that are monomorphized - much like templates-are the standard approach; using vtables or `dyn trait` is a relatively rare use case. This stems from the fact that all code is known at compile time and there is no dynamic loading, allowing the compiler to eliminate polymorphism from the code entirely. > and shared heap objects This might be considered convenient, but in my view, it also leads to code that is harder to maintain when objects can be modified from multiple places. However, I think that is outside the scope of the current discussion. > We get around it with selective use of templates in C++ (or comptime in Zig), but it has to be selective, and it's viral. Yes, monomorphization is the default solution in Rust. It is not always viral either, because when using it, you often define specific types, and they do not spread beyond that scope. I suppose you could say that the programming style I am talking about is complex, inconvenient, unmaintainable, and so on. What I mean is, assuming this programming style is sufficiently convenient—and perhaps even has its own advantages - then none of the optimizations you listed offer an edge, and the Rust code will definitely be faster.

pron 18d ago
> However, such optimization is theoretically possible. The stack is definitely faster than anything else. What you're describing isn't a stack, but an automatic arena, and this optimisation is easier to do in Java. It's easier to do in Java because it requires setting a "current arena" or inlining, both of which Java can do more easily, and then either the arena will be heap allocated (which will be slower in Rust) or associated with the thread, which is not something low-level languages tend to do. > You don't need GC if you allocate data on stack. You also do not need to dereference the pointer. Moving collectors don't need to dereference anything (they don't know and don't want to know when an object is "dead"), and stack allocation works in both languages, only, as you pointed out, is not quite general (not every data structure with a known lifetime can be allocated on the stack). > You mentioned templates. In Rust, traits that are monomorphized - much like templates-are the standard approach; using vtables or `dyn trait` is a relatively rare use case. This stems from the fact that all code is known at compile time and there is no dynamic loading, allowing the compiler to eliminate polymorphism from the code entirely. Sure, except Java does this automatically, and it can do it more aggressively. Dynamic dispatch is rare in low-level languages because it's expensive in those languages. But it's not easy to avoid as programs get larger. That is exactly one of the problems in large programs that the JVM set out to solve. > This might be considered convenient, but in my view, it also leads to code that is harder to maintain when objects can be modified from multiple places. However, I think that is outside the scope of the current discussion. I agree that whether it has downsides is outside the scope of this discussion, but the point is that as programs evolve and grow, the abstractions tend to be more general, and low-level languages suffer from "abstraction cost", where the more general abstraction (which becomes more common over time) is more expensive. Again, this is exactly why large C++ programs suffered from performance issues and what the JVM tried to address. > Yes, monomorphization is the default solution in Rust. ... and in C++. But it is viral, and Java monomorphises without suffering from "zero overhead abstractions". The ability to move pointers, both to data and to code, opens up the possibility of using JITs and moving GCs, which are very powerful optimisations. A JIT does impose two further tradeoffs (aside from the need for an FFI layer), though, which are warmup and the possibility of deoptimisation. We can now cache the generated machine code from one execution to another (https://openjdk.org/jeps/544 https://openjdk.org/jeps/544), but the possibility of deoptimisation remains (in fact, it's what enables the aggressive speculative optimisations), which means you gain average (or even amortised) performance at the cost of the worst case. Anyway, the JVM was designed as a solution for the performance issues low-level languages suffer from as programs grow and/or evolve. It comes with tradeoffs, but those most affect small or short-lived programs. The thing to remember is that low-level languages are not optimised for performance but for low-level control (i.e. pointers are direct addresses etc.). Such control can translate to good performance when programs are small (see next) but it becomes a practical hindrance to performance when they're large. > I suppose you could say that the programming style I am talking about is complex, inconvenient, unmaintainable, and so on. What I mean is, assuming this programming style is sufficiently convenient—and perhaps even has its own advantages That advantage is a performance advantage. The question isn't "does there exist (in the mathematical sense) some program that is fast?" but "how fast is the program we can write within the budget we have?" When programs are small, manual optimisation is practical; when they grow large - not so much. And that's excluding the matter of a moving collector, which is just hard to compete with on speed regardless of program size, unless you use areans, but they're not at all easy to use in most low-level languages except Zig. > and the Rust code will definitely be faster. This is true only in the abstract mathematical sense. The reason we don't write programs that we want to be fast in Assembly (which is faster than anything in the same sense: for any program in any language, there exists and Assembly program that's at least as fast) is not because other languages are fast enough, but because in practice the programs we can actually write in the budget we have will be faster than the Assembly programs we could write. Of course, that could change when AI is able to generate perfect low-level code, but when that happens, it might as well generate machine code directly.
- someone_19 18d ago
  > Assembly (which is faster than anything in the same sense: for any program in any language, there exists and Assembly program that's at least as fast) At least you aren't claiming that the JVM is ~1.5 faster than perfectly written assembly :) I disagree with a lot of what you’re writing. However, we’ve reached the point where we need to run benchmarks and analyze the generated code (this is easy to do for compiled languages using https://godbolt.org/ https://godbolt.org/, but for the JVM, it can be a bit more complex, given the warm-up factor). So, there is one fundamental point I started with: > JIT is effective for languages where the source code lacks sufficient information (dynamic typing, where anything can be null) And your answer is: > A JIT can make such languages decently fast, but that's not how it's used in Java. In Java it is used for speculative optimisation, which allows far more aggressive optimisation than an AOT compiler can do. Essentially, you are saying that the compiler can apply aggressive optimizations when it knows what is happening in the code. But I say that JIT is needed so the compiler can figure out what is happening in the code and perform aggressive optimizations. There are many things that can be inferred from the code without needing to execute it. The question is how difficult it is to make such an inference: in one scenario, the compiler might attempt to track whether specific data changes-and, if it can prove this, mark the data as immutable and apply certain optimizations-whereas in another, it might already possess the information that the data is immutable. Moreover, information about immutability is useful not only to the compiler but also to the programmer. Just like information about types: it benefits both the compiler and the programmer. Imagine a fan of JS or Python joining our conversation and claiming that both Java and Rust are low-level languages because you have to specify types - something they view as complex and a hindrance to development speed. The same applies to the GC: the compiler can perform more optimizations when it knows when memory needs to be cleared (move it to stack or even place the data on registers). The JVM attempts to do this (via escape analysis), but there are limitations; consequently, data ends up on the heap, and GC operations come at a cost (due to data movement). Rust simply makes it easy to obtain far more information, enabling aggressive optimizations that are both immediate and guaranteed. There remain a small number of cases, such as `switch` statements - where one branch executes 99% of the time, while the other 99 branches execute only 1% of the time. In such instances, the JIT could indeed perform further optimizations; however, I am not even sure if the overhead of monitoring wouldn't outweigh the benefits. And the question is when and how to perform PGO, or whether to perform it at all.
gf000 18d ago
> The stack is definitely faster than anything else I have seen it mentioned everywhere, but is this actually true? I mean, of course it is faster than random cold memory, but is it actually faster than a hot, in-cache part of the heap? It is not special in any other way, AFAIK. And for what it's worth, what pron mentioned, Java uses a pretty similar structure for initial allocation, a thread local buffer where you just pointer bump. Another thread can then in the background copy still alive objects from this "arena" and then reset the whole thing.
- someone_19 18d ago
  > I have seen it mentioned everywhere, but is this actually true? Yes, it just adding or subtraction int to stack pointer register. I’m not certain, but the only thing that might be faster is accessing data at a fixed address - that is, global variables.
  - gf000 17d ago
    That's the way of getting the address itself, that's unrelated to how fast the actual memory read/write is. Stack is fast because it is frequently "touched" staying in cache. If you were to continuously read write a small segment of the heap, I don't think it would fair any worse than "the stack". This was my point

pron 18d ago

> However, such optimization is theoretically possible. The stack is definitely faster than anything else. What you're describing isn't a stack, but an automatic arena, and this optimisation is easier to do in Java. It's easier to do in Java because it requires setting a "current arena" or inlining, both of which Java can do more easily, and then either the arena will be heap allocated (which will be slower in Rust) or associated with the thread, which is not something low-level languages tend to do. > You don't need GC if you allocate data on stack. You also do not need to dereference the pointer. Moving collectors don't need to dereference anything (they don't know and don't want to know when an object is "dead"), and stack allocation works in both languages, only, as you pointed out, is not quite general (not every data structure with a known lifetime can be allocated on the stack). > You mentioned templates. In Rust, traits that are monomorphized - much like templates-are the standard approach; using vtables or `dyn trait` is a relatively rare use case. This stems from the fact that all code is known at compile time and there is no dynamic loading, allowing the compiler to eliminate polymorphism from the code entirely. Sure, except Java does this automatically, and it can do it more aggressively. Dynamic dispatch is rare in low-level languages because it's expensive in those languages. But it's not easy to avoid as programs get larger. That is exactly one of the problems in large programs that the JVM set out to solve. > This might be considered convenient, but in my view, it also leads to code that is harder to maintain when objects can be modified from multiple places. However, I think that is outside the scope of the current discussion. I agree that whether it has downsides is outside the scope of this discussion, but the point is that as programs evolve and grow, the abstractions tend to be more general, and low-level languages suffer from "abstraction cost", where the more general abstraction (which becomes more common over time) is more expensive. Again, this is exactly why large C++ programs suffered from performance issues and what the JVM tried to address. > Yes, monomorphization is the default solution in Rust. ... and in C++. But it is viral, and Java monomorphises without suffering from "zero overhead abstractions". The ability to move pointers, both to data and to code, opens up the possibility of using JITs and moving GCs, which are very powerful optimisations. A JIT does impose two further tradeoffs (aside from the need for an FFI layer), though, which are warmup and the possibility of deoptimisation. We can now cache the generated machine code from one execution to another (https://openjdk.org/jeps/544 https://openjdk.org/jeps/544), but the possibility of deoptimisation remains (in fact, it's what enables the aggressive speculative optimisations), which means you gain average (or even amortised) performance at the cost of the worst case. Anyway, the JVM was designed as a solution for the performance issues low-level languages suffer from as programs grow and/or evolve. It comes with tradeoffs, but those most affect small or short-lived programs. The thing to remember is that low-level languages are not optimised for performance but for low-level control (i.e. pointers are direct addresses etc.). Such control can translate to good performance when programs are small (see next) but it becomes a practical hindrance to performance when they're large. > I suppose you could say that the programming style I am talking about is complex, inconvenient, unmaintainable, and so on. What I mean is, assuming this programming style is sufficiently convenient—and perhaps even has its own advantages That advantage is a performance advantage. The question isn't "does there exist (in the mathematical sense) some program that is fast?" but "how fast is the program we can write within the budget we have?" When programs are small, manual optimisation is practical; when they grow large - not so much. And that's excluding the matter of a moving collector, which is just hard to compete with on speed regardless of program size, unless you use areans, but they're not at all easy to use in most low-level languages except Zig. > and the Rust code will definitely be faster. This is true only in the abstract mathematical sense. The reason we don't write programs that we want to be fast in Assembly (which is faster than anything in the same sense: for any program in any language, there exists and Assembly program that's at least as fast) is not because other languages are fast enough, but because in practice the programs we can actually write in the budget we have will be faster than the Assembly programs we could write. Of course, that could change when AI is able to generate perfect low-level code, but when that happens, it might as well generate machine code directly.

someone_19 18d ago
> Assembly (which is faster than anything in the same sense: for any program in any language, there exists and Assembly program that's at least as fast) At least you aren't claiming that the JVM is ~1.5 faster than perfectly written assembly :) I disagree with a lot of what you’re writing. However, we’ve reached the point where we need to run benchmarks and analyze the generated code (this is easy to do for compiled languages using https://godbolt.org/ https://godbolt.org/, but for the JVM, it can be a bit more complex, given the warm-up factor). So, there is one fundamental point I started with: > JIT is effective for languages where the source code lacks sufficient information (dynamic typing, where anything can be null) And your answer is: > A JIT can make such languages decently fast, but that's not how it's used in Java. In Java it is used for speculative optimisation, which allows far more aggressive optimisation than an AOT compiler can do. Essentially, you are saying that the compiler can apply aggressive optimizations when it knows what is happening in the code. But I say that JIT is needed so the compiler can figure out what is happening in the code and perform aggressive optimizations. There are many things that can be inferred from the code without needing to execute it. The question is how difficult it is to make such an inference: in one scenario, the compiler might attempt to track whether specific data changes-and, if it can prove this, mark the data as immutable and apply certain optimizations-whereas in another, it might already possess the information that the data is immutable. Moreover, information about immutability is useful not only to the compiler but also to the programmer. Just like information about types: it benefits both the compiler and the programmer. Imagine a fan of JS or Python joining our conversation and claiming that both Java and Rust are low-level languages because you have to specify types - something they view as complex and a hindrance to development speed. The same applies to the GC: the compiler can perform more optimizations when it knows when memory needs to be cleared (move it to stack or even place the data on registers). The JVM attempts to do this (via escape analysis), but there are limitations; consequently, data ends up on the heap, and GC operations come at a cost (due to data movement). Rust simply makes it easy to obtain far more information, enabling aggressive optimizations that are both immediate and guaranteed. There remain a small number of cases, such as `switch` statements - where one branch executes 99% of the time, while the other 99 branches execute only 1% of the time. In such instances, the JIT could indeed perform further optimizations; however, I am not even sure if the overhead of monitoring wouldn't outweigh the benefits. And the question is when and how to perform PGO, or whether to perform it at all.

gf000 18d ago

> The stack is definitely faster than anything else I have seen it mentioned everywhere, but is this actually true? I mean, of course it is faster than random cold memory, but is it actually faster than a hot, in-cache part of the heap? It is not special in any other way, AFAIK. And for what it's worth, what pron mentioned, Java uses a pretty similar structure for initial allocation, a thread local buffer where you just pointer bump. Another thread can then in the background copy still alive objects from this "arena" and then reset the whole thing.

someone_19 18d ago
> I have seen it mentioned everywhere, but is this actually true? Yes, it just adding or subtraction int to stack pointer register. I’m not certain, but the only thing that might be faster is accessing data at a fixed address - that is, global variables.
- gf000 17d ago
  That's the way of getting the address itself, that's unrelated to how fast the actual memory read/write is. Stack is fast because it is frequently "touched" staying in cache. If you were to continuously read write a small segment of the heap, I don't think it would fair any worse than "the stack". This was my point