5 ms·

Value types are also addressed in the article

vvanders 9y ago
From what I can find[1] it looks like while Julia has compact values it doesn't have what I'd traditionally call value types. Specifically value types that live on the stack unless they are a member of a reference type(which is what C# does). Looking at the performance docs[2] this is pretty clear in that array types(which looks like how Julia does Matrices) get allocated on the heap. You can clearly see the performance impact in the docs(1.95s vs 0.08s) that this type of behavior has. While you can preallocate(which the docs suggest and is the only path for GC'd languages) it's not an ideal solution. If your type is smaller than a single cache line, like in the example above, you've just flushed a whole cache line just to bring in that one pre-allocated value. You also run into the issue if you don't know exactly how many values you need upfront which leads to pooling. In that scenario your pool may be large enough that you're bouncing between cache lines on different pooled objects. This is the type of thing that you really need control over if you want performance "as good as C". Anything less will be a compromise. There's also the whole class of zero-cost abstractions that can get from C++/Rust which leverage all the above to great effect. That lets you get things like nom[3] which gives you high level semantics + productivity while maintaining parity with C. None of this is academic, these are all optimizations I've used on shipping products that went out to millions of users. In each case we had fixed hardware with a limited execution budget and the 5-20x improvements we made were critical to us shipping a product that people wanted to use. [1] https://discourse.julialang.org/t/how-to-know-if-object-memory-resides-on-stack-or-heap/4927 https://discourse.julialang.org/t/how-to-know-if-object-memo... [2] https://docs.julialang.org/en/release-0.4/manual/performance-tips/#pre-allocating-outputs https://docs.julialang.org/en/release-0.4/manual/performance... [3] https://github.com/Geal/nom_benchmarks/tree/master/http https://github.com/Geal/nom_benchmarks/tree/master/http / https://github.com/Geal/nom https://github.com/Geal/nom
- KenoFischer 9y ago
  For values with compiler-visible scoped lifetime, the compiler will automatically promote them to stack variables. There is currently no way to enforce this happening, but it would be perfectly possible to add such an annotation. Regarding preallocation, you tend to want to avoid dynamic memory allocation in high performance applications anyway, so whether you do that in C++ or in Julia, doesn't really make too much of a difference. It is true this is a little harder to control in GC'ed languages than in languages where you have to do memory management manually, but the effect is about the same. Julia provides tools to figure out where you're using dynamic memory allocation and those tools will certainly improve in the future. One thing I think is under appreciated from the performance perspective though is how easily julia lets you express data layout transformation to take better advantage of the cache hierarchy. I touched on this a bit in my JuliaCon presentation [1]. [1] https://youtu.be/uecdcADM3hY?t=32m19s https://youtu.be/uecdcADM3hY?t=32m19s
  - vvanders 9y ago
    > Regarding preallocation, you tend to want to avoid dynamic memory allocation in high performance applications anyway, so whether you do that in C++ or in Julia, doesn't really make too much of a difference. Yes! Which is why it shouldn't take me digging through 3 different documents and still getting it partly wrong! C/C++/Rust make this easy by annotating the type modifiers(&/*/Box/etc) with how the object lives in my runtime system. Rust even gets bonus points for giving me aliasing information as well(&mut vs &).
    - ChrisRackauckas 9y ago
      You just do @. and you're done?
  - bjourne 9y ago
    > For values with compiler-visible scoped lifetime, the compiler will automatically promote them to stack variables. There is currently no way to enforce this happening, but it would be perfectly possible to add such an annotation. Other languages such as Java have that optimization too. But it is quite limited because it can only be applied to data whose size is known at compile-time. There is also the problem that if some data allocated in a procedure is passed to another procedure, it can't be allocated on the stack because it can escape. A programmer working in a low-level language can be smart and choose the best locations (stack or heap) for each piece of data. A language that does not allow explicit stack access (like C does), can never make as optimal decisions.
- StefanKarpinski 9y ago
  The distinction in Julia isn't between value versus reference types (which have fundamentally incompatible semantics), it's immutable types (declared using `struct`) versus mutable types (declared using `mutable struct`). Immutable types are generally stack allocated and need not even be fully materialized, whereas mutable types are typically heap allocated and fully materialized. The built-in array-type is mutable and can change size. As you say, these can be preallocated and modified in place with a rich collection of in-place, mutating algorithms in the standard library, but sometimes that's not quite enough. If you want fixed-size, stack-allocated arrays, you can use the StaticArrays package [1], which provides precisely such types. Aside from immutability, stack allocation and amazing performance, StaticArrays look and behave just like built-in arrays: one of the basic premises of Julia is to allow you to implement types like this and get the exact feature/performance tradeoff you need. [1] https://github.com/JuliaArrays/StaticArrays.jl https://github.com/JuliaArrays/StaticArrays.jl
  - vvanders 9y ago
    Uf, I really don't like intermixing mutability with allocation location. Those seem like two completely separate concerns. One thing that was really common for us to do was to instance a weighted graph(something like this[1]) per-actor. This means that you might have 10-300 floating point values in a block indexed by the node they interact with. It was really common to see one, maybe two values change on a per-frame basis. With the constraint above I'm now copying 300 floating point values every time any node changes which would be brutal for performance. Or I'd take a potential cache miss each time I touched the array which could be even worse if it was a reference type. To be clear, I'm not saying Julia isn't really good at what it does. My complaint stems from the fact that you can't claim performance good as/better than C without having all these tools at your disposal. We haven't even gotten into things like restrict[2] where things like Rust's ownership model let you get it for free[3]. [1] https://docs.unrealengine.com/latest/INT/Engine/Rendering/Materials/Editor/ https://docs.unrealengine.com/latest/INT/Engine/Rendering/Ma... [2] https://en.wikipedia.org/wiki/Restrict https://en.wikipedia.org/wiki/Restrict [3] https://doc.rust-lang.org/nomicon/aliasing.html https://doc.rust-lang.org/nomicon/aliasing.html
    - 9y ago

From what I can find[1] it looks like while Julia has compact values it doesn't have what I'd traditionally call value types. Specifically value types that live on the stack unless they are a member of a reference type(which is what C# does). Looking at the performance docs[2] this is pretty clear in that array types(which looks like how Julia does Matrices) get allocated on the heap. You can clearly see the performance impact in the docs(1.95s vs 0.08s) that this type of behavior has. While you can preallocate(which the docs suggest and is the only path for GC'd languages) it's not an ideal solution. If your type is smaller than a single cache line, like in the example above, you've just flushed a whole cache line just to bring in that one pre-allocated value. You also run into the issue if you don't know exactly how many values you need upfront which leads to pooling. In that scenario your pool may be large enough that you're bouncing between cache lines on different pooled objects. This is the type of thing that you really need control over if you want performance "as good as C". Anything less will be a compromise. There's also the whole class of zero-cost abstractions that can get from C++/Rust which leverage all the above to great effect. That lets you get things like nom[3] which gives you high level semantics + productivity while maintaining parity with C. None of this is academic, these are all optimizations I've used on shipping products that went out to millions of users. In each case we had fixed hardware with a limited execution budget and the 5-20x improvements we made were critical to us shipping a product that people wanted to use. [1] https://discourse.julialang.org/t/how-to-know-if-object-memory-resides-on-stack-or-heap/4927 https://discourse.julialang.org/t/how-to-know-if-object-memo... [2] https://docs.julialang.org/en/release-0.4/manual/performance-tips/#pre-allocating-outputs https://docs.julialang.org/en/release-0.4/manual/performance... [3] https://github.com/Geal/nom_benchmarks/tree/master/http https://github.com/Geal/nom_benchmarks/tree/master/http / https://github.com/Geal/nom https://github.com/Geal/nom

KenoFischer 9y ago
For values with compiler-visible scoped lifetime, the compiler will automatically promote them to stack variables. There is currently no way to enforce this happening, but it would be perfectly possible to add such an annotation. Regarding preallocation, you tend to want to avoid dynamic memory allocation in high performance applications anyway, so whether you do that in C++ or in Julia, doesn't really make too much of a difference. It is true this is a little harder to control in GC'ed languages than in languages where you have to do memory management manually, but the effect is about the same. Julia provides tools to figure out where you're using dynamic memory allocation and those tools will certainly improve in the future. One thing I think is under appreciated from the performance perspective though is how easily julia lets you express data layout transformation to take better advantage of the cache hierarchy. I touched on this a bit in my JuliaCon presentation [1]. [1] https://youtu.be/uecdcADM3hY?t=32m19s https://youtu.be/uecdcADM3hY?t=32m19s
- vvanders 9y ago
  > Regarding preallocation, you tend to want to avoid dynamic memory allocation in high performance applications anyway, so whether you do that in C++ or in Julia, doesn't really make too much of a difference. Yes! Which is why it shouldn't take me digging through 3 different documents and still getting it partly wrong! C/C++/Rust make this easy by annotating the type modifiers(&/*/Box/etc) with how the object lives in my runtime system. Rust even gets bonus points for giving me aliasing information as well(&mut vs &).
  - ChrisRackauckas 9y ago
    You just do @. and you're done?
- bjourne 9y ago
  > For values with compiler-visible scoped lifetime, the compiler will automatically promote them to stack variables. There is currently no way to enforce this happening, but it would be perfectly possible to add such an annotation. Other languages such as Java have that optimization too. But it is quite limited because it can only be applied to data whose size is known at compile-time. There is also the problem that if some data allocated in a procedure is passed to another procedure, it can't be allocated on the stack because it can escape. A programmer working in a low-level language can be smart and choose the best locations (stack or heap) for each piece of data. A language that does not allow explicit stack access (like C does), can never make as optimal decisions.
StefanKarpinski 9y ago
The distinction in Julia isn't between value versus reference types (which have fundamentally incompatible semantics), it's immutable types (declared using `struct`) versus mutable types (declared using `mutable struct`). Immutable types are generally stack allocated and need not even be fully materialized, whereas mutable types are typically heap allocated and fully materialized. The built-in array-type is mutable and can change size. As you say, these can be preallocated and modified in place with a rich collection of in-place, mutating algorithms in the standard library, but sometimes that's not quite enough. If you want fixed-size, stack-allocated arrays, you can use the StaticArrays package [1], which provides precisely such types. Aside from immutability, stack allocation and amazing performance, StaticArrays look and behave just like built-in arrays: one of the basic premises of Julia is to allow you to implement types like this and get the exact feature/performance tradeoff you need. [1] https://github.com/JuliaArrays/StaticArrays.jl https://github.com/JuliaArrays/StaticArrays.jl
- vvanders 9y ago
  Uf, I really don't like intermixing mutability with allocation location. Those seem like two completely separate concerns. One thing that was really common for us to do was to instance a weighted graph(something like this[1]) per-actor. This means that you might have 10-300 floating point values in a block indexed by the node they interact with. It was really common to see one, maybe two values change on a per-frame basis. With the constraint above I'm now copying 300 floating point values every time any node changes which would be brutal for performance. Or I'd take a potential cache miss each time I touched the array which could be even worse if it was a reference type. To be clear, I'm not saying Julia isn't really good at what it does. My complaint stems from the fact that you can't claim performance good as/better than C without having all these tools at your disposal. We haven't even gotten into things like restrict[2] where things like Rust's ownership model let you get it for free[3]. [1] https://docs.unrealengine.com/latest/INT/Engine/Rendering/Materials/Editor/ https://docs.unrealengine.com/latest/INT/Engine/Rendering/Ma... [2] https://en.wikipedia.org/wiki/Restrict https://en.wikipedia.org/wiki/Restrict [3] https://doc.rust-lang.org/nomicon/aliasing.html https://doc.rust-lang.org/nomicon/aliasing.html
  - 9y ago