24 ms·

(Aside about Zig, sorry. Although this applies to Go as well, I think?) Urgh I am so keen to switch to Zig but their attitude towards having vector operators ju

(Aside about Zig, sorry. Although this applies to Go as well, I think?) Urgh I am so keen to switch to Zig but their attitude towards having vector operators just completely kills the viability for me as a graphics programmer. I've asked in their Discord, Andrew Kelley himself passed on commenting (I know his stance, every C++ dev wants their fav feature), but the reality remains that it's just infeasible to do with a DSL so it's just the wrong language for writing graphics code.

Night_Thastus 3y ago
I'm a C++ dev, could you explain a bit of what is missing in Zig for vector operations? Does Zig not have operator overloading?
- klyrs 3y ago
  Nope, no operator overloading. As I understand it, the philosophy is to not hide O(n) behavior behind notation that looks O(1).
  - Night_Thastus 3y ago
    Strange. How is that different from say, a function call? A function may look like a single O(1) operation from the input/output/name, but actually do something much more complex. That seems like the same thing to me, and very common. (and frankly I'm not sure that could even be avoided)
    - klyrs 3y ago
      I didn't mean to volunteer to defend this choice, but without investigating a function you can't really support an opinion about its runtime. A language can make such promises about its basic syntax however. Perhaps I'll rephrase how I understand the philosophy: if it's a function call, it should look like a function call. Operator overloading breaks that. That said, this isn't my hill to die on. Edit to clarify my final sentence there: I have zero interest in debating this any further. Pixelpoet, if you're going to be so fussy, go read Harvey and van der Hoeven and stop trying to win language fights, they're tedious.
      - pixelpoet 3y ago
        I appreciate your being such a good devil's advocate, however as ForkMeOnTinder points out, there is already the super complicated[0] overloaded O(N^1.58+) operator for arbitrary precision integer multiplication, easily argued to be vanishingly less useful than simple vector operators, particularly if they were well-expressed at the IR level and well-mapped to modern hardware / instructions. The ask isn't for general operator overloading (I'm also in favour of not having that), rather just not stopping native algebraic type support at scalars; the C function atan2(y, x) basically just wants to give you the complex argument, for example. Really it would just do so much to unify and simplify everything, besides being able to write vector stuff in a sane way; if every rando has to write their own vector and complex number classes, I'm much less likely to vouch for its correctness. [0] I've recently been looking into Karatsuba multiplication to reduce it from O(N^2) to O(N^1.58): https://en.wikipedia.org/wiki/Karatsuba_algorithm https://en.wikipedia.org/wiki/Karatsuba_algorithm
        Night_Thastus 3y ago
        To me it does seem a weird hill to die on. If I'm using an operator on a non-primitive/non-trivial type - I'm going to consider the possibility that it does something more complex. It would be strange not to.
  - ForkMeOnTinder 3y ago
    And what's funny about that stance is mathematical operators aren't actually O(1). https://en.wikipedia.org/wiki/Computational_complexity_of_mathematical_operations https://en.wikipedia.org/wiki/Computational_complexity_of_ma... You don't want to know what the innocent-looking `*` in this function compiles to: fn square(num: u10000) u10000 { return num * num; }
    - pixelpoet 3y ago
      It's mildly offensive that u10000 is a builtin type but vec2f is not (also note that I am not asking for general operator overloading!). Which has dedicated CPU instructions across all modern architectures, what's the relative usage frequency, what's the cost/benefit of each, etc etc... :( And we all know why u10000 is in there: because you have to have arbitrary-width integers when writing a compiler, so they figured eh, we'll just expose that because we have it already. Absolutely transparent compiler-programming (cf. rendering-programming) bias, almost nobody else needs that. And yeah, I get that Add(Add(Add(a, b), Mul(c, 3)), d) is possible, but come on... imagine if you had to write your normal "a + b + c * 3 + d" with ints/floats like that! What's that, suddenly people care, but nobody cares if it's not their field... Whatever, I will continue to look longingly at Zig for all the bits of C/C++ (and apparently Go, to try bring it back to the original topic) it solves, but missing the trivial and absolutely critical single feature to enable an entire class of performance-critical programming.
      - defen 3y ago
        > And we all know why u10000 is in there: because you have to have arbitrary-width integers when writing a compiler You don't need them as a built-in type to write a compiler. They're there because LLVM was the original backend and you essentially get them for free (in the sense that the backend code generation is already handled for you, so why not include them).
      - bsder 3y ago
        > It's mildly offensive that u10000 is a builtin type but vec2f is not u10000 exists only because we want u3, u7, and u13 as builtin types. u3, u7, and u13 are useful for embedded and systems programming.
truckerbill 3y ago
It is strange that we stop at scalars for algebra in most languages. Odin’s approach is almost more frustrating, allowing for quaternion types, but not clifford algebra which can elegantly describe them and much more(though i know it’s a comically niche request at this point in time)
- deleted 3y ago
  [deleted]
- reactordev 3y ago
  agreed, further with today's parallelism it should be trivial but like you said, most languages stop at scalars and force the onus on you. I get it, higher order math is an implementation detail but to not provide the proper tools for those implementations is frustrating.
- pixelpoet 3y ago
  The main thing that stands out about this comment to me (agreeing 100% about algebraic types) is the "at this point in time" bit, such boundless optimism :D
ForkMeOnTinder 3y ago
That's one of the downsides of a BDFL. Zig is like 98% amazing, and 2% strange decisions that I think could have been avoided if the creator had more experience with languages other than C and Javascript. I'm still a happy Zig user though, and hey, there's still time before 1.0.
- pixelpoet 3y ago
  He's actually fluent in a lot of languages[0]. Just somehow apparently not much sympathy for numerical computing, or UI code (I'm fairly sure you want 2d vectors there too), or plotting, or ... [0] https://andrewkelley.me/post/not-a-js-developer.html https://andrewkelley.me/post/not-a-js-developer.html
- dilyevsky 3y ago
  Not supporting operator overloading is not a “strange decision” it’s widely hated feature outside of very specific domains (like maths and graphics) and every shop i worked at which bothered with a c++ guideline banned it
  - snovv_crash 3y ago
    Banned for in-house use, sure. Require an extra review before including a library, sure. But don't tell me you refuse to use Eigen and force everyone to write their own matrix multiplication routines that don't have operator overloading...
    - dilyevsky 3y ago
      That is why I specifically mentioned math and graphics as exceptions. Honestly I still like explicit DSL model better for these than trying to build dsl on top of your existing language c++/lisp style
  - pcwalton 3y ago
    But you have to use operator overloading to use smart pointers. This isn't a pedantic nerd snipe; the point is that operator overloading really is indispensable in certain contexts.
zoogeny 3y ago
I was watching a recent talk about the new Mojo language [1]. There is a section on SIMD and how they treat scalar values as a special case of vector operations (around the 33 min time). It does seem that tensors are one of the core abstractions for modern ML systems. I've heard people joke that AI is just matrix multiplication. Python is such a flexible language that creating abstractions around its syntax was super easy. I believe that was part of the reason Python has come to dominate this space (not the only reason obviously). I too felt the same as you, but as a distant admirer of Zig. I totally understand that operator overloading can be misused. But I have an intuition that at least providing operators for linear algebra (and probably complex/quaternion) is a really important feature for any languages from this point in history going forward. 1. https://www.youtube.com/watch?v=SEwTjZvy8vw&ab_channel=LLVM https://www.youtube.com/watch?v=SEwTjZvy8vw&ab_channel=LLVM
- sevagh 3y ago
  >I've heard people joke that AI is just matrix multiplication Not sure where the joke is, today's deep learning based AI models are literally matrix multiplications with learned weights.
- xh-dude 3y ago
  Julia has good ideas, interesting trade-offs here as well. ISTM the deep problem is that there’s so much room for optimization under composition of operations or with invariants not captured in types that the typical LinAlg library will never capture. But I agree most languages should be anticipating that users need something helpful here.
- andy99 3y ago
  providing operators for linear algebra (and probably complex/quaternion) is a really important feature for any languages from this point in history going forward. This is why I have started using Fortran for writing AI model inference code. It natively handles array manipulation like python and has a `matmul` intrinsic and compiles into fast code. There are some rough edges, but it's great as a low level matrix programming language which was its original point.
lll-o-lll 3y ago
Doesn’t this open github issue imply that vector operators are still being considered? https://github.com/ziglang/zig/issues/7295 https://github.com/ziglang/zig/issues/7295 Or is it that the three years it’s been around indicate it will never progress?
- pixelpoet 3y ago
  This is far better than my attempts at initiating the discussion, thanks for linking! Also it seems like even the discussion of Geometric Algebra has been covered before 3 years ago: https://github.com/ziglang/zig/issues/7295#issuecomment-738900412 https://github.com/ziglang/zig/issues/7295#issuecomment-7389... Unless it's actually possible after all this time (even in some random beta) to do vector code in infix notation, it doesn't feel like it's going to happen :(
el_oni 3y ago
Odinlang is a language in a similar space that seems to have first class support for matrix and vector operations. As well as having built in support for various graphics apps [0]. Seems like there is a bunch of interesting low level languages gaining steam at the moment. [0] https://odin-lang.org/news/major-graphics-apis/ https://odin-lang.org/news/major-graphics-apis/
- lenkite 3y ago
  Does Odin have support for async/concurrency ?
crest 3y ago
Didn't Zig "just" decide to permanently stay irrelevant for performance critical code by replacing LLVM by a yet to be written home grown optimiser and code generator? Don't get me wrong LLVM has lots of warts, but the a good multi architecture optimiser and code generator is a larger project than the frontend and standard library for any reasonable language.
- throwawaymaths 3y ago
  No. They are turning llvm into an "optional dependency", or plugin, or call it what you want. Day to day/debug compilation will be done with their own compiler and if you want you can cut a release with llvm to get all the optimizations when you're done, or at any time really. Zig won't ship with llvm as part of the standard download, but i imagine it will be easy to get zig+llvm working
__loam 3y ago
Am I crazy or is there nothing stopping us from writing functions that work with vectors? I don't really understand why this is a big headache for graphics programming other than it not being your preferred syntax.
renox 3y ago
What's really sad is that not having overloading "ON" by default is a good idea, but it doesn't mean that language has to stop there, it's really simple to have a small language feature which would say that a #everything_until_the_next_space b is rewritten in #everything_until_the_next_space(a,b) So you'd have: -a #foo b #foo c is foo(foo(a,b),c) -a #foo b #bar c is forbidden, use parenthesis. -and maybe #: for the inverse composition where a #:foo b #:foo c is foo(a, foo(b,c)) It's explicit, no hidden overloading, yet it's "short" enough to be usable: (a #* b) #+ c isn't that much worse than a*b+c and is much more readable than madd(mmul(a,b),c).. If there is no implicit conversion name clashes shouldn't be too bad (both #+ could be usable for matrix and for graphic libraries).

Night_Thastus 3y ago

I'm a C++ dev, could you explain a bit of what is missing in Zig for vector operations? Does Zig not have operator overloading?

klyrs 3y ago
Nope, no operator overloading. As I understand it, the philosophy is to not hide O(n) behavior behind notation that looks O(1).
- Night_Thastus 3y ago
  Strange. How is that different from say, a function call? A function may look like a single O(1) operation from the input/output/name, but actually do something much more complex. That seems like the same thing to me, and very common. (and frankly I'm not sure that could even be avoided)
  - klyrs 3y ago
    I didn't mean to volunteer to defend this choice, but without investigating a function you can't really support an opinion about its runtime. A language can make such promises about its basic syntax however. Perhaps I'll rephrase how I understand the philosophy: if it's a function call, it should look like a function call. Operator overloading breaks that. That said, this isn't my hill to die on. Edit to clarify my final sentence there: I have zero interest in debating this any further. Pixelpoet, if you're going to be so fussy, go read Harvey and van der Hoeven and stop trying to win language fights, they're tedious.
    - pixelpoet 3y ago
      I appreciate your being such a good devil's advocate, however as ForkMeOnTinder points out, there is already the super complicated[0] overloaded O(N^1.58+) operator for arbitrary precision integer multiplication, easily argued to be vanishingly less useful than simple vector operators, particularly if they were well-expressed at the IR level and well-mapped to modern hardware / instructions. The ask isn't for general operator overloading (I'm also in favour of not having that), rather just not stopping native algebraic type support at scalars; the C function atan2(y, x) basically just wants to give you the complex argument, for example. Really it would just do so much to unify and simplify everything, besides being able to write vector stuff in a sane way; if every rando has to write their own vector and complex number classes, I'm much less likely to vouch for its correctness. [0] I've recently been looking into Karatsuba multiplication to reduce it from O(N^2) to O(N^1.58): https://en.wikipedia.org/wiki/Karatsuba_algorithm https://en.wikipedia.org/wiki/Karatsuba_algorithm
      - Night_Thastus 3y ago
        To me it does seem a weird hill to die on. If I'm using an operator on a non-primitive/non-trivial type - I'm going to consider the possibility that it does something more complex. It would be strange not to.
- ForkMeOnTinder 3y ago
  And what's funny about that stance is mathematical operators aren't actually O(1). https://en.wikipedia.org/wiki/Computational_complexity_of_mathematical_operations https://en.wikipedia.org/wiki/Computational_complexity_of_ma... You don't want to know what the innocent-looking `*` in this function compiles to: fn square(num: u10000) u10000 { return num * num; }
  - pixelpoet 3y ago
    It's mildly offensive that u10000 is a builtin type but vec2f is not (also note that I am not asking for general operator overloading!). Which has dedicated CPU instructions across all modern architectures, what's the relative usage frequency, what's the cost/benefit of each, etc etc... :( And we all know why u10000 is in there: because you have to have arbitrary-width integers when writing a compiler, so they figured eh, we'll just expose that because we have it already. Absolutely transparent compiler-programming (cf. rendering-programming) bias, almost nobody else needs that. And yeah, I get that Add(Add(Add(a, b), Mul(c, 3)), d) is possible, but come on... imagine if you had to write your normal "a + b + c * 3 + d" with ints/floats like that! What's that, suddenly people care, but nobody cares if it's not their field... Whatever, I will continue to look longingly at Zig for all the bits of C/C++ (and apparently Go, to try bring it back to the original topic) it solves, but missing the trivial and absolutely critical single feature to enable an entire class of performance-critical programming.
    - defen 3y ago
      > And we all know why u10000 is in there: because you have to have arbitrary-width integers when writing a compiler You don't need them as a built-in type to write a compiler. They're there because LLVM was the original backend and you essentially get them for free (in the sense that the backend code generation is already handled for you, so why not include them).
    - bsder 3y ago
      > It's mildly offensive that u10000 is a builtin type but vec2f is not u10000 exists only because we want u3, u7, and u13 as builtin types. u3, u7, and u13 are useful for embedded and systems programming.

truckerbill 3y ago

It is strange that we stop at scalars for algebra in most languages. Odin’s approach is almost more frustrating, allowing for quaternion types, but not clifford algebra which can elegantly describe them and much more(though i know it’s a comically niche request at this point in time)

deleted 3y ago
[deleted]
reactordev 3y ago
agreed, further with today's parallelism it should be trivial but like you said, most languages stop at scalars and force the onus on you. I get it, higher order math is an implementation detail but to not provide the proper tools for those implementations is frustrating.
pixelpoet 3y ago
The main thing that stands out about this comment to me (agreeing 100% about algebraic types) is the "at this point in time" bit, such boundless optimism :D

ForkMeOnTinder 3y ago

That's one of the downsides of a BDFL. Zig is like 98% amazing, and 2% strange decisions that I think could have been avoided if the creator had more experience with languages other than C and Javascript. I'm still a happy Zig user though, and hey, there's still time before 1.0.

pixelpoet 3y ago
He's actually fluent in a lot of languages[0]. Just somehow apparently not much sympathy for numerical computing, or UI code (I'm fairly sure you want 2d vectors there too), or plotting, or ... [0] https://andrewkelley.me/post/not-a-js-developer.html https://andrewkelley.me/post/not-a-js-developer.html
dilyevsky 3y ago
Not supporting operator overloading is not a “strange decision” it’s widely hated feature outside of very specific domains (like maths and graphics) and every shop i worked at which bothered with a c++ guideline banned it
- snovv_crash 3y ago
  Banned for in-house use, sure. Require an extra review before including a library, sure. But don't tell me you refuse to use Eigen and force everyone to write their own matrix multiplication routines that don't have operator overloading...
  - dilyevsky 3y ago
    That is why I specifically mentioned math and graphics as exceptions. Honestly I still like explicit DSL model better for these than trying to build dsl on top of your existing language c++/lisp style
- pcwalton 3y ago
  But you have to use operator overloading to use smart pointers. This isn't a pedantic nerd snipe; the point is that operator overloading really is indispensable in certain contexts.

zoogeny 3y ago

I was watching a recent talk about the new Mojo language [1]. There is a section on SIMD and how they treat scalar values as a special case of vector operations (around the 33 min time). It does seem that tensors are one of the core abstractions for modern ML systems. I've heard people joke that AI is just matrix multiplication. Python is such a flexible language that creating abstractions around its syntax was super easy. I believe that was part of the reason Python has come to dominate this space (not the only reason obviously). I too felt the same as you, but as a distant admirer of Zig. I totally understand that operator overloading can be misused. But I have an intuition that at least providing operators for linear algebra (and probably complex/quaternion) is a really important feature for any languages from this point in history going forward. 1. https://www.youtube.com/watch?v=SEwTjZvy8vw&ab_channel=LLVM https://www.youtube.com/watch?v=SEwTjZvy8vw&ab_channel=LLVM

sevagh 3y ago
>I've heard people joke that AI is just matrix multiplication Not sure where the joke is, today's deep learning based AI models are literally matrix multiplications with learned weights.
xh-dude 3y ago
Julia has good ideas, interesting trade-offs here as well. ISTM the deep problem is that there’s so much room for optimization under composition of operations or with invariants not captured in types that the typical LinAlg library will never capture. But I agree most languages should be anticipating that users need something helpful here.
andy99 3y ago
providing operators for linear algebra (and probably complex/quaternion) is a really important feature for any languages from this point in history going forward. This is why I have started using Fortran for writing AI model inference code. It natively handles array manipulation like python and has a `matmul` intrinsic and compiles into fast code. There are some rough edges, but it's great as a low level matrix programming language which was its original point.

lll-o-lll 3y ago

Doesn’t this open github issue imply that vector operators are still being considered? https://github.com/ziglang/zig/issues/7295 https://github.com/ziglang/zig/issues/7295 Or is it that the three years it’s been around indicate it will never progress?

pixelpoet 3y ago
This is far better than my attempts at initiating the discussion, thanks for linking! Also it seems like even the discussion of Geometric Algebra has been covered before 3 years ago: https://github.com/ziglang/zig/issues/7295#issuecomment-738900412 https://github.com/ziglang/zig/issues/7295#issuecomment-7389... Unless it's actually possible after all this time (even in some random beta) to do vector code in infix notation, it doesn't feel like it's going to happen :(

el_oni 3y ago

Odinlang is a language in a similar space that seems to have first class support for matrix and vector operations. As well as having built in support for various graphics apps [0]. Seems like there is a bunch of interesting low level languages gaining steam at the moment. [0] https://odin-lang.org/news/major-graphics-apis/ https://odin-lang.org/news/major-graphics-apis/

lenkite 3y ago
Does Odin have support for async/concurrency ?

crest 3y ago

Didn't Zig "just" decide to permanently stay irrelevant for performance critical code by replacing LLVM by a yet to be written home grown optimiser and code generator? Don't get me wrong LLVM has lots of warts, but the a good multi architecture optimiser and code generator is a larger project than the frontend and standard library for any reasonable language.

throwawaymaths 3y ago
No. They are turning llvm into an "optional dependency", or plugin, or call it what you want. Day to day/debug compilation will be done with their own compiler and if you want you can cut a release with llvm to get all the optimizations when you're done, or at any time really. Zig won't ship with llvm as part of the standard download, but i imagine it will be easy to get zig+llvm working

__loam 3y ago

Am I crazy or is there nothing stopping us from writing functions that work with vectors? I don't really understand why this is a big headache for graphics programming other than it not being your preferred syntax.

renox 3y ago

What's really sad is that not having overloading "ON" by default is a good idea, but it doesn't mean that language has to stop there, it's really simple to have a small language feature which would say that a #everything_until_the_next_space b is rewritten in #everything_until_the_next_space(a,b) So you'd have: -a #foo b #foo c is foo(foo(a,b),c) -a #foo b #bar c is forbidden, use parenthesis. -and maybe #: for the inverse composition where a #:foo b #:foo c is foo(a, foo(b,c)) It's explicit, no hidden overloading, yet it's "short" enough to be usable: (a #* b) #+ c isn't that much worse than a*b+c and is much more readable than madd(mmul(a,b),c).. If there is no implicit conversion name clashes shouldn't be too bad (both #+ could be usable for matrix and for graphic libraries).