3 ms·

refer to https://www.youtube.com/watch?v=HNEm8zmkjBU https://www.youtube.com/watch?v=HNEm8zmkjBU

dzaima 3y ago
That's a bit dramatic. Yes, RVV does add to the problem of the same binary running differently on different implementations, but it's not like that doesn't already happen a lot. The main thing that sucks about it imo is that you can't test a piece of code on a single implementation and be sure it'll work the same on all (though I'd imagine CPUs would allow restricting their VLEN to a smaller one, and you can always emulate higher ones), but if your code doesn't work on all implementations, you've really just written as bad code as if it didn't work on any (assuming you didn't do it intentionally). A compiler can be made to handle the variable vector width relatively easily (clang and gcc already have autovectorization obeying the variable width). With explicit intrinsics or ASM, a programmer can of course write bad code, but it's actually surprisingly non-trivial to do so without noticing - yes, an in-register shuffle/permute can easily break, but to be able to run it you need to have to have some source operands for it, and it'll be obvious when you get the to-be-shuffled vector incorrectly - pretty much the only way to get bad length data is to load it from memory (as RVV doesn't provide you any way to make constant vectors, as those don't make sense), and to do that badly you'd have to pass a bad vl to it, which'll be very obvious. Or you can decide to have the to-be-shuffled vector be capped at the minimum guaranteed 128 bits, at which point there's no problem. On the other hand, you can actually use the shuffles in a fully length-agnostic way via constructing the table programmatically. I imagine that there will be a decent amount of things that explicitly stick to the 128 due to not being suited for automatic extension to wider vectors (and RVV does allow doing this very easily!), but I don't think that's particularly bad, nor should it restrict how other vector-width-independent things in the same program have to perform. And RISC-V's openness means that 256-bit and higher width implementations will easily exist, making it not completely ignorable. And explicit intrinsics will be 0.01% of actual RVV usage anyway, the 99.99% of autovectorization will nicely extend to future wide implementations.
- moonchild 3y ago
  My apologies. I linked that video mostly as a cheap shot at risc-v, and because it articulates the problem nicely. But doing that is somewhat antisocial, and further, I do not per se agree with its solution. It paints fixed-width vectors as the lesser of two evils; you seem to me to argue that variable-width vectors are not quite that bad, actually. To begin with, performance is always beholden to microarchitectural concerns. It seems likely that, in general, adding two pairs of numbers at once can be faster than adding one pair and then adding another (and perhaps very significantly so, especially for large values of two). But how much exactly depends on the specifics of the hardware. An algorithm which works well for hardware implementing 128-bit vectors might be quite bad for one implementing 2048-bit vectors. Some problems might be amenable to implementation for variable-length vectors—many will not. But even fewer will be implementable in a performance-portable fashion. The things which be implemented performance-portably are likely to be the simplest. Basically, applications of rank where a given atom is only ever matched with other atoms or with large cells. Few interesting applications are bottlenecked solely by such loops. They will likely have some; but they are likely to also have many loops which cannot be vectorised performance-portably. Hence, the promise of these vector isas, even if it could be realised nominally, would be a lie. Of course, this problem of performance portability afflicts fixed-width vectors, too. Intel goes to some lengths to try to provide it[0], but it's not obvious that that's sustainable in the long term (though, it's not clear that x86 is sustainable in the long term). (An example: I have some half-written code to use vector shuffles in place of loads for indexing small arrays in the j interpreter. What's 'small'? Once you go past a single array, the size of the mux grows quite quickly; but amd has a much better shuffle unit than intel, and a much worse memory subsystem, so I wouldn't be surprised if amd wanted to go a level deeper than intel.) In a recently-oft-circulated thread[1], fabian giesen mentions, among other things, the reason why avx512 hasn't seen wider adoption: if your logical vector-width is n, then you can implement that on an n/2-width alu without too much trouble, but n/4 starts to cause problems. (Where 'without too much trouble' means that the cost model doesn't change much on either the hardware or the software side.) So with fixed-width vectors, you do get one factor of two of hardware flexibility, but no more. Arguably, this gives the software more flexibility (than variable-length vectors), and doesn't harm the hardware, if you can pick the right n. Avx512 seems to demonstrate that you can't; some heavy vector workloads want 512, and some low-power client hardware wants 128. Recently, I predicted[2] that sve would effectively standardise on 256 bits (not 128 bits!—too anemic, and anyway neon already exists). I still hope that that happens. I think 256-bit vectors are a monotonic improvement over variable-width ones. Of course, see the above comment about heavier vector workloads really benefiting from 512-bit vectors; but it's naive to expect that the ideal vector size will grow without bound over the course of time, without other fundamental aspects of our models changing too, invalidating the entire enterprise. And this really gets at the heart of the issue. ISAs are bad[3] (which is why we keep needing to make more of them[4]), because they expose an overly low-level interface. They should be gotten rid of. The correct interface to these sorts of functionality is a higher-level language—a sensible one which can actually be optimised, not c; perhaps a nice apl, perhaps with some concessions. Code should be designed to work uniformly and universally; you should be able to create arrays that logically have any size, and the hardware and software collaborate to execute it as well they know how. If you know something about the underlying microarchitecture, then—uniformly, and as you always have been able to do—you can exploit that. For instance, if you know the underlying vector size is 64 bytes, then you may take care to work with length-32 vectors of 2-byte integers, which may enable further interesting optimisations to be done by the compiler. Making the high-level language expressive and optimisable helps to avoid the need to expose specific low-level functionality (as in c via 'intrinsics'). Where they are necessary—as they may occasionally be[5]—they can still be cast in more generic form and optimised more for clarity, e.g. (taking a pertinent example) acting generically wrt array size. 0. https://lobste.rs/s/gjza8a/#c_ircalq https://lobste.rs/s/gjza8a/#c_ircalq 1. https://mastodon.gamedev.place/@rygorous/110572829749524388 https://mastodon.gamedev.place/@rygorous/110572829749524388 2. https://lobste.rs/s/4cynth/introducing_iguana_extremely_fast_simd#c_qljymi https://lobste.rs/s/4cynth/introducing_iguana_extremely_fast... 3. https://lobste.rs/s/oxzs1q/note_on_metal_shader_converter#c_99aauh https://lobste.rs/s/oxzs1q/note_on_metal_shader_converter#c_.... Sorry I only sketch the argument here; I have been planning to write it up more completely and rigorously, but have not done with that yet. 4. Take care of the implicatory arrow; continually needing to make more of something is not in general an indication that the thing is bad. 5. Programming languages may be prone to the same pathologies as isas; an isa is a programming language. The difference is one of degree, and higher-level languages, in balance, end up in a much nicer and more useful place.

That's a bit dramatic. Yes, RVV does add to the problem of the same binary running differently on different implementations, but it's not like that doesn't already happen a lot. The main thing that sucks about it imo is that you can't test a piece of code on a single implementation and be sure it'll work the same on all (though I'd imagine CPUs would allow restricting their VLEN to a smaller one, and you can always emulate higher ones), but if your code doesn't work on all implementations, you've really just written as bad code as if it didn't work on any (assuming you didn't do it intentionally). A compiler can be made to handle the variable vector width relatively easily (clang and gcc already have autovectorization obeying the variable width). With explicit intrinsics or ASM, a programmer can of course write bad code, but it's actually surprisingly non-trivial to do so without noticing - yes, an in-register shuffle/permute can easily break, but to be able to run it you need to have to have some source operands for it, and it'll be obvious when you get the to-be-shuffled vector incorrectly - pretty much the only way to get bad length data is to load it from memory (as RVV doesn't provide you any way to make constant vectors, as those don't make sense), and to do that badly you'd have to pass a bad vl to it, which'll be very obvious. Or you can decide to have the to-be-shuffled vector be capped at the minimum guaranteed 128 bits, at which point there's no problem. On the other hand, you can actually use the shuffles in a fully length-agnostic way via constructing the table programmatically. I imagine that there will be a decent amount of things that explicitly stick to the 128 due to not being suited for automatic extension to wider vectors (and RVV does allow doing this very easily!), but I don't think that's particularly bad, nor should it restrict how other vector-width-independent things in the same program have to perform. And RISC-V's openness means that 256-bit and higher width implementations will easily exist, making it not completely ignorable. And explicit intrinsics will be 0.01% of actual RVV usage anyway, the 99.99% of autovectorization will nicely extend to future wide implementations.

moonchild 3y ago
My apologies. I linked that video mostly as a cheap shot at risc-v, and because it articulates the problem nicely. But doing that is somewhat antisocial, and further, I do not per se agree with its solution. It paints fixed-width vectors as the lesser of two evils; you seem to me to argue that variable-width vectors are not quite that bad, actually. To begin with, performance is always beholden to microarchitectural concerns. It seems likely that, in general, adding two pairs of numbers at once can be faster than adding one pair and then adding another (and perhaps very significantly so, especially for large values of two). But how much exactly depends on the specifics of the hardware. An algorithm which works well for hardware implementing 128-bit vectors might be quite bad for one implementing 2048-bit vectors. Some problems might be amenable to implementation for variable-length vectors—many will not. But even fewer will be implementable in a performance-portable fashion. The things which be implemented performance-portably are likely to be the simplest. Basically, applications of rank where a given atom is only ever matched with other atoms or with large cells. Few interesting applications are bottlenecked solely by such loops. They will likely have some; but they are likely to also have many loops which cannot be vectorised performance-portably. Hence, the promise of these vector isas, even if it could be realised nominally, would be a lie. Of course, this problem of performance portability afflicts fixed-width vectors, too. Intel goes to some lengths to try to provide it[0], but it's not obvious that that's sustainable in the long term (though, it's not clear that x86 is sustainable in the long term). (An example: I have some half-written code to use vector shuffles in place of loads for indexing small arrays in the j interpreter. What's 'small'? Once you go past a single array, the size of the mux grows quite quickly; but amd has a much better shuffle unit than intel, and a much worse memory subsystem, so I wouldn't be surprised if amd wanted to go a level deeper than intel.) In a recently-oft-circulated thread[1], fabian giesen mentions, among other things, the reason why avx512 hasn't seen wider adoption: if your logical vector-width is n, then you can implement that on an n/2-width alu without too much trouble, but n/4 starts to cause problems. (Where 'without too much trouble' means that the cost model doesn't change much on either the hardware or the software side.) So with fixed-width vectors, you do get one factor of two of hardware flexibility, but no more. Arguably, this gives the software more flexibility (than variable-length vectors), and doesn't harm the hardware, if you can pick the right n. Avx512 seems to demonstrate that you can't; some heavy vector workloads want 512, and some low-power client hardware wants 128. Recently, I predicted[2] that sve would effectively standardise on 256 bits (not 128 bits!—too anemic, and anyway neon already exists). I still hope that that happens. I think 256-bit vectors are a monotonic improvement over variable-width ones. Of course, see the above comment about heavier vector workloads really benefiting from 512-bit vectors; but it's naive to expect that the ideal vector size will grow without bound over the course of time, without other fundamental aspects of our models changing too, invalidating the entire enterprise. And this really gets at the heart of the issue. ISAs are bad[3] (which is why we keep needing to make more of them[4]), because they expose an overly low-level interface. They should be gotten rid of. The correct interface to these sorts of functionality is a higher-level language—a sensible one which can actually be optimised, not c; perhaps a nice apl, perhaps with some concessions. Code should be designed to work uniformly and universally; you should be able to create arrays that logically have any size, and the hardware and software collaborate to execute it as well they know how. If you know something about the underlying microarchitecture, then—uniformly, and as you always have been able to do—you can exploit that. For instance, if you know the underlying vector size is 64 bytes, then you may take care to work with length-32 vectors of 2-byte integers, which may enable further interesting optimisations to be done by the compiler. Making the high-level language expressive and optimisable helps to avoid the need to expose specific low-level functionality (as in c via 'intrinsics'). Where they are necessary—as they may occasionally be[5]—they can still be cast in more generic form and optimised more for clarity, e.g. (taking a pertinent example) acting generically wrt array size. 0. https://lobste.rs/s/gjza8a/#c_ircalq https://lobste.rs/s/gjza8a/#c_ircalq 1. https://mastodon.gamedev.place/@rygorous/110572829749524388 https://mastodon.gamedev.place/@rygorous/110572829749524388 2. https://lobste.rs/s/4cynth/introducing_iguana_extremely_fast_simd#c_qljymi https://lobste.rs/s/4cynth/introducing_iguana_extremely_fast... 3. https://lobste.rs/s/oxzs1q/note_on_metal_shader_converter#c_99aauh https://lobste.rs/s/oxzs1q/note_on_metal_shader_converter#c_.... Sorry I only sketch the argument here; I have been planning to write it up more completely and rigorously, but have not done with that yet. 4. Take care of the implicatory arrow; continually needing to make more of something is not in general an indication that the thing is bad. 5. Programming languages may be prone to the same pathologies as isas; an isa is a programming language. The difference is one of degree, and higher-level languages, in balance, end up in a much nicer and more useful place.