4 ms·

Wider SIMD would be useful, especially with AVX-512 style improvements. 1024 or even 2048 bits wide operations. Of course memory bandwidth should increase prop

Wider SIMD would be useful, especially with AVX-512 style improvements. 1024 or even 2048 bits wide operations. Of course memory bandwidth should increase proportionally otherwise the cores might have no data to process.

TimorousBestie 1y ago
I would love to be able to fit small matrices (4x4 or 16x16 depending on precision) in SIMD registers together with intrinsics for matrix arithmetic.
TinkersW 1y ago
I wouldn't mind, but might need to increase cache line size on x86, as avx512 has reached the current size.
- p_l 1y ago
  AVX-512 size is explicitly a derivative of P54C having 64 byte cache line in fact
owlbite 1y ago
Much better to burn the area for multiple smaller units, its a bit more area for frontend handling, but worth it for the flexibility (see Apple's M-series chips vs intel avx*).
- Remnant44 1y ago
  Yes and no. I think neon is undersized for today at 128bit registers -- if you're working with doubles for example, that's only two values per register, which is pretty anemic. Things like shuffles and other tricky bitops benefit from wider widths as well (see my other reply)
  - adgjlsfhk1 1y ago
    Agreed that 128 bit is undersized, but 512 feels pretty good for the time being. We're unlikely to see further size increases since going to 1024 would require doubling the cache line, register file, and ram bandwidth, while just adding an extra fma port is far less hardware.
    - Remnant44 1y ago
      totally - especially given how bandwidth constrained CPUs still are, going wider than 512 doesn't make much sense. 512 itself was a stretch for quite a long time (and all the negative press on the original implementations was a consequence of being not-quite-ready for primetime), but for current hardware I think it's perfect. But 128bit is just ancient. If you're going to go to significant trouble to rewrite your code in SIMD, you want to at least get a decent perf return on investment!
      - adgjlsfhk1 1y ago
        128 bit is already really nice for things like Int8 comparison (e.g lots of string operations and Swiss Dict key search)
api 1y ago
This would start looking a lot like a GPU.
- Veliladon 1y ago
  GPUs are literal SIMD devices. Usually 32 or 64 ALU lanes.
  - tubs 1y ago
    They are sim-t. It’s slightly different. I made them for a living for quite some time.
    - Bolwin 1y ago
      What's 't'? Textures? Triangles? Tensors?
      - craigacp 1y ago
        SIMT - Single Instruction, Multiple Threads
      - kmeisthax 1y ago
        Threads. It's less a difference in instruction set capability and more a difference in mentality. Like, for SIMD, you have to say "ok, we're working in vector land now" and start doing vector loads into vector registers to do vector ops on them. Otherwise, the standard variables your program uses are scalars and you get less parallelism. On a GPU this is flipped: the regular registers are vector, and the scalar ones (if you have any) are the weird ones. And because of this the code you write is (more or less) scalar code where everything so happens to magically get done sixteen times at once. As you can imagine, this isn't foolproof and there's a lot of other things that have to change on GPU programming in order to be viable. Like, conditional branching has to be scalar, since the instruction pointer register is still scalar. But you can have vectors of condition flags (aka "predicates"), and make all the operations take a predicate register to tell which specific lanes should and shouldn't execute. Any scalar conditional can be compiled into predicates, so long as you're OK with having to chew through all instructions on both branches[0]. [0] A sufficiently smart shader compiler could check if the predicate is all-false or all-true and do a scalar jump over the instructions that won't execute. Whether or not that's a good idea is another question.
account4mypc 1y ago
AMX registers are 1024 *bytes*

TimorousBestie 1y ago

I would love to be able to fit small matrices (4x4 or 16x16 depending on precision) in SIMD registers together with intrinsics for matrix arithmetic.

TinkersW 1y ago

I wouldn't mind, but might need to increase cache line size on x86, as avx512 has reached the current size.

p_l 1y ago
AVX-512 size is explicitly a derivative of P54C having 64 byte cache line in fact

owlbite 1y ago

Much better to burn the area for multiple smaller units, its a bit more area for frontend handling, but worth it for the flexibility (see Apple's M-series chips vs intel avx*).

Remnant44 1y ago
Yes and no. I think neon is undersized for today at 128bit registers -- if you're working with doubles for example, that's only two values per register, which is pretty anemic. Things like shuffles and other tricky bitops benefit from wider widths as well (see my other reply)
- adgjlsfhk1 1y ago
  Agreed that 128 bit is undersized, but 512 feels pretty good for the time being. We're unlikely to see further size increases since going to 1024 would require doubling the cache line, register file, and ram bandwidth, while just adding an extra fma port is far less hardware.
  - Remnant44 1y ago
    totally - especially given how bandwidth constrained CPUs still are, going wider than 512 doesn't make much sense. 512 itself was a stretch for quite a long time (and all the negative press on the original implementations was a consequence of being not-quite-ready for primetime), but for current hardware I think it's perfect. But 128bit is just ancient. If you're going to go to significant trouble to rewrite your code in SIMD, you want to at least get a decent perf return on investment!
    - adgjlsfhk1 1y ago
      128 bit is already really nice for things like Int8 comparison (e.g lots of string operations and Swiss Dict key search)

api 1y ago

This would start looking a lot like a GPU.

Veliladon 1y ago
GPUs are literal SIMD devices. Usually 32 or 64 ALU lanes.
- tubs 1y ago
  They are sim-t. It’s slightly different. I made them for a living for quite some time.
  - Bolwin 1y ago
    What's 't'? Textures? Triangles? Tensors?
    - craigacp 1y ago
      SIMT - Single Instruction, Multiple Threads
    - kmeisthax 1y ago
      Threads. It's less a difference in instruction set capability and more a difference in mentality. Like, for SIMD, you have to say "ok, we're working in vector land now" and start doing vector loads into vector registers to do vector ops on them. Otherwise, the standard variables your program uses are scalars and you get less parallelism. On a GPU this is flipped: the regular registers are vector, and the scalar ones (if you have any) are the weird ones. And because of this the code you write is (more or less) scalar code where everything so happens to magically get done sixteen times at once. As you can imagine, this isn't foolproof and there's a lot of other things that have to change on GPU programming in order to be viable. Like, conditional branching has to be scalar, since the instruction pointer register is still scalar. But you can have vectors of condition flags (aka "predicates"), and make all the operations take a predicate register to tell which specific lanes should and shouldn't execute. Any scalar conditional can be compiled into predicates, so long as you're OK with having to chew through all instructions on both branches[0]. [0] A sufficiently smart shader compiler could check if the predicate is all-false or all-true and do a scalar jump over the instructions that won't execute. Whether or not that's a good idea is another question.

account4mypc 1y ago

AMX registers are 1024 *bytes*