5 ms·

Intel AMX

21 points by fzliu 3y ago

froh 3y ago
full title is: Accelerate Artificial Intelligence (AI) Workloads with Intel Advanced Matrix Extensions (Intel AMX) and it's a brochure ("Solution brief")
camel-cdr 3y ago
Somewhat related, there is ongoing work on an integrated (in vector registers) matrix extension for RISC-V: https://github.com/riscv-admin/integrated-matrix-extension/blob/main/charter.adoc https://github.com/riscv-admin/integrated-matrix-extension/b... They are currently discussing the data layout, as the extension should be vector length agnostic, just like the vector extension: Option A: One matrix per vector registers. Option B: One matrix in multiple vector registers. Option C: Multiple matrices in one vector register. Option D: Streaming buffers for the input matrices. Option E: Variable matrix representation. My favorite of the proposals so far, is to use the vector registers as the accumulator and add a small ammount of new state for the source operands (Option D): https://github.com/riscv-admin/vector/blob/main/minutes/2023/2023-10-30/option-d.pdf https://github.com/riscv-admin/vector/blob/main/minutes/2023...
ysleepy 3y ago
I honestly feel cheated by intel with my 13th gen cpu. They took away avx512, even though the P cores are capable of it, but also did not ship AMX with it. I'm honestly more interested in Apple's AMX "co-processor", in my tests it often beats the GPU in ML tasks with smaller batch sizes. It's really cool. I hope it becomes more common for training/inference/numerics capable accelerators to be included in consumer hardware. Apple's AMX is really under-documented, while the instructions were reverse engineered, Virtually no benchmarks are available comparing current chip generations, models and variants. https://github.com/corsix/amx https://github.com/corsix/amx https://github.com/philipturner/amx-benchmarks https://github.com/philipturner/amx-benchmarks Curious to see how Intel's AMX stacks up to Apple's AMX in real world perf.
bratao 3y ago
It just compares to itself and AMD. It is missing the comparison to GPUs
- gpapilion 3y ago
  It’s like an older gpu. It’s able to perform similar to a v100 for some cases we tested out. It was like a 10x the avx512 solution for the same problem on an icelake cpu.

21 points by fzliu 3y ago

froh 3y ago

full title is: Accelerate Artificial Intelligence (AI) Workloads with Intel Advanced Matrix Extensions (Intel AMX) and it's a brochure ("Solution brief")

camel-cdr 3y ago

Somewhat related, there is ongoing work on an integrated (in vector registers) matrix extension for RISC-V: https://github.com/riscv-admin/integrated-matrix-extension/blob/main/charter.adoc https://github.com/riscv-admin/integrated-matrix-extension/b... They are currently discussing the data layout, as the extension should be vector length agnostic, just like the vector extension: Option A: One matrix per vector registers. Option B: One matrix in multiple vector registers. Option C: Multiple matrices in one vector register. Option D: Streaming buffers for the input matrices. Option E: Variable matrix representation. My favorite of the proposals so far, is to use the vector registers as the accumulator and add a small ammount of new state for the source operands (Option D): https://github.com/riscv-admin/vector/blob/main/minutes/2023/2023-10-30/option-d.pdf https://github.com/riscv-admin/vector/blob/main/minutes/2023...

ysleepy 3y ago

I honestly feel cheated by intel with my 13th gen cpu. They took away avx512, even though the P cores are capable of it, but also did not ship AMX with it. I'm honestly more interested in Apple's AMX "co-processor", in my tests it often beats the GPU in ML tasks with smaller batch sizes. It's really cool. I hope it becomes more common for training/inference/numerics capable accelerators to be included in consumer hardware. Apple's AMX is really under-documented, while the instructions were reverse engineered, Virtually no benchmarks are available comparing current chip generations, models and variants. https://github.com/corsix/amx https://github.com/corsix/amx https://github.com/philipturner/amx-benchmarks https://github.com/philipturner/amx-benchmarks Curious to see how Intel's AMX stacks up to Apple's AMX in real world perf.

bratao 3y ago

It just compares to itself and AMD. It is missing the comparison to GPUs

gpapilion 3y ago
It’s like an older gpu. It’s able to perform similar to a v100 for some cases we tested out. It was like a 10x the avx512 solution for the same problem on an icelake cpu.