3 ms·
Nice and organised work! My only reservation wrt the core of it is that it tests the cpus on a matrix operations task, and power efficiency in this case refers
by dimask 2mo ago
Nice and organised work!
My only reservation wrt the core of it is that it tests the cpus on a matrix operations task, and power efficiency in this case refers specifically to doing matrix operations. This is fine if that's the kind of task one wants to optimise for, but it does not necessarily translate to general energy efficiency for a broader spectrum of tasks that most people may do most of the time.
And thus, if one wants to optimise for matrix operations, wouldn't it make sense to use apple's accelerate framework instead of blis/openblas (I think you used blis?)? I compiled hpl to use accelerate [0] on my m1 max (macbook), and ran it with settings almost identical to the original m1 max settings [1]. I get 422 Gflops vs 264 Gflops in m1 max mac studio (docker) [2]. My average draw (using a wall power meter, but not interfaced with computer to keep a log, just me looking at it) is around 55-57W (probably less because it peaked at 57W but the last half was getting around 52W). With the most conservative estimate of 57W, this gave a 7.4 Gflops/W which is second place, just below the m4 mac mini in the benchmark table. The m1 max in the original did 4 Gflops/W (at 66W).
So tbh I do not really trust the conclusion of the benchmark page, except if I miss sth. It is quite nice, organised work anyway.
[0] https://github.com/tycho/hpl/blob/master/setup/Make.MacOSX_Accelerate https://github.com/tycho/hpl/blob/master/setup/Make.MacOSX_A...
[1] I just fixed the p:q ratio, which was prob not really meant to be 1:10 there, I assume, also judging from the settings in the other tests. I put p=2,q=4. Also I was unable to put more than 8 processes in total.
[2] The original m1 max benchmark: https://github.com/geerlingguy/top500-benchmark/issues/4 https://github.com/geerlingguy/top500-benchmark/issues/4
- geerlingguy 2mo agoCan you please post an issue on the repo? Long on my todo list has been trying to get the Mac tests more repeatable/consistent and optimized (always easier on Linux).
- dimask 2mo agoWill def do!
- lostmsu 2mo agoWhen you test matmul these days you want bf16 max. FP64 is a niche workload 100% unused by 95%+ users.
- pclmulqdq 2mo agoAI is not the only thing in the world. The rendering of this webpage involves a lot of FP64 work.
- dimask 2mo agoThat's probably gonna run in gpu anyway. There is a lot of data work that involves double (or single) precision arithmetic, outside of ML, LLMs etc. It is definitely also worth testing 32 bit floats too, though.