4 ms·

> […] but how much can your code actually use? All of it, and it is transparent to the code. The correct question is «how much data does the code transfer?» W

> […] but how much can your code actually use? All of it, and it is transparent to the code. The correct question is «how much data does the code transfer?» Whether you are scanning large string ropes for a lone character or multiplying huge matrices, no manual code optimisation is required.

tucnak 1y ago
Are you well-read enough into the platform so that you can attest to it requiring no manual code optimisation for high-performance datapaths? I'm only familiar with Apple Silicon-specific code in llama.cpp, and not really familiar with either Accelerate[0] or MLX[1] specifically. Have they really cracked it at homogenous computing so that you could use a single description of computation, and have it emit efficient code for whatever target in the SoC? Or are you merely referring to the full memory capacity/bandwidth being available to CPU in normal operation? [0]: https://developer.apple.com/documentation/accelerate https://developer.apple.com/documentation/accelerate [1]: https://ml-explore.github.io/mlx/build/html/usage/quick_start.html https://ml-explore.github.io/mlx/build/html/usage/quick_star...
- inkyoto 1y ago
  > Are you well-read enough into the platform so that you can attest to it requiring no manual code optimisation for high-performance datapaths? Yes. > Have they really cracked it at homogenous computing […] Yes. > have it emit efficient code […] Yes. I had also written compilers and code generators for a number of platforms (all RISC) decades before Apple Silicon became a thing. > […] for whatever target in the SoC? You are mistaking the memory bus width that I was referring to for CPU specific optimisations. You are also disregarding the fact that the M1-4 Apple SoC's have the same internal CPU architecture, differing mostly in the ARM instruction sets they support (ARM64 v8.2 in M1 through to ARM64 v8.6 in M4). > Or are you merely referring to the full memory capacity/bandwidth being available to CPU in normal operation? Yes. Is there truly a need to be confrontantial in what otherwise could have become an insightgul and engaging conversation?
Rohansi 1y ago
Have you tested it or is that just what you expect?
- inkyoto 1y ago
  Yes, I have actually tested it. Others also have. The https://lemire.me/blog/ https://lemire.me/blog/ blog has a wealth of insights across multiple architectures, which include all of the current incumbents (Intel, Apple, Qualcomm, etc.) Do you have any detailed insights? I would be eager to assess them.

tucnak 1y ago

Are you well-read enough into the platform so that you can attest to it requiring no manual code optimisation for high-performance datapaths? I'm only familiar with Apple Silicon-specific code in llama.cpp, and not really familiar with either Accelerate[0] or MLX[1] specifically. Have they really cracked it at homogenous computing so that you could use a single description of computation, and have it emit efficient code for whatever target in the SoC? Or are you merely referring to the full memory capacity/bandwidth being available to CPU in normal operation? [0]: https://developer.apple.com/documentation/accelerate https://developer.apple.com/documentation/accelerate [1]: https://ml-explore.github.io/mlx/build/html/usage/quick_start.html https://ml-explore.github.io/mlx/build/html/usage/quick_star...

inkyoto 1y ago
> Are you well-read enough into the platform so that you can attest to it requiring no manual code optimisation for high-performance datapaths? Yes. > Have they really cracked it at homogenous computing […] Yes. > have it emit efficient code […] Yes. I had also written compilers and code generators for a number of platforms (all RISC) decades before Apple Silicon became a thing. > […] for whatever target in the SoC? You are mistaking the memory bus width that I was referring to for CPU specific optimisations. You are also disregarding the fact that the M1-4 Apple SoC's have the same internal CPU architecture, differing mostly in the ARM instruction sets they support (ARM64 v8.2 in M1 through to ARM64 v8.6 in M4). > Or are you merely referring to the full memory capacity/bandwidth being available to CPU in normal operation? Yes. Is there truly a need to be confrontantial in what otherwise could have become an insightgul and engaging conversation?

Rohansi 1y ago

Have you tested it or is that just what you expect?

inkyoto 1y ago
Yes, I have actually tested it. Others also have. The https://lemire.me/blog/ https://lemire.me/blog/ blog has a wealth of insights across multiple architectures, which include all of the current incumbents (Intel, Apple, Qualcomm, etc.) Do you have any detailed insights? I would be eager to assess them.