3 ms·
Can someone help me understand when these neural engines kick in in open source software? I typically use python ML libraries like lightgbm, sklearn, xgboost e
by blobbers 8mo ago
Can someone help me understand when these neural engines kick in in open source software?
I typically use python ML libraries like lightgbm, sklearn, xgboost etc.
I also use numpy for large correlation matrices, covariance etc.
Are these operations accelerated? Is there a simple way to benchmark?
I see a lot of benchmarks on what look like C functions, but today in my jobs I rely on higher level libraries. I don't know if they perform any better on apple HW, and unless they have a flag like use_ane I'm inclined to think they do better.
Of course chatgpt suggested I benchmark an Intel Mac vs. newer apple silicon. Thanks chatgpt, there's a reason people still hate AI.
- zozbot234 8mo ago> when these neural engines kick in in open source software? It mostly doesn't because NPUs are bespoke and vendor-specific (which incents neglect by software devs working on open source numerics and ML/AI infrastructure), and the Apple ANE is no exception. Part of this effort is most likely about fixing that for the specific case of the Apple ANE.
- blobbers 8mo agoPart of which effort? The Reverse engineering is so it can be used blog article? I just think: great it seems like I'm paying for a hardware accelerator that makes Siri go faster. And I use siri on my laptop exactly 0 times in the last infinite years.
- bri3d 8mo agoIt also makes a lot of really useful features like on device OCR, captions, voice isolation, temporal antialiasing in metalfx, an enormous host of things in the apple pro apps, etc. work
- blobbers 8mo agoYeah, I don't use any of those features. So it sounds like its for folks who are creatives running lightroom or apple movie, or some kind of apple sound program? I'm a dev, not a creative, unfortunately. I don't use other people's software, I generally write my own (or used to before Claude took over my world).
- this-is-why 8mo agoYes. Numpy will accelerate if it detects hardware that it supports.
- blobbers 8mo agoI can't find any docs that numpy will do this. https://opensource.apple.com/projects/mlx/ https://opensource.apple.com/projects/mlx/ is needed to do this?
- this-is-why 8mo agoNormally I’d help a bro out but I started and googled and got hundreds of results and realized why does everyone need to be spoon fed. Please go do some work yourself mkay?