4 ms·
MLX: NumPy like framework for Apple Silicon by Apple
- jerpint 3y agoThe neural network module looks identical to PyTorch
- blovescoffee 3y agoThat seems reasonable. PyTorch has a well regarded API
- runnerup 3y ago> The design of MLX is inspired by frameworks like PyTorch, Jax, and ArrayFire. A noteable difference from these frameworks and MLX is the unified memory model. Arrays in MLX live in shared memory. Operations on MLX arrays can be performed on any of the supported device types without performing data copies. Currently supported device types are the CPU and GPU. Weird and unfortunate that a framework made by Apple for Apple Silicon doesn't support targeting the Apple Neural Engine.
- domschl 3y agoNeural engine is not helpful for training, its inference hardware, whereas this targets training and research. They use Accelerate and Metal (with seemingly similar/identical performance shaders that their Pytorch adaption uses) which allows for high performance training. This project additionally serves as documentation for other platforms to integrate Silicon, which is good.
- fmajid 3y agoStill, being to run LLaMa2 on the NPU would be awesome due to the unified memory. Apple's restricting its use to only Apple-approved models is frankly irksome.
- domschl 3y agoThe main thing about this framework is, that it uses unified memory with GPU. This gives maximum performance. Neural engine one the other hand is optimized for low-energy inference (which is mostly an advantage on mobile devices), and imposes limitations and restrictions since it's hardware supports only very specific neural network operations. Thus supporting neural engine within a universal machine learning platform doesn't make much sense, it would just be a bottleneck. The way to use neural engine is to convert existing models that strictly adhere to the limitations of the neural engine hardware (excluding many operations used in non-restricted NN models) for use in energy-restricted inference applications only. It's a different application scenario.
- reaperman 3y agoThank you for all this specific information!
- PrayagBhakar 3y agoCould Transformer based models been converted to work on the NPU?
- LeanderK 3y ago> Apple's restricting its use to only Apple-approved models is frankly irksome. I thought you could run arbitrary networks via CoreML, there's just limited precision and maybe not every operation available?
- deleted 3y ago[deleted]
- skavi 3y agodup: https://news.ycombinator.com/item?id=38539153 https://news.ycombinator.com/item?id=38539153
- calaphos 3y agoThe API and featureset so far looks like a one to one reimplementation of JAX without the jit functionality. What does this do differently? AFAIK Jax has an experimental apple GPU backend as well.
- dheera 3y agoIf PyTorch can't use it, it's really not going to pick up at all. They did half of the work, they now need to do the other half, which is contribute to PyTorch so it can use MLX data types so that PyTorch code can run unmodified and have the advantages of Apple silicon.
- Bostonian 3y agoTrying the link, I get Access to ml-explore.github.io has been blocked by Cisco Umbrella. I wonder why.
- tecleandor 3y agoCisco Umbrella is what OpenDNS Umbrella DNS protection is called today after the acquire. I'd say it's due to some DNS "protection" config on a corporate or public access point, or maybe on a corporate laptop or device.
- gentleman11 3y agoDoes numpy not work on apple silicon?
- DerekL 3y agoIt does work.