11 ms·
> Relationship with CVNets > CoreNet evolved from CVNets, to encompass a broader range of applications beyond computer vision. Its expansion facilitated the tr
by gbickford 2y ago
> Relationship with CVNets
> CoreNet evolved from CVNets, to encompass a broader range of applications beyond computer vision. Its expansion facilitated the training of foundational models, including LLMs.
We can expect it to have grown from here: https://apple.github.io/ml-cvnets/index.html https://apple.github.io/ml-cvnets/index.html
It looks like a mid-level implementations of training and inference. You can see in their "default_trainer.py"[1] that the engine uses Tensors from torch but implements its own training method. They implement their own LR scheduler and optimizer; the caller can optionally use Adam from torch.
It's an interesting (maybe very Apple) choice to build from the ground up instead of partnering with existing frameworks to provide first class support in them.
The MLX examples seem to be inference only at this point. It does look like this might be a landing ground for more MLX specific implementations: e.g. https://github.com/apple/corenet/blob/5b50eca42bc97f6146b812a3e3469959da5da0ec/mlx_examples/clip/model.py#L402 https://github.com/apple/corenet/blob/5b50eca42bc97f6146b812...
It will be interesting to see how it tracks over the next year; especially with their recent acquisitions:
Datakalab https://news.ycombinator.com/item?id=40114350 https://news.ycombinator.com/item?id=40114350
DarwinAI https://news.ycombinator.com/item?id=39709835 https://news.ycombinator.com/item?id=39709835
1: https://github.com/apple/corenet/blob/main/corenet/engine/default_trainer.py https://github.com/apple/corenet/blob/main/corenet/engine/de...
- blackeyeblitzar 2y ago> It looks like a mid-level implementations of training and inference I’m not familiar with how any of this works but what does state of the art training look like? Almost no models release their training source code or data sets or pre processing or evaluation code. So is it known what the high level implementation even is?
- spott 2y agohttps://github.com/NVIDIA/Megatron-LM https://github.com/NVIDIA/Megatron-LM This is probably a good baseline to start thinking about LLM training at scale.
- error9348 2y agoThe interface looks very Apple as well. Looks like you create a config file, and you already have a model in mind with the hyperparameters and it provides a simple interface. How useful is this to researchers trying to hack the model architecture? One example: https://github.com/apple/corenet/tree/main/projects/clip#training-clip https://github.com/apple/corenet/tree/main/projects/clip#tra...
- sigmoid10 2y agoNot much. But if you just want to adapt/optimize hyperparams, this is a useful approach. So I can certainly see a possible, less technical audience. If you actually want to hack and adapt architectures it's probably not worth it.
- davedx 2y ago> It's an interesting (maybe very Apple) choice to build from the ground up instead of partnering with existing frameworks to provide first class support in them. It smells of a somewhat panicked attempt to prepare for WWDC to me. Apple has really dropped the ball on AI and now they're trying to catch up.
- pizza 2y agoWouldn’t WWDC-related endeavors be more product-facing? I’m not so sure this has to do with their efforts to incorporate ai into products, and tbh I would say their ai research has been pretty strong generally speaking.
- davedx 2y agoI expect that a lot of WWDC will be Apple trying to get more developers to build AI products for their platforms, because at the moment, Apple products don't have much AI. The other tech companies have integrated user facing LLM products into a significant part of their ecosystem - Google and Microsoft have them up front and center in search. Apple's AI offerings for end users are what exactly? The camera photos app that does minor tweaks to photos (composing from multiple frames). What else actually is there in the first party ecosystem that significantly leverages AI? Siri is still the same trash it's been for the last 10 years - in fact IMO it's become even less useful, often refusing to even do web searches for me. (I WANT Siri to work very well). So because their first party AI products are so non-existent, I think WWDC is a desperate attempt by Apple to get third party developers to build compelling AI products. I say desperate because they're already a year behind the competition in this space. (I can imagine they'll be trying to get developers to build Vision Pro software too, though I hear sales there have collapsed so again, way too little, too late)
- zitterbewegung 2y agoWhat you say is true about the project but both PyTorch works on Mace and Tensorflow was ported to Macs by Apple
- _aavaa_ 2y agoThey were originally available only as binaries, have they released the code changes required or upstreamed them yet?
- zitterbewegung 2y agoTensorflow was always on GitHub and PyTorch was in their source tree in their prerelease branch and then mainlined .
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- big-chungus4 2y agoThey don't implement their own stuff, their optimizers just inherits pytorch optimizers