3 ms·

While the non-commercial Phi2 license continues to be a letdown, I'm excited to see additional development in the space of these ultracompact LLMs. On-device AI

While the non-commercial Phi2 license continues to be a letdown, I'm excited to see additional development in the space of these ultracompact LLMs. On-device AI excites me far more than relying on yet another cloud-based API. On my M1 Macbook Air, current LLMs can run surprisingly quickly locally already.

rablackburn 3y ago
> On-device AI excites me far more than relying on yet another cloud-based API. Agreed. The huge leap-forward in capability already boggles my mind, the fact that I can run it _on my desktop cpu_ and have an interactive, natural-language oracle of the internet in a mere ~11GB file. Everyone seems to be chasing the network-accessible API approach because lock-in is easy and if you're at the bleeding-edge (training) you've got the compute for running it anyway. But now with accessible, local models my bet is on bored young hackers coming up with the best use cases and killer apps, not Microsoft.
- triyambakam 3y ago
  Which local models and CPU are you using?
- Uehreka 3y ago
  > Everyone seems to be chasing the network-accessible API approach because lock-in is easy This feels like a lazy critique. Sure companies may like the lock-in, but people are chasing the network-accessible API approach because it is way more powerful than the stuff you can run locally. Admittedly it’d be great if local models could get up to ChatGPT’s level, but if we’re being honest with ourselves they’re not there yet and are miles away from GPT-4, so it’s totally understandable that people want what OpenAI is selling.
- benterix 3y ago
  > But now with accessible, local models my bet is on bored young hackers coming up with the best use cases and killer apps, not Microsoft. That's only one part of the equation because these young hackers need companies with huge capitalization to deliver the models to them for free. What I hope actually happens is that model building can become modular and therefore could be crowdsourced. I'm not even sure this is even theoretically possible but it would help a lot.
mijoharas 3y ago
I see an MIT license, am I Missing something?
- IanCal 3y ago
  The code is (edit - may be? I can't seem to find it right now), but the model itself is under this license: https://huggingface.co/microsoft/phi-2/resolve/main/LICENSE https://huggingface.co/microsoft/phi-2/resolve/main/LICENSE
- woadwarrior01 3y ago
  The Phi-2 model (the LLM backbone in TinyGPT-V) weights, are licensed under a non-commercial research-only license[1] by Microsoft. [1]: https://huggingface.co/microsoft/phi-2/blob/main/LICENSE https://huggingface.co/microsoft/phi-2/blob/main/LICENSE
  - woadwarrior01 3y ago
    TBF, it's now been re-licensed under a more liberal MIT license.

rablackburn 3y ago

> On-device AI excites me far more than relying on yet another cloud-based API. Agreed. The huge leap-forward in capability already boggles my mind, the fact that I can run it _on my desktop cpu_ and have an interactive, natural-language oracle of the internet in a mere ~11GB file. Everyone seems to be chasing the network-accessible API approach because lock-in is easy and if you're at the bleeding-edge (training) you've got the compute for running it anyway. But now with accessible, local models my bet is on bored young hackers coming up with the best use cases and killer apps, not Microsoft.

triyambakam 3y ago
Which local models and CPU are you using?
Uehreka 3y ago
> Everyone seems to be chasing the network-accessible API approach because lock-in is easy This feels like a lazy critique. Sure companies may like the lock-in, but people are chasing the network-accessible API approach because it is way more powerful than the stuff you can run locally. Admittedly it’d be great if local models could get up to ChatGPT’s level, but if we’re being honest with ourselves they’re not there yet and are miles away from GPT-4, so it’s totally understandable that people want what OpenAI is selling.
benterix 3y ago
> But now with accessible, local models my bet is on bored young hackers coming up with the best use cases and killer apps, not Microsoft. That's only one part of the equation because these young hackers need companies with huge capitalization to deliver the models to them for free. What I hope actually happens is that model building can become modular and therefore could be crowdsourced. I'm not even sure this is even theoretically possible but it would help a lot.

mijoharas 3y ago

I see an MIT license, am I Missing something?

IanCal 3y ago
The code is (edit - may be? I can't seem to find it right now), but the model itself is under this license: https://huggingface.co/microsoft/phi-2/resolve/main/LICENSE https://huggingface.co/microsoft/phi-2/resolve/main/LICENSE
woadwarrior01 3y ago
The Phi-2 model (the LLM backbone in TinyGPT-V) weights, are licensed under a non-commercial research-only license[1] by Microsoft. [1]: https://huggingface.co/microsoft/phi-2/blob/main/LICENSE https://huggingface.co/microsoft/phi-2/blob/main/LICENSE
- woadwarrior01 3y ago
  TBF, it's now been re-licensed under a more liberal MIT license.