15 ms·
Microsoft Phi-2 model changes licence to MIT
- minimaxir 3y agoPreviously it was under a noncommercial license which tempered excitement a bit. Given its performance and size, a commercial-friendly license is actually a big deal.
- ranguna 3y agoThis model has been in the top for quite a while, what's so good about it?
- intellectronica 3y agoExcellent performance for this model size and inference cost. Best model you can run on a device a small as a phone and get performance close to GPT-3.5 level. The structure and the training data are also interesting - sparse model using curated synthetic data to achieve much better accuracy than is achieved in models trained on random internet text.
- stavros 3y agoIs it really close to GPT-3.5 at 2.7B?
- regularfry 3y agoIt's 2.7B, not 1.1. In my experience it goes off the rails and starts generating nonsense after a few paragraphs, but I haven't dug too much into tweaking the kv cache params to see if that's controllable. It also needs a fair bit of prompt massaging to get it to do what you want. So no, not GPT3.5, but it's comfortably better than anything else in its size class.
- eurekin 3y agoWhat are kv cache params?
- regularfry 3y agoKey-value cache in the attention layers. There was a paper a little while back about how maintaining the first N tokens across an extended context helped an LLM keep sane for longer, and it turns out you can replicate it with the right CLI arguments to llama.cpp.
- stavros 3y agoThis sounds much more realistic, thanks!
- realPtolemy 3y agoHow is it compared to 7B LLaMA quantized to run on a raspberry pi?
- regularfry 3y agoProbably similar token rates out of the box, although I havent done a straight comparison. Where they'll differ is in the sorts of questions they're good at. Llama2 was trained (broadly speaking) for knowledge, Phi-2 for reasoning. And bear in mind that you can quantise phi-2 down too. The starting point is f16.
- jasonjmcghee 3y agoIf you can run quantized 7B, nothing beats mistral and its fine tunes- like openhermes2.5
- moffkalast 3y agoClose is a subjective term. Vicuna-33B from over half a year ago gets within 22 ELO of 3.5 in the arena leaderboard, but in practice the refusals are reducing 3.5 and other RLHFd models' ratings a lot and they're not even close. You can try Phi-2 with wasm here, I mostly just get gibberish out of it though: https://huggingface.co/spaces/radames/Candle-phi1-phi2-wasm-demo?model=phi_1_5_q80 https://huggingface.co/spaces/radames/Candle-phi1-phi2-wasm-... Mixtral is the only model that properly matches 3.5 at the moment.
- stavros 3y agoThat makes sense, thank you. There seems to be some inflation, as Mistral is supposed to be GPT-3.5-level, and Mixtral is supposed to be nearer GPT-4, but yeah, it sounds suspicious in practice, even though Mistral is very good.
- moffkalast 3y agoWell in some things it totally can be to some extent, yes. You can almost certainly get a Mistral 7B fine tuned for a specific thing (e.g. coding) and it will likely be about as good as 3.5 in that specific thing (not a super high bar in objective terms). For all the other areas it may suffer in performance relative to its original self, but for some applications that's fine. As for GPT-4 it's about 120 ELO points [0] above Mixtral, and that's even the distilled turbo version. Not even close imo, especially when Mixtral is far less censored. Both 3.5 and 4 have changed drastically over the past year with continued fine tuning, quantization, etc. so what some people consider their level is not exactly a fixed point either. [0] The actual leaderboard I'm referencing, it has its biases but it's the most generally indicative thing available right now: https://chat.lmsys.org https://chat.lmsys.org
- regularfry 3y agoMixtral is only 32 ELO ahead of the best 7B model on that leaderboard, although I suspect that might be understating the difference.
- 3y ago
- WiSaGaN 3y agoIn my experience, it's great at its size, but obviously worse than mistral:7b-instruct-v0.2. Currently mixtral:8x7b-instruct-v0.1 is the lowest inference cost model at similar performance level of GPT-3.5.
- jasonjmcghee 3y agoClose to gpt-3.5? Because I’ve tried it and fine tuned variants and it’s been horrible. Next to useless on general tasks. No where near mistral 7 and absolutely not even close to gpt-3.5. Best 2.7b? sure. I have always thought of this model as something you fine tune for a very specific task or dataset. For others that don’t strongly disagree with the parent comment, can you point me to some examples?
- OhNoNotAgain_99 3y ago[dead]
- blueboo 3y agoIndicates Phi-3 and the next cohort will obsolete Phi-2
- RcouF1uZ4gsC 3y agoIt is really exciting to see these open models. What is interesting is that the AI “ethicists” all want to serve as a high priesthood controlling access to ML models in the name of safety. However, I think the biggest danger from AI is that these models will be used by those who control the models to control and censor what people are allowed to write. These open source models in the hands of the public, are, IMO the best defense against the true danger of AI. Kudos to Facebook and Microsoft and Mistral for pushing this.
- andy99 3y agoFacebook? Have they changed the llama license?
- rat9988 3y agoIt's open enough. Meta contributed a lot to open source ML, and their llama license is open enough.
- RcouF1uZ4gsC 3y agoI think people have lost the difference between defacto and dejure openness. A proprietary software program running on your hardware is more defacto open than an AGPL SAAS running on the cloud.
- Borealid 3y agoIf you had said GPL, I would agree with you. But you said "AGPL". AGPL SAAS running on someone else's computer that you can access requires that they provide you with the source code they're running. Barring shenanigans, that source code would enable you to run the same SAAS yourself if you desired to do so. I'd say having the ability to run the program locally _and_ its source code is "more open" than just having the ability to run the program locally in binary form. With AGPL in your scenario you get all three of source access, local execution, and remote-SAAS exection. Proprietary local code you get one of those three.
- 3y ago
- pk-protect-ai 3y agoI would be more interested in the dataset than the model...
- behohippy 3y agoIt's probably an evolution of the phi-1/1.5 "Textbooks are all you Need" training method: https://arxiv.org/abs/2309.05463 https://arxiv.org/abs/2309.05463
- alecco 3y agoYes. And the cost of these synthetic datasets is very high. Nobody is sharing. I suspect people are underestimating the amount of hardware OpenAI/Microsoft are using to build massive amounts of synthetic data. I doubt they are just training models over and over with the common crawls and such.
- pk-protect-ai 3y agoNobody underestimates it. It is clear that this stuff is not cheap. However, all publications without datasets are garbage because you can't replicate them. Why publish at all? It's just noise.
- orand 3y agoAll world-class scientists who don't cite every book they've ever read or teacher they've ever had are garbage because you can't replicate them. Why be born at all? They're just noise.
- pk-protect-ai 3y agoIt is not the same. If you can't replicate, you can't verify. There is a difference between what you can infer from the provided information and what you can prove. Replication is a cornerstone of scientific experimentation. Thus, the argument you are using here is bullshit.
- dmezzetti 3y agoThis is great. And it's also why independent open source projects are so important. It's hard to think the release of TinyLlama with it's Apache 2.0 license didn't factor into this change.
- qeternity 3y agoWhat’s the rationale that TinyLlama release played a factor?
- deleted 3y ago[deleted]
- jafitc 3y agoImportant to note that this model excels in reasoning capabilities. But it was on purpose not trained on the big “web crawled” datasets to not learn how to build bombs etc, or be naughty. So it is the “smartest thinking” model in weight class or even comparable to higher param models, but it is not knowledgeable about the world and trivia as much. This might change in the future but it is the current state.
- rolisz 3y agoBut that still makes it great for RAG applications, where I want the answer to be based on my data, not on whatever it learned from the web.
- monkeydust 3y agoInteresting. Anyone tried / benchmarked this for RAG?
- refulgentis 3y agoyeah it's good. you'd want* to finetune this before using it (c.f. my reply to it's depressed and insults me for no reason whatsover? @ https://huggingface.co/microsoft/phi-2/discussions/61 https://huggingface.co/microsoft/phi-2/discussions/61) * by want, I mean need. People self-peasantized heavily on "censorsed models" and don't really understand how these work, and the SNR is out of wack because there's a 100000x more waifu creators and culture warriors than knowledgable people sharing on this subject
- dlojudice 3y agoIf you think that LLMs have basically two properties: habitability to use natural language and knowledge to answer questions, then Small language models should being seen just excellent at natural language, and that's great because for many tasks general knowledge is not needed, specially for RAG.
- ethbr1 3y ago
- Donz1 3y ago[dead]