10 ms·
Open source implementation for LLaMA-based ChatGPT
- rnosov 4y agoLooks like they are making ChatGPT clone that would be possible to run a single GPU. HN dream come true!
- wcarss 4y agoChatGPT's stable diffusion moment?
- Taek 4y agoCan't have a stable diffusion moment if you refuse to release the weights to the general public. Stable diffusion only got to where it is because 10,000 people with otherwise zero reputation were able to play around with the code and models. LLaMA is still only available to the elite.
- sinity 4y agoIt was released on 4chan recently :) files_catbox_moe[slash]o8a7xw(dot)torrent
- swyx 4y agolol good luck running a 13B model on a single GPU
- visarga 4y agoYou need a RTX 3090 24gb
- ddren 4y agoSeeing the performance of implementations like FlexGen [1], I don't think it would be entirely unreasonable to run a 13B model on a single GPU for personal usage purposes. You are not going to a run a public service out of it, but it probably would be good enough to run your own ChatGPT or Copilot locally. [1]: https://github.com/FMInference/FlexGen https://github.com/FMInference/FlexGen
- flangola7 4y agoIt would not be a HN dream for long, given the implications it has for the internet
- Jack5500 4y agoThis seems like a great first step to a truly open source LLM
- visarga 4y agoIf we can confirm the 13B model is as good as GPT3 then at least we have a target for reproduction without FB restrictions.
- lurquer 4y agoIndeed. All the weights for all the models will be available one way or the other very soon. The proprietary nature of the weights is not going to be a bottleneck for more than a month, if I had to guess. The other bottle-neck to personal use — the hardware required to run (not train from scratch) the thing - is going to be gone within the year I bet. I would assume some clever bloke is going to be able to prune the model or decrease the precision of the weights and discover you can get good-enough results with 1/10th of the memory. What happens then? Well, probably some Very Bad Things.
- georgehill 4y agoFor anyone wondering what LLaMA is, here are some useful links. https://ai.facebook.com/blog/large-language-model-llama-meta-ai https://ai.facebook.com/blog/large-language-model-llama-meta... https://news.ycombinator.com/item?id=34925944 https://news.ycombinator.com/item?id=34925944
- swyx 4y agothose who know what it is are questioning why you are working on this when the base model itself is not released widely. whats the intention here?
- georgehill 4y agoi am not the author of the repo, i came across this on github. https://twitter.com/nebuly_ai https://twitter.com/nebuly_ai is the author of the repo.
- levesque 4y agoIn what way is this a ChatGPT implementation or equivalent? Seems like a chatbot based on a different backend, therefore it has absolutely zero link to ChatGPT.
- basch 4y agoIt is an analogy. ChatGPT:GPT3::ChatLLaMa:LLaMa
- leerelive 4y ago[dead]
- levesque 4y agoIt uses a different engine, so this is as related to ChatGPT as a Toyota Corolla is related to a BMW car. This is an efficient and open-source chatbot, which is very good news, but the authors just wrote a clickbait title and they know it.
- f_devd 4y ago> Toyota Corolla is related to a BMW car. The analogy is somewhat accurate, but also moot, since within the ML community "ChatGPT" can be used either as the product or the method (more specifically called RLHF) somewhat interchangeably. It's more like Google/Googling, where the largest/most popular provider becomes the defacto way to refer to a method. As someone who develops DL models, the title seems quite apt.
- basch 4y agoIn formal analogies, : is pronounced "is to" and :: is pronounced "as". The purpose here is to use the relationship from a known, to describe the relationship between a partial known. ChatGPT is to GPT3 as ChatLLaMa is to LLaMa. It uses the relationship between ChatGPT and GPT3 to extrapolate a relationship between an unknown and LLaMa. see Analogies.pdf https://resources.finalsite.net/images/v1584287027/brockton/ctbakxiw8qmvpa8nmgox/Analogies.pdf https://resources.finalsite.net/images/v1584287027/brockton/... Corolla:Toyota::3-Series:BMW. If you had heard of a Corolla, Toyota, and BMW, but not a 3-Series, you now roughly know that a 3-Series is BMWs equivalent to a Corolla.
- rvz 4y ago> LLaMA is creating a lot of excitement because it is smaller than GPT-3 but has better performance. For example, LLaMA's 13B architecture outperforms GPT-3 despite being 10 times smaller. Exactly. Best part is that it is open-source. That is worth getting excited about. Not a AI SaaS API owned by a so-called pseudo-non profit company which struggles on API uptime and availablity, just like GitHub. This is the 'revolution' you are looking for that changes everything. Not ChatGPT.
- Ozzie_osman 4y agoIs it open source though? It looks like Meta was only releasing it to academics who apply.
- Taek 4y agoits not open source, only approved researchers are allowed to access the weights. More ivory towering from the AI industry.
- mikie92 4y agoright, and I guess there'll be opensource alternatives of llama very soon
- eigenvalue 4y agoIt's for your own "safety"...
- jstsch 4y agoAre LLaMA's weights generally available/floating around yet?
- gavi 4y agoI think you need to apply for it @Meta https://docs.google.com/forms/d/e/1FAIpQLSfqNECQnMkycAp2jP4Z9TFX0cGR4uf7b_fBxjY_OjhJILlKGA/viewform https://docs.google.com/forms/d/e/1FAIpQLSfqNECQnMkycAp2jP4Z...
- amrb 4y agoNope they pulled a fast one and limited it to academy and government, just put the dam thing on hugging face like everyone else guys..
- swyx 4y agopiratebay, but for model weights
- didntreadarticl 4y agoHave we got any details on the benchmarks that show LLaMa's 13B architecture outperforming GPT-3? Because that seems kindsof fantastical. Is it just a product of a very specific benchmark or does it reflect real world performance?
- rnosov 4y agoThey list 7B, 13B, 33B, 65B architectures. Presumably, they compare 65b one to GPT-3 175B. Chinchilla model which is about 70B outperformed a much larger GPT-3 model. So not that fantastical. EDIT: I stand corrected. They do compare 13B model with a large GPT-3 model which is hard to believe without a bit more concrete evidence
- rileyphone 4y agoYou're incorrect. > For example, LLaMA's 13B architecture outperforms GPT-3 despite being 10 times smaller. Of course this is all on benchmarks but it's a big improvement if true.
- rafaelero 4y agoThe GPT-3 they are comparing to is the one that was released on 2020. Since then OpenAI made a lot of improvements and nowadays I believe GPT-3.5 is competitive to Palm (540b). Still, LLaMa is in the same tier, with much less parameters.
- famouswaffles 4y agoGPT 3.5 is definitely not comparable to Palm. You can see some benchmarks here. https://crfm.stanford.edu/helm/latest/?group=core_scenarios https://crfm.stanford.edu/helm/latest/?group=core_scenarios For example, text-davinci-3 is 56.9 on MMLU, Flan-Palm is 75. https://arxiv.org/abs/2210.11416 https://arxiv.org/abs/2210.11416
- tikkun 4y agoFlan-Palm is 75 with five-shot. text-davinci-003 is it 56.9 with five-shot or zero-shot?
- shmatt 4y agoI don't really understand the benchmarking aspect researchers are touting. The public never cared about LLMs until they had a proper conversation with one. You can beat GPT3 at any benchmark you'd like, but if you can't get people that "feeling" when chatting with your model, is it worth anything? In the future there's going to have to be a way to benchmark the "human-ness" or "intrigue" or "feistiness" of a model to show us if its getting better at what we want
- rnosov 4y agoGPT-3 performed really well on synthetic benchmarks. It was later made palatable for general public consumption. You might say that a LLM needs to be good on synthetic benchmarks first before you can make public facing chatbots with it.
- basch 4y agoThe question is, where does this "human-ness" lie? In the initial neural network, the training data, or the supervised reinforcement? In theory, a significantly smaller neural network that outputs at nearly as good a quality, should be able to chew through training data, and its re-enforcement process, much faster and cheaper, right? A more generalized, lower parameter model, is almost always preferred, as long as it works? If the human feeling is all boltonable to the neural net later, there is no reason to discount this component as lacking potential to exceed current models.
- visarga 4y agoI think it's the feedback from RLHF that is mostly responsible, but it only works if the base model is large. Never seen a small model doing good conversation. They can do ok for classification and open book question answering, but generating long form coherent text is hard.
- antoineMoPa 4y agoI guess one technique is to train a model on various language model outputs to classify these as good/bad/intriguing/robot-sounding/repetitive/etc. A human can tag the answers for the training dataset. Then, we can use this model to compare different LLMs and optimize new models - could be with genetic algorithms or just a human tweaking the model - so that the LLM maximizes whatever we want.
- gersh 4y agoIs the trained model available to download anywhere?
- ddren 4y agoNot really, the LLaMA model is only available on request and access is granted on a "case by case basis" [1], which for most of us is more or less as available as GPT-3 is. [1] https://ai.facebook.com/blog/large-language-model-llama-meta-ai/ https://ai.facebook.com/blog/large-language-model-llama-meta...
- swyx 4y ago> more or less as available as GPT-3 is what? GPT3 is available by a public api that anyone can sign up and pay for and use for commercial use. how is it "as available"?
- ddren 4y agoI was mostly talking about access to the trained model weights. The OpenAI API is certainly better than nothing, but it is very restrictive and cost prohibitive for many purposes. For instance, you have to adhere to the OpenAI usage policies, and while they offer fine-tuning services, it is not likely enough to implement techniques like RLHF, which is the basis for ChatGPT. That said, if LLaMa can achieve performance competitive with GPT-3 with just 13B parameters, I imagine that it is only a matter of time until open source pre-trained models based on this architecture become available, which would render GPT-3 obsolete.
- voytec 4y agoFake title riding on ChatGPT popularity. I think that it should be updated to something like: Open source implementation for LLaMA-based chat bot* Open source implementation for LLaMA-based ChatGPT alternative*
- agolio 4y agoI think the 'alternative' is implied
- amrb 4y agoI would care more about LLaMA architecture when I can get hands on, honestly this project is more interesting and lighting fast on even a 2060 laptop https://github.com/BlinkDL/RWKV-LM https://github.com/BlinkDL/RWKV-LM
- iForgotMyPW 4y ago[flagged]
- lachlan_gray 4y agoHas anyone tried this on an m1 machine?
- tmptmptmp1 4y agoDo you have access to the weights? If so you probably have better ML hardware. Wish this model was actually "open". The perf of a model of that size on the M1 will not be good. That is big enough it won't quite fit on a 3090 (24GB) without quantization.
- pumanoir 4y agoI think is feasible. The description even says is designed to save on vram[1]. I don't get the other comments about needing more vram than a 3090. Also, Neuralmagic may run their sparsification on ARM cpu's in the future, so keep an eye. 1. ChatRWKV v2: with "stream" and "split" strategies. 3G VRAM is enough to run RWKV 14B :)
- IanCal 4y agoYou have to split it up which slows it down a lot. The 14B model doesn't fit fully on a 3090, though the 7B fits easily and is very fast. Other replies either may have meant this or thought the original comment was about llama.
- fswd 4y agoI've ran it on a AMD 3950 which I think is half the speed of a M1, and it's plenty fast. Note I am specifically talked about RWKV
- agolio 4y agoI'm as much a META hater as anyone - their policies have consistently disappointed me in almost every aspect of their business - but their stance on this LLaMA project I must say I am happy with and seems to mark a turn for the better. If they follow through on their promise of making the weights available and share source code that is a big step in the right direction for democratising this technology
- A4ET8a8uTh0 4y agoYeah, but their history ( and accumulated goodwill ) are similar to Microsoft's. They may say it is 'open' or sprinkle appropriately sounding corporate speaking all over the press release, but the actions will, at best, temporarily prevent them from going 'full evil ahead'. And I like that announcement. I just don't think they will actually follow through on this.
- cuuupid 4y agoThe weights are non commercial and while their code is GPLv3 they’ve only released inference and have removed anything that would give away the training methods :)
- alfalfasprout 4y agoThe paper highlights the training methods pretty clearly though including what tweaks they made.
- 4y ago
- davidy123 4y agoI am very far from an expert on this, but I think domain specific conversational AI would be much more useful than these large models. It's fun to ask an AI to compose a fresh 600bpm hip hop song about the relationship between materials science and the breeding habits of mosquitoes, but an open-source medical AI, application support AI, or many other applications would be much more practical, if they could be accurate enough. And especially if they could run "standalone." They could also consult with each other, as a network of specialized AI. Is work inching closer to more specific, more accurate applications? Or is this just a big gimmick/distraction phase around a maybe not so great idea of AI?
- Thebroser 4y agoThere are definitely very solid attempts at least to make LLMs that encode biomedical knowledge such as BioGPT which is trained on Pubmed and other domain specific areas. Source: https://arxiv.org/abs/2210.10341 https://arxiv.org/abs/2210.10341
- PeterisP 4y agoMy opinion is entirely opposite - we need conversational AI for exploring the multitude of possibilities and identify what works, what doesn't work, what is/isn't useful given the obvious limitations on accuracy, truthfulness and interpretability; but once we can identify a specific narrow use case then we can fine-tune a ML system for that isn't conversational but is able to provide better results with whatever domain-specific structure is required (linking to specific sources, including external structured data, providing certainty metrics, filtering results according to domain-specific criteria instead of the conversational political correctness filters which fail some domains, treatment info which was correct but has become outdated, etc, etc) that can be done better in non-conversational systems.
- panarky 4y agoThe P in GPT is for "pretrained". The large pretrained model is a prereq for domain-specific models.
- SakiWatanabe 4y agoWhat is the purpose of this? The model from meta is not available to public. Neither this open source "LLaMA-based ChatGPT" nor the "open source" LLaMA can be downloaded or actually used by public because it would required the actual trained model.
- visarga 4y agoThis whole debate - if a 13B model can really be as good as GPT3 - would have been settled if we had a live demo. I am not sure their licence allows running public demos, even if you get the weights.
- holtkam2 4y agoI don't have a decent gpu at my disposal... has anyone tried to run LLaMA on an EC2 GPU instance? If so, which instance type? (I don't wanna overpay)
- alfalfasprout 4y agoPer the paper the 13B variant runs on a single A100 GPU.
- bethecloud 4y agoopen-assistant from LAION is in the process of creating an OSS RLHF dataset for a personal assistant, may be useful for this project
- threevox 4y agoCan someone leak the weights please
- karmasimida 4y agoIs it of any good?
- louis030195 4y agoThe difference between invention & innovation is that innovation is when you ship your product to the masses. Can I query llama in a line of code? No
- vivegi 4y agoThis obsession with locking up model weights behind a gate-keeping application form and calling it open source is weird. I don't know who the high priests are trying to fool. If your model is really that good, unleash it into the open so that others can truly evaluate it-warts and all-and help improve it by identifying the flaws.
- sinity 4y ago> This obsession with locking up model weights behind a gate-keeping application form and calling it open source is weird. I don't know who the high priests are trying to fool. When they don't do it, people scream at them (see Galactica) "Journalists" react like this: > On November 15 Meta unveiled a new large language model called Galactica, designed to assist scientists. But instead of landing with the big bang Meta hoped for, Galactica has died with a whimper after three days of intense criticism. Yesterday the company took down the public demo that it had encouraged everyone to try out. > Meta’s misstep—and its hubris—show once again that Big Tech has a blind spot about the severe limitations of large language models. There is a large body of research that highlights the flaws of this technology, including its tendencies to reproduce prejudice and assert falsehoods as facts. > However, Meta and other companies working on large language models, including Google, have failed to take it seriously. Yann LeCunn confirmed this: https://twitter.com/pmarca/status/1631185701864865792 https://twitter.com/pmarca/status/1631185701864865792 I wonder if they just leaked it onto 4chan themselves, lol.