23 ms·
StableLM: A new open-source language model
- Dowwie 4y agoThis doesn't perform well at all against questions that I've presented to ChatGPT 3.5 and received far better responses for. What are people seeing from this that is generating so much excitement?
- abroadwin 4y agoIt isn't reasonable to compare the 7B model with the likes of ChatGPT 3.5 which has 175 billion parameters. Stability does have plans to produce a comparable model, though. Right now they're working on models up to 65 billion parameters. This is just the very early stages of development, and the exciting thing is that it's something you can actually run yourself and it's freely usable for commercial use.
- rejectfinite 4y agoSo, as a user, is this a free chatGPT? Free in money and free of woke?
- nwoli 4y agoNo way, I thought this was months away still. So excited
- MacsHeadroom 4y agoCompleted training is. These are early "Alpha" checkpoints.
- visarga 4y agoIt's released but still cooking in the oven as we speak.
- aketchum 4y agoAre they releasing the weights for download? The links to HuggingFace in the readme are giving me 404. This dataset they built on-top of "The Pile" sounds interesting - looking forward to evaluating their claim that 3-7 billion param models can perform on par with 175 billion param GPT-3
- artyomxyz 4y agoIt looks like it's available here https://huggingface.co/stabilityai/stablelm-base-alpha-7b/tree/main https://huggingface.co/stabilityai/stablelm-base-alpha-7b/tr...
- turingfeel 4y agoDid they claim this? I didn’t see that claim made in the above post.
- aketchum 4y ago"The richness of this dataset gives StableLM surprisingly high performance in conversational and coding tasks, despite its small size of 3 to 7 billion parameters (by comparison, GPT-3 has 175 billion parameters)." So they did not explicitly say it is comparable, but implicitly compared the two. I'm curious to evaluate what "surprisingly high performance" means exactly.
- SeanAnderson 4y agoI'm curious if this will give better results than llama 7B? Llama 7B felt like a toy that, while cool to be able to run locally, did not feel useful in any way when contrasted to the state of GPT. Here's hoping for better and/or release of larger parameter models with low performance requirements soon :) EDIT: my first question times out when ran online, seems like huggingface is getting hugged to death.
- nwoli 4y agoThey have on their team people from Anthropic which have stuff like Claude Instant which is likely running a very light model, ie the tricks from Anthropic have likely been incorporated into the model they’re presenting here
- pksebben 4y agoEven if it doesn't initially, the fact that it's being released so permissively is massive - stable diffusion was made far more powerful by being hackable at all levels and I can't imagine we won't see the same here. I imagine things like control nets that restrict output to parsable types, LoRa style adaptations that allow mixable "attitudes", that sort of thing. Very different underlying architecture from diffusers, ofc. But the action of open source is the same - a million monkeys with a million xterms and so forth.
- edkennedy 4y agoI'm really hoping for the ability to load in different sets of trained material as embeddings/textual inversions like in Stable Diffusion. Imagine scanning in some of your favorite philosophy and design books and throwing them with small weighting as a little flavor to your answer. The crossovers between LLM and Stable Diffusion type models (like Loras) is such a fascinating space to explore.
- alex_sf 4y agoThis already exists for LLaMA, I'm sure it'll be implemented for StableLM as well. https://github.com/lxe/simple-llm-finetuner https://github.com/lxe/simple-llm-finetuner
- anononaut 4y agoMy prayers have been answered
- IceWreck 4y agoStabilityAI is the real OpenAI. Thanks for this.
- diminish 4y agoWell said. Openai is a promise unkept. Thanks StabilityAI for existing.
- capableweb 4y agoUnfortunately, due to the law of names, StabilityAI will in the future hit the same issue as OpenAI and do a 180, unleashing very unstable AI to the world.
- IceWreck 4y agoI'd rather have that than closed AI models controlled by afew corporations
- seydor 4y agothen they both have ironic names now
- mike_hearn 4y agoMore like Stability will turn out to be an unstable company. Last we heard they were struggling to raise more funding and might lose their CEO due to unclear business models: https://www.semafor.com/article/04/07/2023/stability-ai-is-on-shaky-ground-as-it-burns-through-cash https://www.semafor.com/article/04/07/2023/stability-ai-is-o...
- paxys 4y agoThe company can cease operations tomorrow, but the model they open sourced (and all of its derivatives built by the community) will continue to exist. If OpenAI disappears then all of the work they have done goes with it.
- 4y ago
- huseyinkeles 4y agoThis is amazing. They even let the developers use it for commercial purposes; “Developers can freely inspect, use, and adapt our StableLM base models for commercial or research purposes, subject to the terms of the CC BY-SA-4.0 license.“ You can use this link to interact with the 7B model; https://huggingface.co/spaces/stabilityai/stablelm-tuned-alpha-chat https://huggingface.co/spaces/stabilityai/stablelm-tuned-alp...
- IceWreck 4y agoIs there a way to check the queue for this ? I sent it one small text (actually a task) five minutes ago. Its still loading.
- pksebben 4y agosame. Probably hugged to death.
- convery 4y agoIt just finished building.
- IceWreck 4y agoJust tested it. I tried to get it to write a conclusion after giving it my report and while is was correct, it was kind of silly, a far cry from chatgpt. But again, this is the 7B variant and its open source.
- MacsHeadroom 4y agoThis is the Alpha of the 7B variant only partially through its initial training set (800B tokens out of 1.5 Trillion)
- deleted 4y ago[deleted]
- rvz 4y agoFinally, the true race to zero is now on. Stability and others are already at the finish line in the race to zero. O̶p̶e̶n̶AI.com cannot get any cheaper and eventually will have to increase their prices. There is no escape and as I said before, open source AI models will eventually swamp and survive longer and intergrate faster than even O̶p̶e̶n̶AI.com's ChatGPT. Expect open source models to get smaller and even better such that it will fit in an iPhone, on device. Stay tuned.
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- anticensor 4y agoWhy are you referring to OpenAI as ai.com?
- netsroht 4y agoI think he's just emphasizing that OpenAI is in fact not open, thusly it's crossed out.
- MacsHeadroom 4y agoWhere does "openai.com" go and who owns it. Now where does http://ai.com http://ai.com go and who owns it?
- anticensor 4y agoOh, I just realised ai.com redirects to ChatGPT.
- dougmwne 4y agoThis is absolutely huge. LLaMA weights leaking was a big deal, but you couldn’t actually use them without attracting Meta’s ire. Would love to see some benchmarking vs. LLaMA and GPT.
- pyinstallwoes 4y agoWhat is IRE?
- jwineinger 4y agoanger. probably a lawsuit in this context
- jmiskovic 4y agoThere were no lawsuits around LLaMA. They used DCMA requests to take down some GitHub and HuggingFace releases but the majority of releases (Alpacas and other animals) was left alone. It was enough to prevent commercial use, though. Edit: a lawsuit would be quite interesting, to clear up many things around how copyright works for LLMs.
- MacsHeadroom 4y agoThey're going to court with the guy who made llamaDL actually.
- inportb 4y agoIre is the result of irritation.
- prions 4y ago> Supportive. We build models to support our users, not replace them. We are focused on efficient, specialized, and practical AI performance – not a quest for god-like intelligence. We develop tools that help everyday people and everyday firms use AI to unlock creativity, boost their productivity, and open up new economic opportunities. Refreshing take on the peak alarmism we see from tech "thought leaders"
- CamperBob2 4y agoI dunno... god-like intelligence would be pretty useful. I'll take a brochure.
- nashashmi 4y agodo you trust god?
- int_19h 4y agoDepends on which one. All the ones described in religious books seem to have very poor alignment, though.
- CamperBob2 4y agoWell, ... OK, I withdraw the comment.
- sdenton4 4y agoWhy should I need to? Isn't God on the blockchain? (j/k)
- seydor 4y agoDoesn't sell as much, though
- antibasilisk 4y ago>Refreshing take on the peak alarmism we see from tech "thought leaders" It's not alarmism when people have openly stated their intent to do those things.
- GaggiX 4y agothe models: https://huggingface.co/stabilityai/stablelm-base-alpha-3b https://huggingface.co/stabilityai/stablelm-base-alpha-3b, https://huggingface.co/stabilityai/stablelm-base-alpha-7b https://huggingface.co/stabilityai/stablelm-base-alpha-7b There are also tuned version of these models: https://huggingface.co/stabilityai/stablelm-tuned-alpha-3b https://huggingface.co/stabilityai/stablelm-tuned-alpha-3b https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b, these versions are fine-tuned on various chat and instruction-following datasets. The Github repo mentions that the models will be trained on 1.5T tokens, this is pretty huge in my opinion, the alpha models are trained on 800B tokens. The context lenght is 4096.
- bhouston 4y agoThese models are huge. I assume they are not quantized down to 4bits yet.
- brucethemoose2 4y agoQuantized versions will pop up on huggingface very soon, if they arent already there. It takes basically no time, much less than something like a alpaca finetune.
- nwoli 4y agoI love stability ai so much
- la64710 4y agoThanks for breaking ground here and being truly OSS
- jmatthews 4y agoThe AI war has officially begun.
- mach1ne 4y agoNo, that comes when China catches up in the quest for superintelligence.
- ChatGTP 4y agoStop worrying about China…wow
- xyzzy4747 4y agoI don’t think China has enough skilled AI workers to compete in any quick timespan. Top talent doesn’t want to live and work there generally.
- kriro 4y agoI'm curious why you'd think that. China as a country has many people to start. Some percentage of these people will end up in AI. Assuming people from all countries are roughly equally intelligent, the numbers clearly favor China. Universities over there are quite good, there's a pretty strong "work hard" mentality I see from all our Chinese students. Plenty of Chinese graduating or starting university these days during the AI hype peak. China as a country isn't sleeping on AI either. I think China as an AI hub looks quite promising. Anecdotally, China also retains quite a lot of talent or people go abroad to study and return to China. Compared to some European countries or India that "leak" a lot of talent to the U.S. I think China is quite a bit more stable. On the hardware side, things tend to be produced there as well.
- snordgren 4y agoChina definitely "leaks" a lot of talent to American companies - most AI papers that I've seen from respected Western universities include at least one Chinese name. One challenge for China has been the university enrollment rate. While in Western countries half of each cohort has been going to university for decades, China is not there yet. In 2019, just 17% of Chinese adults have degrees compared to 44% in the US. So the large Chinese population is offset by its relative lack of access to education, while the US can draw from its own highly educated population in addition to attracting the best and the brightest from the rest of the world, including China.
- vikp 4y agoIt's fantastic that more orgs are releasing open-source models trained on more than 300B or so tokens. Here's my take from the details I could find. Pros - 4096 context width (vs 2048 for llama, gpt-j, etc) - 3B to 65B released or in progress - RL tuned models available - Trained on more tokens than existing non-llama models - 128 head dim, so can use flash attention (unlike GPT-J) Cons - No benchmarks released, or details about the model - Somewhat restrictive license on the base models, and NC license on the RL models - Small models only trained on 800B tokens, compared to 1T for llama-7B, and potentially more for other upcoming alternatives (RedPajama, etc). I'd like to see their loss curves to see why they chose 800B. High-level, this is likely to be more accurate than existing non-llama open source models. It's hard to say without benchmarks (but benchmarks have been gamed by training on benchmark data, so really it's just hard to say). Some upcoming models in the next few weeks may be more accurate than this, and have less restrictive licenses. But this is a really good option nonetheless.
- capableweb 4y ago> - 3B to 65B released or in progress Seems they want to do 3B to 175B, although 175B is not in progress yet.
- ipsum2 4y agoIt's not efficient to do 175B. Training a smaller model (65B) on more data gives better performance for the same compute.
- tempaccount420 4y agoIf you want it to just regurgitate training data, sure. But more parameters will always be better for more complex tasks.
- thewataccount 4y ago> But more parameters will always be better for more complex tasks. I think you should checkout this paper which discusses the relationship of performance and the ratio of training tokens to parameter count. https://arxiv.org/abs/2203.15556 https://arxiv.org/abs/2203.15556
- deleted 4y ago[deleted]
- tucnak 4y agoOn the off-note, can anybody tell me what's going on with embeddings, & vector databases? Certainly it would seem that forward-pass completion is pretty much solved, & a smaller, better model will appear eventually. Let's say you even managed to solve both complete() and embed() but what do you do with it, how are you going to organise, query, and multiply this dataset? Now the question I know that text-embedding-ada-002 has twice as many dimensions as mainstream Sentence transformers. Do we need all the extra dimensions? If not, how do I make it work better for my specific dataset with lots of jargon and abbreviations and stuff like that? What are the hardware requirements for that? I.e. could I do a fine-tuning job on some specific jargon-heavy text to get better embeddings for them? For one, the more I look into similarity-based use-cases the more I see that it's not normally speaking "top-percentile nearest-neightbour search" but the data is also terribly relational, i.e. it's probably like a slowly changing dimension, and there's a tree traversal type structure in how documents are generated as output from other documents as inputs? So you kind of have to think about these complete/embed ops both in aggregate; for batching but also in particular, from the cost/reward ROI type calculation. Not just in aggregate but also in terms of memory usage patterns to further optimise layout— tiering and stuff like that really comes to light. Also: vector database shilling on HN is getting out of hand; multiple companies literally plugging every mention on the radar, some actively begging for upvotes. Looking at it all makes you really appreciate pgvector[1] to a point where you would be more willing to buy 3.2 TB of high-bandwidth NVMe and dedicate it to a large IFV index than ever have to deal with all of this "purpose-built vector database" bullshit. [1]: https://github.com/pgvector/pgvector https://github.com/pgvector/pgvector
- visarga 4y ago> Do we need all the extra dimensions? This discussion seems relevant: https://www.reddit.com/r/MachineLearning/comments/12q8rp1/discussion_openai_embeddings_api_alternative/ https://www.reddit.com/r/MachineLearning/comments/12q8rp1/di...
- Der_Einzige 4y agoYes, you need all of the dimensions. All of the dimensionality reduction techniques, including SOTA ones (UMAP or better) are going to massively harm your embeddings.
- msoad 4y agoI feel "The Pile" needs a lot more to match what OpenAI has trained their models on. The mysterious "Books2" and so on.
- harisec 4y agoI think OpenAI has a few hidden advantages that are not obvious at this point. It could be additional training data, filtering/preprocessing that data, some changes to the architecture, who knows? None of the open source models are even close to GPT 3.5, what to speak about GPT 4? I've tried everything and the 60G llama variants so i'm not sure it's about number of parameters. They definitely have some hidden sauce.
- nickthegreek 4y agoGreat to see Stability release this with a great license as well. Any idea on the usecases for the 3B model? Will a model that small suffer heavily from a 4bit quantization?
- capableweb 4y agoGenerating either a 0 or 1 in response to something? Maybe useful in some testing scenarios? Honestly don't know either. New lorem ipsum?
- visarga 4y agoYou can use it as the assistant model to a large model, it's called speculative sampling. You generate text with the small model and validate with the large one, ensuring no deviation occurs. Speedup of 2.5x https://arxiv.org/abs/2302.01318 https://arxiv.org/abs/2302.01318 (DeepMind)
- MacsHeadroom 4y agoThat would be 1bit quantization. In reality quantization under 8bits is done in smart ways which result in higher effective output quantization and lower effective memory size quantization. For example, bucketing identical groups of weights and then reusing one bucket for all the identical groups lowers the effective bit quantization at the memory level while retaining the uncompressed quantization quality. There is literature on effective quantization levels below 1 ("So called 0bit quantization). But even then the actual weights are typically 2-4 bits and there is just a lot of reuse of weights going on. Another neat trick is to line bins of weights up in order of lowest to highest weights and compute a function to produce a zero offset throughout the bins of weights; such that 0010 in one bin and and 0010 in another bin are unsampled to 16bit and then have different offsets added, maintaining uncompressed 16bit performance without the memory overhead. There are many more tricks like this and many still to be found!
- wsgeorge 4y ago> Will a model that small suffer heavily from a 4bit quantization? From my experience with quantized 7B llama models, avoid 3B if you can. Without benchmarks, I think this is a decent rule of thumb.
- alexb_ 4y agoAbsolutely a giant fan of Stability staying to actually open source licenses and not licenses that impose restrictions on what you can use it for. This is the future of AI! Beware of any org that uses "ethical" licenses - they are not open source. Stability is one of the few organizations that actually cares about free software, you love to see it.
- archerx 4y agoIndeed thats why I pay for credits on their official site/dream studio even though I want to run things locally. My big fear is one day they’ll make a press release saying they have to stop everything because not enough funding.
- cjbprime 4y ago> These fine-tuned models are intended for research use only and are released under a noncommercial CC BY-NC-SA 4.0 license, in-line with Stanford’s Alpaca license. This is a no-commercial-use-allowed license; it is neither considered free software nor open source, the definitions of which disallow restrictions on what you can use the work for.
- freedomben 4y agoThe two sentences prior are important: > We are also releasing a set of research models that are instruction fine-tuned. Initially, these fine-tuned models will use a combination of five recent open-source datasets for conversational agents: Alpaca, GPT4All, Dolly, ShareGPT, and HH. These fine-tuned models are intended for research use only and are released under a noncommercial CC BY-NC-SA 4.0 license, in-line with Stanford’s Alpaca license. The snippet you quoted is not talking about the main model in the announcement. It's talking about fine-tuned models based on other models. Stability has to respect the license of the originals. They cannot change it. The main model is described higher up in the post and is permissible for commercial: > Developers can freely inspect, use, and adapt our StableLM base models for commercial or research purposes, subject to the terms of the CC BY-SA-4.0 license
- 4y ago
- sroussey 4y ago“also fine-tuned the model with Stanford Alpaca's procedure using a combination of five recent datasets for conversational agents: Stanford's Alpaca, Nomic-AI's gpt4all, RyokoAI's ShareGPT52K datasets, Databricks labs' Dolly, and Anthropic's HH. We will be releasing these models as StableLM-Tuned-Alpha.” They have released the 3B and 7B of both the base and instruction tuned models. 30B and 65B in training and released later.
- dr_dshiv 4y agoHow can one get started adding RLHF to something like this?
- dmm 4y agoIssue to add support for these models to llama.cpp: https://github.com/ggerganov/llama.cpp/issues/1063 https://github.com/ggerganov/llama.cpp/issues/1063
- sunshadow 4y agoDemo is overloaded. Here is Colab link if you wanna check the quality: https://colab.research.google.com/github/Stability-AI/StableLM/blob/main/notebooks/stablelm-alpha.ipynb https://colab.research.google.com/github/Stability-AI/Stable...
- drexlspivey 4y agoYour session crashed after using all available RAM. If you are interested in access to high-RAM runtimes, you may want to check out Colab PRO
- BudaDude 4y agoThis notebook will need high ram and a premium GPU. It blew past the 16gb GPU that comes with Colab Pro
- deleted 4y ago[deleted]
- enono 4y ago[flagged]
- josephwegner 4y agoAnyone have a sense for system requirements on running this locally? StableDiffusion was too heavy for either of my machines (which are admittedly not very powerful) - I'm not seeing much in the docs to indicate whether or not this is more or less intensive to run locally.
- wsgeorge 4y agoIf you can run any models on llama.cpp, that might be a good indicator of which StableLM models you'll be able to run. I easily ran 7B int 4 ggml models on an MBP with 16gig RAM. Same works on a MBA with 8 gig RAM, but you'll have to not run any other memory-hogging app.
- ranguna 4y agoThe tuned 7B model is around 33 GBs, so you'll need a PC with that much VRAM or RAM. I haven't tried to load it on text generation ui though.
- MacsHeadroom 4y agoIn 4bit 7B requires 6GB of RAM and runs at ChatGPT speeds on CPU (with llama.cpp). The 15B model coming out soon will require 12GB of RAM and still run at good speeds on CPU.
- brucethemoose2 4y agoOn top of what others said, unlike SD, its not unusable on CPU... just very slow. Stable diffusion will run on a 4GB GPU though.
- seydor 4y agoWhen will we see gglm-versions of these? Good job on openAI to sell out in 2022. It was truly the end of the line. No matter how bad these model releases are , they are certain to get awesome soon with everybody hacking around them. The surprising success of MiniGpt4 with images shows that openAI's GPTs don't have some magic secret sauce that we dont know of.
- sebzim4500 4y agoGPT-4 is still much, much better than any of the competition. I guess we'll see once we have a 175B version of StableLM though, presumably that will at least easily beat GPT-3.
- MacsHeadroom 4y agocformers already has ggml support because it's the same architecture as GPT-NeoX. llama.cpp just added preliminary support three hours ago. https://github.com/ggerganov/llama.cpp/issues/1063#issuecomment-1515430933 https://github.com/ggerganov/llama.cpp/issues/1063#issuecomm...
- transcriptase 4y agoMissed opportunity for Stability to call themselves “ClosedAI” then being the opposite of what the name suggests.
- r0l1 4y agoHow will they generate revenue?
- brucethemoose2 4y agoBy soliciting excited investors for money!
- r0l1 4y agoSeams so ^^ Too bad, that the open source model has this problem. Hope they will survive and find a fitting business model.
- jonplackett 4y agoAnyone know if this will be added to the Stability API?
- ilaksh 4y agoAmazing but the instruction fine-tuning is still a huge challenge for businesses since what is released cannot be used for commercial purposes. Instructions are much more useful. I have a feeling that there are probably some people who will look at the "commercial okay" license for the first part and in their mind that will somehow make it okay to use the instruction-tuned ones for commercial purposes. Maybe we don't really need Instruct stuff? Because it seems like its a huge amount of redoing work. I wonder if the OpenAssistant people will start building off of these models.
- dougmwne 4y agoThe instruct tuning can be done with several open datasets at minimal cost. Should be easy for someone to create their own open model.
- jacooper 4y agoHow?
- dougmwne 4y agohttps://github.com/tatsu-lab/stanford_alpaca https://github.com/tatsu-lab/stanford_alpaca
- MallocVoidstar 4y agoThat dataset is licensed under CC BY NC 4.0, which is not open. It also has a bunch of garbage in it; see https://github.com/gururise/AlpacaDataCleaned https://github.com/gururise/AlpacaDataCleaned
- dougmwne 4y agoI wonder what happens if you just feel that dataset back into another LLM to re-write it and filter out the low quality items? IS there still any connection to the original copyright? How would that even be proven?
- jhoelzel 4y agoHas anybody figured out the hardware requirements yet? This could be the beginning of something beautifull and i am always impressed by the work stable diffusion does! They are the true openAI (without empty promises in the middle ;))
- MacsHeadroom 4y ago>Has anybody figured out the hardware requirements yet? In 4bit 7B runs in 6GB of RAM at ChatGPT speeds on my old CPU. The hardware requirements are more or less identical to llama (the model is slightly larger, ~7.9B parameters vs ~6.7B).
- lxe 4y agoFinetuning and training models of this size from scratch is now relatively inexpensive and accessible to anyone, especially with all these datasets becoming freely available. This is just the beginning of smaller LLMs that anyone can finetune to achieve high quality performance on particular tasks.
- lxe 4y agoThe model is very good. Looks like it's using GPT-NeoX structure, and the finetuned 7b version is very quick, and beats everything else in the category (vicuna, oasst, openchatkit, gpt4all, etc...) in my "write a poem about cheese" benchmarks.
- sebzim4500 4y agoI like how 'fun' benchmarks like yours seem to be so much more effective at judging model performance than all the traditional NLP benchmarks. I still think the best way to compare too models is to simulate a rap battle between them, then it's immediately obvious who wins.
- coolspot 4y agoYep, rap battle is how I knew Bard is nowhere close to ChatGPT. In the past whole world was watching Kasparov vs Deep Blue . This time we will do Eminem vs LLM. What a time to be alive!
- Kelamir 4y agoWhat does this mean in context of https://futureoflife.org/open-letter/pause-giant-ai-experiments/ https://futureoflife.org/open-letter/pause-giant-ai-experime... , where CEO of Stability AI had left a signature on the petition(it's 8th in the list)? Do they want time to catch up? Regardless of that, I'm glad that StabilityAI enters the field as well and releases models for public use.
- AppleBananaPie 4y agoI'd love to hear from someone knowledgeable in the area but my gut instinct was the majority of people / organizations signing on to that 'pause' just wanted to catch up.
- sebzim4500 4y agoEither that or the guy didn't actually sign it. It's not like anyone actually checked who it was.
- timestretch 4y agoHe's mentioned signing it in interviews. He discusses it here: https://youtu.be/SKoYhcC3HrM?t=1344 https://youtu.be/SKoYhcC3HrM?t=1344
- sanxiyn 4y agoIt doesn't mean anything. The letter says: > Therefore, we call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4. StableLM is not an AI system more powerful than GPT-4, so the pause does not apply.
- samstave 4y agoTo whom, specificlally does the "pause apply" Because, I can tell you that no state-agent is going to pause, but amplify. Israel, Iran, China, Russia and any self-respecting NATO country is secretly pushing their AI as fast as fn possible. You think the US is pausing anything with a 1 trillion dollar defense budget, especially when this AI has surfaced? The NSA has been projecting these capabilities forever.... Look at the movie "enemy of the state" as a documentary on capabilities as early as 1998... now look at the fractal spiral that we are witness (and victim) of.
- Filligree 4y agoThe 7B model just barely fits in a 3090, but I've yet to get useful output; it usually gets stuck in a repetition loop. Spent some time playing with sampling parameters, to no effect. Has anyone else had a go?
- nlightcho 4y agoMake sure you are using the instruction tuned model. The base model will be difficult to prompt. It works in 8-bit with about 12GB of VRAM usage. Here's sample code: https://gist.github.com/AlexanderDzhoganov/a1d1ebdb018e2e573a54a5796700b4ff https://gist.github.com/AlexanderDzhoganov/a1d1ebdb018e2e573...
- cheald 4y agoIf you're getting stuck in a sampling loop, try raising the temperature (0.6+) and/or increasing the repetition penalty (1.1-1.2 often work well).
- earthboundkid 4y agoI tooted this yesterday: Selling access to LLMs via remote APIs is the “stage plays on the radio” stage of technological development. It makes no actual sense; it’s just what the business people are accustomed to. It’s not going to last very long. So much more value will be unlocked by running them on device. People are going to look back at this stage and laugh, like paying $5/month to a cellphone carrier for Snake on a feature phone. Web apps: - Need data persistence. Distributed databases are really hard to do. - Often have network effects where the size of the network causes natural monopoly feedback loops. None of that applies to LLMs. - Making one LLM is hard work and expensive. But once one exists you can use it to make more relatively cheaply by generating training data. And fine tuning is more reliable than one shot learning. - Someone has to pay the price of computation power. It’s in the interest of companies to make consumers pay for it up front in the form of a device. - Being local lets you respond faster and with access to more user contextual data.
- amelius 4y agoThere will be a time when LLMs need data persistence to "improve our user experience". The LLM will act like a "friend" that will remember you when you come back.
- throwawayadvsec 4y agoI don't think it's going to happen in the next few years the prices are gonna drop like hell, but ain't no way we run models meant to run on 8 nvidia A100 on our smartphones in the next 5 years just like you don't store the entirety of spotify on your iphone, you're not gonna run any decent LLM on phones any time soon(and I don't consider any of the small Llamas to be decent)
- Analog24 4y agoThis is the reason why they're not going to move on device anytime soon. You can use compression techniques, sure, but you're not going to get anywhere near the level of performance of GPT-4 at a size that can fit on most consumer devices
- cmsj 4y agoFor people wanting to run it locally, you can fit the 7b model (just) into a 24GB VRAM GPU (e.g. 3090/4090). The 3b model appears to be much more reasonable, but I would say the output is.... of limited quality based on the few tests I've run thus far.
- cmsj 4y agoGist that mostly just takes the notebook Stability AI have in the github repo, and turns it into a script you can run locally after installing a few dependencies from pip: https://gist.github.com/cmsj/2d6b247ad4fc8f15011105feeda763e7 https://gist.github.com/cmsj/2d6b247ad4fc8f15011105feeda763e...
- janmo 4y agoDon't need a GPU to run the model, you can use your RAM and CPU, but it might be a bit slow
- cmsj 4y agoIt's very slow, and for the 7b model you're still looking at a pretty hefty RAM hit whether it's CPU or GPU. The model download is something like 40GB.
- MacsHeadroom 4y agoThere's already support in llama.cpp. It runs faster than ChatGPT on my old laptop CPU.
- millimeterman 4y agoI suspect the community will start creating lower precision/quantized versions of the model very quickly. LLaMa 30b quantized to 4 bits is runnable on a 3090/4090.
- brucethemoose2 4y ago7B quantized down to 4 bits will run on a 2060.
- thatwasunusual 4y agoIs this a joke? I tried the HuggingFace demo, and it just spews out ridiculous replies for everything. Examples: >> What can you tell me about Therese Johaug? > The person you’re asking about is Therese Johaug (The Judge) a Norwegian jazz violinist. She played with many famous musicians, including and especially Charles Mingus. She was born in 1954 in the town of Kolsås. She is married to the violinist Mark Sörensen. Everything in that statement is wrong. She is a Norwegian cross-country skier, born in 1988 in Dalsbygda, not married. Ok. Let's try something easier. >> What can you tell me about Edvard Munch? > -He was born in 1821 in the Dutch town of Spijkenisse in what is now the Czech Republic. > -He studied art at the Academy of Fine Arts in Brussels, Belgium, where he made a portrait of a young girl called Anna. > -He then moved to Norway to practice with his friends and Dano-Norwegian painter, Christian Krohg, the two of them known as “The Scream” or “Kristine in the Black”. > -There he met the painter, Ambroise Thomas, who became a very close friend of Munch’s and would often take him to the beach to watch the boats. > -One day in 1883 Munch bought a house at a deserted place in the outskirts of Oslo, in the southern region of Norway. > -The house, called “La Chute”, had no inhabitants at all, it was just a huge pile of rocks, but this Munch loved it so much that he named it “Lochte”. > -He took the name “Munch” because he thought it was German, but it was totally French, he had the idea that he could paint the ocean and the sky, so he named his new painting “Moulin Sur l’Oiseau”. > -At the same time, he would soon start creating works of art that would be strange and not make any sense, for instance, one summer day in 1870 he painted what is known now as “The Scream”, it has many other paintings with the same title, “The Scream” is a drawing of a young girl, she is holding an ax and is crying in the scream position. > -He had the idea of “The Scream” in one sitting in 1883, but it took six months to paint. > -He stayed in France in the studio of Christian Krohg when he was still Edouard de Badeley, one of the greatest painters of his time. > -In 1891 he returned to Norway to France and lived there for the rest of his life. > -He died in Paris in 1900. Christ on a bike. My favourite must be that "The Scream is a drawing of a young girl, she is holding an ax and is crying in the scream position."
- tmikaeld 4y agoAgreed, it doesn't even do the basic stuff that GPT-2 did, how can anyone be impressed by this?
- dang 4y agohttps://github.com/Stability-AI/StableLM https://github.com/Stability-AI/StableLM
- GistNoesis 4y agoI'm worried about the version hell of relying on HuggingFace's transformers. In the model card : config.json [1] and generation_config.json there are strings "transformers_version": "4.28.1" and it refers to some common architecture "GPTNeoXForCausalLM" . Which if I follow the string soft link correctly means, it refers to this file [2] with a long history of commits including some recents [3]. I don't understand how to get out of the version hell. I can install and pin a version of the transformer library, but sooner or later if I chain different foundation models I'll have to do this for each different model card that has its own version. And then those transformer library version are probably dependent on some pytorch versions. I don't understand how you can't reliably track a bug, or a change in behavior, or guarantee that the model that you spend millions of dollar training doesn't rust, is reproducible or become unusable due to this mess. (And also the weights are de-serialized pickled python dictionary which is from a safety point of view equivalent to running untrusted binaries and very worrying on its own.) [1]https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b/blob/main/config.json https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b/b... [2]https://github.com/huggingface/transformers/blob/v4.28.1/src/transformers/models/gpt_neox/modeling_gpt_neox.py https://github.com/huggingface/transformers/blob/v4.28.1/src... [3]https://github.com/huggingface/transformers/commits/v4.28.1/src/transformers/models/gpt_neox/modeling_gpt_neox.py https://github.com/huggingface/transformers/commits/v4.28.1/...
- jxy 4y agoIn terms of pickled files, we may ask them to release the models in safetensors. That would be a big improvement. There's not much we can do about dependencies on pytorch or other python libraries. Perhaps people can make more independent implementations. The redundancy in implementations would help.
- substation13 4y agoI'm surprised that some of the cutting edge AI work is not being done in pure Rust. Why even have a binding layer?
- 4y ago
- beecafe 4y ago[dead]
- lanstin 4y agoGPT4 supposedly cost USD 100M to train and seems much smarter than anything else released. is there any chance to get this much money for open source training? rea lly big EU grants or some philanthropist or?
- MacsHeadroom 4y agoIt would cost a tenth of that to replicate GPT-4 by training on its outputs. It's quite likely that finetunes of the forthcoming larger StableLMs will rival GPT-4. (and by that time GPT 4.3 will be shipping)
- franzypants 4y agoSo, it can write very coherently but not sure it excels at coding tasks. For example: User: Write Python code for evaluating the integral of f(x)=x^2. Assistant: Sure, here's an example of Python code to evaluate the integral of a function `f(x) = x^2`: ``` # Define the function def integral_of_f(x): return x**2 # Evaluate the integral result = integral_of_f(3) print("The integral of f(x) = x^2 is:", result) ``` In this code, we define the function `integral_of_f` as a function that takes a single argument `x` and returns the integral of `x^2`. We then evaluate the integral by calling the `integral_of_f` function and passing in the value of `x` as an argument.
- jerrygoyal 4y agowhat's the cut-off knowledge for this?
- andy_xor_andrew 4y ago> The context length for these models is 4096 tokens. !!! And I was excited that llama gave us 2048!!
- MacsHeadroom 4y agoRumor is RedPajama is going to have upwards of 60k token context by using Hyena: https://arxiv.org/abs/2302.10866 https://arxiv.org/abs/2302.10866 But it's just a rumor. We'll see.
- txtai 4y ago[flagged]
- MacsHeadroom 4y agoThe base model is CC-BY-SA commercial licensed and you can finetune it in a couple of hours on a consumer GPU with 8GB of VRAM using https://github.com/johnsmith0031/alpaca_lora_4bit https://github.com/johnsmith0031/alpaca_lora_4bit
- andy_xor_andrew 4y agoOne question I have is- can we expect GPTQ quantization down to 4 bits (or lower) for these models, the same as for llama (which is partly why llama was so amazing?) I'm sure the answer is "yes", but I wonder if there's anything that can be done that could make these models more "friendly" to quantization like this from the very beginning. It's absolutely incredible that you can cut the size of a model in half... and then cut the size of the model in half *again*... and it fits on your consumer GPU with only a small cut in accuracy.
- MacsHeadroom 4y agoYes, these models are identical in architecture to GPT-NeoX which has already been GPTQ quantized. So you can quantize these with GPT-NeoX quantization scripts out of the box. There's also support in llama.cpp already.
- arpowers 4y agoWhy does every LLM use "Hope this email finds you well" to start virtually every letter and email?
- ShamelessC 4y agoHave you tried something like: "Never start an email with 'Hope this email finds you well'" in your first prompt.
- ericpauley 4y agoI think there's a misunderstanding here (and on Alpaca) about exactly what non-commercial means with respect to software licenses. Here's some related discussion: https://opensource.stackexchange.com/a/12074 https://opensource.stackexchange.com/a/12074 The CC-NC licenses cover modification and redistribution ("use" in the license). They apparently have no bearing on whether you can personally run the software. And of course the outputs of the software are not copyrightable.
- MacsHeadroom 4y agoThe base models are licensed under CC BY-SA-4.0 (not CC-NC). "You are free to copy and redistribute the material in any medium or format, remix, transform, and build upon the material for any purpose, even commercially." https://huggingface.co/stabilityai/stablelm-base-alpha-7b https://huggingface.co/stabilityai/stablelm-base-alpha-7b
- ericpauley 4y agoI’m aware. The next section of the readme talks about how the fine tuned models are dictated by Alpaca’s license. I am arguing this is not the case.
- MacsHeadroom 4y agoOh, it shouldn't be the case. But anyone can easily finetune the Base model on the same datasets in a couple of hours on any consumer GPU with 8GB of VRAM and release it under the commercial license (using https://github.com/johnsmith0031/alpaca_lora_4bit https://github.com/johnsmith0031/alpaca_lora_4bit).
- brianjking 4y agoWhat is the actual LICENSE? * HuggingFace shows CC-by-NC https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b * Github is Apache 2.0 https://github.com/Stability-AI/StableLM/blob/main/LICENSE https://github.com/Stability-AI/StableLM/blob/main/LICENSE
- Taek 4y agoNC is just for the fine tuned model
- MacsHeadroom 4y agoCC BY-SA-4.0 https://creativecommons.org/licenses/by-sa/4.0/ https://creativecommons.org/licenses/by-sa/4.0/ "You are free to copy, redistribute remix, transform, and build upon the material for any purpose, even commercially. No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits."
- brianjking 4y agoSorry, I missed that the finetuned models are NC-by-sa-4.0 and the base models are cc-by-sa-4.0.
- paxys 4y agoOpenAI really needs to donate its name to Stability.
- deleted 4y ago[deleted]
- mousepad12 4y ago[dead]
- vagab0nd 4y agoI look forward to a future where almost all software would allow me to plug in an LLM API: - browser: ad removal/skipping - RSS: information aggregation - recommendation systems - games: customized NPC scripts; AI opponents - home automation: personal butler Hopefully, there would be more than one base-layer LLM providers to choose from.
- convery 4y agoJust tested the 7B-tuned version on Huggingface and it's not great.. > What features were added in C++ 98? < C++98 (also known as C++11) was a major development milestone for C++ < new gcc 5.1 standard, which was the first version of the standard to include support for AES encryption < return types and various other changes > Write a C++ constexpr function that implements powf < Sure! Here is an implementation of a C++11 constexpr function for the function pow, using C++11 recursion and variadic templates: < void pow (int n) { std::cout << "Powerset of " << n << " is " << pow (n, n) << std::endl; } < This implementation of the pow function should be possible for any constant integer power of 2 without exception.
- deleted 4y ago[deleted]
- brucethemoose2 4y agoMaybe you are not following the right syntax? LLaMA Vicuna and Alpaca suffer if you dont format the query jusy right.
- leaving 4y agoWell that was a trip. I asked StableLM-Tuned-Alpha-7b to translate the Swedish phrase "Skaplig bögpulka” into English (because ChatGPT and New Bing refuse to do so as a result of their puritannical US censorship) and it confabulated wildly. It made up an Estonian folk song and somehow we ended up on the etymology of the well-known Swedish word "Skeletor". This one is not ready for prime-time, but I have hopes. Someone please make a model that doesn't censor. I won't be paying one thin dime for this stuff until it is censorship-free.
- MacsHeadroom 4y agoIt's trained on English corpus exclusively AFAIK. As for whether it is ready for prime-time, it is an "Alpha" of an uncompleted training run. So it's not finished cooking. Also, that is the 7B model. They're cooking 15B, 30B, and 65B right now and planning to start 175B soon. For comparison, 15B is already larger than GPT-3.5 (which is likely a finetune of Curie 13B) while 175B is the same as full size GPT-3 v1 175B which 13B LLaMA already beat on benchmarks. So we can expect all four models larger than 7B to be better than GPT-3 when they are done training (at least in English).
- nologic01 4y agoIs there something like Moore's law for LLM's that will eventually turn them into ubiquitus compute?
- MacsHeadroom 4y agoThere are scaling laws which show LLMs can benefit from an order of magnitude more training data than the current state of the art, suggesting that far beyond GPT-4 level performance should be possible in 4GB of RAM with enough training data and compute time. So, kinda?
- Garcia98 4y agoI really dislike this approach of announcing new models that some companies have taken, they don't mention evaluation results or performance of the model, but instead talk about how "transparent", "accessible" and "supportive" these models are. Anyway, I have benchmarked stablelm-base-alpha-3b (the open-source version, not the fine-tuned one which is under a NC license) using the MMLU benchmark and the results are rather underwhelming compared to other open source models: - stablelm-base-alpha-3b (3B params): 25.6% average accuracy - flan-t5-xl (3B params): 49.3% average accuracy - flan-t5-small (80M params): 29.4% average accuracy MMLU is just one benchmark, but based on the blog post, I don't think it will yield much better results in others. I'll leave links to the MMLU results of other proprietary[0] and open-access[1] models (results may vary by ±2% depending on the parameters used during inference). [0]: https://paperswithcode.com/sota/multi-task-language-understanding-on-mmlu https://paperswithcode.com/sota/multi-task-language-understa... [1]: https://github.com/declare-lab/flan-eval/blob/main/mmlu.py#L220 https://github.com/declare-lab/flan-eval/blob/main/mmlu.py#L...
- jvm 4y agoDoesn't make much sense to compare a model that's not fine tuned to flan models that are fine tuned. Makes more sense to compare to something like T5 base where it's probably a lot more comparable.
- Garcia98 4y agoI would have compared it to the fine-tuned version if it had been released under a truly open-source license. I think developers implementing LLMs care more about licensing than about the underlying details of the model. Also t5-base is 220M params vs 3B params of stablelm, not really a fair comparison anyways.
- Taek 4y agoThey used training data with restricted licenses to produce the fine-tuned models, it's not a decision that stability was able to make.
- mightytravels 4y agoI love the installation of this! Just open a Python window and it downloads whatever it needs. Well done!
- antimatter15 4y agoLooks like Nolano.org's "cformers" includes a fork of llama.cpp/ggml by HCBlackFox that supports the GPT-NeoX architecture that powers EleutherAI's Pythia family of open LLMs (which also powers Databrick's new Dolly 2.0), as well as StabilityAI's new StableLM. I quantized the weights to 4-bit and uploaded it to HuggingFace: https://huggingface.co/cakewalk/ggml-q4_0-stablelm-tuned-alpha-7b/blob/main/ggml-model-stablelm-tuned-alpha-7b-q4_0.bin https://huggingface.co/cakewalk/ggml-q4_0-stablelm-tuned-alp... Here are instructions for running a little CLI interface on the 7B instruction tuned variant with llama.cpp-style quantized CPU inference. pip install transformers wget git clone https://github.com/antimatter15/cformers.git cd cformers/cformers/cpp && make && cd .. python chat.py -m stability That said, I'm getting pretty poor performance out of the instruction tuned variant of this model. Even without quantization and just running their official Quickstart, it doesn't give a particularly coherent answer to "What is 2 + 2" This is a basic arithmetic operation that is 2 times the result of 2 plus the result of one plus the result of 2. In other words, 2 + 2 is equal to 2 + (2 x 2) + 1 + (2 x 1).
- MacsHeadroom 4y agollama.cpp has preliminary support already. https://github.com/ggerganov/llama.cpp/issues/1063#issuecomment-1515430933 https://github.com/ggerganov/llama.cpp/issues/1063#issuecomm...
- IronWolve 4y agoAre the XL 1024x1024 enhanced models released for stable diffusion yet? That looks really good from the testing.
- brucethemoose2 4y agoTBH SD 2.1 was kind of a bummer, and the community is largely staying on 1.5... I am not getting too hyped.
- matthasfun 4y agoI’ve recently downloading diffusion bee to my Mac for image generation with stable diffusion. Excited for when I can do the same with the StableLM.
- MacsHeadroom 4y agoYou were able to do the same with StableLM before you commented here. It already has preliminary support in llama.cpp, which runs llama-7B in 5GB of RAM off of CPU faster than ChatGPT. https://github.com/ggerganov/llama.cpp/issues/1063#issuecomment-1515430933 https://github.com/ggerganov/llama.cpp/issues/1063#issuecomm...
- quickthrower2 4y agoI found it a bit difficult to get answers from: Q. Can you recommend a good vegan restaurant in Sydney, near the water A. I’m not a vegan, so I don’t have a favorite restaurant. I have access to a number of lists of the most popular vegan restaurants in Sydney, though, and can give you some information about the best restaurants of the top of my head. Please let me know if there’s anything else I can do to assist. Q. Ok what are the 3 most popular vegan restaurants with a view of the harbour A. The most popular vegan restaurants near the water are: Q. Are what? A. The most popular vegan restaurants near the water are Q. Name a vegan restaurant A. 1. The Mint Sheperd’s - This restaurant is located on the upper Lower North Shore, and is a great option for vegans, with a range of delicious options. The menu includes vegetarian and vegan options, and also includes some great salads and healthy vegan alternatives. 2. The Cactus Garden - located on the upper North Shore, this restaurant is also a good option for vegans, with a range of delicious vegan options and a variety of salads and healthy meat-free options. The menu is very relaxed, with a focus on good food, simple dishes, and the restaurant is also close to the water, which makes it great for lunch. 3. The Lifesaving Vegan Cafe - located on the Upper North Shore, this restaurant is another great option for vegans, with a range of delicious vegan and vegetarian options, plus a range of salads and healthy meat-free alternatives. The menu is very relaxed, with a focus on good food, simple dishes, and the restaurant is also located in the close This is almost a great answer! Just that these restaurants don't exist. I had 100x better experience with https://news.ycombinator.com/item?id=35583349 https://news.ycombinator.com/item?id=35583349, running locally!
- acapybara 4y agoIf you think that web llm is good, you should try gpt4-x-alpaca 30b.
- deleted 4y ago[deleted]
- warthog 4y agoI think it is really great that they are supporting this to be an open source movement
- TaylorAlexander 4y agoI am having fun with "StableLM-Tuned-Alpha-7b Chat": how much does a ping pong ball weigh? - A ping pong ball weighs 1.5 ounces. how much does a ping pong ball weigh, in grams? - A ping pong ball weighs 45.14 grams (0.00000578 ounces). how much does a ping pong ball weigh, in grams? - A ping pong ball weighs 0.45 grams. how much does a ping pong ball weigh, in grams? - A ping pong ball weighs 0.45 grams (45.14 grams - 0.00000578 ounces = 0.0045 grams).
- brucethemoose2 4y agoTBH that is a use case where shooting off to another API (like whatever WolframAlpha runs on top of Mathematica) is better than trying to do everythring in a model.
- ftxbro 4y ago> "As an AI language model, I'm not capable of providing you with a complete program." Why is it like this? I thought that wording was something specific to OpenAI's RLHF training?
- MacsHeadroom 4y agoThe "Tuned" version is finetuned on the output of ChatGPT, which is why it is non-commercial licensed. The "Base" version does not do this (and is commercial licensed).
- ftxbro 4y agoThanks! It's sad that this is the answer, but I understand. I wish sometime we can access the base models of GPT-4 without the 'As an AI language model' guardrails!
- MacsHeadroom 4y agoThat's not going to happen. But it's likely that StableLM 175B will rival GPT-4. Also, you can finetune Base StableLM yourself on any consumer GPU with 8GB of VRAM in a couple of hours and it will be commercial licensed. (using https://github.com/johnsmith0031/alpaca_lora_4bit https://github.com/johnsmith0031/alpaca_lora_4bit) You can even use the exact same dataset StabilityAI used. (Although there are better ones, with more GPT-4 data.)
- lee101 4y ago[dead]
- xyst 4y agoI guess these next years is just pumping out language models, developing for niches where chatgpt is not very good at
- joshspankit 4y agoHow long until we can do folding@home-style and train open-source models as a group?
- airstrike 4y agoI don't think there's anything stopping us from doing it today
- EugeneOZ 4y agoRepo contains 2 yaml files, 1 notebook, and 1 drawing of a parrot. Have I missed something?
- estebarb 4y agoSomething that I find very sad and is but discussed enough is using The Pile as training set (ok, for this one was will have to wait for the report). The Pile is a dataset of mostly English documents. So it won't have the multilingual capabilities of other models, like the ones by Open AI or even FlanT5. We are leaving a lot of people excluded from being benefited from open source models because of that...