5 ms·
StableLM Zephyr 3B
- josh-sematic 3y agoNote that it uses a non-commercial license. Still pretty cool though!
- Reubend 3y agoYeah, I think this is a great release, but I also suspect that most people won't end up using it just because of the license. It's actually a lot more restrictive than what I would personally consider "commercial" usage: > Non-Commercial Uses does not include any production use of the Software Products or any Derivative Works. So even if you want to launch a free service using this, that's not allowed.
- m3kw9 3y agoHow would one go about making a .llamafile for this?
- simonw 3y agoConvert it to GGML and use a zip tool to add that to a llamafile package. https://huggingface.co/TheBloke?search_models=Zephyr https://huggingface.co/TheBloke?search_models=Zephyr doesn't have a GGML for it yet but I wouldn't be surprised to see one by the end of the day.
- deleted 3y ago[deleted]
- pleks 3y agoAnd it's been uploaded https://huggingface.co/TheBloke/stablelm-zephyr-3b-GGUF/tree/main https://huggingface.co/TheBloke/stablelm-zephyr-3b-GGUF/tree... Getting 20toks/s on M1 mba where as LLaVa I ground to a halt. Very impressed
- adamkochanowicz 3y agoCan't wait for someone smarter than me to make this compatible with MLC on iPhone.
- haltist 3y agoYou can do this by just following a tutorial: https://huggingface.co/docs/diffusers/main/en/using-diffusers/using_safetensors https://huggingface.co/docs/diffusers/main/en/using-diffuser.... ML/AI models are just function graphs and most of the frameworks support saving and loading safetensor serialized graphs.
- simonw 3y ago"This model is being released under a non-commercial license that permits non-commercial use." I'm very interested in high quality 3B models, but it's hard to get excited about this given the increasing array of commercially usable models.
- brianjking 3y agoYeah, Replit is likely the best option out there for a 3B model size, right?
- filterfiber 3y agoRefact has a decent 1.6B model that I think is better https://huggingface.co/smallcloudai/Refact-1_6B-fim https://huggingface.co/smallcloudai/Refact-1_6B-fim
- anigbrowl 3y agoThey have more fully open stuff in the pipeline. IMHO it's good that they put out stuff for hobbyists to play around with so that they're not immediately overtaken by people ready to deploy things at commercial scale.
- emadm 3y agoIt will be included under our membership next week which starts at $1 a month after grant ($20 base)
- makin 3y agoThe parent comment was referring to free ($0 a month) models. With those, companies don't need to plan around the possibility that Stability AI hikes up the price afterwards.
- emadm 3y agoIt’ll be static with cpi max, flat membership for all core models Just released video, sdxl turbo and 3d, code and more coming Very positive reaction so far and we will still do our grants for OSS and do OSS collaborations, done over 10m A100 hours over last year Launches next few days https://x.com/emostaque/status/1732197072290353455?s=46 https://x.com/emostaque/status/1732197072290353455?s=46
- deleted 3y ago[deleted]
- simlevesque 3y agoHow fast are these small models on a 4090, is it like 100ms ? 500ms ?
- stavros 3y agoAm I reading it right that performance was roughly comparable with GPT-3.5? How is this even possible?
- filterfiber 3y agoNo it's not (according to their benchmarks). Zephyr-7B-B still beats it in most benchmarks but it's close. This model is almost Zephyr-7B-B performance at 3B size which is a lot better for inference requirements.
- emadm 3y agoYeah got a way to beat 3.5 but it beats most of the first generation llama tunes even guacano 65b Lots of improvements to go
- alsodumb 3y agoBy comparing on benchmarks that are either limited, or have data leaks, or in most cases just don't make sense in terms of usability - I've personally stopped looking at benchmarks to compare models. Personally, if I want to try a new model I hear a lot of chatter about, I use it for a few hours in my daily workflow. My baseline is GPT3.5 and GPT4, and I compare the models with them in terms of my day to day usage.
- kouteiheika 3y agoSo in your experience which open model is currently the best?
- Version467 3y agoNot really. They already chose to show the benchmark where it does best and even then it’s still quite a bit worse (though definitely impressive for its size). If you take a look at other benchmarks, for example MMLU@5-shot then this does 46.3, while gpt-3.5 does 70. But there might be some use cases where this one is close enough in performance and the difference in cost and speed make it a better choice.
- 3y ago
- filterfiber 3y ago> Hardware: StableLM Zephyr 3B was trained on the Stability AI cluster across 8 nodes with 8 A100 80GBs GPUs for each nodes. I might be missing it but do they say the number of training tokens that was used to train this? This would help with efforts like TinyLlama in trying to figure out how well the scaling works with training tokens vs parameter size and challenging the chinchilla model.
- emadm 3y agoWe included full training details for the base model on 4 trillion tokens including wandb etc https://stability.wandb.io/stability-llm/stable-lm/reports/StableLM-3B-4E1T--VmlldzoyMjU4?accessToken=u3zujipenkx5g7rtcj9qojjgxpconyjktjkli2po09nffrffdhhchq045vp0wyfo https://stability.wandb.io/stability-llm/stable-lm/reports/S...
- nextworddev 3y agoStability is apparently up for sale, hence the recent steady stream of releases
- supermatt 3y agoThis space is so confusing when it comes to licenses. “Zephyr” is MIT, but “Stability Zephyr” is non commercial. They could have at least used a different name. “Inspired” in all but license it would seem
- mirekrusin 3y agoHow those licenses work for generated content? If it's non-commercial does it mean I can still use it for work to generate stuff? In other words - is it similar to ie. using GIMP, which is open source, but I can still use created content in commercial product without attribution?