5 ms·
WizardLM 2
- exe34 3y agoAny idea what the context length is? Can't see it from the page linked.
- titaniumtown 3y agoI assume the same as the models that they're fine tuned off of.
- spmurrayzzz 3y agoFor future reference, you can always check the `config.json` file in the repo for this info. Specifically `max_position_embeddings` is the key you're looking for. 7B: 32k 8x22B: 65k
- exe34 3y agoThanks!
- brokensegue 3y agoseems to be roughly the same capacity as Command R+ if the arena leader board is to be trusted. I think Command R+ has fewer params?
- lolinder 3y agoI don't see WizardLM 2 on the leaderboard yet—are you looking at the previous version? Or do you have a link to a different leader board? https://chat.lmsys.org https://chat.lmsys.org
- brokensegue 3y agoI said this based on the comparisons in the OP to other models which are on the leaderboard
- reissbaker 3y agoWizardLM-2 8x22B appears to beat Command-R+ in their measurements.
- tosh 3y agoa bit hidden but afaiu: * 8x22B (based on Mistral 8x22B) * 70B (based on llama2?) * 7B (based on Mistral-7B-v0.1) and the 70B model is not available yet on huggingface https://huggingface.co/collections/microsoft/wizardlm-661d403f71e6c8257dbd598a https://huggingface.co/collections/microsoft/wizardlm-661d40...
- smusamashah 3y agoWhat does "state-of-art" mean? Every one seems to be using this while in my head SOTA means "top of the line". When it says comparable results in benchmarks, how is it SOTA? Or does SOTA mean topping on really really specific benchmarks?
- striking 3y agoThere's a table of MT-Bench scores. I presume we'll have to wait for the preprint before we know anything else.
- spxneo 3y agoWizardLLM beats GPT4-0314 nearly close to the latest release GPT-4-1106-preview
- mrtesthah 3y agoBy that benchmark, Claude Sonnet beats both.
- MacsHeadroom 3y agoAnd Claude Sonnet is great. It's what I recommend people use if they don't want to pay for an LLM or run one locally. But Claude Sonnet is not open source and can't be run on your own hardware like Wizard.
- stavros 3y agoHow do you use Sonnet for free?
- MacsHeadroom 3y agoAt https://poe.com/ https://poe.com/
- stavros 3y agoThanks!
- Vetch 3y agoPoe's Sonnet is limited to 15 free messages per day. The best freely accessible LLM with a generous daily allotment (300 msgs/day) is Bing Green Precise Mode. It's at about GPT4 level.
- spxneo 3y agohow are they offering this at discounts from the original? looks like the goal is to get subs to cover for the most frequent users?
- YetAnotherNick 3y agoWhile I want to be very optimistic about open models, the fact that almost all top open models heavily use GPT-4 for synthetic data is discouraging. Even now it is against TOS and over time I think over time OpenAI would become better at detecting meaning the gap between open and closed models would increase.
- dimask 3y agoConsidering these models are released by "Microsoft AI", I doubt they do anything against the ToS of "Open"AI.
- dartos 3y agoI doubt it. They’re using gpt-4 data because it’s cheaper than getting real data, not because it’s better. If openai cares enough to stop people scraping responses, then we’ll just crowdsource them like open assistant did with their dataset.
- YetAnotherNick 3y agoThere is some news OpenAI spent 4 billion dollars for data sourcing.
- dartos 3y agoYeah but that’s what OpenAI might’ve spent
- 0x008 3y agoWhat can OpenAI realistically do about it except block the user account? I think the TOS should have no effect on the license and copyright of the resulting model, since outputs of AI Models cannot be copyrighted (at least in the US).
- YetAnotherNick 3y agoOpenAI would likely start using AI to detect the content generation and ban the users. It wouldn't even need to be GPT level. Just they could use some AI labelling and counting.
- Patrick_Devine 3y agoThe 7B model is available on ollama if you want to try it: `ollama run wizardlm2` or `ollama run wizardlm2:7b`. We're still crunching the 8x22B model to get it ready, and the 70B model isn't yet available.
- syntaxing 3y agoIf you can computationally afford it, 7b-q5_K_M is a way better choice. Default :7B goes to q4_0 which might give you subpar results.
- xyc 3y agoTried 7b q5 with some RAG tasks. Seems quite impressive.
- dimask 3y agoAnd quantised gguf files by Bartowski if somebody wants to download and run through llama.cpp directly https://huggingface.co/bartowski/WizardLM-2-7B-GGUF https://huggingface.co/bartowski/WizardLM-2-7B-GGUF
- xyc 3y agoAlso gguf files by abroxis: https://huggingface.co/ABX-AI/WizardLM-2-7B-GGUF-IQ-Imatrix https://huggingface.co/ABX-AI/WizardLM-2-7B-GGUF-IQ-Imatrix
- Patrick_Devine 3y agoFor those adventurous souls who have gobs of memory, a good GPU, and plenty of disk space, some of the 8x22b models are now up. Use `ollama run wizardlm2:8x22b-q4_0`, `ollama run wizardlm2:8x22b-q8_0`, or `ollama run wizardlm2:8x22b-fp16`. The 4 bit quantized version is about 80GB, the 8 bit quantized version is about 150GB, and the non-quantized version is 281GB (!).
- Me1000 3y agoThe 4bit quant doesn't seem to work for me, I keep getting: Error: exception create_tensor: tensor 'blk.0.ffn_gate.0.weight' not found I've tried downloading it twice now.
- mcbuilder 3y agoMixtral8x22B looking very strong! Finetunes seem comparable to GPT4!
- littlestymaar 3y agoThat's quite crazy to see that a model that's barely bigger than GPT-3 (and only uses a fraction of the compute due to its MoE architecture) can achieve such a thing. It looks like the people who focasted that AI models would need to keep growing to improve their performance where completely misguided. I wonder if in 3 to 4 years we'll end up with models with less than 4B parameters we comparable performance as today's State of the Art.
- int_19h 3y agoIt can also simply mean that benchmarks are not particularly representative of real-world performance on challenging reasoning tasks.
- littlestymaar 3y agoOf course they aren't, but it's still pretty evident that most opensource models are miles ahead of GTP-3, even the ones that are only a fraction of its size so there's still some massive improvement that doesn't depend on the model size itself.
- int_19h 3y agoGPT-3 sure. There's no model that's even close to GPT-4, though. And we don't really know much about the actual size of that, only lots of speculation.
- littlestymaar 3y agoMy entire point since the beginning is that the models out there improved a ton compared to what used to be the state of the art just 4 years ago, with an equivalent when not much smaller number of parameters (GPT-3 was 175B params and on MMLU it performs worse than Mistral 7B [1]). [1]: https://paperswithcode.com/sota/multi-task-language-understanding-on-mmlu https://paperswithcode.com/sota/multi-task-language-understa...
- zamalek 3y agoHow are folks running 8x22B (or MoE in general) in an affordable way?
- dartos 3y agoFor personal inference, llama.cpp can run some MoEs on CPU. So like that.
- PcChip 3y agoCan I run this locally with a 4090? If so, what’s the easiest way?
- vunderba 3y agoEven a 4-bit quant version of the MoE 8x22 is going eat ~80GB of VRAM. No way you're running this on a 4090 without setting it up as a GGUF to split between VRAM and regular RAM, and then you're going to have to deal with the low token rate as a result. Running WizardLM-2 70B or lower WizardLM-2 7B is much more feasible however. Look into Ollama: https://ollama.com/library/wizardlm2/tags https://ollama.com/library/wizardlm2/tags
- 0x008 3y agoThey are based on llama2 architecture which performs a lot worse.
- mark_l_watson 3y agoI have just started experimenting with the 7B Q4 model by running "ollama run wizardlm2" and so far it seems to be very good at arithmetic word problems. I usually use Mistral 7B-Q4 with Ollama's REST interface. I am curious about and looking forward to slightly adjust the prompts I use for wizardlm2 and see how it does.
- samspenc 3y agoLatest update: this link no longer works, leads to a 404 on GitHub. Folks on Reddit are reporting that it has disappeared from HuggingFace as well. Does anyone know what is going on? Were there any announcements on its being unpublished?
- segmondy 3y agoUnpublished, they yanked everything off.
- 0xDEADFED5 3y agohere's the 7B: https://huggingface.co/amazingvince/Not-WizardLM-2-7B https://huggingface.co/amazingvince/Not-WizardLM-2-7B
- stevenhuang 3y agoapparently it's because they forgot to censor it: https://www.reddit.com/r/LocalLLaMA/comments/1c586rm/wizardlm2_was_deleted_because_they_forgot_to_test/ https://www.reddit.com/r/LocalLLaMA/comments/1c586rm/wizardl...
- Aissen 3y agoDeleted. Archive: http://web.archive.org/web/20240415221214/https://wizardlm.github.io/WizardLM2/ http://web.archive.org/web/20240415221214/https://wizardlm.g... https://github.com/nlpxucan/WizardLM/commit/4218c0ba64a84cfd0b6650ac4da8110bd17aae76 https://github.com/nlpxucan/WizardLM/commit/4218c0ba64a84cfd...
- Aissen 3y agoReason: https://twitter.com/WizardLM_AI/status/1780101465950105775 https://twitter.com/WizardLM_AI/status/1780101465950105775 (they forgot toxicity testing before release and are doing it now)
- Alifatisk 3y agoWhere do people check the leaderboard of the best performing LLM? The latest news I had was the best perfoming was claude.ai.
- sunaookami 3y agoA good leaderboard is https://arena.lmsys.org/ https://arena.lmsys.org/ (the "Leaderboard" tab).