6 ms·
First test I tried to run a random taxation question through it Output: https://gist.github.com/IAmStoxe/7fb224225ff13b1902b6d1724678ea49 https://gist.github.c
by byteknight 2y ago
First test I tried to run a random taxation question through it
Output: https://gist.github.com/IAmStoxe/7fb224225ff13b1902b6d1724678ea49 https://gist.github.com/IAmStoxe/7fb224225ff13b1902b6d172467...
Within the first paragraph, it outputs:
> GET AN ESSAY WRITTEN FOR YOU FROM AS LOW AS $13/PAGE
Thought that was hilarious.
- orost 2y agoThat's not the model this post is about. You used the base model, not trained for tasks. (The instruct model is probably not on ollama yet.)
- byteknight 2y agoI absolutely did not: ollama run mixtral:8x22b EDIT: I like how you ninja-editted your comment ;)
- orost 2y agoConsidering "mixtral:8x22b" on ollama was last updated yesterday, and Mixtral-8x22B-Instruct-v0.1 (the topic of this post) was released about 2 hours ago, they are not the same model.
- byteknight 2y agoAre we looking at the same page? https://imgur.com/a/y6XfpBl https://imgur.com/a/y6XfpBl And even the direct tag page: https://ollama.com/library/mixtral:8x22b https://ollama.com/library/mixtral:8x22b shows 40-something minutes ago: https://imgur.com/a/WNhv70B https://imgur.com/a/WNhv70B
- orost 2y agoLet me clarify. Mixtral-8x22B-v0.1 was released a couple days ago. The "mixtral:8x22b" tag on ollama currently refers to it, so it's what you got when you did "ollama run mixtral:8x22b". It's a base model only capable of text completion, not any other tasks, which is why you got a terrible result when you gave it instructions. Mixtral-8x22B-Instruct-v0.1 is an instruction-following model based on Mixtral-8x22B-v0.1. It was released two hours ago and it's what this post is about. (The last updated 44 minutes ago refers to the entire "mixtral" collection.)
- belter 2y agoI get: ollama run mixtral:8x22b Error: exception create_tensor: tensor 'blk.0.ffn_gate.0.weight' not found
- gliptic 2y agoAnd where does it say that's the instruct model?
- deleted 2y ago[deleted]
- mysteria 2y agoYeah this is exactly what happens when you ask a base model a question. It'll just attempt to continue what you already wrote based off its training set, so if you say have it continue a story you've written it may wrap up the story and then ask you to subscribe for part 2, followed by a bunch of social media comments with reviews.
- Sohcahtoa82 2y agoIt can be fun, though, to prompt a text completion with something like "I'm thinking about" and just seeing what random thing it completes it with.
- woadwarrior01 2y agoLooks like an issue with the quantization that ollama (i.e llama.cpp) uses and not the model itself. It's common knowledge from Mixtral 8x7B that quantizing the MoE gates is pernicious to model perplexity. And yet they continue to do it. :)
- deleted 2y ago[deleted]
- cjbprime 2y agoNo, it's unrelated to quantization, they just weren't using the instruct model.
- jmorgan 2y agoThe `mixtral:8x22b` tag still points to the text completion model – instruct is on the way, sorry! Update: mixtral:8x22b now points to the instruct model: ollama pull mixtral:8x22b ollama run mixtral:8x22b
- Zuiii 2y agoWait. Isn't it a breaking change to change the underlying model like this? Wouldn't people start running into consistency issues in production? (given ollama appears to be oriented towards backend use)
- orra 2y agoSure, in theory. But if you move so fast that you already are running the base 8x22B model from last week, you can easily fix this. I've long thought that if you want reproducibility and reliability, you need to pin your deps. So, IMO, the change is very much worth it to reduce confusion going forward.
- renewiltord 2y agoNot instruct tuned. You're (actually) "holding it wrong".
- enduser 2y ago[dead]