2 ms·
EDIT: it seems this is likely an Ollama bug, please keep that in mind for the rest of this comment :) I ran Gemma in Ollama and noticed two things. First, it i
by LorenDB 3y ago
EDIT: it seems this is likely an Ollama bug, please keep that in mind for the rest of this comment :)
I ran Gemma in Ollama and noticed two things. First, it is slow. Gemma got less than 40 tok/s while Llama 2 7B got over 80 tok/s. Second, it is very bad at output generation. I said "hi", and it responded this:
```
Hi, . What is up? melizing with you today!
What would you like to talk about or hear from me on this fine day??
```
With longer and more complex prompts it goes completely off the rails. Here's a snippet from its response to "Explain how to use Qt to get the current IP from https://icanhazip.com https://icanhazip.com":
``` python
print( "Error consonming IP arrangration at [local machine's hostname]. Please try fufing this function later!") ##
guanomment messages are typically displayed using QtWidgets.MessageBox
```
Do you see similar results on your end or is this just a bug in Ollama? I have a terrible suspicion that this might be a completely flawed model, but I'm holding out hope that Ollama just has a bug somewhere.
- mark_l_watson 3y agoI was going to try these models with Ollama. Did you use a small number of bits/quantization?
- LorenDB 3y agoThe problem exists with the default 7B model. I don't know if different quantizations would fix the problem. The 2B model is fine, though.