4 ms·
My test for open models is surprisingly simple. I just ask "What is the capital of France?" and I haven't had a correct answer yet in any model I tried. They o
by seszett 3y ago
My test for open models is surprisingly simple.
I just ask "What is the capital of France?" and I haven't had a correct answer yet in any model I tried. They often get Paris right at least, but most other details are wrong.
Guanaco says:
> The current capital of France is Paris. It has been so since 1982 when it replaced the previous one which was Vichy.
- simonw 3y agoI just got "The capital of France is Paris." from vicuna-v1-7B running entirely on my iPhone (using the MLC Chat app).
- edgyquant 3y ago>They often get Paris right at least, but most other details are wrong.
- whimsicalism 3y agoWhat other detail in that statement is wrong?
- ac29 3y agoGuanaco 7B gives the same exact answer.
- jhbadger 3y agoWhat model size? 7B? 13B? 33B? When people say "Guanaco" or "Alpaca" or "Manticore" they are referring to a whole family of models. The really small ones like 7B often are very weird.