3 ms·

Mistral 7B parameter models are quite good Already fine tuned and conversational its like education is more important than needing a trillion parameter braini

• ben_w 3y ago
  There's a lot we don't know. Human brains appear to be a few hundred trillion parameters, while small rodents are in the realm of tens to hundreds of billions. Would you guess a single sufficiently trained ferret could write on demand short stories about Dracula, Winnie the Poo, and Sherlock teaming up, and follow this up with a bit of university student level web development, and finally give you a decent apple cake recipe? I wouldn't have, and yet the LLMs exist and are much better than I was expecting. (People who dismiss SotA models as "stochastic parrots" confuse me as much as people who think they're already superhuman; the Markov chains and RNNs I coded a few years back didn't come close to last year's LLMs). That even smaller models can do well is both unsurprising (why would we expect our existing design efforts to already have the most efficient architecture?) and very surprising (how come we can get something so much less complex than our biological brains to do so much so well?)
  • yieldcrv 3y ago
    maybe ferrets would do better with better interfaces. with better ways to interact with the world and better co-processor. but the main brain might already be capable of those aforementioned things.