3 ms·
Why don't we need them? If I need to run a hundred small models to get a given level of quality, what's the difference to me between that and running one large
by stavros 9mo ago
Why don't we need them? If I need to run a hundred small models to get a given level of quality, what's the difference to me between that and running one large model?
- spockz 9mo agoYou can run smaller models on smaller compute hardware and split the compute. For large models you need to be able to fit the whole model in memory to get any decent throughput.
- stavros 9mo agoAh interesting, I didn't realize MoE doesn't need to all run in the same place.
- 5o1ecist 9mo ago[flagged]
- Someone 9mo ago> There is not one person on the planet, who wouldn't prefer a doctor who is deeply considerate of the complexities and feedback-loops of the human body, over a doctor who is simply not smart enough to do so and, thus, can't. He can learn texts all he wants, but the memorization of text does not require deeper understanding. But a smart person who hasn’t read all the texts won’t be a good doctor, either. Chess players spend enormous amounts of time studying openings for a reason. > Multiple small models, specifically trained for high reasoning/cognitive capabilities, given access to relevant texts So, even assuming that one can train a model on reasoning/cognitive abilities, how does one pick the relevant texts for a desired outcome?