3 ms·

I don’t think we should use an AI trained in 5 minutes on a laptop to infer what small models are capable of… Sure they still have massive problems with halluc

• gambiting 1y ago
  Why not? And I'm not being flippant, but like....isn't that the whole point of small models?
  • kevinventullo 1y ago
    As I understand it, the most effective small models are synthesized from larger models.
  • remexre 1y ago
    For one thing, the model is trained on a language modelling task, not a question-answering task?
  • Closi 1y ago
    I guess different small models will have different points/goals, but you can still have a small model with lots of training effort or a large model with little training effort. I think the point of most (frontier) small models is usually to provide the best answer possible given small inference resources, rather than to reduce training time. This is more of a toy model, so fun and an interesting project but it doesn't necessarily tell us what the art of the possible is for small models.