3 ms·
This is really cool, I'm curious how much knowledge can their be in smaller models? It seems the current trade off is you need sizeably larger models for more p
by tolugenius 2mo ago
This is really cool, I'm curious how much knowledge can their be in smaller models? It seems the current trade off is you need sizeably larger models for more performance but I'm curious if in your work how far this is true, as edge ai is really what needs to get better before physical ai can take off (my two cents).
- msdz 2mo agoI imagine at such a low parameter count, there would be little to no world knowledge whatsoever, and the entire focus is on getting the structure of tool calling etc. right…? But yeah, in terms of “physical” AI, robotics definitely comes to mind for me as well, where tool calls/structured “device” use in a “realtime”/edge application are highly beneficial (if you wanted to go with LLMs), but beefy hardware can’t be easily used.
- rshemet 2mo agoRoman from Cactus here - yes you're right, there's only so much a 14MB model can do. Needle excels at in-conext inference, with tightly defined environments. In our experience: accurate descriptions + narrow tool scope = success