4 ms·
Question for dudes building modern nn's... what's the thinking on estimating structural capacity for real world problem? How should I estimate how many paramete
by sgt101 1y ago
Question for dudes building modern nn's... what's the thinking on estimating structural capacity for real world problem? How should I estimate how many parameters to choose for the model?
- p1esk 1y agoYou test different models on your real world problem, and pick the smallest one that works.
- sgt101 1y agoI just think that there has to be some heuristic..
- BoorishBears 1y agoClosest thing to a heuristic is trying the task with non fine-tuned models and building an intuition for how far off each model is, what directions it's off in, and how easily you can improve that direction via fine-tuning. For example, for classification, if is hallucinating semantically similar, but not technically valid classes, you can probably fine-tune your way out of the gap with a smaller model. But if your task requires world knowledge, you likely need a larger model. It's not cheap, efficient, or generally useful to fine-tune for additional world knowledge directly.