4 ms·
Having some idea of the task will guide the choice of model, which will be an enormous factor in memory use (I.e. whether it will startup or not) Do you need a
by malux85 2y ago
Having some idea of the task will guide the choice of model, which will be an enormous factor in memory use (I.e. whether it will startup or not)
Do you need a 70b param model or a 7b model? Theres thousands and thousands of dollars hardware difference there
With no idea of the task, one can’t even ball park it
- tbrownaw 2y agoThis particular tool has a page listing recommended models: https://neuml.github.io/txtai/models/ https://neuml.github.io/txtai/models/