3 ms·
There's a difference between "how many CPU-hours will my task need" and "how much memory does this program use to even start up".
by tbrownaw 2y ago
There's a difference between "how many CPU-hours will my task need" and "how much memory does this program use to even start up".
- deleted 2y ago[deleted]
- malux85 2y agoHaving some idea of the task will guide the choice of model, which will be an enormous factor in memory use (I.e. whether it will startup or not) Do you need a 70b param model or a 7b model? Theres thousands and thousands of dollars hardware difference there With no idea of the task, one can’t even ball park it
- tbrownaw 2y agoThis particular tool has a page listing recommended models: https://neuml.github.io/txtai/models/ https://neuml.github.io/txtai/models/