3 ms·
Even the README of this project is lacking in details. They cover sizes of system RAM, but not of VRAM. Are they running the models on CPU exclusively? How does
by HexDecOctBin 3y ago
Even the README of this project is lacking in details. They cover sizes of system RAM, but not of VRAM. Are they running the models on CPU exclusively? How does that compare to running them on GPUs? What are the corresponding numbers for GPUs?
Not to mention more meta-topics, like which is preferred for inferencing - CPU or GPU? What are the corresponding numbers for fine-tuning or training for various model sizes? and so on.
- deleted 3y ago[deleted]