4 ms·
How many tokens a second? Really trying to figure out viability. 4x NVIDIA A100 at lamda labs is $4.40 an hour and I really have not had an issue getting them.
by ttt3ts 3y ago
How many tokens a second? Really trying to figure out viability.
4x NVIDIA A100 at lamda labs is $4.40 an hour and I really have not had an issue getting them.
- hnfong 3y agoNote this is a M1, not M2. https://www.reddit.com/r/LocalLLaMA/comments/16o4ka8/running_ggufs_on_an_m1_ultra_is_an_interesting/ https://www.reddit.com/r/LocalLLaMA/comments/16o4ka8/running...
- ttt3ts 3y agoThanks! Ya, I opted for dual 3090 for my workstation (keeping full LLM in VRAM is crit) was wondering what lift was for M2. OP implied that there were workloads where it out competes renting in terms of cost. Was hoping it was true for something than a single user interactive session (which can be done a lot cheaper)