3 ms·
I only run small models (70b at my hardware gets me around 10-20 TOPS) for just random things (personal assistant kind of thing) but not for coding tasks. For
by SamDc73 10mo ago
I only run small models (70b at my hardware gets me around 10-20 TOPS) for just random things (personal assistant kind of thing) but not for coding tasks.
For coding related tasks I consume 30-80M tokens per day and I want something as fast as it gets