4 ms·
Running models locally brings my beefy rig to the knees for about half a minute for each querry for smaller models. Answering querries has to be expensive too?
by rightbyte 2y ago
Running models locally brings my beefy rig to the knees for about half a minute for each querry for smaller models. Answering querries has to be expensive too?