3 ms·
Oh nvm, you mean non inference calls. ...But still, these are not very common API calls? Generally an OpenAI endpoint is mostly inference calls, right? And lla
by brucethemoose2 3y ago
Oh nvm, you mean non inference calls.
...But still, these are not very common API calls? Generally an OpenAI endpoint is mostly inference calls, right? And llama.cpp's slowness is going to blow that advantage away.