3 ms·
I assumed it was coming straight from the API because of the token-by-token generation effect but maybe you're right.
by iknownthing 2y ago
I assumed it was coming straight from the API because of the token-by-token generation effect but maybe you're right.
- rodiger 2y agoThe token-by-token responses are probably API, while the "instant" loads seem to be cached. Probably using groq based on speed of response