4 ms·
totally. to get it out the door I kept it in the cloud but that would be a good optimization. Honestly the biggest issue I've been facing is google's ability t
by tommywilczek 8mo ago
totally. to get it out the door I kept it in the cloud but that would be a good optimization.
Honestly the biggest issue I've been facing is google's ability to host its 3 flash/3.1 flash-lite models. They CONSTANTLY return 503: model unavailable due to high usage... Next step will be to play with a more reliable LLM that's still fast and smart enough.