2 ms·

I have not pushed my local commits to GitHub lately (and probably should), but my experience with the Gemini API so far has been relatively positive: https://g

by ryao 1y ago

I have not pushed my local commits to GitHub lately (and probably should), but my experience with the Gemini API so far has been relatively positive: https://github.com/ryao/gemini-chat https://github.com/ryao/gemini-chat The main thing I do not like is that token counting is rated limited. My local offline copies have stripped out the token counting since I found that the service becomes unusable if you get anywhere near the token limits, so there is no point in trimming the history to make it fit. Another thing I found is that I prefer to use the REST API directly rather than their Python wrapper. Also, that comment about 500 errors is obsolete. I will fix it when I do new pushes.

yorick 1y ago
It looks like you can use the gemma tokenizer to count tokens up to at least the 1.5 models. The docs claim that there's a local compute_tokens function in google-genai, but it looks like it just does an API call. Example for 1.5: https://github.com/googleapis/python-aiplatform/blob/main/vertexai/tokenization/_tokenizer_loading.py https://github.com/googleapis/python-aiplatform/blob/main/ve...