2 ms·
I am really interested in a LLM router that can track the usage and tokens used to measure the cost and put int rate limits. What this will do is help keep cost
by mrbishalsaha 2y ago
I am really interested in a LLM router that can track the usage and tokens used to measure the cost and put int rate limits. What this will do is help keep cost down even while using multiple LLMs.
I think this project will be a great starting point.
- vinhnx 2y agoThank you for your compliment! I still learning and experimenting with LLM and gen AI for now Looking at LiteLLM docs, which is quite extensive I think for tracking usages and tokens management is a great use case, which I think also have covered in their docs. Cheers, Vinh