3 ms·
I've been having an interesting challenge relating to this recently. I'm trying to calculate costs for LLM usage, but the amounts of money involved are so tiny.
by simonw 1y ago
I've been having an interesting challenge relating to this recently. I'm trying to calculate costs for LLM usage, but the amounts of money involved are so tiny. Gemini 1.5 Flash 8B is $0.0375 per million tokens!
Should I be running my accounting system on units of 10 billionths of a dollar?
- outurnate 1y agoYou're better off representing values as rationals; a ratio between two different numbers. For example, 0.0375 would be represented as 375 over 10000, or 3 over 80
- simonw 1y agoSounds hard to model in SQLite.
- deleted 1y ago[deleted]
- teaearlgraycold 1y agoTwo columns?
- anthk 1y agoFrom Forth, here's how I'd set the rationals: : gcd begin dup while tuck mod repeat drop ; : lcm 2dup \* abs -rot gcd / ; : reduce 2dup gcd tuck / >r / r> ; : q+ rot 2dup \* >r rot \* -rot \* + r> reduce ; : q- swap negate swap q+ ; : q\* rot \* >r \* r> reduce ; : q/ >r \* swap r> \* swap reduce ; Example: to compute 70 * 0.25 = 35/2 70 1 1 4 q* reduce .s 35 2 ok On stack managing words like 2dup, rot and such, these are easily grasped under either Google/DDG or any Forth with the words "see" and/or "help". as a hint, q- swaps the top two numbers in the stack, (which compose a rational), makes the last one negative and then turns back its position. And then it calls q+. So, 2/5 - 3/2 = 2/5 + -3/2.
- marcosdumay 1y agoAccounting happens on the unities people pay, not the ones that generate expenses. But you probably should run your billing in fixed point or floating decimals with a billionth of a dollar precision, yes. Either that or you should consolidate the expenses into larger bunches.
- scott_w 1y agoFixed point Decimal is your friend here. I’m guessing you buy tokens in increments of 1,000,000 so it isn’t too much of an issue to account for. You can then normalise in your accounting so 1,000,000 is just “1 unit,” or you can just account in increments of 1,000,000 but that does start looking weird (but might be necessary!)
- latchkey 1y agoEthereum is 1e18 or 1 wei. https://ethereum.stackexchange.com/questions/158517/does-solidity-interpret-1e18-as-1-000-000-000-000-000-000-or-as-1-00000000000000 https://ethereum.stackexchange.com/questions/158517/does-sol...
- kolbe 1y agoI've used Auroa Units to do this. You can define the dollars dimension, and then all the nano-micro-whatever scale comes with.
- klysm 1y agoConvert to money as late as possible
- roryirvine 1y agoThis is surely the right answer: simply count the number of tokens used, and do the billing reconciliation as a separate step. As an added benefit, it makes it much easier to deal with price changes.