3 ms·

I didn't really distinguish between those token types, since I didn't want to complicate the calculation too much. But yes, you are right, there is a large diff

I didn't really distinguish between those token types, since I didn't want to complicate the calculation too much. But yes, you are right, there is a large difference between input and output tokens.

jjmarr 1y ago
Input management is the biggest difference between monthly plans like Copilot and pay-per-use like Cline. Monthly plans hide the input from the user and try to minimize it while pay-per-use plans visibly use the full context window. Mentally modelling the pricing as being determined by output doesn't match reality in my experience. It's also fundamentally different economics since input is gated by VRAM capacity while output is gated by compute.