2 ms·
I was hoping I could use this to reduce the cost of using the openai API. But since it sends both the custom prompt telling gpt4 to compress the intent and the
by superkuh 4y ago
I was hoping I could use this to reduce the cost of using the openai API. But since it sends both the custom prompt telling gpt4 to compress the intent and the full original prompt this does not reduce the tokens used. It increases it by the number of tokens in the custom prompt saying to compress things. And then if you want to use the output you have to pay again for the full tokens of the compressed form.
I guess this might have uses and it's neat. But it won't be for saving money.
- babyshake 4y agoMy impression is this is more useful for making efficient use of the context window and not as relevant for decreasing costs. It would decrease costs if you want to reuse the compressed information many times, similar to how the cost savings of a compressed file would be apparent when reusing the compressed file.
- weird-eye-issue 4y agoYeah except this doesn't actually work reliably.