8 ms·
My use case is IRC bots. If you just have the bot responding to only a single line and not knowing any of the chat history, yeah, it can be fairly cheap. But on
by superkuh 4y ago
My use case is IRC bots. If you just have the bot responding to only a single line and not knowing any of the chat history, yeah, it can be fairly cheap. But once you try to start giving it short term memory by feeding in the prior ~handful of lines you blow through that $18 of free credit in a couple weeks. Something that costs $25/mo is not cheap for a human person.
I am not happy with your implication that gpt3.5-turbo only doesn't respond to "nazi" stuff and that my users are such people. But I guess getting Godwin'd online isn't new. It literally won't even respond to innocuous questions.
- Spivak 4y agoWhat kinda volume are you pushing though because I also do that and even have it ingest whole pdfs/word docs as conversation context and I get charged like $3/mo on average. Edit: I’m literally agreeing with you and describing innocuous questions that it doesn’t respond to. I’m saying that if all it refused to do was write hate and erotica it would be fine and I would use it but the filter catches things like code.
- superkuh 4y agoWith gpt3 davinci we were doing about ~30 requests for text completion per hour (at peak activity) each having ~11 lines of chat history (including up to 190 token responses from davinci) which added up to about ~1000 to 5000 tokens each. So 30*3000 at $0.0200/1000 tokens equals a few dollars per day.