6 ms·

I think this is under appreciated. I run a "talk-to-your-files" website with 5ish K MRR and a pretty generous free tier. My OpenAI costs have not exceeded $200

I think this is under appreciated. I run a "talk-to-your-files" website with 5ish K MRR and a pretty generous free tier. My OpenAI costs have not exceeded $200 / mo. People talk about using smaller, cheaper models but unless you have strong data security requirements you're burdening yourself with serious maintenance work and using objectively worse models to save pennies. This doesn't even consider OpenAI continuously lowering their prices. I've talked to a good amount of businesses and 90% of custom use cases would also have negligible AI costs. In my opinion, unless you're in a super regulated industry or doing genuinely cutting edge stuff, you should probably just be using the best that's available (OpenAI).

vsreekanti 3y ago
I completely agree — open-source models and custom deployments just can't compete with the cost and efficiency here. The only exception here is if open-source models can get way smaller and faster than they are now while maintaining existing quality. That will make private deployments and custom fine-tuning way more likely.
- SkyMarshal 3y ago
  Or FOSS models remain the same size and speed, but hardware for running them, especially locally, steadily improves till the AI is "good enough" for a large enough segment of the market.
hobs 3y ago
How do you deal with the fact that Azure et al are not appearing to sell anyone additional capacity?
jejeyyy77 3y ago
how do ur customers feel about you uploading potentially confidential documents to a 3rd party?
- CDSlice 3y ago
  If they are confidential they probably shouldn’t be uploaded to any website no matter if it calls out to OpenAI or does all the processing on their own servers.
- yunohn 3y ago
  It’s simple really, lots of businesses share data with 3rd parties to enable various services. OpenAI provides a service contract claiming they do not mine/reshare/etc the data shared via their API. As the SaaS provider, you just need to call it out your user service agreement.
- yieldcrv 3y ago
  most people don't care even when its required by their industry, and privacy isn’t a selling point on its own
- pnt12 3y ago
  Ifnyoure a big company dealing with Microsoft , you're not just going to "pay as you go". You'll have a sales rep, dedicated support, custom contracts and special prices. In the contract, Microsoft just needs to promise they won't share your data ever: if they do, the company can sue. Microsoft is a behemoth at dealing with enterprises and has been doing it for decades. Even old school enterprises are OK with uploading data to Azure.
euazOn 3y ago
Just curious, could you briefly mention some of the custom use cases with negligible AI costs? Thanks
cyode 3y ago
Are any OpenAI powered flows available to public, logged-out user traffic? I’ve worried (maybe irrationally) about doing this in a personal project and then dealing with malicious actors and getting stuck with a big bill.
- wrsh07 3y ago
  You can set hard limits on the api when you configure billing, so you know your monthly bill will never be more than $X
Bukhmanizer 3y ago
The bleeding obvious is that OpenAI is doing what most tech companies for the last 20 years have done. Offer the product for dirt cheap to kill off competition, then extract as much value from your users as possible by either mining data or hiking the price. I don’t understand how people are surprised by this anymore. So yeah, it’s the best option right now, when the company is burning through cash, but they’re planning on getting that money back from you eventually.
- jaredklewis 3y ago
  > Offer the product for dirt cheap to kill off competition, then extract as much value from your users as possible by either mining data or hiking the price. Genuine question, what are some examples of companies in that "hiking the price" camp? I can think of tons of tech companies that sold or sell stuff at a loss for growth, but struggling to find examples where the companies then are able to turn dominant market share into higher prices. To be clear, I'm definitely not implying they are not out there, just looking for examples.
  - loganfrederick 3y ago
    Uber, Netflix and the online content streaming services. These are probably the most prominent examples from this recent 2010s era.
  - spacebanana7 3y ago
    The Google Maps API price hike of 2018 [1] is a relevant example. [1] https://kobedigital.com/google-maps-api-changes https://kobedigital.com/google-maps-api-changes
  - beezlebroxxxxxx 3y ago
    Uber is probably the biggest pure example. When I was in uni when they first spread, Uber's entire business model was flood the market with hilariously low prices and steep discounts. People overnight started using them like crazy. They were practically giving away their product. Now, they're as expensive, if not sometimes more expensive, than any other taxi or ridesharing service in my area. One thing I'll add is that it's not always that this ends with higher prices in an absolute sense, but that the tech company is able to essentially cut the knees out of their competitors until they're a shell of their former selves. Then when the prices go "up", they're in a way a return to the "norm", only they have a larger and dominant market share because of their crazy pricing in the early stages.
goosinmouse 3y ago
Are you using 3.5 turbo? Its always funny when i test a new fun chatbot or something and see my API usage 10x just from a single GPT 4 API call. Although i only usually have a $2 bill every month from openAI.
littlestymaar 3y ago
> you should probably just be using the best that's available (OpenAI). Sure, if you want to let a monopoly have all the added value while you get to keep the rest you can do that. Just make sure you're never successful enough to inspire them though, otherwise you're dead the next minute. Oops.
- wrsh07 3y ago
  5k monthly revenue (or even 100k) isn't big enough for OpenAI to care. It's barely big enough for a competitor to care. I suspect that OpenAI doesn't have the bandwidth to build most uses of ai and so is in the bill gates platform land: the ecosystem should be pocketing more money than the owner of the platform is.
  - littlestymaar 3y ago
    Now
    - deleted 3y ago
      [deleted]
    - wrsh07 3y ago
      Was this an accident? If it's a complete thought intending to say "for now" - eh, I would bet on it for quite a while. They don't care about your little use case. It's possible gpt next or next++ ends up making whatever work you do trivial, but it's likely you'll still have customers
quickthrower2 3y ago
Not only that there is Azure Services for OpenAI which is a more corporate offering with regions, Azure AD integration etc.

vsreekanti 3y ago

I completely agree — open-source models and custom deployments just can't compete with the cost and efficiency here. The only exception here is if open-source models can get way smaller and faster than they are now while maintaining existing quality. That will make private deployments and custom fine-tuning way more likely.

SkyMarshal 3y ago
Or FOSS models remain the same size and speed, but hardware for running them, especially locally, steadily improves till the AI is "good enough" for a large enough segment of the market.

hobs 3y ago

How do you deal with the fact that Azure et al are not appearing to sell anyone additional capacity?

jejeyyy77 3y ago

how do ur customers feel about you uploading potentially confidential documents to a 3rd party?

CDSlice 3y ago
If they are confidential they probably shouldn’t be uploaded to any website no matter if it calls out to OpenAI or does all the processing on their own servers.
yunohn 3y ago
It’s simple really, lots of businesses share data with 3rd parties to enable various services. OpenAI provides a service contract claiming they do not mine/reshare/etc the data shared via their API. As the SaaS provider, you just need to call it out your user service agreement.
yieldcrv 3y ago
most people don't care even when its required by their industry, and privacy isn’t a selling point on its own
pnt12 3y ago
Ifnyoure a big company dealing with Microsoft , you're not just going to "pay as you go". You'll have a sales rep, dedicated support, custom contracts and special prices. In the contract, Microsoft just needs to promise they won't share your data ever: if they do, the company can sue. Microsoft is a behemoth at dealing with enterprises and has been doing it for decades. Even old school enterprises are OK with uploading data to Azure.

euazOn 3y ago

Just curious, could you briefly mention some of the custom use cases with negligible AI costs? Thanks

cyode 3y ago

Are any OpenAI powered flows available to public, logged-out user traffic? I’ve worried (maybe irrationally) about doing this in a personal project and then dealing with malicious actors and getting stuck with a big bill.

wrsh07 3y ago
You can set hard limits on the api when you configure billing, so you know your monthly bill will never be more than $X

Bukhmanizer 3y ago

The bleeding obvious is that OpenAI is doing what most tech companies for the last 20 years have done. Offer the product for dirt cheap to kill off competition, then extract as much value from your users as possible by either mining data or hiking the price. I don’t understand how people are surprised by this anymore. So yeah, it’s the best option right now, when the company is burning through cash, but they’re planning on getting that money back from you eventually.

jaredklewis 3y ago
> Offer the product for dirt cheap to kill off competition, then extract as much value from your users as possible by either mining data or hiking the price. Genuine question, what are some examples of companies in that "hiking the price" camp? I can think of tons of tech companies that sold or sell stuff at a loss for growth, but struggling to find examples where the companies then are able to turn dominant market share into higher prices. To be clear, I'm definitely not implying they are not out there, just looking for examples.
- loganfrederick 3y ago
  Uber, Netflix and the online content streaming services. These are probably the most prominent examples from this recent 2010s era.
- spacebanana7 3y ago
  The Google Maps API price hike of 2018 [1] is a relevant example. [1] https://kobedigital.com/google-maps-api-changes https://kobedigital.com/google-maps-api-changes
- beezlebroxxxxxx 3y ago
  Uber is probably the biggest pure example. When I was in uni when they first spread, Uber's entire business model was flood the market with hilariously low prices and steep discounts. People overnight started using them like crazy. They were practically giving away their product. Now, they're as expensive, if not sometimes more expensive, than any other taxi or ridesharing service in my area. One thing I'll add is that it's not always that this ends with higher prices in an absolute sense, but that the tech company is able to essentially cut the knees out of their competitors until they're a shell of their former selves. Then when the prices go "up", they're in a way a return to the "norm", only they have a larger and dominant market share because of their crazy pricing in the early stages.

goosinmouse 3y ago

Are you using 3.5 turbo? Its always funny when i test a new fun chatbot or something and see my API usage 10x just from a single GPT 4 API call. Although i only usually have a $2 bill every month from openAI.

littlestymaar 3y ago

> you should probably just be using the best that's available (OpenAI). Sure, if you want to let a monopoly have all the added value while you get to keep the rest you can do that. Just make sure you're never successful enough to inspire them though, otherwise you're dead the next minute. Oops.

wrsh07 3y ago
5k monthly revenue (or even 100k) isn't big enough for OpenAI to care. It's barely big enough for a competitor to care. I suspect that OpenAI doesn't have the bandwidth to build most uses of ai and so is in the bill gates platform land: the ecosystem should be pocketing more money than the owner of the platform is.
- littlestymaar 3y ago
  Now
  - deleted 3y ago
    [deleted]
  - wrsh07 3y ago
    Was this an accident? If it's a complete thought intending to say "for now" - eh, I would bet on it for quite a while. They don't care about your little use case. It's possible gpt next or next++ ends up making whatever work you do trivial, but it's likely you'll still have customers

quickthrower2 3y ago

Not only that there is Azure Services for OpenAI which is a more corporate offering with regions, Azure AD integration etc.