5 ms·

> $150/Mtok input, $600/Mtok output What use case could possibly justify this price?

by bakugo 2y ago

icelancer 2y ago
Full file refactoring. But I just use the webUI for this and will continue to at these prices... probably.
serjester 2y ago
Synthetic data generation. You can have a really powerful, expensive model create evals so you can tune a faster, cheaper system with similar performance.
- jsheard 2y ago
  You could do that, but OpenAI specifically doesn't want you to: https://openai.com/policies/row-terms-of-use/ https://openai.com/policies/row-terms-of-use/ What you cannot do. You may not use our Services for any illegal, harmful, or abusive activity. For example, you may not: Use Output to develop models that compete with OpenAI. Presumably you run the risk of getting banned if they realize what you're doing.
  - serjester 2y ago
    Synthetic data is just as useful for building app layers evals. Probably significantly cheaper ways to get the data if you’re training your own model.
  - echelon 2y ago
    Screw their TOS. OpenAI trained on the world's data. Data they didn't license. Anyone should be able to "rip them off" and copy their capabilities on the cheap.
    - jsheard 2y ago
      The irony isn't lost on me, but irony isn't going to stop them kicking you off their platform if they feel like it.
  - littlestymaar 2y ago
    I suspect they have no way to enforce that without risking false positive hurting their rich customers (and their business).
  - levocardia 2y ago
    I wonder if some of the high pricing is specifically an attempt to ward off this sort of "slow distillation" of a powerful model
  - andyferris 2y ago
    > You may not use our Services for any illegal, harmful, or abusive activity. For example, you may not: Use Output to develop models that compete with OpenAI. This reads as if they consider developing models that compete with OpenAI as illegal, harmful or abusive. Which is crazy. (The other dot points in their list in the linked terms seem better).
  - kelseyfrog 2y ago
    I compete with AI, not my models.
  - SJC_Hacker 2y ago
    If it was possible: 1) Why wasn't OpenAI doing it themselves? 2) This means we've reached technological singularity if AI models can improve themselves (as in getting a smarter model, not just compressing existing ones like Deepseek)
    - sgillen 2y ago
      Compressing existing models is exactly what we are talking about here.
      - SJC_Hacker 2y ago
        Then why wasn't OpenAI doing that themselves ?
        nickthegreek 2y ago
        Who says they arent?
    - throwup238 2y ago
      It’s not a singularity because the synthetic data generated by the previous frontier model isn’t usually fed directly into the training for the next frontier model - a “discriminator” is applied to select only the highest quality responses. That discriminator could be a field expert or mechanical turk or another model trained to select for higher quality responses (i.e. trained on a dataset of books rather than internet content). As far as I know, OpenAI has been doing this, using both experts and Kenyan workers as well as their own discriminator models. Unfiltered synthetic data is generally used more for distilled models and fine tunes for a specific use case.
refulgentis 2y ago
It enables obscene unnatural things at a fraction of most SWE hourly rates. One win that jumps to mind was writing a complete implementation of a Windows PCM player, as a flutter plugin, with some unique design properties and emergent API behavior that it needed to replicate from existing iOS/Android code
- zipy124 2y ago
  Does it really? Your average software engineer is like £20-30 an hour, for the cost of 1m output tokens you can get a Dev for a full week.
  - sheepscreek 2y ago
    The math doesn’t check out. A day maybe. Also it’s not just about a placeholder dev. The person needs to know your use-case and have the tech chops to deliver successfully in that timeframe. Now to have that delivered to you in less than an hour? That’s a huge win.
    - zipy124 2y ago
      No, even contractors who are more expensive per day can be had for £200-300 per diem. Employee costs you are looking at closer to £165 including national insurance and pension on a standard £35k a year salary. Even an employee on £50k a year is only £241.97 a day. IR35 rules largely closed down a lot of the high-value contracting business and pushed day rates down heavily.
      - refulgentis 2y ago
        Sure. So why are you saying we'd get a dev for a week for $600? :) Even we take that at face value, now we're claiming devs are $30K/year. (This was sort of baseline full-time pay when I was in high school, 20 years ago) Why do I need 1,000,000 tokens for 500 loc? I don't think further futzing with the numbers makes this work, it's off by multiple OOMs while barely credible

> $150/Mtok input, $600/Mtok output What use case could possibly justify this price?

by bakugo 2y ago

> $150/Mtok input, $600/Mtok output What use case could possibly justify this price?

icelancer 2y ago

Full file refactoring. But I just use the webUI for this and will continue to at these prices... probably.

serjester 2y ago

Synthetic data generation. You can have a really powerful, expensive model create evals so you can tune a faster, cheaper system with similar performance.

jsheard 2y ago
You could do that, but OpenAI specifically doesn't want you to: https://openai.com/policies/row-terms-of-use/ https://openai.com/policies/row-terms-of-use/ What you cannot do. You may not use our Services for any illegal, harmful, or abusive activity. For example, you may not: Use Output to develop models that compete with OpenAI. Presumably you run the risk of getting banned if they realize what you're doing.
- serjester 2y ago
  Synthetic data is just as useful for building app layers evals. Probably significantly cheaper ways to get the data if you’re training your own model.
- echelon 2y ago
  Screw their TOS. OpenAI trained on the world's data. Data they didn't license. Anyone should be able to "rip them off" and copy their capabilities on the cheap.
  - jsheard 2y ago
    The irony isn't lost on me, but irony isn't going to stop them kicking you off their platform if they feel like it.
- littlestymaar 2y ago
  I suspect they have no way to enforce that without risking false positive hurting their rich customers (and their business).
- levocardia 2y ago
  I wonder if some of the high pricing is specifically an attempt to ward off this sort of "slow distillation" of a powerful model
- andyferris 2y ago
  > You may not use our Services for any illegal, harmful, or abusive activity. For example, you may not: Use Output to develop models that compete with OpenAI. This reads as if they consider developing models that compete with OpenAI as illegal, harmful or abusive. Which is crazy. (The other dot points in their list in the linked terms seem better).
- kelseyfrog 2y ago
  I compete with AI, not my models.
- SJC_Hacker 2y ago
  If it was possible: 1) Why wasn't OpenAI doing it themselves? 2) This means we've reached technological singularity if AI models can improve themselves (as in getting a smarter model, not just compressing existing ones like Deepseek)
  - sgillen 2y ago
    Compressing existing models is exactly what we are talking about here.
    - SJC_Hacker 2y ago
      Then why wasn't OpenAI doing that themselves ?
      - nickthegreek 2y ago
        Who says they arent?
  - throwup238 2y ago
    It’s not a singularity because the synthetic data generated by the previous frontier model isn’t usually fed directly into the training for the next frontier model - a “discriminator” is applied to select only the highest quality responses. That discriminator could be a field expert or mechanical turk or another model trained to select for higher quality responses (i.e. trained on a dataset of books rather than internet content). As far as I know, OpenAI has been doing this, using both experts and Kenyan workers as well as their own discriminator models. Unfiltered synthetic data is generally used more for distilled models and fine tunes for a specific use case.

refulgentis 2y ago

It enables obscene unnatural things at a fraction of most SWE hourly rates. One win that jumps to mind was writing a complete implementation of a Windows PCM player, as a flutter plugin, with some unique design properties and emergent API behavior that it needed to replicate from existing iOS/Android code

zipy124 2y ago
Does it really? Your average software engineer is like £20-30 an hour, for the cost of 1m output tokens you can get a Dev for a full week.
- sheepscreek 2y ago
  The math doesn’t check out. A day maybe. Also it’s not just about a placeholder dev. The person needs to know your use-case and have the tech chops to deliver successfully in that timeframe. Now to have that delivered to you in less than an hour? That’s a huge win.
  - zipy124 2y ago
    No, even contractors who are more expensive per day can be had for £200-300 per diem. Employee costs you are looking at closer to £165 including national insurance and pension on a standard £35k a year salary. Even an employee on £50k a year is only £241.97 a day. IR35 rules largely closed down a lot of the high-value contracting business and pushed day rates down heavily.
    - refulgentis 2y ago
      Sure. So why are you saying we'd get a dev for a week for $600? :) Even we take that at face value, now we're claiming devs are $30K/year. (This was sort of baseline full-time pay when I was in high school, 20 years ago) Why do I need 1,000,000 tokens for 500 loc? I don't think further futzing with the numbers makes this work, it's off by multiple OOMs while barely credible