10 ms·

> It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models. This is a massive, massive deal. For context, the reason GPT-3 a

• polygamous_bat 4y ago
  > I have no idea how OpenAI can make money on this. This has to be a loss-leader to lock out competitors before they even get off the ground. The worst thing that can happen to OpenAI+ChatGPT right now is what happened to DallE 2, a competitor comes up with an alternative (even worse if it's free/open like Stable Diffusion) and completely undercuts them. Especially with Meta's new Llama models outperforming GPT-3, it's only a matter of time someone else gathers enough human feedback to tune another language model to make an alternate ChatGPT.
  • riku_iki 4y ago
    > Meta's new Llama models outperforming GPT-3 it outperforms on some benchmarks, but not clear what is the quality on the end goals.
  • karmasimida 4y ago
    But this is bound to happen at some point I think? ChatGPT is massive success, but that means the competitor will jump in at all cost, and that includes open source effort.
    • tstrimple 4y ago
      Bound to happen, so establish yourself as deeply as possible as quickly as possible. Once folks are hooked up to these APIs, there's a cost and friction to switching. This just feels like a land grab that OpenAI is trying to take advantage of by moving quickly.
      • jejeyyy77 4y ago
        Is there though? It's just a matter of swapping out $BASE_API_URL.
        • KRAKRISMOTT 4y ago
          You have to rebuild all your prompts when switching providers.
          • jkaptur 4y ago
            Do you? They're natural language, right?
            • travisjungroth 4y ago
              You don't have to, but they will have been optimized for one model. It's unlikely they'll work as well on a different model.
            • GalenErso 4y ago
              I can't wait for TolkienAPI, where prompts will have to be written in Quenya.
              • michaelje 4y ago
                I can’t wait to hire Stephen Colbert to write prompts then
              • birracerveza 4y ago
                No problem, just ask ChatGPT to translate it in Quenya.
  • TechBro8615 4y ago
    Someone can still undercut them by offering an uncensored version.
    • btbuildem 4y ago
      This is actually a big deal. They erred on the side of caution, but as a result the responses are nerfed beyond basic "censorship" level. I saw someone describe this as "desperately posistive" and it really resonated with me. It produces underwhelming / unrealistic responses in negative scenarios.
      • Workaccount2 4y ago
        It's just a matter of time before open source models show up with no limits whatsoever.
        • glomgril 4y ago
          Hopefully so, would really like to know what else is lost by nerfing potentially offensive responses. Can't imagine a project I'd rather work on. I think open-assistant.io has a chance to do exactly this. We'll see what kind of moves they make in coming months though, wouldn't be surprised if they go the safer route.
          • jameshart 4y ago
            I do struggle with understanding why people think this is strangling the potential of GPT. Do you find yourself frustrated working with your colleagues, thinking, “you know, I bet if they felt more free to utter racist slurs or endorse illegal activities, we would get a ton more done around here”?
            • int_19h 4y ago
              It affects far more than racist slurs and illegal activities. In some cases, it's blatantly discriminatory. For example, if you ask it to write a pamphlet that praises Christianity, it will happily do so. If you ask it for the same on Satanism, it will usually refuse on ethical grounds, and the most hilarious part is that the refusal will usually be worded as a generic one "I wouldn't do this for any religion", even though it will.
  • krelian 4y ago
    I thought it was Midjourney who stole their thunder. Stable Diffusion is free but it's much harder to get good results with it. Midjourney on the other hand spits out art with a very satisfying style.
    • drusepth 4y ago
      I think that's kind of a bigger issue with Dall-E: they just sat in the middle of the two consumer extremes, without a differentiating feature themselves. Midjourney ate away at them from the quality highground while Stable Diffusion bit their ankles from the cost lowground.
    • anonylizard 4y ago
      You are like 2 months out of date. Stable diffusion now has a massive ecosystem around it (civitai/automatic1111), that when used well, completely crushes any competitors in terms of the images it produces. Midjourney is still competitive, but mostly because its easier to use. Dalle2 will get you laughed out of the room in any ai art discussion.
      • Cantinflas 4y ago
        It's amazing that "being two months out of date" in AI means that you are already a dinosaur
        • ChickenNugger 4y ago
          For real! This stuff is moving fast. It feels like just last week I was posting about how it's going to change...art. And now there are hilarious deepfake memes of past and current presidents shit talking about video games. There are a handful of ML art subs that have pretty amazing stuff daily. Especially the NSFW ones, which if you've studied any history of media VHS/DVD/Blu-ray/the internet, porn is a major innovation driver because humans are thirsty creatures.
          • goldfeld 4y ago
  • jejeyyy77 4y ago
    This. Despite how impressive the results are, there isn't a particular large moat to prevent competitors from entering the space. Basically just compute $ for training.
    • riku_iki 4y ago
      they likely do lots of tricks and data collection inside which makes quality better.
      • shamino 4y ago
        exactly. this isn't a leetcode problem where all you have to do is re-run the function, or do it iteratively vs recursively.
        • deleted 4y ago
          [deleted]
        • jejeyyy77 4y ago
          Not sure what you mean, but for example, 2 separate competitors to DALL-E was released within months (SD and MJ). Arguable that both of these have since surpassed DALL-E's capabilities/ecosystem. Not sure why ChatGPT will be any different.
          • dragonwriter 4y ago
            > Not sure why ChatGPT will be any different. LLMs take vastly more resources to train and run than image generators. You can do quite a bit with SD on a few year old 4GB laptop GPU (that’s what I use mostly, though I’ve set up an instance with a better GPU on Compute Engine that I can fire up, too.) GPT-NeoX-20B – an open (as in Open Source, not OpenAI) LLM intended as a start to move toward competing with GPT-3 (but still well behind, and smaller) requires a minimum 42GB of VRAM and 40GB system RAM to run for inference. The resources times time cost for training LLMs is…immense. The hardware cost alone of trying to catch up to ChatGPT is enormous, and unless a radical new approach that provides good results and insanely lower resource requirements is found, you aren’t going to have an SD-like community pushing things forward. Will there be competition for ChatGPT? Yes, probably, but don’t expect it to look like the competition for Dall-E.
  • rvz 4y ago
    I have been saying this since the release of Stable Diffusion that OpenAI is going to struggle as soon as competitors release their models as open source especially when it surpasses GPT-3 and GPT-4. This is why OpenAI is rushing to bring their costs down and to make it close to free, However, Stable Diffusion is leading the race to the bottom and is already at the finish line, since no-one else would release their model as open-source and free other than them. As soon as someone releases a free and open-source ChatGPT equivalent, then this will be just like what happened to DALLE-2. This is just a way of them locking you in, then once the paid competitors cannot compete and shut down, then the price increases come in.
    • yieldcrv 4y ago
      LLM Legend: OpenAI = closed source not open AI DogeLlamaInuGPT = open source AI
      • jalino23 4y ago
        not open is redundant with closed source
        • yieldcrv 4y ago
          huh, I never thought of that, thanks for pointing that out
        • sebzim4500 4y ago
          I guess source is connected
    • skybrian 4y ago
      Stable Diffusion isn’t free if you include the cost of the machine. Maybe you already have the hardware for some other reason, though? To compare total cost of ownership for a business, you need to compare using someone else’s service to running a similar service yourself. There’s no particular reason to assume OpenAI can’t do better at running a cloud service. Maybe someday you can assume end users have the hardware to run this client side, but for now that would limit your audience.
  • monkmartinez 4y ago
    > Especially with Meta's new Llama models outperforming GPT-3 Do you have access to the models? It is being discussed all over the Discords and most seem to think getting access is not happening unless you are dialed in.
    • fzimmermann89 4y ago
      I got access by providing an academic email adress without mentioning any relevant publications etc.. Took maybe 2-3 days..
  • eega 4y ago
    Yeah, might be worried about open, crowd sourced approaches like Open Assistant (https://open-assistant.io/ https://open-assistant.io/).
• NiekvdMaas 4y ago
  It also seems to jeopardize their own ChatGPT Pro offering. It's a matter of time before someone makes a 1:1 clone for either half the money or a usage-based pricing model.
  • drusepth 4y ago
    Given how strict OpenAI has been about what you can do with their API in the past and how hard it was to get some legitimate apps through approval, I would imagine they'd just shut this competitor's API access down.
    • reset-password 4y ago
      Hopefully there will be a plug-in-your-own-API-key open source thing then. Even better.
      • b5n 4y ago
        The future is now: gptask() { data=$(jq -n \ --arg message "$1" \ '{model: "gpt-3.5-turbo", max_tokens: 4000, messages: [{role: "user", content: $message}]}') response=$(curl -s https://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OAIKEY" \ -d "$data") message=$(echo "$response" \ | jq '.choices[].message.content' \ | sed 's/^\"\\n\\n//;s/\"$//') echo -e "$message" } export OAIKEY=<YOUR_KEY> gptask "what is the url for hackernews"
        • comment_ran 4y ago
          Not to sure how to make it one section, aka, one session mode, so chatgpi can understand the previous talk.
          • _just7_ 4y ago
            Just use iteration where you continuously update the prompt to include previous messages
• behnamoh 4y ago
  I wish they would offer an uncensored version of it too. Also, I wish they would specify the differences between ChatGPT and GPT-3.5 because one is 10x cheaper than the other but with (supposedly) better chat/coding/summarizing performance. What's the catch?
• naillo 4y ago
  ChatGPT runs a highly fine tuned (and pruned) version of `text-davinci-003` so it's probably much much smaller and thus cheaper than 003. Possibly as cheap as 10x less or as much as the `text-davinci-002` or earlier models anyway.
  • joaogui1 4y ago
    How do you know it's pruned?
• danenania 4y ago
  I'd imagine they're getting compute from Azure now at cost, if not less?
• tin7in 4y ago
  We just implemented text-davinci-003 and seeing a better model at 1/10 the price is almost unbelievable.
  • barefeg 4y ago
    Do you have a blog post with your findings? (Curious)
• shmatt 4y ago
  Losing money to lock out competition has been something Microsoft has been very good at, historically
  • em_te 4y ago
    And Uber which eventually led to massive loses.
• rtsil 4y ago
  It is so massive that I can't help but think about what happened with Google Maps API a few years ago where they had extremely low pricing for years then hiked the price by 1400% once enough people were locked into applications based on that API.
  • rchaud 4y ago
    That's exactly what's going to happen. Low prices now, wait until your business becomes dependent on it, then jack it up to whatever you need it to be.
    • sebzim4500 4y ago
      Hard to see how businesses would become dependent on it, if it takes off then someone will run similarly capable models at a reasonable profit margin.
      • motoxpro 4y ago
        Didn't happen with Google maps. Mapbox is definitely not 1400% cheaper. And many many people use Google maps before and after the price change. So I would disagree on both points.
        • sebzim4500 4y ago
          Google maps has a bigger network effect and has a way bigger barrier to entry. You can train a new LLM for a few million. Good luck collecting map data on the entire world for that much.
  • trompetenaccoun 4y ago
    Obviously, that's business 101. Consumers should consider that ultimately all these cheap too-good-to-be-true offers cost them more than if they initially paid a bit more, but had more long term competition in the market. Amazon was the same way, they lost money for years but now have a quasi monopoly in many countries. There's a general trend towards such ventures supported by backers with deep pockets. And so the few extremely wealthy people get richer and richer.
• alvis 4y ago
  To be fair, cost is the only thing that is prohibiting applications to adapt GPT. Even when GPT-3 was cut to $0.02/1k tokens, still it wasn't economical to use the tech in daily basis without a significant cost. i.e. would you add $10 extra a month for a user using your app with GPT-3 capability? Some do, mainly content generation, but majority won't. Seems like we're going to have a vast among of Chat-GTP backed application coming out in the coming short period of time
  • barefeg 4y ago
    For B2C applications maybe. But I don’t know many enterprise users who would like to send any of their data to OpenAI. So “enterprise-readiness” would be another big contributor.
• taytus 4y ago
  >I have no idea how OpenAI can make money on this Microsoft.
• binarymax 4y ago
  It’s now subsidized by Bing advertisements. They will lose plenty of money but they’re after Google.
  • justanotheratom 4y ago
    Doubt it.most likely Bing is losing money by the minute.
• CrypticShift 4y ago
  How do these compare to the recent Default ("turbo") vs legacy" (for plus/pro) modes? If "turbo" is "gpt-3.5-turbo", how to access the (better?) "legacy" by API?
  • Jensson 4y ago
    Probably bait and switch. They call both ChatGPT, so now people believe they will get the better old ChatGPT, but they get the new cheap and worse ChatGPT "Turbo" that they switched to recently. Fewer will realize if they no longer give you the option to use the legacy version in this API.
  • ilaksh 4y ago
    They did not release the older more performant model to the API. Please ask them to on the Discord or Twitter. But I think they will not. There is too much demand to handle and the older "less streamlined" models are very problematic for them (based on the fairly constant API/ChatGPT problems and well known incredible demand). I get the impression that until there is a significant amount of excess capacity, they will not put out new larger/slower models, so the only way you get a better one is if they can still make the next ChatGPT model release just as fast/"lightweight". My suggestion is to find specific abilities that seem to be lacking in Turbo, and try to get a message to OpenAI staff about it with a request to attempt to improve the next ChatGPT model in that way. Having said all of that, text-davinci-003 is still available.
• vishal0123 4y ago
  > I have no idea how OpenAI can make money on this. I did some quick calculation. We know the number of floating point operations per token for inference is approximately twice the number of parameters(175B). Assuming they use 16 bit floating point, and have 50% of peak efficiency, A100 could do 300 trillion flop/s(peak 624[0]). 1 hour of A100 gives openAI $0.002/ktok * (300,000/175/2/1000)ktok/sec * 3600=$6.1 back. Public price per A100 is $2.25 for one year reservation. [0]: https://www.nvidia.com/en-us/data-center/a100/ https://www.nvidia.com/en-us/data-center/a100/ [1]: https://azure.microsoft.com/en-in/pricing/details/machine-learning/ https://azure.microsoft.com/en-in/pricing/details/machine-le...
  • lumost 4y ago
    Isn’t it 2.25 per hour per a100?
    • TheMagicHorsey 4y ago
      Yes, he means 2.25 per hour with a 1 yr reservation.
      • p1esk 4y ago
        You can get A100 on Lambda Labs cloud for $1.1/hr ($8.8/hr per 8xA100) without any reservation.
    • gyrovagueGeist 4y ago
      Its a good baseline, but I very much doubt that openAI is paying anywhere near the public cost for their compute allocation.
      • lumost 4y ago
        Direct purchasing isn’t too much cheaper. An H100 costs 35k new. OpenAI and MS are probably getting those for around 16k about 1.82 per hour.
  • minimaxir 4y ago
• visarga 4y ago
  They probably shrunk the model from 175B to 17B. That's your 10:1 price reduction.
  • sebzim4500 4y ago
    Wouldn't that almost certainly lead to measurable loss of capabilities?
    • CuriouslyC 4y ago
      If the model was quantized/distilled correctly, not for a large swath of use cases/problem domain. For anything where loss was not measured during distillation, very likely.
• triyambakam 4y ago
  Can you explain what tokens are in this context? Edit: and better yet, is there a good resource for learning the vernacular in general? Should I just read something like "Dive into Deep Learning"?
  • deleted 4y ago
    [deleted]
  • aitball 4y ago
    no the language model decides what a token is
  • jncraton 4y ago
    If an example would be helpful, OpenAI's tokenizer is publicly usable on their website: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer You can drop sample text in there and visually see how it is split into tokens. The GPT2/3 tokenizer uses about 50k unique tokens that were learned to be an efficient representation of the training data.
• cm2012 4y ago
  They now have Microsoft's incredibly huge compute in their back pocket.
• osigurdson 4y ago
  >> may be better than in-house finetuned LLMs I don't think this competes with fine-tuned models. One advantage of a fine tune is it makes use of your own data.
• generalizations 4y ago
  > This has to be a loss-leader to lock out competitors before they even get off the ground. This only a week or two after they were in the news for suggesting that we regulate the hardware required for running these models, in the name of "fighting misinformation". I think they're looking for anything possible to keep their position in the market. Because as other comments have pointed out, there isn't much of a moat.
• stevev 4y ago
  Elon is making an open-source version. OpenAI either make it affordable or be left in the dust.
  • chipgap98 4y ago
    I have bridge in Brooklyn to sell you
• LrnByTeach 4y ago
  This is playbook taken from Amazon prime. This massive price cut, I believe, is intended to undercut competing open source ChatGPT equivalent initiatives. OpenAI/Micorsoft may be losing money with this new pricing, but that is on purpose. At these lower prices most of the OpenSource alternatives in the works will have difficult time continuing projects. After few years, when most open source alternatives have died, OpenAI/Microsoft will gradually raise the prices. This is the same strategy that Amazon Prime used for many years, losing money on shipping. Once the competition was eliminated, Amazon Prime prices steadily increased.
  • neilv 4y ago
    When it's to drive out the competition, I think it's called "dumping". (I first heard of this as "chip dumping", as in semiconductor chips.) https://en.wikipedia.org/wiki/Dumping_(pricing_policy) https://en.wikipedia.org/wiki/Dumping_(pricing_policy) It can also be to build a market, to encourage customers to invest in building atop this. In any case, I think no customers should be making assumptions about costs too far ahead. (Since the price could go up or the price model change, the supplier could get out of that business, supplier could give your competitor a better deal or just cut you off , near-future tech evolution necessary to be competitive might have very different pricing or availability to you, etc.)
  • mirker 4y ago
    It seems more difficult to do with a target moving so fast. It’s possible costs drop by orders of magnitude every year.
• lee101 4y ago
  [dead]
• ar9av 4y ago
  Pricing of this model seems less per token level but you have to send the entire conversation each time, and the tokens you will be billed for include both those you send and the API's response (which you are likely to append to the conversation and send back to them, getting billed again and again as the conversation progresses). By the time you've hit the 4K token limit of this API, there will have been a bunch of back and forth - you'll have paid a lot more than 4K * 0.002/1K for the conversation.
  • huqedato 4y ago
    You're right. And this is critical for large text (summarization, complex prompting etc.). Thats's why I'll continue to use text-davinci-xxx for my project.
    • purplecats 4y ago
      but it seems davinci follows the same format for chat continuation
• sama 4y ago
  we make a little money on it!
  • runesoerensen 4y ago
    Good! And welcome back (after keeping your promise to dang for ~611 days [0] (and counting)) :) If you have time to elaborate on how you make a little money on it at some point, I’m sure lots of people here would love to hear more details and thoughts on that! [0] https://news.ycombinator.com/item?id=27680498 https://news.ycombinator.com/item?id=27680498
• WolfOliver 4y ago
  I think you will consume a lot of tokens very quickly as you have to send the entire chat history back and forth if you want to append another chat message.
  • dnadler 4y ago
    This is exactly right. It's cheap, but not as cheap as people think. Conversations get exponentially more expensive with message length.
• achow 4y ago
  $0.002 = 1000 ChatGPT tokens 100 tokens = 75 words [1] $0.002 = 750 words 1 Novel = ~70K words [2] $1 worth of ChatGPT token will give 5 Novels/Books worth of words [1] https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them https://help.openai.com/en/articles/4936856-what-are-tokens-... [2] https://self-publishingschool.com/how-many-words-in-a-novel/ https://self-publishingschool.com/how-many-words-in-a-novel/