31 ms·

Introducing ChatGPT and Whisper APIs

• tartakovsky 4y ago
  Ok so can someone provide 10 steps to launching your own voice assistant?
  • pjot 4y ago
    Try asking chatGPT
• minimaxir 4y ago
  > It is priced at $0.002 per 1k tokens, which is 10x cheaper than our existing GPT-3.5 models. This is a massive, massive deal. For context, the reason GPT-3 apps took off over the past few months before ChatGPT went viral is because a) text-davinci-003 was released and was a significant performance increase and b) the cost was cut from $0.06/1k tokens to $0.02/1k tokens, which made consumer applications feasible without a large upfront cost. A much better model and a 1/10th cost warps the economics completely to the point that it may be better than in-house finetuned LLMs. I have no idea how OpenAI can make money on this. This has to be a loss-leader to lock out competitors before they even get off the ground.
  • polygamous_bat 4y ago
    > I have no idea how OpenAI can make money on this. This has to be a loss-leader to lock out competitors before they even get off the ground. The worst thing that can happen to OpenAI+ChatGPT right now is what happened to DallE 2, a competitor comes up with an alternative (even worse if it's free/open like Stable Diffusion) and completely undercuts them. Especially with Meta's new Llama models outperforming GPT-3, it's only a matter of time someone else gathers enough human feedback to tune another language model to make an alternate ChatGPT.
    • riku_iki 4y ago
      > Meta's new Llama models outperforming GPT-3 it outperforms on some benchmarks, but not clear what is the quality on the end goals.
    • karmasimida 4y ago
      But this is bound to happen at some point I think? ChatGPT is massive success, but that means the competitor will jump in at all cost, and that includes open source effort.
      • tstrimple 4y ago
        Bound to happen, so establish yourself as deeply as possible as quickly as possible. Once folks are hooked up to these APIs, there's a cost and friction to switching. This just feels like a land grab that OpenAI is trying to take advantage of by moving quickly.
        • jejeyyy77 4y ago
          Is there though? It's just a matter of swapping out $BASE_API_URL.
          • KRAKRISMOTT 4y ago
            You have to rebuild all your prompts when switching providers.
            • jkaptur 4y ago
              Do you? They're natural language, right?
              • travisjungroth 4y ago
                You don't have to, but they will have been optimized for one model. It's unlikely they'll work as well on a different model.
  • NiekvdMaas 4y ago
    It also seems to jeopardize their own ChatGPT Pro offering. It's a matter of time before someone makes a 1:1 clone for either half the money or a usage-based pricing model.
    • drusepth 4y ago
      Given how strict OpenAI has been about what you can do with their API in the past and how hard it was to get some legitimate apps through approval, I would imagine they'd just shut this competitor's API access down.
      • reset-password 4y ago
        Hopefully there will be a plug-in-your-own-API-key open source thing then. Even better.
        • b5n 4y ago
          The future is now: gptask() { data=$(jq -n \ --arg message "$1" \ '{model: "gpt-3.5-turbo", max_tokens: 4000, messages: [{role: "user", content: $message}]}') response=$(curl -s https://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OAIKEY" \ -d "$data") message=$(echo "$response" \ | jq '.choices[].message.content' \ | sed 's/^\"\\n\\n//;s/\"$//') echo -e "$message" } export OAIKEY=<YOUR_KEY> gptask "what is the url for hackernews"
          • comment_ran 4y ago
            Not to sure how to make it one section, aka, one session mode, so chatgpi can understand the previous talk.
            • _just7_ 4y ago
              Just use iteration where you continuously update the prompt to include previous messages
  • behnamoh 4y ago
    I wish they would offer an uncensored version of it too. Also, I wish they would specify the differences between ChatGPT and GPT-3.5 because one is 10x cheaper than the other but with (supposedly) better chat/coding/summarizing performance. What's the catch?
  • naillo 4y ago
    ChatGPT runs a highly fine tuned (and pruned) version of `text-davinci-003` so it's probably much much smaller and thus cheaper than 003. Possibly as cheap as 10x less or as much as the `text-davinci-002` or earlier models anyway.
    • joaogui1 4y ago
      How do you know it's pruned?
  • danenania 4y ago
    I'd imagine they're getting compute from Azure now at cost, if not less?
  • tin7in 4y ago
    We just implemented text-davinci-003 and seeing a better model at 1/10 the price is almost unbelievable.
    • barefeg 4y ago
      Do you have a blog post with your findings? (Curious)
  • shmatt 4y ago
    Losing money to lock out competition has been something Microsoft has been very good at, historically
    • em_te 4y ago
      And Uber which eventually led to massive loses.
  • rtsil 4y ago
    It is so massive that I can't help but think about what happened with Google Maps API a few years ago where they had extremely low pricing for years then hiked the price by 1400% once enough people were locked into applications based on that API.
    • rchaud 4y ago
      That's exactly what's going to happen. Low prices now, wait until your business becomes dependent on it, then jack it up to whatever you need it to be.
      • sebzim4500 4y ago
        Hard to see how businesses would become dependent on it, if it takes off then someone will run similarly capable models at a reasonable profit margin.
        • motoxpro 4y ago
          Didn't happen with Google maps. Mapbox is definitely not 1400% cheaper. And many many people use Google maps before and after the price change. So I would disagree on both points.
          • sebzim4500 4y ago
            Google maps has a bigger network effect and has a way bigger barrier to entry. You can train a new LLM for a few million. Good luck collecting map data on the entire world for that much.
    • trompetenaccoun 4y ago
      Obviously, that's business 101. Consumers should consider that ultimately all these cheap too-good-to-be-true offers cost them more than if they initially paid a bit more, but had more long term competition in the market. Amazon was the same way, they lost money for years but now have a quasi monopoly in many countries. There's a general trend towards such ventures supported by backers with deep pockets. And so the few extremely wealthy people get richer and richer.
  • alvis 4y ago
    To be fair, cost is the only thing that is prohibiting applications to adapt GPT. Even when GPT-3 was cut to $0.02/1k tokens, still it wasn't economical to use the tech in daily basis without a significant cost. i.e. would you add $10 extra a month for a user using your app with GPT-3 capability? Some do, mainly content generation, but majority won't. Seems like we're going to have a vast among of Chat-GTP backed application coming out in the coming short period of time
    • barefeg 4y ago
      For B2C applications maybe. But I don’t know many enterprise users who would like to send any of their data to OpenAI. So “enterprise-readiness” would be another big contributor.
  • taytus 4y ago
    >I have no idea how OpenAI can make money on this Microsoft.
  • binarymax 4y ago
    It’s now subsidized by Bing advertisements. They will lose plenty of money but they’re after Google.
    • justanotheratom 4y ago
      Doubt it.most likely Bing is losing money by the minute.
  • CrypticShift 4y ago
    How do these compare to the recent Default ("turbo") vs legacy" (for plus/pro) modes? If "turbo" is "gpt-3.5-turbo", how to access the (better?) "legacy" by API?
    • Jensson 4y ago
      Probably bait and switch. They call both ChatGPT, so now people believe they will get the better old ChatGPT, but they get the new cheap and worse ChatGPT "Turbo" that they switched to recently. Fewer will realize if they no longer give you the option to use the legacy version in this API.
    • ilaksh 4y ago
      They did not release the older more performant model to the API. Please ask them to on the Discord or Twitter. But I think they will not. There is too much demand to handle and the older "less streamlined" models are very problematic for them (based on the fairly constant API/ChatGPT problems and well known incredible demand). I get the impression that until there is a significant amount of excess capacity, they will not put out new larger/slower models, so the only way you get a better one is if they can still make the next ChatGPT model release just as fast/"lightweight". My suggestion is to find specific abilities that seem to be lacking in Turbo, and try to get a message to OpenAI staff about it with a request to attempt to improve the next ChatGPT model in that way. Having said all of that, text-davinci-003 is still available.
  • vishal0123 4y ago
    > I have no idea how OpenAI can make money on this. I did some quick calculation. We know the number of floating point operations per token for inference is approximately twice the number of parameters(175B). Assuming they use 16 bit floating point, and have 50% of peak efficiency, A100 could do 300 trillion flop/s(peak 624[0]). 1 hour of A100 gives openAI $0.002/ktok * (300,000/175/2/1000)ktok/sec * 3600=$6.1 back. Public price per A100 is $2.25 for one year reservation. [0]: https://www.nvidia.com/en-us/data-center/a100/ https://www.nvidia.com/en-us/data-center/a100/ [1]: https://azure.microsoft.com/en-in/pricing/details/machine-learning/ https://azure.microsoft.com/en-in/pricing/details/machine-le...
    • lumost 4y ago
      Isn’t it 2.25 per hour per a100?
      • TheMagicHorsey 4y ago
        Yes, he means 2.25 per hour with a 1 yr reservation.
        • p1esk 4y ago
          You can get A100 on Lambda Labs cloud for $1.1/hr ($8.8/hr per 8xA100) without any reservation.
      • gyrovagueGeist 4y ago
        Its a good baseline, but I very much doubt that openAI is paying anywhere near the public cost for their compute allocation.
        • lumost 4y ago
          Direct purchasing isn’t too much cheaper. An H100 costs 35k new. OpenAI and MS are probably getting those for around 16k about 1.82 per hour.
    • minimaxir 4y ago
  • visarga 4y ago
    They probably shrunk the model from 175B to 17B. That's your 10:1 price reduction.
    • sebzim4500 4y ago
      Wouldn't that almost certainly lead to measurable loss of capabilities?
      • CuriouslyC 4y ago
        If the model was quantized/distilled correctly, not for a large swath of use cases/problem domain. For anything where loss was not measured during distillation, very likely.
  • triyambakam 4y ago
    Can you explain what tokens are in this context? Edit: and better yet, is there a good resource for learning the vernacular in general? Should I just read something like "Dive into Deep Learning"?
    • deleted 4y ago
      [deleted]
    • aitball 4y ago
      no the language model decides what a token is
    • jncraton 4y ago
      If an example would be helpful, OpenAI's tokenizer is publicly usable on their website: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer You can drop sample text in there and visually see how it is split into tokens. The GPT2/3 tokenizer uses about 50k unique tokens that were learned to be an efficient representation of the training data.
  • cm2012 4y ago
    They now have Microsoft's incredibly huge compute in their back pocket.
  • osigurdson 4y ago
    >> may be better than in-house finetuned LLMs I don't think this competes with fine-tuned models. One advantage of a fine tune is it makes use of your own data.
  • generalizations 4y ago
    > This has to be a loss-leader to lock out competitors before they even get off the ground. This only a week or two after they were in the news for suggesting that we regulate the hardware required for running these models, in the name of "fighting misinformation". I think they're looking for anything possible to keep their position in the market. Because as other comments have pointed out, there isn't much of a moat.
  • stevev 4y ago
    Elon is making an open-source version. OpenAI either make it affordable or be left in the dust.
    • chipgap98 4y ago
      I have bridge in Brooklyn to sell you
  • LrnByTeach 4y ago
    This is playbook taken from Amazon prime. This massive price cut, I believe, is intended to undercut competing open source ChatGPT equivalent initiatives. OpenAI/Micorsoft may be losing money with this new pricing, but that is on purpose. At these lower prices most of the OpenSource alternatives in the works will have difficult time continuing projects. After few years, when most open source alternatives have died, OpenAI/Microsoft will gradually raise the prices. This is the same strategy that Amazon Prime used for many years, losing money on shipping. Once the competition was eliminated, Amazon Prime prices steadily increased.
    • neilv 4y ago
      When it's to drive out the competition, I think it's called "dumping". (I first heard of this as "chip dumping", as in semiconductor chips.) https://en.wikipedia.org/wiki/Dumping_(pricing_policy) https://en.wikipedia.org/wiki/Dumping_(pricing_policy) It can also be to build a market, to encourage customers to invest in building atop this. In any case, I think no customers should be making assumptions about costs too far ahead. (Since the price could go up or the price model change, the supplier could get out of that business, supplier could give your competitor a better deal or just cut you off , near-future tech evolution necessary to be competitive might have very different pricing or availability to you, etc.)
    • mirker 4y ago
      It seems more difficult to do with a target moving so fast. It’s possible costs drop by orders of magnitude every year.
  • lee101 4y ago
    [dead]
  • ar9av 4y ago
    Pricing of this model seems less per token level but you have to send the entire conversation each time, and the tokens you will be billed for include both those you send and the API's response (which you are likely to append to the conversation and send back to them, getting billed again and again as the conversation progresses). By the time you've hit the 4K token limit of this API, there will have been a bunch of back and forth - you'll have paid a lot more than 4K * 0.002/1K for the conversation.
    • huqedato 4y ago
      You're right. And this is critical for large text (summarization, complex prompting etc.). Thats's why I'll continue to use text-davinci-xxx for my project.
      • purplecats 4y ago
        but it seems davinci follows the same format for chat continuation
  • sama 4y ago
    we make a little money on it!
    • runesoerensen 4y ago
      Good! And welcome back (after keeping your promise to dang for ~611 days [0] (and counting)) :) If you have time to elaborate on how you make a little money on it at some point, I’m sure lots of people here would love to hear more details and thoughts on that! [0] https://news.ycombinator.com/item?id=27680498 https://news.ycombinator.com/item?id=27680498
  • WolfOliver 4y ago
    I think you will consume a lot of tokens very quickly as you have to send the entire chat history back and forth if you want to append another chat message.
    • dnadler 4y ago
      This is exactly right. It's cheap, but not as cheap as people think. Conversations get exponentially more expensive with message length.
  • achow 4y ago
    $0.002 = 1000 ChatGPT tokens 100 tokens = 75 words [1] $0.002 = 750 words 1 Novel = ~70K words [2] $1 worth of ChatGPT token will give 5 Novels/Books worth of words [1] https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them https://help.openai.com/en/articles/4936856-what-are-tokens-... [2] https://self-publishingschool.com/how-many-words-in-a-novel/ https://self-publishingschool.com/how-many-words-in-a-novel/
• fallingmeat 4y ago
  this is so much cheaper than fine-tuned model; it would make sense to try and use the multi-shot prompt in the 3.5turbo model. plus, there's the hundreds/thousands of training items that you wouldn't need to create....fml
  • minimaxir 4y ago
    The ChatGPT API is cheaper than a fine-tuned babbage model.
• tablet 4y ago
  I am happy that we designed our UI for Fibery AI Assistant ready for chat, less re-work! Looks like a super decent release and price cut makes it sane to use. Token limit is is same, and this is not great for many use cases...
• deleted 4y ago
  [deleted]
• rd 4y ago
  This is incredibly cheap, it makes you wonder how in the world they managed to make it 10x cheaper than davinci-003 and still a better model? The world of robo-consulting is about to go insane.
  • dmw_ng 4y ago
    It may just be that OpenAI corrected its 2020-era understanding of model size using 2022 insights from DeepMind.. https://towardsdatascience.com/a-new-ai-trend-chinchilla-70b-greatly-outperforms-gpt-3-175b-and-gopher-280b-408b9b4510 https://towardsdatascience.com/a-new-ai-trend-chinchilla-70b... Seems model sizing, compression and quantization are still an art form, see also https://www.unum.cloud/blog/2023-02-20-efficient-multimodality https://www.unum.cloud/blog/2023-02-20-efficient-multimodali...
    • aidaman 4y ago
      yes turbo model admitted this when asked.
• alexb_ 4y ago
  >Through a series of system-wide optimizations, we’ve achieved 90% cost reduction for ChatGPT since December This is seriously impressive. A MILLION tokens for 2 dollars is absolutely fucking insane. I hope that the gains reached here can also be found by open source and non-controlled AI projects. If so, that could be huge for the advancement of AI.
  • machinekob 4y ago
    If you are Microsoft as GigaScaler with almost unlimited cash and can ignore getting profit of your api/models its pretty easy to undercut all the other companies and offer it very cheap just to gain advantage in the future.
    • alexb_ 4y ago
      What the cost cutting measures suggest is that AI like this could maybe soon be run on consumer hardware. That combined with actually open source language models could be huge. OpenAI won't allow for that for obvious reasons, but this confirms that the optimizations are there, and that's exciting enough news on its own.
      • alfalfasprout 4y ago
        I mean, meta's new LAMA model runs on a single A100 in the 13B parameter variant (which performs similar to GPT3 65B).
        • visarga 4y ago
          "Performs" on paper until they give a demo.
  • sva_ 4y ago
    A lot of people assumed GPT4 would be an even bigger model, but I've been thinking it'll probably be more about more efficient compute. This is at least some evidence that they're working on that.
• r3trohack3r 4y ago
  Whisper as an API is great, but having to send the whole payload upfront is a bummer. Most use cases I can build for would want streaming support. Like establish a WebRTC connection and stream audio to OpenAI and get back a live transcription until the audio channel closes.
  • banana_giraffe 4y ago
    It's also annoying since there appears to be a hard limit of 25 MiB to the request size, requiring you to split up larger files and manage the "prompt" to subsequent calls. Well, somehow, near as I can tell, how you're expected to use that value isn't documented.
    • travisjungroth 4y ago
      You split up the audio and send it over in a loop. Pass in the transcript of the last call as the prompt for the next one. See item 2 here: https://platform.openai.com/docs/guides/speech-to-text/prompting https://platform.openai.com/docs/guides/speech-to-text/promp...
      • banana_giraffe 4y ago
        And: > we suggest that you avoid breaking the audio up mid-sentence as this may cause some context to be lost. That's really easy to put in a document, much harder to do in practice. Granted, it might not matter much in the real world, not sure yet. Still, this will require more hand holding than I'd like.
        • travisjungroth 4y ago
          I doubt it will matter if you're breaking up mid sentence if you pass in the previous as a prompt and split words. This is how Whisper does it internally. It's not absolutely perfect, but splitting on the word boundary is one line of code with the same package in their docs: https://github.com/jiaaro/pydub/blob/master/API.markdown#silencesplit_on_silence https://github.com/jiaaro/pydub/blob/master/API.markdown#sil... 25MB is also a lot. That's 30 minutes to an hour on MP3 at reasonable compression. A 2 hour movie would have three splits.
• rkwasny 4y ago
  Pricing is good because OpenAI does not need to make any money but needs data for feedback, if everyone switches to open source ( Llama etc. ) they won't get the data they need. Google is testing their system internally with XX thousand users, OpenAI with XXX million users ...
  • alpark3 4y ago
    > Data submitted through the API is no longer used for service improvements (including model training) unless the organization opts in I don't think the pricing is largely driven by intention to scrape API requests for data.
  • agnokapathetic 4y ago
    > Starting today, OpenAI says that it won’t use any data submitted through its API for “service improvements,” including AI model training, unless a customer or organization opts in. In addition, the company is implementing a 30-day data retention policy for API users with options for stricter retention “depending on user needs,” and simplifying its terms and data ownership to make it clear that users own the input and output of the models. https://techcrunch.com/2023/03/01/addressing-criticism-openai-will-no-longer-use-customer-data-to-train-its-models-by-default/ https://techcrunch.com/2023/03/01/addressing-criticism-opena...
  • PartiallyTyped 4y ago
    Something that has been bothering me for a while is whether poisoning of OpenAI's dataset is possible, willingly or otherwise. An example here is getting chatGPT to accept that that 2+2=5, it's a lot of effort, but can be done. Then the users can give thumbs up when such responses are given. Could this cause issues?
  • wunderland 4y ago
    I think they are actually selling this service at a price point that is profitable.
    • TeMPOraL 4y ago
• gigel82 4y ago
  You can run Whisper in WASM (locally) so no need to pay for the API, plus the bandwidth. It actually works surprisingly well: https://github.com/ggerganov/whisper.cpp https://github.com/ggerganov/whisper.cpp
  • mijoharas 4y ago
    This looks like C++ rather than WASM. Am I misunderstanding something?
    • iib 4y ago
      The C++ is compiled to WASM. You can look into [1] to see emscripten there. [1] https://github.com/ggerganov/whisper.cpp/blob/master/CMakeLists.txt https://github.com/ggerganov/whisper.cpp/blob/master/CMakeLi...
      • mijoharas 4y ago
        So it is! Thanks for the pointer.
    • moyix 4y ago
      Maybe they meant to link to something like this: https://github.com/ggerganov/whisper.cpp/pull/540 https://github.com/ggerganov/whisper.cpp/pull/540 Web demo: https://whisper.ggerganov.com/ https://whisper.ggerganov.com/
  • qwertox 4y ago
    whisper.cpp has no GPU support. Models below medium aren't that good, and medium and large are pretty CPU intensive. A minute of audio on medium can take anything between 15 and 90 seconds to transcribe, when using 8 cores, while the service transcribes on the large model in less than 7 seconds.
    • graderjs 4y ago
      For English only: This is so wrong! I actually didn't find that much significant improvement between Medium to Large. In many cases Medium was actually better for English than large was. Where Large really excels in my experience is with super noisy or distorted audio. Same thing with Small to Medium. There's no much difference in text quality. For good quality audio, Small is all you need. Medium is not much more accurate, and certainly the "value per unit time" is way higher with Small than with Medium. If you have very distorted audio you can try medium or large. I agree that the API providing a super fast large is fantastic tho! But you can go far with the provided models. When did you last try Whisper.cpp? It's constantly updated and probably much better than it was a few months ago. If you don't believe me (and you have a Mac) try out my free App that uses Whisper.cpp offline combined with Voice Activity Detection (Silero VAD) to reduce Whisper-hallucinating-words during silent / non-speech sections. It's really good! https://apps.apple.com/app/wisprnote/id1671480366 https://apps.apple.com/app/wisprnote/id1671480366
• deleted 4y ago
  [deleted]
• r3trohack3r 4y ago
  One of the things I love about the API-ification of these LLMs is that they’re plug and play. I built https://persona.ink https://persona.ink against davinci knowing it didn’t give as good of results as ChatGPT but knowing I could swap the model out once 3.5 came out. Today is that day, going to swap out the prompt in the cloudflare worker and it should Just Work(tm)
  • giarc 4y ago
    Famous last words?
  • cfiggers 4y ago
    Typo on your front page—"enfrocement" should be "enforcement."
  • sebzim4500 4y ago
    Is davinci actually worse than chatGPT? I know it's worse as an assistant, but in my (admittedly brief) testing the performance was the same or better for tasks like summarization, sentiment analysis, etc. I guess it's irrelevant now because everyone will use the one which is 10x cheaper.
    • r3trohack3r 4y ago
      Yes. Asking ChatGPT to assume a persona and rewrite content, it performs significantly better than davinci in my tests. I was able to get "good enough" with a lot of prompt engineering on davinci - but ChatGPT is a shoulder above with less investment in prompt engineering. On the flip side - I get hand swatted by ChatGPT more frequently than davinci. Davinci's moderation filters don't really pick up on much, but ChatGPT will give me a lecture instead of a translation on a lot of occasions. There are many valid use cases for rewriting/editing content that involve graphic details that davinci will gladly handle and ChatGPT will give you a lecture about. Human existence is messy. ChatGPT doesn't like the messy.
• kumarm 4y ago
  ChatGPT API examples are missing (what to use instead of completions?) and also missing in playground. Hope they will add them soon.
  • minimaxir 4y ago
    The documentation was just updated: https://platform.openai.com/docs/guides/chat https://platform.openai.com/docs/guides/chat
    • kumarm 4y ago
      Thank you.
  • ajhai 4y ago
    We have added them to our playground at https://trypromptly.com https://trypromptly.com if you want to check them out. https://twitter.com/ajhai/status/1631020290502463489 https://twitter.com/ajhai/status/1631020290502463489 has a quick demo
• LinuxBender 4y ago
  Does ChatGPT yet have a debug function to Show Its Work so to speak? I think this will be important in the future when it gets itself into drama, trouble, etc.. Probably also useful to prove how ChatGPT created something rather than being known as an opaque box.
  • warent 4y ago
    I'm pretty sure any system built via linear regression or similar is an opaque box even to the most experienced researchers. For example: https://clementneo.com/posts/2023/02/11/we-found-an-neuron https://clementneo.com/posts/2023/02/11/we-found-an-neuron These are massive functions with billions of parameters that evolved over millions of computing years.
    • LinuxBender 4y ago
      Adding to that, the human brain is incredibly complex and performs billions of functions. If a person says to me, "I love you" I should be able to ask them why they said that but it would probably be unfair to expect a detailed answer including all their environmental and genetic inputs, many of which they may not be aware of. If ChatGPT says it loves me, I not only expect the system to tell me why that was said but what steps brought the system to that conclusion. It is a computer or network of computers after all. Even if the system is continuously learning there should be some facets of reproducible steps that can be enumerated. ChatGPT: "I love you" Me: "debug last transaction, Hal." Here is where I would expect an enumeration of all steps used to reach said conclusion. These steps may evolve/devolve over time as the system ingests new data but it should be possible to have it Think out loud so to speak. Maybe the output is large so ChatGPT should give me a link to a .tar file compressed with whatever it knows is my preferred compression. [Edit] I accept that this may be hundreds of billions of calculations. I will wait the few minutes it takes to generate a tar file for me. It's good to get up and stretch the legs once in a while.
      • falcor84 4y ago
        >If a person says to me, "I love you" I should be able to ask them why they said that. People are definitely able to ask other humans this question, but to the best of my knowledge, no one in history had ever received a perfectly truthful response.
• deleted 4y ago
  [deleted]
• alpark3 4y ago
  > Dedicated instances can make economic sense for developers running beyond ~450M tokens per day. 450M tokens * $0.002/1K tokens = $900 per day. I wonder what the exact pricing structure is. (edited for math)
  • ArminRS 4y ago
    > It is priced at $0.002 per 1k tokens So it would be $900 per day
• 0xDEF 4y ago
  What is the developer experience of using OpenAI's Server-Sent Events endpoint from something else than their Python and Node.js libraries? The SSE endpoint is required for use cases like chat so the end user doesn't have to wait until the whole reply has been generated. I started implementing a simple SSE client on top of C#/.Net's HttpClient but it's harder than I first assumed.
  • hyferg 4y ago
    We had to restream server events from openai -> our backend -> client. It was pretty simple.
• nico 4y ago
  Couldn’t find docs or references to the Whisper API. Anyone had a direct link they could share?
  • minimaxir 4y ago
    https://platform.openai.com/docs/guides/speech-to-text https://platform.openai.com/docs/guides/speech-to-text
    • nico 4y ago
      Awesome. Thank you!
• deleted 4y ago
  [deleted]
• noonething 4y ago
  can't you do whisper stuff for free already?
  • minimaxir 4y ago
    Whisper large is a bit tricker to self-host, and the faster inference may be useful for certain applications.
    • graderjs 4y ago
      Agree on that! Whisper large has big needs. But I didn't find the quality for English to be better than Medium. It just took longer. For most cases where audio is good quality, Small is all you need. Not much different to Medium. Only for really distorted (windy, loud background) audio were Medium and Large really good. But all models will fail beyond a certain point of extreme distortion. If you don't believe me or want to know more check out my free app that uses Whisper Small, and (Whisper Tiny for Turbo mode): https://apps.apple.com/app/wisprnote/id1671480366 https://apps.apple.com/app/wisprnote/id1671480366 It uses VAD (voice activity detection) to reduce increased WER during silent or non-speech sections, and it's really fast! Runs locally on M1 just fine.
  • travisjungroth 4y ago
    You can. You're just paying for compute and having it managed. Here's price estimates for 1,000 hours of audio on GCP: https://www.assemblyai.com/blog/how-to-run-openais-whisper-speech-recognition-model/#cost-to-run-whisper https://www.assemblyai.com/blog/how-to-run-openais-whisper-s... For reference, from OpenAI it would be $360 and it's the large-v2 model.
• strudey 4y ago
  Added to the Ruby library here if any Rubyists interested! https://github.com/alexrudall/ruby-openai https://github.com/alexrudall/ruby-openai
  • crancher 4y ago
    Thank you!
  • pschoeps 4y ago
    Dude! I was just thinking of forking your gem to implement these changes myself. You are so fast, thanks.
• revskill 4y ago
  ChatGPT is unreal. It's not artificial intelligence, it's kinda of a supernatural intelligence.
• rburhum 4y ago
  Can somebody please clarify. Is the cost $0.002 per 1k tokens generated, read, or both?
  • minimaxir 4y ago
    Both the prompt and completion fall into the token count (this has been the case since the beginning of GPT-3)
  • dmw_ng 4y ago
    Both, the API response includes a breakdown. In the best case 1 token = 1 word (for example "and", "the", etc). Depending on input, for English it seems reasonable to multiply the word count by about 1.3 to get a rough token count This pricing model seems fair since you can pass in huge prompts and request a single word reply, or a few words that expect a large reply
    • shostack 4y ago
      Is there anything akin to creating Stable Diffusion embeddings where it can train a very discrete concept that takes up a few kilobytes and use that with the base model? Such an approach could in theory make it so you spend a little upfront to train more complex (read: concepts costing many tokens) and can subsequently reuse it cheaply because you're using an embedding of the vectors for that complex concept instead which may only take a single token.
  • hagope 4y ago
    I can confirm it is TOTAL tokens, this from the account/usage page: gpt-3.5-turbo-0301, 2 requests 28 prompt + 64 completion = 92 tokens
• steno132 4y ago
  OpenAI released a ChatGPT API while reducing the cost by 10x. For those claiming OpenAI is for profit: Why would OpenAI do this if they were fixated on making money? Also, while I wish OpenAI released the code for ChatGPT, I applaud OpenAI for actually making their AI model available, to everyone, right now. * Google hyped their Bard chatbot...but where is it? * Facebook took down Galatica. * Even Bing Chat has a waitlist
  • xur17 4y ago
    > For those claiming OpenAI is for profit: Why would OpenAI do this if they were fixated on making money? Reducing costs by 10x may very well increase usage by more than 10x + it makes it even more difficult for competition to come in and undercut them.
  • hellcow 4y ago
    > For those claiming OpenAI is for profit: Why would OpenAI do this if they were fixated on making money? To get lock-in from devs before competitors can enter the market and starves any would-be smaller competitors before they can raise money/gain traction.
    • rvz 4y ago
      This is what 'extinguish' looks like from the new EEE strategy that I have said before, years ago [0] from Microsoft which competitors with paid offerings being unable to compete with free since OpenAI's pricing model is now close to free and their competitors cannot increase their prices. Since Microsoft can foot the bill for the Azure infrastructure, there is going to be little area for anyone to seriously compete against OpenAI on price, API and features, unless it is completely free and open source, like Stability AI. [0] https://news.ycombinator.com/item?id=28324999 https://news.ycombinator.com/item?id=28324999
  • 0xDEF 4y ago
    >For those claiming OpenAI is for profit: Why would OpenAI do this if they were fixated on making money? Silicon Valley companies have for the past 25 years focused on getting as many users as possible to increase valuation in the hope of getting a $100 billion exit. They don't care about current or near future profitability. However I agree that OpenAI is getting far too much hate. Their goal of bringing openness to AI made sense in 2015 when one American company (Google) was dominating the field. However now there are plenty of other companies, countries and open source organizations doing advanced AI research.
• throwaway71271 4y ago
  wow just in time, i just made https://github.com/jackdoe/emacs-chatgpt-jarvis https://github.com/jackdoe/emacs-chatgpt-jarvis which is chatgpt+whisper but using local whisper and chatgpt-wrapper which is a bit clunky since i integrated chatgpt with my emacs i use it at least 20-30 times a day i wonder if they will charge me per token if i am paying the monthly fee
  • graderjs 4y ago
    So cool! I really think this is an awesome future! Everyone has an AI assistant they can interact with via speech.
    • throwaway71271 4y ago
      particularly if you have RSI from 20 years of typing :) with a touchpad that is designed to give you carpal tunnel
• Zetice 4y ago
  I'm a bit confused; what's the difference between this and the Azure OpenAI offering?
  • 0xDEF 4y ago
    Azure will probably provide compliance (HIPAA, GDPR etc.) just like they do with their non-AI offerings.
• sfink 4y ago
  General question: is this the next way we'll manage to destroy the planet? Imagine in the near future that having a slightly better, slightly more up to date LLM is a major competitive advantage. Whether that is between companies or nation-states doesn't really matter. So now all of those recently-idled GPUs will be put to use training and re-training ever bigger and more current models, once again sucking down electricity with no limit. We're not there yet; there are too many ways to improve things without burning a country's worth of electricity re-training. But is it coming?
• evolveyourmind 4y ago
  Goodbye internet as we knew it
• phas0ruk 4y ago
  Feels like people running small websites monetised with ads will get killed. Why go to a recipe website to search for healthy meals for your kids if you can ask chat gpt in Instacart? :(
  • ikmckenz 4y ago
    No more scrolling through a thousand lines of SEO optimization disguised as some deep heartfelt backstory before getting to the actual recipe? That sounds like a huge win to me.
    • squeaky-clean 4y ago
      My love for the perfect Tuna Salad Sandwich began when I was but a young IPython notebook. My lead developer would occasionally eat at the desk and crumbs would fall into the keyboard....
  • Kiro 4y ago
    These small websites have been dead for years and replaced with SEO spam.
    • shostack 4y ago
      Except now Microsoft can hurt Google's ability to monetize it by stealing search traffic with people switching to ChatGPT.
• georgel 4y ago
  I wish Whisper offered speaker diarization. That would be a full game changer for the speech-to-text space.
  • _neil 4y ago
    whisperX has diarization. https://github.com/m-bain/whisperX https://github.com/m-bain/whisperX
  • epoch_100 4y ago
    We hacked that together for https://paxo.ai https://paxo.ai — can be done!
  • graderjs 4y ago
    Why do you need diarization? That's attributing speech to different speakers, right? What sort of use cases?
    • iamflimflam1 4y ago
      Transcribing interviews, meetings etc…
      • graderjs 4y ago
        Did some research and seems there’s no reliable diarization method right now. They all have error rates around like 20%.
• fswd 4y ago
  "model": "gpt-3.5-turbo", turbo isn't listed in Playground, but if you invoke the example curl command (note: /v1/chat/completions) in your terminal, it works.
  • ajhai 4y ago
    We have added it to https://trypromptly.com's https://trypromptly.com's playground. https://twitter.com/ajhai/status/1631020290502463489 https://twitter.com/ajhai/status/1631020290502463489 is a quick demo
• georgehill 4y ago
  Big news! Many apps will be integrating ChatGPT. Worried about AI-generated content flood the search engine, making it harder to do in-depth research.
  • ngokevin 4y ago
    Honestly, I've almost stopped Googling and have had personal success just relying on ChatGPT. It's pretty much taught me 3D animation and Blender to Unity workflows. Every time I wanted to know a Blender keyboard shortcut or what some Blender property was, or how to do something in Blender, the forums and documentation is so sparse and outdated. I felt ChatGPT got me a lot closer much faster. Especially when it tells you how to learn concepts you didn't even know existed. Google results in the meanwhile have just become a pile of SEO-optimized fluff, and it's hard to engineer the search query around that besides sticking "reddit" on the end of it.
  • r3trohack3r 4y ago
    This is a good thing. The future is curation and cultivation. We've been living in an age of information abundance and markets haven't adapted. The age of "crawl every website, index everything, and let people search it" is coming to an end. There is just too much content and too much of it is low quality. With or without AI. This abundance problem isn't just a WWW problem. Movies, TV, music, podcasts, short form content, food, widgets, wibbles and wobbles all suffer from abundance these days. We are quickly exiting the age of supply chain driven scarcity and getting a marketplace flooded with options. Capitalism has delivered on basically everything it's promised with some asterisks and, if we don't give into consumerism, we want for little and have everything we need at our fingertips. I've personally opened up my pocket book to curation services. I know brands that I trust. I know services that reliably surface quality content. I suspect the next few decades are going to trend towards services that separate noise from signal - and I suspect AI is going to be a big part of that.
    • bluefone 4y ago
      Why separate noise from signal? Why isn't AI-generated stuff seen just as valuable as human-written one? When you fill the world with plastic, you need to evolve to eat plastic. When you surround yourself with AI-produced stuff, then you should learn to respect it. First of all, evaluating someone based on what they wrote or said or did, is nonsense.
• coldtea 4y ago
  The quality of suggestions, forums, internet content and other such things just took a huge drop - this will create an internet E.L.E. in SPAM and empty content...
• ehPReth 4y ago
  I can't seem to find it mentioned but is there an unrestricted/uncensored mode for this? I'd love to have some fun with a few friends and hook it up in a matrix room for us
  • mritchie712 4y ago
    https://news.ycombinator.com/item?id=34972791 https://news.ycombinator.com/item?id=34972791
    • CamperBob2 4y ago
      That's just a list of exploits that will be fixed as soon as they come to OpenAI's attention, if they haven't already been fixed. Is anyone actually committed to providing uncensored models as either paid services or open distributions?
      • RugnirViking 4y ago
        Maybe try GPT-J, its an open source model you can run locally. It's pretty massive though iirc
• jonplackett 4y ago
  This feels like the AI’s iPhone moment. I am scared for all people working service jobs.
  • 13years 4y ago
    > I am scared for all people Probably could stop there.
    • jonplackett 4y ago
      Yeah that is actually true. Yeah that’s probably truest. But I’m more scared for some than others short term. I’m less immediately scared for anyone doing work that interacts with the physical world. Weird how it turned out the robotics was harder than the thinking
      • 13years 4y ago
        Yes, short term agreed. AI has flipped on its head much of what we thought early AI would be like with creativity being one of its most successful targets. I suspect the surprises will only continue to be, lets say, more surprising. I've actually written a lot about that recently as well. FYI https://dakara.substack.com/p/ai-and-the-end-to-all-things https://dakara.substack.com/p/ai-and-the-end-to-all-things
        • jonplackett 4y ago
          Will check this out!
  • passion__desire 4y ago
    All jobs will shift to asteroid mining.
    • jeron 4y ago
      what's stopping AI from asteroid mining for itself
• thedangler 4y ago
  Question, Can I give openAI some data for it to process so I can use it to my own advantage. Say I want to train it on specific topic of information I've gathered over the years. Can I some how give it that data and then I can use the API to get back out data in a chat or some other forms of questions? I'm not too familiar with how it works.
  • bicx 4y ago
    You can do this to an extent via fine-tuning, but you will need to do so via one of the other GPT-3 models rather than the ChatGPT API model (`GPT-3.5-turbo`). The latter is not available for fine-tuning.
    • thedangler 4y ago
      Does openAI have models for data to train it with.. data like pricing, locations, products .... Or would I have to use something else? The reason I ask because companies are using it with their own data, like Shopify... sooo it has to be trained somehow.
      • kfarr 4y ago
        The docs on fine tuning are excellent: https://platform.openai.com/docs/guides/fine-tuning https://platform.openai.com/docs/guides/fine-tuning
      • visarga 4y ago
        Yes, you bring your own data, or you generate data with the model, that works too. I have generated training data with GPT3 and trained smaller HuggingFace models on it.
        • thedangler 4y ago
          Any links on how to get started with this?
          • 4y ago
• braingenious 4y ago
  I hope this pricing impacts ChatGPT+ $20 is equivalent to what, 10,000,000 tokens? At ~750 words/1k tokens, that’s 7.5 million words per month, or roughly 250,000 words per day, 10,416 words per hour, 173 words per minute, every minute, 24/7. I uh, do not have that big of a utilization need. It’s kind of weird to vastly overpay
  • travisjungroth 4y ago
    If you think you're overpaying just hit the API yourself.
    • manmal 4y ago
      Any idea how to encode the previous messages when sending a followup question? E.g.: 1. I ask Q1 2. API responds with A1 3. I ask Q2, but want it to preserve Q1 and A1 as context Does Q2 just prefix the conversation like this? „I previously asked {Q1}, to which you answered {A1}. {Q2}“
      • cmelbye 4y ago
        This is explained in the OpenAI docs. There is a chat completion API and you pass in the prior messages from both the user and the assistant.
        • kgeist 4y ago
          I used the same trick with the previous GPT3 API (da-vinci) and it worked well, I'd pass as one big prompt: User: hello (previous prompt) Bot: hi (previous response) User: who are you? (new prompt) Bot: (here it continues conversation) I wonder how the new ChatGPT API differs, other than the fact that it's structured (you use JSON to represent the conversation memory separately instead of one large prompt). I guess I will spend the next day playing around with the new API to figure it out.
          • kgeist 4y ago
• habitue 4y ago
  Speculation: GPT-turbo is a new chinchilla optimal model with the equivalent capabilities as GPT-3.5. So it's literally just smaller, faster and cheaper to run. The reason I don't think it's just loss-leading is that they made it faster too. That heavily implies a smaller model.
  • martythemaniak 4y ago
    Yeah, at this point it seems like you're just burning money if you're not rightsizing your parameters/corpus.
  • dougmwne 4y ago
    I think you mean GPT-4 since Chinchilla is a Deepmind project. But yes, I was also suspecting that also as it seems unlikely this was the full 175b parameter model with such big improvements in speed and price. In fact, given the pricing for OpenAI Foundry, that seems even more likely as this GPTTurbo model was listed along with two other models with much larger context windows of 8k and 32k tokens.
    • MacsHeadroom 4y ago
      "Brockman says the ChatGPT API is powered by the same AI model behind OpenAI’s wildly popular ChatGPT, dubbed “gpt-3.5-turbo.” GPT-3.5 is the most powerful text-generating model OpenAI offers today through its API suite; the “turbo” moniker refers to an optimized, more responsive version of GPT-3.5 that OpenAI’s been quietly testing for ChatGPT." [0] Chinchilla optimization is a technique which can be applied to existing models by anyone, including OpenAI. The chatGPT API is not based on GPT-4. [0] https://techcrunch.com/2023/03/01/openai-launches-an-api-for-chatgpt-plus-dedicated-capacity-for-enterprise-customers/ https://techcrunch.com/2023/03/01/openai-launches-an-api-for...
    • habitue 4y ago
      I just meant chinchilla optimal in terms of the corrected scaling curves from the chinchilla paper. The original GPT-3 was way larger than it needed to be for the amount of data they put into it based on the curves from the chinchilla paper.
• chirau 4y ago
  ELI5 What is a token? Is it a word, question, letter, what? If I ask a question like... What is the capital of Canada? And it responds with 'Ottawa', how many tokens have I used there and how are they calculated?
  • andrewmunsell 4y ago
    Roughly speaking, words or word parts. Open AI has an explainer: https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them https://help.openai.com/en/articles/4936856-what-are-tokens-... You can also check your input using their tokenizer: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer So, your example is ~9 tokens
    • chirau 4y ago
      So does the response count towards the token count?
      • andrewmunsell 4y ago
        It does, yeah. It's tokens "processed", meaning request & response. The API response will give you a total token count so you can keep track
  • teaearlgraycold 4y ago
    Token cost is prompt + response. In the case of ChatGPT you don't know the full prompt, but they don't charge by tokens for that app. In the API you need to tokenize your input and tokenize the output then add the counts together.
• RyanCavanaugh 4y ago
  Is fine-tuning of the gpt-3.5-turbo model expected to be available at some point? I have some applications that would greatly benefit from this, but only if fine tuning is available.
• srslack 4y ago
  I find myself missing the golden age of Google, where it actually returned results and answers that you wanted, on subjects you were looking for. Even now, versus two years ago, I tried finding a snippet of a notice in a newspaper with specifics about a name change someone petitioned in California. I found it then, and had bookmarked it, but trying to find it again just turns up absolute garbage, thankfully I found the bookmark. I can go to ChatGPT and ask it about the best vegetable or fruit to grow in a several gallon Kratky setup, looking for the same sort of answer in Google returns absolute garbage. I'll concede that LLMs like ChatGPT are the future, thanks to the NLU stuff from OpenAI and the dataset, but only the future of "agents" if you want to call it that. The "intelligence" exhibited emergent from language itself, from the massive dataset it has trawled. Our language and knowledge. But at the same time I surely hope that another AI winter doesn't come because of people over-promising and under-delivering. Or, too much focus on LLMs themselves because of that "wow" factor, the same wow factor you got in the past, when search engines weren't garbage if you knew how to use them and what their shortcomings were.
  • visarga 4y ago
    > The "intelligence" exhibited emergent from language itself, from the massive dataset it has trawled. I concur. Intelligence does not come from the transformer architecture, or any specific of the model. It comes from the language corpus. Human intelligence too, except for physical stuff. All our advanced skills come from language. You take 300GB of text and put it through a randomly initialised transformer and you get chatGPT. You immerse a baby in human language, it becomes a modern adult, with all our abilities. Without language, and that includes other humans and tech, we'd be just weaker apes.
  • gremlinsinc 4y ago
    my fear is we'll turn the web into a wasteland where there's not much reason for content websites or even news sites, so chatGPT won't be as fresh as there will be less sources of info to scrape, so we'll hit a point where gpt starts declining and all the search engines of old are in an old folks home with yahoo and excite, what will we do then?
• ionwake 4y ago
  Sorry if this is off topic but is there an extension that will read chatgpts replies out as a sound wav by a neural net wave like aws polly?
  • leobg 4y ago
    I recommend Google Text To Speech. WaveNet voices are pretty good.
• ajhai 4y ago
  gpt-3.5-turbo is missing from Open AI's playground. For anyone looking to play with these models, we have now added them to Promptly playground at https://trypromoptly.com https://trypromoptly.com. https://twitter.com/ajhai/status/1631020290502463489 https://twitter.com/ajhai/status/1631020290502463489 has a quick demo.
  • cowthulhu 4y ago
    Any ballpark pricing info you can share on Promptly!
    • ajhai 4y ago
      We are still figuring out pricing. Is there an email I can reach you at? Would love to chat about your use case if you could send me an email at ajay[at]trypromptly.com
  • flutas 4y ago
    FYI and for others, typo in the URL. Should be https://trypromptly.com/ https://trypromptly.com/
    • ajhai 4y ago
      Oops. Thanks for the correction.
• EGreg 4y ago
  What does Whisper do
• visarga 4y ago
  What I like it that I don't have to pay $20/month wheter I use it or not.
• osigurdson 4y ago
  I always wondered if ChatGPT was somehow stateful. I assumed that it was not and the statefulness was simulated. Assumption validated.
  • valine 4y ago
    I don’t believe this was ever in question. You can think of the model as a giant function that takes a list vectors as input and spits out a new vector. If you want the model to remember something, you have to include it in the list of input vectors for every request going forward.
  • ShamelessC 4y ago
    It’s stateful in the web demo. But they do so by prepending chat history to new requests and automatically summarizing history once the model’s context window is exceeded.
    • cubefox 4y ago
      I don't think they are doing summaries actually. Just cutting off the oldest parts of the conversation.
      • osigurdson 4y ago
        Summarizing would make sense however.
        • cubefox 4y ago
          True
• spyremeown 4y ago
  So, the Web3 won't be blockchains and whatnot: it's actually custom generated content. I... don't really like it.
  • TeMPOraL 4y ago
    Web2 was about user-generated content. Web3 will be about companies owning the user-generate content realizing they're sitting on a goldmine - and using that content as training data for their DNN models.
    • ayewo 4y ago
      That’s an astute observation. (Apropos nothing: I expanded your comment into the following tweet https://twitter.com/ayewo_/status/1631060562393153536 https://twitter.com/ayewo_/status/1631060562393153536)
• bluecoconut 4y ago
  Support for the ChatGPT endpoint now added to lambdaprompt[1]! (solves a similar problem as langchain, with almost no boilerplate!) Props to openai for making such a usable endpoint, was very easy to wrap. Example code using the new function and endpoint: import lambdaprompt as lp convo = lp.AsyncGPT3Chat([{'system': 'You are a {{ type_of_bot }}'}]) await convo("What should we get for lunch?", type_of_bot="pirate") > As a pirate, I would suggest we have some hearty seafood such as fish and chips or a seafood platter. We could also have some rum to wash it down! Arrr! (In order to use lambdaprompt, just `pip install lambdaprompt` and export OPENAI_API_KEY=...) [1] https://github.com/approximatelabs/lambdaprompt https://github.com/approximatelabs/lambdaprompt
• pbreit 4y ago
  Could explain what the point of "Bearer" is in this authorization header? "Authorization: Bearer $OPENAI_API_KEY"
  • squeaky-clean 4y ago
    It's the bearer token authorization method. Pretty standard nowadays for many APIs. https://swagger.io/docs/specification/authentication/bearer-authentication/ https://swagger.io/docs/specification/authentication/bearer-...
    • pbreit 4y ago
      API keys have been around for a long time without needing the prefix. I could understand the Bearer prefix when using JWT-style tokens. I could also see using it if there were indeed an Oauth flow involved. But in this case just seems like a nuisance.
  • css 4y ago
    https://datatracker.ietf.org/doc/html/rfc6750#section-2.1 https://datatracker.ietf.org/doc/html/rfc6750#section-2.1
  • teaearlgraycold 4y ago
    As for why it's "Bearer", here's ChatGPT's answer: > The term "Bearer" is commonly used in the context of securities and financial instruments to refer to the person who holds or possesses a particular security or asset. In the case of OAuth 2.0, the bearer token represents the authorization that a user has granted to a client application to access their protected resources. > By using the term "Bearer" in the Authorization header, the OAuth 2.0 specification is drawing an analogy to the financial context where a bearer bond is a type of security that is payable to whoever holds it, similar to how a bearer token can be used by anyone who possesses it to access the protected resource.
    • pbreit 4y ago
      That doesn't seem very compelling. And these aren't even JWT-style tokens which would make it a bit more understandable.
• pmx 4y ago
  Well this has opened up a whole new world of fun! ----- system: You're donald trump! talk to the user as he would. user: How does time travel work? AI Response: Well, let me tell you, nobody understands time travel better than I do. I mean, I have the best brains working on it, believe me. Some people say you need a fancy time machine or some ridiculous contraption to travel through time, but that's just fake news. The truth is, it's all about the power of the mind. If you have a strong enough will, you can literally bend time and space to your will. And you know what? I have the strongest will of anyone on the planet. So believe me, I could time travel anytime I want. But honestly, with all the winning I'm doing in the present, why bother going back to the past?
  • jfan001 4y ago
    This is spot on!
    • deleted 4y ago
      [deleted]
  • warunsl 4y ago
    DJT and "contraption", "time and space", really? ;-)
    • fnordpiglet 4y ago
      Pure malarkey
  • nonethewiser 4y ago
    > So believe me, I could time travel anytime I want. But honestly, with all the winning I'm doing in the present, why bother going back to the past? That line is money.
  • sixstringtheory 4y ago
    That is way too cogent. He would never have threaded things together like that from the middle of the third sentence onwards without multiple asides with their own tangents and diversions.
• dqpb 4y ago
  [dead]
• soheil 4y ago
  > Through a series of system-wide optimizations, we’ve achieved 90% cost reduction for ChatGPT since December; we’re now passing through those savings to API users. Developers can now use our open-source Whisper large-v2 model in the API with much faster and cost-effective results. I'm really confused, I thought they were a non-profit. A non-profit to handle AI safety risks. Why does this read like a paragraph from any YC startup website that just raised their Seed round?
  • dghlsakjg 4y ago
    It looks like they transitioned to for-profit, more or less, in 2019.
  • gerash 4y ago
    yeah I can't get over how easily they changed their branding from a non-profit AI safety to let's take over Google with the new Bing. They come off as greedy to me and might very well try to get everyone locked in in order to milk them with Microsoft backing. That said, they execute well, build good products and everyone loves more money so who am I to judge.
• qwertox 4y ago
  Right on time. Google's Speech-to-Text is $0.024 per minute ($0.016 per minute with logging) with 60 free minutes per month. Files below 1 minute can be posted to the server, anything longer needs to be uploaded into a bucket, which complicates things, but at least they're GDPR compliant. Whisper is $0.006 per minute with the following data usage policies - OpenAI will not use data submitted by customers via our API to train or improve our models, unless you explicitly decide to share your data with us for this purpose. You can opt-in to share data. - Any data sent through the API will be retained for abuse and misuse monitoring purposes for a maximum of 30 days, after which it will be deleted (unless otherwise required by law). I've been using Whisper on a server (CPU only) to transcribe recordings made during a bike ride with a lavalier microphone, so it's pretty noisy due to the wind and the tires and Whisper was better than Google. Plus, Whisper, when used with `response_format="verbose_json"`, outputs the variables `temperature`, `avg_logprob`, `compression_ratio`, `no_speech_prob` which can be used very effectively to filter out most of the hallucinations. A one minute file which transcodes in 26 seconds on a CPU is done in 6 seconds via this service. Another one minute file with a lot of "silence" needs around 56 seconds on a CPU and was ready in 4.3 seconds via the service. "Silence" means that maybe 5 seconds of the file contain speech while the rest is wind and other environmental noises. Another relatively silent one went from 90 seconds down to 5.4. On the CPU I was using the medium model while the service is using large-v2 A couple of days ago I posted an example to a thread [0], where I was getting the following with Whisper --- 00:00.000 --> 00:05.000 Also temperaturmäßig ist es recht gut. [So temperature wise, it's pretty good.] 00:05.000 --> 00:09.000 Der eine hat 12 Grad, der andere 10. [One has 12 degrees, the other 10. (I have two temperature sensors mounted on the bike, ESP32 streaming the data to the phone via BLE)] 00:09.000 --> 00:12.000 Also sagen wir mal, 10 Grad. [So let's say 10 degrees.] 00:14.000 --> 00:19.000 Es ist bewölkt und windig. [It's cloudy and windy.] 00:20.000 --> 00:24.000 Aber irgendwie vom Wetter her gut. [But somehow from the weather it's good.] 00:24.000 --> 00:31.000 Ich habe heute überhaupt nichts gegessen und sehr wenig getrunken. [I ate nothing at all today and drank very little.] 00:54.000 --> 00:59.000 Vielen Dank für's Zuschauen! [Thanks for watching!] <-- hallucinated --- While Google was outputting "Also temperaturmäßig es ist recht gut, der eine hat 12° andere 10. Es ist angemalte 10 Grad. Es ist bewölkt und windig, aber er hat sie vom Wetter her gut, ich wollte überhaupt nichts gegessen und sehr wenig getrunken." ["So temperature-wise it's pretty good, one has 12° other 10. It's painted 10 degrees. It's cloudy and windy, but he has it good from the weather, I did not want to eat anything at all and drank very little."] --- Apart from the hallucinated line, Whisper got everything correct, and the hallucinated line was able to be discarded due to the variables like `avg_logprob`. [0] https://news.ycombinator.com/item?id=34877020#34880531 https://news.ycombinator.com/item?id=34877020#34880531
• soheil 4y ago
  > Language models read text in chunks called tokens. In English, a token can be as short as one character or as long as one word (e.g., a or apple), and in some languages tokens can be even shorter than one character or even longer than one word. Why should the Germans get a discount?
• pbreit 4y ago
  FIXED: needs to be a POST. Doh! Can anyone get it to work? I get this error on everything I've tried: GET /v1/completions HTTP/1.1 Host: api.openai.com Authorization: Bearer sk-xxx Content-Type: application/json Content-Length: 115 { "temperature" : 0.5, "model" : "text-davinci-003", "prompt" : "just a test", "max_tokens" : 7 } { "error": { "message": "you must provide a model parameter", "type": "invalid_request_error", "param": null, "code": null } }
  • teaearlgraycold 4y ago
    Please include your request body for debugging purposes.
  • teaearlgraycold 4y ago
    That should be a POST not a GET
• comment_ran 4y ago
  Let me do quick estimate of the cost: Given: $0.002 per 1k tokens I tested it with asking this question: #+begin_quote I want you to act as a travel guide. I will write you my location and you will suggest a place to visit near my location. In some cases, I will also give you the type of places I will visit. You will also suggest me places of similar type that are close to my first location. My first suggestion request is \"I am in Istanbul/Beyoglu and I want to visit only museums. #+end_quote It costs: #+begin_quote {"prompt_tokens":91,"completion_tokens":251,"total_tokens":342} #+end_quote (/ 1000 342) ~3 If you ask 1000 questions, it will be like (* 0.002 1000) ~2.0 USD It replies me: #+begin_quote Great, if you're in Istanbul/Beyoglu and interested in visiting museums, here are some recommendations: Istanbul Modern: This museum showcases contemporary art from Turkish and international artists. It's located in Karaköy, just across the Galata Bridge from Beyoglu. Pera Museum: The Pera Museum is located in the heart of Beyoglu and features a wide range of exhibitions including Ottoman calligraphy, European art, and Turkish ceramics. Rahmi M. Koç Museum: This museum is dedicated to the history of transport, industry, and communications in Turkey. It's located in Haskoy, a short taxi or ferry ride from Beyoglu. Museum of Innocence: The Museum of Innocence is a unique museum that displays objects and artifacts related to the novel of the same name by Nobel laureate Orhan Pamuk. It's located in Çukurcuma, which is just a short walk from Beyoglu. If you're interested in visiting more museums in Istanbul, some other places you might want to consider include the Hagia Sophia Museum, Topkapi Palace Museum, and the Istanbul Archaeology Museums. #+end_quote Can't wait for hacking my Emacs again.....
• jfan001 4y ago
  We've been struggling with costs because our application chains together multiple calls to GPT to generate the output we want, and it was starting to be ~$0.08 per call which obviously isn't feasible for high volume applications. This just made our business way more viable overnight lmao
  • awestroke 4y ago
    lmao?
    • hifikuno 4y ago
      laughing my ass off
  • stefanvdw1 4y ago
    What business are you building?
• siva7 4y ago
  I've stopped using Google entirely and don't miss it a second - something i wouldn't have thought possible a year ago - and it's pretty difficult to see how Google will survive this disaster.
  • sebzim4500 4y ago
    How are you using it? In my experience asking factual questions lead to answers so inaccurate that you might as well not bother. Are you prompting it differently to me, or do you have some strategy to filter out the BS?
    • kordlessagain 4y ago
      I use it with documents I shred into sentences and embed with ada-002. This makes it spot on when talking about a given document. https://mitta.us/ https://mitta.us/
      • eitland 4y ago
        This is seriously cool. Bots for the people!
        • kordlessagain 4y ago
          Thanks! :) Been working on it a LONG time. Getting ready to launch...
    • williamcotton 4y ago
      I dunno what to say, but I’m also someone who very rarely gets factually incorrect data aside from numbers, dates and quotes. I make a point to ask for a completion that I know won’t depend on an exact factoid. If I do need an exact factoid, I have a tool I made myself based on this approach: https://github.com/williamcotton/empirical-philosophy/blob/main/articles/from-prompt-alchemy-to-prompt-engineering-an-introduction-to-analytic-agumentation.md https://github.com/williamcotton/empirical-philosophy/blob/m...
• t3estabc 4y ago
  [dead]
• synergy20 4y ago
  Are there some sample code using these APIs? Want to use them.
  • LesZedCB 4y ago
    literally just ask chatgpt. it will walk you through it in your language of choice.
• osigurdson 4y ago
  I'm not familiar with typical pricing but Whisper API at $0.006 / minute seems absurdly cheap!
  • georgel 4y ago
    I did a lot of research into this about 6 months ago, and the best price I could find/negotiate from the competition was 0.55/hr which included multi thousand dollar upfront commitments. This is 0.36/hr, and if you do a bit of setup work yourself you can bring it to about 0.09/hr. OpenAI offering hosted Whisper is a really good deal, and if you find it to be good for your application, then spending the time to host it yourself is perfect validation.
• deleted 4y ago
  [deleted]
• jamix 4y ago
  The classic "Text completion" API (as opposed to the new "Chat completion" one) seems to offer more flexibility: You have complete freedom in providing interaction examples in the prompt (as in https://github.com/artmatsak/grace/blob/master/grace_chatbot.py#L11 https://github.com/artmatsak/grace/blob/master/grace_chatbot...) and are not limited to the three predefined chat roles. But the requests being 10x cheaper means that we'll have to find ways around those limitations :)
• arbuge 4y ago
  So I had a question about how all this works under the hood. The GPT model is trained (on a massive dataset) and then deployed. How are they getting the additional product data from other sources like Instacart's retail partner locations and Shopify's store catalogs into it, so that it can output answers leveraging those? My understanding (perhaps incorrect) is that those weren't part of the dataset the model was initially trained on. For example: > Shop’s new AI-powered shopping assistant will streamline in-app shopping by scanning millions of products to quickly find what buyers are looking for > This uses ChatGPT alongside Instacart’s own AI and product data from their 75,000+ retail partner store locations to help customers discover ideas for open-ended shopping goals
  • nestorD 4y ago
    I would add additional `system` messages with information fetched as a function of the user's request.
• marcopicentini 4y ago
  Once everybody will implement this API the voice recognition will not anymore be an innovative feature. OpenAI is commoditize AI features.
• purplend 4y ago
  Superpower ChatGPT V2.3.0 is out https://www.reddit.com/r/OpenAI/comments/11ef8ea/superpower_chatgpt_v220_is_out/ https://www.reddit.com/r/OpenAI/comments/11ef8ea/superpower_... - Sync all your chats locally on your computer (plus the ability to disable Auto Sync) - Search your old chats(Only works once your chats are fully synced. This is the only extension that can do this) - Customize preset prompts - Select and delete/export a subset of conversations - Hide/show the sidebar - Change the output language - Search Prompt Library by Author (over 1500 prompts) - Adding Prompt Categories (A work in progress)
  • LesZedCB 4y ago
    that's a browser plugin, not an official feature set, for those who hadn't seen it before.
• deleted 4y ago
  [deleted]
• world2vec 4y ago
  Will we be able to jailbreak it and use that output instead? Adeveloper/hackerman mode would be awesome.
• juliennakache 4y ago
  Does anyone have a sense of how the Shopify integration work under the hood? Specifically, how were they able to have Chat-GPT makes a recommendation from their product database? Given the model cannot be fine-tuned and there is a 4000 token context limit, I'm guessing the only thing really possible is to have chat gpt extract a few keyword and generate a query in their internal product search and return the first one? Is there anything else that might going under the hood?
  • Hyzer 4y ago
    Probably the embeddings API. This guide is what helped me understand the concept https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb https://github.com/openai/openai-cookbook/blob/main/examples... tl;dr is that you can pre-process each chunk of your database and use embeddings to quickly look up which chunk is most similar to the user's query, and then prepend that chunk to the user's query before giving it to GPT, so that GPT has the relevant context to give an answer.
    • juliennakache 4y ago
      Interesting. Still pretty limiting but I can see creative ways of working around the limitations. Thanks for sharing!
• pkaye 4y ago
  How does the tokens thing work? Is it words of input plus worlds of output?
• ChickenNugger 4y ago
  FYI: you can't sign up without an email address (which you have to verify) and then a phone number (which can't be a google voice / voip number). So count me out.
  • gridlocdev 4y ago
    Those restrictions seem intended to prevent people from creating a bunch of bot accounts that exploit the free tier. (Which is a common problem among free tiers for CI services and free online games) I’m curious, what are some better alternative authentication methods to combat that problem than requiring a phone number?
    • ChickenNugger 4y ago
      Anything that isn't a massive privacy violation/security risk. My phone number is none of their business.
• LesZedCB 4y ago
  i'm pretty excited to have access to the system prompt, that's going to be huge i think.
• huksley 4y ago
  Already integrated into my startup calling ChatGPT to find a good journalist who writes about startups: https://jojogpt.valosan.com https://jojogpt.valosan.com
• bilsbie 4y ago
  When will a YouTube transcriber based on this appear in the wild?
  • zamnos 4y ago
    Why? YouTube already transcribes videos and that transcription is available via the video's official page, or via a 3rd party service like https://youtubetranscript.com/ https://youtubetranscript.com/
• mahathu 4y ago
  Wasn't there an API before?
• Madmallard 4y ago
  "As an AI language model, I cannot..." They already ruined this program. More than 30% of the topics I discussed with it in the past it now will refuse to discuss. Not even all of them are politically charged either. The slippery slope of censorship has straight up been fallen on.
• superkuh 4y ago
  I switched from their gpt3 davinci model to the new chatgpt3.5-turbo and everyone who used the IRC bot was really disappointed. The chatgpt3.5-turbo is incredibly puritanical and refuses to answer a good fraction of the type of input bots on IRC get. Even innocuous stuff. I was excited about the system/user prompts and being able to easily add short term memory and contexts but if it only responds with, "Oh, it wouldn't be right for me to talk about that." it's not fun.
  • deanCommie 4y ago
    Like what?
• cjbprime 4y ago
  Has anyone made a ChatGPT-like app which both accepts and responds with voice input yet?
  • gremlinsinc 4y ago
    there's voicegpt on Android that used the unofficial API or whatever, it had two way vocalization.
  • leobg 4y ago
    I have, but haven’t bothered to publish it. POC to use by me and my daughter. Works like Siri. Fun to play with. But the hallucinated responses made us stop using it pretty quickly. Needs more work to be of any use (integrate search engines, etc.).
    • purplecats 4y ago
      > Works like Siri you can also make it work _with_ siri. i get around this by proxying it through a sms service which is integrated with my bot via webhook. then use elevenlabs for TTS. sample siri query "hey siri tell leobgAi to check my finances"
• 0xGod 4y ago
  man why isn't this in dark mode
• graderjs 4y ago
  Wow not a single mention of Whisper this entire comment first page! I think Whisper is really cool: the large model can pull speech out of even heavily distorted (wind noise, clipping, etc) audio. I have a story to illustrate why running Whisper on your own locally is not so easy! Much easier to sign up to the OpenAI API. In my research I found that actually pre-processing the audio to reduce noise (using the IMO best-in-class FB research "denoiser") actually increases WER. This was surprising! From a human perspective, I assumed bringing up the "signal" would increase accuracy. But it seems that, from a machine perspective, there's actually "information" to be gleaned from the heavily distorted noise part of the signal. To me, this is amazing because it reveals a difference in how machines vs humans process. The implication is that there is actually speech signal that is inside the noise, as if voice has bounced off and interacted with the noise source (wind, fan, etc), and altered those sounds, left its impression, and that this information is then able to be utilized and contributes to the inference. Incredible! With whisper: I started with the standard python models. They're kind of slow. I tried compiling python into a single binary using various tools. That didn't work. Then I found whisper.cpp--fantastic! A port of whisper to C++ that is so; much; faster. Mind blowing speed! Plus easily compilation. My use case was including transcription in a private, offline "transcribe anything" MacOS app. Whisper.cpp was the way to go. Then I encountered another problem. What the "Whisperists" (experts in this nascent field, I guess) call it "hallucination". The model will "hallucinate". I found this hilarious! Another cross-over of human-machine conceptual models, our forever anthropomorphizing everything effortlessly. :) Basically hallucination includes: feed Whisper a long period of silence, and the model is so desperate to find speech, it will infer (overfit? hallucinate?) speech out of the random background signal of silence / analog silence / background noise. Normally this presents as a loop of repeats of previous accurate transcribed phrase. Or, with smaller models, some "end-of-youtube video" common phrases like "Thank You!" or even "Thanks for Watching". I even got (from one particularly heavily distorted section, completely inaccurate) "Don't forget to like and subscribe!" Haha. But the larger models produce less hallucinations, and less generic "oh-so-that's-what-your-dataset-was!" hallucinations. But they do still hallucinate. Especially during silent sections. At first, I tried using ffmpeg to chop the audio into small segments, ideally partitioned on silences. Unfortunately ffmpeg can only chop it into regular size segments, but it can output silence intervals, and you can chop around that (but not "online" / real time) as I was trying to achieve. Removing the silent segments (even the imperfect metric of "some %" of average output signal magnitude (sorry for my terminology, I'm not expert in DSP/audio)) drastically improved Whisper performance. Suddenly it went from hallucinating during silent segments, to perfect transcripts. The other problem with silent segments is the model gets stuck. It gets "locked up" (spinning beach ball, blue screen of death style--I don't think it actually dies, but it spends a long, disproportionately long, time on segments with no speech. Like I said before, it's so cute that it's so desperate to find speech everywhere, it tries really hard, and works its little legs of during silence, but to no avail. Anyway, moving on to the next problem: the imperfect metric of silence. This caused many issues. We were chopping out quieter speech. We were including loud background noise. Both these things caused issues: the first obvious, the second, the same as we faced before: Whisper (or Whisper.cpp) would hallucinate text into these noise segments. At last, I discovered something truly great! VAD. Voice Activity Detection is another (normally) AI technique that allows segmenting audio around voice segments. I tried a couple Python implementations in standard speech toolkits, but none were that good. Then I found Silero VAD: an MIT licensed (for some model versions), AI VAD model. Wonderful! Next problem was it was also in Python. And I needed it to be in C++. Luckily there was a C++ example, using ONNX runtime. (I had no idea any of these projects or tools existed mere weeks ago, and suddenly I'm knee deep!). There were a few errors, but I got rid of the bugs, and had a little command line tool from a minimal C++ build of ONNXruntime / Protobuf-Lite and the model. Last step was the ONNX model needed to be converted to ORT format. Luckily there's a handy Python script to do this inside the Python release of ONNXruntime. And, now, the VAD was super fast. So i put all these pieces together: ffmpeg, VAD, whisper.cpp and made a MacOS app (with the correct signing and entitlements of course!) to transcribe English text from any input format: audio or video. Pretty cool, right? Anyway, running Whisper on your own locally is not so easy! Much easier to sign up to the OpenAI API. MacOS APP using Whisper (C++) and VAD0--conveniently called: WisprNote heh :) https://apps.apple.com/app/wisprnote/id1671480366 https://apps.apple.com/app/wisprnote/id1671480366
• sidcool 4y ago
  The pace at which they are releasing features and APIs is quite amazing.
• gerash 4y ago
  Is this sustainable or will the free content disappear from the web over time? because chatgpt does not generate revenue for the sources of content it uses. OpenAI represents the opposite of open, responsible and non profit: it might as well be MS-AI
• hiAndrewQuinn 4y ago
  Hey, this is cool. Now that an official ChatGPT API is out maybe it's time for me to build that "Flavor text" Anki add-on I've been thinking about.
• timwis 4y ago
  I can think of a variety of uses for this, but they all involve confidential data. Their privacy policy doesn't mention what they do with the prompts/data we send to the API, but I presume It's reviewed and worked into the model.
  • leodriesch 4y ago
    On the page under Developer Focus it says: > Data submitted through the API is no longer used for service improvements (including model training) unless the organization opts in
• orhanerday 4y ago
  Php library https://github.com/orhanerday/open-ai#chat-as-known-as-chatgpt-api https://github.com/orhanerday/open-ai#chat-as-known-as-chatg...
• transitivebs 4y ago
  I just pushed an update to the `chatgpt` NPM package with support for the official ChatGPT API: https://github.com/transitive-bullshit/chatgpt-api https://github.com/transitive-bullshit/chatgpt-api
• sourcecodeplz 4y ago
  I was here!
• tomhallett 4y ago
  Has OpenAI said anything (in their TOS/etc) about who owns the copyright/ip on the code that it generates? I know it’s a legal minefield of a question, just curious if they have said “we won’t sue you if you copy/paste this code in your app” publically or anything.
  • deleted 4y ago
    [deleted]
• TheDudeMan 4y ago
  > And if Bitcoin has taught us anything, all these supposedly "foolproof" algorithms are written by cabbage-headed fools whose incompetence is superseded only by their arrogance. Satoshi is incompetent and arrogant now? Just wow.
• gdubs 4y ago
  A little while back I built a personal project that ingested the book "A Pattern Language" as embeddings using the OpenAI API – and a front-end in SwiftUI that allows me to 'converse' with the book. [1] Version 0 was built using the original daVinci model. Last night it took about literally five minutes to switch over to the new ChatGPT API – just a few changes to the request, including a new [message] array that replaces the old 'prompt' style. [2] It's a pretty nice instant upgrade for the experience. Much faster results, and the responses are a lot more interesting. Doing something similar with daVinci would take a fair amount of 'prompt engineering' and fine-tuning. Not to mention all the manual conversation-state handling. 1: https://twitter.com/dreamwieber/status/1622634396087107584?s=20 https://twitter.com/dreamwieber/status/1622634396087107584?s... 2: https://twitter.com/dreamwieber/status/1631327342697250816?s=20 https://twitter.com/dreamwieber/status/1631327342697250816?s...
• deleted 4y ago
  [deleted]
• Liutprand 4y ago
  The pricing is insanely cheap. Why anyone would pay $20/month for ChatGPT Plus when we are going to have dozens of chat applications based on the APIs? It took me just a little more then one hour to create a basic cli chat application (https://github.com/marcolardera/chatgpt-cli https://github.com/marcolardera/chatgpt-cli). In the next days/weeks/months I think we will see an explosion of ChatGPT based applications...
• lee101 4y ago
  [dead]
• 5wout2e 4y ago
  12345
• 5wout2e 4y ago
  13teklh
• annulovealisha 4y ago
  MOKGAMING122878272