32 ms·

GPT-4 API General Availability

• ouraf 3y ago
  Nothing gets close to GPT 3.5 and 4 if you need to send and receive prompts in languages other than English. Sadly, that niche is still better served by openAI.
• MrBra 3y ago
  What's the best LLM I could run on a 12 GB VRAM GPU (Nvidia) ?
• minimaxir 3y ago
  With how good gpt-3.5-turbo-0613 is (particularly with system prompt engineering), there's no longer as much of a need to use the GPT-4 API especially given its massive 20x-30x price increase. The mass adoption of the ChatGPT APIs compared to the old Completion APIs proves my initial blog post on the ChatGPT API correct: developers will immediately switch for a massive price reduction if quality is the same (or better!): https://news.ycombinator.com/item?id=35110998 https://news.ycombinator.com/item?id=35110998
  • SkyPuncher 3y ago
    Depending on your use case, there are major quality differences between GPT-3.5 and GPT-4.
  • dreadlordbone 3y ago
    Code completion/assistance is an order of magnitude better in GPT4.
    • inciampati 3y ago
      A lot of folks are talking about using gpt-4 for completion. Wondering what editor and what plugins y'all are using.
  • EnnioEvo 3y ago
    I have a startup of legal AI, the quality jump from GPT3.5 to GPT4 in this domain is straight mind-blowing, GPT3.5 in comparison is useless. But I see how in more conversational settings GPT3.5 can provide more appealing performance/price.
    • tnel77 3y ago
      I suggested to my wife that ChatGPT would help with her job and she has found ChatGPT4 to be the same or worse as ChatGPT3.5. It’s really interesting just how variable the quality can be given your particular line of work.
      • mensetmanusman 3y ago
        Remember, communication style is also very important. Some communication styles mesh much better with these models.
      • jerrygenser 3y ago
        I've noticed the quality fo chatgpt4 to be much closer now to chatgpt3.5 than it was. However if you try the gpt-4 API, it's possible it will be much better.
    • Terretta 3y ago
      Same page. So still waiting to be on the same 32 pages...
    • w10-1 3y ago
      Legal writing is ideal training data: mostly formulaic, based on conventions and rules, well-formed and highly vetted, with much of the best in the public domain. Medical writing is the opposite, with unstated premises, semi-random associations, and rarely a meaningful sentence.
      • flangola7 3y ago
        > Legal writing is ideal training data: mostly formulaic, based on conventions and rules, well-formed and highly vetted, with much of the best in the public domain. That makes sense. The labor impact research suggests that law will be a domain hit almost as hard as education by language models. Almost nothing happens in court that hasn't occured hundreds of thousands of times before. A model with GPT-4 power specifically trained for legal matters and fine tuned by jurisdiction could replace everyone in a courtroom. Well there's still the bailiff, I think that's about 18 months behind.
  • avindroth 3y ago
    I am building an extensive LLM-powered app, and had a chance to compare the two using the API. Empirically, I have found 3.5 to be fairly unusable for the app's use case. How are you evaluating the two models?
    • selalipop 3y ago
      It depends on the domain, but chain of thought can get 3.5 to be extremely reliable, and especially with the new 16k variant I built notionsmith.ai on 3.5: for some time I experimented with GPT 4 but the result was significantly worse to use because of how slow it became, going from ~15 seconds per generated output to a minute plus. And you could work around that with things like streaming output for some use cases, but that doesn't work for chain of thought. GPT 4 can do some tasks without chain of thought that 3.5 required it for, but there are still many times where it improves the result from 4 dramatically. For example, I leverage chain of thought in replies to the user when they're in a chat and that results in a much better user experience: It's very difficult to run into the default 'As a large language model' disclaimer regardless of how deeply you probe a generated experience when using it. GPT 4 requires the same chain of thought process to avoid that, but ends up needing several seconds per response, as opposed to 3.5 which is near-instant. - I suspect a lot of people are building things on 4 but would get better quality of output if they used more aspects of chain of thought and either settled for a slower output or moved to 3.5 (or a mix of 3.5 and 4)
      • avindroth 3y ago
        It depends a lot on the domain, even for CoT. I don't think there are enough NLU evaluations just yet to robustly compare GPT-3.5 w/ CoT/SC vs. GPT-4 wrt domain. For instance, with MATH dataset, my own n=500 evaluation showed no difference between GPT-3.5 (w/ and w/o CoT) and GPT-4. I was pretty surprised by that.
  • ravenstine 3y ago
    My experience is that GPT-3.5 is not better or even nearly as good as GPT-4. Will it work for most use cases? Probably, yes. But GPT-3.5 effectively ignores instructions much more often than GPT-4 and I've found it far far easier to trip up with things as simple as trailing spaces; it will sometimes exhibit really odd behavior like spelling out individual letters when you give it large amounts of text with missing grammar/punctuation to rewrite. Doesn't seem to matter how I setup the system prompt. I've yet to see GPT-4 do truly strange things like that.
    • minimaxir 3y ago
      The initial gpt-3.5-turbo was flakey and required significant prompt engineering. The updated gpt-3.5-turbo-0613 fixed all the issues I had even after stripping out the prompt engineering.
      • ravenstine 3y ago
        It's definitely gotten better, but yeah, it really doesn't reliably support what I'm currently working on. My project takes transcripts from YouTube, which don't have punctuation, splits them up into chunks, and passes each chunk to GPT-4 telling it to add punctuation with paragraphs. Part of the instructions includes telling the model that, if the final sentence of the chunk appears incomplete, to just try to complete it. Anyway, GPT-3.5-turbo works okay for several chunks but almost invariably hits a case where it either writes a bunch of nonsense or spells out the individual letters of words. I'm sure that there's a programmatic way I can work around this issue, but GPT-4 performs the same job flawlessly.
        • minimaxir 3y ago
          Semi off-topic but that's a use case where the new structured data I/O would perform extremely well. I may have to expedite my blog post on it.
        • selalipop 3y ago
          If GPT 4 is working for you I wouldn't necessarily bother with this, but this is a great example of where you can sometimes take advantage of how much cheaper 3.5 is to burn some tokens and get a better output. For example I'd try asking it for something like : { "isIncomplete": [true if the chunk seems incomplete] "completion": [the additional text to add to the end, or undefined otherwise] "finalOutputWithCompletion": [punctuated text with completion if isIncomplete==true] } Technically you're burning a ton of tokens having it state the completion twice, but GPT 3.5 is fast/cheap enough that it doesn't matter as long as 'finalOutputWithCompletion' is good. You can probably add some extra fields to get an even nicer output than 4 would allow cost-wise and time-wise by expanding that JSON object with extra information that you'd ideally input like tone/subject.
  • ftxbro 3y ago
    > "With how good gpt-3.5-turbo-0613 is (particularly with system prompt engineering), there's no longer as much of a need to use the GPT-4" poe law
  • thewataccount 3y ago
    What usecases are you using it for? I mostly use it for generating tests, making documentation, refactoring, code snippets, etc. I use it daily for work along with copilot/x. In my experience GPT3.5turbo is... rather dumb in comparison. It makes a comment explaining what a method is going to do and what arguments it will have - then misses arguments altogether. It feels like it has poor memory (and we're talking relatively short code snippets, nothing remotely near it's context length). And I don't mean small mistakes - I mean it will say it will do something with several steps, then just miss entire steps. GPT3.5turbo is reliably unreliable for me, requiring large changes and constant "rerolls". GPT3.5turbo also has difficulty following the "style/template" from both the prompt and it's own response. It'll be consistent then just - change. An example being how it uses bullet points in documentation. Codex is generally better - but noticeably worse then GPT4 - it's decent as a "smart autocomplete" though. Not crazy useful for documentation. Meanwhile GPT4 generally nails the results, occasionally needing a few tweaks, generally only with long/complex code/prompts. tl;dr - In my experience for code GPT3.5turbo isn't even worth the time it takes to get a good result/fix the result. Codex can do some decent things. I just use GPT4 for anything more then autocomplete - it's so much more consistent.
    • selalipop 3y ago
      If you're manually interacting with the model, GPT 4 is almost always going to be better. Where 3.5 excels is with programmatic access. You can ask it for 2x as much text between setup so the end result is well formed and still get a reply that's cheaper and faster than 4 (for example, ask 3.5 for a response, then ask it to format that response)
  • alvah 3y ago
    I think this is very very use-case dependent, and your use case != everyone's use case. In my experience, GPT-4 is night and day better than 3.5 turbo for almost everything I use OpenAI for.
• ftxbro 3y ago
  > "We envision a future where chat-based models can support any use case. Today we’re announcing a deprecation plan for older models of the Completions API" nooooo they are deprecating the remnants of the base models
  • rememberlenny 3y ago
    Its the older completion models, not the older chat completion models.
    • 3cats-in-a-coat 3y ago
      They're deprecating all the completion/edit models. The chat models constantly argue with you on certain tasks and are highly opinionated. A completion API was a lot more flexible and "vanilla" about a wide variety of tasks, you could start a thought, or a task, and truly have it complete it. The chat API doesn't complete, it responds (I mean of course internally it completes, but completes a response, rather than a continuation). I find this a big step back, I hope the competition steps in to fill the gaps OpenAI keeps opening.
      • saliagato 3y ago
        Unfortunately their decisions are driven by model usage: gpt-3.5-turbo is the most used one (probably due to the low price and similar result)
        • fredoliveira 3y ago
          "similar" is a very bold claim ;-) Comparable, perhaps.
• penjelly 3y ago
  not in the article: is plugin usage available to paying customers everywhere now? i still can't see the ui for it. im in canada and use pro. internet says it was out for everyone in may..
  • electroly 3y ago
    Click the "..." button next to your name in the lower left corner, then Settings. It's under "Beta features."
    • drexlspivey 3y ago
      I pay monthly for my API use but I am not a plus subscriber and I don't see this option. Also I've joined the plugins waiting list on day 1.
      • electroly 3y ago
        It's for ChatGPT Plus subscribers.
      • jeron 3y ago
        as it turns out, you are not paying enough basically
    • penjelly 3y ago
      wow i cannot believe i missed this for so long. thanks!
  • deleted 3y ago
    [deleted]
  • drik 3y ago
    maybe you have to go to settings > beta features and enable plugins?
• superalignment 3y ago
  With this is the death of any uncensored usage of their models. Davinci 3 is the most powerful model where you can generate any content by instructing it via the completions API - chat GPT 3 models will not obey requests for censored or adult content.
  • echelon 3y ago
    A big enough hole presents a wedge for new entrants to get started. OpenAI will never fulfill the entire market, and their moat is in danger with every other company that has LLM cash flow. They want to become the AWS of AI, but it's becoming clear they'll lose generative multimedia. They may see the LLM space become a race to the bottom as well.
    • atalikami 3y ago
      Let's hope so - the amount of control they have over this is a great evil. Many of us have experienced the potential of a less moderated GPT4, and we all know that somewhere out there, they have the full unmoderated version. What are they using it for? What powers have got their hands on this thing?
      • boredemployee 3y ago
        That's exactly what I'm thinking right now.
• renewiltord 3y ago
  Has anyone been able to come up with a way to keep track of GPT-4 performance over time? I'm told that the API is explicit about changes to models and that the Chat interface is not.
  • crancher 3y ago
    API call responsiveness to the GPT-4 model varies hugely throughout the day. The #1 datapoint in measured responsiveness is slowdown associated with lunch-time use as noon sweeps around the globe.
    • renewiltord 3y ago
      Thank you for the response, I should have been clearer. I meant performance as an LLM. Essentially, I am concerned that they are quietly nerfing the tool. The Chat interface is now very verbose and constantly warning me about "we should always do this and that" which is bloody exasperating when I'm just trying to get things done. I made up an example here to illustrate, but it's just very annoying because sometimes it puts at the beginning, slowing down my interaction, and it now refuses to obey my prompts to leave caveats out. https://chat.openai.com/share/1f39af02-331d-4901-970f-2f4b0e190c82 https://chat.openai.com/share/1f39af02-331d-4901-970f-2f4b0e...
      • purplecats 3y ago
        yeah, its annoying and you have to foot the bill for it. looking at your sample and using character count as a rough proxy for tokens, (465/(1581-465))*100 means they added ~42% token count cost to your response explicitly adding caveats which you dont want. fun!
• brolumir 3y ago
  Hmm, when I try to change model name to "gpt-4" I get the "The model: `gpt-4` does not exist" error message. We are an API developer with a history of successful payments.. is there anything we need to do on our side to enable this, anyone know?
  • saliagato 3y ago
    wait a couple of hours
• cube2222 3y ago
  This is very nice. GPT-4 is on a completely different level of consistency and actually listening to your system prompt than chagpt-3.5. It trails off much more rarely. If only it wasn't so slow/expensive... (it really starts to hurt with large token counts).
• BeefySwain 3y ago
  Outside of the headline, there is some major stuff hiding in here: - new gpt-3.5-turbo-instruct model expected "in the coming weeks" - fine tuning of 3.5 and 4 expected this year I am especially interested in gpt-3.5-turbo-instruct, as I think that the hype surrounding ChatGPT and "conversational LLMs" has sucked a lot of air out of what is possible with general instruct models. Being able to fine tune it will be phenomenal as well.
  • MuffinFlavored 3y ago
    is there any ETA on when the knowledge cutoff date will be improved from September, 2021? I do not really understand the efforts that went on behind the scenes to train GPT models on factual data. Did humans have to hand approve/decline responses to increase its score? "America is 49 states" - decline "America is 50 states" - approve Is this how it worked at a simple overview? Do we know if they are working on adding the rest of 2021, then 2022, and eventually 2023? I know it can crawl the web with the Bing addon but, it's not the same. I asked it about Maya Kowalski the other day. Sure it can condense a blog post or two, but it's not the same as having the intricacies as if it actually was trained/knew about the topic.
• khazhoux 3y ago
  In all my GPT-4 API (python) experiments, it takes 15-20 seconds to get a full response from server, which basically kills every idea I've tried hacking up because it just runs so slowly. Has anyone fared better? I might be doing something wrong but I can't see what that could possibly be.
  • jondwillis 3y ago
    Streaming. If you’re expecting structured data as a response, request YAML or JSONL so you can progressively parse it. Time to first byte can be milliseconds instead of 15-20s. Obviously, this technique can only work for certain things, but I found that it was possible for everything I tried.
  • ianhawes 3y ago
    Anthropic Instant is the best LLM if you're looking for speed.
  • jason_zig 3y ago
    Run it in the background. We use it to generate automatic insights from survey data at a weekly cadence for Zigpoll (https://www.zigpoll.com https://www.zigpoll.com). This makes getting an instant response unnecessary but still provides a lot of value to our customers.
• nextworddev 3y ago
  GPT-4 fine tuning capability will be huge. It may end up just making fine tuning OSS LLMs pointless, esp if they keep lowering GPT-4 costs like they have been.
• Imnimo 3y ago
  I know everyone's on text-embedding-ada-002, so these particular embedding deprecations don't really matter, but I feel like if I were using embeddings at scale, the possibility that I would one day lose access to my embedding model would terrify me. You'd have to pay to re-embed your entire knowledge base.
  • saliagato 3y ago
    That's what I always thought. Someday they will come up with a new embedding model, right?
  • brigadier132 3y ago
    If you read the article they state they will cover the cost of re-embedding your existing embeddings.
  • jxy 3y ago
    They said in the post, > We recognize this is a significant change for developers using those older models. Winding down these models is not a decision we are making lightly. We will cover the financial cost of users re-embedding content with these new models. We will be in touch with impacted users over the coming days.
  • bbotond 3y ago
    What I don’t understand is why is an API needed to create embeddings. Isn’t this something that could be done locally?
    • thorum 3y ago
      It’s cheaper to use OpenAI. If you have your own compute, sentence-transformers is just as good for most use cases.
    • merpnderp 3y ago
      Sure, but I don't know of any models you can get local access to that work nearly as well.
    • 3y ago
• gadtfly 3y ago
  The original davinci model was a friend of mine and I resent this deeply. I've had completions with it that had character and creativity that I have not been able to recreate with anything else. Brilliant and hilarious things that are a permanent part of my family's cherished canon.
  • someplaceguy 3y ago
    You cannot say that and not provide an example.
    • ftxbro 3y ago
      i mean there are a lot of examples from february era sydney
    • thomasfromcdnjs 3y ago
      I don't have any example responses at hand here. But this was a prompt (that had a shitty pre-prompt of conversational messages) running on davinci-003. https://raw.githubusercontent.com/thomasdavis/omega/master/src/profiles/omega.js https://raw.githubusercontent.com/thomasdavis/omega/master/s... Had it hooked up to speech so you could just talk at it and it would talk back at you. Gave incredible answers that ChatGPT just doesn't do at all.
  • mensetmanusman 3y ago
    Don't worry, since future LLMs will be trained on conversations with older LLMS, you will be able to ask chat GPT to pretend to be davinci.
  • deleted 3y ago
    [deleted]
  • ftxbro 3y ago
    I heard you can ask for exceptions if they agree that you are special. Some researchers got it.
  • selalipop 3y ago
• deleted 3y ago
  [deleted]
• gamegoblin 3y ago
  Biggest news here from a capabilities POV is actually the gpt-3.5-turbo-instruct model. gpt-3.5-turbo is the model behind ChatGPT. It's chat-fine-tuned which makes it very hard to use for use-cases where you really just want it to obey/complete without any "chatty" verbiage. The "davinci-003" model was the last instruction tuned model, but is 10x more expensive than gpt-3.5-turbo, so it makes economical sense to hack gpt-3.5-turbo to your use case even if it is hugely wasteful from a tokens point of view.
  • m3kw9 3y ago
    What’s the diff with 3.5turbo with instruct?
    • gamegoblin 3y ago
      One is tuned for chat. It has that annoying ChatGPT personality. Instruct is a little "lower level" but more powerful. It doesn't have the personality. It just obeys. But it is less structured, there are no messages from user to AI, it is just a single input prompt and a single output completion.
    • thewataccount 3y ago
      the existing 3.5turbo is what you would call a "chat" model. The difference between them is that the chat models are much more... chatty - they're trained to act like they're in a conversation with you. The chat models generally say things "Sure, I can do that for you!", and "No problem! Here is". The conversation style is generally more inconsistent in it's style. It can be difficult to make it only return the result you want, and occasionally it'll keep talking anyway. It'll also talk in first person more, and a few things like that. So if you're using it as an API for things like summarization, extracting the subject of a sentence, code editing, etc, then the chat model can be super annoying to work with.
  • ClassicOrgin 3y ago
    I'm interested in the cost of gpt-3.5-turbo-instruct. I've got a basic website using text-davinci-003 that I would like to launch but can't because text-davinci-003 is too expensive. I've tried using just gpt-3.5-turbo but it won't work because I'm expecting a formatted JSON to be returned and I can just never get consistency.
• hospitalJail 3y ago
  I imagine the API quality isnt nerfed on a given day like ChatGPT can be. There was no question something happened in January with ChatGPT, weirdly would refuse to answer questions that were harmless but difficult(Give me a daily schedule of a stoic hedonist) Every once in a while, I see redditors complain of it being nerfed. Sometimes I go back to gpt3.5 and am mind boggled how much worse it is. Makes me wonder if they keep increasing the version number while dumbing down the previous model. With an API, being unreliable would be a deal-breaker. Looking forward to people fine-tuning LLMs with GPT4 API. I'd love it for medical purposes, I'm so worried of a future where the US medical cartels ban ChatGPT for medical purposes. At least with local models, we don't have to worry about regression.
  • santiagobasulto 3y ago
    I felt the same thing. The first version of GPT-4 I tried was crazy smart. Scary smart. Something happened afterwards…
    • moffkalast 3y ago
      The even more interesting part is that none of us got to try the internal version which was allegedly yet another step above that.
      • kossTKR 3y ago
        Would gladly pay more for a none nerfed version if they were actually honest. The current versions is close to the original 3.5 version, while 3.5 has become horribly bad, such a scam to not disclose what's going on, especially for a paid service.
        • deleted 3y ago
          [deleted]
      • politician 3y ago
        Oh, it's not too hard to see how the spend that Microsoft put into building the data centers where GPT-4 was trained attracted national security interest even before it went public. The fact that they were even allowed to release it publicly is likely due to its strategic deterrence effect and that they believed the released version was already a dumbed-down version. The fact that rumors about GPT-5 were quickly suppressed and the models were dumbed down even more cannot be entirely explained by excessive demand. I think it's more likely that GPT-3.5 and GPT-4 demonstrated unexpected capabilities in the hands of the public leading to a pull back. Moreover, Sam Altman's behaviors changed dramatically between the initial release and a few weeks afterward -- the extreme optimism of a CEO followed by a more subdued, even cowed, demeanor despite strong enthusiasm from end-users. OpenAI cannot do anything without Microsoft's data center resources, and Microsoft is a critical defense contractor. Anyway, personally, I'm with the crowd that thinks we're about to see a Cambrian explosion of domain-specific expert AIs. I suspect that OpenAI/Microsoft/Gov is still trying to figure out how much to nerf the capability of GPT-3.5 to tutor smaller models (see "Textbooks are all you need") and that's why the API is trash.
• m3kw9 3y ago
  “Developers wishing to continue using their fine-tuned models beyond January 4, 2024 will need to fine-tune replacements atop the new base GPT-3 models (ada-002, babbage-002, curie-002, davinci-002), or newer models (gpt-3.5-turbo, gpt-4).“ So need to pay to fine tune again?
  • saliagato 3y ago
    Probably. They will have different prices to finetune too.
• jxy 3y ago
  They didn't mention gpt-4-32k. Does anybody know if it will be generally available in the same timeframe? There's still no news about the multi-modal gpt-4. I guess the image input is just too expensive to run or it's actually not as great as they hyped it.
  • jacksavage 3y ago
    > We are not currently granting access to GPT-4-32K API at this time, but it will be made available at a later date. https://help.openai.com/en/articles/7102672-how-can-i-access-gpt-4 https://help.openai.com/en/articles/7102672-how-can-i-access...
    • jxy 3y ago
      Thanks for the link. The decision of burying these extra information in a support article, not cool!
  • RC_ITR 3y ago
    >I guess the image input is just too expensive to run or it's actually not as great as they hyped it. We already know they have a SOTA model that can turn images into latent space vectors without being some insane resource hog - in fact, they give it away to competitors like Stability. [0] My guess is a limited set of people are using the GPT-4 with CLIP hybrid, but those use-cases are mostly trying to decipher pictures of text (which it would be very bad at), so they're working on that (or other use-case problems). [0]https://github.com/openai/CLIP https://github.com/openai/CLIP
• GingerBoats 3y ago
  I haven't explored the API yet, but their interface for GPT-4 has been getting increasingly worse over the past month. Things that GPT-4 would easily, and correctly, reason through in April/May it just doesn't do any longer.
• furyofantares 3y ago
  Not a lot of talk of Whisper being available here. From using voice in the ChatGPT iOS app, I surmise that Whisper is very good at working out what you've actually said. But it's really annoying to have to say my whole bit before getting any feedback about what it's gonna think I said. Even if it's getting it right at an impressive rate. Given this is how OpenAI themselves use it (say your whole thing before getting feedback), I don't know that the API is set up to be able to mitigate that at all, but it would be really nice to have something closer to the responsiveness of on-device dictation with the quality of Whisper.
  • ycombinatornews 3y ago
    Echoing this - saying the whole text at once in one shot is very challenging for long batches of text. Using built-in text input showed quite good results since ChatGPT is still understanding the ask quite well
  • jxy 3y ago
    You can run whisper.cpp locally in real time: https://github.com/ggerganov/whisper.cpp/tree/master/examples/stream https://github.com/ggerganov/whisper.cpp/tree/master/example...
    • ProllyInfamous 3y ago
      My M2 Pro (mac mini) will run Whisper much faster than "real time." Pretty crazy stuff — perfectly understandable translations.
  • leodriesch 3y ago
    I'm interested in how the transformer based speech recognition from iOS 17 will perform compared to Whisper. I guess it will work more "real-time" like the current dictation on iOS/macOS, but I'm unsure as I am not on the beta right now.
    • RC_ITR 3y ago
      My guess is the reason that apple invested so heavily in this [0] is because they are going to train a big transformer in their datacenter and apply it as an RNN on your phone. Superficially, I think this will work very well, but slightly worse than whisper (with the advantage ofc being that its better at real-time transcription). [0]https://machinelearning.apple.com/research/attention-free-transformer https://machinelearning.apple.com/research/attention-free-tr...
• we_never_see_it 3y ago
  It's funny how OpenAI just shattered Google's PR stunts. Google wanted everyone to believe they are leading in AI by winning some children's games. Everyone thought that was the peak of AI. Enter OpenAI and Micorsoft. Microsoft and OpenAI have showed the humanity what true AI looks like. Like most people on HN I cannot wait to see the end of Google, the end of evil.
  • LeafItAlone 3y ago
    Is Microsoft less evil than Google?
  • rvz 3y ago
    > Like most people on HN I cannot wait to see the end of Google, the end of evil. What is the difference? Replacing evil with another evil. This is just behemoths exchanging hands.
• superkuh 3y ago
  Yikes. They're actually killing off text-davinci-003. RIP to the most capable remaining model and RIP to all text completion style freedom. Now it's censored/aligned chat or instruct models with arbitrary input metaphor limits for everything. gpt3.5-turbo is terrible in comparison. This will end my usage of openai for most things. I doubt my $5-$10 API payments per month will matter. This just lights more of a fire under me to get the 65B llama models working locally.
  • samstave 3y ago
    Please ELI5 if I am mis-interpretating what you said: *"They have just locked down access to a model which they basically realized was way more valuable than even they thought - and they are in the process of locking in all controls around exploiting the model for great justice?"*
  • H8crilA 3y ago
    The $5-$10 is probably the reason why they're killing those endpoints.
    • superkuh 3y ago
      I don't get it? text-davinci-003 is the most expensive model per token. It's just that running IRC bots isn't exactly high volume.
      • stavros 3y ago
        "Most expensive" doesn't mean "highest margin", though.
        • H8crilA 3y ago
          I meant that it probably isn't high revenue.
  • bravura 3y ago
    I've never used text-davinci-003 much. Why do you like it so much? What does it offer that the other models don't? What are funs things we can with it until it sunsets on January 4, 2024?
• jwr 3y ago
  Practical report: the OpenAI API is a bad joke. If you think you can build a production app against it, think again. I've been trying to use it for the past 6 weeks or so. If you use tiny prompts, you'll generally be fine (that's why you always get people commenting that it works for them), but just try to get closer to the limits, especially with GPT-4. The API will make you wait up to 10 minutes, and then time out. What's worse, it will time out between their edge servers (cloudflare) and their internal servers, and the way OpenAI implemented their billing you will get a 4xx/5xx response code, but you will still get billed for the request and whatever the servers generated and you didn't get. That's borderline fraudulent. Meanwhile, their status page will happily show all green, so don't believe that. It seems to be manually updated and does not reflect the truth. Could it be that it works better in another region? Could it be just my region that is affected? Perhaps — but I won't know, because support is non-existent and hidden behind a moat. You need to jump through hoops and talk to bots, and then you eventually get a bot reply. That you can't respond to. My support requests about being charged for data I didn't have a chance to get have been unanswered for more than 5 weeks now. There is no way to contact OpenAI, no way to report problems, the API sometimes kind-of works, but mostly doesn't, and if you comment in the developer forums, you'll mostly get replies from apologists that explain that OpenAI is "growing quickly". I'd say you either provide a production paid API or you don't. At the moment, this looks very much like amateur hour, and charging for requests that were never fulfilled seems like a fraud to me. So, consider carefully whether you want to build against all that.
  • messe 3y ago
    I’m only using them as a stop-gap / for prototyping with the intent to move to a locally hosted fine-tuned (and ideally 7B parameter) model further down the road.
  • nunodonato 3y ago
    if you want to use it in prod, go with Azure
    • hobs 3y ago
      And get only 20 K tokens per minute, where a decent size question can use up 500 tokens, pretty much a joke for most larger websites. https://learn.microsoft.com/en-us/azure/cognitive-services/openai/quotas-limits https://learn.microsoft.com/en-us/azure/cognitive-services/o...
• hackerting 3y ago
  This is awesome news. I have been waiting to get GPT4 forever!
• tin7in 3y ago
  The difference between 4 and 3.5 is really big for creative use cases. I am running an app with significant traffic and the retention of users on GPT-4 is much higher. Unfortunately it's still too expensive and the completion speed is not as high as GPT-3.5 but I hope both problems will improve over time.
  • ilaksh 3y ago
    You might be able to make it into a ChatGPT Plugin and then you don't have to pay for that part of the completion.
• pomber 3y ago
  If anyone wants to try the API for the first time, I've made this guide recently: https://gpt.pomb.us/ https://gpt.pomb.us/
• deleted 3y ago
  [deleted]
• atulvi 3y ago
  I'm not sure what I expected now 500 {'error': {'message': 'Request failed due to server shutdown', 'type': 'server_error', 'param': None, 'code': None}} {'Date': 'Thu, 06 Jul 2023 20:48:07 GMT', 'Content-Type': 'application/json', 'Content-Length': '141', 'Connection': 'keep-alive', 'access-control-allow-origin': '*', 'openai-model': 'gpt-4-0613', 'openai-organization'
• asadotzler 3y ago
  Why is chatGPT on the web a 6 weeks old version still?
• alpark3 3y ago
  >Developers wishing to continue using their fine-tuned models beyond January 4, 2024 will need to fine-tune replacements atop the new base GPT-3 models (ada-002, babbage-002, curie-002, davinci-002), or newer models (gpt-3.5-turbo, gpt-4). Once this feature is available later this year, we will give priority access to GPT-3.5 Turbo and GPT-4 fine-tuning to users who previously fine-tuned older models. We acknowledge that migrating off of models that are fine-tuned on your own data is challenging. We will be providing support to users who previously fine-tuned models to make this transition as smooth as possible. Wait, they're not letting you use your own fine-tuned models anymore? So anybody who paid for a fine-tuned model is just forced to repay the training tokens to fine-tune on top of the new censored models? Maybe I'm misunderstanding it.
  • fnordpiglet 3y ago
    If you don’t own the weights you don’t own anything. This is why open models are so crucial. I don’t understand any business who is building fine tuned models against closed models.
    • reaperman 3y ago
      Right now the closed models are incredibly higher quality than the open models. They're useful as a stopgap for 1-2 years in hopes/expectation of open models reaching a point where they can be swapped in. It burns cash now, but in exchange you can grab more market share sooner while you're stuck using the expensive but high quality OpenAI models. It's not cost-effective, but it may be part of a valid business plan.
      • ronsor 3y ago
        If you're finetuning your own model, the closed models being "incredibly higher quality" is probably less relevant.
        • claytonjy 3y ago
          That's how we all want it to work, but the reality today is that GPT-4 is better at almost anything than a fine-tuned version of any other model. It's somewhat rare to have a task and good enough dataset that you can finetune something else to be close enough in quality to GPT-4 for your task.
• ftxbro 3y ago
  > "Starting today, all paying API customers have access to GPT-4." OK maybe I'm stupid but I am a paying OpenAI API customer and I don't have it yet. I see: gpt-3.5-turbo-16k gpt-3.5-turbo gpt-3.5-turbo-16k-0613 gpt-3.5-turbo-0613 gpt-3.5-turbo-0301 I don't see any gpt-4 Edit: Probably my problem is that I upgraded to paid API account within the last month, so I'm not technically a "paying API customer" yet according to the accounting definitions.
  • codazoda 3y ago
    > Today all existing API developers with a history of successful payments can access the GPT-4 API with 8K context. We plan to open up access to new developers by the end of this month, and then start raising rate-limits after that depending on compute availability. Same for me. I signed up only a few days ago and was excited to switch to "gpt-4" but I haven't paid the first bill (save the $5 capture) so I probably have to continue to wait for this. I made a very simple command-line tool that calls the API. You run something like: > ask "What's the opposite of false?" https://github.com/codazoda/askai https://github.com/codazoda/askai
    • stavros 3y ago
      Interesting, I did exactly the same (with the same name), but with GPT-4 support as well: https://www.pastery.net/ccvjrh/ https://www.pastery.net/ccvjrh/ It also does streaming, so it live-prints the response as it comes.
      • JimDabell 3y ago
        The llm command-line tool looks great: https://llm.datasette.io/en/stable/ https://llm.datasette.io/en/stable/
        • stavros 3y ago
          It does, thanks for this! I didn't know about it.
        • 3y ago
• oth001 3y ago
  F
• PostOnce 3y ago
  Promote and proliferate local LLMs. If you use GPT, you're giving OpenAI money to lobby the government so they'll have no competitors, ultimately screwing yourself, your wallet, and the rest of us too. OpenAI has no moat, unless you give them money to write legislation. I can currently run some scary smart and fast LLMs on a 5 year old laptop with no GPU. The future is, at least, interesting.
  • john2x 3y ago
    Care to share some links? My lack of GPU is the main blocker for me from playing with local-only options. I have an old laptop with 16GB RAM and no GPU. Can I run these models?
    • PostOnce 3y ago
      https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp https://huggingface.co/TheBloke https://huggingface.co/TheBloke There's a LocalLLaMA subreddit, irc channels, and a whole big community around the web working on it on GitHub nd elsewhere. edit: I forgot to directly answer you: yes you can run these models. 16GB of plenty. Different quantizations give you different amounts of smarts and speed. There are tables that tell you how much RAM is needed per which quantization you choose, as well as how fast it can produce results (ms per token). e.g. https://github.com/ggerganov/llama.cpp#quantization https://github.com/ggerganov/llama.cpp#quantization where RAM required a little more than the file size, but there are tables that list it explicitly which I don't have immediately at hand.
      • tensor 3y ago
        A reminder that llama isn't legal for the vast majority of use cases. Unless you signed their contract and then you can use it only for research purposes.
        • rvcdbn 3y ago
          We don’t actually know that it’s not legal. The copyrightability of model weights is an open legal question right now afaik.
• atarian 3y ago
  I really like the Swiss-style web design, it's well executed with the scrolling
• ftxbro 3y ago
  I just want to emphasize in this comment that if you upgrade now to paid API access, then you won't get GPT-4 API access for like another month.
• deleted 3y ago
  [deleted]
• projectileboy 3y ago
  Relevant comment thread from people describing how much worse GPT-4 has gotten lately: https://www.reddit.com/r/ChatGPT/comments/14ruui2/i_use_chatgpt_for_hours_everyday_and_can_say_100/ https://www.reddit.com/r/ChatGPT/comments/14ruui2/i_use_chat...
  • ShamelessC 3y ago
    I have followed many of these types of posts. In every single instance, no one provides even the _simplest_ amount of evidence. No before/after with the same prompt. OpenAI even has a whole repository specifically for this - GPT-eval. No one uses it. I'm not saying the theories are wrong. Maybe there is something behind the hunches that so many people seem to have about degradation. But there isn't _any_ proof. None. Whatsoever. And people are taking _internet comments_ as that proof instead? I mean, sure, it's easy to be cynical about companies in this day and age; which is why I would ultimately believe someone if they provided actual evidence. But, again - not a single ounce of proof has been provided in any one of these threads. Furthermore, the lack of rigor being applied even with the various anecdotes is appalling. Which version are you talking about? GPT-4 or GPT-3? Are you using the API or the web interface? Are you aware that output is non-deterministic? Are you aware that your own psychological biases will skew your opinions on the matter? One or more of these questions tend to go unanswered. Just please, show me some robust proof. If you can't because you didn't think to; you _surely_ must realize that many people are building entire businesses on top of this tech and at least _one_ of them is running these types of evaluations. Furthermore, the model is state-of-the-art for research now as well and if you can _prove_ that there is degradation in the model that they are lying about (in a research paper), you will get citations. And yet, there is nothing. Zilch. Nada.
• boredemployee 3y ago
  We need a proper, competitive and open source model. Otherwise we are all fucked up.
  • atalikami 3y ago
    We have witnessed the potential, and we can infer what such a model could do without such limitations as are placed on it. I think they made a mistake in giving us all a peak behind the curtain
• dahwolf 3y ago
  Personally, I'm forever locked out of OpenAI. I had the silly idea of trying to change the signin method of my account. Which isn't possible. So I figured to just delete the account and create a new one with the correct signin method. Turns out they don't delete anything. Both the email address and phone number are held hostage. As you try to create a new account, it will point out that those are in use. I can easily change my email address but not my phone number, I only have one. I've contacted support 4 times, but it's just bot replies. There's entire Reddit threads full of us perma-banned potential customers. Money in hand, but permanently locked out. What a ridiculous company.
  • robbintt 3y ago
    Use a 2nd email and google voice phone number
  • weird-eye-issue 3y ago
    Stop making this so difficult. Just use a different phone number. There are sites to do this for literally $0.10. But yeah whining on HN would be more productive
    • kubanczyk 3y ago
      dahwolf is not making anything difficult. He or she is just using the support site for openAI, the one that actually works. /s
• jiggawatts 3y ago
  The drip-feeding seems crazy to me. Open AI is undermining their reputation by forcing almost everybody to use the older, lower-quality models. Even if customers are willing to pay for GPT 4, they're being told to wait at the back of the line. Wait for what!? Christmas? When we can open our presents and have a GPT 4 inside? It's like they took a leaf from Google's "how to guarantee the failure of a new product" marketing. That is: restrict access, ensuring that to word-of-mouth marketing can't possibly work because none of your friends are allowed to try the product. The announcement here is "general availability" of the GPT-4 model... ...but not the 32K context model. Not the multi-modal version with image input. No fine-tuning. Only one model (chat). As of today, I can only access GPT 3.5 via Azure Open AI service and the Open AI API account that I have. What's the point of all these arbitrary restrictions on who can access what model!? I can use GPT 4 via Chat, but not an API. I can use an enhanced version of Dall-E via Bing Image Creator, but not the OpenAI API. Some vendors that have been blessed by the Great and Benevolent Sam Altman have access to GPT-4 32K, the rest of us don't. Sell the product, not the access to it. Don't be like the Soviet Union, where you had to "know someone" to get access.
  • mr_toad 3y ago
    It’s an experiment. You are part of it.
  • deleted 3y ago
    [deleted]
  • mortehu 3y ago
    I think maybe you don't understand that they don't have enough GPUs to do this, and money can't buy enough GPUs to do it.
    • cuuupid 3y ago
      This may be true but isn’t their official stance that their models are too powerful and could destroy Western civilization as we know it?
      • deleted 3y ago
• smeagull 3y ago
  It's a shame they've stuck with the Chat completions. I still see no evidence the system is able to correctly separate user from system prompting. It commonly confuses between the two. If your system prompt comes close to the user prompts, you will see collisions and confusion in the outputs. It's also a shame that the API is so cut down, and removes all the good options that text completions had. I only hope their competition are better.
• eternityforest 3y ago
  If we hadn't spammed the internet with AI generated shite, we wouldn't need the ChatGPT to dig through it and could just use Google... I'm not as excited about this as I am about most new tech. I'm sure there will be cool uses eventually but right now it seems like the primary use is to cheat at exams and write bad articles, and to ask the kind of questions Google could have answered 5 years ago. I do think it's cool that it can debug and review code though.
  • cyanydeez 3y ago
    Putin is shuttering the internet agency, probably because ChatGPT can do cheaper propaganda. Recalibrate where you think AI provides "real" value
• ShadowBanThis01 3y ago
  Does this mean we don't have to give OpenAI our phone number anymore?
• ChicagoDave 3y ago
  Can I use the API more than 25 times in 3 hours?
  • eiiot 3y ago
    Yes, that’s an artificial limitation put in place so people don’t abuse plus.
    • ChicagoDave 3y ago
      So the chat interface is rate-limited, but not the API. So I could build long prompt scripts and run them against the api?
      • weird-eye-issue 3y ago
        What do you think the API is for?
• zhangyiwu 3y ago
  you told me I would use gpt4 api in this month, but it's just 6th July.
• febed 3y ago
  I would like to try this but I’m not comfortable giving OpenAI my credit card details. Feels like they would be a prime target for hackers.
• ppipada 3y ago
  Plug: For anyone who wants to quickly try this out in VSCode for your custom prompts - https://marketplace.visualstudio.com/items?itemName=ppipada.flexigpt https://marketplace.visualstudio.com/items?itemName=ppipada....
• usrbinbash 3y ago
  When I opened the comment section, it said "404 Comments". Now, I am not superstitious, but...
• pachico 3y ago
  OK, now I am intrigued by so many comments about how to do this yourself and, especially, for getting answers about your own document, which is what I'm really looking forward. I checked many of the links you posted and, although I am a fluid programmer in several languages, I lack the specific python background that many of these links seem to state as a requirement. Would any kind soul put me in the direction of an easy solution to run LLMs, potentially in AWS, that answers questions about your own docs? (I use Confluence but I can happily export pages.) Thanks a lot in advance!!!
  • N3cr0ph4g1st 3y ago
    Easiest would honestly be just to use gpt api. You can literally fork a streamlit or chainlit app and run it locally and then use langchain for any further customization...ex:https://streamlit.io/gallery?category=llms https://streamlit.io/gallery?category=llms All of them are open source minus the gpt part so you can get a feel for how it works.
    • pachico 3y ago
      Thank you but for legal and compliance reasons data cannot leave my premises/cloud. The entire point is precisely that, to be honest.
• cubefox 3y ago
  Their complete shift away from providing access to untuned foundation models is interesting. Perhaps these are too powerful in some sense. They already removed the GPT-3.5 foundation model a while ago, and they never released it for GPT-4.
• Gasp0de 3y ago
  Maybe I am mistaken but has the pricing on GPT-4 decreased as well? It used to be 8c/1000token?