8 ms·
GPT4 is up to 6 times more expensive than GPT3.5
- lewdev 4y agoAnd website still says "openai."
- newZWhoDis 4y agoMaybe they mean “open” as in “open your wallet”?
- saberd 4y agoAt last they didn't call it freeai
- icapybara 4y agoShould they not be able to make money off a world changing new technology they bet billions on? Does your company offer all its products for free?
- JohnFen 4y agoThe problem with the name isn't that they're charging. It's that they're implying that they're something in the ballpark of open source when they're not.
- djbusby 4y agoOpenTable isn't Open Source. Open, as a word, has more than one context.
- JohnFen 4y agoBut OpenTable was never open source to begin with, so that association was never made. Open AI started with a commitment to being open source (thus the "open"), but then they changed their minds and went closed source. I think in this context, keeping the "Open" in their name is a bit deceptive. It's not the biggest issue in the world, admittedly, but it does leave a bad taste in the mouth.
- djbusby 4y agoOh! that's new information for me. That case does feel like a -n-switch.
- evilduck 4y agoOpenTable references restaurants tables and times that are still open to be reserved. What contextual definition of "open" does OpenAI have? A website open to subscribers?
- wmf 4y agoA website open to subscribers? You can scoff at this but Google's models weren't available at any price before today.
- icapybara 4y agoAn open API you can build products off of. Consider the alternative. They could not let anyone use their models and just release AI products (their own search engine, their own version of Teams, their own self driving car software, their own photoshop competitor, etc).
- deelowe 4y agoThis is revisionist history. OpenAI very much started as a (somewhat) open source (for whatever definition of "open" makes sense for LLMs) initiative.
- icapybara 4y agoI’m not arguing they stuck with their original position. I’m saying the name is still reasonable. I don’t care if they changed positions.
- jackmott 4y ago[dead]
- sebzim4500 4y agoI think it was borderline defensible for them to not release the weights, but now they aren't even telling people the size of the model. From their website: >Our mission is to ensure that artificial general intelligence benefits all of humanity
- OmarAssadi 4y agoI pay for ChatGPT Plus and have paid a fair bit in OpenAI API fees; I don't care that they are making money. The problem is the openwashing, not the monetization.
- sparrc 4y ago6 times? My reading of these prices is that it's 45x more expensive (for 8K) and 90x more expensive (32k)
- behnamoh 4y agoThe title compares GPT3.5 (not ChatGPT) with GPT4. Although the question is: If ChatGPT can do almost anything GPT3.5 can do, then why is it priced almost 10x less? Is it because GPT3.5 is less "censored"?
- dragonwriter 4y ago> Although the question is: If ChatGPT can do almost anything GPT3.5 can do, then why is it priced almost 10x less? ChatGPT is GPT-3.5; with specific fine tuning / chat-oriented training, and without customer fine-tuning (at least, currently) available. It’s the particular GPT-3.5 interface that OpenAI wants people to preferentially use, so the price structure artificially encourages this.
- eldenring 4y agoThe ChatGPT API is using a smaller model than the original "GPT 3.5". The original, presumably larger, model is used by Legacy ChatGPT.
- behnamoh 4y agoWait is that true? I couldn't find anything that verifies this. Do you have a source? If it is in fact true, doesn't it fall into "false advertisement" category? That is, OpenAI using ChatGPT for demonstration purposes but then charging for an API that pretends to be ChatGPT but is in fact based on a much smaller model?
- wongarsu 4y agoThe normal ChatGPT interface also switched to the cheaper model, and lots of people complained when they switched (though not enough to make a big media splash). With ChatGPT Pro you can select between the new and the "Legacy" model (and now GPT4 as a third option)
- tagawa 4y agoI think you mean “up to 60” ($0.12 / 1K tokens as opposed to $0.002).
- speedgoose 4y agoThe OpenAI pricing models seem to be decided by ChatGPT.
- sebzim4500 4y agoHow so? They seem reasonable to me. You have the models which are pretty good that are cheap, and the models which are far ahead of the competition which are expensive.
- obblekk 4y agoWorth remembering GPT3 was 10-20 times more expensive 2yrs ago. There's a super-moore's law type learning function going on here and I suspect in 1yr GPT4.5 will be the same cost as GPT3.5 today. It's insane... feels like iPhone 3G to iPhone 4 level quality improvement every year.
- Maxion 4y agoGPT-4 is a scary improvement over 3.5, especially for handling code. It will be the literal definition of awesome when these models get a large enough context space to hold a small-medium sized codebase. I've been playing around with it for an hour seeing what it can do to refactor some of the things we have with the most tech debt, and it is astounding how well it does with how little context I give it.
- speedgoose 4y agoThe 32k tokens version should do fairly well on such codebase. I don’t know if it’s the one used in ChatGPT.
- Maxion 4y agoI’m not even kidding when I say I just did 6h or so of work in around 45 minutes
- deleted 4y ago[deleted]
- worldsayshi 4y agoDo you let it refactor your code base? How do you feed the code to it? Just copy paste?
- Maxion 4y agoYup, it’s a bit of a learning curve how to get it to output the most useful stuff with the least amount of prompts. Figure out what your goal is, and then start by giving a wide context and at first it will give wrong answers due to lack of context. With each wrong answer give it some more context fro what you think the solution provide is missing the most. Eventually you get 100% working code, or something so close to it that you can easily and quickly finish it yourself. This is one strategy, but there are many that you can use to get it to reduce the burden of refactoring.
- icapybara 4y agoWell just the other day people were shocked at how cheap GPT3.5 was.
- fswd 4y agoIt's $3.80 for the full 1000 prompt tokens plus 32k token long context. But OpenAI doesn't have a monopoly on long context, RWKV is free and open source and has virtually an unlimited context (as long as you remind it every once and a while). However ChatGPT really cannot be matched at this point perhaps except LLaMa Alpaca 7B EDIT: I accidentially typed Alpine, it's Alpaca. https://github.com/tatsu-lab/stanford_alpaca https://github.com/tatsu-lab/stanford_alpaca
- sebzim4500 4y agoAlpaca is supposed to be roughly comparable to GPT-3.5, which means it is probably far worse than GPT-4.
- stavros 4y agoBut also gpt-3.5-turbo is far worse than davinci-003.
- sebzim4500 4y agoBy what metric? For most stuff 3.5 is better in my experience. Assuming you are trying to complete an actual task rather than have fun.
- stavros 4y agoI've tried to make an assistant with ChatGPT, because it's cheaper, and it's abysmally worse than davinci-003 at following instructions and reasoning.
- deleted 4y ago[deleted]
- artdigital 4y ago3.5-turbo is already worse than 3.5-legacy, you can try this with chatgpt plus. Use the normal (turbo) model and then use the legacy model, you will almost certainly feel a difference in quality Turbo prioritizes speed (probably due to cost?), the legacy model has far higher quality of output. You can confirm this on Reddit where none of the chatgpt plus userbase seems happy with turbo and the general recommendation is to switch it back to legacy. I tried porting a mini project from davinci to the new turbo api and quickly reverted it, turbo output is a lot more all over the place and very hard to get into something useful. It’s a fun chatbot though and still great for simple tasks You get what you pay for, and there’s a reason 3.5-turbo is so cheap
- Mizza 4y agoDoes anybody have their hands on it? Is it actually 6 times better at performing tasks, or are we paying more for improved corporate-friendly bumper bowling?
- nonfamous 4y agoIf you've used Bing Chat, you've used GPT-4: https://blogs.bing.com/search/march_2023/Confirmed-the-new-Bing-runs-on-OpenAI%E2%80%99s-GPT-4 https://blogs.bing.com/search/march_2023/Confirmed-the-new-B...
- alchemist1e9 4y agoBing Chat is more useful in many cases because it provides references that connect to it’s responses. I don’t like the mobile UI and that it doesn’t save your chats for you. Maybe I should try the web interface.
- wetpaws 4y agoheavily lobotomized version of GPT-4
- simonw 4y agoNot the API, but if you pay for ChatGPT Plus you can try it out there. It's very, very impressive so far - I've tried a bunch of things I've previously run against GPT-3 and got noticeably better results - maybe not a 2x or 6x multiple in "quality" (depending on how you measure that) but still very clearly improved. Posted a few examples here: https://fedi.simonwillison.net/@simon/110022949941148725 https://fedi.simonwillison.net/@simon/110022949941148725
- Veen 4y agoI imagine this price will come down over time. OpenAI has repeatedly said that they haven't the infrastructure in place yet to handle too much load and expect to be "severely capacity constrained", so I assume the high pricing and usage caps on the ChatGPT version are to keep the load manageably low for the moment.
- avisser 4y agoI imagine those dollars from papa Nadella are buying LOTS of capacity as we speak.
- aiappreciator 4y agoIts not $ constrained. The existing cloud providers don't have 10000 A100s lying around to ramp up at any time. There's a very limited amount of A100s and H100s on the market. TSMC is printing them like candy, but it still takes time to ramp up. Azure also needs time to install them in datacenters. These A100s stacks probably produce monstrous amounts of heat compared to normal CPU racks.
- jazzyjackson 4y agoYou're not wrong but I think many people still have an intuition informed by a pre-silicon-capacity-crunch era, it cannot be taken for granted that the demand for GPU / TPU will be met over the next few years.
- albntomat0 4y agoThis may be because they recently found ways to make GPT3.5 much more efficient. https://openai.com/blog/introducing-chatgpt-and-whisper-apis https://openai.com/blog/introducing-chatgpt-and-whisper-apis
- colesantiago 4y agoNo surprise here, this is Classic Microsoft & OpenAI. If you know anything about business models this is by design. The minute that ChatGPT's pricing came close to free, it was obvious they had a way better model. It was never about OpenAI having found a more efficient way to reduce costs, it was to price out competitors like Google, Meta, Open Source, etc with a "good-enough" breakthrough model. Then introduce an expensive superior corrected model.
- wongarsu 4y agoI'm more impressed that it's less than 3 times as expensive as Davinci GPT3. In usecases where most tokens are in the context even only 50% more expensive, while allowing 4x as much context. And you can pay a 100% surcharge to get another fourfold increase in context size. It only looks expensive when compared to the GPT-3.5-Turbo ChatGPT offering which is incredibly cheap (or alternatively Davinci is overpriced by now)
- tlogan 4y agoCan you please explain how GPT-4 is cheaper than davinci GPT3. Maybe I do not understand the pricing here…
- IanCal 4y agoIt's not. I think you've perhaps misread "less than 3 times as expensive as" as "3 times less expensive as" or similar. It's more expensive but not more than 3x the cost.
- dogma1138 4y agoDesigner leather jackets boutiques world wide are rejoicing.
- aiappreciator 4y agoI remember as a young gamer, asking my Dad whether Nvidia or Intel was a bigger company. It seemed obvious to me, since people always bragged about their GPUs online, but never the CPU, that Nvidia is the more important company. At that time, Nvidia was worth like 1% of Intel. Today, regarding online discussions, nothing has changed. People still brag about GPUs, never the CPU. But now Nvidia is worth 5 times as much as Intel.
- mdasen 4y agoI feel like this is comparing apples and oranges a bit, depending on when you were a kid. Even within the GPU market, Nvidia wasn't a huge player for a while. It's not just that GPUs are more important today, but that Nvidia went from being a small player in the GPU market to essentially being the GPU market. At the same time, Intel's presence in the CPU market has never been lower. Most "computers" are now smartphones running ARM processors, data centers are using AArch64 more and more, there's a slowdown in businesses and consumers buying computers, and AMD is taking a huge share of the X64 market. Likewise, Nvidia is priced at 102x earnings while Intel is priced at 14x earnings. Some of that might be that people are anticipating the GPU market to be where all the money is in the future, but a decent amount of that is pessimism about Intel's place in the market. You're not even comparing the CPU market to the GPU market. You're comparing what was possibly the 3rd ranked GPU marker when you were a kid and Intel when you were a kid dominating with 90-95% of CPU sales with Nvidia being 90% of the GPU market's profits today and Intel being more of a bit player in the CPU market. Some of Nvidia's size today is because they beat all the other graphics card companies (yes, ATI/AMD is still around, but Nvidia's marketshare is above 85%). Some of that comes from GPUs becoming a much bigger factor in many things like AI. But when comparing Intel and Nvidia today, Intel is still getting double the revenue and profits of Nvidia. This is despite the fact that most CPUs have moved away from their architecture, despite the fact that they've had at least half a decade of terrible engineering performance, despite AMD starting to kick their butt and taking serious marketshare on their architecture, etc. I don't want to say that GPUs aren't important, but Nvidia being worth more than Intel probably says more about Nvidia executing well while Intel stumbles than anything else. If you were asking your father this in 1998, Nvidia was just releasing the RIVA TNT, 3dfx was still king, and Intel was so dominant it looked like they'd be unstoppable forever. Now you're looking at an Intel that doesn't even look like the king of their own architecture, has been having foundry issues for years, and an Nvidia that no other GPU maker has been able to touch. If another GPU maker beats Nvidia and Nvidia becomes smaller than Intel, does that mean that GPUs aren't important any more? No. The problem isn't your conclusion that GPUs are important today, but the logic getting there. People bragging about things isn't necessarily a great barometer for future success. Lots of things are trendy and lots of companies come and go. Everyone was talking about Pogs and Beanie Babies at one point. That hasn't made them last. People were talking about smartphone/handheld devices since the 80s, but almost all of those companies aren't in that market today (Palm, Handspring, Blackberry, Microsoft). Instead, Apple won that market. Heck, you could have asked your father if 3dfx were bigger than Intel and seen them go bankrupt a few years later. Where's Matrox and S3? This is an anecdote that just feels like survivorship bias. Yep, you asked about the GPU maker that won the market and the CPU maker that would face big declines (and still Intel's revenue/profits are double Nvidia's).
- sebzim4500 4y agoWhy is the 32K context only twice as expensive as the 8K context? Are they using sparse attention or something? I don't think flash attention on it's own can explain it. EDIT: Oh right, if the cost is per token so if you actually fill the context then it is 8x more, which makes much more sense.
- cscheid 4y agoAsk it the other way around, "Why is the 8K context only twice as cheap as the 32K context?" and your original answer is clearer: because they think the demand curve supports it. Price is not determined by cost, but by how much people are willing to pay.
- sebzim4500 4y agoNo I'm just an idiot. Since the price is per token the actual ratio is more like 8x more (assuming you fill both contexts) which is plausibly the ratio between the costs.
- Maxion 4y agoIn the end the cost for you is how many tokens consumed, if you can get more value out of one request with 32k context that multiple with lower, then it’s cheaper (and faster). Some problems also require a larger context to be able to solve.
- jsyolo 4y agoCan GPT be used for something like feeding it a 2000 page PDF of pure text(not english) and ask questions about its contents?
- wmf 4y agoNo. In the near future they will support ~50 pages.
- Kiro 4y agoThis supports up to 2000 pages: https://www.chatpdf.com/ https://www.chatpdf.com/
- jimmyechan 4y agoNot yet, but my bet is that we will be able to in the near future. The gpt-4-32k (with a 32K context window) allows for about 52 pages of text
- IanCal 4y agoNot without other tooling. Things like langchain and llama_index would be good starting points. An approach would be to use llama_index to create embedding vectors for each section of the pdf, then you query and it gets a vector for your query -> gets the context -> puts it into gpt + your query -> returns the result. I've seen people say it's better to ask gpt for a fake answer then use the embedding of that answer to search (so you're looking for context that looks like the answer). I don't know if that's supported in those tools.
- tlogan 4y agoThe title should be changed to “GPT4 is up to 90x more expensive than GPT3.5”
- thelittleone 4y agoIs it conceivable that ChatGPT could review itself and produce a superior revision to a human?
- bckr 4y agoExciting thought!
- antibasilisk 4y ago*alarming
- falcor84 4y agoThat's one of the definitions of the Singularity. Assuming it doesn't quickly hit diminishing returns, at that point in time, you would get a so-called "fast takeoff" of superintelligence.
- pixl97 4y agoChatGPT, unlikely. But with that said, we're already using AI optimizers to make better versions of AI chips like the H100, so take that with what you will. Current GPT context is too low for this, I believe. We'll have to have something a little closer to AGI for this, and I'm unsure if a cyclic working memory (come up with an output, review the output, review that output, output) will get us to AGI itself. We still need selection functions that bind AI to the real world and not the dream state it lives in. Maybe just hooking it to compilers and allowing it to run test functions will get it a lot farther in self learning, but again, no one seems to know at this point.
- thelittleone 4y agoGenuinely curious as to why ChatGPT could not so I asked it. Note the final bullet point. I posit, that if it is possible, it will happen (regardless of ethics), because the risk of not doing it (when your adversary will) outweighs the risk of doing it. "You say an instance of the GPT cannot experiment with new architectures, but could it if a) it was modified by the developers and b) it was given access to sufficient computational resources?" Response: "In theory, if a GPT model were to be modified by developers to include capabilities for analyzing, designing, and testing new architectures, and if it were given access to sufficient computational resources, it could potentially experiment with new architectures. This would involve incorporating additional machine learning and deep learning algorithms and methods, such as reinforcement learning, evolutionary algorithms, or neural architecture search, which can be used to explore new model designs. However, there are a few caveats to consider: - Developing such a system would require significant expertise in AI research, as well as an understanding of the specific problems that need to be addressed. - The GPT model is a complex system, and adding the ability to self-improve or create new architectures could make the model even more complex and challenging to manage. - The computational resources required to develop, test, and implement new architectures are substantial, and optimizing resource allocation would be a critical concern. - Ensuring the safety and ethical use of such a self-improving AI system would be an essential consideration, as it could potentially lead to unintended consequences or malicious use. Although it is theoretically possible to create a GPT-like model with the ability to experiment with new architectures, it would require significant advancements in AI research, development, and ethical considerations.
- dirkg 4y agoIf this is public I can only imagine what Google has internally. Does DeepMind compete in this space or is it fundamentally different than a llm like Bard? With their massive codebase and already deep investment in AI/ML, I'm pretty sure Google and likely MS already have the ability to do massive refactoring, validate it using tests, reiterate, train, rinse and repeat.
- dang 4y ago"Please use the original title, unless it is misleading or linkbait; don't editorialize." - https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html If you want to say what you think is important about an article, that's fine, but do it by adding a comment to the thread. Then your view will be on a level playing field with everyone else's: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&sort=byDate&type=comment&query=%22level%20playing%20field%22%20by:dang https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
- nwienert 4y agoThe title of the launch of my product a couple months ago - a labor of love - was nerfed, despite being backed by benchmarks.
- dang 4y agoHN's guidelines don't call for using the original title in every case; there are quite a few exceptions. Most likely your title broke one of those rules. For example, it's certainly possible for a title to be linkbait even when it is backed by benchmarks. But I'd have to know which submission it was to answer in detail!
- pg_1234 4y agoSometimes you have to abbreviate to keep a title under the character count. Sometimes what is in the page's <title/> tag is not the same as the readable title on the article ... and as many news aggregators link with the content's of the <title/> tag, which is what caught your attention, it is not unreasonable to use that. And something I've observed recently is titles on news articles being updated after they are published ... political pressure on editors ¯\_(ツ)_/¯ ... it usually manifests as a "watering down" of the original title. This doesn't justify wild re-writes ... but "original title" is not as clear cut as one might wish.
- dang 4y ago> Sometimes you have to abbreviate to keep a title under the character count. Correct! > Sometimes what is in the page's <title/> tag is not the same as the readable title on the article ... and [...] it is not unreasonable to use that. Also quite correct! > And something I've observed recently is titles on news articles being updated after they are published Also entirely correct. (I don't know if it's pressure - sometimes they're just correcting things - but we don't really need to know why.) > This doesn't justify wild re-writes ... but "original title" is not as clear cut as one might wish. Absolutely right!