16 ms·
OpenAI's Foundry leaked pricing says a lot
- seydor 4y agoIt looks like OpenAI's first mover advantage won't last long, considering the speed at which these models are improving and compacting themselves. Seems like this new 'Moore's law' will be even more steep, at least in the beginning . So steep, that we can hope to be running these in our desktops instead of on someone else's computer.
- sharemywin 4y agobut what kind of model will run in the cloud?
- eppp 4y agoWe could call it a weather model
- interestica 4y agoA model that augments the local? Runs above it? A supermodel?
- Tepix 4y agoI don't think Moore's law is the most important factor, instead algorithmic improvements will enable to have smaller models that are as capable as these humongeous models. Llama 65b is hopefully just the beginning of this trend. It outperforms OPT-175b.
- buildbot 4y agoThat holds true if scaling laws don’t hold true, otherwise a hypothetical Llama 175b would be even better. So the high end will always be on big clusters.
- deleted 4y ago[deleted]
- 2OEH8eoCRo0 4y ago'Moore's Law' was in quotes for a reason.
- sgk284 4y agoWe saw the same phenomenon with DALL-E. It was state-of-the-art for a blip in time before Midjourney and StableDiffusion surpassed it. And now with ControlNet + the ecosystem of domain-specific models, if you're doing serious generative art, OpenAI isn't even a part of the conversation. If OpenAI makes their revenue off of charging exorbitant fees to use the LLMs, they'll no longer have incentive to ever open them (even if abuse / misuse concerns are addressed) AND they'll have no incentive to make them more efficient. OpenAI has yet to show it can have a sustainable advantage. Every other player in the space benefits from an open model ecosystem and efficiency gains.
- l33tman 4y agoThey do pay for the compute time in some way or another so surely they have the same incentive as everybody else to make it more efficient, at least, to increase their margin. But yeah, they probably need to evolve from a (very good) two-trick pony into "the microsoft of AI" or something to stay afloat in the long run. That goes for the rest of the smaller AI companies as well though..
- andybak 4y agoMidjourney and StableDiffusion have not surpassed Dall-E for many types of image. They are still well behind in certain critical factors. I often jump to Dall-E when I hit a brick wall with SD. My money is on Deep Floyd (if and when it gets released)
- tmaly 4y agoWill be interesting to see what Nvidia H100s will do to the scene and what will come after them.
- Imnimo 4y ago>Handle customer interactions – natural language Q&A, appointment setting, account management, and even tech support, available 24/7, pick up right where you left off, and switch from text to voice as needed. Customer service and experience will improve dramatically. For example, Microsoft will let companies create their own custom versions of ChatGPT — read here. If I can prompt-hack your ChatGPT customer service agent into giving me a discount or accepting an otherwise invalid return or giving me an appointment at 3AM, how binding is that? And if the answer is "obviously it's not binding!", why should I trust anything else your bot tells me?
- mike_hearn 4y agoThat would presumably be treated the same as a human CSA doing the same thing.
- vorpalhex 4y agoTypically an agent of the company can act within their realm of authority. A human CSA can give you a discount, and that is usually binding. They can't declare X corp is going to mail a box of donuts to you every day for the rest of your life and have it be. So my expectation would be (in the absence of real world cases currently) that if you negotiate the AI into giving you a 10% discount on your bill.. it would be binding typically. Now if there's prompt hacking involved or obvious attempts to trick the AI, that would not be binding.
- arlcode 4y agoAs usual courts will also look at the evidence (aka the whole Transcript) and it'll matter whether the AI made a mistake on it's own (ie. Hallucinating an additional plausible rebate) or whether the customer performed an obvious prompt injection attack. A judge will not look favorably on your 5000 token prompt of carefully selected instructions telling the AI to go wild
- darkwater 4y ago
- ilaksh 4y agoThe Foundry pricing, $78,000 for a few months minimum, is absolutely the opposite of Open. It could completely kill my small startup if the API goes from less than a penny per request to requiring a huge up-front investment. It means that anyone bootstrapping is now locked out.
- layer8 4y agoThat’s the general risk of basing a startup on someone else’s service without having a solid contractual agreement, though. It’s always a gamble, in particular when the underlying service is an entirely new type of business.
- tikkun 4y agoThey only require an up front payment if you're renting a dedicated instance.
- riskneutral 4y agoDedicated instances (and high price tags, like $1.5 million per year) are necessary for large corporations to feel safe about sending their proprietary/private data to OpenAI, and to have performance/availablity SLAs in place if/when they start depending on OpenAI for critical workloads. Right now many companies are blocking access to OpenAI entirely because of the data privacy issues. I wouldn't be surprised if this "leak" is intentional and a way of getting feedback from the market on proposed product/pricing. I also wouldn't be surprised if some larger customers are knocking on OpenAI's door and demanding to run OpenAI's models on their own infrastructure to avoid sending any data outside their network.
- visarga 4y agoAbout $100K .. $200K are inference costs. They have a 10x markup.
- wmf 4y agoYeah but will "DV" be available any other way?
- mc32 4y agoWith reports[1] (maybe exaggerated, maybe apocryphal) of companies replacing FTEs with chatGPT, even at these high prices, it may make sense in some use cases, no, tho presumably this kills the playpen use cases. [1]https://it.slashdot.org/story/23/02/27/009234/survey-claims-some-companies-are-already-replacing-workers-with-chatgpt https://it.slashdot.org/story/23/02/27/009234/survey-claims-...
- buildbot 4y agoNobody is doing this, unless you have a source you would like to link?
- celestialcheese 4y agoHere's one anecdotal source confirming this - specifically some copywriting contractors. Significantly less hours needed per month writing. A lot more hours for our editors but it's still significant cost savings when netted out.
- mc32 4y agoAccording to this survey: https://it.slashdot.org/story/23/02/27/009234/survey-claims-some-companies-are-already-replacing-workers-with-chatgpt https://it.slashdot.org/story/23/02/27/009234/survey-claims-... "Earlier this month, job advice platform Resumebuilder.com surveyed 1,000 business leaders who either use or plan to use ChatGPT. It found that nearly half of their companies have implemented the chatbot. And roughly half of this cohort say ChatGPT has already replaced workers at their companies...." There are anecdotes elsewhere as well.
- gjsman-1000 4y agoSo much for "Open" AI. Open-source algorithm? No. Open access for free, or reasonable prices? No. Open emails and conversations of directors (accountability)? No. Open conversations of issues and ethics? No.
- rnd0 4y agoThank fuck they weren't called "HomelandAI" or "LibertyAI" or somesuch!
- honkler 4y agowell atleast now they are Open about their intentions :)
- HDThoreaun 4y agoOpen access for a fee? Yes. The original point of openAI was to make sure google and facebook don't completely dominate AI and keep their work hidden due to the innovator's dilemma they face. Seems like they've absolutely accomplished that. Yes, they've stretched the meaning of "open" but I don't think they've ever tried to claim they had some sort of open source aspirations.
- meghan_rain 4y agoOpenAI = Microsoft, which you have conveniently left out in your FAANG enumeration
- HDThoreaun 4y ago
- dragonwriter 4y ago> This really should not be a surprise, because even the standard-issue ChatGPT can pass the Bar Exam No, it can’t. The two things that together have sometimes gotten misrepresented that way in “game of telephone” presentations are: (1) that when tested on the multiple choice component of the multistate bar exam (not the whole bar exam), it got passing grades in two subjects (evidence and torts), not the whole multiple choice section; which is very much not the same thing as being able to pass the exam, and 50.3% overall (better than chance, since its four choices per question, but also very much not passing.) https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4314839 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4314839 (2) that a set of law professors gave it the exams for four individual law courses and it got an average of C+ scores on those exams, which are minimally-passing grades (but not on the bar exam.) https://www.reuters.com/legal/transactional/chatgpt-passes-law-school-exams-despite-mediocre-performance-2023-01-25/ https://www.reuters.com/legal/transactional/chatgpt-passes-l...
- version_five 4y agoI know this isnt the same, but: I've long had it on my list of things to try to train a classifier to predict the answer to multiple choice tests based on embedding of the questions. Many tests I've seen don't require actual intelligence to pass, just a plausible answer relative to the question phrasing
- tshaddox 4y agoA lot of multiple choice tests are only there to provide some minimum bar of plausible deniability for the examiner. The multiple choice section of a driving test (at least in the two U.S. states I've taken them) is a great example. The questions are almost entirely of the form: Which of these activities is legally permitted while operating an automobile? A. Wearing your seatbelt. B. Being intoxicated. C. Driving 120 miles per hour. D. Intentionally colliding with pedestrians. That way, when someone drives drunk, it clearly isn't the fault of the examiner, because they clearly verified that the person knew that was illegal! (Or at least, if they got that question wrong, they got enough other ones correct.)
- ccakes 4y ago> Streamline hiring – in such a hot market, personalizing outreach, assessing resumes, summarizing & flagging profiles, and suggesting interview questions. For companies who have an overabundance of candidates, perhaps even conducting initial interviews? That's a hiring red flag if I've ever seen one. The nightmare dystopia is just around the corner it seems.
- 3throwaway3141 4y agoDystopia? I think of it as a great comedy! We could be only months away from the first-ever hire where both the applicant and the company have automated the entire process and may not even realize there's been an accepted offer.
- MagicMoonlight 4y agoThat's a good point. I could pay OpenAI for a corporate account and then get it to apply for a hundred remote jobs. Then plug it into slack etc and let it work those jobs while I collect the money.
- daveguy 4y agoPretty sure that would fit the legal definition of fraud.
- flangola7 4y agoThe work is getting done isn't it?
- l33t233372 4y agoMaybe it’s more important that you personally do the work than the work get done.
- deleted 4y ago[deleted]
- sourcecodeplz 4y agoSuch a great read. Nothing eye opening really but well put and tied together. At this point any OpenAI leak is THE leak.
- blibble 4y agophoning customer support is already painful when there's a human on the end a bot that gives sometimes random answers seems like a particularly cruel form of torture to inflict on your customers
- risyachka 4y agoTrue but many companies won't care much as long as it saves them a few bucks.
- Quarrelsome 4y agoI would anticipate the application of this technology might be bi-directional. Why wait in a queue myself when I can use an AI to book my appointment for me or cancel my subscription for me?
- woeirua 4y agoThe single most valuable thing that Microsoft/OpenAI could use ChatGPT for, is to periodically crawl the sprawling SharePoint sites, Teams chat logs, and Github codebases for large companies. Having a customized ChatGPT for your company could be very valuable. That is assuming of course, that it doesn't just hallucinate a bunch of bullshit and can link back to the source documents, which BingGPT has had minimal success with so far.
- bilbo0s 4y agoUnless I miss my guess, the models that the big corporate cos pay millions for will not "just hallucinate a bunch of bullshit". It's the models that consumers have access to via APIs that will be decidedly less effective. The AI that makes movies for Disney will far outperform whatever movies the rest of us can put together on Stability's successors. If you thought the digital divide was bad, well, you ain't seen nothing yet.
- piperswe 4y ago> AI that makes movies for Disney At least, for now, works entirely made by AI aren't eligible for copyright, so we'll have to change the law before we see things like that.
- sanxiyn 4y ago> Attention mechanism inference costs rise with the square of the inference window. This is incorrect. It is correct for the original Transformer, but OpenAI isn't using the original Transformer since GPT-3. Sparse Transformer scales O(N sqrt(N)). https://openai.com/research/sparse-transformer https://openai.com/research/sparse-transformer
- boh 4y agoPeople don't yet understand where the value for AI services are. Most of the hype is coming from technology-centric sources and so isn't considering the tech in practical terms. The idea that any business would use AI in a decision making capacity is absolutely absurd and irresponsible. As it stands today, while AI may help filter and suggest, there will always be a person at the end of the decision-making process. Evangelizing the tech with unsubstantiated assumptions doesn't help legitimize it.
- atemerev 4y agoOh no. There will be AIs everywhere. AI CEOs. AI generals deploying AI autonomous weapons. AI hackers. AI information warfare specialists. Science isn’t about why, it’s about why not!
- nutanc 4y agoAround $250k for getting your own chatGPT. Sounds reasonable. Of course this is for the enterprises. It's upto their managers to define how they can extract $250k worth of value from the investment which I think is easy given the AI frenzy. Of course for startups they will have to use the direct OpenAI API on a pay per use model. One question that is not clear, can I take a ChatGPT instance and provide an API similar to OpenAI API and charge for it?
- transitivebs 4y agoI'm the author of the original leak this article is based on: https://twitter.com/transitive_bs/status/1628118163874516992 https://twitter.com/transitive_bs/status/1628118163874516992 You can read a full screenshot of the google doc that OpenAI shared publicly with partners in that thread, including pricing info.
- dandiep 4y agoDoes anyone known if OpenAI is allowing fine tuning to GPT35 for foundry customers?
- deleted 4y ago[deleted]
- pharmakom 4y ago> In short, anything for which there is an established, documented, the standard operating procedure will be transformed first. Work that requires original thought, sophisticated reasoning, and advanced strategy will be much less affected in the immediate term. then goes on to say that programmers are next.
- 9dev 4y agoI do believe that too, at least in the sense of programmers as the 99% working on trivial CRUD apps, which includes pretty much all of us at a given period of their career. Future engineers will need to up their ante, I suppose. Maybe this was overdue?
- ronnykylin 4y agoAbout the jobs that might be soon replaced by artificial intelligence, some would say, half jokingly, that AI will never replace accountants since it cannot work as a scapegoat.
- berniedurfee 4y agoAs CEO of Acme Mousetraps Inc, does this mean I can pay OpenAI $1.2M/yr to develop a mousetrap-aware model that can augment (replace?) my R&D team? Will I be able to ask this custom model to describe a new, novel (patentable) mousetrap that’s better and cheaper to build than anything ever before? Or will I at least be able to pay OpenAI to ensure Bing suggests using Acme mousetraps (clearly the best) when they ask what mousetraps they should buy?
- qualudeheart 4y ago“And can we really afford another in-house ML Ph.D. after we just dropped $1.5M on DV 32K?“ Here we see automation negatively impact the job market for elite talent. Deepmind can train a new model faster than you can go back to grad school.
- joshvince 4y agoPeople are so desperate for the next bubble