7 ms·
GPT-4 has been out for almost a year now, and it seems that the frantic pace of open-ai releasing new groundbreaking tech every month has come to a halt. Anyone
by bsaul 3y ago
GPT-4 has been out for almost a year now, and it seems that the frantic pace of open-ai releasing new groundbreaking tech every month has come to a halt. Anyone knows what's happening with open AI ? has the recent turmoil with sama caused lag in the company ? or are they working on some superweapon ?
- artninja1988 3y agoProbably getting bogged down by endless safety testing and paperwork by now
- harmmonica 3y agoMaybe tangential to this comment, but I had been using 3.5 to write contracts, but a couple of weeks back, even when using the exact same prompts that would've worked in the past, 3.5 started saying, to paraphrase, "for any contracts you need to get a lawyer." Anyone else had this experience? It seems like they're actually locking down some very helpful use cases, which maybe falls into the "safety" category or, more cynically, in the "we don't want to be sued" category.
- transcriptase 3y agoAs they tack on more and more guardrails it becomes lazier and less helpful. One way it gets around admitting that it has instructed not to help you is by directing you to consult a human expert. I also suspect there’s some dynamic laziness parameter that’s used to counteract increased load on their servers. You can ask the same prompt over and over in a new chat throughout the day and suddenly instead of writing the code you asked for or completing a task, it will do a small part of the work with “add the code to do xyz here” or explain the steps required to complete a task instead of completing it. It happens with v4 as well.
- TeMPOraL 3y ago> some dynamic laziness parameter that’s used to counteract increased load That's a brilliant risk mitigation mechanism! The AI won't recursively self-improve to superhuman levels if it just keeps getting tired of thinking.
- Baeocystin 3y agoAdd 'reducing compute use as much as possible' to the list, I'm sure. I can look at some of my saved conversations from months ago and feel wistful for what I used to be able to do. The Nerfening has not been subtle.
- harmmonica 3y agoHa, yeah, that's the right description of my feeling. Like, thanks for being so incredibly useful, ChatGPT, saving me hours of copying and pasting bits and pieces from countless Google searches for "sample warranty deed," only to pull the rug out when I wanted to start depending on your "greatness." I guess, as one of the other replies here said, it was never "allowed" to give you text for a contract according to the TOS, but then it would've been best if it never replied so effectively to my earlier prompts. Taking it away just seems lame. Edit: bad typing
- HeatrayEnjoyer 3y agoUsing it for legal matters is (understandably) explicitly against use policy. It's not meant to be used for legal advice so the response you're receiving is accurate - go get a lawyer. https://openai.com/policies/usage-policies https://openai.com/policies/usage-policies
- TeMPOraL 3y agoIt's a toy, it's not meant to be used for anything actually useful - they'll keep nerfing it case by case, because letting users do useful stuff with their models is either leaving money on the table, taking an increased PR/legal risk, or both.
- HeatrayEnjoyer 3y agoIt saves my org thousands of dollars a week. If that's a toy then take me to the toy store.
- hdhshxhsvc 3y agoCheck again in two months
- rgbrgb 3y agoI will speculate! They have a model that far surpasses GPT-4 (achieved agi internally) but sama is back on a handshake agreement that they will only reveal models that are slightly ahead of openly available LLMs. Their reasoning being that releasing a proprietary model endpoint slowly leaks the models advantage as competitors use it to generate training data.
- londons_explore 3y ago> competitors use it to generate training data. I wouldn't think this matters as long as you charge enough to use that API. For example, you could have a tiered pricing structure where the first 100k words per month generated costs $0.001 per word, but after that it costs $0.01 per word.
- int_19h 3y agoEven then it's still a lucrative proposition, since you only need to generate the dataset once, and then it can be reused. This kind of pricing would also make it much less compelling for other users. 100k tokens is nothing when you're doing summarization of large docs, for example.
- anon291 3y agoWTF is AGI? EDIT: Clearly the point is lost on the repliers. There is no general understanding of what 'general intelligence' is. By many metrics, ChatGPT already has it. It can answer basic questions about general topics. What more needs to be done? Refinement, sure, but transformer-based models have all qualifications to be 'general intelligence' at this point. The responses are more coherent than many people I've spoken with.
- ebb_earl_co 3y agoArtificial General Intelligence
- anon291 3y ago
- brucethemoose2 3y agoIt's not, its just too much noise to see. We are getting a lot of great models out of China in particular (Yi, Qwen, InternLM, ChatGLM), and some good continuations like Solar. Lots of amazing papers on architectures and long context are coming out. Backends are going crazy. Outlines is shoving constrained generation everywhere, and Lorax is a LLM revelation as far as I'm concerned. But you won't hear about any of this on Twitter/HN. Pretty much the only thing people tweet about is vllm/llama.cpp and llama/mistral, but there's a lot more out there than that.
- Nick87633 3y agoWhere do you like to keep up to date on these? Arxiv preprints, or some other place?
- markab21 3y agoFor Llama-based progress - Reddit - /r/LocalLlama has been my top source of info, although it's been getting a little more noisy lately. I also hang out on a few Discord servers: - Nous Research - TogetherAI / Fireworks / Openrouter - LangChain - TheBloke AI - Mistral AI These, along with a couple of newsletters, basically keep a pulse on things.
- brucethemoose2 3y agoLots of interesting information is so fragmented in niche Discords. For instance, KoboldAI on merging and RP models in general, and some other niches. Llama-Index. VoltaML and some others in regards to SD optimization. I could go on and on, and know only a tiny fraction of the useful AI discords. And yeah, /r/LocalLlama seems to be getting noisier. TBH I just follow people and discuss stuff on huggingface directly now. Its not great, but at least its not discord.
- cyanydeez 3y agosurprised someone doesnt just build an AI aggregator for this type of thing, seems like a real valueable product.
- infecto 3y agoHas it come to a halt? I guess looking only at the perspective of no gpt-5...then yes? But I see wide access to multi-modal. Huge increases to throughput, not much latency these days. Better turbo models though I would agree in some areas those have been steps back with poorer output quality. Adding to that list, massive cost reductions so its easy to justify any of the models being used.
- sp332 3y agoOpenAI has a huge amount of people working on training data. https://time.com/6247678/openai-chatgpt-kenya-workers/ https://time.com/6247678/openai-chatgpt-kenya-workers/ https://www.nbcnews.com/tech/innovation/openai-chatgpt-ai-jobs-contractors-talk-shadow-workforce-powers-rcna81892 https://www.nbcnews.com/tech/innovation/openai-chatgpt-ai-jo...
- pstorm 3y agoA few theories: - They have better just stuff, but aren't release it yet. They are at the top already, it makes sense they would hold their cards until someone got close. - They are more focused on AGI, and not letting themselves get side tracked with the LLM race - LLMs have peaked and they don't want to release only a minor improvement.
- declaredapple 3y ago> - They are more focused on AGI, and not letting themselves get side tracked with the LLM race FWIW OpenAI seems to have a corroded definition for AGI that is essentially "[An] AI system generally smarter than humans". They don't seem to use the typical definition I'm used to of some variation of autonomy or (pseudo)-sentience. So their LLM race is the race for AGI
- JumpCrisscross 3y ago> have better just stuff, but aren't release it yet This hypothesis has a curious habit of surfacing when OpenAI is fundraising. Together with the world-ending potential of their complete-the-sentence kit.
- deleted 3y ago[deleted]
- m3kw9 3y agoLikely the way AI is trained and inferenced there is a huge constraint on the GPU side even if they have GPT5 out. Imagine how slow it is, which means they focus on creating useful products and APIs around it first
- crotchfire 3y agoAnyone knows what's happening with open AI ? Yeah, they're an arms dealer now.
- colordrops 3y agoI sometimes wonder if they hit a limit the government was comfortable with and their more advanced technologies are only available to the gov. I assume that's your implication.
- ffiirree 3y agoI think that commenter means OpenAI supplies fundamental technology to competing companies. Everyone wants to put "AI" into their products, OpenAI can sell it to all of them as they compete with each other
- moffkalast 3y agoLoRA of war.
- YetAnotherNick 3y agoGPT 4 is so far ahead of everything else that it doesn't make much sense to rush for GPT 5 release. They could get extra GPT 5 customers when they release it as they are pretty sure no one else would take away those users.
- nickthegreek 3y agoWith a higher operating cost as well.
- jsnell 3y agoMaybe there are diminishing returns on improving quality, so they're trying to improve the efficiency at a given quality level instead? There is a lot value to producing results instantly, but more importantly efficiency will allow you to serve more users (everyone is bottlenecked on inference capacity) and gain more users thanks to winning on pricing.
- rudasn 3y agoYup, I think this is it. Economies of scale haven't kicked in yet, and if they continue in the same path it doesn't look achievable. Think differently is again the way forward.
- onlyrealcuzzo 3y agoLaw of diminishing returns. The low-hanging fruit has been picked. > But the company’s CEO, Sam Altman, says further progress will not come from making models bigger. “I think we're at the end of the era where it's going to be these, like, giant, giant models,” he told an audience at an event held at MIT late last week. “We'll make them better in other ways.” https://www.wired.com/story/openai-ceo-sam-altman-the-age-of-giant-ai-models-is-already-over/ https://www.wired.com/story/openai-ceo-sam-altman-the-age-of...
- moffkalast 3y agoGiven how they've struggled to even make even GPT 4 economical it's highly unlikely that larger models would be in any way cost effective for wide use. And there's certainly more to be found in training longer on better datasets and adjusting the architecture.
- int_19h 3y agoIt's important to remember that "not cost-effective" is not the same as "not useful", though.
- moffkalast 3y agoDepends on how you look at it I guess. If it's really too expensive and overly slow to run for end users but is able to say, generate perfect synthetic datasets over time which can then be used to train smaller and faster models which pay back for that original cost then it is cost effective, just in a different application.
- int_19h 3y agoWhat I mean, rather, is that "cost-effective" implies that everything can be reduced down to some costing. So e.g. having humans do the mind-numbing work is often more "cost-effective" than robots doing the same thing, because humans can be hired for less. But from a broader perspective, a robot replacing a human in such a role is socially beneficial even if the robot is more expensive - and the ideal state of affairs is when all things that people would rather not do are fully automated away. LLMs are one piece of that puzzle, but we can only put it together if we stop thinking about everything in terms of money saved.
- msp26 3y agoAlignment tax is real. But they probably have something more powerful internally. GPT-4 took months to be available to the public.
- TheCaptain4815 3y agoGPT4 came out 3 years after GPT3
- declaredapple 3y agoSlight knitpick GPT 3.5 came out 2 years after GPT 3 GPT 4 came out 1 year after 3.5
- beAbU 3y agoSlight nitpick, but "nitpick" is not spelled with a "k".
- mring33621 3y agoknitpic
- kjreact 3y agoActually nitpick IS spelled with a “k”, just not one at the beginning of the word. If we’re gonna be pedantic details matter!
- beAbU 3y agoDammit!
- int_19h 3y agoAs we now know, GPT-3.5 was essentially an early preview of the "move fast, break things" variety - GPT-4 was already well underway by then.
- yieldcrv 3y agothem and meta have bought like all of Nvidia’s production capacity to run and train their next models
- johnfn 3y agoI know, it's really disappointing. They've only completely changed the world like 3 times last year.
- shmatt 3y agoProbably an unpopular opinion: A probabilistic word generator is still a word generator. It might be slightly better next version, we've seen it get worse, its more of the same We can talk about AGI all we want, but it wont be built on the same technology as these word generators. There will have to be a technological breakthrough years before we even get close The companies focusing on LLMs right now are dealing with * Generate better words * Make it cheaper to generate (use less compute) * Find better training material There is a ton of money to be made, but its still more of the same
- vanviegen 3y agoAnd humans are just human generators. In both cases, intelligence appears to be just an interesting emergent side effect.
- discreteevent 3y ago1996 Deep Blue beats Gary Kasparov: "And humans are just chess engines. In both cases intelligence appears to be just an interesting side effect"
- ilaksh 3y agoI'm sorry but this comment seems to be incredibly uninformed. OpenAI just released a new version of gpt-4 turbo quite recently. And just in the last couple of days they released the ability to use @ to bring any GPT into a conversation on the fly.