5 ms·
"OpenAI’s is called GPT-4, the fourth LLM the company has developed since its 2015 founding." - that sentence doesn't fill me with confidence in the quality of
by simonw 2y ago
"OpenAI’s is called GPT-4, the fourth LLM the company has developed since its 2015 founding." - that sentence doesn't fill me with confidence in the quality of the rest of the article, sadly.
- 404mm 2y agoQuite funny that an article about AI was not fed to AI to proof read it.
- ToucanLoucan 2y agoBold of you to assume AI didn't write it, too.
- viraptor 2y agoEditing mistakes that AI wouldn't make is the new "proof of human input".
- KTibow 2y agoI've been messing around with base (not instruction tuned) LLMs; they often evade AI detectors and I wouldn't be surprised if they evade this kind of detection too, at least with a high temperature
- staunton 2y ago> with a high temperature More like: with the right prompting
- deleted 2y ago[deleted]
- jacobsimon 2y agoThere’s nothing grammatically offensive about this. It’s like saying, “Cars come in all colors. Mine is red.”
- simonw 2y agoNo, I'm complaining that just because GPT-4 is called GPT-4 doesn't mean it's the fourth LLM from OpenAI. Off the top of my head: GPT-2, Codex, GPT-3 in three different flavors (babbage, curie, davinci), GPT-3.5. Suggesting that GPT-4 was "fourth" simply isn't credible. Just the other day they announced a jump from o1 to o3, skipping o2 purely because it's already the name of a major telecommunications brand in Europe. Deriving anything from the names of OpenAI's products doesn't make sense.
- benatkin 2y agoWhile I’m sure it’s unintentional, that amounts to nitpicking. I can easily find three to include and pass over the rest. Face value turns out to be a decent approximation.
- simonw 2y agoIf this was a random blog post I wouldn't nitpick, but this is the Wall Street Journal.
- benatkin 2y agoThe thing is that I think it could be an optimal way of saying it. Should we not put it into context of making a particular LLM? Why count three versions of three LLMs? They made it hard to choose the one that makes up for not having GPT 1. GPT 3.5 and Codex are both good candidates. And of course calling GPT 4 the third and fifth could be considered as well.
- simonw 2y ago"OpenAI's fourth family of LLMs" or "fourth generation of LLMs" would work for me.
- benatkin 2y agoThat doesn’t resolve the problem of whether third or fifth is better than fourth. I have yet to be convinced that their wording here shows that they fail to grasp the pace of the development.
- dheera 2y agoArticles these days are probably written by ChatGPT
- MichaelDickens 2y agoI doubt it, if you ask ChatGPT whether GPT-4 is OpenAI's fourth LLM, it gives the correct answer. That's the sort of thing GPT-2 might have said.
- Grimblewald 2y agoWell, here's the interesting part - gpt2 has been writing news since well before gpt3 was launched. Remember when "news" started getting weirdly reptative? When just about any product had a review avaliable? When the amount of slop content just _exploded_? Thats when the ai colonization of the internet began.
- maxrmk 2y agoI was wondering about this one too... > At best, they say, Orion performs better than OpenAI’s current offerings, but hasn’t advanced enough to justify the enormous cost of keeping the new model running. wdym "keep it running"?
- overgard 2y agoWell, those server farms don't pay for themselves.
- maxrmk 2y agosure, but once it's trained there isn't a running maintenance cost
- bhouston 2y agoWell if it takes a ton of memory/compute for inference because of its size, it may be cost prohibitive to run compared to the ROI it generates?
- wongarsu 2y agoIf you offer an API you need to dedicate servers to it that keep the model loaded in GPU memory. Unless you don't care about latency at all. Though I wouldn't be surprised if the bigger reason is the PR cost of releasing with an exciting name but unexciting results. The press would immediately declare the end of the AI growth curve
- wavemode 2y agoOf course running inference costs money. You think GPUs are free?
- overgard 2y agoThere definitely is, storage, machines at the ready, data centers, etc. Also OpenAI basically loses money every time you interact with ChatGPT https://www.wheresyoured.at/subprimeai/ https://www.wheresyoured.at/subprimeai/
- overgard 2y agoThe article definitely has issues, but to me what's relevant is where it's published. The smart money and experts without a vested interest have been well aware LLMs are an expensive dead for over a year and have been saying as much (Gary Marcus for instance). That this is starting to enter mainstream consciousness is what's newsworthy.
- icpmacdo 2y agoGary Marcus is continuously lambasted and not taken seriously
- overgard 2y agoBy whom? He seems highly credible to me, and his credentials check out, especially compared to hype men like Sam Altman. All youre doing is spreading FUD by an unnamed "they"
- icpmacdo 2y agoHe only criticizes ai capabilities, without creating anything himself. Credentials are effectively meaningless. With every new release, he clamors for attention to prove how right he was—and always will be. That’s precisely why he lacks credibility.
- overgard 2y agoHe started and then sold a machine learning startup to Uber. He's also written multiple books about the construction of the human mind and he has a PhD from MIT. I would hardly call that creating nothing. He's not clamoring for attention, he's asking that AI be regulated and pointing out a lot of the glaring issues with the field.
- astrange 2y agoGary Marcus is just an anti-AI crank to balance out the pro-AI cranks. He's not credible.
- Narretz 2y ago> Several months later, Google launched the most viral new AI application of the year, called NotebookLM When I read this I was honestly confused. I had never heard of NotebookLM before.
- Rodmine 2y agoAlso, it was not a company when it was founded in 2015. It was in 2019 when they decided to change the non-profit org to a for-profit company.