8 ms·
These announcements have started to look like a template. - Our state-of-the-art model. - Benchmarks comparing to X,Y,Z. - "Better" reasoning. It might be a
by Oras 2y ago
These announcements have started to look like a template.
- Our state-of-the-art model.
- Benchmarks comparing to X,Y,Z.
- "Better" reasoning.
It might be an excellent model, but reading the exact text repeatedly is taking the excitement away.
- Mistletoe 2y agoI’m sure the AI helps write the announcements.
- ototot 2y agoMaybe they just asked Gemini 2.5 to write the announcement.
- cpeterso 2y agoAnd it was trained on the previous announcements.
- xlbuttplug2 2y ago... which were also written by earlier Gemini versions.
- belter 2y agoLLMs all the way down
- cratermoon 2y agoNot all the way. At the bottom are a bunch of unpaid writers and artists and a horde of low-paid mturk workers in Nigeria.
- finebalance 2y agoThat's ok. AI will kill those off soon enough, and like all winners, rewrite history enough so that that inconvenient theft never happened anyway. It's manifest destiny, or something.
- throwaway2037 2y agoI love this comment. It made me laugh. > mturk workers in Nigeria Serious question: Has anyone tested how much money you can actually make doing a month of Amazon Mechanical Turk? (It would make for an interesting YouTube video!) I am curious if it is middle class wages in very poor countries (like Nigeria). Some light Googling tells me that middle class salary in Nigeria is about 6K USD, so about 3 USD/hour (assuming: 50 weeks/year * 40 hours/week = 2000 hours/year). Is this possible with MTurk?
- rvnx 2y agowhich was written by ChatGPT3.5
- andai 2y agoWell hey, OpenAI did the exact opposite, and nobody liked that either.
- Oras 2y agoHow did you measure “nobody” liked OpenAI announcements?
- exitb 2y agoI think people were fine with OpenAI demos. They were less fine with not actually ever releasing the demoed tech.
- emptysongglass 2y agoI liked their announcements and demos and continue to like them.
- andai 2y agoTo clarify, by "doing the opposite" I mean OpenAI releasing GPT-4.5, a non-reasoning model that does worse on benchmarks (but supposed to be qualitatively better). People shit on OpenAI hard for doing that.
- bhouston 2y ago> It might be an excellent model, but reading the exact text repeatedly is taking the excitement away. This is the commodification of models. There is nothing special about the new models but they perform better on the benchmarks. They are all interchangeable. This is great for users as it adds to price pressure.
- flir 2y agoMan, I hope those benchmarks actually measure something.
- Legend2440 2y agoI would say they are a fairly good measure of how well the model has integrated information from pretraining. They are not so good at measuring reasoning, out-of-domain performance, or creativity.
- Workaccount2 2y agoSooner or later someone is going to find "secret sauce" that provides a step-up in capability, and it will be closely guarded by whoever finds it. As big players look to start monetizing, they are going to desperately be searching for moats.
- bangaladore 2y agoReasoning was supposed to be that for "Open" AI, that's why they go to such lengths to hide the reasoning output. Look how that turned out. Right now, in my opinion, OpenAI has actually a useful deep research feature which I've found nobody else matches. But there is no moat to be seen there.
- CamperBob2 2y agoIf you've seen DeepSeek R1's <think> output, you'll understand why OpenAI hides their own. It can be pretty "unsafe" relative to their squeaky-clean public image.
- diego_sandoval 2y agoI take this as a good thing, because they're beating each other every few weeks and using benchmarks as evidence. If these companies start failing to beat the competition, then we should prepare ourselves for very creative writing in the announcements.
- gtirloni 2y agoThe improvements have been marginal at best. I wouldn't call that beating.
- devsda 2y agoReminds me of how nobody is too excited about flagship mobile launches anymore. Most flagships for sometime now are just incremental updates over previous gen and only marginally better. Couple that with the chinese OEMs launching better or good enough devices at a lower price point, new launches from established players are not noteworthy anymore. It's interesting how the recent AI announcements are following the same trend over a smaller timeframe.
- flakiness 2y agoPhones are limited by hardware manufacturing, plus maybe the annual shopping cycle peaking at Christmas. People won't have bought multiple iPhones even in its heyday. These LLM models were supposedly limited by the training run, but these point-version models are mostly post-training driven, which seems to be taking less time. If models were tied to a specific hardware (say, a "AI PC" or whatever) the cycle would get slower and we'll get a slower summer which I'm secretly wishing.
- breppp 2y agoI think the greatest issue with buying a new phone today is ironically the seamless migration. once you get all your apps, wallpaper, shortcut order and same OS, you really quickly get the feeling you spent 1000$ for the exact same thing
- atonse 2y ago100% agree with you. But it needs to be seamless to remove any friction from the purchase, but at the same time if it feels the same then we felt like we wasted money. So what I usually do is buy a different colored phone and change the wallpaper. My MacBook was the same. Seamless transition and 2 hours later I was used to the new m4 speeds.
- schainks 2y agoI wish I wish I wish Google put better marketing into these releases. I've moved entire workflows to Gemini because it's just _way_ better than what openai has to offer, especially for the money. Also, I think google's winning the race on actually integrating the AI to do useful things. The agent demo from OpenAI is interesting, but frankly, I don't care to watch the machine use my computer. A real virtual assistant can browse the web headless and pick flights or food for me. That's the real workflow unlock, IMO.
- throwaway2037 2y ago> I've moved entire workflows to Gemini because it's just _way_ better than what openai has to offer, especially for the money. This is useful feedback. I'm not here to shill for OpenAI, nor Google/Gemini, but can you share a concrete example? It would be interesting to hear more about your use case. More abstractly: Do you think these "moved entire workflows" offset a full worker, or X% of a full worker? I am curious to see how and when we will see low-end/junior knowledge workers displaced by solid LLMs. Listening to the Oxide and Friends podcast, I learned that they make pretty regular use of LLMs to create graphs using GNU plot. To paraphrase, they said "it is like have a good intern".
- schainks 2y ago> can you share a concrete example? Upload a complicated PDF of presentation and ask for insights that require some critical thinking about them. > Do you think these "moved entire workflows" offset a full worker, or X% of a full worker It can replace many junior analysts IMO.
- belter 2y agoCancelled my account long time ago. Gemini models are like a McDonalds Croissant. You always give them an extra chance, but they always fall apart on your hands...
- tibbar 2y agoFor me, the most exciting part is the improved long-context performance. A lot of enterprise/RAG applications rely on synthesizing a bunch of possibly relevant data. Let's just say it's clearly a bottleneck in current models and I would expect to see a meaningful % improvement in various internal applications if long-context reasoning is up. Gemini was already one of my favorite models for this usecase. So, I think these results are very interesting, if you know what features specifically you are using.
- zwaps 2y agoBut they score it on their own benchmark, on which coincidentally Gemini models always were the only good ones. In Nolima or Babilong we see that Gemini models still cant do long context. Excited to see if it works this time.
- taf2 2y agoWe have incrementally improved 1% better then we were yesterday. Our competition is 1 day behind us now.
- throwaway2037 2y agoLike! No trolling: This could be a sarcastic comment written by an LLM!
- kh_hk 2y agoWas going to comment the same thing, which has been bugging me off lately on all announcements that start with "our" followed by empty superlatives. Happy to not be alone on this!
- cratermoon 2y agoGlaringly missing from the announcements: concrete use cases and products. The Achilles heel of LLMs is the distinct lack of practical real-world applications. Yes, Google and Microsoft have been shoving the tech into everything they can fit, but that doesn't a product make.
- sebzim4500 2y agoChatGPT has like 500M weekly active users, what are you on about?
- cratermoon 2y ago"Well, Ed, there are 300 million weekly users of ChatGPT. That surely proves that this is a very real industry!" https://www.wheresyoured.at/longcon/ https://www.wheresyoured.at/longcon/
- sebzim4500 2y agoIs that article trying to argue that 500M people every week are visiting ChatGPT for the first (or second) time after reading about it in the news? If I'm being incredibly generous I will concede that this could have been the case for the first few weeks when it was making headlines, but it clearly isn't true now. It would be literally impossible to keep up these figures for as long as ChatGPT has without a ton of repeat users. There simply aren't enough people/devices.
- deleted 2y ago[deleted]
- kiratp 2y agohttps://www.osmos.io/fabric https://www.osmos.io/fabric Practical, real-world application.
- throwaway2037 2y agoI would say Adobe is doing an excellent job of commercialising image manipulation and generation using LLMs. When I see adverts for their new features, they seem genuinely useful for normie users who are trying to edit some family/holiday photos.
- cadamsdotcom 2y agoWhy not snooze the news for a year and see what’s been invented when you get back. That’ll blow your mind properly. Because each of these incremental announcements contributes to a mind blowing rate of improvement. The rate of announcements is a sign that models are increasing in ability at an amazing rate, and the content is broadly the same because they’re fungible commodities. The latter, that models are fungible commodities, is what’s driving this explosion and leading to intense competition that benefits us all.
- sagarpatil 2y agoAI labs, it seems, use a template for system cards as well. OpenAI stands out because they showcase their employees using their tools for various use cases, which is refreshing.