18 ms·
As of now yes. But we are still in day 0.1 of GenAI. Do you think this will be the case when o3 models are 10x better and 100x cheaper? There will be a turning
by digitcatphd 1y ago
As of now yes. But we are still in day 0.1 of GenAI. Do you think this will be the case when o3 models are 10x better and 100x cheaper? There will be a turning point but it’s not happened yet.
- nradov 1y ago10× better by what metric? Progress on LLMs has been amazing but already appears to be slowing down.
- jaggederest 1y agoAll these folks are once again seeing the first 1/4 of a sigmoid curve and extrapolating to infinity.
- drodgers 1y agoNo doubt from me that it’s a sigmoid, but how high is the plateau? That’s also hard to know from early in the process, but it would be surprising if there’s not a fair bit of progress left to go. Human brains seem like an existence proof for what’s possible, but it would be surprising if humans also represent the farthest physical limits of what’s technologically possible without the constraints of biology (hip size, energy budget etc).
- leoedin 1y agoBiological muscles are proof that you can make incredibly small and forceful actuators. But the state of robotics is nowhere near them, because the fundamental construction of every robotic actuator is completely different. We’ve been building actuators for 100s of years and we still haven’t got anything comparable to a muscle. And even if you build a better hydraulic ram or brushless motor driven linear actuator you will still never achieve the same kind of behaviour, because the technologies are fundamentally different. I don’t know where the ceiling of LLM performance will be, but as the building blocks are fundamentally different to those of biological computers, it seems unlikely that the limits will be in any way linked to those of the human brain. In much the same way the best hydraulic ram has completely different qualities to a human arm. In some dimensions it’s many orders of magnitudes better, but in others it’s much much worse.
- lazide 1y agoBiological muscles come with a lot of baggage, very constrained operating environments, and limited endurance. It’s not just that ‘we don’t know how to build them’, it’s that the actuators aren’t a standalone part - and we don’t know how to build (or maintain/run in industrial enviroments!) the ‘other stuff’ economically either.
- audunw 1y agoI don’t think it’s hard to know. We’re already seeing several signs of being near the plateau in terms of capabilities. Most big breakthrough these days seems to be in areas where we haven’t spent the effort in training and model engineering. Like recent improvements in video generation. So of course we could get improvements in areas where we haven’t tried to use ML yet. For text generation, it seems like the fast progress was mainly due to feeding the models exponentially more data and exponentially more compute power. But we know that the growth in data is over. The growth in compute has a shifted from a steep curve (just buy more chips) to a slow curve (have to make exponentially more factories if we want exponentially more chips) Im sure we will have big improvements in efficiency. Im sure nearly everyone will use good LLMs to support them in their work, and they may even be able to do all they need to do on-device. But that doesn’t make the models significantly smarter.
- jaggederest 1y agoThe wonderful thing about a sigmoid is that, just as it seems like it's going exponential, it goes back to linear. So I'd guess we're not going to see 1000x from here - I could be wrong, but I think the low hanging fruit has been picked. I would be surprised in 10 years if AI were 100x better than it is now (per watt, maybe, since energy devoted to computing is essentially the limiting factor) The thing about the latter 1/3rd of a sigmoid curve is, you're still making good progress, it's just not easy any more. The returns have begun to diminish, and I do think you could argue that's already happening for LLMs.
- GoblinSlayer 1y agoHuman brains are easy to do, just run evolution for neural networks.
- formerly_proven 1y agoProgress so far has been half and half technique and brute force. Overall technique has now settled for a few years, so that's mostly in the tweaking phase. Brute force doesn't scale by itself and semiconductors have been running into a wall for the last few years. Those (plus stagnating outcomes) seem decent reasons to suspect the plateau is neigh.
- elif 1y agowith autonomous vehicles, the narrative of imperceptibly slow incremental change about chasing 9's is still the zeitgeist despite an actual 10x improvement in homicidality compared to humans already existing. There is a lag in how humans are reacting to AI which is probably a reflexive aspect of human nature. There are so many strategies being employed to minimize progress in a technology which 3 years ago did not exist and now represents a frontier of countless individual disciplines.
- intended 1y agoThis is my favorite thing to point out from the day we started talking about autonomous vehicles on tech sites. If you took a Tesla or a Waymo and dropped into into a tier 2 city in India, it will stop moving. Driving data is cultural data, not data about pure physics. You will never get to full self driving, even with more processing power, because the underlying assumptions are incorrect. Doing more of the same thing, will not achieve the stated goal of full self driving. You would need to have something like networked driving, or government supported networks of driving information, to deal with the cultural factor. Same with GenAI - the tooling factor will not magically solve the people, process, power and economic factors.
- binoct 1y agoOne of my favorite things to question about autonomous driving is the goalposts. What do you mean the “stated goal of full self driving”, which is unachievable? Any vehicle, anywhere in the world, in any conditions? That seems an absurd goal that ignores the very real value in having vehicles that do not require drivers and are safer than humans but are limited to certain regions. Absolutely driving is cultural (all things people do are cultural) but given 10’s of millions of miles driven by Waymo, clearly it has managed the cultural factor in the places they have been deployed. Modern autonomous driving is about how people drive far more than the rules of the road, even on the highly regulated streets of western countries. Absolutely the constraints of driving in Chennai are different, but what is fundamentally different? What leads to an impossible leap in processing power to operate there?
- nothercastle 1y agoI think they will be 10-100x cheaper id be really surprised if we even doubled the quality though
- makeitdouble 1y agoHow does it work if they get 10x better in 10 years ? Everything else will have already moved on and the actual technology shift will come from elsewhere. Basically, what if GenAI is the Minitel and what we want is the internet.
- directevolve 1y agoWe’re already heading toward the sigmoid plateau. The GPT 3 to 4 shift was massive. Nothing since had touched that. I could easily go back to the models I was using 1-2 years ago with little impact on my work. I don’t use RAG, and have no doubt the infrastructure for integrating AI into a large codebase has improved. But the base model powering the whole operation seems stuck.
- threeseed 1y ago> I don’t use RAG, and have no doubt the infrastructure for integrating AI into a large codebase has improved It really hasn't. The problem is that a GenAI system needs to not only understand the large codebase but also the latest stable version of every transitive dependency it depends on. Which is typically in the order of hundreds or thousands. Having it build a component with 10 year old, deprecated, CVE-riddled libraries is of limited use especially when libraries tend to be upgraded in interconnected waves. And so that component will likely not even work anyway. I was assured that MCP was going to solve all of this but nope.
- HumanOstrich 1y agoHow did you think MCP was going to solve the issue of a large number of outdated dependencies?
- threeseed 1y agoThose large number of outdated dependencies are in the LLM "index" which can't be rapidly refreshed because of the training costs. MCP would allow it to instead get this information at run-time from language servers, dependency repositories etc. But it hasn't proven to be effective.
- chrsw 1y ago> I could easily go back to the models I was using 1-2 years ago with little impact on my work. I can't. GPT-4 was useless for me for software development. Claude 4 is not.
- apwell23 1y ago> Do you think this will be the case when o3 models are 10x better and 100x cheaper? why don't you bring it up then. > There will be a turning point but it’s not happened yet. do you know something that rest of us don't ?
- ricardobayes 1y agoFrankly, we don't know. That "turning point" that seemed so close for many tech, never came for some of them. Think 3D-printing that was supposed to take over manufacturing. Or self-driving, that is "just around the corner" for a decade now. And still is probably a decade away. Only time will tell if GenAI/LLMs are color TV or 3D TV.
- kergonath 1y ago> Think 3D-printing that was supposed to take over manufacturing. 3D printing is making huge progress in heavy industries. It’s not sexy and does not make headlines but it absolutely is happening. It won’t replace traditional manufacturing at huge scales (either large pieces or very high throughput). But it’s bringing costs way down for fiddly parts or replacements. It is also affecting designs, which can be made simpler by using complex pieces that cannot be produced otherwise. It is not taking over, because it is not a silver bullet, but it is now indispensable in several industries.
- godelski 1y agoYou're misunderstanding the parent's complaint and frankly the complaints with AI. Certainly 3D printing is powerful and hasn't changed things. But you forgot that 30 years ago people were saying there would be one in every house because a printer can print a printer and how this would revolutionize everything because you could just print anything at home. The same thing with AI. You'd be blind or lying if you said it hasn't advanced a lot. People aren't denying that. But people are fed up being constantly being promised the moon and getting a cheap plastic replica instead. The tech is rapidly advancing and doing good. But it just can't keep up with the bubble of hype. That's the problem. The hype, not the tech. Frankly, the hype harms the tech too. We can't solve problems with the tech if we're just throwing most of our money at vaporware. I'm upset with the hype BECAUSE I like the tech. So don't confuse the difference. Make sure you understand what you're arguing against. Because it sounds like we should be on the same team, not arguing against one another. That just helps the people selling vaporware
- Ray20 1y ago>Think 3D-printing that was supposed to take over manufacturing This was never the case, and this is obvious to anyone who has ever been to factories that doing mass-produced plastic >Or self-driving, that is "just around the corner" for a decade now. But it is really around the corner, all that remains is to accept it. That is, to start building and modifying the road infrastructure and changing the traffic rules to enable effective integration self-driving cars into road traffic.
- threeseed 1y agoHow are we in 0.1 of GenAI ? It's been developed for nearly a decade now. And each successive model that has been released has done nothing to fundamentally change the use cases that the technology can be applied to i.e. those which are tolerant of a large percentage of incoherent mistakes. Which isn't all that many. So you can keep your 10x better and 100x cheaper models because they are of limited usefulness let alone being a turning point for anything.
- Flemlo 1y agoA decade? The explosion of funding, awareness etc only happened after gpt-3 launch
- hyperadvanced 1y agoFunding is behind the curve. Social networks existed in 2003 and Facebook became a billion dollar company a decade later. AI horror fantasies from the 90’s still haven’t come true. There is no god, there is no Skynet.
- imtringued 1y agoThat was five years ago not yesterday.
- Flemlo 1y agoI didn't say yesterday. Nonetheless it took openai til Nov 2022 for 1 Million users. The overall awareness and breakthrough was probably not at 2020.
- sbm_au 1y agoAlphaGo beating the top human player was in 2016. To my memory, that was one of the first public breakthroughs of the new era of machine learning. Around 2010 when I was at university, a friend did their undergraduate thesis on neural networks. Among our cohort it was seen as a weird choice and a bit of a dead-end from the last AI winter.
- godelski 1y agoYet we're what? 5 years into "AI will replace programmers in 6 months"? 10 years into "we'll have self driving cars next year" We're 10 years into "it's just completely obvious that within 5 years deep learning is going to replace radiologists" Moravec's paradox strikes again and again. But this time it's different and it's completely obvious now, right?
- tsunamifury 1y ago[flagged]
- godelski 1y agoI named 3 things... You're going to have to specify which 2 you think happened
- croes 1y agoWhere did it happen? They try it, but it’s not reliable
- seanhunter 1y agoI consulted a radiologist more than 5 years after Hinton said that it was completely obvious that radiologists would be replaced by AI in 5 years. I strongly suspect they were not an AI. Why do I think this? 1) They smelled slightly funny. 2) They got the diagnosis wrong. OK maybe #2 is a red herring. But I stand by the other reason.
- HDThoreaun 1y agoI know a radiologist and talk a decent bit about AI usage in the field. Every radiologist today is making heavy use of AI. They pre screen everything and from what I understand it has led to massive productivity gains. It hasnt led to job losses yet but theres so much money on the line it really feels to me like we're just waiting for the straw that broke the camels back. No one wants to be the first to fully get rid of radiologists but once one hospital does the rest will quickly follow suit.
- croes 1y agoIf not when.
- solumunus 1y agoI use LLM’s daily and live them but at the current rate of progress it’s just not really something worth worrying about. Those that are hysterical about AI seem to think LLM’s are getting exponentially better when in fact diminishing returns are hitting hard. Could some new innovation change that? It’s possible but it’s not inevitable or at least not necessarily imminent.
- kbelder 1y agoI agree that the core models are only going to see slow progression from here on out, until something revolutionary happens... which might be a year from now, or maybe twenty years. Who knows. But we are going to see a huge explosion in how those models are integrated into the rest of the tech ecosystem. Things that a current model could do right now, if only your car/watch/videogame/heart monitor/stuffed animal had a good working interface into an AI. Not necessarily looking forward to that, but that's where the growth will come.
- AvAn12 1y agoRemember when RPA was going to replace everyone?
- AvAn12 1y agoOr low-code / no-code?
- johnnyanmac 1y agoThere's a lot of "when" people are betting on, and not a lot of action to back it. If "when" is 20 years, then I still got plenty career ahead of me before I need to worry about that.
- xnx 1y ago> 5 years into "AI will replace programmers in 6 months"? Programmers that don't use AI will get replaced by those that do. (no just by mandate, but by performance) > 10 years into "we'll have self driving cars next year" They're here now. Waymo does 250K paid rides/week.
- player1234 1y agoHow have you measured this performance boost?