4 ms·

> It does sort of give me the vibe that the pure scaling maximalism really is dying off though I think the big question is if/when investors will start giving

> It does sort of give me the vibe that the pure scaling maximalism really is dying off though I think the big question is if/when investors will start giving money to those who have been predicting this (with evidence) and trying other avenues. Really though, why put all your eggs in one basket? That's what I've been confused about for awhile. Why fund yet another LLMs to AGI startup. Space is saturated with big players and has been for years. Even if LLMs could get there that doesn't mean something else won't get there faster and for less. It also seems you'd want a backup in order to avoid popping the bubble. Technology S-Curves and all that still apply to AI Though I'm similarly biased, but so is everyone I know with a strong math and/or science background (I even mentioned it in my thesis more than a few times lol). Scaling is all you need just doesn't check out

morleytj 1y ago
I'm pretty curious about the same thing. I think a somewhat comparable situation is in various online game platforms now that I think about it. Investors would love to make a game like Fortnite, and get the profits that Fortnite makes. So a ton of companies try to make Fortnite. Almost all fail, and make no return whatsoever, just lose a ton of money and toss the game in the bin, shut down the servers. On the other hand, it may have been more logical for many of them to go for a less ambitious (not always online, not a game that requires a high player count and social buy-in to stay relevant) but still profitable investment (Maybe a smaller scale single player game that doesn't offer recurring revenue), yet we still see a very crowded space for trying to emulate the same business model as something like Fortnite. Another more historical example was the constant question of whether a given MMO would be the next "WoW-killer" all through the 2000's/2010's. I think part of why this arises is that there's definitely a bit of a psychological hack for humans in particular where if there's a low-probability but extremely high reward outcome, we're deeply entranced by it, and investors are the same. Even if the chances are smaller in their minds than they were before, if they can just follow the same path that seems to be working to some extent and then get lucky, they're completely set. They're not really thinking about any broader bubble that could exist, that's on the level of the society, they're thinking about the individual, who could be very very rich, famous, and powerful if their investment works. And in the mind of someone debating what path to go down, I imagine a more nebulous answer of "we probably need to come up with some fundamentally different tools for learning and research a lot of different approaches to do so" is a bit less satisfying and exciting than a pitch that says "If you just give me enough money, the curve will eventually hit the point where you get to be king of the universe and we go colonize the solar system and carve your face into the moon." I also have to acknowledge the possibility that they just have access to different information than I do! They might be getting shown much better demos than I do, I suppose.
- eru 1y ago
  We see both things: almost all games are 'not fortnite'. But that doesn't (commercially) invalidate some companies' quest for building the next fortnite. Of course, if you limit your attention to these 'wanabe fortnites', then you only see these 'wannabe fortnites'.
- godelski 1y ago
  I'm pretty sure the answer is people buying into the scaling is all you need argument. Because if you have that framing then it can be solved through engineering, right? I mean there's still engineering research and it doesn't mean there's no reason to research but everyone loves the simple and straight forward path, right? > I think a somewhat comparable situation is in various online game platforms I think it is common in many industries. The weird thing is that being too risk adverse creates more risk. There's a balance that needs to be struck. Maybe another famous one is movies. They go on about pirating and how Netflix is winning but most of the new movies are rehashes or sequels. Sure, there's a lot of new movies, but few get nearly the same advertising budgets and so people don't even hear about it (and sequels need less advertising since there's a lot of free advertising). You'd think there'd be more pressure to find the next hit that can lead to a few sequels but instead they tend to be too risk adverse. That's the issue of monopolies though... or any industry where the barrier to entry is high... > psychological hack While I'm pretty sure this plays a role (along with other things like blind hope) I think the bigger contributor is risk aversion and observation bias. Like you say, it's always easier to argue "look, it worked for them" then "this hasn't been done before, but could be huge." A big part of the bias is that you get to oversimplify the reasoning for the former argument compared to the latter. The latter you'll get highly scrutinized while the former will overlook many of the conditions that led to success. You're right that the big picture is missing. Especially that a big part of the success was through the novelty (not exactly saying Fortnite is novel via gameplay...). For some reason the success of novelty is almost never seen as motivation to try new things. I think that's the part that I find most interesting and confusing. It's like an aversion of wanting to look just one layer deeper. We'll put in far more physical and mental energy to justify a shallow thought than what would be required to think deeper. I get we're biased towards being lazy, so I think this is kinda related to us just being bad at foresight and feeling like being wrong is a bad thing (well it isn't good, but I'm pretty sure being wrong and not correcting is worse than just being wrong).
famouswaffles 1y ago
>Really though, why put all your eggs in one basket? That's what I've been confused about for awhile. I mean that's easy lol. People don't like to invest in thin air, which is what you get when you look at non-LLM alternatives to General Intelligence. This isn't meant as a jab or snide remark or anything like that. There's literally nothing else that will get you GPT-2 level performance, never-mind an IMO Gold Medalist. Invest in what else exactly? People are putting their eggs in one basket because it's the only basket that exists. >I think the big question is if/when investors will start giving money to those who have been predicting this (with evidence) and trying other avenues. Because those people have still not been proven right. Does "It's an incremental improvement over the model we released a few months ago, and blows away the model we released 2 years ago." really scream, "See!, those people were wrong all along!" to you ?
- godelski 1y ago
  > which is what you get when you look at non-LLM alternatives to General Intelligence. I disagree with this. There are a good ideas that are worth pursuit. I'll give you that few, if any, have been shown to work at scale but I'd say that's a self-fulfilling prophecy. If your bar is that they have to be proven at scale then your bar is that to get investment you'd have to have enough money to not need investment. How do you compete if you're never given the opportunity to compete? You could be the greatest quarterback in the world but if no one will let you play in the NFL then how can you prove that? On the other hand, investing in these alternatives is a lot cheaper, since you can work your way to scale and see what fails along the way. This is more like letting people try their stuff out in lower leagues. The problem is there's no ladder to climb after a certain point. If you can't fly then how do you get higher? > Invest in what else exactly? ... it's the only basket that exists. I assume you don't work in ML research? I mean that's okay but I'd suspect that this claim would come from someone not on the inside. Though tbf, there's a lot of ML research that is higher level and not working on alternative architectures. I guess the two most well known are Mamba and Flows. I think those would be known by the general HN crowd. While I think neither will get us to AGI I think both have advantages that shouldn't be ignored. Hell, even scaling a very naive Normalizing Flow (related to Flow Matching) has been shown to compete and beat top diffusion models[0,1]. The architectures aren't super novel here but they do represent the first time a NF was trained above 200M params. That's a laughable number by today's standards. I can even tell you from experience that there's a self-fulfilling filtering for this kind of stuff because having submitted works in this domain I'm always asked to compare with models >10x my size. Even if I beat them on some datasets people will still point to the larger model as if that's a fair comparison (as if a benchmark is all the matters and doesn't need be contextualized). > Because those people have still not been proven right. You're right. But here's the thing. *NO ONE HAS BEEN PROVEN RIGHT*. That condition will not exist until we get AGI. > scream, "See!, those people were wrong all along!" to you ? Let me ask you this. Suppose people are saying "x is wrong, I think we should do y instead" but you don't get funding because x is currently leading. Then a few years later y is proven to be the better way of doing things, everything shifts that way. Do you think the people who said y was right get funding or do you think people who were doing x but then just switched to y after the fact get funding? We have a lot of history to tell us the most common answer... [0] https://arxiv.org/abs/2412.06329 https://arxiv.org/abs/2412.06329 [1] https://arxiv.org/abs/2506.06276 https://arxiv.org/abs/2506.06276
eru 1y ago
> Really though, why put all your eggs in one basket? That's what I've been confused about for awhile. Why fund yet another LLMs to AGI startup. Funding multiple startups means _not_ putting your eggs in one basket, doesn't it? Btw, do we have any indication that eg OpenAI is restricting themselves to LLMs?
- godelski 1y ago
  > Funding multiple startups means _not_ putting your eggs in one basket, doesn't it? Different basket hierarchy. Also, yes. They state this and given how there are plenty of open source models that are LLMs and get competitive performance it at least indicates that anyone not doing LLMs is doing so in secret. If OpenAI isn't using LLMs then doesn't that support my argument?
l33tman 1y ago
I started such an alternative project just before GPT-3 was released, it was really promising (lots of neuroscience inspired solutions, pretty different to Transformers) but I had to put it on hold because the investors I approached seemed like they would only invest in LLM-stuff. Now a few years later I'm trying to approach investors again, only to find now they want to invest in companies USING LLMs to create value and still don't seem interested in new foundational types of models... :/ I guess it makes sense, there is still tons of value to be created just by using the current LLMs for stuff, though maybe the low hanging fruits are already picked, who knows. I heard John Carmack talk a lot about his alternative (also neuroscience-inspired) ideas and it sounded just like my project, the main difference being that he's able to self-fund :) I guess funding an "outsider" non-LLM AI project now requires finding someone like Carmack to get on board - I still don't think traditional investors are that disappointed yet that they want to risk money on other types of projects..
- godelski 1y ago
  > I guess funding an "outsider" non-LLM AI project now requires finding someone like Carmack to get on board And I think this is a big problem. Especially since these investments tend to be a lot cheaper than the existing ones. Hell, there's stuff in my PhD I tabled and several models I made that I'm confident I could have doubled performance with less than a million dollars worth of compute. My methods could already compete while requiring less compute, so why not give them a chance to scale? I've seen this happen to hundreds of methods. If "scale is all you need" then shouldn't the belief that any of those methods would also scale?
- andrepd 1y ago
  Markets are very good and squeezing profit out of sometimes ludicrous ventures, not good at all for foundational research. It's the problem of having organised our economic life in this way, or rather, exclusively this way.
  - godelski 1y ago
    > exclusively this way. I think an important part is to recognize that fundamental research is extremely foundational. We often don't recognize the impacts because by the time we see them they have passed through other layers. Maybe in the same way that we forget about the ground existing and being the biggest contributor to Usain Bolt's speed. Can't run if there's no ground. But to see economic impact, I'll make the bet that a single mathematical work (technically two) had a greater economic impact than all technologies in the last 2 centuries. Calculus. I haven't run the calculations (seems like it'd be hard and I'd definitely need calculus to do them), but I'd be willing to bet that every year Calculus results in a greater economic impact than FAANG, MANGO, or whatever the hot term is these days, does. It seems almost silly to say this and it is obviously so influential. But things like this fade away into the background the same way we almost never think about the ground beneath our feet. I have to say this because we're living in a time where people are arguing we shouldn't build roads because cars are the things that get us places. But this is just framing, and poor framing at that. Frankly, part of it is that roads are typically built through public funds and cars through private. It's this way because the road is able to make much higher economic impact by being a public utility rather than a private one. Incidentally, this makes the argument to not build roads self-destructive. It's short sighted. Just like actual roads, research has to be continuously performed. The reality is more akin to those cartoon scenes where a character is laying down the railroad tracks just in front of the speeding train.[0] I guess if you're not Gromit placing down the tracks it is easy to assume they just exist. But unlike actual roads, research is relatively cheap. Sure, maybe a million mathematicians don't produce anything economically viable for that year and maybe not 20, but one will produce something worth trillions. And they do this at a mathematician's salary! You can hire research mathematicians at least 2 to 1 for a junior SWE. 10 to 1 for a staff SWE. It's just crazy to me that we're arguing we don't have the money for these kinds of things. I mean just look at the impact of Tim Berners Lee and his team. That alone offsets all costs for the foreseeable future. Yet somehow his net worth is in the low millions? I think we really should question this notion that wealth is strongly correlated to impact. [0] Why does a 10hr version of this exist... https://www.youtube.com/watch?v=fwJHNw9jU_U https://www.youtube.com/watch?v=fwJHNw9jU_U
csomar 1y ago
The current money made its money following the market. They do not have the capacity for innovation or risk taking.

morleytj 1y ago

I'm pretty curious about the same thing. I think a somewhat comparable situation is in various online game platforms now that I think about it. Investors would love to make a game like Fortnite, and get the profits that Fortnite makes. So a ton of companies try to make Fortnite. Almost all fail, and make no return whatsoever, just lose a ton of money and toss the game in the bin, shut down the servers. On the other hand, it may have been more logical for many of them to go for a less ambitious (not always online, not a game that requires a high player count and social buy-in to stay relevant) but still profitable investment (Maybe a smaller scale single player game that doesn't offer recurring revenue), yet we still see a very crowded space for trying to emulate the same business model as something like Fortnite. Another more historical example was the constant question of whether a given MMO would be the next "WoW-killer" all through the 2000's/2010's. I think part of why this arises is that there's definitely a bit of a psychological hack for humans in particular where if there's a low-probability but extremely high reward outcome, we're deeply entranced by it, and investors are the same. Even if the chances are smaller in their minds than they were before, if they can just follow the same path that seems to be working to some extent and then get lucky, they're completely set. They're not really thinking about any broader bubble that could exist, that's on the level of the society, they're thinking about the individual, who could be very very rich, famous, and powerful if their investment works. And in the mind of someone debating what path to go down, I imagine a more nebulous answer of "we probably need to come up with some fundamentally different tools for learning and research a lot of different approaches to do so" is a bit less satisfying and exciting than a pitch that says "If you just give me enough money, the curve will eventually hit the point where you get to be king of the universe and we go colonize the solar system and carve your face into the moon." I also have to acknowledge the possibility that they just have access to different information than I do! They might be getting shown much better demos than I do, I suppose.

eru 1y ago
We see both things: almost all games are 'not fortnite'. But that doesn't (commercially) invalidate some companies' quest for building the next fortnite. Of course, if you limit your attention to these 'wanabe fortnites', then you only see these 'wannabe fortnites'.
godelski 1y ago
I'm pretty sure the answer is people buying into the scaling is all you need argument. Because if you have that framing then it can be solved through engineering, right? I mean there's still engineering research and it doesn't mean there's no reason to research but everyone loves the simple and straight forward path, right? > I think a somewhat comparable situation is in various online game platforms I think it is common in many industries. The weird thing is that being too risk adverse creates more risk. There's a balance that needs to be struck. Maybe another famous one is movies. They go on about pirating and how Netflix is winning but most of the new movies are rehashes or sequels. Sure, there's a lot of new movies, but few get nearly the same advertising budgets and so people don't even hear about it (and sequels need less advertising since there's a lot of free advertising). You'd think there'd be more pressure to find the next hit that can lead to a few sequels but instead they tend to be too risk adverse. That's the issue of monopolies though... or any industry where the barrier to entry is high... > psychological hack While I'm pretty sure this plays a role (along with other things like blind hope) I think the bigger contributor is risk aversion and observation bias. Like you say, it's always easier to argue "look, it worked for them" then "this hasn't been done before, but could be huge." A big part of the bias is that you get to oversimplify the reasoning for the former argument compared to the latter. The latter you'll get highly scrutinized while the former will overlook many of the conditions that led to success. You're right that the big picture is missing. Especially that a big part of the success was through the novelty (not exactly saying Fortnite is novel via gameplay...). For some reason the success of novelty is almost never seen as motivation to try new things. I think that's the part that I find most interesting and confusing. It's like an aversion of wanting to look just one layer deeper. We'll put in far more physical and mental energy to justify a shallow thought than what would be required to think deeper. I get we're biased towards being lazy, so I think this is kinda related to us just being bad at foresight and feeling like being wrong is a bad thing (well it isn't good, but I'm pretty sure being wrong and not correcting is worse than just being wrong).

famouswaffles 1y ago

>Really though, why put all your eggs in one basket? That's what I've been confused about for awhile. I mean that's easy lol. People don't like to invest in thin air, which is what you get when you look at non-LLM alternatives to General Intelligence. This isn't meant as a jab or snide remark or anything like that. There's literally nothing else that will get you GPT-2 level performance, never-mind an IMO Gold Medalist. Invest in what else exactly? People are putting their eggs in one basket because it's the only basket that exists. >I think the big question is if/when investors will start giving money to those who have been predicting this (with evidence) and trying other avenues. Because those people have still not been proven right. Does "It's an incremental improvement over the model we released a few months ago, and blows away the model we released 2 years ago." really scream, "See!, those people were wrong all along!" to you ?

godelski 1y ago
> which is what you get when you look at non-LLM alternatives to General Intelligence. I disagree with this. There are a good ideas that are worth pursuit. I'll give you that few, if any, have been shown to work at scale but I'd say that's a self-fulfilling prophecy. If your bar is that they have to be proven at scale then your bar is that to get investment you'd have to have enough money to not need investment. How do you compete if you're never given the opportunity to compete? You could be the greatest quarterback in the world but if no one will let you play in the NFL then how can you prove that? On the other hand, investing in these alternatives is a lot cheaper, since you can work your way to scale and see what fails along the way. This is more like letting people try their stuff out in lower leagues. The problem is there's no ladder to climb after a certain point. If you can't fly then how do you get higher? > Invest in what else exactly? ... it's the only basket that exists. I assume you don't work in ML research? I mean that's okay but I'd suspect that this claim would come from someone not on the inside. Though tbf, there's a lot of ML research that is higher level and not working on alternative architectures. I guess the two most well known are Mamba and Flows. I think those would be known by the general HN crowd. While I think neither will get us to AGI I think both have advantages that shouldn't be ignored. Hell, even scaling a very naive Normalizing Flow (related to Flow Matching) has been shown to compete and beat top diffusion models[0,1]. The architectures aren't super novel here but they do represent the first time a NF was trained above 200M params. That's a laughable number by today's standards. I can even tell you from experience that there's a self-fulfilling filtering for this kind of stuff because having submitted works in this domain I'm always asked to compare with models >10x my size. Even if I beat them on some datasets people will still point to the larger model as if that's a fair comparison (as if a benchmark is all the matters and doesn't need be contextualized). > Because those people have still not been proven right. You're right. But here's the thing. *NO ONE HAS BEEN PROVEN RIGHT*. That condition will not exist until we get AGI. > scream, "See!, those people were wrong all along!" to you ? Let me ask you this. Suppose people are saying "x is wrong, I think we should do y instead" but you don't get funding because x is currently leading. Then a few years later y is proven to be the better way of doing things, everything shifts that way. Do you think the people who said y was right get funding or do you think people who were doing x but then just switched to y after the fact get funding? We have a lot of history to tell us the most common answer... [0] https://arxiv.org/abs/2412.06329 https://arxiv.org/abs/2412.06329 [1] https://arxiv.org/abs/2506.06276 https://arxiv.org/abs/2506.06276

eru 1y ago

> Really though, why put all your eggs in one basket? That's what I've been confused about for awhile. Why fund yet another LLMs to AGI startup. Funding multiple startups means _not_ putting your eggs in one basket, doesn't it? Btw, do we have any indication that eg OpenAI is restricting themselves to LLMs?

godelski 1y ago
> Funding multiple startups means _not_ putting your eggs in one basket, doesn't it? Different basket hierarchy. Also, yes. They state this and given how there are plenty of open source models that are LLMs and get competitive performance it at least indicates that anyone not doing LLMs is doing so in secret. If OpenAI isn't using LLMs then doesn't that support my argument?

l33tman 1y ago

I started such an alternative project just before GPT-3 was released, it was really promising (lots of neuroscience inspired solutions, pretty different to Transformers) but I had to put it on hold because the investors I approached seemed like they would only invest in LLM-stuff. Now a few years later I'm trying to approach investors again, only to find now they want to invest in companies USING LLMs to create value and still don't seem interested in new foundational types of models... :/ I guess it makes sense, there is still tons of value to be created just by using the current LLMs for stuff, though maybe the low hanging fruits are already picked, who knows. I heard John Carmack talk a lot about his alternative (also neuroscience-inspired) ideas and it sounded just like my project, the main difference being that he's able to self-fund :) I guess funding an "outsider" non-LLM AI project now requires finding someone like Carmack to get on board - I still don't think traditional investors are that disappointed yet that they want to risk money on other types of projects..

godelski 1y ago
> I guess funding an "outsider" non-LLM AI project now requires finding someone like Carmack to get on board And I think this is a big problem. Especially since these investments tend to be a lot cheaper than the existing ones. Hell, there's stuff in my PhD I tabled and several models I made that I'm confident I could have doubled performance with less than a million dollars worth of compute. My methods could already compete while requiring less compute, so why not give them a chance to scale? I've seen this happen to hundreds of methods. If "scale is all you need" then shouldn't the belief that any of those methods would also scale?
andrepd 1y ago
Markets are very good and squeezing profit out of sometimes ludicrous ventures, not good at all for foundational research. It's the problem of having organised our economic life in this way, or rather, exclusively this way.
- godelski 1y ago
  > exclusively this way. I think an important part is to recognize that fundamental research is extremely foundational. We often don't recognize the impacts because by the time we see them they have passed through other layers. Maybe in the same way that we forget about the ground existing and being the biggest contributor to Usain Bolt's speed. Can't run if there's no ground. But to see economic impact, I'll make the bet that a single mathematical work (technically two) had a greater economic impact than all technologies in the last 2 centuries. Calculus. I haven't run the calculations (seems like it'd be hard and I'd definitely need calculus to do them), but I'd be willing to bet that every year Calculus results in a greater economic impact than FAANG, MANGO, or whatever the hot term is these days, does. It seems almost silly to say this and it is obviously so influential. But things like this fade away into the background the same way we almost never think about the ground beneath our feet. I have to say this because we're living in a time where people are arguing we shouldn't build roads because cars are the things that get us places. But this is just framing, and poor framing at that. Frankly, part of it is that roads are typically built through public funds and cars through private. It's this way because the road is able to make much higher economic impact by being a public utility rather than a private one. Incidentally, this makes the argument to not build roads self-destructive. It's short sighted. Just like actual roads, research has to be continuously performed. The reality is more akin to those cartoon scenes where a character is laying down the railroad tracks just in front of the speeding train.[0] I guess if you're not Gromit placing down the tracks it is easy to assume they just exist. But unlike actual roads, research is relatively cheap. Sure, maybe a million mathematicians don't produce anything economically viable for that year and maybe not 20, but one will produce something worth trillions. And they do this at a mathematician's salary! You can hire research mathematicians at least 2 to 1 for a junior SWE. 10 to 1 for a staff SWE. It's just crazy to me that we're arguing we don't have the money for these kinds of things. I mean just look at the impact of Tim Berners Lee and his team. That alone offsets all costs for the foreseeable future. Yet somehow his net worth is in the low millions? I think we really should question this notion that wealth is strongly correlated to impact. [0] Why does a 10hr version of this exist... https://www.youtube.com/watch?v=fwJHNw9jU_U https://www.youtube.com/watch?v=fwJHNw9jU_U

csomar 1y ago

The current money made its money following the market. They do not have the capacity for innovation or risk taking.