6 ms·

There’s a shocking lack of introspection in the AI hype bubble — we are continually promised world-changing advances, but no one is very clear about which direc

There’s a shocking lack of introspection in the AI hype bubble — we are continually promised world-changing advances, but no one is very clear about which direction these advances are pushing us in. Even ‘AGI’ is nebulous and undefined at best. Where is the focus on the foundational principles of growth? The author writes that endless growth is impossible, but that is only true in the absence of novel breakthroughs. We need creations and inventions which bring us the future we once imagined: limitless energy and abundance. How have we completely lost focus on new physics or foundational sciences and devoted the smartest minds of our time to transformers and data scraping? There is some critical human component which is completely absent here. Where is the renaissance, and how do we make it happen?

dotnet00 3y ago
I think part of the issue is that a lot of the 'big' advances in physics nowadays aren't constrained by brain power as much as they are by cost (or rather, money allocation), and due to the corruption of the scientific funding system in most rich countries, waste is extremely high, severely limiting the money actually going into science. Eg to make radical progress in physics, we need more sensitive telescopes, larger particle accelerators, more efficient space missions, more actual iteration on fusion reactors etc. All these things cost a lot of money, yet the majority of it gets eaten by systems designed to minimize the money actually going towards the research work. This isn't even really getting into how the researchers are paid. I agree that AI researchers are concerningly blasé about what they're actually aiming to do. AGI is poorly defined, and the only negative externalities they seem to pay lip service to is culture war stuff. There's near zero consideration for the real negative externalities they're perpetrating upon the world. I think partly this also manifests in how they interact with fields where even older AI developments would be useful. At the physics lab I work at, I've found that many scientists have a negative impression of AI 'researchers', where they don't care to understand the problem they're trying to solve, preferring to just treat everything - even things where the solution space can be constrained by scientific understanding - as a black box. It's almost the opposite of science in approach.
- sebmellen 3y ago
  It’s still surprising to me that not one decabillionaire has funded a good basic sciences institute. With a few hundred million dollars (paid primarily to salaries) and an extremely rigorous selection process, one could establish a highly prestigious research university and bring the brightest minds together in one place. Just go from there! If Leland Stanford could do it…
  - rsanek 3y ago
    https://arcinstitute.org/ https://arcinstitute.org/
    - sebmellen 3y ago
      Arc is a cool concept, but these initiatives are ultimately handicapped by their reliance on the infrastructure of "grants". Grants suck. What matters is having smart people together in one place. If you pay smart people enough money, and offer them enough insulation and protection from academic politics, they will come to your institute. Grant-making to people at institutions which are already politically and socially captured does not fix the root issue.
  - timeforcomputer 3y ago
    Not sure if relevant as it is not just one institute but the Simons Foundation (https://en.wikipedia.org/wiki/Simons_Foundation https://en.wikipedia.org/wiki/Simons_Foundation) seems along this line.
- timeforcomputer 3y ago
  I am studying physics now (with a CS and math background) and I feel obligated to get up to date on AI and to develop a good working philosophy of how it can be meaningfully used in scientific work. Not that all "apply machine learning to X" approaches aren't interesting, but I lack enough understanding to know whether these are popping up everywhere because people feel obligated to apply new methods. For example, the Fourier transform is deep and interesting and there are libraries and standards and ways to transform different objects over clusters, etc., but I wouldn't say good scientific research is about finding a place to apply a Fourier transform (maybe :)). I am new to this though.
KuriousCat 3y ago
This is the consequence of giving up privacy and agency very easily. At some point universities stopped encouraging independent exploration and started minting out industry ready vocational humans eager to join the rat race. That needs to be fixed but at this point there are not many direct beneficiaries who would want to address that problem so this is going to get worse for a while.
- vkou 3y ago
  > At some point universities stopped encouraging independent exploration and started minting out industry ready vocational humans eager to join the rat race. Universities absolutely encourage independent exploration. Every year, they mint ~10x more grad students than there are research professor positions for them. You want universities to produce even more of them?
  - HarryHirsch 3y ago
    Back then we had industry research labs. There was Bell Labs, the Frythe, DuPont Experimental Station, BASF's Zentralforschung, and nowadays we have the 737-MAX.
    - daveguy 3y ago
      There are still a bunch of industry research labs. Here's a currated list of just those doing computer science research: https://cra.org/cra-wp/research-labs/ https://cra.org/cra-wp/research-labs/ Just because Boeing sucks these days doesn't mean there isn't good research happening in industry.
      - leetrout 3y ago
        I wonder how accurate that list is now. I see Twitter listed and I assume they have stopped doing CS research.
        daveguy 3y ago
renegade-otter 3y ago
AI is just evolution of Big Data. It's not transformative, like the Internet was. The Internet was a new medium. A more advanced Hadoop will not change your life. It will not change the way you read, listen, watch, communicate (fundamentally). The Internet did. Especially at a time when the "legacy" internet is falling apart, a new way to find information is rather a necessity than some breakthrough. The "good" AI needs to fight it out with the "bad" AI before we see if the net positive is even there. A podcast can be translated by AI into a different language, you say? The price is super-realistic fraudulent phone and video calls from your "family".
- tomrod 3y ago
  I'd argue this take is a bit dated, as if you're looking at ML/AI in terms of training of data. If you look at the enablement of features, AI is a new (and, currently, often flaky) medium. It would be analogous to looking at the Internet as limited to making it simpler to order from a catalog over the phone. While technically true, there is more that it can do besides. Much like we don't use copper in our walls for dialup anymore, I don't anticipate the initial architectures of LLM and MoE to last terribly long, but they do enable proving that the concept works.
  - chewxy 3y ago
    Current AI/ML is in fact a reflection on training data. Change the manifold, and change the response. It's unfortunate that of course the data is the entire internet.
    - Eisenstein 3y ago
      Can you please explain what that means? I'm not sure I get it.
      - chewxy 3y ago
        GP mentioned that the current slate of transformer based AIs are not transformative in the same way the Internet was. Rather it's more of a triumph of data engineering practices. OP disagrees with GP. OP's main thesis is that AI enables a lot new applications. OP claims that GP is simply looking at it as if it were training data. I stated that current AI techniques ARE indeed just reflections of the data used in training. I agree with GP that the current "AI"s are simply not transformative in the same way the Internet was. If you change the training data for the current generation of AI, you get different behaviours. The training data forms a manifold - which you can think of as a landscape with features forming valleys and hills. What the current generation of AI does is that it tries to find a shape that fits the landscape - think of it like taking a very large sheet of cloth to cover a landscape. The stiffer the cloth, the less well the cloth fits to the landscape. The "stiffness" of the cloth is the amount of parameters that a neural network has. Modern deep nets are highly overparameterized - imagine a very soft pliable cloth - of course it fits to a landscape well. So if you have a different training data - the neural network will fit to this different landscape as well. Hence the response will be different. It's unfortunate that the training data is the entire internet for a few reasons: 1. Only the rich can train a vaguely competent AI. You're at the whims of those well-resourced enough. 2. There's no "alternate" training dataset anymore. (Though a clever thing people at OpenAI are doing are Mixture of Experts models, where you train multiple NNs using different subsets of the full training set, so you get multiple competencies)
deleted 3y ago
[deleted]
tazu 3y ago
> How have we completely lost focus on new physics or foundational sciences and devoted the smartest minds of our time to transformers and data scraping? I'm not smart enough to be a physicist, but I like listening to Eric Weinstein[1]. He thinks string theory is essentially a dead-end honeypot doing exactly what you describe with our smartest minds. [1]: https://youtu.be/eOvqJwgY8ow https://youtu.be/eOvqJwgY8ow
- sebmellen 3y ago
  If Weinstein could tone down his conspiratorial edges and focus on the primary substance of what he’s saying, I think he would be more effective at achieving his goals. I like his podcast appearances — they are fun to listen to — but the solution to political machinations destroying established institutions is not to focus on the politics! We need to escape that frame entirely, and focus instead on building new institutions that are sufficiently reverent of smart minds and brilliant people. Yes, we need new physics! We get there by escaping the current career trap which stops brilliant people from trying new approaches. Give the top, boldest, most daring researchers an alternative to tenure — $10m vested in a secure position at a new research institute. Then they won’t have to be scared of string theory boogeymen.
- mianos 3y ago
  Sabine Hossenfelder just did a talk on this. I wonder if the smartest AIs will go down the same rabbit-hole and chew up a billion CPUs in a loop?
jdmdmdmdmd 3y ago
[dead]
protocolture 3y ago
People are working hard in all fields. Just because you see lots of AI news doesnt mean its the only thing being done.
tim333 3y ago
Tech innovation doesn't really happen through grand planning and "focus on the foundational principles of growth." It happens through lots of people trying to make stuff. Many people think about the diretion this is going but tend to get dismissed as singularuty cranks if they think far enough ahead.

dotnet00 3y ago

I think part of the issue is that a lot of the 'big' advances in physics nowadays aren't constrained by brain power as much as they are by cost (or rather, money allocation), and due to the corruption of the scientific funding system in most rich countries, waste is extremely high, severely limiting the money actually going into science. Eg to make radical progress in physics, we need more sensitive telescopes, larger particle accelerators, more efficient space missions, more actual iteration on fusion reactors etc. All these things cost a lot of money, yet the majority of it gets eaten by systems designed to minimize the money actually going towards the research work. This isn't even really getting into how the researchers are paid. I agree that AI researchers are concerningly blasé about what they're actually aiming to do. AGI is poorly defined, and the only negative externalities they seem to pay lip service to is culture war stuff. There's near zero consideration for the real negative externalities they're perpetrating upon the world. I think partly this also manifests in how they interact with fields where even older AI developments would be useful. At the physics lab I work at, I've found that many scientists have a negative impression of AI 'researchers', where they don't care to understand the problem they're trying to solve, preferring to just treat everything - even things where the solution space can be constrained by scientific understanding - as a black box. It's almost the opposite of science in approach.

sebmellen 3y ago
It’s still surprising to me that not one decabillionaire has funded a good basic sciences institute. With a few hundred million dollars (paid primarily to salaries) and an extremely rigorous selection process, one could establish a highly prestigious research university and bring the brightest minds together in one place. Just go from there! If Leland Stanford could do it…
- rsanek 3y ago
  https://arcinstitute.org/ https://arcinstitute.org/
  - sebmellen 3y ago
    Arc is a cool concept, but these initiatives are ultimately handicapped by their reliance on the infrastructure of "grants". Grants suck. What matters is having smart people together in one place. If you pay smart people enough money, and offer them enough insulation and protection from academic politics, they will come to your institute. Grant-making to people at institutions which are already politically and socially captured does not fix the root issue.
- timeforcomputer 3y ago
  Not sure if relevant as it is not just one institute but the Simons Foundation (https://en.wikipedia.org/wiki/Simons_Foundation https://en.wikipedia.org/wiki/Simons_Foundation) seems along this line.
timeforcomputer 3y ago
I am studying physics now (with a CS and math background) and I feel obligated to get up to date on AI and to develop a good working philosophy of how it can be meaningfully used in scientific work. Not that all "apply machine learning to X" approaches aren't interesting, but I lack enough understanding to know whether these are popping up everywhere because people feel obligated to apply new methods. For example, the Fourier transform is deep and interesting and there are libraries and standards and ways to transform different objects over clusters, etc., but I wouldn't say good scientific research is about finding a place to apply a Fourier transform (maybe :)). I am new to this though.

KuriousCat 3y ago

This is the consequence of giving up privacy and agency very easily. At some point universities stopped encouraging independent exploration and started minting out industry ready vocational humans eager to join the rat race. That needs to be fixed but at this point there are not many direct beneficiaries who would want to address that problem so this is going to get worse for a while.

vkou 3y ago
> At some point universities stopped encouraging independent exploration and started minting out industry ready vocational humans eager to join the rat race. Universities absolutely encourage independent exploration. Every year, they mint ~10x more grad students than there are research professor positions for them. You want universities to produce even more of them?
- HarryHirsch 3y ago
  Back then we had industry research labs. There was Bell Labs, the Frythe, DuPont Experimental Station, BASF's Zentralforschung, and nowadays we have the 737-MAX.
  - daveguy 3y ago
    There are still a bunch of industry research labs. Here's a currated list of just those doing computer science research: https://cra.org/cra-wp/research-labs/ https://cra.org/cra-wp/research-labs/ Just because Boeing sucks these days doesn't mean there isn't good research happening in industry.
    - leetrout 3y ago
      I wonder how accurate that list is now. I see Twitter listed and I assume they have stopped doing CS research.
      - daveguy 3y ago

renegade-otter 3y ago

AI is just evolution of Big Data. It's not transformative, like the Internet was. The Internet was a new medium. A more advanced Hadoop will not change your life. It will not change the way you read, listen, watch, communicate (fundamentally). The Internet did. Especially at a time when the "legacy" internet is falling apart, a new way to find information is rather a necessity than some breakthrough. The "good" AI needs to fight it out with the "bad" AI before we see if the net positive is even there. A podcast can be translated by AI into a different language, you say? The price is super-realistic fraudulent phone and video calls from your "family".

tomrod 3y ago
I'd argue this take is a bit dated, as if you're looking at ML/AI in terms of training of data. If you look at the enablement of features, AI is a new (and, currently, often flaky) medium. It would be analogous to looking at the Internet as limited to making it simpler to order from a catalog over the phone. While technically true, there is more that it can do besides. Much like we don't use copper in our walls for dialup anymore, I don't anticipate the initial architectures of LLM and MoE to last terribly long, but they do enable proving that the concept works.
- chewxy 3y ago
  Current AI/ML is in fact a reflection on training data. Change the manifold, and change the response. It's unfortunate that of course the data is the entire internet.
  - Eisenstein 3y ago
    Can you please explain what that means? I'm not sure I get it.
    - chewxy 3y ago
      GP mentioned that the current slate of transformer based AIs are not transformative in the same way the Internet was. Rather it's more of a triumph of data engineering practices. OP disagrees with GP. OP's main thesis is that AI enables a lot new applications. OP claims that GP is simply looking at it as if it were training data. I stated that current AI techniques ARE indeed just reflections of the data used in training. I agree with GP that the current "AI"s are simply not transformative in the same way the Internet was. If you change the training data for the current generation of AI, you get different behaviours. The training data forms a manifold - which you can think of as a landscape with features forming valleys and hills. What the current generation of AI does is that it tries to find a shape that fits the landscape - think of it like taking a very large sheet of cloth to cover a landscape. The stiffer the cloth, the less well the cloth fits to the landscape. The "stiffness" of the cloth is the amount of parameters that a neural network has. Modern deep nets are highly overparameterized - imagine a very soft pliable cloth - of course it fits to a landscape well. So if you have a different training data - the neural network will fit to this different landscape as well. Hence the response will be different. It's unfortunate that the training data is the entire internet for a few reasons: 1. Only the rich can train a vaguely competent AI. You're at the whims of those well-resourced enough. 2. There's no "alternate" training dataset anymore. (Though a clever thing people at OpenAI are doing are Mixture of Experts models, where you train multiple NNs using different subsets of the full training set, so you get multiple competencies)

deleted 3y ago

[deleted]

tazu 3y ago

> How have we completely lost focus on new physics or foundational sciences and devoted the smartest minds of our time to transformers and data scraping? I'm not smart enough to be a physicist, but I like listening to Eric Weinstein[1]. He thinks string theory is essentially a dead-end honeypot doing exactly what you describe with our smartest minds. [1]: https://youtu.be/eOvqJwgY8ow https://youtu.be/eOvqJwgY8ow

sebmellen 3y ago
If Weinstein could tone down his conspiratorial edges and focus on the primary substance of what he’s saying, I think he would be more effective at achieving his goals. I like his podcast appearances — they are fun to listen to — but the solution to political machinations destroying established institutions is not to focus on the politics! We need to escape that frame entirely, and focus instead on building new institutions that are sufficiently reverent of smart minds and brilliant people. Yes, we need new physics! We get there by escaping the current career trap which stops brilliant people from trying new approaches. Give the top, boldest, most daring researchers an alternative to tenure — $10m vested in a secure position at a new research institute. Then they won’t have to be scared of string theory boogeymen.
mianos 3y ago
Sabine Hossenfelder just did a talk on this. I wonder if the smartest AIs will go down the same rabbit-hole and chew up a billion CPUs in a loop?

jdmdmdmdmd 3y ago

[dead]

protocolture 3y ago

People are working hard in all fields. Just because you see lots of AI news doesnt mean its the only thing being done.

tim333 3y ago

Tech innovation doesn't really happen through grand planning and "focus on the foundational principles of growth." It happens through lots of people trying to make stuff. Many people think about the diretion this is going but tend to get dismissed as singularuty cranks if they think far enough ahead.