10 ms·

Of course AI is extractive, everything is lately

81 points by cdme 3y ago

sebmellen 3y ago
There’s a shocking lack of introspection in the AI hype bubble — we are continually promised world-changing advances, but no one is very clear about which direction these advances are pushing us in. Even ‘AGI’ is nebulous and undefined at best. Where is the focus on the foundational principles of growth? The author writes that endless growth is impossible, but that is only true in the absence of novel breakthroughs. We need creations and inventions which bring us the future we once imagined: limitless energy and abundance. How have we completely lost focus on new physics or foundational sciences and devoted the smartest minds of our time to transformers and data scraping? There is some critical human component which is completely absent here. Where is the renaissance, and how do we make it happen?
- dotnet00 3y ago
  I think part of the issue is that a lot of the 'big' advances in physics nowadays aren't constrained by brain power as much as they are by cost (or rather, money allocation), and due to the corruption of the scientific funding system in most rich countries, waste is extremely high, severely limiting the money actually going into science. Eg to make radical progress in physics, we need more sensitive telescopes, larger particle accelerators, more efficient space missions, more actual iteration on fusion reactors etc. All these things cost a lot of money, yet the majority of it gets eaten by systems designed to minimize the money actually going towards the research work. This isn't even really getting into how the researchers are paid. I agree that AI researchers are concerningly blasé about what they're actually aiming to do. AGI is poorly defined, and the only negative externalities they seem to pay lip service to is culture war stuff. There's near zero consideration for the real negative externalities they're perpetrating upon the world. I think partly this also manifests in how they interact with fields where even older AI developments would be useful. At the physics lab I work at, I've found that many scientists have a negative impression of AI 'researchers', where they don't care to understand the problem they're trying to solve, preferring to just treat everything - even things where the solution space can be constrained by scientific understanding - as a black box. It's almost the opposite of science in approach.
  - sebmellen 3y ago
    It’s still surprising to me that not one decabillionaire has funded a good basic sciences institute. With a few hundred million dollars (paid primarily to salaries) and an extremely rigorous selection process, one could establish a highly prestigious research university and bring the brightest minds together in one place. Just go from there! If Leland Stanford could do it…
    - rsanek 3y ago
      https://arcinstitute.org/ https://arcinstitute.org/
      - sebmellen 3y ago
        Arc is a cool concept, but these initiatives are ultimately handicapped by their reliance on the infrastructure of "grants". Grants suck. What matters is having smart people together in one place. If you pay smart people enough money, and offer them enough insulation and protection from academic politics, they will come to your institute. Grant-making to people at institutions which are already politically and socially captured does not fix the root issue.
    - timeforcomputer 3y ago
      Not sure if relevant as it is not just one institute but the Simons Foundation (https://en.wikipedia.org/wiki/Simons_Foundation https://en.wikipedia.org/wiki/Simons_Foundation) seems along this line.
  - timeforcomputer 3y ago
    I am studying physics now (with a CS and math background) and I feel obligated to get up to date on AI and to develop a good working philosophy of how it can be meaningfully used in scientific work. Not that all "apply machine learning to X" approaches aren't interesting, but I lack enough understanding to know whether these are popping up everywhere because people feel obligated to apply new methods. For example, the Fourier transform is deep and interesting and there are libraries and standards and ways to transform different objects over clusters, etc., but I wouldn't say good scientific research is about finding a place to apply a Fourier transform (maybe :)). I am new to this though.
- KuriousCat 3y ago
  This is the consequence of giving up privacy and agency very easily. At some point universities stopped encouraging independent exploration and started minting out industry ready vocational humans eager to join the rat race. That needs to be fixed but at this point there are not many direct beneficiaries who would want to address that problem so this is going to get worse for a while.
  - vkou 3y ago
    > At some point universities stopped encouraging independent exploration and started minting out industry ready vocational humans eager to join the rat race. Universities absolutely encourage independent exploration. Every year, they mint ~10x more grad students than there are research professor positions for them. You want universities to produce even more of them?
    - HarryHirsch 3y ago
      Back then we had industry research labs. There was Bell Labs, the Frythe, DuPont Experimental Station, BASF's Zentralforschung, and nowadays we have the 737-MAX.
      - daveguy 3y ago
        There are still a bunch of industry research labs. Here's a currated list of just those doing computer science research: https://cra.org/cra-wp/research-labs/ https://cra.org/cra-wp/research-labs/ Just because Boeing sucks these days doesn't mean there isn't good research happening in industry.
        leetrout 3y ago
        I wonder how accurate that list is now. I see Twitter listed and I assume they have stopped doing CS research.
        daveguy 3y ago
- renegade-otter 3y ago
  AI is just evolution of Big Data. It's not transformative, like the Internet was. The Internet was a new medium. A more advanced Hadoop will not change your life. It will not change the way you read, listen, watch, communicate (fundamentally). The Internet did. Especially at a time when the "legacy" internet is falling apart, a new way to find information is rather a necessity than some breakthrough. The "good" AI needs to fight it out with the "bad" AI before we see if the net positive is even there. A podcast can be translated by AI into a different language, you say? The price is super-realistic fraudulent phone and video calls from your "family".
  - tomrod 3y ago
    I'd argue this take is a bit dated, as if you're looking at ML/AI in terms of training of data. If you look at the enablement of features, AI is a new (and, currently, often flaky) medium. It would be analogous to looking at the Internet as limited to making it simpler to order from a catalog over the phone. While technically true, there is more that it can do besides. Much like we don't use copper in our walls for dialup anymore, I don't anticipate the initial architectures of LLM and MoE to last terribly long, but they do enable proving that the concept works.
    - chewxy 3y ago
      Current AI/ML is in fact a reflection on training data. Change the manifold, and change the response. It's unfortunate that of course the data is the entire internet.
      - Eisenstein 3y ago
        Can you please explain what that means? I'm not sure I get it.
        chewxy 3y ago
        GP mentioned that the current slate of transformer based AIs are not transformative in the same way the Internet was. Rather it's more of a triumph of data engineering practices. OP disagrees with GP. OP's main thesis is that AI enables a lot new applications. OP claims that GP is simply looking at it as if it were training data. I stated that current AI techniques ARE indeed just reflections of the data used in training. I agree with GP that the current "AI"s are simply not transformative in the same way the Internet was. If you change the training data for the current generation of AI, you get different behaviours. The training data forms a manifold - which you can think of as a landscape with features forming valleys and hills. What the current generation of AI does is that it tries to find a shape that fits the landscape - think of it like taking a very large sheet of cloth to cover a landscape. The stiffer the cloth, the less well the cloth fits to the landscape. The "stiffness" of the cloth is the amount of parameters that a neural network has. Modern deep nets are highly overparameterized - imagine a very soft pliable cloth - of course it fits to a landscape well. So if you have a different training data - the neural network will fit to this different landscape as well. Hence the response will be different. It's unfortunate that the training data is the entire internet for a few reasons: 1. Only the rich can train a vaguely competent AI. You're at the whims of those well-resourced enough. 2. There's no "alternate" training dataset anymore. (Though a clever thing people at OpenAI are doing are Mixture of Experts models, where you train multiple NNs using different subsets of the full training set, so you get multiple competencies)
- deleted 3y ago
  [deleted]
- tazu 3y ago
  > How have we completely lost focus on new physics or foundational sciences and devoted the smartest minds of our time to transformers and data scraping? I'm not smart enough to be a physicist, but I like listening to Eric Weinstein[1]. He thinks string theory is essentially a dead-end honeypot doing exactly what you describe with our smartest minds. [1]: https://youtu.be/eOvqJwgY8ow https://youtu.be/eOvqJwgY8ow
  - sebmellen 3y ago
    If Weinstein could tone down his conspiratorial edges and focus on the primary substance of what he’s saying, I think he would be more effective at achieving his goals. I like his podcast appearances — they are fun to listen to — but the solution to political machinations destroying established institutions is not to focus on the politics! We need to escape that frame entirely, and focus instead on building new institutions that are sufficiently reverent of smart minds and brilliant people. Yes, we need new physics! We get there by escaping the current career trap which stops brilliant people from trying new approaches. Give the top, boldest, most daring researchers an alternative to tenure — $10m vested in a secure position at a new research institute. Then they won’t have to be scared of string theory boogeymen.
  - mianos 3y ago
    Sabine Hossenfelder just did a talk on this. I wonder if the smartest AIs will go down the same rabbit-hole and chew up a billion CPUs in a loop?
- jdmdmdmdmd 3y ago
  [dead]
- protocolture 3y ago
  People are working hard in all fields. Just because you see lots of AI news doesnt mean its the only thing being done.
- tim333 3y ago
  Tech innovation doesn't really happen through grand planning and "focus on the foundational principles of growth." It happens through lots of people trying to make stuff. Many people think about the diretion this is going but tend to get dismissed as singularuty cranks if they think far enough ahead.
ohyes 3y ago
This is a good insight, but it needs to scream the point louder. AI in the form of an LLM is a tool. We’re doing extractive things with all of these other tools as well. It is not AI that is the problem, we are the problem.
anonzzzies 3y ago
After social media anything goes; the rest are just extensions of the mind control social media made possible. Crypto nor AI would’ve been the hype they are without the current depressing iteration of social media and popular ‘influencers’ promoting their ‘businesses’ to somehow legally take your money. Of the only communities still out there that have some sort of free thought, like HN, even on HN, people are defending TikTok for being ‘great for discovery’. Even though it’s likely a Chinese state weapon and responsible for controlling a billion+ people their every waking (and probably therefore sleeping) moments. With seemingly smart people defending garbage like that, what does the rest matter? Who is going to listen to reason while scrolling through endless empty money grabs (which somehow they believe are adding actual value to their lives)? There is an article on the HN homepage about leaded petrol lowering IQ; I bet that in 100 years TikTok etc will be considered far more detrimental than that.
- xvector 3y ago
  I've come across countless great artisans and small businesses I would have never discovered otherwise, all thanks to social media. I've spent tens of thousands of dollars on products from these small businesses. I will go so far as to say that millions of small businesses exist because social media provides them with global reach. As an example, if you forge, say, artisan chefs knives made out of meteorite, no one in your town/village/etc likely cares, but there are thousands of people worldwide that will gladly buy from you. TikTok/Reels is simply the next evolution in information condensation. It's entertaining but also educative. I might not care to spend 20 minutes learning about the intricacies of, say, forging Damascus steel, but I'll gladly watch a couple TikToks on it.
  - sebmellen 3y ago
    The problem with condensed information is that it is extremely lossy. Sure, you may learn something with the bite-sized entertainment offered by a TikTok video or Instagram Reel, but the deeper substance is not there. It’s like saying that by eating 200 individual cheerios, or maybe 200 individual pieces of all kinds of cereal, you’ve eaten a full meal. But you haven’t, really. Learning and understanding requires expending some kind of effort, and easy access to condensed information actually precludes you from experiencing that. If we don’t digest what we learn we are no better than LLMs — mindlessly regurgitating small bits of non-integrated information back and forth to each other.
deleted 3y ago
[deleted]
smallmancontrov 3y ago
> a compulsion to demonstrate the possibility of unending growth which is, frankly, impossible It's only confusing when seen from the bottom. The purpose of growth, seen from the perspective of the wealth-weighted-people served by capitalism, is to deliver a return on capital, to pay them for being rich in proportion to how rich they are. It's possible for everyone to win if the growth is real, but in a pinch fake redistributive growth will work (see: Cantillon pump). In either case, the expectation of growth rationalizes the outcome and the pulling of any policy levers needed to get there.
IMTDb 3y ago
First sentence of the article: "It's easy to pick on AI, because, well, it's costing a whole lot and providing, at best, dubious benefits." "At best dubious benefits" ??? I feel like there is a whole essay missing there, because my analysis of the situation is that it provides at minimum minor benefits to million of people using it on a daily basis, and at best it saves lives to some of them. That it not to say there is 0 harm anywhere; we definitely need to do some cost/benefit analysis. But the initial premise is so obviously flawed that it's hard to consider the rest of the arguments.
- dbtc 3y ago
  Can you give some examples?
  - plagiarist 3y ago
    I use AI instead of a thesaurus, it's pretty adequate.
    - dbtc 3y ago
      The username, the adjective, it's all fabulous.
  - data-ottawa 3y ago
    LLMs/ChatGPT is saving me a lot of time, and it’s actually a very good tool to find a jumping off point for research. Copilot saves me quite a bit of time writing documentation and boilerplate. For hobbies AI is nice because I can ask my beginner questions and get results, where often you end up on low quality blogs or reddit threads arguing about things you shouldn’t bother thinking of when starting out.
    - miek 3y ago
      I agree; It's odd that you received enough downvotes to be partially grayed out.
  - 3y ago
xvector 3y ago
I'm so done with this ridiculous techno-pessimism trend. > AI [is] costing a whole lot and providing, at best, dubious benefits A wild take. Technology like GPT-4 being available today was unimaginable to us just 5 years ago. Our company saves hundreds of thousands of engineering-hours a month using generative AI. It has let me automate the humdrum boilerplate bullshit out of my workflows, both professional and personal. It's also essentially replaced Google for me. Just today I've already used GPT-4 at least 5-10 times. Immensely useful, the $20/mo is a drop in the bucket for the value it provides to me. > Ride-hailing services? Those flouted regulations, hooked users on cheap prices, drove down wages and made employment more precarious. Has this guy ever stepped foot outside a major city? Has he ever gotten drunk and needed a ride home without friends in the immediate vicinity? Uber/Lyft provide immense value that simply wasn't there before. Has he even ever spoken to any Uber drivers, or ex-taxi Uber drivers? Does he realize that people often prefer working for Uber as opposed to a "regular, stable job" and that customers actually like using it over taxis? Compared to taxis, Ubers tend to be cleaner, the drivers tend to be nicer, and it's just a easier, better experience. The taxi cartel is even more powerful and "extractive" than Uber in many cities, and they exert undue market power, especially at airports and resorts, not Uber. > streaming platforms that are actively rolling back up into something analogous to cable Spotify still provides me with far more value/$ than having to buy every single song from iTunes or whatever. Apple TV is still quite useful to me and I far prefer it to ad-riddled cable with set showtimes for what you want to see. Tale as old as time: people seeing the past with rose-tinted glasses so they can have a pessimistic "hot take" about today vs the good old days.
- gorjusborg 3y ago
  Where there is extreme hype, there is extreme skepticism (and often, it is well deserved). If you are tired of hearing the emperor has no clothes, getting mad at the crowd for saying it is not likely to help.
  - xvector 3y ago
    > getting mad at the crowd for saying it is not likely to help Except the emperor does have clothes, this pessimism simply isn't based in reality, it's based on a rose-tinted view of the past and a misunderstanding of why people use these services over the older alternatives. The author seems to have such limited life experience that he can't even fathom why someone would prefer Uber over the taxis of old, or find great value in using generative AI every day, or prefer Spotify over something like the iTunes Store. The value proposition is so obvious and clear, and yet the author can't even see it.
CatWChainsaw 3y ago
Infinite growth is not possible with finite resources. I'm not sure why this is so hard for some people (cough, economists) to understand. They treat it like an actual law of physics, but conveniently ignore every time a curve that looks exponential reveals itself to be sigmoidal on a long enough timescale.
- 65 3y ago
  > Infinite growth is not possible with finite resources. I would argue software is an infinite resource. Therefore, we can have infinite growth in a world of software.
  - CatWChainsaw 3y ago
    Software exists on hardware. You can only put so many bits on so many atoms. Finite.
    - 65 3y ago
      Software can be re-written on the same hardware though. Is it all technically finite? Yes. Though it's infinite in terms of human ability to use software.
- erulabs 3y ago
  A casual glance at nuclear physics points to a world where we’re nowhere near our energy production capabilities. Automation sings along. And that argument would be salient if it was required - “economists say infinite growth is possible” is a strawman argument.
  - CatWChainsaw 3y ago
    "Nowhere near" is not "literally infinite". How many nuclear reactors can you build with a finite amount of matter? See other comment. I'm not sure why pointing out that economists say, believe, and rely on infinite growth is a strawman when it is a statement of fact.
Aerroon 3y ago
>a compulsion to demonstrate the possibility of unending growth which is, frankly, impossible. I hate this saying. Why is unending growth impossible? Nobody seems to every try to explain it. Economic growth is generally improvement of society in some way. Usually it's about new inventions and ideas that spread more. Better ways to make things that people want, more efficient ways etc. if you come up with a more efficient fuel then that's still economic growth and the upside is that resource usage can decrease for the same benefit. Math is essentially proven to be unending. There's always going to be something more to be discovered. Doesn't this mean that unending growth is possible? Of course, in practice "unending" only means "for a very long time". We are nowhere near a point where we have exhausted new advancements.
- jackblemming 3y ago
  Because there’s only so many people in the world and of those people they only have so much money.
  - xvector 3y ago
    There is literally an solar system, galaxy, universe for the taking. We have not even scratched the surface of the resources available to us.
    - deleted 3y ago
      [deleted]
  - Aloisius 3y ago
    Growth is about value produced, not money. Unless technological progress ends, we will continue to produce more and more valuable things. Even if the amount of money never changes or even if money disappears, growth still happens.
- protocolture 3y ago
  99% sure that unending growth would be proven the second we saw a von Neumann probe.
samatman 3y ago
Crypto is finally in the rear view mirror, he says, as Bitcoin hits another all time high...
- protocolture 3y ago
  Yeah this one struck me as the first of several poorly thought out arguments. I think people are satisfied as long as stuff they dont like remains absent from their new sources.
naet 3y ago
AI feels different to me because it's "retroactively" extractive. Social media may be extractive if I give it my attention and data to resell. Crypto is extractive to the people buying into it and losing money. But I can avoid doing those things if I want. AI these days takes anything I've written (from literature to code) or illustrated, and retroactively extracts value from it to be repackaged and redistributed. Even if I was proactive about securing the trademark or copyright, they can seemingly just ignore it without consequence, without me ever participating or opting in to it. It feels massively different to have social media extract value from and image I uploaded willingly, vs AI extracting value (eg making things "in the style" of me) after training on my work against my will.
- dantheman 3y ago
  You mean like how you learned from others? Against their will?
  - Nevermark 3y ago
    Scale has a quality all its own. You create a highly useful website full of dense hard earned knowledge. Some people will just use that knowledge, but a few create their own sites, using what they learned. A company sweeps up hard earned knowledge from millions of sites, with automation but not permission, and goes a long way to making those millions of informational sites redundant without any compensation. Simplifying things obviously. But these two scenarios are not equivalent.
  - jj999 3y ago
    Why can't we agree there are differences between other human beings learning from other people and a multinational corporation learning from everyone at industrial scale, with industrial means?
    - CatWChainsaw 3y ago
      I can't po$$ibly imagine.
  - 3y ago
anonu 3y ago
> everything is lately. Not lately - everything is extractive and has always been. This is the very foundation of economic development and creation. If we were against all extractive things then nobody would work for a living and our entire society would crumble.
- MarcelOlsz 3y ago
  Or maybe we'd just be dancing around fires fornicating and singing and hunting as intended. Wish "cities" were strictly medicinal hubs and the rest free to roam.
  - idle_zealot 3y ago
    That sounds terrible. I'm glad most of our ancestors disagreed with you.
    - MarcelOlsz 3y ago
      Yeah man I love working and never using my natural skills.
  - mianos 3y ago
    The idea of abandoning cities to frolic in the wilderness is naive and disconnected from historical reality. For most of human history, life was defined by grueling labor, scarcity, disease and servitude for all but a privileged elite. Modern development, for all its flaws, has brought immense progress in living standards and quality of life that shouldn't be romanticized away.
    - MarcelOlsz 3y ago
      I meant more of the sustainable earth home type than getting lime disease all day and walking barefoot in the amazon. What we have now is too much and disconnected from the nature around us. For example I like old japanese architecture because it was all done by hand (fun, commnunity, passion), with wood that grew locally so that it contracted and expanded with the weather and could stand it. >Modern development, for all its flaws, has brought immense progress in living standards and quality of life that shouldn't be romanticized away. Of course but it's imbalanced. The whole "infinite growth at any cost" thing combined with intense individualism is a recipe for solitude and mental health disaster. I'm not surprised populations are dropping.
grogenaut 3y ago
Is there anything in humanity or the universe that isn't extractive at some level? Machinery, used oil or metal. Hell if you go all the way to solar or wood, you're still extracting energy from the sun which is extracting extracting energy from fusion. Everything constructive is just refining entropy into something.
1vuio0pswjnm7 3y ago
https://www.ipsos.com/sites/default/files/ct/news/documents/2023-07/Ipsos%20Global%20AI%202023%20Report-WEB.pdf https://www.ipsos.com/sites/default/files/ct/news/documents/... Only 37% of people surveyed in the US believe AI has more benefits than drawbacks AI is curiously well-received in countries with high levels of corruption
tim333 3y ago
Not sure what he means by extractive. Googling a bit has >extractive industry can be defined as a processes that involve different activities that lead to the extraction of raw materials from the earth (such as oil, metals, mineral and aggregates), processing and utilization by consumers. AI seems the opposite. It takes nothing from the earth and outputs vast quantities of iffy writing photos and similar.

81 points by cdme 3y ago

There’s a shocking lack of introspection in the AI hype bubble — we are continually promised world-changing advances, but no one is very clear about which direction these advances are pushing us in. Even ‘AGI’ is nebulous and undefined at best. Where is the focus on the foundational principles of growth? The author writes that endless growth is impossible, but that is only true in the absence of novel breakthroughs. We need creations and inventions which bring us the future we once imagined: limitless energy and abundance. How have we completely lost focus on new physics or foundational sciences and devoted the smartest minds of our time to transformers and data scraping? There is some critical human component which is completely absent here. Where is the renaissance, and how do we make it happen?

dotnet00 3y ago
I think part of the issue is that a lot of the 'big' advances in physics nowadays aren't constrained by brain power as much as they are by cost (or rather, money allocation), and due to the corruption of the scientific funding system in most rich countries, waste is extremely high, severely limiting the money actually going into science. Eg to make radical progress in physics, we need more sensitive telescopes, larger particle accelerators, more efficient space missions, more actual iteration on fusion reactors etc. All these things cost a lot of money, yet the majority of it gets eaten by systems designed to minimize the money actually going towards the research work. This isn't even really getting into how the researchers are paid. I agree that AI researchers are concerningly blasé about what they're actually aiming to do. AGI is poorly defined, and the only negative externalities they seem to pay lip service to is culture war stuff. There's near zero consideration for the real negative externalities they're perpetrating upon the world. I think partly this also manifests in how they interact with fields where even older AI developments would be useful. At the physics lab I work at, I've found that many scientists have a negative impression of AI 'researchers', where they don't care to understand the problem they're trying to solve, preferring to just treat everything - even things where the solution space can be constrained by scientific understanding - as a black box. It's almost the opposite of science in approach.
- sebmellen 3y ago
  It’s still surprising to me that not one decabillionaire has funded a good basic sciences institute. With a few hundred million dollars (paid primarily to salaries) and an extremely rigorous selection process, one could establish a highly prestigious research university and bring the brightest minds together in one place. Just go from there! If Leland Stanford could do it…
  - rsanek 3y ago
    https://arcinstitute.org/ https://arcinstitute.org/
    - sebmellen 3y ago
      Arc is a cool concept, but these initiatives are ultimately handicapped by their reliance on the infrastructure of "grants". Grants suck. What matters is having smart people together in one place. If you pay smart people enough money, and offer them enough insulation and protection from academic politics, they will come to your institute. Grant-making to people at institutions which are already politically and socially captured does not fix the root issue.
  - timeforcomputer 3y ago
    Not sure if relevant as it is not just one institute but the Simons Foundation (https://en.wikipedia.org/wiki/Simons_Foundation https://en.wikipedia.org/wiki/Simons_Foundation) seems along this line.
- timeforcomputer 3y ago
  I am studying physics now (with a CS and math background) and I feel obligated to get up to date on AI and to develop a good working philosophy of how it can be meaningfully used in scientific work. Not that all "apply machine learning to X" approaches aren't interesting, but I lack enough understanding to know whether these are popping up everywhere because people feel obligated to apply new methods. For example, the Fourier transform is deep and interesting and there are libraries and standards and ways to transform different objects over clusters, etc., but I wouldn't say good scientific research is about finding a place to apply a Fourier transform (maybe :)). I am new to this though.
KuriousCat 3y ago
This is the consequence of giving up privacy and agency very easily. At some point universities stopped encouraging independent exploration and started minting out industry ready vocational humans eager to join the rat race. That needs to be fixed but at this point there are not many direct beneficiaries who would want to address that problem so this is going to get worse for a while.
- vkou 3y ago
  > At some point universities stopped encouraging independent exploration and started minting out industry ready vocational humans eager to join the rat race. Universities absolutely encourage independent exploration. Every year, they mint ~10x more grad students than there are research professor positions for them. You want universities to produce even more of them?
  - HarryHirsch 3y ago
    Back then we had industry research labs. There was Bell Labs, the Frythe, DuPont Experimental Station, BASF's Zentralforschung, and nowadays we have the 737-MAX.
    - daveguy 3y ago
      There are still a bunch of industry research labs. Here's a currated list of just those doing computer science research: https://cra.org/cra-wp/research-labs/ https://cra.org/cra-wp/research-labs/ Just because Boeing sucks these days doesn't mean there isn't good research happening in industry.
      - leetrout 3y ago
        I wonder how accurate that list is now. I see Twitter listed and I assume they have stopped doing CS research.
        daveguy 3y ago
renegade-otter 3y ago
AI is just evolution of Big Data. It's not transformative, like the Internet was. The Internet was a new medium. A more advanced Hadoop will not change your life. It will not change the way you read, listen, watch, communicate (fundamentally). The Internet did. Especially at a time when the "legacy" internet is falling apart, a new way to find information is rather a necessity than some breakthrough. The "good" AI needs to fight it out with the "bad" AI before we see if the net positive is even there. A podcast can be translated by AI into a different language, you say? The price is super-realistic fraudulent phone and video calls from your "family".
- tomrod 3y ago
  I'd argue this take is a bit dated, as if you're looking at ML/AI in terms of training of data. If you look at the enablement of features, AI is a new (and, currently, often flaky) medium. It would be analogous to looking at the Internet as limited to making it simpler to order from a catalog over the phone. While technically true, there is more that it can do besides. Much like we don't use copper in our walls for dialup anymore, I don't anticipate the initial architectures of LLM and MoE to last terribly long, but they do enable proving that the concept works.
  - chewxy 3y ago
    Current AI/ML is in fact a reflection on training data. Change the manifold, and change the response. It's unfortunate that of course the data is the entire internet.
    - Eisenstein 3y ago
      Can you please explain what that means? I'm not sure I get it.
      - chewxy 3y ago
        GP mentioned that the current slate of transformer based AIs are not transformative in the same way the Internet was. Rather it's more of a triumph of data engineering practices. OP disagrees with GP. OP's main thesis is that AI enables a lot new applications. OP claims that GP is simply looking at it as if it were training data. I stated that current AI techniques ARE indeed just reflections of the data used in training. I agree with GP that the current "AI"s are simply not transformative in the same way the Internet was. If you change the training data for the current generation of AI, you get different behaviours. The training data forms a manifold - which you can think of as a landscape with features forming valleys and hills. What the current generation of AI does is that it tries to find a shape that fits the landscape - think of it like taking a very large sheet of cloth to cover a landscape. The stiffer the cloth, the less well the cloth fits to the landscape. The "stiffness" of the cloth is the amount of parameters that a neural network has. Modern deep nets are highly overparameterized - imagine a very soft pliable cloth - of course it fits to a landscape well. So if you have a different training data - the neural network will fit to this different landscape as well. Hence the response will be different. It's unfortunate that the training data is the entire internet for a few reasons: 1. Only the rich can train a vaguely competent AI. You're at the whims of those well-resourced enough. 2. There's no "alternate" training dataset anymore. (Though a clever thing people at OpenAI are doing are Mixture of Experts models, where you train multiple NNs using different subsets of the full training set, so you get multiple competencies)
deleted 3y ago
[deleted]
tazu 3y ago
> How have we completely lost focus on new physics or foundational sciences and devoted the smartest minds of our time to transformers and data scraping? I'm not smart enough to be a physicist, but I like listening to Eric Weinstein[1]. He thinks string theory is essentially a dead-end honeypot doing exactly what you describe with our smartest minds. [1]: https://youtu.be/eOvqJwgY8ow https://youtu.be/eOvqJwgY8ow
- sebmellen 3y ago
  If Weinstein could tone down his conspiratorial edges and focus on the primary substance of what he’s saying, I think he would be more effective at achieving his goals. I like his podcast appearances — they are fun to listen to — but the solution to political machinations destroying established institutions is not to focus on the politics! We need to escape that frame entirely, and focus instead on building new institutions that are sufficiently reverent of smart minds and brilliant people. Yes, we need new physics! We get there by escaping the current career trap which stops brilliant people from trying new approaches. Give the top, boldest, most daring researchers an alternative to tenure — $10m vested in a secure position at a new research institute. Then they won’t have to be scared of string theory boogeymen.
- mianos 3y ago
  Sabine Hossenfelder just did a talk on this. I wonder if the smartest AIs will go down the same rabbit-hole and chew up a billion CPUs in a loop?
jdmdmdmdmd 3y ago
[dead]
protocolture 3y ago
People are working hard in all fields. Just because you see lots of AI news doesnt mean its the only thing being done.
tim333 3y ago
Tech innovation doesn't really happen through grand planning and "focus on the foundational principles of growth." It happens through lots of people trying to make stuff. Many people think about the diretion this is going but tend to get dismissed as singularuty cranks if they think far enough ahead.

ohyes 3y ago

This is a good insight, but it needs to scream the point louder. AI in the form of an LLM is a tool. We’re doing extractive things with all of these other tools as well. It is not AI that is the problem, we are the problem.

anonzzzies 3y ago

After social media anything goes; the rest are just extensions of the mind control social media made possible. Crypto nor AI would’ve been the hype they are without the current depressing iteration of social media and popular ‘influencers’ promoting their ‘businesses’ to somehow legally take your money. Of the only communities still out there that have some sort of free thought, like HN, even on HN, people are defending TikTok for being ‘great for discovery’. Even though it’s likely a Chinese state weapon and responsible for controlling a billion+ people their every waking (and probably therefore sleeping) moments. With seemingly smart people defending garbage like that, what does the rest matter? Who is going to listen to reason while scrolling through endless empty money grabs (which somehow they believe are adding actual value to their lives)? There is an article on the HN homepage about leaded petrol lowering IQ; I bet that in 100 years TikTok etc will be considered far more detrimental than that.

xvector 3y ago
I've come across countless great artisans and small businesses I would have never discovered otherwise, all thanks to social media. I've spent tens of thousands of dollars on products from these small businesses. I will go so far as to say that millions of small businesses exist because social media provides them with global reach. As an example, if you forge, say, artisan chefs knives made out of meteorite, no one in your town/village/etc likely cares, but there are thousands of people worldwide that will gladly buy from you. TikTok/Reels is simply the next evolution in information condensation. It's entertaining but also educative. I might not care to spend 20 minutes learning about the intricacies of, say, forging Damascus steel, but I'll gladly watch a couple TikToks on it.
- sebmellen 3y ago
  The problem with condensed information is that it is extremely lossy. Sure, you may learn something with the bite-sized entertainment offered by a TikTok video or Instagram Reel, but the deeper substance is not there. It’s like saying that by eating 200 individual cheerios, or maybe 200 individual pieces of all kinds of cereal, you’ve eaten a full meal. But you haven’t, really. Learning and understanding requires expending some kind of effort, and easy access to condensed information actually precludes you from experiencing that. If we don’t digest what we learn we are no better than LLMs — mindlessly regurgitating small bits of non-integrated information back and forth to each other.

deleted 3y ago

[deleted]

smallmancontrov 3y ago

> a compulsion to demonstrate the possibility of unending growth which is, frankly, impossible It's only confusing when seen from the bottom. The purpose of growth, seen from the perspective of the wealth-weighted-people served by capitalism, is to deliver a return on capital, to pay them for being rich in proportion to how rich they are. It's possible for everyone to win if the growth is real, but in a pinch fake redistributive growth will work (see: Cantillon pump). In either case, the expectation of growth rationalizes the outcome and the pulling of any policy levers needed to get there.

IMTDb 3y ago

First sentence of the article: "It's easy to pick on AI, because, well, it's costing a whole lot and providing, at best, dubious benefits." "At best dubious benefits" ??? I feel like there is a whole essay missing there, because my analysis of the situation is that it provides at minimum minor benefits to million of people using it on a daily basis, and at best it saves lives to some of them. That it not to say there is 0 harm anywhere; we definitely need to do some cost/benefit analysis. But the initial premise is so obviously flawed that it's hard to consider the rest of the arguments.

dbtc 3y ago
Can you give some examples?
- plagiarist 3y ago
  I use AI instead of a thesaurus, it's pretty adequate.
  - dbtc 3y ago
    The username, the adjective, it's all fabulous.
- data-ottawa 3y ago
  LLMs/ChatGPT is saving me a lot of time, and it’s actually a very good tool to find a jumping off point for research. Copilot saves me quite a bit of time writing documentation and boilerplate. For hobbies AI is nice because I can ask my beginner questions and get results, where often you end up on low quality blogs or reddit threads arguing about things you shouldn’t bother thinking of when starting out.
  - miek 3y ago
    I agree; It's odd that you received enough downvotes to be partially grayed out.
- 3y ago

xvector 3y ago

I'm so done with this ridiculous techno-pessimism trend. > AI [is] costing a whole lot and providing, at best, dubious benefits A wild take. Technology like GPT-4 being available today was unimaginable to us just 5 years ago. Our company saves hundreds of thousands of engineering-hours a month using generative AI. It has let me automate the humdrum boilerplate bullshit out of my workflows, both professional and personal. It's also essentially replaced Google for me. Just today I've already used GPT-4 at least 5-10 times. Immensely useful, the $20/mo is a drop in the bucket for the value it provides to me. > Ride-hailing services? Those flouted regulations, hooked users on cheap prices, drove down wages and made employment more precarious. Has this guy ever stepped foot outside a major city? Has he ever gotten drunk and needed a ride home without friends in the immediate vicinity? Uber/Lyft provide immense value that simply wasn't there before. Has he even ever spoken to any Uber drivers, or ex-taxi Uber drivers? Does he realize that people often prefer working for Uber as opposed to a "regular, stable job" and that customers actually like using it over taxis? Compared to taxis, Ubers tend to be cleaner, the drivers tend to be nicer, and it's just a easier, better experience. The taxi cartel is even more powerful and "extractive" than Uber in many cities, and they exert undue market power, especially at airports and resorts, not Uber. > streaming platforms that are actively rolling back up into something analogous to cable Spotify still provides me with far more value/$ than having to buy every single song from iTunes or whatever. Apple TV is still quite useful to me and I far prefer it to ad-riddled cable with set showtimes for what you want to see. Tale as old as time: people seeing the past with rose-tinted glasses so they can have a pessimistic "hot take" about today vs the good old days.

gorjusborg 3y ago
Where there is extreme hype, there is extreme skepticism (and often, it is well deserved). If you are tired of hearing the emperor has no clothes, getting mad at the crowd for saying it is not likely to help.
- xvector 3y ago
  > getting mad at the crowd for saying it is not likely to help Except the emperor does have clothes, this pessimism simply isn't based in reality, it's based on a rose-tinted view of the past and a misunderstanding of why people use these services over the older alternatives. The author seems to have such limited life experience that he can't even fathom why someone would prefer Uber over the taxis of old, or find great value in using generative AI every day, or prefer Spotify over something like the iTunes Store. The value proposition is so obvious and clear, and yet the author can't even see it.

CatWChainsaw 3y ago

Infinite growth is not possible with finite resources. I'm not sure why this is so hard for some people (cough, economists) to understand. They treat it like an actual law of physics, but conveniently ignore every time a curve that looks exponential reveals itself to be sigmoidal on a long enough timescale.

65 3y ago
> Infinite growth is not possible with finite resources. I would argue software is an infinite resource. Therefore, we can have infinite growth in a world of software.
- CatWChainsaw 3y ago
  Software exists on hardware. You can only put so many bits on so many atoms. Finite.
  - 65 3y ago
    Software can be re-written on the same hardware though. Is it all technically finite? Yes. Though it's infinite in terms of human ability to use software.
erulabs 3y ago
A casual glance at nuclear physics points to a world where we’re nowhere near our energy production capabilities. Automation sings along. And that argument would be salient if it was required - “economists say infinite growth is possible” is a strawman argument.
- CatWChainsaw 3y ago
  "Nowhere near" is not "literally infinite". How many nuclear reactors can you build with a finite amount of matter? See other comment. I'm not sure why pointing out that economists say, believe, and rely on infinite growth is a strawman when it is a statement of fact.

Aerroon 3y ago

>a compulsion to demonstrate the possibility of unending growth which is, frankly, impossible. I hate this saying. Why is unending growth impossible? Nobody seems to every try to explain it. Economic growth is generally improvement of society in some way. Usually it's about new inventions and ideas that spread more. Better ways to make things that people want, more efficient ways etc. if you come up with a more efficient fuel then that's still economic growth and the upside is that resource usage can decrease for the same benefit. Math is essentially proven to be unending. There's always going to be something more to be discovered. Doesn't this mean that unending growth is possible? Of course, in practice "unending" only means "for a very long time". We are nowhere near a point where we have exhausted new advancements.

jackblemming 3y ago
Because there’s only so many people in the world and of those people they only have so much money.
- xvector 3y ago
  There is literally an solar system, galaxy, universe for the taking. We have not even scratched the surface of the resources available to us.
  - deleted 3y ago
    [deleted]
- Aloisius 3y ago
  Growth is about value produced, not money. Unless technological progress ends, we will continue to produce more and more valuable things. Even if the amount of money never changes or even if money disappears, growth still happens.
protocolture 3y ago
99% sure that unending growth would be proven the second we saw a von Neumann probe.

samatman 3y ago

Crypto is finally in the rear view mirror, he says, as Bitcoin hits another all time high...

protocolture 3y ago
Yeah this one struck me as the first of several poorly thought out arguments. I think people are satisfied as long as stuff they dont like remains absent from their new sources.

naet 3y ago

AI feels different to me because it's "retroactively" extractive. Social media may be extractive if I give it my attention and data to resell. Crypto is extractive to the people buying into it and losing money. But I can avoid doing those things if I want. AI these days takes anything I've written (from literature to code) or illustrated, and retroactively extracts value from it to be repackaged and redistributed. Even if I was proactive about securing the trademark or copyright, they can seemingly just ignore it without consequence, without me ever participating or opting in to it. It feels massively different to have social media extract value from and image I uploaded willingly, vs AI extracting value (eg making things "in the style" of me) after training on my work against my will.

dantheman 3y ago
You mean like how you learned from others? Against their will?
- Nevermark 3y ago
  Scale has a quality all its own. You create a highly useful website full of dense hard earned knowledge. Some people will just use that knowledge, but a few create their own sites, using what they learned. A company sweeps up hard earned knowledge from millions of sites, with automation but not permission, and goes a long way to making those millions of informational sites redundant without any compensation. Simplifying things obviously. But these two scenarios are not equivalent.
- jj999 3y ago
  Why can't we agree there are differences between other human beings learning from other people and a multinational corporation learning from everyone at industrial scale, with industrial means?
  - CatWChainsaw 3y ago
    I can't po$$ibly imagine.
- 3y ago

anonu 3y ago

> everything is lately. Not lately - everything is extractive and has always been. This is the very foundation of economic development and creation. If we were against all extractive things then nobody would work for a living and our entire society would crumble.

MarcelOlsz 3y ago
Or maybe we'd just be dancing around fires fornicating and singing and hunting as intended. Wish "cities" were strictly medicinal hubs and the rest free to roam.
- idle_zealot 3y ago
  That sounds terrible. I'm glad most of our ancestors disagreed with you.
  - MarcelOlsz 3y ago
    Yeah man I love working and never using my natural skills.
- mianos 3y ago
  The idea of abandoning cities to frolic in the wilderness is naive and disconnected from historical reality. For most of human history, life was defined by grueling labor, scarcity, disease and servitude for all but a privileged elite. Modern development, for all its flaws, has brought immense progress in living standards and quality of life that shouldn't be romanticized away.
  - MarcelOlsz 3y ago
    I meant more of the sustainable earth home type than getting lime disease all day and walking barefoot in the amazon. What we have now is too much and disconnected from the nature around us. For example I like old japanese architecture because it was all done by hand (fun, commnunity, passion), with wood that grew locally so that it contracted and expanded with the weather and could stand it. >Modern development, for all its flaws, has brought immense progress in living standards and quality of life that shouldn't be romanticized away. Of course but it's imbalanced. The whole "infinite growth at any cost" thing combined with intense individualism is a recipe for solitude and mental health disaster. I'm not surprised populations are dropping.

grogenaut 3y ago

Is there anything in humanity or the universe that isn't extractive at some level? Machinery, used oil or metal. Hell if you go all the way to solar or wood, you're still extracting energy from the sun which is extracting extracting energy from fusion. Everything constructive is just refining entropy into something.

1vuio0pswjnm7 3y ago

https://www.ipsos.com/sites/default/files/ct/news/documents/2023-07/Ipsos%20Global%20AI%202023%20Report-WEB.pdf https://www.ipsos.com/sites/default/files/ct/news/documents/... Only 37% of people surveyed in the US believe AI has more benefits than drawbacks AI is curiously well-received in countries with high levels of corruption

tim333 3y ago

Not sure what he means by extractive. Googling a bit has >extractive industry can be defined as a processes that involve different activities that lead to the extraction of raw materials from the earth (such as oil, metals, mineral and aggregates), processing and utilization by consumers. AI seems the opposite. It takes nothing from the earth and outputs vast quantities of iffy writing photos and similar.