6 ms·

> For example, if you are an ELO 1000 chess player would you yourself be able to tell if Magnus Carlson or another grandmaster were better by playing them indiv

> For example, if you are an ELO 1000 chess player would you yourself be able to tell if Magnus Carlson or another grandmaster were better by playing them individually? No, but I wouldn't be able to tell you what the player did wrong in general. By contrast, the shortcomings of today's LLMs seem pretty obvious to me.

make3 1y ago
the argument is for in the future, not now
- runarberg 1y ago
  The future had us abandon traditional currency in favor of bitcoin, it had digital artists being able to sell NFTs for their work, it had supersonic jet travel, self driving or even flying cars. It had population centers on the moon, mines on asteroids, fusion power plants, etc. I think large language models have the same future as supersonic jet travel. It’s usefulness will fail to realize, with traditional models being good enough but for a fraction of the price, while some startups keep trying to push this technology but meanwhile consumers keep rejecting it.
  - Mackena 1y ago
    [flagged]
  - germandiago 1y ago
    I also have the intuition that something like this is the most likely outcome.
  - eru 1y ago
    Even if models keep stagnating at roughly the current state of the art (with only minor gains), we are still working through the massive economic changes they will bring. Unlike supersonic passenger jet travel, which is possible and happened, but never had much of an impact on the wider economy, because it never caught on.
    - rusk 1y ago
      Cost was what brought supersonic down. Comparatively speaking, it may be the cost/benefit curve that will decide the limit of this generation of technology. It seems to me the stuff we are looking at now is massively subsidised by exuberant private investment. The way these things go, there will come a point where investors want to see a return, and that will be a decider on wether the wheels keep spinning in the data centre. That said, supersonic flight is yet very much a thing in military circles …
      - eru 1y ago
        Yes, cost is important. Very important. AI is a bit like railways in the 19th century: once you train the model (= once you put down the track), actually running the inference (= running your trains) is comparatively cheap. Even if the companies later go bankrupt and investors lose interest, the trained models are still there (= the rails stay in place). That was reasonably common in the US: some promising company would get British (and German etc) investors to put up money to lay down tracks. Later the American company would go bust, but the rails stayed in America.
        runarberg 1y ago
        I think there is a fundamental difference though. In the 19th century when you had a rail line between two places it pretty much established the only means of transport between those places. Unless there was a river or a canal in place, the alternative was pretty much walking (or maybe a horse and a carriage). The large language models are not that much better than a single artist / programmer / technical writer (in fact they are significantly worse) working for a couple of hours. Modern tools do indeed increase the productivity of workers to the extent where AI generated content is not worth it in most (all?) industries (unless you are very cheap; but then maybe your workers will organize against you). If we want to keep the railway analogy, training an AI model in 2025 is like building a railway line in 2025 where there is already a highway, and the highway is already sufficient for the traffic it gets, and won’t require expansion in the foreseeable future.
  - ants_everywhere 1y ago
    I've seen this take a lot, but I don't know why because it's extremely divorced from reality. Demand for AI is insanely high. They can't make chips fast enough to meet customer demand. The energy industry is transforming to try to meet the demand. Whomever is telling you that consumers are rejecting it is lying to you, and you should honestly probably reevaluate where you get your information. Because it's not serving you well.
    - refactor_master 1y ago
      I think the above post has a fair point. Demand for chatbot customer service in various forms is surely "insanely high" - but demand from whom? Because I don't recall any end-user ever asking for it. No, instead it'll be the new calculator that you can use to lazy-draft an email on your 1.5 hour Ryanair economy flight to the South. Both unthinkable luxuries just decades ago, but neither of which have transformed humanity profoundly.
      - ants_everywhere 1y ago
        This is just the same argument. If you believe demand for AI is low then you should be able to verify that with market data. Currently market data is showing a very high demand for AI. These arguments come down to "thumbs down to AI". If people just said that it would at least be an honest argument. But pretending that consumers don't want LLMs when they're some of the most popular apps in the history of mankind is not a defensible position
        runarberg 1y ago
        I‘m not sure this works in reverse. If demand is indeed high, you could show that with market data. But if you have marked data e.g. showing high valuation of AI companies, or x many requests over some period, that doesn’t mean necessarily that demand is high. In other words, marked data is necessary but not sufficient to prove your claim. Reasons for market data seemingly showing high demand without there actually being one include: Market manipulation (including marketing campaigns), artificial or inflated demand, forced usage, hype, etc. As an example NFTs, Bitcoin, and supersonic jet travel all had “an insane market data” which seemed at the time to show that there was a huge demand for these things. My prediction is that we are in the early Concord era of supersonic jet travel and Boeing is racing to catch up to the promise of this technology. Except that in an unregulated market such as the current tech market, we have forgone all the safety and security measures and the Concord has made its first passenger flight in 1969 (as opposed to 1976), with tons of fan fare and all flights fully booked months in advance. Note that in the 1960 it was market forecasts had the demand for Concord to build 350 airplanes by 1980, and at the time the first prototypes were flying they had 74 options. Only 20 were every built for passenger flight.
  - School-Cotton 1y ago
    LLMs are already extremely useful today
    - xwolfi 1y ago
      Any sort of argument ?
      - School-Cotton 1y ago
        Personal experience: I use them.
  - ngcazz 1y ago
    A future where anything has to be paid (but it's crypto) doesn't sound futuristic to me at all.
alfalfasprout 1y ago
Actually, chess commentators do this all the time. They have the luxury of consulting with others, and discussing + analyzing freely. Even without the use of an engine.
- komaromy 1y ago
  Anyone more than ~300 points below the players can only contribute to the discussion in a superficial capacity though
- nopinsight 1y ago
  Au contraire, AlphaGo made several “counterintuitive” moves that professional Go players thought were mistakes during the play, but turned out to be great strategic moves in hindsight. The (in)ability to recognize a strange move’s brilliance might depend on the complexity of the game. The real world is much more complex than any board game. https://en.m.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol https://en.m.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol
  - tonyhart7 1y ago
    I think its doable tbh, if you pour enough resources (smart people,energy,compute power etc) like the entire planet resources of course we can have AGI (damned if we don't) because we put so much, it better works but the problem we cant do that right because its so expensive, AGI is not matter of if but when but even then it always about the cost
    - jsrozner 1y ago
      There may be philosophical (i.e. fundamental) challenges to AGI. Consider, e.g., Godel's Incompleteness Theorem. Though Scott Aaronson argues this does not matter (see e.g., youtube video, "How Much Math Is Knowable?"). There would also seem to be limits to the computation of potentially chaotic systems. And in general, verifying physical theories has required the carrying out of actual physical experiment. Even if we were to build a fully reasoning model, "pondering" is not always sufficient.

make3 1y ago

the argument is for in the future, not now

runarberg 1y ago
The future had us abandon traditional currency in favor of bitcoin, it had digital artists being able to sell NFTs for their work, it had supersonic jet travel, self driving or even flying cars. It had population centers on the moon, mines on asteroids, fusion power plants, etc. I think large language models have the same future as supersonic jet travel. It’s usefulness will fail to realize, with traditional models being good enough but for a fraction of the price, while some startups keep trying to push this technology but meanwhile consumers keep rejecting it.
- Mackena 1y ago
  [flagged]
- germandiago 1y ago
  I also have the intuition that something like this is the most likely outcome.
- eru 1y ago
  Even if models keep stagnating at roughly the current state of the art (with only minor gains), we are still working through the massive economic changes they will bring. Unlike supersonic passenger jet travel, which is possible and happened, but never had much of an impact on the wider economy, because it never caught on.
  - rusk 1y ago
    Cost was what brought supersonic down. Comparatively speaking, it may be the cost/benefit curve that will decide the limit of this generation of technology. It seems to me the stuff we are looking at now is massively subsidised by exuberant private investment. The way these things go, there will come a point where investors want to see a return, and that will be a decider on wether the wheels keep spinning in the data centre. That said, supersonic flight is yet very much a thing in military circles …
    - eru 1y ago
      Yes, cost is important. Very important. AI is a bit like railways in the 19th century: once you train the model (= once you put down the track), actually running the inference (= running your trains) is comparatively cheap. Even if the companies later go bankrupt and investors lose interest, the trained models are still there (= the rails stay in place). That was reasonably common in the US: some promising company would get British (and German etc) investors to put up money to lay down tracks. Later the American company would go bust, but the rails stayed in America.
      - runarberg 1y ago
        I think there is a fundamental difference though. In the 19th century when you had a rail line between two places it pretty much established the only means of transport between those places. Unless there was a river or a canal in place, the alternative was pretty much walking (or maybe a horse and a carriage). The large language models are not that much better than a single artist / programmer / technical writer (in fact they are significantly worse) working for a couple of hours. Modern tools do indeed increase the productivity of workers to the extent where AI generated content is not worth it in most (all?) industries (unless you are very cheap; but then maybe your workers will organize against you). If we want to keep the railway analogy, training an AI model in 2025 is like building a railway line in 2025 where there is already a highway, and the highway is already sufficient for the traffic it gets, and won’t require expansion in the foreseeable future.
- ants_everywhere 1y ago
  I've seen this take a lot, but I don't know why because it's extremely divorced from reality. Demand for AI is insanely high. They can't make chips fast enough to meet customer demand. The energy industry is transforming to try to meet the demand. Whomever is telling you that consumers are rejecting it is lying to you, and you should honestly probably reevaluate where you get your information. Because it's not serving you well.
  - refactor_master 1y ago
    I think the above post has a fair point. Demand for chatbot customer service in various forms is surely "insanely high" - but demand from whom? Because I don't recall any end-user ever asking for it. No, instead it'll be the new calculator that you can use to lazy-draft an email on your 1.5 hour Ryanair economy flight to the South. Both unthinkable luxuries just decades ago, but neither of which have transformed humanity profoundly.
    - ants_everywhere 1y ago
      This is just the same argument. If you believe demand for AI is low then you should be able to verify that with market data. Currently market data is showing a very high demand for AI. These arguments come down to "thumbs down to AI". If people just said that it would at least be an honest argument. But pretending that consumers don't want LLMs when they're some of the most popular apps in the history of mankind is not a defensible position
      - runarberg 1y ago
        I‘m not sure this works in reverse. If demand is indeed high, you could show that with market data. But if you have marked data e.g. showing high valuation of AI companies, or x many requests over some period, that doesn’t mean necessarily that demand is high. In other words, marked data is necessary but not sufficient to prove your claim. Reasons for market data seemingly showing high demand without there actually being one include: Market manipulation (including marketing campaigns), artificial or inflated demand, forced usage, hype, etc. As an example NFTs, Bitcoin, and supersonic jet travel all had “an insane market data” which seemed at the time to show that there was a huge demand for these things. My prediction is that we are in the early Concord era of supersonic jet travel and Boeing is racing to catch up to the promise of this technology. Except that in an unregulated market such as the current tech market, we have forgone all the safety and security measures and the Concord has made its first passenger flight in 1969 (as opposed to 1976), with tons of fan fare and all flights fully booked months in advance. Note that in the 1960 it was market forecasts had the demand for Concord to build 350 airplanes by 1980, and at the time the first prototypes were flying they had 74 options. Only 20 were every built for passenger flight.
- School-Cotton 1y ago
  LLMs are already extremely useful today
  - xwolfi 1y ago
    Any sort of argument ?
    - School-Cotton 1y ago
      Personal experience: I use them.
- ngcazz 1y ago
  A future where anything has to be paid (but it's crypto) doesn't sound futuristic to me at all.

alfalfasprout 1y ago

Actually, chess commentators do this all the time. They have the luxury of consulting with others, and discussing + analyzing freely. Even without the use of an engine.

komaromy 1y ago
Anyone more than ~300 points below the players can only contribute to the discussion in a superficial capacity though
nopinsight 1y ago
Au contraire, AlphaGo made several “counterintuitive” moves that professional Go players thought were mistakes during the play, but turned out to be great strategic moves in hindsight. The (in)ability to recognize a strange move’s brilliance might depend on the complexity of the game. The real world is much more complex than any board game. https://en.m.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol https://en.m.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol
- tonyhart7 1y ago
  I think its doable tbh, if you pour enough resources (smart people,energy,compute power etc) like the entire planet resources of course we can have AGI (damned if we don't) because we put so much, it better works but the problem we cant do that right because its so expensive, AGI is not matter of if but when but even then it always about the cost
  - jsrozner 1y ago
    There may be philosophical (i.e. fundamental) challenges to AGI. Consider, e.g., Godel's Incompleteness Theorem. Though Scott Aaronson argues this does not matter (see e.g., youtube video, "How Much Math Is Knowable?"). There would also seem to be limits to the computation of potentially chaotic systems. And in general, verifying physical theories has required the carrying out of actual physical experiment. Even if we were to build a fully reasoning model, "pondering" is not always sufficient.