6 ms·

All of science is getting harder as the easiest discoveries are all pretty much behind us. LLMs were a breakthrough I didn't expect and it's likely the last on

All of science is getting harder as the easiest discoveries are all pretty much behind us. LLMs were a breakthrough I didn't expect and it's likely the last one we'll see in our lifetime.

iterance 9mo ago
Specific fields may not advance for decades at a time, but we are hardly in a scientific drought. There have been dramatic advances in countless fields over the last 20 years alone and there is no good reason to expect such advances to abruptly cease. Frankly this is far too pessimistic.
- threethirtytwo 9mo ago
  I don't understand what is wrong with pessimism. That's not a valid critique. If someone is pessimistic but his description of the world matches REALITY, then there's nothing wrong with his view point. Either way this is also opinion based. There hasn't been a revolutionary change in technology in the last 20 years. I don't consider smart phones to be revolutionary. I consider going to the moon revolutionary and catching a rocket sort of revolutionary. Actually I take that back I predict mars as a possible break through along with LLMs, but we got lucky with musk.
  - tehjoker 9mo ago
    genetic technology and computing technology have been the biggest drivers for a while. i do think it is remarkable to video call another continent. communication technology is disruptive and revolutionary though it looks like chaos. ai is interesting too if it lives up to the hype even slightly. catching a rocket is very impressive, but its just a lower cost method for earth orbit. it does unlock megaconstellations tho
    - threethirtytwo 9mo ago
      Yeah none of those are step function changes. Video calling another continent is like a tiny step from TV. Yeah I receive video wirelessly on my tv not that amazed when I can stretch the distance further with a call that has video. Big deal. AI is the step function change. The irony is that it became so pervasive and intertwined with slop people like you forget that what it does now (write all code) was unheard of just a couple years ago. ai surpassed the hype, now it’s popular to talk shit about it.
      - incr_me 9mo ago
        A step in which function are you talking about, exactly?
        threethirtytwo 9mo ago
        If you want it stated precisely, the function is human cognitive labor per unit time and cost. For decades, progress mostly shifted physical constraints or communication bandwidth. Faster chips, better networks, cheaper storage. Those move slopes, not discontinuities. Humans still had to think, reason, design, write, debug. The bottleneck stayed human cognition. LLMs changed that. Not marginally. Qualitatively. The input to the function used to be “a human with training.” The output was plans, code, explanations, synthesis. Now the same class of output can be produced on demand, at scale, by a machine, with latency measured in seconds and cost approaching zero. That is a step change in effective cognitive throughput. This is why “video calling another continent” feels incremental. It reduces friction in moving information between humans. AI reduces or removes the human from parts of the loop entirely. You can argue about ceilings, reliability, or long term limits. Fine. But the step already happened. Tasks that were categorically human two years ago are now automatable enough to be economically and practically useful. That is the function. And it jumped.
        deleted 9mo ago
        [deleted]
  - iterance 9mo ago
    My critique is not due to pessimism, it is due to afactuality. Breakthroughs in science are plenty in the modern era and there is no reason to expect them to slow or halt. However, from your later comments, it sounds as though you feel the only operating definition of a "breakthrough" is a change inducing a rapid rise in labor extraction / conventional productivity. I could not disagree more strongly with this opinion, as I find this definition utterly defies intuition. It rejects many, if not most, changes in scientific understanding that do not directly induce a discontinuty in labor extraction. But admittedly if one restricts the definition of a breakthrough in this way, then, well, you're probably about right. (Though I don't see what Mars has to do with labor extraction.)
    - threethirtytwo 9mo ago
      That’s only one dimension. The step function is multidimensional. My critique is more about the Euclidean distance between the initial point and the end point. To which AI is the only technology that has enough distance to be classified as a “breakthrough”.
  - andrewflnr 9mo ago
    You imply your view "matches REALITY", then fall back to "Either way this is also opinion based." Nicely played. But the actual reality is that scientific discovery is proceeding at least as fast as it ever has. These things take time. 20 years is a laughably short time in which to declare defeat, even ignoring the fact that genetic and other biological tech has advanced leaps and bounds in that time. There's important work happening in solid state physics and materials science. JWST is overturning old theories and spawning new ones in cosmology. There's every reality-based reason to believe there will be plenty of big changes in science in the next 20 years or so.
    - threethirtytwo 9mo ago
      [flagged]
      - andrewflnr 9mo ago
        No, your opinions bias toward negativity, and we can see it in this comment by the way you shift the goalposts for every achievement until you can poo-poo it. Oh, except for the ones you just omitted from your quote, maybe because even you can't rationalize why CRISPR isn't a step change.
        threethirtytwo 9mo ago
        >No, your opinions bias toward negativity, and we can see it in this comment by the way you shift the goalposts for every achievement until you can poo-poo it. Oh, except for the ones you just omitted from your quote, maybe because even you can't rationalize why CRISPR isn't a step change. Not true at all. CRISPR isn't a step change because it only made genetic engineering more efficient and it didn't effect the lives of most people. It's still a research thing. I didn't poo-poo AI did I? That's the favorite thing for everyone to poo-poo these days and ironically it's the one thing that effects everyones life and is causing paradigm shifting changes in society right now.
        andrewflnr 9mo ago
  - layer8 9mo ago
    > If someone is pessimistic but his description of the world matches REALITY, then there's nothing wrong with his view point. A description that matches reality is realist, not pessimist.
    - threethirtytwo 9mo ago
      Technically this is true. Practically speaking most realists are perceived to be pessimists. There are tons of scientific studies to back this up as well. People who are judged to be pessimistic experimentally have more accurate perceptions of the real world. This means that most people who you would term as "realists" are likely optimists and not realists at all.
selfmodruntime 9mo ago
The additional irony here is that LLMs are a tool that is likely forever damned to regurgitate knowledge of the past, with the inability to derive new information.
- threethirtytwo 9mo ago
  This is not true at all. Just query any LLM and ask it for new information. Literally ask it to create something that doesn't exist. It will give it to you.
- kingstnap 9mo ago
  It depends on what you mean, specifically on your distance metric. If you mean nearest neighbours search like autocorrect then LLMs are extrapolative. You can easily generate combinations not seen before. I mean you can prove this with parametric prompting. Like "Generate a poem about {noun} in {place} in {language}" or whatever. This is a simplistic example but it doesn't take much to come up with a space that has quadrillion of possibilities. Then if you randomly sample 10 and they all seem to be "right" then you have proven it's not pure neighbour recall. Same is true of the image generators. You can prove its not memorizing because you can generate random varients and show that the number of images realizable is more than the training data possibly contains. If you mean on the underlying manifold of language and ideas. Its definitely interpolation, which is fundamentally a limitation of what can be done using data alone. But I know this can be expanded over iteration (I have done experiments related to this). The trick to expanding it actually running experiments/simulation on values at the boundry of the manifold. You have to run experiments on the unknown.
  - threethirtytwo 9mo ago
    It is interpolation but that is what human thinking is as well. Interpolation is so broad it can cover agi conceptually. But I get it, the interpolation you’re talking about is limited. But I think you missed this insight: human interpolation is limited too. In the short term everything we do is simply recombination of ideas as you put it. But that’s the short term. In the long term we do things that are much greater. But I think this is just an aggregation of small changes. Change the words in a poem 5000 times: have the LLM do the same task 5000 times. Let it pick a random word. The result is wholly original. And I think in the end this what human cognition is as well.
8note 9mo ago
famous last words before quantum physics hit

All of science is getting harder as the easiest discoveries are all pretty much behind us. LLMs were a breakthrough I didn't expect and it's likely the last one we'll see in our lifetime.

iterance 9mo ago

Specific fields may not advance for decades at a time, but we are hardly in a scientific drought. There have been dramatic advances in countless fields over the last 20 years alone and there is no good reason to expect such advances to abruptly cease. Frankly this is far too pessimistic.

threethirtytwo 9mo ago
I don't understand what is wrong with pessimism. That's not a valid critique. If someone is pessimistic but his description of the world matches REALITY, then there's nothing wrong with his view point. Either way this is also opinion based. There hasn't been a revolutionary change in technology in the last 20 years. I don't consider smart phones to be revolutionary. I consider going to the moon revolutionary and catching a rocket sort of revolutionary. Actually I take that back I predict mars as a possible break through along with LLMs, but we got lucky with musk.
- tehjoker 9mo ago
  genetic technology and computing technology have been the biggest drivers for a while. i do think it is remarkable to video call another continent. communication technology is disruptive and revolutionary though it looks like chaos. ai is interesting too if it lives up to the hype even slightly. catching a rocket is very impressive, but its just a lower cost method for earth orbit. it does unlock megaconstellations tho
  - threethirtytwo 9mo ago
    Yeah none of those are step function changes. Video calling another continent is like a tiny step from TV. Yeah I receive video wirelessly on my tv not that amazed when I can stretch the distance further with a call that has video. Big deal. AI is the step function change. The irony is that it became so pervasive and intertwined with slop people like you forget that what it does now (write all code) was unheard of just a couple years ago. ai surpassed the hype, now it’s popular to talk shit about it.
    - incr_me 9mo ago
      A step in which function are you talking about, exactly?
      - threethirtytwo 9mo ago
        If you want it stated precisely, the function is human cognitive labor per unit time and cost. For decades, progress mostly shifted physical constraints or communication bandwidth. Faster chips, better networks, cheaper storage. Those move slopes, not discontinuities. Humans still had to think, reason, design, write, debug. The bottleneck stayed human cognition. LLMs changed that. Not marginally. Qualitatively. The input to the function used to be “a human with training.” The output was plans, code, explanations, synthesis. Now the same class of output can be produced on demand, at scale, by a machine, with latency measured in seconds and cost approaching zero. That is a step change in effective cognitive throughput. This is why “video calling another continent” feels incremental. It reduces friction in moving information between humans. AI reduces or removes the human from parts of the loop entirely. You can argue about ceilings, reliability, or long term limits. Fine. But the step already happened. Tasks that were categorically human two years ago are now automatable enough to be economically and practically useful. That is the function. And it jumped.
        deleted 9mo ago
        [deleted]
- iterance 9mo ago
  My critique is not due to pessimism, it is due to afactuality. Breakthroughs in science are plenty in the modern era and there is no reason to expect them to slow or halt. However, from your later comments, it sounds as though you feel the only operating definition of a "breakthrough" is a change inducing a rapid rise in labor extraction / conventional productivity. I could not disagree more strongly with this opinion, as I find this definition utterly defies intuition. It rejects many, if not most, changes in scientific understanding that do not directly induce a discontinuty in labor extraction. But admittedly if one restricts the definition of a breakthrough in this way, then, well, you're probably about right. (Though I don't see what Mars has to do with labor extraction.)
  - threethirtytwo 9mo ago
    That’s only one dimension. The step function is multidimensional. My critique is more about the Euclidean distance between the initial point and the end point. To which AI is the only technology that has enough distance to be classified as a “breakthrough”.
- andrewflnr 9mo ago
  You imply your view "matches REALITY", then fall back to "Either way this is also opinion based." Nicely played. But the actual reality is that scientific discovery is proceeding at least as fast as it ever has. These things take time. 20 years is a laughably short time in which to declare defeat, even ignoring the fact that genetic and other biological tech has advanced leaps and bounds in that time. There's important work happening in solid state physics and materials science. JWST is overturning old theories and spawning new ones in cosmology. There's every reality-based reason to believe there will be plenty of big changes in science in the next 20 years or so.
  - threethirtytwo 9mo ago
    [flagged]
    - andrewflnr 9mo ago
      No, your opinions bias toward negativity, and we can see it in this comment by the way you shift the goalposts for every achievement until you can poo-poo it. Oh, except for the ones you just omitted from your quote, maybe because even you can't rationalize why CRISPR isn't a step change.
      - threethirtytwo 9mo ago
        >No, your opinions bias toward negativity, and we can see it in this comment by the way you shift the goalposts for every achievement until you can poo-poo it. Oh, except for the ones you just omitted from your quote, maybe because even you can't rationalize why CRISPR isn't a step change. Not true at all. CRISPR isn't a step change because it only made genetic engineering more efficient and it didn't effect the lives of most people. It's still a research thing. I didn't poo-poo AI did I? That's the favorite thing for everyone to poo-poo these days and ironically it's the one thing that effects everyones life and is causing paradigm shifting changes in society right now.
        andrewflnr 9mo ago
- layer8 9mo ago
  > If someone is pessimistic but his description of the world matches REALITY, then there's nothing wrong with his view point. A description that matches reality is realist, not pessimist.
  - threethirtytwo 9mo ago
    Technically this is true. Practically speaking most realists are perceived to be pessimists. There are tons of scientific studies to back this up as well. People who are judged to be pessimistic experimentally have more accurate perceptions of the real world. This means that most people who you would term as "realists" are likely optimists and not realists at all.

selfmodruntime 9mo ago

The additional irony here is that LLMs are a tool that is likely forever damned to regurgitate knowledge of the past, with the inability to derive new information.

threethirtytwo 9mo ago
This is not true at all. Just query any LLM and ask it for new information. Literally ask it to create something that doesn't exist. It will give it to you.
kingstnap 9mo ago
It depends on what you mean, specifically on your distance metric. If you mean nearest neighbours search like autocorrect then LLMs are extrapolative. You can easily generate combinations not seen before. I mean you can prove this with parametric prompting. Like "Generate a poem about {noun} in {place} in {language}" or whatever. This is a simplistic example but it doesn't take much to come up with a space that has quadrillion of possibilities. Then if you randomly sample 10 and they all seem to be "right" then you have proven it's not pure neighbour recall. Same is true of the image generators. You can prove its not memorizing because you can generate random varients and show that the number of images realizable is more than the training data possibly contains. If you mean on the underlying manifold of language and ideas. Its definitely interpolation, which is fundamentally a limitation of what can be done using data alone. But I know this can be expanded over iteration (I have done experiments related to this). The trick to expanding it actually running experiments/simulation on values at the boundry of the manifold. You have to run experiments on the unknown.
- threethirtytwo 9mo ago
  It is interpolation but that is what human thinking is as well. Interpolation is so broad it can cover agi conceptually. But I get it, the interpolation you’re talking about is limited. But I think you missed this insight: human interpolation is limited too. In the short term everything we do is simply recombination of ideas as you put it. But that’s the short term. In the long term we do things that are much greater. But I think this is just an aggregation of small changes. Change the words in a poem 5000 times: have the LLM do the same task 5000 times. Let it pick a random word. The result is wholly original. And I think in the end this what human cognition is as well.

8note 9mo ago

famous last words before quantum physics hit