3 ms·

Depends who's using it. Like many tools the force multiplier depends on the operator. I'm confident it's an amplifier for people who know how learning works a

by jonahx 1mo ago

Depends who's using it. Like many tools the force multiplier depends on the operator. I'm confident it's an amplifier for people who know how learning works and already do a lot of it, successfully. However the level of "learning fluency" I'm talking about isn't reached for many until late college or grad school, and sometimes not at all. So I'm not surprised by the quoted results for 12-18 year olds.

bawolff 1mo ago
And people who want to learn. Most teenagers lack agency in their studies. They aren't in high school because they love it but because they have no choice.
ChrisSD 1mo ago
Sometimes things are just common sense pure and simple.
- dan_ggggg 1mo ago
  > Sometimes things are just common sense pure and simple. The discussion is about "AI", so common sense is out the window. These people's professional reputations depend on addict-level "AI" usage remaining socially acceptable.
sebastiennight 1mo ago
> Depends who's using it. Like many tools the force multiplier depends on the operator. How would you prove/disprove this assumption without falling into a True Scotsman fallacy?
- jonahx 1mo ago
  Fair question. In theory you could do an experiment where the subjects were grad students or professors, or top performing college students. Give them some fixed amount of time to understand some new subject on which they'll be tested, only 1 group has access to an LLM with appropriate context for the learning task, etc. That's just one idea...
  - sebastiennight 1mo ago
    This experimental setup would only tell you whether this demographic (grad students or professors, or top performing college students) benefits from access to LLMs for acquiring a new knowledge or skill. It would not prove that "the results depend on the operator". To prove this (and figure out the traits that make a proficient operator) you would either need a massive dataset to which you'd apply some machine learning to figure out the correlations, or at least you'd need a hypothesis on the traits you want to test. Do left-handed people perform better with LLMs? Analytical thinkers? Dyslexic people? It's not enough to just claim "some individuals will perform better with these tools" without any notion of who does, and whether it's possible to become one of these people, and what is the expected gain in this group. Otherwise, sure, we could sell unsecured chainsaws as a tool for making ice sculptures and observe that "some people" are indeed more proficient at not getting their face cut off ; but if the tool is being marketed to schools, such a vague statement is not helpful in making a case for it.
    - jonahx 1mo ago
      Yes, I'm using "high-performing college students" or "phd students" or whatever as a proxy for "people skilled at learning". It's not perfect but I think it'd be good enough. You could do lots of refinements on the idea. You could try to do some kind of "pre-test" that more directly tested "meta-learning skills". My only point was I think it's possible to do. I personally think it's pretty clear. And there's nothing special about AI. In rural places, you could replace it with "access to books" or "access to a good tutor" and so on. I also think you'd see the same effect with "motivation" (if you could test it), with performance on standardized tests, and with grades. AI will boost the better (by these metrics) students more.
      - sebastiennight 1mo ago
        My understanding of this study is that it disproves your assumption: "The negative learning effects are larger for students with higher initial achievement." It seems that the students who had the most skill and motivation from the start, have the most to lose.

by jonahx 1mo ago

bawolff 1mo ago

And people who want to learn. Most teenagers lack agency in their studies. They aren't in high school because they love it but because they have no choice.

ChrisSD 1mo ago

Sometimes things are just common sense pure and simple.

dan_ggggg 1mo ago
> Sometimes things are just common sense pure and simple. The discussion is about "AI", so common sense is out the window. These people's professional reputations depend on addict-level "AI" usage remaining socially acceptable.

sebastiennight 1mo ago

> Depends who's using it. Like many tools the force multiplier depends on the operator. How would you prove/disprove this assumption without falling into a True Scotsman fallacy?

jonahx 1mo ago
Fair question. In theory you could do an experiment where the subjects were grad students or professors, or top performing college students. Give them some fixed amount of time to understand some new subject on which they'll be tested, only 1 group has access to an LLM with appropriate context for the learning task, etc. That's just one idea...
- sebastiennight 1mo ago
  This experimental setup would only tell you whether this demographic (grad students or professors, or top performing college students) benefits from access to LLMs for acquiring a new knowledge or skill. It would not prove that "the results depend on the operator". To prove this (and figure out the traits that make a proficient operator) you would either need a massive dataset to which you'd apply some machine learning to figure out the correlations, or at least you'd need a hypothesis on the traits you want to test. Do left-handed people perform better with LLMs? Analytical thinkers? Dyslexic people? It's not enough to just claim "some individuals will perform better with these tools" without any notion of who does, and whether it's possible to become one of these people, and what is the expected gain in this group. Otherwise, sure, we could sell unsecured chainsaws as a tool for making ice sculptures and observe that "some people" are indeed more proficient at not getting their face cut off ; but if the tool is being marketed to schools, such a vague statement is not helpful in making a case for it.
  - jonahx 1mo ago
    Yes, I'm using "high-performing college students" or "phd students" or whatever as a proxy for "people skilled at learning". It's not perfect but I think it'd be good enough. You could do lots of refinements on the idea. You could try to do some kind of "pre-test" that more directly tested "meta-learning skills". My only point was I think it's possible to do. I personally think it's pretty clear. And there's nothing special about AI. In rural places, you could replace it with "access to books" or "access to a good tutor" and so on. I also think you'd see the same effect with "motivation" (if you could test it), with performance on standardized tests, and with grades. AI will boost the better (by these metrics) students more.
    - sebastiennight 1mo ago
      My understanding of this study is that it disproves your assumption: "The negative learning effects are larger for students with higher initial achievement." It seems that the students who had the most skill and motivation from the start, have the most to lose.