7 ms·

From my experience, you have to be good at math to trust an LLM to do the math.

jerf 2mo ago
There is an old koan in the old hacker literature: "A novice was trying to fix a broken Lisp machine by turning the power off and on. "Knight[, one of the principle designers of the Lisp machine], seeing what the student was doing, spoke sternly: 'You cannot fix a machine by just power-cycling it with no understanding of what is going wrong.' "Knight turned the machine off and on. "The machine worked." I feel like AI is manifesting this even more concretely. I don't feel like I'm guiding the AI super intensely as I work on it with software engineering. I'd have a hard time pointing you at where in the prompt my decades of experience are manifesting. But I definitely can have better results, even with a less frontier-level AI, than people who don't know the same amount of stuff. Terence Tao also released some unedited transcripts of some of his conversations with AI, and many people observed that while many mathematicians may have been able to formulate the initial question, very few people could have given the same feedback to the AI. Perhaps someday AI will eliminate the need for competence to use it properly. But that day is not today. And to be honest, that tech is probably not LLMs, no matter how large they get. Some other breakthrough will be necessary to truly eliminate the human element. Those psychopathic elites making plans to turn Earth into one of the Spacer worlds from Asimov's works with a small elite population supported entirely with robots take notes... it's not possible yet.
- najmbajwa123 2mo ago
  Exactly. AI is an amplifier, not a substitute, if you multiply high domain knowledge by AI you get incredible leverage, but 0 knowledge times AI is still effectively 0.
  - cindyllm 2mo ago
    [dead]
paulpauper 2mo ago
This is why laypeople have not used Ai to make math breakthroughs . it has all been ppl with lots of knowledge already.
- contubernio 2mo ago
  People with lots of knowledge and (monetary and computing) resources.
- najmbajwa123 2mo ago
  Exactly, without knowledge, thats where ai slop comes in
annzabelle 2mo ago
That's been an interesting thing for me with Claude Code and working with non-technical coworkers. Often they ask me to write something that could be a simple script and takes maybe 3 prompts to get it right, and I find myself wondering why they can't just do it themselves. But then I have them asking me what a .md file is and how to open it, and they're unable to follow readme instructions to use commandline programs, so now I have to throw a "make a gui" onto every prompt.
- najmbajwa123 2mo ago
  true, basic tech literacy is also a real bottleneck.

jerf 2mo ago

There is an old koan in the old hacker literature: "A novice was trying to fix a broken Lisp machine by turning the power off and on. "Knight[, one of the principle designers of the Lisp machine], seeing what the student was doing, spoke sternly: 'You cannot fix a machine by just power-cycling it with no understanding of what is going wrong.' "Knight turned the machine off and on. "The machine worked." I feel like AI is manifesting this even more concretely. I don't feel like I'm guiding the AI super intensely as I work on it with software engineering. I'd have a hard time pointing you at where in the prompt my decades of experience are manifesting. But I definitely can have better results, even with a less frontier-level AI, than people who don't know the same amount of stuff. Terence Tao also released some unedited transcripts of some of his conversations with AI, and many people observed that while many mathematicians may have been able to formulate the initial question, very few people could have given the same feedback to the AI. Perhaps someday AI will eliminate the need for competence to use it properly. But that day is not today. And to be honest, that tech is probably not LLMs, no matter how large they get. Some other breakthrough will be necessary to truly eliminate the human element. Those psychopathic elites making plans to turn Earth into one of the Spacer worlds from Asimov's works with a small elite population supported entirely with robots take notes... it's not possible yet.

najmbajwa123 2mo ago
Exactly. AI is an amplifier, not a substitute, if you multiply high domain knowledge by AI you get incredible leverage, but 0 knowledge times AI is still effectively 0.
- cindyllm 2mo ago
  [dead]

paulpauper 2mo ago

This is why laypeople have not used Ai to make math breakthroughs . it has all been ppl with lots of knowledge already.

contubernio 2mo ago
People with lots of knowledge and (monetary and computing) resources.
najmbajwa123 2mo ago
Exactly, without knowledge, thats where ai slop comes in

annzabelle 2mo ago

That's been an interesting thing for me with Claude Code and working with non-technical coworkers. Often they ask me to write something that could be a simple script and takes maybe 3 prompts to get it right, and I find myself wondering why they can't just do it themselves. But then I have them asking me what a .md file is and how to open it, and they're unable to follow readme instructions to use commandline programs, so now I have to throw a "make a gui" onto every prompt.

najmbajwa123 2mo ago
true, basic tech literacy is also a real bottleneck.