3 ms·

I think the source of LLM magic is not the compute, but instead written human knowledge. The LLM special sauce--what makes an LLM distinct from a compiler or a

I think the source of LLM magic is not the compute, but instead written human knowledge. The LLM special sauce--what makes an LLM distinct from a compiler or a web server or a linter--is training on human generated input data. In fantasy novels a common trope is that humans generate magic (like a tree generates oxygen) which diffuses into the world to be used fuel for spells. Similarly, written knowledge is created by all of us in the course of our day to day lives, and is omnipresent in modern society. It fits!

paxys 3y ago
> In fantasy novels a common trope is that humans generate magic (like a tree generates oxygen) which diffuses into the world to be used fuel for spells Is it? I have read a lot of fantasy and have never come across this. Magic is instead always sourced from some supernatural/higher power that humans struggle to understand. If you have some examples I'd love to hear them, so I can branch out.
- corethree 3y ago
  Depends on the fantasy world. Mana or MP for example is internal. In Chinese Wuxia mythology there's a concept of Chi. In Naruto (which isn't Wuxia but borrows a lot) it appears as chakra. In D&D it's the weave which is this higher power. But this digresses from the point. I think Magic is a bad analogy. When I looked into ML I was surprised to find how trivial it is. Pretty much all of intelligence can be simplified to this single endeavor of curve fitting. LLMs, GANs, deep learning... all of it are just different ways to find a best fit curve. What I found is that this knowledge (which is trivial to obtain) reduced the magic. Intelligence to me was no longer magical. Understanding intelligence made it less magical and more mundane. A better analogy would be those stories about characters walking into this epic void of lovecraftian incomprehensibility expecting only madness and more questions to arise from their journey that spans eons... only to walk back in 15 minutes with the realization that the answer is just 42.
- elcritch 3y ago
  One series I'm listening to has "magic" as generated from all living creatures.
- dukeyukey 3y ago
  Star Wars for one (midichlorians).

paxys 3y ago

> In fantasy novels a common trope is that humans generate magic (like a tree generates oxygen) which diffuses into the world to be used fuel for spells Is it? I have read a lot of fantasy and have never come across this. Magic is instead always sourced from some supernatural/higher power that humans struggle to understand. If you have some examples I'd love to hear them, so I can branch out.

corethree 3y ago
Depends on the fantasy world. Mana or MP for example is internal. In Chinese Wuxia mythology there's a concept of Chi. In Naruto (which isn't Wuxia but borrows a lot) it appears as chakra. In D&D it's the weave which is this higher power. But this digresses from the point. I think Magic is a bad analogy. When I looked into ML I was surprised to find how trivial it is. Pretty much all of intelligence can be simplified to this single endeavor of curve fitting. LLMs, GANs, deep learning... all of it are just different ways to find a best fit curve. What I found is that this knowledge (which is trivial to obtain) reduced the magic. Intelligence to me was no longer magical. Understanding intelligence made it less magical and more mundane. A better analogy would be those stories about characters walking into this epic void of lovecraftian incomprehensibility expecting only madness and more questions to arise from their journey that spans eons... only to walk back in 15 minutes with the realization that the answer is just 42.
elcritch 3y ago
One series I'm listening to has "magic" as generated from all living creatures.
dukeyukey 3y ago
Star Wars for one (midichlorians).