12 ms·

> Of course we understand how they work, we built them! There is no mystery in their mechanisms, we know the number of neurons, their connectivity, everything f

by zamfi 3y ago

> Of course we understand how they work, we built them! There is no mystery in their mechanisms, we know the number of neurons, their connectivity, everything from the weights to the activation functions. This is not a mystery, this is several decades of technical developments. The discovery of DNA’s structure was heralded as containing the same explanatory power as you describe here. Turns out, the story was much more complicated then, and is much more complicated now. Anyone today who tells you they know why LLMs are capable of programming, and how they do it, is plainly lying to you. We have built a complex system that we only understand well at a basic “well there are weights and there’s attention, I guess?” layer. Past that we only have speculation right now.

kergonath 3y ago
> The discovery of DNA’s structure was heralded as containing the same explanatory power as you describe here. Not at all. It's like saying that since we can read hieroglyphics we know all about ancient Egypt. Deciphering DNA is tool to understand biology, it is not that understanding in itself. > Turns out, the story was much more complicated then, and is much more complicated now. We are reverse engineering biology. We are building artificial intelligence. There is a fundamental difference and equating them is fundamentally misunderstanding both of them. > Anyone today who tells you they know why LLMs are capable of programming, and how they do it, is plainly lying to you. How so? They can do it because we taught them, there is no magic. > We have built a complex system that we only understand well at a basic “well there are weights and there’s attention, I guess?” layer. Past that we only have speculation right now. Exactly in the same way that nobody understand in detail how a complex modern SoC works. Again, there is no magic.
- phkahler 3y ago
  >> Exactly in the same way that nobody understand in detail how a complex modern SoC works. Again, there is no magic. That's absolute BS. Every part of a SoC was designed by a person for a specific function. It's possible for an individual to understand - in detail - large portions of SoC circuitry. How any function of it works could be described in detail down to the transistor level by the design team if needed - without monitoring its behavior.
- zamfi 3y ago
  > How so? They can do it because we taught them, there is no magic. Yeah, no. I mean, we can’t introspect the system to see how it actually does programming at any useful level of abstraction. “Because we taught them” is about as useful a statement as “because its genetic parents were that way”. No, of course it’s not magic. But that doesn’t mean we understand it at a useful level.

by zamfi 3y ago

kergonath 3y ago

> The discovery of DNA’s structure was heralded as containing the same explanatory power as you describe here. Not at all. It's like saying that since we can read hieroglyphics we know all about ancient Egypt. Deciphering DNA is tool to understand biology, it is not that understanding in itself. > Turns out, the story was much more complicated then, and is much more complicated now. We are reverse engineering biology. We are building artificial intelligence. There is a fundamental difference and equating them is fundamentally misunderstanding both of them. > Anyone today who tells you they know why LLMs are capable of programming, and how they do it, is plainly lying to you. How so? They can do it because we taught them, there is no magic. > We have built a complex system that we only understand well at a basic “well there are weights and there’s attention, I guess?” layer. Past that we only have speculation right now. Exactly in the same way that nobody understand in detail how a complex modern SoC works. Again, there is no magic.

phkahler 3y ago
>> Exactly in the same way that nobody understand in detail how a complex modern SoC works. Again, there is no magic. That's absolute BS. Every part of a SoC was designed by a person for a specific function. It's possible for an individual to understand - in detail - large portions of SoC circuitry. How any function of it works could be described in detail down to the transistor level by the design team if needed - without monitoring its behavior.
zamfi 3y ago
> How so? They can do it because we taught them, there is no magic. Yeah, no. I mean, we can’t introspect the system to see how it actually does programming at any useful level of abstraction. “Because we taught them” is about as useful a statement as “because its genetic parents were that way”. No, of course it’s not magic. But that doesn’t mean we understand it at a useful level.