4 ms·
How do you know how something with 5.3 trillion transistors works? If it's Micron's 2TB 3D-stacked NAND, we know exactly how it works, because we understand how
by danaris 4y ago
How do you know how something with 5.3 trillion transistors works? If it's Micron's 2TB 3D-stacked NAND, we know exactly how it works, because we understand how it was made.
Just putting a very high number on something doesn't mean it's automatically sentient, or even incomprehensible.
Just because we don't know the precise means by which ChatGPT arrives at one particular answer to a prompt doesn't mean we don't understand the underlying computations and data structures that make it up, and that they don't add up to sentience any more than Eliza does.
- spion 4y agoWe don't understand them. There is a key difference between building something from parts and running gradient descent to automatically find the network connectivity. That difference is that we don't understand the final result at all.
- throwaway290 4y agoWe understand exactly how it works. It just works in such a way that we cannot predict the outcome, which makes it pretty bad for many applications. If you can't explain how it works doesn't mean it's not understood how it works.
- spion 4y agoWe really don't know why LLMs have the most of their emergent capabilities. The entire GPT-3 paper was about being surprised about it.
- throwaway290 4y agoMaybe we don't disagree then. We know how LLM work. Parameters. Training data. Random number generator. That stuff. We don't know why it outputs what it outputs because rngs are notoriously unpredictable and we know it. So we are surprised but that in itself is unsurprising.