Maybe a bit, they aren’t just asking why scissors cut paper, but also why we landed on that design. What about it makes it ergonomic to hold and efficient, and

Maybe a bit, they aren’t just asking why scissors cut paper, but also why we landed on that design. What about it makes it ergonomic to hold and efficient, and more to the point, why does being sharp cut it? In the language model case, why can we model language this way so effectively and why does it follow these statistical patterns? It turns out that maybe a major reason has something to do with self description. This is also a more interesting question because we understand language less than we understand cutting paper, and also because the process humans used to design large language models is more indirect and alien than traditional industrial design. My takeaway was that people reading language modeling a person writing about themselves imagine it was written by a person, more than text written by people not about themselves. That’s an interesting trick! Describing human experiences in language makes people attribute the language to a human right now. Maybe this will change after a generation of people knowing about this, but that seems important to think about.