51 ms·
> From Socrates worrying that writing would destroy memory [...] Think of it as a partnership: the computer handles the organizational heavy lifting, while you
by Terr_ 2y ago
> From Socrates worrying that writing would destroy memory [...] Think of it as a partnership: the computer handles the organizational heavy lifting, while you focus on the thinking.
I'm less worried about memory per se and more about failing to think, or getting brainwashed/ring-led by a system with its own biases and quirks. Any sufficiently complex organizing is thinking!
Perhaps the simplest example is when quantities (numbers, easy to record) get a mental weight that overshadows and hides their dimensions (the definitions, what they really mean.) For example, a tendency to automatically assume a rising GDP number is an unqualified good sign.
Stuff like LLMs bring that into newer and more-dangerous territory, because the model also contains uncountable subtle biases from its training data, and even if you know it isn't aligned (heh) with your own mental models you can't reliably change it. Much like false-memories implanted by interrogators, patterns in those systems can and will leak into the users. Whenever we can't "think about how our thinking is being changed", I'd say that's axiomatically bad.
- Terr_ 2y agoP.S.: Lest anyone think I'm a Luddite--not that I think that appellation is actually that bad--an example where I would use an LLM would be to help me generate synonyms or alternate inputs to a more-traditional search for a discrete external piece of information. For example, I might remember a book with a jester playing a lute and singing about ogres, and I just can't find any clear search results, because it was actually a bard strumming a harp with a poem about giants. This is much less dangerous than just throwing every dang thing into LLM inputs (since prompt injection isn't a fluke, it's a way of life) or filtering the results back through the same model in an opaque fashion.