Sure, but what I'm getting at is that even if you scale a GPT-4 style transformer up 1000x, it won't suddenly develop a human-like memory. It will still be read

Sure, but what I'm getting at is that even if you scale a GPT-4 style transformer up 1000x, it won't suddenly develop a human-like memory. It will still be read only, whereas there's no such thing as a read only human unless they are unfortunate enough to have severe brain damage.