>how many books is 500gb at, say, a few bits per character? In what reality is a character taking up only a few bits ? Certainly isn't this one. Your denial i

>how many books is 500gb at, say, a few bits per character? In what reality is a character taking up only a few bits ? Certainly isn't this one. Your denial is so weird. Why does ICL use Higher-Order Optimization Methods when it's just a lookup table ? https://arxiv.org/abs/2310.17086 https://arxiv.org/abs/2310.17086 How do you explain small language models that have much less space than the text they were trained on even with your nonsensical calculations ? LLMs are not lookup tables and there's plenty evidence to support that. You look insane insisting they are. >even, just ask chatgpt to repeat the first paragraph of some book -- say, a dickens novel Not only will this not work for the vast majority of books it sees during training, Why only the first paragraph ? are you not insisting they memorize everything ? Why can't it repeat the whole page ? the whole chapter ? the whole book ?