3 ms·

just do the calculation yourself: how many books is 500gb at, say, a few bits per character? more than all every written -- and so on perhaps apply a single d

just do the calculation yourself: how many books is 500gb at, say, a few bits per character? more than all every written -- and so on perhaps apply a single drop of scepticism to this credulity even, just ask chatgpt to repeat the first paragraph of some book -- say, a dickens novel

famouswaffles 3y ago
>how many books is 500gb at, say, a few bits per character? In what reality is a character taking up only a few bits ? Certainly isn't this one. Your denial is so weird. Why does ICL use Higher-Order Optimization Methods when it's just a lookup table ? https://arxiv.org/abs/2310.17086 https://arxiv.org/abs/2310.17086 How do you explain small language models that have much less space than the text they were trained on even with your nonsensical calculations ? LLMs are not lookup tables and there's plenty evidence to support that. You look insane insisting they are. >even, just ask chatgpt to repeat the first paragraph of some book -- say, a dickens novel Not only will this not work for the vast majority of books it sees during training, Why only the first paragraph ? are you not insisting they memorize everything ? Why can't it repeat the whole page ? the whole chapter ? the whole book ?
lbalazscs 3y ago
I actually did that, and I asked for the first paragraph from "Great Expectations", and the answer was correct. Of course the first paragraph of any famous novel is in itself famous, so I also asked for the first paragraph from Chapter XL, from the same novel. Interestingly the first 12 words were correct, but the rest was hallucination. Original: It was fortunate for me that I had to take precautions to ensure (so far as I could) the safety of my dreaded visitor; for, this thought pressing on me when I awoke, held other thoughts in a confused concourse at a distance. ChatGPT 3.5: "It was fortunate for me that I had to take precautions to ensure the safety of my treasure, for with such a boy as Compeyson on the loose, I was not likely to find much mercy at his hands. I had not told Herbert about the money, and I was glad to have the chance to count it before he came back. I found that I had still twenty-one sovereigns, six shillings, and sixpence left."

famouswaffles 3y ago

>how many books is 500gb at, say, a few bits per character? In what reality is a character taking up only a few bits ? Certainly isn't this one. Your denial is so weird. Why does ICL use Higher-Order Optimization Methods when it's just a lookup table ? https://arxiv.org/abs/2310.17086 https://arxiv.org/abs/2310.17086 How do you explain small language models that have much less space than the text they were trained on even with your nonsensical calculations ? LLMs are not lookup tables and there's plenty evidence to support that. You look insane insisting they are. >even, just ask chatgpt to repeat the first paragraph of some book -- say, a dickens novel Not only will this not work for the vast majority of books it sees during training, Why only the first paragraph ? are you not insisting they memorize everything ? Why can't it repeat the whole page ? the whole chapter ? the whole book ?

lbalazscs 3y ago

I actually did that, and I asked for the first paragraph from "Great Expectations", and the answer was correct. Of course the first paragraph of any famous novel is in itself famous, so I also asked for the first paragraph from Chapter XL, from the same novel. Interestingly the first 12 words were correct, but the rest was hallucination. Original: It was fortunate for me that I had to take precautions to ensure (so far as I could) the safety of my dreaded visitor; for, this thought pressing on me when I awoke, held other thoughts in a confused concourse at a distance. ChatGPT 3.5: "It was fortunate for me that I had to take precautions to ensure the safety of my treasure, for with such a boy as Compeyson on the loose, I was not likely to find much mercy at his hands. I had not told Herbert about the money, and I was glad to have the chance to count it before he came back. I found that I had still twenty-one sovereigns, six shillings, and sixpence left."