Humans have long and short term memory. And more recent theories from Neuroscience are that the brain is a 'prediction machine'. Predicting the next action. T

Humans have long and short term memory. And more recent theories from Neuroscience are that the brain is a 'prediction machine'. Predicting the next action. This whole concept of the large context windows seems to be filling a little bit of the gap where people could argue AI will never reach human intelligence. This article steered clear of sentience or consciousness. But what happens with a billion token context window? Or more. Humans have read a lot of books from the time they are in 1st grade to grad school. Lot of magazines, online, movies. What happens when the context window can absorb all of it. Seems like scaling still has room to grow.