2 ms·
I will assume you’re asking in good faith. Text continuation to match what people around the internet are saying requires the model internally to “program itse
by EGreg 2y ago
I will assume you’re asking in good faith.
Text continuation to match what people around the internet are saying requires the model internally to “program itself”. It is a non-von-neumann architecture but it encodes durable structures in BILLIONS of bytes, that’s like a codebase of gigabytes. It can pass the Bar exam, various pre-med exams, knows more than any particular human on every subject, and it’s only been a couple years.
The text continuation was the first interface, now they are doing multimodal. As it learns to predict the latent space for those things, it will be like predicting things in the real world. Training physical robots will become faster and easier too.
In short, this is basically a system to search through the space of all possible alhorithms in order to “reason about” a huge model of the world, enough to understand it and predict it with small loss.
AlphaZero used MCTS and can beat every human in chess now, and even every program written by a human, with beautiful gameplay. And even without AI, just the fact that the best performing algorithms are downloaded onto millions of computers is already multiplying the power far more than the 1% of humans who might do better in some area. So it can be better than 95% of humans at most tasks !