3 ms·
We are miles away from the fundamental constraint. We know that our current training methodologies are scandalously data inefficient compared to human/animal br
by schmidtleonard 1y ago
We are miles away from the fundamental constraint. We know that our current training methodologies are scandalously data inefficient compared to human/animal brains. Augmenting observations with dreams has long been theorized to be (part of) the answer.
- vanviegen 1y ago> current training methodologies are scandalously data inefficient compared to human/animal brains Are you sure? I've been ingesting boatloads of high definition multi-sensory real-time data for quite a few decades now, and I hardly remember any of it. Perhaps the average quality/diversity of LLM training data has been higher, but they sure remember a hell of a lot more of it than I ever could.