3 ms·
You are confusing the entirety of all text corpus to _all data_. The next step is multi-modal data. Think videos, sounds, touch, etc. An 18 year old in fact "
by stevenhuang 4y ago
You are confusing the entirety of all text corpus to _all data_.
The next step is multi-modal data. Think videos, sounds, touch, etc.
An 18 year old in fact "trained" on much more input data than GPT4, orders of magnitudes more, considering the complete informational content of our 5 senses and implicit structure of our brains "learned" through evolution.
Not to mention recent research in LLM scaling (chinchilla) reveals how our current models are undertrained and over-parameterized.
We have not plateaued yet.
- PaulDavisThe1st 4y ago> The next step is multi-modal data. Think videos, sounds, touch, etc. Remember that the data must be annotated. This puts limits on what can be usefully ingested.
- stevenhuang 4y agoNot necessarily, see the developments in self supervised learning. It's what's relied on for the majority of the RLHF phase in current LLMs.
- mlboss 4y agoWhat if it can annotate itself ? With some feedback loop it can learn the correct annotation by experiments like a child. Earlier there was a comment of an agent learning to run a python program by itself and in the process of writing code to file and install python interpreter.