4 ms·
memory, internal iteration and scratchpads can and have been added. Foundation models don't include this yet, but it's not unexplored territory. Online learning
by the8472 3y ago
memory, internal iteration and scratchpads can and have been added. Foundation models don't include this yet, but it's not unexplored territory. Online learning is more difficult but I'd argue that various fine-tuning and control approaches are slowly working towards that.
So what is a brain other than a very complex non-linear function which takes inputs, iterates on them, integrates them with memory and eventually generates output?
> is that it doesn't truly understand anything at all, but is just instead (in simple terms) a large matrix of inputs to outputs
The first step to defeating the tiger is to realize that it cannot hurt you, for it is only made of simple atoms.
- mgn115 3y ago> So what is a brain other than a very complex non-linear function which takes inputs, iterates on them, integrates them with memory and eventually generates output? A thing that processes and embodies sensory input, makes informed guesses about other people's intentions and values, grows tumors, worries about the dirty dishes.... How many other "complex non-linear functions" are made of meat?
- ukuina 3y ago>> is that it doesn't truly understand anything at all, but is just instead (in simple terms) a large matrix of inputs to outputs > The first step to defeating the tiger is to realize that it cannot hurt you, for it is only made of simple atoms. This is now my favorite rebuttal when people claim LLMs are simply some form of math.