3 ms·
Aren't transformers universal function approximators? It seems pretty easy to see executive function as a simple computation. So it would be trivially true that
by roenxi 4mo ago
Aren't transformers universal function approximators? It seems pretty easy to see executive function as a simple computation. So it would be trivially true that a sufficiently large transformer could model executive function because it could approximate [current transformer] + [an approximation of the executive function algorithm] + [whatever bloat is needed to store state in a transformer].
It seems hard to come up with an argument that executive function can't possibly be approximated with an algorithm. Executive function is basic once the clustering into objects part of the process is done. The only real questions are whether a transformer of sufficient scale is feasible on current hardware and if the engineers with access to the hardware have figured out what to train for yet.
- fc417fc802 4mo ago> whether a transformer of sufficient scale is feasible on current hardware Just because a particular approach (such as a neural network or transformer) can approximate something doesn't necessarily mean it can do so efficiently. However I share what I infer to be your suspicion that executive function can in fact be easily modeled. I think it's likely to develop on its own depending on the training methodology used.