4 ms·
Of course that's ok. That's also how brains work, delegating certain tasks to specific specialized parts and processes within the brain. But it's no longer a t
by spywaregorilla 3y ago
Of course that's ok. That's also how brains work, delegating certain tasks to specific specialized parts and processes within the brain.
But it's no longer a token prediction model.
- criddell 3y agoIf GPT transforms are Turing-complete[1], then why couldn't the ability to write and execute a program be part of a prediction model? [1]: https://aclanthology.org/2020.conll-1.37.pdf https://aclanthology.org/2020.conll-1.37.pdf
- spywaregorilla 3y agoBecause the optimization logic of tweaking the gpt transforms via training data and nudging weights and biases to achieve token prediction is not a path to utilizing the full potential of the gpt transforms.
- criddell 3y agoI don’t see why not. I haven’t tried this, but I think I could describe a very basic logical CPU to it and have it execute opcodes and tell me the CPU state at each step (like Knuth’s MIX). Couldn’t I ask it to write programs and execute programs for this computer when doing so would help it give better answers?
- spywaregorilla 3y agoHighly unlikely to work in the near future. LLMs are bad state machines. But even if it did, you are basically describing programming in a contrived language, not the llm learning to do math. You can explicitly teach a model to follow an algorithm. But that's not the same as the algorithm being "learned" during training and "invoked" when asked what's 2 + 2?