3 ms·
Because the optimization logic of tweaking the gpt transforms via training data and nudging weights and biases to achieve token prediction is not a path to util
by spywaregorilla 3y ago
Because the optimization logic of tweaking the gpt transforms via training data and nudging weights and biases to achieve token prediction is not a path to utilizing the full potential of the gpt transforms.
- criddell 3y agoI don’t see why not. I haven’t tried this, but I think I could describe a very basic logical CPU to it and have it execute opcodes and tell me the CPU state at each step (like Knuth’s MIX). Couldn’t I ask it to write programs and execute programs for this computer when doing so would help it give better answers?
- spywaregorilla 3y agoHighly unlikely to work in the near future. LLMs are bad state machines. But even if it did, you are basically describing programming in a contrived language, not the llm learning to do math. You can explicitly teach a model to follow an algorithm. But that's not the same as the algorithm being "learned" during training and "invoked" when asked what's 2 + 2?