3 ms·

>Actual calculation on values - not really. Uh yes really. https://www.alignmentforum.org/posts/N6WM6hs7RQMKDhYjB/a-mechanistic-interpretability-analysis-of-gr

>Actual calculation on values - not really. Uh yes really. https://www.alignmentforum.org/posts/N6WM6hs7RQMKDhYjB/a-mechanistic-interpretability-analysis-of-grokking https://www.alignmentforum.org/posts/N6WM6hs7RQMKDhYjB/a-mec... >You can tell which mode is used by using large random numbers and seeing what kind of error happens. Missed digits - it's not actually doing the calculation. Missed digits doesn't mean anything other than the wrong calculation. The assertion that it isn't doing calculation is absurd. >Also, we don't know how much gpt4 cheats at math. OpenAI may be replacing those bits with external calculation Pick one. Either it's so bad it's obviously not doing any calculations at all or it's so good you suspect Open AI are passing numbers through a calculator behind the scenes. Not only is the latter baseless speculation, the 2 options are mutually exclusive.

viraptor 3y ago
> Uh yes really. This is based on integer addition up to 113 with a data set that only trains that. Yes, you can achieve that in a special case. No, LLMs don't achieve that. Conclusion from the post you linked: "Epistemic status: I feel confident in the empirical results, but the generalisation to non-toy settings is more speculative" > Pick one. Either it's so bad it's obviously not doing any calculations at all or it's so good you suspect Open AI are passing numbers through a calculator behind the scenes. That's not the claim. I'm saying LLMs don't do real, precise calculation, but can do digit level operation and some implementations of systems around an LLM could pass the problem through a calculator. LLMs can rewrite the problem into a function, but the execution of it doesn't happen inside LLMs. If you squint it's like the json output - LLMs can kind of do it most of the time, but you can implement a system around the LLM which ensures that any json output will be valid.

viraptor 3y ago

> Uh yes really. This is based on integer addition up to 113 with a data set that only trains that. Yes, you can achieve that in a special case. No, LLMs don't achieve that. Conclusion from the post you linked: "Epistemic status: I feel confident in the empirical results, but the generalisation to non-toy settings is more speculative" > Pick one. Either it's so bad it's obviously not doing any calculations at all or it's so good you suspect Open AI are passing numbers through a calculator behind the scenes. That's not the claim. I'm saying LLMs don't do real, precise calculation, but can do digit level operation and some implementations of systems around an LLM could pass the problem through a calculator. LLMs can rewrite the problem into a function, but the execution of it doesn't happen inside LLMs. If you squint it's like the json output - LLMs can kind of do it most of the time, but you can implement a system around the LLM which ensures that any json output will be valid.