3 ms·
It does have access to its thoughts. This is literally what thinking models do. They write out thoughts to a scratch pad (which you can see!) and use that as
by jmalicki 6mo ago
It does have access to its thoughts. This is literally what thinking models do. They write out thoughts to a scratch pad (which you can see!) and use that as part of the prompt.
- deleted 6mo ago[deleted]
- grey-area 6mo agoThey do not in fact do that. The ‘thoughts’ are not a chain of logic.
- mmoll 6mo agoIt doesn’t mean that these “thoughts” influenced their final decision the way they would in humans. An LLM will tell you a lot of things it “considered” and its final output might still be completely independent of that.
- jmalicki 6mo agoIts output quite literally is not independent, as the "thinking tokens" are attended to by the attention mechanism.
- fc417fc802 6mo agoIt's important to be aware that while those "thoughts" can be a useful aid for human understanding they don't seem to reliably reflect what's going on under the hood. There are various academic papers on the matter or you can closely inspect the traces of a more logically oriented question for yourself and spot impossible inconsistencies.
- sumeno 6mo agoYou have a fundamental misunderstanding of what the model is doing. It's not your fault though, you're buying into the advertising of how it works
- eleumik 6mo agoThose are a funny progress bar made by a micro model , is just ui