3 ms·
> How much does it matter whether or not we can accurately observe, characterize, and validate the processes at work inside large reasoning models If you don't
by astrobe_ 2mo ago
> How much does it matter whether or not we can accurately observe, characterize, and validate the processes at work inside large reasoning models
If you don't or cannot, then it's the kind of sufficiently advanced technology that looks like magic. Except it is the problematic type of magic, like X-ray. They "magically" let you see the bones inside bodies. But you discover that you shouldn't do it too often only when the damage has already been done.
- notrealyme123 2mo agoThe assumption is that those reasoning traces are the lowest level. But we can't observe the reasoning-for-reasoning (the reasoning in a transformer block.) Saying that one is enough is pretty arbitrary.