3 ms·
Consistency drive. The base model always wants to generate an output that's consistent with its context! It's what it was trained to do! Every LLM is just a ba
by ACCount37 1y ago
Consistency drive. The base model always wants to generate an output that's consistent with its context! It's what it was trained to do!
Every LLM is just a base model with a few things bolted on the top of it. And loops are extremely self-consistent. So LLMs LOVE their loops!
By the way, "no no no, that's a reasoning loop, I got to break it" is a behavior that larger models learn by themselves under enough RLVR stress. But you need a lot of RLVR to get to that point. And sometimes this generalizes to what looks like the LLM is just... getting bored by repetition of any kind. Who would have though.