3 ms·

Steering can degrade and bias output Even with basic experiments I've done, it frequently introduces much more hallucination etc and allowing arbitrary steerin

• 0c3ca83 18d ago
  > Not to mention you just can't trust a model's judgement if the highest bidder chooses what it thinks But you already can't trust a model's judgement, and there's an entire industry around "GEO" or "AEO", which is basically poisoning training data so that AI mentions your products. The post above is the owner of the model taking a cut of that.
• charcircuit 18d ago
  Are the models you are using trained with steering being applied.
• achierius 17d ago
  But ultimately this is just an engineering problem, no? Yes, if you just hack steering into a model it's going to hurt performance, because doing so takes the model out of the regime it was trained for and validated in. But if that steering were to be accounted for (e.g. by rearchitecting the training process) there's no reason why it couldn't work. Diffusion-based image generation models, for example, 'by default' just generated random images out of the noise; steering (i.e. the user prompt) was added on as a secondary input, which models had to be re-trained in order to use.