The popular LLMs definitely do have a bias toward certain phrases. For local models, someone was annoyed enough to make a modified sampler that backtracks when

The popular LLMs definitely do have a bias toward certain phrases. For local models, someone was annoyed enough to make a modified sampler that backtracks when the model outputs a "GPT-ism" and regenerates with the output token probabilities tweaked to avoid it: https://github.com/sam-paech/antislop-sampler/ https://github.com/sam-paech/antislop-sampler/