3 ms·

One interesting tidbit from this article that I haven't seen mentioned yet is that you can use glitch tokens to figure out what model someone is using behind th

One interesting tidbit from this article that I haven't seen mentioned yet is that you can use glitch tokens to figure out what model someone is using behind the scenes. Put a glitch token in a prompt, and see if it reacts normally or response with this kind of glitchy behavior.

sails 1y ago
Yes I thought so too. I wonder if it will mean more or less revealing of the models that are running agentic flows (we currently abstract with Fast/Smart) It is also possible that the first model calls other models, and you could reverse engineer the tool call structure by seeing when glitches occur based on different branches of the tool calling
willvarfar 1y ago
You can imagine LLM fingerprinting to be part of future pentest workflows where they identify the model and know it's weaknesses and vulnerabilities etc...
- varispeed 1y ago
  Will be cat and mouse game, because models can be trained to mimic response of other models under such conditions to obfuscate.
  - dboreham 1y ago
    See also: browser user-agent field.
lyu07282 1y ago
Isn't the only reason we can do that because we have access to the tokenizer? Do we have the Claude and Gemini tokens? I mean if they didn't publish that, would it defeat this attack?
- sebzim4500 1y ago
  I don't think we do, but maybe we could reverse engineer them by using the API to count the tokens in a bunch of strings. I think you can do that for free through both APIs.
joemazerino 1y ago
One reason to do this is to test what the underlying model is for a stealth model.