3 ms·

They say that including todays date in the system prompt is what "causes" this vulnerability, but they don't suggest a fix. Having the current date in the syste

They say that including todays date in the system prompt is what "causes" this vulnerability, but they don't suggest a fix. Having the current date in the system prompt tends to help a lot, and even if you don't have that and make some prompt that says "today" or "now" or whatever, the model will promptly call "date" to figure it out regardless. So what is the proposed "fix" here if there is any?

eru 2mo ago
Btw, if it's just the date (and nothing else) researchers or anyone interested can explore that by just providing future dates and see if anything changes.
- embedding-shape 2mo ago
  Well, you could have these "sleeper weights" only act on a specific day, rather than "$day + N", then people would have to prod every single future date to see if it could be malicious on that specific day, seems like a hassle. But I guess would be the only way really to detect it?
  - eru 2mo ago
    If it's only the day (and not time of day), then securing the next ten years would only take around ~3650 trials. Doesn't seem too bad. (And the can't make it too precise, if they want their attack to work even if you don't use the tool at exactly the right microsecond.)
    - embedding-shape 2mo ago
      Yeah, at a glance doesn't seem too bad, would depend on how fast you can actually do one prefill+full decode run. Say you do it with 10 seconds per iteration/test, then it's only ~10 hours to verify next ten years. Definitely doable.
      - eru 2mo ago
        You wouldn't need to run all ten years ahead of time. Every Dec 31 you could test out the coming year, if you wanted to.
- rcxdude 2mo ago
  Yeah, this seems pretty straightforward to test.
villish 2mo ago
The NSA’s fast16 driver silently modified nuclear simulation data on computers in Iran. This type of attack is possible with open weight models. For instance give slightly inaccurate gps coordinates within China if model might be used in a weapon system. There should be absolutely zero trust in any closed or open weight models.

eru 2mo ago

Btw, if it's just the date (and nothing else) researchers or anyone interested can explore that by just providing future dates and see if anything changes.

embedding-shape 2mo ago
Well, you could have these "sleeper weights" only act on a specific day, rather than "$day + N", then people would have to prod every single future date to see if it could be malicious on that specific day, seems like a hassle. But I guess would be the only way really to detect it?
- eru 2mo ago
  If it's only the day (and not time of day), then securing the next ten years would only take around ~3650 trials. Doesn't seem too bad. (And the can't make it too precise, if they want their attack to work even if you don't use the tool at exactly the right microsecond.)
  - embedding-shape 2mo ago
    Yeah, at a glance doesn't seem too bad, would depend on how fast you can actually do one prefill+full decode run. Say you do it with 10 seconds per iteration/test, then it's only ~10 hours to verify next ten years. Definitely doable.
    - eru 2mo ago
      You wouldn't need to run all ten years ahead of time. Every Dec 31 you could test out the coming year, if you wanted to.
rcxdude 2mo ago
Yeah, this seems pretty straightforward to test.

villish 2mo ago

The NSA’s fast16 driver silently modified nuclear simulation data on computers in Iran. This type of attack is possible with open weight models. For instance give slightly inaccurate gps coordinates within China if model might be used in a weapon system. There should be absolutely zero trust in any closed or open weight models.