5 ms·
It could be an attack surface. Maybe one day, when we find a chatbot online, we could let it guess a random number repeatedly, then accurately infer the underly
by maxloh 5mo ago
It could be an attack surface. Maybe one day, when we find a chatbot online, we could let it guess a random number repeatedly, then accurately infer the underlying model based on the resulting distribution.
- vidarh 5mo agoAt least some Claude models have a thing for numbers that contains "47"...
- smokel 5mo agoIn order to find out how real humans reply: Please guess a number between 1 and 100.
- bestouff 5mo ago69
- relativeadv 5mo agonice
- zulban 5mo ago101
- deleted 5mo ago[deleted]
- rithdmc 5mo ago√67
- orphea 5mo ago49.5
- Barbing 5mo agoSure!
- Ekaros 5mo agoe
- snerbles 5mo agoτ
- deleted 5mo ago[deleted]
- maxloh 5mo ago50
- pieter_mj 5mo ago137
- casper14 5mo agoand
- deleted 5mo ago[deleted]
- nom 5mo agonull
- Wonkey 5mo agopi
- HappMacDonald 5mo agoi+7up
- Grimblewald 5mo ago6*7=42
- alistairSH 5mo agoProto-Voight-Kampff Test?
- dijksterhuis 5mo agoi did something in my phd developing an attack against mozilla deepspeech. deepspeech used the CTC algorithm [0], which adds a “blank” character token to indicate repeats of a predicted normal alphabet character token over a sequence of audio/speech feature inputs. so "h==e=l===l===o====" maps to "hello" the model becomes super biased towards predicting that blank token. one speech feature is like 0.1 second of audio or less (can’t remember off hand). so there are a lot of alphabet character token repeats. off hand i seem to remember the predicted token distribution over like 1000 audio files was 50% blank token and then 50% distributed across the rest of the alphabet. as a result, you can get significantly smaller perturbations when generating adversarial examples. by like a factor of 2-4 or something. all you need to do is prioritise blank tokens in your target output. i spent 2 years trying to find a super clever attack. turns out all i needed to do was make one simple graph counting characters. xD [0]: https://en.wikipedia.org/wiki/Connectionist_temporal_classification https://en.wikipedia.org/wiki/Connectionist_temporal_classif...