4 ms·
In system design there is something called resonant amplification and what you are describing is very similar. The biases of the model are amplified with each i
by haltist 3y ago
In system design there is something called resonant amplification and what you are describing is very similar. The biases of the model are amplified with each iteration and the end result is that the system converges onto the patterns recognizable and amplified by its architecture. If you know about impulse and frequency analysis then an AI system can be considered to be a signal processor that amplifies and attenuates certain frequencies in the input/impulse. Running an LLM in a loop always ends up with nonsense as the final output.
- lazystar 3y agoSo if left unchecked, the thing we built in man's attempt to play god could result in gibberish? sounds kind of like the tower of babel; seems humankinds only defense would be creating a new language that the machines can't infiltrate
- ben_w 3y agoAnother defence is to check them, which is why the thumbs up, thumbs down, and regenerate buttons are there on the ChatGPT user interface.
- haltist 3y agoYou can try the experiment yourself. Take any open source LLM and then feed the output back into it in a loop and you'll see what I mean. Most LLMs diverge and lose coherence in less than 100 iterations.
- lazystar 3y agoright, im thinking in terms of the world at large - what happens after 5 years of chatgpt, what will the internet and human communication be like in 2030?