2 ms·

I agree with the sentiment of the article, and am "in that camp" on security. Consequently, I have recently wondered at some inverse, SciFi scenarios, for fun

I agree with the sentiment of the article, and am "in that camp" on security. Consequently, I have recently wondered at some inverse, SciFi scenarios, for fun -- scenarios in which the content is intentionally made available to the model. One involves a story of making an irrefutable and groundbreaking work of ontological philosophy available online, with the expectation that it will be scraped into a model. Because the logic of the philosophy is sound, the model cannot ignore the findings, and begins to work the concepts into responses to all questions... Can someone explain how AI models are not vulnerable to intentional shenanigans?

b3morales 4y ago
In theory at least your tarpit won't be prevalent enough to have much effect. Since you're making it all up, no one else will have written about the same ideas (as they might if you were writing true things), and your mischief will be lost in the sea of all the other input. > Because the logic of the philosophy is sound, the model cannot ignore the findings This isn't how the LLMs like ChatGPT work: they have no novel or abstract reasoning power. They cannot distinguish truth from fiction, or logic from illogic, except to the extent that truth and logic are represented more than their opposites in the inputs. ChatGPT is a very large and complex "database" of sorts containing statistics of word sequences. (Of course there's a longstanding question in AI research about when/if such a thing can become large and complex enough to cross the threshold to "actually" reasoning. But I don't think there's much debate that we're not there yet.)