3 ms·
Models aren’t trained as much from random internet text as they used to in pre-2024 era. Like they are, but specialized datasets get more attention.
by tokioyoyo 3mo ago
Models aren’t trained as much from random internet text as they used to in pre-2024 era. Like they are, but specialized datasets get more attention.
- MikeNotThePope 3mo agoYour model/harness will indiscriminately do web searches to get answers. I believe that’s where the real risk is.
- brunoarueira 3mo agoI don't know for sure, but will it require something like with high "SEO" rankings to fake the reputation of the sources? So if the LLM search for a specific spreaded knowledge splitted across many bad sites, it can poison the model maybe.
- Terr_ 3mo agoYou mean, how difficult will it be for an attacker to get their text seen by the LLM that does arbitrary searches? It could be as simple as a malicious prospectus for AcmeCo, and then try to get AcmeCo on the radar so that the LLM-tools find your document and incorporate it. The malicious bits don't even need to be AcmeCo-related, they could be to pump (or dump) practically anything.