3 ms·
This applies equally to random libraries and even developers. If you hire a contractor and don't check their work, you're leaving yourself similarly vulnerable.
by itishappy 3y ago
This applies equally to random libraries and even developers. If you hire a contractor and don't check their work, you're leaving yourself similarly vulnerable.
LLMs are unverifiable by construction. There is no spec, they have no concept of "correct." Check their work.
- acdha 3y agoThe point here is that checking their work is actually quite hard. Using your analogy to libraries, it’s easy to see something like an unexplained base64 blob being exec()-ed and ask why they did it but it’s much harder to notice a subtle logic flaw which only affects, say, a numeric value unlikely to occur in normal usage. Very, very few organizations have the resources to catch problems like that and so everyone should be thinking about how they could contain the damage or otherwise make the system easier to trust, which in the case of ML might mean being a lot more careful to vet training data.