3 ms·
Models are hard to train. So if someone is offering you a large model, you can be fairly sure whoever is offering it has substantial compute resources. Turns
by londons_explore 4y ago
Models are hard to train.
So if someone is offering you a large model, you can be fairly sure whoever is offering it has substantial compute resources.
Turns out most bad guys don't yet have access to compute on the necessary scale.
That in turn means you can be fairly sure most big models you find online are in fact made by a trustworthy party, even if you download them from a random WeTransfer link...
- mtlmtlmtlmtl 4y agoIf it's in pickle format, containing arbitrary code, what's stopping a bad actor from simply generating a random untrained model, with a malicious payload attached? If you want to embed some sort of sneaky backdoor into a model, sure I buy this logic, but most malicious actors just want to take over your machine or something. No need to actually train a model to do that.
- aflag 4y agoThat said, it's probably a much better investment to do supply chain sort of attacks than trying to trick people into downloading your pickled model. Although, I would be surprised if there aren't pickled models with some malicious code out there. It doesn't feel like it's a very sought after target.
- londons_explore 4y agoThere are recent attackers who want to steal ssh keys of developers to do things like inject malicious code into any git repos that developer owns. That malicious code in turn, when pulled and installed by another developer does the same - so it's a worm that spreads via npm, makefiles, requirements.txt, etc. No reason it couldn't also spread by pickle files.
- aflag 4y agoI think plenty bad guys have the necessary resources, but models tend to be just a large array of numbers. I think the main reason it's unlikely is just that there's not that much value in messing with your model. What are they realistically going to get out of it?
- kwertyoowiyop 4y agoCan all those crypto-hashing computers now be used for ML training?