4 ms·
> historically ML research has used insecure file formats (like pickle) that has made these exploits fairly common Not to downplay this but it links to an old
by janalsncm 2y ago
> historically ML research has used insecure file formats (like pickle) that has made these exploits fairly common
Not to downplay this but it links to an old GitHub issue. Safetensors are pretty much ubiquitous. Without it sites like civitai would be unthinkable. (Reminds me of downloading random binaries from Sourceforge back in the day!)
Other than that, it’s a good write up. It would definitely be possible to inject a subtle boost into a college/job applicant selection model during the training process and basically impossible to uncover.
- sshh12 2y agoDefinitely! Although I'd be lieing if I said I haven't used pickle for a few models even relatively recently when safetensors wasn't convenient
- samtheprogram 2y agoTo clarify this further, pickle was more common ~10+ years ago I’d say? Hence the “historically” It wasn’t designed (well enough?) to be read safely so malware or other arbitrary data could be injected into models (to compromise the machine running the model, as opposed to the outputs like in the article), which safetensors was made to avoid.
- NitpickLawyer 2y ago> Safetensors are pretty much ubiquitous. Agreed. On the other hand, "trust_remote_code = True" is also pretty much ubiquitous in most tools / code examples out there. And this is RCE, as intended.