4 ms·

I agree that we can define a safe serialization format for models with given assumptions about architecture. I.e. when the model is just the matrices and cannot

I agree that we can define a safe serialization format for models with given assumptions about architecture. I.e. when the model is just the matrices and cannot supply custom inference code needed to process the matrices. But, I expect we're going to have additional rounds of insecure practice just like we've had in every other popularized tech movement. People are going to develop frameworks with code-injection flaws, where they assume mode outputs (tokens) can be trusted and can contain executable content. Then, the inscrutable and untrustworthy models are going to be a problem as well. Think everything from MS Office macro abuse to human drivers blindly following their GPS guidance into a lake. This can and most probabaly will be repeated with AI models, due to the prevalence of naive and over-trusting practitioners and consumers.

Me1000 3y ago
100%. Software takes time to make it secure, since after all it’s written by us flawed humans. The runtime that consumes the model files might have bugs, but those will be fixed over time. How people use model outputs (or inputs; I.e prompt injections) is a whole other area ripe for exploits, especially while these technologies are being adopted by people who don’t really understand them. But I view this as fundamentally different than the above. A model format can be secure in that it won’t just randomly delete files on your computer. This is a computer science problem and thus provably securable. I guess that was the point I was trying to make when differentiating why they work vs how they work.