3 ms·
Model weights are data, they shouldn't contain general purpose code with access to the host environment. Sure, any library/tool that harnesses models might con
by the8472 3y ago
Model weights are data, they shouldn't contain general purpose code with access to the host environment.
Sure, any library/tool that harnesses models might contain malicious code. The models themselves should not be able to.
This is more like distributing music as .exe files instead of .flac
- bongoman37 3y agoThere are action models based on LLMs like Gorilla LLM or autoGPT that take a natural language input and convert that to API calls that could do all sorts of things from sending emails to performing stock trades.
- the8472 3y agoTo my knowledge it's still the harnesses that perform those actions, not baked into the weights (other than perhaps special tokens denoting that an action should be taken)
- Filligree 3y agoYou'd hope, but a lot of models are still distributed as .pt files, which are Pickle — which means they can contain and execute arbitrary code on load. Some popular applications (Automatic1111 etc.) take measures against that, but the real fix is to ignore anything that isn't safetensors.
- fragmede 3y agoWhich, large platforms like Hugging Faces bear the responsibility to do what they can to prohibit python pickles.
- _ea1k 3y agoI think some of the models require remote code execution to support things like custom tokenizers.