2 ms·

maybe it is more like - one million transactions in good faith and then an actor or group does a Bad Faith action.. is the tolerance for 'perfect' in the way ?

maybe it is more like - one million transactions in good faith and then an actor or group does a Bad Faith action.. is the tolerance for 'perfect' in the way ? How can ordinary use be enabled while correctly blocking out malicious use, without crazy and ugly intrusive security engineering ? put another way, maybe the cost of a few bad faith actors is much higher than anticipated among people doing skilled work? How is that handled in a human way? I can guess that de-platforming based on whatever is much easier for the server side. This dynamic is how we got to the personal computer "revolution" thirty+ years ago

the8472 3y ago
Model weights are data, they shouldn't contain general purpose code with access to the host environment. Sure, any library/tool that harnesses models might contain malicious code. The models themselves should not be able to. This is more like distributing music as .exe files instead of .flac
- bongoman37 3y ago
  There are action models based on LLMs like Gorilla LLM or autoGPT that take a natural language input and convert that to API calls that could do all sorts of things from sending emails to performing stock trades.
  - the8472 3y ago
    To my knowledge it's still the harnesses that perform those actions, not baked into the weights (other than perhaps special tokens denoting that an action should be taken)
- Filligree 3y ago
  You'd hope, but a lot of models are still distributed as .pt files, which are Pickle — which means they can contain and execute arbitrary code on load. Some popular applications (Automatic1111 etc.) take measures against that, but the real fix is to ignore anything that isn't safetensors.
  - fragmede 3y ago
    Which, large platforms like Hugging Faces bear the responsibility to do what they can to prohibit python pickles.
- _ea1k 3y ago
  I think some of the models require remote code execution to support things like custom tokenizers.

the8472 3y ago

Model weights are data, they shouldn't contain general purpose code with access to the host environment. Sure, any library/tool that harnesses models might contain malicious code. The models themselves should not be able to. This is more like distributing music as .exe files instead of .flac

bongoman37 3y ago
There are action models based on LLMs like Gorilla LLM or autoGPT that take a natural language input and convert that to API calls that could do all sorts of things from sending emails to performing stock trades.
- the8472 3y ago
  To my knowledge it's still the harnesses that perform those actions, not baked into the weights (other than perhaps special tokens denoting that an action should be taken)
Filligree 3y ago
You'd hope, but a lot of models are still distributed as .pt files, which are Pickle — which means they can contain and execute arbitrary code on load. Some popular applications (Automatic1111 etc.) take measures against that, but the real fix is to ignore anything that isn't safetensors.
- fragmede 3y ago
  Which, large platforms like Hugging Faces bear the responsibility to do what they can to prohibit python pickles.
_ea1k 3y ago
I think some of the models require remote code execution to support things like custom tokenizers.