3 ms·

Are you telling me that when I run software on my computer I could potentially be running software on my computer?

by bongodongobob 2y ago

foobiekr 2y ago
Nobody expects a model file to be code thrat executes whatever.
- foolishbard 2y ago
  I'd say anybody who is working in the field has this expectation. But the outside observer who is excited to try a new model does not expect it.
- squigz 2y ago
  This is like expecting any random binary to only do what it 'claims' to do.
  - erikerikson 2y ago
    No, this is like PDFs that can contain and will execute PostScript but without a sandbox and with more general libraries
- LunaSea 2y ago
  That is borderline incompetence
hn92726819 2y ago
This is more akin to downloading a jpeg and the jpeg running arbitrary code. Models should be like jpegs and I believe safetensors treat them that way, while the old pickle format didn't.
- catgary 2y ago
  The weights are like jpegs, the model itself is still a piece of software that executes code.
- amelius 2y ago
  You typically need more than the model to actually use it, as you might need to "massage" the data into the right form before and after the model call.
- bravetraveler 2y ago
  I think the gap between 'should' and [short-lived] reality is the point - that's obviously the reasonable setup, but pickle happened. In the quest of being first or whatever, amateur rediscoveries are happening. Consistently. I'm making a disgusting amount of money in the field by doing day-one packaging work any commodity project would also be doing. They just wouldn't decide to redefine things in a clearly worse way. I don't actually know jack about AI, yet here we are. The hype/gold rush is real; competence in the shared fundamentals can go a long way. The trick is knowing/applying the basics for the wizards. There's a steep premium so they can play around, apparently. Case in point: most of these humongous container images and how they're naively moved around Now that more people know how to publish (containers/packages/articles), we're seeing more drivel published. It's ridiculous yet inevitable. I just wonder how much value the rush provides. IIRC 'safetensors' aren't necessarily even that safe. I could have sworn I read they're fairly nebulous. Grain of salt etc
  - fragmede 2y ago
    no such exploits have been publicised against known safetensors, so I'd need more than an unqualified Internet rumor to consider them unsafe. Not saying it's not possible but the entire point of them is that they're a blob of numbers, and not a known exploitable bit of python. How're you making money? is it a contracting via network thing or can anybody apply?
    - bravetraveler 2y ago
      I tried to explain that it's nothing more than a rumor. I wrapped that in poor recollection as much as I could. Three absolving statements for the one! Did you, somehow, feel I was trying to convince? I'm fishing for education. I'm making my money doing 'SRE', which really, amounts to doing housekeeping for the AI people. Anyone can apply. What are you insinuating? Say it. My situation (other than comp) isn't special, they generally need help. I don't really need to know about what they're running to keep the systems online and running well. Another day, different workload. I may bother to learn a thing or two about it... but it's not a requirement. At all. Same toys all the way down. It's shiny old news at this level. I don't mean to belittle it. It's jazzy stuff. I just build the boring parts - OS, VLANs, etc. My 'ire'/elbowing is that 'they' (the AI folks), almost pathologically, refuse to engage with the basics like Python packaging. It's with love, honestly.
pheatherlite 2y ago
Data intended to be read as instructions for the interpreter or the cpu, is a whole different ballgame than data intended to convey values of something. High order sparse/dense matrices serialized in some xyz format is what most people think of when they hear the word "model". To switch it up and send some arbitrary python file and execute it on the client is a security nightmare. This outrageous.

Are you telling me that when I run software on my computer I could potentially be running software on my computer?

foobiekr 2y ago

Nobody expects a model file to be code thrat executes whatever.

foolishbard 2y ago
I'd say anybody who is working in the field has this expectation. But the outside observer who is excited to try a new model does not expect it.
squigz 2y ago
This is like expecting any random binary to only do what it 'claims' to do.
- erikerikson 2y ago
  No, this is like PDFs that can contain and will execute PostScript but without a sandbox and with more general libraries
LunaSea 2y ago
That is borderline incompetence

hn92726819 2y ago

This is more akin to downloading a jpeg and the jpeg running arbitrary code. Models should be like jpegs and I believe safetensors treat them that way, while the old pickle format didn't.

catgary 2y ago
The weights are like jpegs, the model itself is still a piece of software that executes code.
amelius 2y ago
You typically need more than the model to actually use it, as you might need to "massage" the data into the right form before and after the model call.
bravetraveler 2y ago
I think the gap between 'should' and [short-lived] reality is the point - that's obviously the reasonable setup, but pickle happened. In the quest of being first or whatever, amateur rediscoveries are happening. Consistently. I'm making a disgusting amount of money in the field by doing day-one packaging work any commodity project would also be doing. They just wouldn't decide to redefine things in a clearly worse way. I don't actually know jack about AI, yet here we are. The hype/gold rush is real; competence in the shared fundamentals can go a long way. The trick is knowing/applying the basics for the wizards. There's a steep premium so they can play around, apparently. Case in point: most of these humongous container images and how they're naively moved around Now that more people know how to publish (containers/packages/articles), we're seeing more drivel published. It's ridiculous yet inevitable. I just wonder how much value the rush provides. IIRC 'safetensors' aren't necessarily even that safe. I could have sworn I read they're fairly nebulous. Grain of salt etc
- fragmede 2y ago
  no such exploits have been publicised against known safetensors, so I'd need more than an unqualified Internet rumor to consider them unsafe. Not saying it's not possible but the entire point of them is that they're a blob of numbers, and not a known exploitable bit of python. How're you making money? is it a contracting via network thing or can anybody apply?
  - bravetraveler 2y ago
    I tried to explain that it's nothing more than a rumor. I wrapped that in poor recollection as much as I could. Three absolving statements for the one! Did you, somehow, feel I was trying to convince? I'm fishing for education. I'm making my money doing 'SRE', which really, amounts to doing housekeeping for the AI people. Anyone can apply. What are you insinuating? Say it. My situation (other than comp) isn't special, they generally need help. I don't really need to know about what they're running to keep the systems online and running well. Another day, different workload. I may bother to learn a thing or two about it... but it's not a requirement. At all. Same toys all the way down. It's shiny old news at this level. I don't mean to belittle it. It's jazzy stuff. I just build the boring parts - OS, VLANs, etc. My 'ire'/elbowing is that 'they' (the AI folks), almost pathologically, refuse to engage with the basics like Python packaging. It's with love, honestly.

pheatherlite 2y ago

Data intended to be read as instructions for the interpreter or the cpu, is a whole different ballgame than data intended to convey values of something. High order sparse/dense matrices serialized in some xyz format is what most people think of when they hear the word "model". To switch it up and send some arbitrary python file and execute it on the client is a security nightmare. This outrageous.