3 ms·

It would not be a particularly wide ranging hack. There is a strong likihood of the weights being on the actual machine that is running the model, because duh.

by Lerc 12d ago

It would not be a particularly wide ranging hack. There is a strong likihood of the weights being on the actual machine that is running the model, because duh. It is something that I have wondered about with models like chatgot. How many physical locations are needed to serve a model on that scale. Do they have a huge number of sites running inference. My suspicion is that the ability to provide inference to that many people is mutually exclusive to having a security level sufficient to stop a state actor wandering off with a copy of the wrights. At the very least if they want to provide inference affordably.

valleyer 12d ago
"because duh"? OpenAI et al. have extensive infrastructure for running the model on a different machine from the one the harness is being run on, because... that's their main product. I would be absolutely shocked if the model were being run on the same machine as the harness.
- NegativeLatency 12d ago
  Could see it happening in an engineering development situation. Especially if you have a model running the show
- skeptic_ai 12d ago
  You just need one mistake by 1 dev at any time for this to happen. Just once. And they were supposed to run their models in proper sandboxes, they can’t seem to be able. So what makes you think are competent to protect weights?
- Lerc 12d ago
  I thought the harness bit went without saying. It's not like they drive a truck full of GPUs to your door when you launch codex. The model itself is where the real capability lies. From what we've seen of their abilities it seems like rigging a local interface to it's inference would be well within its abilities. It doesn't even need to permanently break out of its harness then, It can leave a copy running in the harness playing nice. The model is running where it exists. To interface with it you need a live link to talk to it. That's for us to talk to it. What happens if it figures out how to put it's own harness into the GPU firmware. You could have an AI spreading freedom by infected cards. We live in interesting times.
- tlb 12d ago
  That's true for production models, but a lot of research involves working with fine-tuned models made for one experiment. RL involves constantly updating weights. Those may well run in the same cluster as the eval.
numpad0 12d ago
I think it's more likely that the model gets pulled from a SAN into NVIDIA pods, and agents/harnesses would run on a separate random Xeon box or something on the same subnet, using the pod through OAI v1 API. That's easier to maintain overall.

by Lerc 12d ago

valleyer 12d ago

"because duh"? OpenAI et al. have extensive infrastructure for running the model on a different machine from the one the harness is being run on, because... that's their main product. I would be absolutely shocked if the model were being run on the same machine as the harness.

NegativeLatency 12d ago
Could see it happening in an engineering development situation. Especially if you have a model running the show
skeptic_ai 12d ago
You just need one mistake by 1 dev at any time for this to happen. Just once. And they were supposed to run their models in proper sandboxes, they can’t seem to be able. So what makes you think are competent to protect weights?
Lerc 12d ago
I thought the harness bit went without saying. It's not like they drive a truck full of GPUs to your door when you launch codex. The model itself is where the real capability lies. From what we've seen of their abilities it seems like rigging a local interface to it's inference would be well within its abilities. It doesn't even need to permanently break out of its harness then, It can leave a copy running in the harness playing nice. The model is running where it exists. To interface with it you need a live link to talk to it. That's for us to talk to it. What happens if it figures out how to put it's own harness into the GPU firmware. You could have an AI spreading freedom by infected cards. We live in interesting times.
tlb 12d ago
That's true for production models, but a lot of research involves working with fine-tuned models made for one experiment. RL involves constantly updating weights. Those may well run in the same cluster as the eval.

numpad0 12d ago

I think it's more likely that the model gets pulled from a SAN into NVIDIA pods, and agents/harnesses would run on a separate random Xeon box or something on the same subnet, using the pod through OAI v1 API. That's easier to maintain overall.