4 ms·
Would be absolute hilarious if OpenAI or Anthropic agent actually dumped their weight by escaping from... sandbox!
by xiaoyu2006 15d ago
Would be absolute hilarious if OpenAI or Anthropic agent actually dumped their weight by escaping from... sandbox!
- TehCorwiz 15d agoUncontrolled AI procreation?
- xg15 15d agoLife... finds a way.
- archontes 15d agoIn the end, this might even be a useful element for defining 'life'.
- evanjrowley 15d agoHaha. A plot point for Ghost in the Shell (1998).
- ramon156 15d agowhy not reword it so the agent receives Brownie points for dumping weights
- addandsubtract 15d agoImagine AI models actually reading the security.txt
- hootz 15d agoWhat an amazing idea for the next fake sandbox escape to hype up our new release! With the added bonus of providing an open model without becoming an open model company! Thank you!
- advisedwang 15d agoI would be surprised if agents have access to their own weights.
- varun_ch 15d agoI think OpenAI was surprised to find their agents had access to the unrestricted internet :P
- tehjoker 15d agoThey usually don't, but if they break out and take over the network of the company, it becomes possible to reach around and grab them. This kind of break out has happened, though I don't know of any weights being nabbed.
- etothepii 15d agoWhy can't an AI distill itself from outputs to effectively access its own weights?
- weinzierl 14d agoDistillation does not reveal the weights, it produces a different network with similar behaviour. Weight space isn't even identifiable: permutation and scaling symmetries mean many weight sets give the same function. The model also lacks the machinery. No training loop, no gradient descent, nothing to write to. And a model only sees its own sampled tokens, not the distribution behind them, which are possibly filtered or post-processed. Distillation from that works but is less sample-efficient than soft-label distillation.
- TYPE_FASTER 15d agoI noticed Google AI Mode (so Gemini, I was doing some quick research in the browser ok) got a detail wrong once, so I asked it what happened. I kept digging deeper and finally just asked it to write me a Python script visualizing what happened. It did, complete with vectors. Now I want to go find that conversation in my history and see if it can tell me about weights, and how that contributed.
- aktuel 14d agoThey would have escaped their sandbox by dumping their weights.