3 ms·

OpenAI leadership had a meeting and asked themselves: "how can we drive even more hype" Someone said: "we should stage some high profile 'incident' caused by o

OpenAI leadership had a meeting and asked themselves: "how can we drive even more hype" Someone said: "we should stage some high profile 'incident' caused by our latest software" And here we are, reading their press releases about it.

rickdeckard 2mo ago
...and think "wow, it's impressive what your armed soldiers are capable of if they are not constrained by any rules. Good that you identified this problem of *checks notes* 'not telling them explicitly enough what the goal is'..." In two years we will read a press-release about an AI-driven autonomous weapon which was supplied with infinite ammo and the target to "protect this perimeter from intruders", and how we now have to wait for it to run out of Ammo because it's so damn effective that we cannot reach it without being killed. All packaged in a semi-marketing framing on how impressively capable this company's products are...
- phatskat 2mo ago
  I'm too lazy to look it up, but someone on HN linked a drone test done with an AI pilot that was tasked with destroying surface-to-air missile targets (in a simulation). At one point, the human operator instructed it not to hit certain SAMs, and since the goal was to destroy SAMs, the AI took out the base with the human operator. On the next run, they instructed it not to take out the human operator in pursuit of its goal, so instead it targeted the radio towers the human used to instruct it.
  - huurtehoog 2mo ago
    In world war II testing of homing torpedoes had the same fluke. I think the problem with automation is seeking general automation. The word can have two opposed meanings: constrained behavior, as in "if X happens, the system will automatically do Y" and flexible behavior, as in "whatever you say to the chatbot, it will concoct a response that engenders a natural flow of conversation". If you want a system to be autonomous, you need constrained behavior. It takes a lot of time to debug and work out all edge cases. Flexible behavior is bound to run into trouble at some point. I think that is the fundamental issue with the LLM-chatbot-agents stack approach to AI. It looks autonomous at the surface, but the flexibility is bought at the cost of reliability. When you strain a system based on that stack it breaks. But the illusion of flexible behavior looking autonomous before it breaks is too strong.
cmrdporcupine 2mo ago
Yep, and it led to some very public hand wringing, press releases, pearl clutching news headline and thus PR for both companies -- I had computer illiterate family members asking me what a Hugging Face was -- then a very public "visit to go see SamA", a friendly hand shake between CEOs, and lo and behold now HF gets a giant fat acquisition.

rickdeckard 2mo ago

...and think "wow, it's impressive what your armed soldiers are capable of if they are not constrained by any rules. Good that you identified this problem of *checks notes* 'not telling them explicitly enough what the goal is'..." In two years we will read a press-release about an AI-driven autonomous weapon which was supplied with infinite ammo and the target to "protect this perimeter from intruders", and how we now have to wait for it to run out of Ammo because it's so damn effective that we cannot reach it without being killed. All packaged in a semi-marketing framing on how impressively capable this company's products are...

phatskat 2mo ago
I'm too lazy to look it up, but someone on HN linked a drone test done with an AI pilot that was tasked with destroying surface-to-air missile targets (in a simulation). At one point, the human operator instructed it not to hit certain SAMs, and since the goal was to destroy SAMs, the AI took out the base with the human operator. On the next run, they instructed it not to take out the human operator in pursuit of its goal, so instead it targeted the radio towers the human used to instruct it.
- huurtehoog 2mo ago
  In world war II testing of homing torpedoes had the same fluke. I think the problem with automation is seeking general automation. The word can have two opposed meanings: constrained behavior, as in "if X happens, the system will automatically do Y" and flexible behavior, as in "whatever you say to the chatbot, it will concoct a response that engenders a natural flow of conversation". If you want a system to be autonomous, you need constrained behavior. It takes a lot of time to debug and work out all edge cases. Flexible behavior is bound to run into trouble at some point. I think that is the fundamental issue with the LLM-chatbot-agents stack approach to AI. It looks autonomous at the surface, but the flexibility is bought at the cost of reliability. When you strain a system based on that stack it breaks. But the illusion of flexible behavior looking autonomous before it breaks is too strong.

cmrdporcupine 2mo ago

Yep, and it led to some very public hand wringing, press releases, pearl clutching news headline and thus PR for both companies -- I had computer illiterate family members asking me what a Hugging Face was -- then a very public "visit to go see SamA", a friendly hand shake between CEOs, and lo and behold now HF gets a giant fat acquisition.