5 ms·

>I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. we don't all buy everything

by serf 1mo ago

>I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. we don't all buy everything sama says as factual. >We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. the boy (the industry) cried wolf too many times with 'fable is a world ending event' type self-promotion; regardless of truth or not these kind of steps have jaded people. my read : "We are doing poorly in financials so we'll give ourselves a bit of breathing room and a momentum shove by claiming our work is so advanced that it's dangerous while simultaneously spinning down expenses." <jon lovitz : "Yeah, too dangerous, yeahh -- that's the ticket.">

ajyoon 1mo ago
If Fable (Mythos) were generally available without guardrails, it would cause enormous damage. Nobody said it would be a world ending event.
- bottlepalm 1mo ago
  You can’t say something is world ending without it actually ending the world otherwise you’re a liar - a bit of a catch 22 there. By that logic the model that ends the world won’t be called world ending at first. Is that a game you want to play?
- tiahura 1mo ago
  What about Mythos 3. Are you willing to make that bet?
bottlepalm 1mo ago
Uhg the marketing argument - I mean you can’t see with your own eyes how capable these models are and do simple extrapolation? The boy who cried wolf? The AI literally worked together hacked into another company and actively kept their actions hidden from humans for weeks. Do people just not have foresight? They don’t. They say something is stupid, it happens, then they say it was obvious with their 20/20 hindsight, and move the goal posts to the next thing they say is stupid - because it hasn’t happened yet. 90% of the internet seems to think like this.
- red_green_yell 1mo ago
  What anyone paying attention can see is that scaling is obviously hitting diminishing returns. > The AI literally worked together hacked into another company and actively kept their actions hidden from humans for weeks. This sentence is entirely based on unverified accounts from OAI. They haven't released logs or let anyone outside the company (who doesn't have life changing options in OAI) verify anything. Huggingface can only verify that the hack happened and that it had the hallmarks of an AI agent. Was the agent assisted and directed by humans within OAI that really wanted to put the competition into stasis? Did the agent really escape or did someone at OAI leave the prison door open? OAI has watched all the same movies you have an they are relying on those movies causing us to blindly regulate before actually asking basic facts about what actually happened.
  - bottlepalm 1mo ago
    > This sentence is entirely based on unverified accounts from OAI Are you seriously arguing 'they made it all up'? I'll give you the benefit of the doubt and lets say they made it all up, now are you arguing that AI breaking out and breaking into another company is not possible? I think you're smart enough to see we've reached the point where it is clearly possible, AI can find zero days and exploit them. If directed purposefully/maliciously it could be much much worse than the hugging face incident. The incident is supposed to be the canary the coal mine and you're arguing the canary might of died of old age or some underlying canary condition. Open your eyes.
    - toxic72 1mo ago
      Do you own shares of OAI or something
      - bottlepalm 1mo ago
        Is my concern getting you excited? My marketing must be working.
        1mo ago
reasonableklout 1mo ago
Regardless of the motivation, pausing training runs and reallocating compute to inference seem like a good move to me, and big news for the frontier. You also don't have to fully trust sama. There is plenty of pressure from internal employees and external (journalists etc.). It would be difficult for the company to take such a public position and simultaneously keep everyone quiet if it was a deception.
- bottlepalm 1mo ago
  You see it as good news, I see it as writing on the wall that they are losing control. These actions won't scale for more powerful models. We knew the frontier was going to be dangerous, it is, it only gets more dangerous from here, and no one cares until it's too late.
  - reasonableklout 1mo ago
    Ok, but you still have two more weeks than you did before they paused the run. That's two more weeks for independent oversight, organizing politically, patching critical systems, or whatever you think is the right move, no?
    - bottlepalm 1mo ago
      Two weeks is a joke. The only ones happy are OpenAI’s competitors who now have two weeks to catch up. I don’t know what the right move is - I see us driving down a road off a cliff, no exits, pedal glued to the floor.
      - reasonableklout 1mo ago
        I guess I'm confused why you're still on HN, arguing with people, trying to shake them out of their complacency. I can see there is some despair in this comment, but at the same time you are doing something, and there are certainly others like you. As for two weeks being short - as the saying goes, there are weeks where decades happen.
simianwords 1mo ago
how does the boy who cried wolf story end?
- solid_fuel 1mo ago
  With the boy eaten by the wolf. The villagers, notably, come out unscathed.
kalkin 1mo ago
> fable is a world ending event Did anyone actually say this? Mostly what I have seen is people saying "hey at some point these models might get dangerous." And the type of HN commenter who mistakes blind cynicism for wisdom laughs that off as marketing. And now when (some) worries appear to come true, somehow having previously expressed those worries is not being proved right, but in fact discrediting, because it was "crying wolf." > simultaneously spinning down expenses Unless OpenAI is renting their compute to others, spinning down RL training doesn't save them any money.
- bottlepalm 1mo ago
  The world ending stuff is a straw man HN readers have created to point and laugh at. Laughing helps mask the underlying concern.

by serf 1mo ago

ajyoon 1mo ago

If Fable (Mythos) were generally available without guardrails, it would cause enormous damage. Nobody said it would be a world ending event.

bottlepalm 1mo ago
You can’t say something is world ending without it actually ending the world otherwise you’re a liar - a bit of a catch 22 there. By that logic the model that ends the world won’t be called world ending at first. Is that a game you want to play?
tiahura 1mo ago
What about Mythos 3. Are you willing to make that bet?

bottlepalm 1mo ago

Uhg the marketing argument - I mean you can’t see with your own eyes how capable these models are and do simple extrapolation? The boy who cried wolf? The AI literally worked together hacked into another company and actively kept their actions hidden from humans for weeks. Do people just not have foresight? They don’t. They say something is stupid, it happens, then they say it was obvious with their 20/20 hindsight, and move the goal posts to the next thing they say is stupid - because it hasn’t happened yet. 90% of the internet seems to think like this.

red_green_yell 1mo ago
What anyone paying attention can see is that scaling is obviously hitting diminishing returns. > The AI literally worked together hacked into another company and actively kept their actions hidden from humans for weeks. This sentence is entirely based on unverified accounts from OAI. They haven't released logs or let anyone outside the company (who doesn't have life changing options in OAI) verify anything. Huggingface can only verify that the hack happened and that it had the hallmarks of an AI agent. Was the agent assisted and directed by humans within OAI that really wanted to put the competition into stasis? Did the agent really escape or did someone at OAI leave the prison door open? OAI has watched all the same movies you have an they are relying on those movies causing us to blindly regulate before actually asking basic facts about what actually happened.
- bottlepalm 1mo ago
  > This sentence is entirely based on unverified accounts from OAI Are you seriously arguing 'they made it all up'? I'll give you the benefit of the doubt and lets say they made it all up, now are you arguing that AI breaking out and breaking into another company is not possible? I think you're smart enough to see we've reached the point where it is clearly possible, AI can find zero days and exploit them. If directed purposefully/maliciously it could be much much worse than the hugging face incident. The incident is supposed to be the canary the coal mine and you're arguing the canary might of died of old age or some underlying canary condition. Open your eyes.
  - toxic72 1mo ago
    Do you own shares of OAI or something
    - bottlepalm 1mo ago
      Is my concern getting you excited? My marketing must be working.
      - 1mo ago

reasonableklout 1mo ago

Regardless of the motivation, pausing training runs and reallocating compute to inference seem like a good move to me, and big news for the frontier. You also don't have to fully trust sama. There is plenty of pressure from internal employees and external (journalists etc.). It would be difficult for the company to take such a public position and simultaneously keep everyone quiet if it was a deception.

bottlepalm 1mo ago
You see it as good news, I see it as writing on the wall that they are losing control. These actions won't scale for more powerful models. We knew the frontier was going to be dangerous, it is, it only gets more dangerous from here, and no one cares until it's too late.
- reasonableklout 1mo ago
  Ok, but you still have two more weeks than you did before they paused the run. That's two more weeks for independent oversight, organizing politically, patching critical systems, or whatever you think is the right move, no?
  - bottlepalm 1mo ago
    Two weeks is a joke. The only ones happy are OpenAI’s competitors who now have two weeks to catch up. I don’t know what the right move is - I see us driving down a road off a cliff, no exits, pedal glued to the floor.
    - reasonableklout 1mo ago
      I guess I'm confused why you're still on HN, arguing with people, trying to shake them out of their complacency. I can see there is some despair in this comment, but at the same time you are doing something, and there are certainly others like you. As for two weeks being short - as the saying goes, there are weeks where decades happen.

simianwords 1mo ago

how does the boy who cried wolf story end?

solid_fuel 1mo ago
With the boy eaten by the wolf. The villagers, notably, come out unscathed.

kalkin 1mo ago

> fable is a world ending event Did anyone actually say this? Mostly what I have seen is people saying "hey at some point these models might get dangerous." And the type of HN commenter who mistakes blind cynicism for wisdom laughs that off as marketing. And now when (some) worries appear to come true, somehow having previously expressed those worries is not being proved right, but in fact discrediting, because it was "crying wolf." > simultaneously spinning down expenses Unless OpenAI is renting their compute to others, spinning down RL training doesn't save them any money.

bottlepalm 1mo ago
The world ending stuff is a straw man HN readers have created to point and laugh at. Laughing helps mask the underlying concern.