3 ms·

The degradation, polarization, and weaponization of or media landscape over the era of social media has left us utterly incapable of believing anything that any

by NiloCK 14d ago

The degradation, polarization, and weaponization of or media landscape over the era of social media has left us utterly incapable of believing anything that anybody says. Dario in particular has consistently been risk-wary on model improvements for going on a decade - long before he was CEO of Anthropic. He believes what he is saying. Human beings can plainly say things that they think are true. Not every utterance by every person is a cynical ploy. Please at least consider the possibility.

blfr 14d ago
I have considered it, I've been hearing way more of it that I like and it's rationalist slop with little to no predictive power: https://foom.hyperplex.org/ https://foom.hyperplex.org/
- kalkin 14d ago
  Hmm, I clicked that link and the first claimed bad prediction I see, from 1996, is "singularity 2035 (actually 2025)". Predicting 2025-2035 as, at least, the period when AI becomes a really big deal, seems pretty good, even if the jury's still out on "singularity". Broadly, the rationalists seem to have been pretty early to realizing LLMs were a big deal, and certainly seem to have had a much more accurate picture of how they'd develop than the people who denounce "TESCREAL" and talk about "stochastic parrots". It's fair (and IMHO correct) to ding rationalists for lots of things, but specifically poor prediction about AI seems like a bad one, insofar as anything has been tested so far.
  - zero_shift 13d ago
    > Broadly, the rationalists seem to have been pretty early to realizing LLMs were a big deal You can't make that assertion when these same people made the LLM revolution happen, by securing the capital and human resources to realise their dream / nightmare
    - kalkin 13d ago
      The people who made the LLM revolution happen were by no means exclusively rationalists--but more importantly, lots of people have lots of dreams about the future and even when pursued very, very few of those dreams pan out in anything like this way. LLMs are not what they are because of rationalists' sheer force of will.
- 0xDEAFBEAD 14d ago
  Amodei and Yudkowsky are different people with different views. See e.g. https://intelligence.org/2014/01/13/miri-strategy-conversation-with-steinhardt-karnofsky-and-amodei/ https://intelligence.org/2014/01/13/miri-strategy-conversati...
Avicebron 14d ago
We've had enough utterances from "effective altruists" to know what they say and claim to believe is a cynical ploy.
youoy 14d ago
This sounds to me like a cry for help from someone thats held hostage. His previous self is writting this from the possition of his current self who is too deep in the economic consequeces to be able to do anything meaningful other than write a consequenceless text. Any other action would now carry too much personal risk for him.
- 0xDEAFBEAD 14d ago
  What course of action would you suggest for Dario at this point?
  - youoy 14d ago
    For example I believe the world would be more secure with an open frontier AI lab, but the consequences of him doing that are too big for him.
    - stratos123 14d ago
      You mean, if Anthropic made all of their models open-weights from now on? I'm pretty sure Dario thinks this will be extremely unsafe, because it'd provide unrestricted access to all the most capable/dangerous models to everyone, and hence doesn't do it. Why do you think it'd make the world more secure, if he did?
      - youoy 14d ago
        AI technology is now only audited by people who think like them. The ones that dont either dont join the company, or leave after a short stint as we have seen. That creates an information or feedback bubble which is not healthy nor productive.
        stratos123 14d ago
        Okay, but suppose that Hypothetical Opensource Anthropic trains a model that turns out to be very dangerous, and releases it. Suppose that the public investigates and, not being limited by an information bubble, correctly notices that it's very dangerous. What then? The model's already released, there's effectively no way to prevent it from being used. Whatever the risks of its release were, they will now materialize, regardless of what the public wants. How is this better than the current world, either by Dario's values or by yours?
ThrowawayR2 14d ago
> "Human beings can plainly say things that they think are true. Not every utterance by every person is a cynical ploy." Every person, no, but every utterance of a CEO is, in fact, a cynical ploy. That's not even a cynical statement itself; it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation. If he believes what he's saying, then when Anthropic is sued for their AI harming another party, his statements are evidence that Anthropic knew _in advance_ that their AI safeguards were likely insufficient to keep their product from harming people. It would be a blatant admission that they were reckless and negligent. That other AI companies are doing the same would not mitigate that.
- stratos123 14d ago
  > Every person, no, but every utterance of a CEO is, in fact, a cynical ploy. That's not even a cynical statement itself; it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation. This seems like useless pedantry. Suppose a person really does have belief A and expresses it for years, they become a CEO of a company, and keep saying A. Are they lying? Well, maybe their beliefs magically changed when they became a CEO, but the more likely explanation is that they think expressing their belief in A is more important than their company's interests.
- sailfast 14d ago
  Not sure you can judge a PBC under those same umbrella as a for profit company, and voting rights make a big difference in terms of responsibility. There are a number of ways to govern companies that can reduce the amount of cynicism and “shareholder value” issues - you just don’t see it very often.
- itajaja 14d ago
  > it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation. not really. if you really want to go by definition in the book and apply it the CEO role, Amodei is CEO of an LTBT, so it's in the CEO's goal to benefit humanity. You can believe in his sincerity or not if you want, but using the CEO label as a definitional reason as to why he must lie is factually incorrect.

by NiloCK 14d ago

blfr 14d ago

I have considered it, I've been hearing way more of it that I like and it's rationalist slop with little to no predictive power: https://foom.hyperplex.org/ https://foom.hyperplex.org/

kalkin 14d ago
Hmm, I clicked that link and the first claimed bad prediction I see, from 1996, is "singularity 2035 (actually 2025)". Predicting 2025-2035 as, at least, the period when AI becomes a really big deal, seems pretty good, even if the jury's still out on "singularity". Broadly, the rationalists seem to have been pretty early to realizing LLMs were a big deal, and certainly seem to have had a much more accurate picture of how they'd develop than the people who denounce "TESCREAL" and talk about "stochastic parrots". It's fair (and IMHO correct) to ding rationalists for lots of things, but specifically poor prediction about AI seems like a bad one, insofar as anything has been tested so far.
- zero_shift 13d ago
  > Broadly, the rationalists seem to have been pretty early to realizing LLMs were a big deal You can't make that assertion when these same people made the LLM revolution happen, by securing the capital and human resources to realise their dream / nightmare
  - kalkin 13d ago
    The people who made the LLM revolution happen were by no means exclusively rationalists--but more importantly, lots of people have lots of dreams about the future and even when pursued very, very few of those dreams pan out in anything like this way. LLMs are not what they are because of rationalists' sheer force of will.
0xDEAFBEAD 14d ago
Amodei and Yudkowsky are different people with different views. See e.g. https://intelligence.org/2014/01/13/miri-strategy-conversation-with-steinhardt-karnofsky-and-amodei/ https://intelligence.org/2014/01/13/miri-strategy-conversati...

Avicebron 14d ago

We've had enough utterances from "effective altruists" to know what they say and claim to believe is a cynical ploy.

youoy 14d ago

This sounds to me like a cry for help from someone thats held hostage. His previous self is writting this from the possition of his current self who is too deep in the economic consequeces to be able to do anything meaningful other than write a consequenceless text. Any other action would now carry too much personal risk for him.

0xDEAFBEAD 14d ago
What course of action would you suggest for Dario at this point?
- youoy 14d ago
  For example I believe the world would be more secure with an open frontier AI lab, but the consequences of him doing that are too big for him.
  - stratos123 14d ago
    You mean, if Anthropic made all of their models open-weights from now on? I'm pretty sure Dario thinks this will be extremely unsafe, because it'd provide unrestricted access to all the most capable/dangerous models to everyone, and hence doesn't do it. Why do you think it'd make the world more secure, if he did?
    - youoy 14d ago
      AI technology is now only audited by people who think like them. The ones that dont either dont join the company, or leave after a short stint as we have seen. That creates an information or feedback bubble which is not healthy nor productive.
      - stratos123 14d ago
        Okay, but suppose that Hypothetical Opensource Anthropic trains a model that turns out to be very dangerous, and releases it. Suppose that the public investigates and, not being limited by an information bubble, correctly notices that it's very dangerous. What then? The model's already released, there's effectively no way to prevent it from being used. Whatever the risks of its release were, they will now materialize, regardless of what the public wants. How is this better than the current world, either by Dario's values or by yours?

ThrowawayR2 14d ago

> "Human beings can plainly say things that they think are true. Not every utterance by every person is a cynical ploy." Every person, no, but every utterance of a CEO is, in fact, a cynical ploy. That's not even a cynical statement itself; it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation. If he believes what he's saying, then when Anthropic is sued for their AI harming another party, his statements are evidence that Anthropic knew _in advance_ that their AI safeguards were likely insufficient to keep their product from harming people. It would be a blatant admission that they were reckless and negligent. That other AI companies are doing the same would not mitigate that.

stratos123 14d ago
> Every person, no, but every utterance of a CEO is, in fact, a cynical ploy. That's not even a cynical statement itself; it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation. This seems like useless pedantry. Suppose a person really does have belief A and expresses it for years, they become a CEO of a company, and keep saying A. Are they lying? Well, maybe their beliefs magically changed when they became a CEO, but the more likely explanation is that they think expressing their belief in A is more important than their company's interests.
sailfast 14d ago
Not sure you can judge a PBC under those same umbrella as a for profit company, and voting rights make a big difference in terms of responsibility. There are a number of ways to govern companies that can reduce the amount of cynicism and “shareholder value” issues - you just don’t see it very often.
itajaja 14d ago
> it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation. not really. if you really want to go by definition in the book and apply it the CEO role, Amodei is CEO of an LTBT, so it's in the CEO's goal to benefit humanity. You can believe in his sincerity or not if you want, but using the CEO label as a definitional reason as to why he must lie is factually incorrect.