4 ms·

Apparently NIST employees dislike Christiano's beliefs and associations. It doesn't seem justified to label their or Christiano's beliefs as "religious". If any

Apparently NIST employees dislike Christiano's beliefs and associations. It doesn't seem justified to label their or Christiano's beliefs as "religious". If any disagreement with a common position is "religious war", then all such disagreements are arbitrary and religion itself is just as arbitrary.

lmeyerov 3y ago
There are immediate cybercrime etc activities happening around AI -- eg, content abuse is rampant rn -- so someone advising on active standards that explicitly doesn't (relatively) care about these, especially at the federal standards leadership level, is insane. That's the job! Internally to gov, depts deploying AI are terrible at data security even before LLMs hoover & regurgitate everything. Most senior leaders don't understand this stuff, like why secure RAG indexing is hard. (Some are sharp, just too few.) That's even before other basics like ai vs human decision making for diff tasks. Getting a future-ethics person asleep at the wheel on the on-fire problems right as deployments & ATOs are getting figured out in every agency.. ouch.
- mcguire 3y ago
  It is a great way to prevent those currently deploying and using AI from being impeded by any icky current concerns.
  - lmeyerov 3y ago
    yeah that's beyond my paygrade :) At least for our work, it just takes 1 hat out of the ring for those that we would be otherwise paying attention to, but afaict doesn't really change much otherwise. There are parallel efforts across the DOD etc to set AI compliance standards for internal policies. Likewise, as usual, legal innovators at the state level are also doing legislation to govern commerce, like the recent California thing. Precedent will still get set... just not by NIST? Ex: Sounds like this person would not be focused on basic efforts to update practical aspects of NIST SP 800-53 for telling teams how it is safe to do AI in the cloud. So either teams will default to NO -- we've seen this as a common reaction in different govs + banks -- or need to point to something like the IC/DOD AI policies. Or it'll continue business as usual for the folks doing the work.
kragen 3y ago
the particular positions christiano is being criticized for are positions on questions such as the following: - what are the correct standards of right and wrong, if any exist? - what is the ultimate purpose of human existence? - what, if anything, will bring about the extinguishment of the human race? - what sorts of immaterial more-than-human consciousnesses exist, or can exist, and what are the relationships between them and humanity? it doesn't seem especially controversial or metaphorical to me to describe beliefs about such topics as 'religious', particularly when they are coupled with associations organized around those beliefs. buddhism is routinely described as a 'religion' even though it doesn't involve the worship of any deities and is often described by buddhists as entailing atheism
- burkaman 3y ago
  Where did that list come from? I don't see anything like that in the article.
  - kragen 3y ago
    it came from reading about effective altruism and longtermism since before they got those names
    - burkaman 3y ago
      Ok I see, I interpreted the phrase "association with" to be more about the people he's associating with than the beliefs, but the article is incredibly vague so there's no way for us to know what these criticisms actually are.
      - kragen 3y ago
        the criticisms are that other people have different positions on those questions; it should hardly be surprising that people with conflicting positions on questions like the correct standards of right and wrong will criticize one another oh and also some bad people are effective altruists, which is something everyone agrees on, including the other effective altruists

lmeyerov 3y ago

There are immediate cybercrime etc activities happening around AI -- eg, content abuse is rampant rn -- so someone advising on active standards that explicitly doesn't (relatively) care about these, especially at the federal standards leadership level, is insane. That's the job! Internally to gov, depts deploying AI are terrible at data security even before LLMs hoover & regurgitate everything. Most senior leaders don't understand this stuff, like why secure RAG indexing is hard. (Some are sharp, just too few.) That's even before other basics like ai vs human decision making for diff tasks. Getting a future-ethics person asleep at the wheel on the on-fire problems right as deployments & ATOs are getting figured out in every agency.. ouch.

mcguire 3y ago
It is a great way to prevent those currently deploying and using AI from being impeded by any icky current concerns.
- lmeyerov 3y ago
  yeah that's beyond my paygrade :) At least for our work, it just takes 1 hat out of the ring for those that we would be otherwise paying attention to, but afaict doesn't really change much otherwise. There are parallel efforts across the DOD etc to set AI compliance standards for internal policies. Likewise, as usual, legal innovators at the state level are also doing legislation to govern commerce, like the recent California thing. Precedent will still get set... just not by NIST? Ex: Sounds like this person would not be focused on basic efforts to update practical aspects of NIST SP 800-53 for telling teams how it is safe to do AI in the cloud. So either teams will default to NO -- we've seen this as a common reaction in different govs + banks -- or need to point to something like the IC/DOD AI policies. Or it'll continue business as usual for the folks doing the work.

kragen 3y ago

the particular positions christiano is being criticized for are positions on questions such as the following: - what are the correct standards of right and wrong, if any exist? - what is the ultimate purpose of human existence? - what, if anything, will bring about the extinguishment of the human race? - what sorts of immaterial more-than-human consciousnesses exist, or can exist, and what are the relationships between them and humanity? it doesn't seem especially controversial or metaphorical to me to describe beliefs about such topics as 'religious', particularly when they are coupled with associations organized around those beliefs. buddhism is routinely described as a 'religion' even though it doesn't involve the worship of any deities and is often described by buddhists as entailing atheism

burkaman 3y ago
Where did that list come from? I don't see anything like that in the article.
- kragen 3y ago
  it came from reading about effective altruism and longtermism since before they got those names
  - burkaman 3y ago
    Ok I see, I interpreted the phrase "association with" to be more about the people he's associating with than the beliefs, but the article is incredibly vague so there's no way for us to know what these criticisms actually are.
    - kragen 3y ago
      the criticisms are that other people have different positions on those questions; it should hardly be surprising that people with conflicting positions on questions like the correct standards of right and wrong will criticize one another oh and also some bad people are effective altruists, which is something everyone agrees on, including the other effective altruists