4 ms·

The most aggrivating thing about EA "longtermism" AI Safety stuff is that is takes the oxygen out of the room for actual AI safety research. Using ML for objec

The most aggrivating thing about EA "longtermism" AI Safety stuff is that is takes the oxygen out of the room for actual AI safety research. Using ML for object detection, object tracking, or prediction on an L2-L5 driver assistant system? AI safety research sounds like a capability you'd really want. Using ML for object detection, object tracking, or prediction on an industrial robot that is going to work alongside humans or could cost $$$ when it fails? AI safety research sounds like a capability you'd really want. Using classifiers or any form of optimization for algorithmic trading? AI safety research sounds like a capability you'd really want. Building decision support systems to optimize resource allocation (in an emergency, in a data center, in a portfolio, ...)? AI safety research sounds like a capability you'd really want. Hell, want to use an LLM as part of a customer service chatbot? You probably don't want it to be hurling racial slurs at your customers. AI safety research sounds like a capability you'd really want. Unfortunately, now "AI Safety" no longer means "building real world ML systems for real world problems with bounds on their behavior" and instead means... idk, something really stupid EA longtermism nonsense.

ethanbond 4y ago
You just listed 5 completely distinct applications of AI safety (and surely can name countless others) and then concluded with a complaint that the concept of AI safety is not well-defined? The whole point is that the technology itself and its capabilities are not well defined, people are constantly inventing new applications and new methods now at breakneck speed, so the question of how to mitigate its risks is going to be at least as squishy a concept as the underlying tech + applications.
- thwayunion 4y ago
  Right, that's the problem. Before 2016 or so, "AI Safety" meant the types of things I listed above: how to design AI systems that are safe to use in safety-critical settings. Unfortunately, the term "AI Safety" has been overrun with effective altruist longtermism folks talking about something closer to https://en.wikipedia.org/wiki/Existential_risk_from_artificial_general_intelligence https://en.wikipedia.org/wiki/Existential_risk_from_artifici... which is ill-defined and, frankly, usually completely vapid nonsense. > so the question of how to mitigate its risks is going to be at least as squishy a concept as the underlying tech + applications. No, no, no!!! AI safety methods -- methods that really increase confidence in real systems -- are not generic. ALL of the useful AI safety work I know of is deeply related to one or more of: the specific model architecture, the specific optimization algorithm, the specific data, or the specific application. Almost always all of the above.
  - ethanbond 4y ago
    “Field is squishy” != all parts of the field are equally squishy. The presence of some clear problem-and-solution pairs does not preclude the existence of more ambiguous problems. This is especially true as we’re deploying AI against increasingly vague goals with increasingly vague constraints, such as in the case of the non-lying Diplomacy bot that absolutely did deceive human players.
PaulHoule 4y ago
I'm going to add that "AI Safety" (allied with longtermism) is also part of the hype machine for big tech. Pressing the idea that AI is dangerous makes it seem like these companies are even more powerful than they are and could drive up their stock price. When the AI Safety people get into some conflict and get fired they are really doing their job because now it looks like big tech is in a conspiracy to cover up how dangerous their technology is.

ethanbond 4y ago

You just listed 5 completely distinct applications of AI safety (and surely can name countless others) and then concluded with a complaint that the concept of AI safety is not well-defined? The whole point is that the technology itself and its capabilities are not well defined, people are constantly inventing new applications and new methods now at breakneck speed, so the question of how to mitigate its risks is going to be at least as squishy a concept as the underlying tech + applications.

thwayunion 4y ago
Right, that's the problem. Before 2016 or so, "AI Safety" meant the types of things I listed above: how to design AI systems that are safe to use in safety-critical settings. Unfortunately, the term "AI Safety" has been overrun with effective altruist longtermism folks talking about something closer to https://en.wikipedia.org/wiki/Existential_risk_from_artificial_general_intelligence https://en.wikipedia.org/wiki/Existential_risk_from_artifici... which is ill-defined and, frankly, usually completely vapid nonsense. > so the question of how to mitigate its risks is going to be at least as squishy a concept as the underlying tech + applications. No, no, no!!! AI safety methods -- methods that really increase confidence in real systems -- are not generic. ALL of the useful AI safety work I know of is deeply related to one or more of: the specific model architecture, the specific optimization algorithm, the specific data, or the specific application. Almost always all of the above.
- ethanbond 4y ago
  “Field is squishy” != all parts of the field are equally squishy. The presence of some clear problem-and-solution pairs does not preclude the existence of more ambiguous problems. This is especially true as we’re deploying AI against increasingly vague goals with increasingly vague constraints, such as in the case of the non-lying Diplomacy bot that absolutely did deceive human players.

PaulHoule 4y ago

I'm going to add that "AI Safety" (allied with longtermism) is also part of the hype machine for big tech. Pressing the idea that AI is dangerous makes it seem like these companies are even more powerful than they are and could drive up their stock price. When the AI Safety people get into some conflict and get fired they are really doing their job because now it looks like big tech is in a conspiracy to cover up how dangerous their technology is.