5 ms·

But nobody is failing to take the risks seriously. The people who actually work on these models understand the risks far better than the outside ethicists. I wo

But nobody is failing to take the risks seriously. The people who actually work on these models understand the risks far better than the outside ethicists. I work on ML in research. Reading their work is like listening to a drunk relative at Thanksgiving ranting about Bill Gates putting nanobots in vaccines. It's completely uninformed pseudoscience that comes from a place of strong political bias. For example Timnit's "parrots" paper confused training with inference and GPUs with TPUs, making specific quantitative estimates that were off by orders of magnitude. If she had talked to a single person working on large language models, she would have recognized the error. But these people work in a bubble where facts don't matter and identify politics is everything.

trention 4y ago
There are enough people criticizing both current language models and the overall "quest" towards AGI that come from a non-political (unless you subscribe to an aristotelian everything-is-politics) perspective. I personally don't think any of the companies with significant AI research is actually doing anything meaningful in terms of safety. Also, from their public "utterings", it's quite clear to me that both Altman and Hassabis (not to mention Lecun) don't actually care about safety or consequences.
- mgraczyk 4y ago
  > I personally don't think any of the companies with significant AI research is actually doing anything meaningful in terms of safety I assume this is just speculation on your part? Do you have any reason to make that claim? I personally know multiple people doing this full time at large tech companies. I can give you some examples of serious safety oriented criticism of large language models to contrast with what plays out in the press and amongst "ethicists". It's well understood that one can generate so-called "adversarial examples" for image classifiers. These adversarial examples can be chosen so that to a human they look like thing A, but the model classifies them as thing B with high probability. Methods of finding these adversaries are well understood. Methods of preventing them from being problematic are less developed but rapidly advancing. For language models, the situation is much worse. I don't know of any effective way to prevent a large language model from being searched for adversarial inputs, trivially. That is, an attacker could find inputs from large curated input spaces that cause the model to output a specific, desired sequence. For example, an attacker with access to the model weights could probably find an innocuous looking input that causes the model to output "kill yourself". Is this a risk that AI researchers are aware of? Yes, of course. But the difference between AI researchers and "ethicists" is that AI researchers understand the implications of the risk and will work on mitigations. "Ethicists" do not care about mitigating risk, and they don't care that the people who build the models already understand them and are comfortable with them.
  - adamsmith143 4y ago
    >I can give you some examples of serious safety oriented criticism of large language models to contrast with what plays out in the press and amongst "ethicists". To clarify I think the poster above was talking about the AI Alignment/Control Problem and not the specifics failure modes of particular models, LLM, CNNs etc. Very few people at OpenAI or Deepmind for example are seriously engaging with Alignment. Paul Cristiano at least acknowledges the problem but seems to think there will be available solutions in time to avert serious consequences which may or may not be the case. The folks at MIRI certainly don't seem optimistic.
  - trention 4y ago
    >I personally know multiple people doing this full time at large tech companies. The failure mode of internal "ethical" control at private enterprises is well-known and has already played out (at least) once when we tried to regulate medical experiments in the 2 decades after WW2. I personally consider the current AI safety positions to be just blatant whitewashing. The lemoine fiasco is a specifically hilarious case in point combining both a) a person that is utterly incompetent and biased to work at that position and b) total failure of leadership to adequately engage with an issue (or even admit it's possible in principle). At the current point, AI safety is roughly as useful as tobacco lobbying (exaggerated for effect).
adamsmith143 4y ago
Well I definitely wasn't talking about people like Timnit but rather researchers like Stuart Russell who actually are at the forefront of the field and discuss AI safety broadly.

trention 4y ago

There are enough people criticizing both current language models and the overall "quest" towards AGI that come from a non-political (unless you subscribe to an aristotelian everything-is-politics) perspective. I personally don't think any of the companies with significant AI research is actually doing anything meaningful in terms of safety. Also, from their public "utterings", it's quite clear to me that both Altman and Hassabis (not to mention Lecun) don't actually care about safety or consequences.

mgraczyk 4y ago
> I personally don't think any of the companies with significant AI research is actually doing anything meaningful in terms of safety I assume this is just speculation on your part? Do you have any reason to make that claim? I personally know multiple people doing this full time at large tech companies. I can give you some examples of serious safety oriented criticism of large language models to contrast with what plays out in the press and amongst "ethicists". It's well understood that one can generate so-called "adversarial examples" for image classifiers. These adversarial examples can be chosen so that to a human they look like thing A, but the model classifies them as thing B with high probability. Methods of finding these adversaries are well understood. Methods of preventing them from being problematic are less developed but rapidly advancing. For language models, the situation is much worse. I don't know of any effective way to prevent a large language model from being searched for adversarial inputs, trivially. That is, an attacker could find inputs from large curated input spaces that cause the model to output a specific, desired sequence. For example, an attacker with access to the model weights could probably find an innocuous looking input that causes the model to output "kill yourself". Is this a risk that AI researchers are aware of? Yes, of course. But the difference between AI researchers and "ethicists" is that AI researchers understand the implications of the risk and will work on mitigations. "Ethicists" do not care about mitigating risk, and they don't care that the people who build the models already understand them and are comfortable with them.
- adamsmith143 4y ago
  >I can give you some examples of serious safety oriented criticism of large language models to contrast with what plays out in the press and amongst "ethicists". To clarify I think the poster above was talking about the AI Alignment/Control Problem and not the specifics failure modes of particular models, LLM, CNNs etc. Very few people at OpenAI or Deepmind for example are seriously engaging with Alignment. Paul Cristiano at least acknowledges the problem but seems to think there will be available solutions in time to avert serious consequences which may or may not be the case. The folks at MIRI certainly don't seem optimistic.
- trention 4y ago
  >I personally know multiple people doing this full time at large tech companies. The failure mode of internal "ethical" control at private enterprises is well-known and has already played out (at least) once when we tried to regulate medical experiments in the 2 decades after WW2. I personally consider the current AI safety positions to be just blatant whitewashing. The lemoine fiasco is a specifically hilarious case in point combining both a) a person that is utterly incompetent and biased to work at that position and b) total failure of leadership to adequately engage with an issue (or even admit it's possible in principle). At the current point, AI safety is roughly as useful as tobacco lobbying (exaggerated for effect).

adamsmith143 4y ago

Well I definitely wasn't talking about people like Timnit but rather researchers like Stuart Russell who actually are at the forefront of the field and discuss AI safety broadly.