3 ms·

This article initially seems to be arguing over semantics, saying that AI isn't AGI, which most people would agree with. Then talks about data dignity, which is

This article initially seems to be arguing over semantics, saying that AI isn't AGI, which most people would agree with. Then talks about data dignity, which is interesting but a bit of a tangent. My understanding of the concern over AGI, or even an all-powerful "AI" isn't that it's not truly self-aware, but rather that it's motives are not in line with humanity's motives. It doesn't matter if it trains on data by humans if it still doesn't have the moral compass of a human. Not really sure what data dignity will provide if we truly do have a misaligned "AGI"

lcnPylGDnU4H9OF 3y ago
Data dignity would give us insight into how the AI “thinks” so we can better understand how it’s aligned.
majormajor 3y ago
My read is that the author doesn't think today's tools are particularly close to AGI that has its own motives, or that it's at all useful to characterize these tools as "having motives" at all. There are those who disagree with how close we are to "AGI" with today's LLMs, but as an outsider, the author seems pretty well-qualified in his position at MS Research compared to some of the less-informed prognosticators. Even the linked-out discussion with Altman, who the author says is more pessimistic than he is, doesn't appear to be a case of Altman saying these models are close to AGI or dangerous for "alignment" reasons vs "they can be used for bad things." Worrying about the uses of tools is a VERY different thing than worrying about if the tools are gonna overthrow us, and I think that's what the author is really pushing for here. If we have that conversation we can get closer to real, specific policies, laws, etc to make sure the tools benefit us instead of the other way around.
- zarzavat 3y ago
  Why would AGI need to have its own motives? AGI = artificial, general intelligence. Meaning intelligence that is generally applicable to all domains. AGI doesn’t mean building an artificial human with all human quirks.
toss1 3y ago
>>It doesn't matter if it trains on data by humans if it still doesn't have the moral compass of a human. True, and I'm not sure that even the moral compass of humans is entirely trustworthy. The quick version is what happened to Tay in the hands of random humans, and the larger context is the fact that the authoritarianism and racism are still very active threats in the arc of history. In 10,000 years of civilization, they still haven't been eliminated. Not sure how even training on human works would eliminate it...
- coffeebeqn 3y ago
  I’m not sure what other works an AI would be trained on. I guess we should hope they’re wiser than us humans
  - toss1 3y ago
    Exactly! We should hope, but hope is not a method for attaining success; that requires a sound plan and work...
batch12 3y ago
I try to remember that it doesn't matter if the algorithms are really AI or just cosplaying. If the end result is something that can behave like a malicious AI, whether or not it's aware may not matter so much. Maybe it's worse really since it's really not aware of what it does.
topspin 3y ago
> This article initially seems to be arguing over semantics This article is conducting the same goal post shift operation that happens every time machines reach another fraction of human capability. Ironically, we're maybe a year or so from the day when one can generate an issue of The New Yorker given a handful of recent event headlines and a few photos.

lcnPylGDnU4H9OF 3y ago

Data dignity would give us insight into how the AI “thinks” so we can better understand how it’s aligned.

majormajor 3y ago

My read is that the author doesn't think today's tools are particularly close to AGI that has its own motives, or that it's at all useful to characterize these tools as "having motives" at all. There are those who disagree with how close we are to "AGI" with today's LLMs, but as an outsider, the author seems pretty well-qualified in his position at MS Research compared to some of the less-informed prognosticators. Even the linked-out discussion with Altman, who the author says is more pessimistic than he is, doesn't appear to be a case of Altman saying these models are close to AGI or dangerous for "alignment" reasons vs "they can be used for bad things." Worrying about the uses of tools is a VERY different thing than worrying about if the tools are gonna overthrow us, and I think that's what the author is really pushing for here. If we have that conversation we can get closer to real, specific policies, laws, etc to make sure the tools benefit us instead of the other way around.

zarzavat 3y ago
Why would AGI need to have its own motives? AGI = artificial, general intelligence. Meaning intelligence that is generally applicable to all domains. AGI doesn’t mean building an artificial human with all human quirks.

toss1 3y ago

>>It doesn't matter if it trains on data by humans if it still doesn't have the moral compass of a human. True, and I'm not sure that even the moral compass of humans is entirely trustworthy. The quick version is what happened to Tay in the hands of random humans, and the larger context is the fact that the authoritarianism and racism are still very active threats in the arc of history. In 10,000 years of civilization, they still haven't been eliminated. Not sure how even training on human works would eliminate it...

coffeebeqn 3y ago
I’m not sure what other works an AI would be trained on. I guess we should hope they’re wiser than us humans
- toss1 3y ago
  Exactly! We should hope, but hope is not a method for attaining success; that requires a sound plan and work...

batch12 3y ago

I try to remember that it doesn't matter if the algorithms are really AI or just cosplaying. If the end result is something that can behave like a malicious AI, whether or not it's aware may not matter so much. Maybe it's worse really since it's really not aware of what it does.

topspin 3y ago

> This article initially seems to be arguing over semantics This article is conducting the same goal post shift operation that happens every time machines reach another fraction of human capability. Ironically, we're maybe a year or so from the day when one can generate an issue of The New Yorker given a handful of recent event headlines and a few photos.