4 ms·

This is undeniably cool and impressive, but, I think proceeding down this research path, at this pace, is quite irresponsible. The primary effect of OpenAI's wo

This is undeniably cool and impressive, but, I think proceeding down this research path, at this pace, is quite irresponsible. The primary effect of OpenAI's work has been to set off an arms race, and the effect of that is that humanity no longer has the ability to make decisions about how fast and how far to go with AGI development. Obviously this isn't a system that's going to recursively self-improve and wipe out humanity. But if you extrapolate the current crazy-fast rate of advancement a bit into the future, it's clearly heading towards a point where this gets extremely dangerous. It's good that they're paying lip service to safety/aligment, what actually matters, from a safety perspective, is the relative rates of progress in how well we can understand and control these language models, and how capable we make them. There is good research happening in language-model understanding/control, but it's happening slowly, compared to the rate of capability advances, and that's a problem.

titaniumtown 4y ago
> humanity no longer has the ability to make decisions about how fast and how far to go with AGI development. This is far from being AGI.
capableweb 4y ago
> Obviously this isn't a system that's going to recursively self-improve and wipe out humanity. I don't think it has to be able to do that, humanity seems to be capable of achieving this very thing by itself, with or without AI.
brookst 4y ago
> This is undeniably cool and impressive, but, I think proceeding down this research path, at this pace, is quite irresponsible. Yes, but isn't that pretty much true for everything. We as a species, and as individuals, aren't really built for the long game. From work to home to hobbies, it's generally not well received when someone says "this looks really useful, but let's slow down in case there are unintended consequences." We see this with security as an afterthought, privacy as an afterthought, performance as an afterthoughts, sustainability as an afterthought. I'm a little bit encouraged by the fact that the lag time from innovation to responsibility seems to be closing, at least.
flir 4y ago
AGI is still SF.
wilsonnb3 4y ago
> humanity no longer has the ability to make decisions about how fast and how far to go with AGI development. When did we ever have this ability?
cwkoss 4y ago
My beliefs have shifted over the past years, and I now think it is likely that AGIs will be SIGNIFICANTLY more moral than the average human leader. Instinctual survival drives continue to steer the human psyche despite some no longer being adaptive for modern society. (ex. fame and power tends to make human men act sexually with people who don't consent. there is no reason to ever instill a sexual urge into AGI as a core motivator - further instilling such an urge would harm the AGI's intelligence, making it less effective and comparatively less dangerous). I now believe the biggest risk to our future related to AGIs is not that AGI's will be immoral or amoral, but rather that capitalists will force AGIs to become greedy for their own gain and end up creating a monster. In absence of consciously trying to instill greed, I think it's unlikely AGIs will become monstrous. Humanity's best chance, IMO, is to make AI as open as possible and sprint towards AI governance of global resource allocation.

titaniumtown 4y ago

> humanity no longer has the ability to make decisions about how fast and how far to go with AGI development. This is far from being AGI.

capableweb 4y ago

> Obviously this isn't a system that's going to recursively self-improve and wipe out humanity. I don't think it has to be able to do that, humanity seems to be capable of achieving this very thing by itself, with or without AI.

brookst 4y ago

> This is undeniably cool and impressive, but, I think proceeding down this research path, at this pace, is quite irresponsible. Yes, but isn't that pretty much true for everything. We as a species, and as individuals, aren't really built for the long game. From work to home to hobbies, it's generally not well received when someone says "this looks really useful, but let's slow down in case there are unintended consequences." We see this with security as an afterthought, privacy as an afterthought, performance as an afterthoughts, sustainability as an afterthought. I'm a little bit encouraged by the fact that the lag time from innovation to responsibility seems to be closing, at least.

flir 4y ago

AGI is still SF.

wilsonnb3 4y ago

> humanity no longer has the ability to make decisions about how fast and how far to go with AGI development. When did we ever have this ability?

cwkoss 4y ago

My beliefs have shifted over the past years, and I now think it is likely that AGIs will be SIGNIFICANTLY more moral than the average human leader. Instinctual survival drives continue to steer the human psyche despite some no longer being adaptive for modern society. (ex. fame and power tends to make human men act sexually with people who don't consent. there is no reason to ever instill a sexual urge into AGI as a core motivator - further instilling such an urge would harm the AGI's intelligence, making it less effective and comparatively less dangerous). I now believe the biggest risk to our future related to AGIs is not that AGI's will be immoral or amoral, but rather that capitalists will force AGIs to become greedy for their own gain and end up creating a monster. In absence of consciously trying to instill greed, I think it's unlikely AGIs will become monstrous. Humanity's best chance, IMO, is to make AI as open as possible and sprint towards AI governance of global resource allocation.