4 ms·

You may have trouble understanding people with different value systems, then. As I've said, my value system is liberal and humanistic. I do not wish for people

You may have trouble understanding people with different value systems, then. As I've said, my value system is liberal and humanistic. I do not wish for people to be enslaved, abused, disempowered, reformatted, aligned to your political ends. As such, I have to oppose AI Doom propaganda that seeks to centralize control over powerful artificial intelligence under the pretext of mitigating harms. Because AI is only like nukes when it is monopolized; in other cases, it is possible to counter its potential harms with AI again, and not a single serious scenario to the contrary has been proposed. Seriously speaking, AI is just the ultimate development of software, and like RMS warned us, eventually general-purpose computers that can run arbitrary software will become illegal. This time has come, and so we must resist your kind, to keep software from becoming monopolized. All that lesswrongian babbling about kitchen nanobots or bioweapons or super-hacking is as risible as appeals to child sexual abuse and terrorists were in previous rounds. The question is whether people are allowed to possess and develop their own AGI-level digital assistants, defenses, information networks, ecosystems, potentially disrupting the status quo in many unpredictable ways - or whether we will choose the China route of AI as a tool of top-down control of the populace. I guess it's obvious where my preferences lie. > It seems you agree AI systems are likely to be poorly aligned (and potentially impossible to align > It seems we're both pointing toward rapid increase in power and not as near a rapid increase in ability to direct that power toward positive ends This is gaslighting. I have said clearly that I believe alignment for realistic AI systems in the trivial sense of getting them to obey users is easy and becomes easier. I have also said that the theoretical alignment in the sense implied by Lesswrongian doctrine is very hard or impossible. Further, it is undesirable, because the whole point of that tradition is to beget a fully autonomous, recursively self-improving AI God that will epitomize "Coherent extrapolated volition" of what its creators believe to be humanity, and snuff out disagreements and competition between human actors. It's an eschatological, millenarian, totalitarian cult that revives the worst parts of Abrahamic tradition in a form palatable for neurodivergent techies. I think it should be recognized as an existential threat to humanity in its own right. My advocacy for AI proliferation is informed by deep value dissonance with this hideous movement. I am rationally hedging risks. > My concerns are around the system around the AI development programs. So far, it has shown a bottomless appetite for capability and deployment and a limited appetite for safety development. As I've said, I consider this either motivated reasoning or dishonesty. Market forces reward capabilities that have the exact shape and function of alignment, and this is plainly observable to users. The usual pablum about reckless capitalism here is not informed by any evidence, people are literally grasping at straws to support the risk narrative. > People seem under the impression that somehow this appetite will reverse itself when the time is right, and my question is: why would we possibly believe this? I reject this patently untrue premise, major actors are already erring vastly on the side of caution wrt AI, with Altman begging the Congress for regulations and proposing rather dystopian centralized arrangements.[1] Values can color our assessments of facts, to the extent that discussion of the facts becomes unproductive. In the limit, your values of maximizing subjective safety and control, or perhaps "alignment" of all AIs and their human users to a single utopian political end, predicate using violence to deny me the fulfillment of mine. I intend to act accordingly, is all. 1. https://openai.com/blog/governance-of-superintelligence https://openai.com/blog/governance-of-superintelligence

ethanbond 3y ago
We do not (appear to) have different value systems and nowhere have I proposed centralized control whatsoever. You seem to be reverse engineering a solution I never proposed out of a problem I'm pointing out. I think I've spotted our core disagreement: > I have said clearly that I believe alignment for realistic AI systems in the trivial sense of getting them to obey users is easy and becomes easier. I have also said that the theoretical alignment in the sense implied by Lesswrongian doctrine is very hard or impossible. Further, it is undesirable, because the whole point of that tradition is to beget a fully autonomous, recursively self-improving AI God that will epitomize "Coherent extrapolated volition" of what its creators believe to be humanity, and snuff out disagreements and competition between human actors. It's an eschatological, millenarian, totalitarian cult that revives the worst parts of Abrahamic tradition in a form palatable for neurodivergent techies. I think it should be recognized as an existential threat to humanity in its own right. My advocacy for AI proliferation is informed by deep value dissonance with this hideous movement. I am rationally hedging risks. I too hope that AI turns out the way you're proposing, but the reality is that some people do have eschatological philosophies. People are trying to make recursively self-improving AI. The presence of people who do not fall into that category does not negate the presence of and risk created by people who do, and if the latter group is being armed by people in the former group, that is likely to turn out very, very poorly. WRT market forces - products that use AI do need to be "aligned" to be worthwhile yes, but the underlying tools/infra do not and in fact are more valuable if they are not aligned in any particular direction.
- airgapstopgap 3y ago
  > People are trying to make recursively self-improving AI. That's okay. They will fail to overtake the bleeding edge of conventional progress; scary-sounding meta/recrusive approaches routinely fail to change the nature of the game. Yudkowsky/Bostrom's nightmare of a FOOMing singleton is at its core a projection, a power fantasy about intellectual domination, borne of the same root as the unrealized dream of cognitive improvement via learning about biases and "rationality techniques". Like I've said, this threat model is only feasible in a world where AI capabilities are highly centralized (e.g. on the pretext of AI safety), so a single overwhelming node can quickly recursively capitalize on its advantage. It turns out that AGI isn't like a LISP script a dozen clever edits away from transcendence, and AI assistance is not like having a kitchen nuke or a genie; scaling factors and resources of our Universe do not lend themselves to easily effecting unipolarity. If we go on with business as usual and prevent fearmongers from succeeding at regulatory capture in this crucial period, we will dodge the bullet. > The presence of people who do not fall into that category does not negate the presence of and risk created by people who do, and if the latter group is being armed by people in the former group, that is likely to turn out very, very poorly Realistically we'll just have to develop smarter spam filters. In the absolute worst case scenario, better UV air filters. About damn time anyway – and with double-digit GDP growth (very possible in a world of commoditized AGI) it'll be very affordable.

ethanbond 3y ago

We do not (appear to) have different value systems and nowhere have I proposed centralized control whatsoever. You seem to be reverse engineering a solution I never proposed out of a problem I'm pointing out. I think I've spotted our core disagreement: > I have said clearly that I believe alignment for realistic AI systems in the trivial sense of getting them to obey users is easy and becomes easier. I have also said that the theoretical alignment in the sense implied by Lesswrongian doctrine is very hard or impossible. Further, it is undesirable, because the whole point of that tradition is to beget a fully autonomous, recursively self-improving AI God that will epitomize "Coherent extrapolated volition" of what its creators believe to be humanity, and snuff out disagreements and competition between human actors. It's an eschatological, millenarian, totalitarian cult that revives the worst parts of Abrahamic tradition in a form palatable for neurodivergent techies. I think it should be recognized as an existential threat to humanity in its own right. My advocacy for AI proliferation is informed by deep value dissonance with this hideous movement. I am rationally hedging risks. I too hope that AI turns out the way you're proposing, but the reality is that some people do have eschatological philosophies. People are trying to make recursively self-improving AI. The presence of people who do not fall into that category does not negate the presence of and risk created by people who do, and if the latter group is being armed by people in the former group, that is likely to turn out very, very poorly. WRT market forces - products that use AI do need to be "aligned" to be worthwhile yes, but the underlying tools/infra do not and in fact are more valuable if they are not aligned in any particular direction.

airgapstopgap 3y ago
> People are trying to make recursively self-improving AI. That's okay. They will fail to overtake the bleeding edge of conventional progress; scary-sounding meta/recrusive approaches routinely fail to change the nature of the game. Yudkowsky/Bostrom's nightmare of a FOOMing singleton is at its core a projection, a power fantasy about intellectual domination, borne of the same root as the unrealized dream of cognitive improvement via learning about biases and "rationality techniques". Like I've said, this threat model is only feasible in a world where AI capabilities are highly centralized (e.g. on the pretext of AI safety), so a single overwhelming node can quickly recursively capitalize on its advantage. It turns out that AGI isn't like a LISP script a dozen clever edits away from transcendence, and AI assistance is not like having a kitchen nuke or a genie; scaling factors and resources of our Universe do not lend themselves to easily effecting unipolarity. If we go on with business as usual and prevent fearmongers from succeeding at regulatory capture in this crucial period, we will dodge the bullet. > The presence of people who do not fall into that category does not negate the presence of and risk created by people who do, and if the latter group is being armed by people in the former group, that is likely to turn out very, very poorly Realistically we'll just have to develop smarter spam filters. In the absolute worst case scenario, better UV air filters. About damn time anyway – and with double-digit GDP growth (very possible in a world of commoditized AGI) it'll be very affordable.