5 ms·
For the sake of argument, let's assume OP is correct: the finance industry and corporations in general are indeed blood-sucking vampire squids jamming their fun
by gwern 13y ago
For the sake of argument, let's assume OP is correct: the finance industry and corporations in general are indeed blood-sucking vampire squids jamming their funnels into everything that smells of money and are antithetical to human values. (I strongly disagree with it, but others are criticizing that aspect, so I'll move on.)
So what? How does this justify his conclusion:
> But they have chosen a fantasy version of the problem, when human interests are being fucked over by actual existing systems right now. All that brain-power is being wasted on silly hypotheticals, because those are fun to think about, whereas trying to fix industrial capitalism so it doesn’t wreck the human life-support system is hard, frustrating, and almost certainly doomed to failure.
Even if the finance industry / corporations are a parasite, so what? That doesn't wish away AI: AI will still remain a threat (or not a threat) regardless of whether Goldman Sachs is dissolved. Threats are not a zero-sum game: if you show that finance is a parasite, that doesn't suddenly alter the universe to guarantee AI is safe and easy.
Even if we interpret him charitably as making a nuanced cost-effectiveness argument that 'we face many threats, have very limited resources, and corporations are more cost-effective to fight than working on AI', that still seems pretty darn unlikely. Corporations are a major political issue which attract the attention and effort of hundreds of thousands of people and inspire real world action like Occupy Wall Street; why does this fight need a few more people in it? Diminishing returns set in a very long time ago. And fundamentally, financial types have (or have not) been parasites throughout history. Their impact is bounded. They are not anything new under the sun.
Whereas with AI, we don't know whether AI will ever exist; whether the impact will be huge or close to zero; hardly anyone takes it remotely seriously (quick question: have you ever read a mainstream article which did not make either a _Termintor_ or 'Rapture of the Nerds' allusion?); and the sum total of all efforts on thinking about the topic is, what, maybe $7m a year? (Google's safety board + MIRI + FHI + the new existential risk org whose name escapes me). You have to assume some pretty extreme values to make cost-effectiveness criterion spit out 'nobody should be thinking about AI! Everyone should be swotting up their Marx!'
Conflating socially-harmful corporations with 'hostile AI' does both topics a disservice.
- ispolin 13y agoAn even more charitable interpretation might be: We can gain insight on how to manage fully autonomous rule-based systems (AI) by looking at existing semi-autonomous rule-based systems (Corporations, Governments).
- mtraven 13y agoIndeed, that is a nice and concise way of putting it.
- sutterbomb 13y agoI'm honestly not sure if I misunderstood OP's argument or if you did. I understood his argument as such: 1) Our current economic system is already a form of AI. Using the wikipedia definition, it is a system that perceives its environment and takes actions that maximize its chances of success. Humans that are in this system are inputs in the environment. They may have individual agency, but cannot direct the system as a whole. 2) The "success" that this AI is maximizing is not currently in line with optimizing human interest. 3) Instead of theorizing about how to make sure a future AI system is in line with human interest, we should look at the current dominant AI and attempt to improve that. All three of those premises can be argued, and I've come to reasonable arguments on both sides just on my own with a few minutes of thought. But I think those are the premises to argue.
- mtraven 13y agoOP here, nonplussed that his obscure blog suddenly is near the top of Hacker News. First, I don't think that corporations are nothing but parasites, and said as much in the article. I did say that finance was parasitical. That was probably too strong. Allocation of capital is a useful function; the problem is that the allocators are helping themselves to a far larger cut of the proceedings than the value they produce (and I don't particularly want to argue that here, just wanted to clarify what I meant). Second, the more central point about whether the Singularity Institute types are wasting their time and should refocus their thinking. It is true that not many people are thinking about the effects of superhuman AIs, and there's no harm and possibly some good in a few smart weirdos focusing on that, especially since corporations have been around and critiqued for so long. However, we are so far away from producing anything remotely close to that level of AI, that I fear their work is ungrounded. It strikes me as fun quasi-SF rather than serious engineering. But I take believe they are seriously interested in world improvement, so my suggestion to them in their work on "benevolent goal architectures" is to study how existing goal architectures work, that is to look at existing economic and social systems where disparate individual goals are coordinated into cooperative or conflicting action, and how the results are related to human goals and human flourishing. For my part, I am in the process of trying to understand their friendly AI work better so I critique it from a more informed standpoint.
- topynate 13y ago> Allocation of capital is a useful function; the problem is that the allocators are helping themselves to a far larger cut of the proceedings than the value they produce Yeah, I find this (that financial actors are acting in a hostile manner) more tenable than the impression I took from your blog post, which was that the financial system itself was acting with intentionality. > look at existing economic and social systems where disparate individual goals are coordinated into cooperative or conflicting action I hope you can bring that perspective to the table; I've noticed that a lot of the public work is about preserving goals while modifying structure, while actually mediating between conflicting goals/values is a bit handwavey. And yeah, I hope we're far away from human level AI, but we're talking about our uncertainty on conceptual distance to an undiscovered theoretical framework, here. It's not quite as bad as trying to guess when the Riemann Hypothesis will be resolved, but it has that quality.