3 ms·

If you have ideas, I encourage you to write a blog post and submit it to HN. Somewhere I got the notion that the UN is generally considered to be dysfunctional

If you have ideas, I encourage you to write a blog post and submit it to HN. Somewhere I got the notion that the UN is generally considered to be dysfunctional these days, and that's part of the reason there's been a shift in diplomacy to other venues like the G20. Representativeness is important, but it's just one of several important desiderata in AI governance. It seems to me that the current important decisions facing OpenAI are for the most part fairly technical, so it's good for their oversight to be technical. Representativeness becomes more important later. I'd say if OpenAI actually does succeed in developing AGI, that's the point at which the board should be replacing itself with some other, more representative institution. >Of course the race is on and the first party to cross the threshold will get the gold and never mind the consequences so I expect any attempt at oversight to be murdered the second they have it working. Maybe a good question from a governance perspective would be, how can we configure the institutional incentives to discourage this, or at least select for leaders with strong character and ethics who won't ruthlessly seek power at all costs? I've been criticizing Sam Altman a fair amount lately, but it sounds like you take it much farther in essentially viewing him as some sort of crime boss, or else you're assuming that anyone with significant power will automatically adopt crime boss ethics as a result. I doubt the latter statement is true -- for example, I don't believe that Obama adopted crime boss ethics when he became POTUS.

jacquesm 3y ago
I think the main reason the UN is dysfunctional is because the largest countries have been effectively in the driving seat and the rest is along for show. As long as there is veto power for a select few and the big military entities ignore any restrictions laid in their way to do what they want to do (by way of veto or otherwise) it will continue that way. But that breeds a ton of resentment elsewhere in the world and the people in those countries effectively have no representation, even if they have the vote. It's apartheid writ large, based on weapons and money. > It seems to me that the current important decisions facing OpenAI are for the most part fairly technical For OpenAI, yes. But for society the stakes are much higher and non-technical in nature. For instance: mass disruption of the labour market. Note that when we had the industrial revolution the world population was a fraction of what it is today. > so it's good for their oversight to be technical But that could change in a heartbeat and by then it is too late. Winner takes all. > Representativeness becomes more important later. There won't be a 'later', whoever gets this first will instantly weaponize it and seize power, though you won't know about it until you realize that it's a done deal. Stable dictatorships are one of the more likely outcomes of AI. > I'd say if OpenAI actually does succeed in developing AGI, that's the point at which the board should be replacing itself with some other, more representative institution. Right now our best hope is that they can't. If not, all bets are off and you may want to brush up on your Charles Stross. > Maybe a good question from a governance perspective would be, how can we configure the institutional incentives to discourage this, or at least select for leaders with strong character and ethics who won't ruthlessly seek power at all costs? Leaders with strong power and those who ruthlessly seek power at all costs have strong overlap, assuming they are not congruent. > I've been criticizing Sam Altman a fair amount lately, but it sounds like you take it much farther in essentially viewing him as some sort of crime boss, or else you're assuming that anyone with significant power will automatically adopt crime boss ethics as a result. I see him just like I see Elon Musk: brilliant but dangerous, not necessarily a crime boss, more like someone who doesn't realize that they don't have the right to make the decisions they make and too little in terms of reflective capabilities and humility. > I doubt the latter statement is true -- for example, I don't believe that Obama adopted crime boss ethics when he became POTUS. Obama was a huge disappointment, so no argument there, but the flipside is that Trump was exactly what he promised to be and more and the world came out for the worse anyway. It makes you wonder how much worse that can get. Probably a lot worse. I probably don't want to know.

jacquesm 3y ago

I think the main reason the UN is dysfunctional is because the largest countries have been effectively in the driving seat and the rest is along for show. As long as there is veto power for a select few and the big military entities ignore any restrictions laid in their way to do what they want to do (by way of veto or otherwise) it will continue that way. But that breeds a ton of resentment elsewhere in the world and the people in those countries effectively have no representation, even if they have the vote. It's apartheid writ large, based on weapons and money. > It seems to me that the current important decisions facing OpenAI are for the most part fairly technical For OpenAI, yes. But for society the stakes are much higher and non-technical in nature. For instance: mass disruption of the labour market. Note that when we had the industrial revolution the world population was a fraction of what it is today. > so it's good for their oversight to be technical But that could change in a heartbeat and by then it is too late. Winner takes all. > Representativeness becomes more important later. There won't be a 'later', whoever gets this first will instantly weaponize it and seize power, though you won't know about it until you realize that it's a done deal. Stable dictatorships are one of the more likely outcomes of AI. > I'd say if OpenAI actually does succeed in developing AGI, that's the point at which the board should be replacing itself with some other, more representative institution. Right now our best hope is that they can't. If not, all bets are off and you may want to brush up on your Charles Stross. > Maybe a good question from a governance perspective would be, how can we configure the institutional incentives to discourage this, or at least select for leaders with strong character and ethics who won't ruthlessly seek power at all costs? Leaders with strong power and those who ruthlessly seek power at all costs have strong overlap, assuming they are not congruent. > I've been criticizing Sam Altman a fair amount lately, but it sounds like you take it much farther in essentially viewing him as some sort of crime boss, or else you're assuming that anyone with significant power will automatically adopt crime boss ethics as a result. I see him just like I see Elon Musk: brilliant but dangerous, not necessarily a crime boss, more like someone who doesn't realize that they don't have the right to make the decisions they make and too little in terms of reflective capabilities and humility. > I doubt the latter statement is true -- for example, I don't believe that Obama adopted crime boss ethics when he became POTUS. Obama was a huge disappointment, so no argument there, but the flipside is that Trump was exactly what he promised to be and more and the world came out for the worse anyway. It makes you wonder how much worse that can get. Probably a lot worse. I probably don't want to know.