4 ms·

That’s not correct. What you are saying is that there is no plausible organized effort that could stop or slow the creation of signal processing models that wil

by septor 7y ago

That’s not correct. What you are saying is that there is no plausible organized effort that could stop or slow the creation of signal processing models that will have pronounced negative impacts. The error is on two levels: you are using too much analogy with other technologies. And you are writing off the possibility of stopping ai when it’s still not clear that it can’t be stopped. This isn’t something that can be built and tested in isolation like other things we are familiar with. Training these models is not an exact science. Nothing about ai is an exact science. Progress only comes with trial and error. And each trial requires huge compute resources; at least for the most capable and dangerous models. It can’t be done in your basement. Not without significant effort and drawing attention to yourself. Could we sense whenever someone was trying to do it? Could we form a global coalition to stop every attempt? That brings us to the next thing. What you are doing is the following: we are both in a car that is about to roll off a cliff. I propose that we try pressing the brakes. You respond by saying that, geez it looks like we probably wouldn’t stop in time — we are going awfully fast and it probably wouldn’t work to press the brakes so why even try? Let’s just brace our heads and hope the impact doesn’t kill us. Obviously the better thing to do is to try and press the brakes. Even if you aren’t sure if you can stop in time.

CamperBob2 7y ago
Correction: You are in a car that doesn't have any brakes.
- krick 7y ago
  I don't understand why it's downvoted (at the moment of writing, at least). I think it's absolutely correct. Imagine a thing "we should not do", let it be creation of really powerful language model, genetic engineering in order to produce smarter, stronger children, or whatever. Whatever you are opposing to, really. The simple fact is that if you (and by "you" I mean any entity you associate yourself with, be it literally you, or your company, or a group of researchers in your country, or the government of your country) can do something, anybody can. Even if it will be a couple of years later. You are not unique, nor alone. You can stop "yourselves" as long as you want, but there are other people, companies, research groups, governments, and they don't give a fuck about what you think "we" should do. So, in the end, the car really doesn't have any brakes. Even if it truly means the end of it all, it's just unfortunate, but really, really unavoidable.
jcims 7y ago
I think if you try to be less cynical about what OpenAI is doing you might even find them an ally to your perspective. Facebook throwing pocket change at an AI ethics org and Google's rather embarrassing failure at staffing an ethics panel of its own is evidence that we've got bicycle brakes on a freight train. OpenAI suffered a ton of blowback for not just releasing the full model from the start. You can read their intitial blog post [1] looking particularly at the sections for Policy Implications and Release Strategy. I would also highly recommend you listen to Lex Fridman's podcast with Greg Brockman [2] to hear their rationale about the recent org changes at Open AI. Obviously you can posit that everything they say is bullshit and they are only after almighty dollars. I can't prove it's not true and personally believe there is at least a kernel of truth to it, but we live in a messy world and finding imperfect allies is generally better than having none at all. [1] - https://openai.com/blog/better-language-models/ https://openai.com/blog/better-language-models/ [2] - https://www.youtube.com/watch?v=bIrEM2FbOLU https://www.youtube.com/watch?v=bIrEM2FbOLU
- septor 7y ago
  I’ve already read the blog post and watched the video. I’m not just some idiot who reads a few headlines and then comes to this conclusion. I found lex’s conversation to be frustrating beyond description. These people simply don’t understand the consequences of this technology. They think that an ethics board will help in some way. They are completely missing the point. “What’s going to stop bad things from happening?” “The charter” Good fucking god.
PostOnce 7y ago
What do you mean "Could we sense when someone was trying to do it", are we talking mandatory computer inspections, government-enforced walled gardens, and the death of the general purpose computer? lol, if you thought gun control was hard... Also, yeah, you could do it in your basement without being detected. You can do it in the cloud, too, a lot of us here have the skills and resources to do it, but why spend time on this toy instead of another one, especially if it's going to cost money we could spend on something more fun?
- septor 7y ago
  Ai in general is an area of active research. The AIs that will bring about the end of the world do not exist — they are not simply in need of an implementation. Right now ai is driven by the collective effort of many research entities. Universities and private businesses. Ai research in those places can be shut down very effectively with regulation. That would probably be enough to buy us centuries because progress in ai depends heavily on this open community of research. Making the use of high level signal processing illegal for business would further delay ai because there would be no incentive for anyone to develop models for profit. We audit taxes and lots of other things, auditing signal processing is not obviously impossible. As you mention, cloud computing is a thing. Cloud computing will likely be the nucleation point from which the first agi springs. It should be shut down. Through regulation or through the dismantling of the internet it should be shut down. Shutting it down would buy us lots of valuable time. ICs are not something you can make in your back yard. Controlling the compute density and energy efficiency or any other aspect of ICs is trivial because they require huge factories to manufacture. Huge amounts of space, energy and money are required to run those fabs. And there aren’t very many of them. Therefore they are highly susceptible to regulation and public oversight. We could stop the production of ICs that are too powerful. So, here you are in your basement. You are trying to train an AGI. You are stuck with old hardware that you somehow bought on the black market. You are risking your life to do it because it’s highly illegal. You don’t have any investment, legal or otherwise, because it’s highly illegal and also stands to make no profit for anyone. You don’t have access to cloud computing. The training is very slow. It is made even slower by the fact that you have to slowly charge a gigantic battery array before each training because without batteries the spike in wattage would be inexplicable for a house in the suburbs and probably trigger a visit from the power authority. The probability that anyone would succeed in this scenario is not zero, but it’s very low. Very, very low. And I think it would save the world.
feanaro 7y ago
> The error is on two levels: you are using too much analogy with other technologies. And you are writing off the possibility of stopping ai when it’s still not clear that it can’t be stopped. No. What I am saying is that it's impossible to centrally control the actions of 7.7 billion free humans. A lot of them will disagree with your position (and any other position as well). By trying to "put a stop to it" in a central manner, you are only making it harder (but not impossible) for some subset to learn about this phenomenon, to improve on it and to understand its strengths and weaknesses. I am unconvinced by your argument that we could reliably detect, much less stop, attempts to train a large and useful machine learning model.

by septor 7y ago

CamperBob2 7y ago

Correction: You are in a car that doesn't have any brakes.

krick 7y ago
I don't understand why it's downvoted (at the moment of writing, at least). I think it's absolutely correct. Imagine a thing "we should not do", let it be creation of really powerful language model, genetic engineering in order to produce smarter, stronger children, or whatever. Whatever you are opposing to, really. The simple fact is that if you (and by "you" I mean any entity you associate yourself with, be it literally you, or your company, or a group of researchers in your country, or the government of your country) can do something, anybody can. Even if it will be a couple of years later. You are not unique, nor alone. You can stop "yourselves" as long as you want, but there are other people, companies, research groups, governments, and they don't give a fuck about what you think "we" should do. So, in the end, the car really doesn't have any brakes. Even if it truly means the end of it all, it's just unfortunate, but really, really unavoidable.

jcims 7y ago

I think if you try to be less cynical about what OpenAI is doing you might even find them an ally to your perspective. Facebook throwing pocket change at an AI ethics org and Google's rather embarrassing failure at staffing an ethics panel of its own is evidence that we've got bicycle brakes on a freight train. OpenAI suffered a ton of blowback for not just releasing the full model from the start. You can read their intitial blog post [1] looking particularly at the sections for Policy Implications and Release Strategy. I would also highly recommend you listen to Lex Fridman's podcast with Greg Brockman [2] to hear their rationale about the recent org changes at Open AI. Obviously you can posit that everything they say is bullshit and they are only after almighty dollars. I can't prove it's not true and personally believe there is at least a kernel of truth to it, but we live in a messy world and finding imperfect allies is generally better than having none at all. [1] - https://openai.com/blog/better-language-models/ https://openai.com/blog/better-language-models/ [2] - https://www.youtube.com/watch?v=bIrEM2FbOLU https://www.youtube.com/watch?v=bIrEM2FbOLU

septor 7y ago
I’ve already read the blog post and watched the video. I’m not just some idiot who reads a few headlines and then comes to this conclusion. I found lex’s conversation to be frustrating beyond description. These people simply don’t understand the consequences of this technology. They think that an ethics board will help in some way. They are completely missing the point. “What’s going to stop bad things from happening?” “The charter” Good fucking god.

PostOnce 7y ago

What do you mean "Could we sense when someone was trying to do it", are we talking mandatory computer inspections, government-enforced walled gardens, and the death of the general purpose computer? lol, if you thought gun control was hard... Also, yeah, you could do it in your basement without being detected. You can do it in the cloud, too, a lot of us here have the skills and resources to do it, but why spend time on this toy instead of another one, especially if it's going to cost money we could spend on something more fun?

septor 7y ago
Ai in general is an area of active research. The AIs that will bring about the end of the world do not exist — they are not simply in need of an implementation. Right now ai is driven by the collective effort of many research entities. Universities and private businesses. Ai research in those places can be shut down very effectively with regulation. That would probably be enough to buy us centuries because progress in ai depends heavily on this open community of research. Making the use of high level signal processing illegal for business would further delay ai because there would be no incentive for anyone to develop models for profit. We audit taxes and lots of other things, auditing signal processing is not obviously impossible. As you mention, cloud computing is a thing. Cloud computing will likely be the nucleation point from which the first agi springs. It should be shut down. Through regulation or through the dismantling of the internet it should be shut down. Shutting it down would buy us lots of valuable time. ICs are not something you can make in your back yard. Controlling the compute density and energy efficiency or any other aspect of ICs is trivial because they require huge factories to manufacture. Huge amounts of space, energy and money are required to run those fabs. And there aren’t very many of them. Therefore they are highly susceptible to regulation and public oversight. We could stop the production of ICs that are too powerful. So, here you are in your basement. You are trying to train an AGI. You are stuck with old hardware that you somehow bought on the black market. You are risking your life to do it because it’s highly illegal. You don’t have any investment, legal or otherwise, because it’s highly illegal and also stands to make no profit for anyone. You don’t have access to cloud computing. The training is very slow. It is made even slower by the fact that you have to slowly charge a gigantic battery array before each training because without batteries the spike in wattage would be inexplicable for a house in the suburbs and probably trigger a visit from the power authority. The probability that anyone would succeed in this scenario is not zero, but it’s very low. Very, very low. And I think it would save the world.

feanaro 7y ago

> The error is on two levels: you are using too much analogy with other technologies. And you are writing off the possibility of stopping ai when it’s still not clear that it can’t be stopped. No. What I am saying is that it's impossible to centrally control the actions of 7.7 billion free humans. A lot of them will disagree with your position (and any other position as well). By trying to "put a stop to it" in a central manner, you are only making it harder (but not impossible) for some subset to learn about this phenomenon, to improve on it and to understand its strengths and weaknesses. I am unconvinced by your argument that we could reliably detect, much less stop, attempts to train a large and useful machine learning model.