5 ms·
After reading the NYT interview, I don’t understand why he still chose to invent, in his words, a dangerous technology and publish the results openly. Not a cr
by politician 3y ago
After reading the NYT interview, I don’t understand why he still chose to invent, in his words, a dangerous technology and publish the results openly.
Not a criticism of the man, but of the article.
- arkitaip 3y agoFame and greed, what else.
- trgdr 3y agoYeah I don't want to be unfair or unkind, but his responses in this article seem to reflect rather poorly on his character. The thought process seems to be something like: "There was an opportunity for someone to gain notoriety and money at a profound cost to the human race. Someone was going to do it. I don't actually feel bad about being the one to benefit, but it is fashionable to pretend to have a conscience about such things."
- mindvirus 3y agoIsn't it true of everything though? Explosives, airplanes, electricity, computers - all double edged swords that have both greatly benefited humanity and caused great harm (with the potential for a lot more of both).
- trgdr 3y agoIt is certainly true that there are many inventions that pose some sort of threat to humanity, and that they are generally pursued by people who have some sort of personal/professional interest in their development. In that respect, this isn't particularly different. The sentiment of "Oh by the way this stuff is super bad and dangerous so we should be careful" just rings pretty hollow from someone who is at the tail-end of a career spent in pursuit of that exact bad, dangerous thing. If he were 20 years younger or not wealthy it's hard to believe that he would be saying this out loud, even if he believed it. Also this sentiment rings _extra_ hollow from someone who supposedly left CMU because he didn't want to accept Pentagon funds or work on things that would be used for war. That feels like either an incoherent value system or some pretty substantial half-truths to me.
- cma 3y agoHe's also saying though that his estimate of when it may surpass human intelligence is much less than he estimated just a few years ago.
- mitchdoogle 3y agoIts a science fiction trope, perhaps a trope in real life as well. Brilliant scientist gets paid to work on potentially dangerous thing. They know it is potentially dangerous so they warn about it and are reassured over and over again that nothing will be done without their consent, or that things will be done with the utmost care and security. And then scientist finally succeeds in creating the thing and the business owner's greed takes over and releases it in a premature way.
- rscho 3y agoBecause it's their best shot at ensuring their kids' well-being? And if they don't have children, maybe they simply don't care.
- berkeleyjunk 3y agoAfter the war, Robert Oppenheimer remarked that the physicists involved in the Manhattan project had "known sin". Von Neumann's response was that "sometimes someone confesses a sin in order to take credit for it." - From Norman Macrae's John von Neumann book
- yowzadave 3y agoI have this same question about the (apparently many) AI researchers who believe it poses significant risks to humanity, yet still push forward developing it as fast as they can.
- beanbean01 3y agoHere's my theory: if you look at surveys, it does say a 10% chance or so of an extremely bad outcome. BUT it says a ~20% chance of an extremely good outcome, and an 80% chance of at least a neutral one. Simple cost benefit analysis.
- mitchdoogle 3y agoJust guessing, but I'm sure they get paid very well and receive promises from their companies that everything will be done ethically, nothing rushed, etc. We've seen now that OpenAI and Microsoft and Google care more about the business case rather than doing things ethically and carefully.
- code_runner 3y agoIf a whistleblower for these companies came out and said "For the last decade advanced research has been conducted on extraordinarily big LLMs and they won't even give the public a clue of what it is and how it works" you would get a combination of people that a) don't care and b) vilify the companies for not being open and having some demonstration of this secret super power. "why can't joe-schmo get his hands on this technology", "how can we trust something we can't see and use", etc. A lot of the capabilities of these models are emerging as people discover them. I truly don't believe you can make everyone happy with this tech, but isn't it better than the general public can at least explore it? Do people think that nobody was ever going to try to improve on transformers with more compute, more data, and more parameters? We knew splitting an atom was going to cause a big boom.... thats not really how this tech emerged.
- broast 3y ago"Intelligence, uh, finds a way."
- 3y ago
- ncr100 3y agoThat's assuming something. Think about it otherwise: how do you know it's dangerous until you've seen it in real life? You raise a kid, they end up being a murderer, should you have aborted them?
- politician 3y agoPer the article, he had early misgivings — moving to Canada and refusing money from DoD. It’s not anything like your hypothetical.
- ncr100 3y agoAhh - I see - thank you @politician, that is right.
- abm53 3y agoHe partly answers this in the article: “because if I didn’t, someone else would”. He states himself that it’s not a convincing argument to some. But it surely carries some weight: in developing nuclear weapons many scientists made the same calculation even though the invention is a wicked one, in and of itself.
- tgv 3y agoSo, let someone else do it. It's the laziest excuse.
- Satam 3y agoUntamed nature is far more dangerous to humanity than human technology. As recently as in the 1900s, the average life expetency at birth was 30-40 years. We're shooting guns, nuking nukes and engineering viruses, and still, on average, we're better off with all that than without it.
- karaterobot 3y agoI imagine there will be a lot of people who agree that AI is dangerous, but continue to use it, because provides something of value to them in the short term. In his case, he might really believe AI is a potential danger, but also wanted to get the notoriety of publishing, and the money and excitement of founding a successful startup. There's not a big difference between our kind of hypocrisy — supporting something we suspect is destructive in the long term because it is neat, convenient, or popular in the short term — and his kind. Both are part of the reason things get worse rather than better. His kind is more lucrative, so it's actually less surprising in a way.