13 ms·
OpenAI
- IIAOPSW 4y agoA genius plan indeed. If he can get the AI to write his papers, he'll never have to stop being on sabbatical.
- jks 4y agoIn academia, a "sabbatical" means you take some time off from teaching courses, advising students and doing administrative work so you can concentrate on your research. So in order to stay on sabbatical, he'd need to get the AI to do that other stuff.
- ksm1717 4y ago
- yoquan 4y agoObviously it will relate to complexity in deep learning but I can't resist thinking about some AI models involving quantum computation stuffs :-)
- chis 4y agoSeeing someone as trustworthy as Scott choose to work on AI safety is a pretty good sign for the state of the field IMO. It seems like a lot of studious people agree AI alignment is important but then end up shoehorning the problem into whatever framework they are most expert in. When all you have is a hammer etc... I feel like he has good enough taste to avoid this pitfall. Semi-related - I'd want to see some actual practical application for this research to prove they're on the right track. But maybe conceptually that's just impossible without a strong AI to test with, at which point it's already over? Alignment papers are impressively complex and abstract but I have this feeling while reading them that it's just castles made of sand.
- arturkane7 4y agoSymmetrically someone like him transitioning from quantum computing should imply something negative about the state of quantum computing.
- deleted 4y ago[deleted]
- gspr 4y agoMaybe he just wants to use his sabbatical to try something different? Someone in his position doesn't have to remain laser focus on their own field.
- astrange 4y agoHe mostly studies computational complexity. Quantum computing is a part of that, but there's other subfields. Though the kind of AI safety described in this post seems more like an extremely fancy version of program verification, so out of CS bloggers you'd expect John Regehr to get into it.
- sigmoid10 4y ago>the kind of AI safety described in this post seems more like an extremely fancy version of program verification It kind of is. The field of AI safety is actually much more advanced than most people realise, with actual, real techniques to e.g. make sure neural networks are aligned with certain goals even under fluctuating parameters. Granted, we're still far from soothing an AGI before it can do something bad, but the tools we have today are already pushing in that direction (assuming neural networks are the right way to AGI of course).
- sydthrowaway 4y agoProbably realized quantum computing is going nowhere fast.
- sgt101 4y agoDepends what you mean by quantum computing; if you mean the warehouse sized labs full of lasers and cryogenics then you are correct. If you mean the study of quantum algorithms then actually it's been an exciting year. The first polynomial speed-up for an NP unstructured pseudo random function was published in April. It's not even clear how big a deal it is!
- FartyMcFarter 4y agoQuantum computing is not everything he does, so even if you're right it wouldn't explain this. Going on a sabbatical is not that weird.
- ComplexSystems 4y ago"When you start reading about AI safety, it’s striking how there are two separate communities—the one mostly worried about machine learning perpetuating racial and gender biases, and the one mostly worried about superhuman AI turning the planet into goo" - great quote.
- ductsurprise 4y agoAgreed. Maybe... Short sighted vs. far sighted vs. ignorance is bliss?
- astrange 4y agoIt's a secular/religious divide. (As it says in the post.) Though it's possible the people who think a theoretical future AI will turn the planet into paperclips have merely forgotten that perpetual motion machines aren't possible.
- eru 4y agoCould you please explain your reasoning for the comparison to perpetual motion machines? (Keep in mind that biological machines, ie life, have managed to turn the surface of the planet into 'green goo'.)
- astrange 4y agoSelf-reproducing machines are capable of covering the surface of the planet, yes. There's one right now (covid). But there's lots of energy and oxygen up here and they rarely displace other such machines (species) or even displace much of any earth and water. And because they're self-contained and self-reproducing, all of their instructions can be lost over time to entropy including the ones we're afraid of. None of em replace the entire planet though. That's a lot of rock to digest without any more energy to help you do it. And a paperclip factory isn't self-reproducing (that would be a paperclip factory factory). It's just a regular machine that can break down. The people afraid of that one are imagining a perfect non-breaking-down non-energy-requiring machine because they've accidentally joined a religion.
- tominous 4y agoCongrats to Scott. I'm still somewhat disappointed Brendan Gregg didn't pop up there making the AI Alignment equivalent of flame graphs!
- hadlock 4y agoI thought we decided 20 threads ago, that OpenAI is anything but open?
- jb1991 4y agoExactly. It’s time for a name change.
- dubswithus 4y agoHow about... AnythingButAGI?
- deleted 4y ago[deleted]
- Jerry2 4y agoSo, physics is a dead-end? Given that Scott is running his own research lab, a year is a very long time and him working out of his field is an indication that physics is in a big trouble.
- ivalm 4y agoQuantum computing is slow to mature. Physics is a huge field.
- eru 4y agoMaybe. Or maybe it's just an indication that he wants to do something else for a while?
- deleted 4y ago[deleted]
- texaslonghorn5 4y agoI am confused, how did you reach that conclusion? How does this announcement relate to the future of physics research? Sure, Scott's research is at the intersection of complexity and physics, but he is a CS professor at Texas working in the theoretical computer science department. His work leans far more towards TCS, with some work having connections to cosmology (he cares about physical limits of the universe and the information theory of things like black holes) and other interesting ideas from physics. But the main themes of his work have been quantum algorithms and complexity for a while. He's also nowhere near the experimental side of physics.
- texaslonghorn5 4y agoAlso copied from a couple blog posts ago he doesn't self-identify as a physicist either. https://scottaaronson.blog/?p=6457 https://scottaaronson.blog/?p=6457 I also had the following exchange at my birthday dinner: Physicist: So I don’t get this, Scott. Are you a physicist who studied computer science, or a computer scientist who studied physics? Me: I’m a computer scientist who studied computer science. Physicist: But then you… Me: Yeah, at some point I learned what a boson was, in order to invent BosonSampling. Physicist: And your courses in physics… Me: They ended at thermodynamics. I couldn’t handle PDEs. Physicist: What are the units of h-bar? Me: Uhh, well, it’s a conversion factor between energy and time. (*) Physicist: Good. What’s the radius of the hydrogen atom? Me: Uhh … not sure … maybe something like 10-15 meters? Physicist: OK fine, he’s not one of us.
- d--b 4y agoAI is not going to become self aware and destroy the world. AI is going to cause something like the industrial revolution of the 19th century: massive changes in who is rich, massive changes in the labor market, massive changes in how people make war, etc. It’s already started really. What worries me most is that as long as society is capitalist, AI will be used to optimize for self-enrichment, likely causing an even greater concentration of capital than what we have today. I wouldn’t be surprised that the outcome is a new kind of aristocracy, where society is divided between those who have access to Ai and those who don’t. And that I don’t think falls into the “Ai safety” field. Especially since OpenAi is Vc-backed
- atmosx 4y agoTo have access to the forefront of AI you have to be super-rich. I don’t see how AI will change that, if anything will make it harder to change by giving yet another advantage to those who already have plenty.
- zzzzzzzza 4y agouhm, i spend less than 40 cents a day on avg on gpt3 queries... a bit more accessible than like a hackerspace membership or building a factory or something
- omnicognate 4y agoYou don't have access to the forefront of AI. You have the ability to give money to those that do so you can use what they've made. To have access to the forefront of AI means being able to make, own and profit from things like GPT-3, and it requires access to vast computational and data resources.
- sva_ 4y agoI doubt GPT-3 is profitable as of now.
- 4y ago
- natch 4y agoIf only we could get the AI to help us figure out how to eliminate the use of full justify once and for all.
- encryptluks2 4y agoFrom the website header: > Also, next pandemic, let's approve the vaccines faster! This is obviously very important to them. Is there some proof that the vaccine was unnecessarily delayed or just that they believe if we mess up and humanity suffers, so what?
- drdeca 4y agoProof? The point aiui is mostly arguing that the FDA errs too much on the side of caution in this area, and the trade-off would have been worth it to approve earlier. Not insinuating that like, there was some corruption (or laziness or something) that delayed it.
- pas 4y agoprobably the best phrasing would be even faster basically let's set up a standing pipeline to develop multivalent vaccines for every coming season (we already have the yearly for influenza)
- Liron 4y agoI'm really happy this is happening and hope to see more. Namely, the AI safety & alignment challenge attracting our best minds who would previously have prioritized other math, physics and comp sci.
- newbye5 4y agoX
- ksm1717 4y agoLots of really cogent points. Hopefully he reads this comment and swiftly begins working on designing a Diplomatic Agent to avoid the MAD or getting a good peace treaty
- nlqrt 4y ago"[...] where the misuse of AI for spambots, surveillance, propaganda, and other nefarious purposes is already a major societal concern [...]" I'm curious what he will do and whether for example he approves of the code laundering CoPilot tool. I also hope he'll resist being used as an academic promoter of such tools, explicitly or implicitly (there are many ways, his mere association with the company buys goodwill already).
- pas 4y agowhat's wrong with copilot as a concept and as the concrete implementation? it's a fancy autocomple. we had stack overflow based autocomplete before. this got a bigger training data set.
- cillian64 4y agoThe objection here is that its training was based on code on github without paying any attention to the license of that code. It’s generally considered ok for people to learn from code and then produce new code without the new code being considered a derived work of what they learned from (I’m not sure if there is a specific fair use clause covering this). But it’s not obvious that copilot should be able to ignore the licenses of the code it was trained on, especially given it sometimes outputs code from the training set verbatim. One could imagine a system very similar to copilot which reads in GPL or proprietary code and writes functionally equivalent code while claiming it’s not a derived work of the original and so isn’t subject to its licensing constraints.
- hansword 4y ago'OpenAI, of course, has the word “open” right in its name, ... don’t expect me to share any proprietary information' Yeah, Mr Aaronson just lost quite a bit of respect from my side. Going into AI is a great move, moving to the ClosedAI corporation.......? Why? (Edit: Removed an outdated reference to Elon Musk, thanks @pilaf !)
- pilaf 4y agoElon Musk left the board of OpenAI in 2018.
- hansword 4y agoMuch thanks, didn't know that, edited my comment.
- nopinsight 4y agoThe latest huge AI models that may germinate into an AGI all come from private corporations, largely because of the requisite resources and currently there’re very few if any public or nonprofit AI-focused organizations with such resources.
- hansword 4y ago> huge AI models that may germinate into an AGI are science fiction.
- dreeves 4y agoScott Aaronson adds the following in the comment on his blog post in response to a question about this: > the NDA is about OpenAI’s intellectual property, e.g. aspects of their models that give them a competitive advantage, which I don’t much care about and won’t be working on anyway. They want me to share the research I’ll do about complexity theory and AI safety.
- frozencell 4y agoWorking on OpenAI instead of trying unconventional options such as decentralized models governance might increase inequalities . Why would the community decide to repeat what they denounce in big tech?
- fny 4y agoNaive question: Isn't the easiest way to prevent an general purpose AI from taking over the world to not make a general purpose AI?
- cal85 4y agoUnfortunately that just means you don’t create one. To prevent one being created, you have to either somehow figure out a way to get everyone in the world to agree not to create one, or obtain enough global power that you can forcibly stop anyone creating one. Not exactly easy!
- afthonos 4y agoA powerful AI could help with that! >.>
- lrhegeba 4y agoreminds me of a very frightening quote from security specialist Gavin de Becker (https://en.wikipedia.org/wiki/Gavin_de_Becker https://en.wikipedia.org/wiki/Gavin_de_Becker), paraphrased: "every evil that you can think of, someone will have done it"
- orzig 4y agoThe assumption many in the field make is that _someone_ is going to create General Purpose AI, and we'd rather it be people who want it to be 'good' (aka 'aligned'). Best case, that AI can prevent the creation of harmful AI, though that's glossing over a lot of details that I'm not qualified to describe.
- astrange 4y agoIf you want to stop the world from containing general intelligence, you'd have to stop everyone from having children, which are equally generally intelligent to AGIs (but possibly less specifically intelligent) and are even more dangerous since then actually exist. The reason people don't accuse every random child of possibly ending the world is because things that actually exist are just less exciting.
- shadycuz 4y agoWould love to see how OpenAI does CI/CD. Both with the models and while training.
- gitfan86 4y agoThe debate around "What is AGI?" is becoming increasingly irrelevant. If in two iterations of DallE it can do 30% of graphic design work just as well as a human, who cares if it really "understands" art. It is going to start making an impact on the world. Same thing with self driving. If the car doesn't "understand" a complex human interaction, but still achieves 10x safety at 5% of the cost of a human, it is going to have a huge impact on the world. This is why you are seeing people like Scott change their tune. As AI tooling continue to get better and cheaper and Moore's law continue for a couple years, GTP will be better than humans at MANY tasks.
- humanistbot 4y ago> If in two iterations of DallE it can do 30% of graphic design work just as well as a human, who cares if it really "understands" art. It is going to start making an impact on the world. From an AI safety perspective, it is because understanding is a key step towards general-purpose AI that can improve / reprogram itself in any arbitrary way.
- abeppu 4y agoWhy does the question of defining AGI even need to enter into this? Aaronson's post only sort of obliquely touches on AGI, via OpenAI's stated founding mission, and Yudkowsky's very dramatic views. Most of the post is on there being signs that the field is ready for real progress. AI safety can be an interesting, important, fruitful area without AI approaching AGI, or even surpassing human performance on some tasks. We would still like to be able to establish confidently that a pretty dumb delivery drone won't decide to mow down pedestrians to shorten its delivery time, right?
- theptip 4y agoIt’s worth being clear about what AI risk is. This has nothing to do with “AI may do some harm by putting lots of people out of work”. The idea is that there is _existential risk_ (ie species-extinction) once an AI can self-modify to improve itself, therefore increasing its own power. A powerful AI can change the world however it wants, and if this AI is not aligned to human interests it can easily decide to make humans extinct. Scott said in the OP that he now sees AGI as potentially close enough that one can do meaningful research into alignment, ie it’s plausible that this powerful AI could arrive in our lifetimes. So he is claiming the opposite of you; AGI is more relevant than ever, hence the career change. I agree with your premise that non-General AI will continue to improve and add lots of value, but I don’t think your conclusion follows from that premise.
- tlringer 4y agoI'm not sure how Scott ended up buying the party line of the weird AGI Doomsday Cult but so be it. In any case, none of the things he says about verifying AI in this post make any sense at all, and if OpenAI actually cares about verifying AI and not just about hiring people who believe in the AGI Doomsday party line, probably they should hire verification people. Alas, that is not the point.