4 ms·
I was recently in a work meeting with some higher ups where we discussed the near-term future of AI. The idea that this is the interface revolution on-par with
by aimor 4y ago
I was recently in a work meeting with some higher ups where we discussed the near-term future of AI. The idea that this is the interface revolution on-par with GUIs and the WWW was mentioned, and attributed to Bill Gates (I guess he must have talked about this recently). I've been dwelling on everyone's expectations for a few days and I see the hype cycle.
A good reason to consider "what if" is that it's still going to take a lot of effort to get from where we are to where expectations are. There's been many false starts recently: self-driving cars, 3D printing, VR/AR, crypto. 10 years on and they're all real technologies anyone can use and making progress every day, but the wild expectations hit reality. When we get there with AI (and we will, because expectations are a moving goalpost) it's good to have considered what we want to do about it. We don't want to waste years of resources on insurmountable roadblocks, we want ideas about why what worked before stopped and what easier paths forward might exist.
That is, I don't see many people downplaying AI but I do see people with lots of unanswerable questions about how long this current boom will last and where we'll wind up when it's over.
- pixl97 4y agoBill Gates AI episode on the Microsoft Youtube channel recently https://youtu.be/bHb_eG46v2c https://youtu.be/bHb_eG46v2c
- Symmetry 4y agoHe also wrote his thoughts out longform. https://www.gatesnotes.com/The-Age-of-AI-Has-Begun https://www.gatesnotes.com/The-Age-of-AI-Has-Begun
- kurthr 4y agoI agree and I find some of the statements a bit odd (not thought out?). I think Moore’s Law could keep going for decades.[2] But even if it doesn’t... If 1e35 FLOP is enough to train a transformative AI (henceforth, TAI) system, which seems plausible, I think we could get TAI by 2040... First, I don't see Moore's Law going sub-atomic without a complete change in methodology, which would delay the results. Wafer/die stacking are cool, but stop-gap measures. In particular he acknowledges that power consumption hasn't scaled since 2005 (in fact chiplets help by only sqrt2!). The challenge is that it doesn't help that much to increase the number of transistors, if their latency/power/cost don't fall as well. We're approaching that even ignoring any geopolitical issues. Second, power consumption is not improving except by going to 8&16 bit... there's not a lot of room there. Currently, we get <1000GFlOPs/W and even if we get 100x up to 100TFLOPs/W, you still need 10^17kWhr. OK algorithms get us another 10,000x... and it costs $1T to train 1 transformative AI. How many proof of concepts trials will be do at only $100B a pop? It all just seems a bit flip and hopeful like Feb 2000. The 10^35 number seems like its just pulled out to be a number, when it could orders of magnitude up/dn. https://www.researchgate.net/publication/354573934_Compute_and_Energy_Consumption_Trends_in_Deep_Learning_Inference/figures?lo=1 https://www.researchgate.net/publication/354573934_Compute_a...
- erwald 4y agoI'm curious, did you read beyond the summary? (I don't mean that in a snide way, it's totally fine just to read the summary -- that's why it's there.) The 1e35 FLOP number is meant as a conservative upper bound and comes from here: https://www.lesswrong.com/s/5Eg2urmQjA4ZNcezy/p/rzqACeBGycZtqCfaX#comments https://www.lesswrong.com/s/5Eg2urmQjA4ZNcezy/p/rzqACeBGycZt... The major fabs all have roadmaps for approaching 1 nm, and there are other advances that could allow you to keep going either if transistor size scaling stops (e.g., vertical scaling). (That said, I definitely don't think it's a given that HW price-performance keeps doubling at the same rate 10+ years.)