5 ms·
Do you have a rationale for that? I get a feeling progress in both machine learning and understanding biological intelligence is fairly rapid and has been accel
by TrainedMonkey 4y ago
Do you have a rationale for that? I get a feeling progress in both machine learning and understanding biological intelligence is fairly rapid and has been accelerating. I believe two primary contributing factors are cheaper compute and vast amount of investment poured into machine learning, see https://venturebeat.com/ai/report-ai-investments-see-largest-year-over-year-growth-in-20-years/ https://venturebeat.com/ai/report-ai-investments-see-largest...
Now, the question of whether we are going to have AGI is incredibly broad. So I am going to split it into two smaller ones:
- Are we going to have enough compute by year X to implement AGI. Note that we are not talking about super intelligence or singularity here. This AGI might be below human intelligence and incredibly uneconomical to run.
- Assuming we have enough compute, will we a way to get AGI working.
The compute advancements scale with new Chip Fabs linearly and tech node improvements exponentially. I think it is reasonable for compute to get cheaper and more accessible throughout at least 2030. I expect this because TSMC is starting 3nm node production, Intel is decoupling fabing and chip design (aka TSMC model), and the strategic investments into into chap manufacturing driven by supply chain disruptions. See https://www.tomshardware.com/news/tsmc-initiates-3nm-chips-production-next-month https://www.tomshardware.com/news/tsmc-initiates-3nm-chips-p...
How much compute do we need? This is hard to estimate, but amount of human connections in human brain is estimated at 100 trillion, that is 1e14. Current largest model has 530B parameters, that is 5.3e11: https://developer.nvidia.com/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/ https://developer.nvidia.com/blog/using-deepspeed-and-megatr... . That is factor of 500 or 9 doublings off. To get there by 2040 we would need a doubling every 2 years. This is slower that recent progress, but past performance does not predict future results. Still, I believe getting models with 1e14 parameters by 2040 is possible for tech giants. I believe it is likely that a model with 1e14 parameters is sufficient for AGI if we know how structure and train it.
Will we know how to structure and train it? I think is mostly driven by investment into the AI field. More money means more people and given the venture beat link above the investment seems to be accelerating. A lot of that investment will be unprofitable, but we are not looking to make a profit - we are looking for breakthroughs and larger model sizes. Self-driving, stock trading, and voice controls are machine learning applications which are currently deployed in the real world. At the very least it is reasonable to expect continuous investment to improve those applications.
Based on the above I believe we would need to mess things up royally to not get AGI by 2100. Remember this could be below human and super uneconomical AGI. I am rather optimistic, so my personal prediction is that we have 50% chance to get AGI by 2040 and 5-10% chance of getting there by 2030.