3 ms·

Title isn't quite right... The article is estimating the probability of widespread/consumer accessible/cost effective AGI. But AGI can be transformative without

Title isn't quite right... The article is estimating the probability of widespread/consumer accessible/cost effective AGI. But AGI can be transformative without being widespread. Replacing all human functions is not nessecary for transformation. A single superhumam AI could cost 1e6 that of a single human, but still do transformative work if it's 1.5x smarter than us. Edit: nevermind... This was an intentional definition of "transformative" made by the authors. Seems wrong to me, but they were up front about it. Not impressed though. Anyone can drum attention by using a non-standard definition of something. But of a cheap trick imo, but that's academia these days I guess... For reference, my definition of transformative would be "responsible for large scale changes to society". In other words, it doesn't need to happen at scale for impact to happen at scale. Replication of a solution is cheap once it is discovered.

carlmr 3y ago
I think if we get 1.5x as smart as the smartest human, then that will automatically scale after a few iterations of improving itself.
- kibibu 3y ago
  There should be a name for this idea. I propose The Kurzweilarity.
  - headcanon 3y ago
    There already is a term for it: "Foom" https://en.wiktionary.org/wiki/foom https://en.wiktionary.org/wiki/foom
- TrapLord_Rhodo 3y ago
  "As Smart" I don't think it needs that at all. It just needs to 'Code' 1.01 times better than the average programmer to enter a self improvement cycle.
pama 3y ago
It appears like a very poor analysis overall. These closing arguments suggest lack of deep thinking: “Consider Alphabet, the parent company of Google DeepMind. Alphabet is not constrained by financial capital: as of March 2023, Alphabet had $115B of cash on hand.202 Alphabet is also not constrained by human capital: as of March 2023, Alphabet had 191,000 employees.203 If Alphabet wanted to, it could easily hire more AI researchers and allocate more software engineers to AI research infrastructure. But it chooses not to. Why? This deliberate choice presumably implies that Alphabet management doesn’t believe there is enough ROI from hiring more smart people to accelerate progress on AGI research. To us, this suggests that Alphabet management is bearish on the likelihood of transformative AGI by 2043. We also see this as a stronger signal than public markets, as Alphabet management is likely much more informed about AGI progress than the average stock trader. Another implicit signal of low likelihood of transformative AGI is the comparatively low valuation of AGI companies. Consider Anthropic, which reportedly raised money in early 2023 at a valuation of ~$4B. $4B is ~0.004% of world product each year. If transformative AGI is soon able to do most human work, we expect it to be worth multiples of today’s world product. Supposing transformative AGI is worth $100T/yr, and Anthropic has a 10% chance of capturing 10% of the value for an expected duration of 10 years, that would imply a valuation of $10T (before time discounting). $4B is less than 1/1,000th of $10T, which seems to imply that Anthropic and its investors do not collectively believe that Anthropic has a chance of inventing transformative AGI by 2043 and capturing its value.”
antman 3y ago
AI can be disruptive and innovative without being AGI. I think AGI discussion is more useful for financing academic projects. Lower cost of gpt and whisper are good enough for business and dictatorships. Infinite drones with the current human recognition capability can do the leg work (pun), no universal AGI soldier needed. Just current AI and batteries!

carlmr 3y ago

I think if we get 1.5x as smart as the smartest human, then that will automatically scale after a few iterations of improving itself.

kibibu 3y ago
There should be a name for this idea. I propose The Kurzweilarity.
- headcanon 3y ago
  There already is a term for it: "Foom" https://en.wiktionary.org/wiki/foom https://en.wiktionary.org/wiki/foom
TrapLord_Rhodo 3y ago
"As Smart" I don't think it needs that at all. It just needs to 'Code' 1.01 times better than the average programmer to enter a self improvement cycle.

pama 3y ago

It appears like a very poor analysis overall. These closing arguments suggest lack of deep thinking: “Consider Alphabet, the parent company of Google DeepMind. Alphabet is not constrained by financial capital: as of March 2023, Alphabet had $115B of cash on hand.202 Alphabet is also not constrained by human capital: as of March 2023, Alphabet had 191,000 employees.203 If Alphabet wanted to, it could easily hire more AI researchers and allocate more software engineers to AI research infrastructure. But it chooses not to. Why? This deliberate choice presumably implies that Alphabet management doesn’t believe there is enough ROI from hiring more smart people to accelerate progress on AGI research. To us, this suggests that Alphabet management is bearish on the likelihood of transformative AGI by 2043. We also see this as a stronger signal than public markets, as Alphabet management is likely much more informed about AGI progress than the average stock trader. Another implicit signal of low likelihood of transformative AGI is the comparatively low valuation of AGI companies. Consider Anthropic, which reportedly raised money in early 2023 at a valuation of ~$4B. $4B is ~0.004% of world product each year. If transformative AGI is soon able to do most human work, we expect it to be worth multiples of today’s world product. Supposing transformative AGI is worth $100T/yr, and Anthropic has a 10% chance of capturing 10% of the value for an expected duration of 10 years, that would imply a valuation of $10T (before time discounting). $4B is less than 1/1,000th of $10T, which seems to imply that Anthropic and its investors do not collectively believe that Anthropic has a chance of inventing transformative AGI by 2043 and capturing its value.”

antman 3y ago

AI can be disruptive and innovative without being AGI. I think AGI discussion is more useful for financing academic projects. Lower cost of gpt and whisper are good enough for business and dictatorships. Infinite drones with the current human recognition capability can do the leg work (pun), no universal AGI soldier needed. Just current AI and batteries!