3 ms·

> But it's bogus because the exponential growth of Moore's law involves an exponential growth of inputs A continuing Moore's law is only one factor in the emer

> But it's bogus because the exponential growth of Moore's law involves an exponential growth of inputs A continuing Moore's law is only one factor in the emergence of AGI. It's possible the current progress of GPUs today is sufficient to run AGI given advancements in software. > unlikely to get a 100% kill I don't take much comfort in 99% of the human race dying, and the remaining 1% living in constant fear of being next. (Substitute 99:1 ratio with whichever you please.) > it will die if we die Why would you think this? Certainly an AGI that is 10x more intelligent than homo sapiens would be able to leapfrog Boston Dynamics's technology level by decades or perhaps centuries. > could not do it without an extended program of experimentation and development We're in that extended program period now. But when the AI can write its successors better and faster than we can, we should expect the iteration cycle to shorten exponentially upon each iteration, hence the short time to liftoff.

PaulHoule 4y ago
The progress of GPUs themselves has already hit the wall (look at the pricing of the 40-series.) We need to switch to something other than 2-d silicon structures if hardware progress is going to be great again. I'll grant that there are 1-3 orders of magnitude gain possible from better models as it seems really silly to use floats to encode the kind of knowledge that language models have and we are just starting to understand how that knowledge is represented in that kind of system. Insofar as the thing develops something like Drexler's nanotechnology it will develop it first, not call up some rando to make an order from a DNA synthesis house. That in particular is part of Yudkowsky's M.O. in that, like a 419 letter, he inserts a lot of garbage into everything he writes designed to drive away real rational thinkers because he is looking for followers for a cult that is some mash-up of L. Ron Hubbard, Teilhard de Chardin, and H. P. Lovecraft. That's the whole point of Sequences: the smart people realize it is crap, the not-so-smart people struggle with it and think it is profound.
- andrewfromx 4y ago
  I had to google "what's a 419 letter" but now I get it. Wow, I thought I was smart but I'm in the "not-so-smart people struggle with it and think it is profound." group today.

PaulHoule 4y ago

The progress of GPUs themselves has already hit the wall (look at the pricing of the 40-series.) We need to switch to something other than 2-d silicon structures if hardware progress is going to be great again. I'll grant that there are 1-3 orders of magnitude gain possible from better models as it seems really silly to use floats to encode the kind of knowledge that language models have and we are just starting to understand how that knowledge is represented in that kind of system. Insofar as the thing develops something like Drexler's nanotechnology it will develop it first, not call up some rando to make an order from a DNA synthesis house. That in particular is part of Yudkowsky's M.O. in that, like a 419 letter, he inserts a lot of garbage into everything he writes designed to drive away real rational thinkers because he is looking for followers for a cult that is some mash-up of L. Ron Hubbard, Teilhard de Chardin, and H. P. Lovecraft. That's the whole point of Sequences: the smart people realize it is crap, the not-so-smart people struggle with it and think it is profound.

andrewfromx 4y ago
I had to google "what's a 419 letter" but now I get it. Wow, I thought I was smart but I'm in the "not-so-smart people struggle with it and think it is profound." group today.