2 ms·
> 1. The intelligence of an AI model roughly equals the log of the resources used to train and run it. > 2. The cost to use a given level of AI falls about 10x
by abetusk 2y ago
> 1. The intelligence of an AI model roughly equals the log of the resources used to train and run it.
> 2. The cost to use a given level of AI falls about 10x every 12 months, and lower prices lead to much more use.
> 3. The socioeconomic value of linearly increasing intelligence is super-exponential in nature.
My own editorial:
Point 1 is pretty interesting as if P != NP then this is essentially "we can't do better than random search". So, in order to find progressively better solutions as we increase the (linear) input, we need exponentially more resources to find the answer. While I believe P != NP it's interesting to see this play out in the context of learning and AI.
Point 2 is semi well-known. I'm having trouble finding it but there was an article a while back talking about algorithmic efficiencies to the DFT (or DCT?) were outpacing efficiencies that could be just attributed to Moore's law. Meaning the DFT was improving a few orders of magnitude faster than just Moore's law would imply. I assume this is essentially a Wright's law but for attention, in some sense, where more attention to problems leads to better optimizations that dovetail with Moore's law.
Point 3 seems like it's almost a corollary, at least in the short term. If intelligence is capturing the exponential search and it can be re-used to find further efficiency, as in point 2, you get super-exponential growth. I think Kurzweil mentioned something about this as well.
I haven't read the whole article but this jumped out at me:
> Our mission is to ensure that AGI (Artificial General Intelligence) benefits all of humanity.
A bald faced lie. Their mission is to capture value from developing AGI. Any benefit to humanity is incidental.