4 ms·

Here's the thing, current computational power isn't enough to achieve the scale of model that would be required for "god like" AI. So isn't that kind of a natur

Here's the thing, current computational power isn't enough to achieve the scale of model that would be required for "god like" AI. So isn't that kind of a natural brake, where we have to wait for computing speed to catch up with our ambitions? Why do we need additional handicaps and slowdowns on top of that? I've seen a theory that the big tech companies are the ones pushing for regulation in order to achieve regulatory capture of this powerful new technology, which seems like a compelling theory indeed...

gsatic 4y ago
Yup Big tech, which is really big advertising if you take Google+Meta online ad market share, get their main revenue from Likes/clicks/links/views counting + personal data. As info exploded you needed this crap to work out what to show ppl. And ppl have learnt how to accumulate status/power/cash by gaming those numbers. Chat-GPT et al dont rely on any of these number. So all the infrastructure of the Advertising-Industrial Attention capture complex can be thrown in the sea. Which should have been done long ago but we got free email, video, chat, games, movies etc so we gave away attention for free. Behind the morons talking abt "god like" ai and similar ridiculous bullshit are mainly ppl worried about how they are going to capture the chimp troupes attention(and through it cash/status/power), if the tools they have relied on arent going to work anymore. Eventually AI startups will end up where Google and Facebook ended up cause how else to fund their server farms? Subscriptions are never enough. Just ask Newspapers or Netflix.
flangola7 4y ago
> Here's the thing, current computational power isn't enough to achieve the scale of model that would be required for "god like" AI. I don't see how that is the case at all. The human brain only needs about 100 watts. A GPT-3 instance only needs a few GPUs and maybe 1000 watts. GPT-4 is presumably in the same ballpark, let's say 10000 watts to be generous (even though it's probably closer to 1000 too), but still small enough to fit inside a full height rack. Many datacenters are megawatt-level. Seeing how GPT-4 is already human-level at many tasks and watt-to-subjective-capability increases seem to grow linearly at worst, an AI could be ran in a single data center and perform 100x GPT-4 abilities. I'm not sure what 10x, much less 100x GPT-4 abilities would be, but that sure sounds like godlike or near-god to me.
- verdverm 4y ago
  Simply scaling up the current LLM models will not fix the fundamental problems of hallucination and lack of planing abilities. If they cost $10M+ to train, will scaling that cost 100X (to $1B) really make sense. I find it doubtful that the ROI is there Consider that GPT4 is 15-30X the cost of GPT3.5, is it really 15-30X better? Not in my experience. I would expect that we are approaching the point of diminishing returns with text based LLMs, at least for the current architectures and training regimes. I'm interested to see what the Large Multi-Modal Models produce, but I suspect the planning & lying will still be there. Chaining these models will increase the cost even more, without certainty that it will produce usable results for anything beyond simple, context limited tasks.
  - rbanffy 4y ago
    When it becomes that large and expensive, it makes sense for governments to do it and open up the result so everyone can benefit.
    - verdverm 4y ago
      That also makes sense when we account for how gov'ts are really good at wasting money for below average results... This is not something I would trust them to do competently or within a reasonable budget. If it was done through the NSF or DARPA, with an industry partner, maybe? But I can't imagine the current politicians and them arguing about if & how to put restraints on it. It will be interesting to see how the CCP goes about this. Maybe the EU can offer a third alternative?
      - rbanffy 4y ago
        > That also makes sense when we account for how gov'ts are really good at wasting money for below average results... The goal of governments is not efficiency, but fairness and strategic action aiming at long-term results. I like to joke that one of the key goals of NASA is to prevent rocket scientists from moving to North Korea.
  - 4y ago
Llamamoe 4y ago
> I've seen a theory that the big tech companies are the ones pushing for regulation in order to achieve regulatory capture This... would track with what's historically been the case. Damn, that's an eye-opening take, thank you.
mach1ne 4y ago
>Here's the thing, current computational power isn't enough to achieve the scale of model that would be required for "god like" AI. We don’t know that. It would seem likely, perhaps, but we don’t know that.

gsatic 4y ago

Yup Big tech, which is really big advertising if you take Google+Meta online ad market share, get their main revenue from Likes/clicks/links/views counting + personal data. As info exploded you needed this crap to work out what to show ppl. And ppl have learnt how to accumulate status/power/cash by gaming those numbers. Chat-GPT et al dont rely on any of these number. So all the infrastructure of the Advertising-Industrial Attention capture complex can be thrown in the sea. Which should have been done long ago but we got free email, video, chat, games, movies etc so we gave away attention for free. Behind the morons talking abt "god like" ai and similar ridiculous bullshit are mainly ppl worried about how they are going to capture the chimp troupes attention(and through it cash/status/power), if the tools they have relied on arent going to work anymore. Eventually AI startups will end up where Google and Facebook ended up cause how else to fund their server farms? Subscriptions are never enough. Just ask Newspapers or Netflix.

flangola7 4y ago

> Here's the thing, current computational power isn't enough to achieve the scale of model that would be required for "god like" AI. I don't see how that is the case at all. The human brain only needs about 100 watts. A GPT-3 instance only needs a few GPUs and maybe 1000 watts. GPT-4 is presumably in the same ballpark, let's say 10000 watts to be generous (even though it's probably closer to 1000 too), but still small enough to fit inside a full height rack. Many datacenters are megawatt-level. Seeing how GPT-4 is already human-level at many tasks and watt-to-subjective-capability increases seem to grow linearly at worst, an AI could be ran in a single data center and perform 100x GPT-4 abilities. I'm not sure what 10x, much less 100x GPT-4 abilities would be, but that sure sounds like godlike or near-god to me.

verdverm 4y ago
Simply scaling up the current LLM models will not fix the fundamental problems of hallucination and lack of planing abilities. If they cost $10M+ to train, will scaling that cost 100X (to $1B) really make sense. I find it doubtful that the ROI is there Consider that GPT4 is 15-30X the cost of GPT3.5, is it really 15-30X better? Not in my experience. I would expect that we are approaching the point of diminishing returns with text based LLMs, at least for the current architectures and training regimes. I'm interested to see what the Large Multi-Modal Models produce, but I suspect the planning & lying will still be there. Chaining these models will increase the cost even more, without certainty that it will produce usable results for anything beyond simple, context limited tasks.
- rbanffy 4y ago
  When it becomes that large and expensive, it makes sense for governments to do it and open up the result so everyone can benefit.
  - verdverm 4y ago
    That also makes sense when we account for how gov'ts are really good at wasting money for below average results... This is not something I would trust them to do competently or within a reasonable budget. If it was done through the NSF or DARPA, with an industry partner, maybe? But I can't imagine the current politicians and them arguing about if & how to put restraints on it. It will be interesting to see how the CCP goes about this. Maybe the EU can offer a third alternative?
    - rbanffy 4y ago
      > That also makes sense when we account for how gov'ts are really good at wasting money for below average results... The goal of governments is not efficiency, but fairness and strategic action aiming at long-term results. I like to joke that one of the key goals of NASA is to prevent rocket scientists from moving to North Korea.
- 4y ago

Llamamoe 4y ago

> I've seen a theory that the big tech companies are the ones pushing for regulation in order to achieve regulatory capture This... would track with what's historically been the case. Damn, that's an eye-opening take, thank you.

mach1ne 4y ago

>Here's the thing, current computational power isn't enough to achieve the scale of model that would be required for "god like" AI. We don’t know that. It would seem likely, perhaps, but we don’t know that.