4 ms·

The way I see it, it's clear that human-level intelligence can be achieved with hardware that's toaster-sized and consumes 100 watts, as demonstrated by our bra

The way I see it, it's clear that human-level intelligence can be achieved with hardware that's toaster-sized and consumes 100 watts, as demonstrated by our brains. Obviously there is some minimum requirements and limitations, but they aren't huge, there are no physical or info-theoretical limits that superhuman intelligence must require a megawatt-sized compute cluster and all the data on the internet (which obviously no human could ever see). The only reason why currently it takes far, far more computing power is that we have no idea how to build effective intelligence, and we're taking lots of brute force shortcuts because we don't really understand how the emergent capabilities emerge as we just throw a bunch of matrix multiplication at huge data and hope for the best. Now if some artificial agent becomes powerful enough to understand how it works and is capable of improving that (and that's a BIG "if", I'm not saying that it's certain or even likely, but I am asserting that it's possible) then we have to assume that it might be capable of doing superhuman intelligence with a quite modest compute budget - e.g. something that can be rented on the cloud with a million dollars (for example, by getting a donation from a "benefactor" or getting some crypto through a single ransomware extortion case), which is certainly below the level which would draw attention. Perhaps it's unlikely, but it is plausible, and that is dangerous enough to be a risk worth considering even if it's unlikely.

SanderNL 3y ago
Using this logic, flexible, self-healing, marathon running, juggling and childbearing robots that run on the occasional pizza are just around the corner, because, nature. It might us a thousand years to get anywhere close? I don’t see the good arguments for all of this happening soon.
- rl3 3y ago
  It'd be interesting if we could calculate the amount of power consumed in aggregate by evolutionary processes over millions of years. Unfortunately we could probably optimize it, a lot.
- pixl97 3y ago
  So with your closing comment you're arguing it is possible, now you're just talking about time frames. All I have to say is that in 1900 many thought flights by heavier than aur craft were tens of thousands of years away. 3 years later it was achieved.
  - Hasu 3y ago
    Of course in the 1950s many thought that computer vision and artificial intelligence were only a few months to years away, and here we are 70 years later and we're still working on those problems. Predicting the future is hard. Some problems are harder than expected, others are easier than expected. But generally I'd say history favors the pessimists, the cases where a problem gets solved suddenly and there's a major breakthrough get a lot of press and attention, but they're a minority in the overall story of technological progress. They're also unpredictable black swan events - someone might crack AGI or a unified theory of physics tomorrow, or it might not happen for ten thousand years, or ever.
  - spookie 3y ago
    I firmly believe that we are severely underestimating the problem space. Given there are a multitude of scientific areas focusing on human nature, and even then, they've shown difficulty on explaining each of its parts. Look, we can make assumptions given a much simpler technology, and the outcomes of our past selves. But, while the physics for those wings was pretty much good enough at the time, the aforementioned scientific areas aren't. And we know it.
williamtrask 3y ago
> there are no physical or info-theoretical limits that superhuman intelligence must require a megawatt-sized compute cluster and all the data on the internet (which obviously no human could ever see). Much of your "intelligence" is a function of natural selection. This is billions of years X bajillions of creatures in parallel, each processing tons of data at a crazy fast sampling rate in an insanely large/expensive environment (the real world). Humanity's algorithm is evolution moreso than the brain. Humans learn for a little while, start unlearning, and then die — which is an important inner for loop in the overall learning process. Taken together, there is some evidence to suggest that superhuman intelligence must require megawatt-sized compute cluster and all the data on the internet (and a lot... LOT more)
- pixl97 3y ago
  LLMs we're messing with are text data only, we're barely starting to eat video data in multi modal LLMs. This world doesn't lack data.
  - mrtranscendence 3y ago
    I'm not sure why watching Linus Tech Tips and makeup tutorials is going to give AI a better shot at super-intelligence, but sure?
- jabradoodle 3y ago
  Evolved creatures are somewhat handicapped by needing to make only incremental changes from one form to the next, and needing to not be eaten by predators while doing so. That isn't strong evidence of what would be required by a well engineered system with none of those constraints. Intelligence is not the end of goal of evolution, it is a by product.

SanderNL 3y ago

Using this logic, flexible, self-healing, marathon running, juggling and childbearing robots that run on the occasional pizza are just around the corner, because, nature. It might us a thousand years to get anywhere close? I don’t see the good arguments for all of this happening soon.

rl3 3y ago
It'd be interesting if we could calculate the amount of power consumed in aggregate by evolutionary processes over millions of years. Unfortunately we could probably optimize it, a lot.
pixl97 3y ago
So with your closing comment you're arguing it is possible, now you're just talking about time frames. All I have to say is that in 1900 many thought flights by heavier than aur craft were tens of thousands of years away. 3 years later it was achieved.
- Hasu 3y ago
  Of course in the 1950s many thought that computer vision and artificial intelligence were only a few months to years away, and here we are 70 years later and we're still working on those problems. Predicting the future is hard. Some problems are harder than expected, others are easier than expected. But generally I'd say history favors the pessimists, the cases where a problem gets solved suddenly and there's a major breakthrough get a lot of press and attention, but they're a minority in the overall story of technological progress. They're also unpredictable black swan events - someone might crack AGI or a unified theory of physics tomorrow, or it might not happen for ten thousand years, or ever.
- spookie 3y ago
  I firmly believe that we are severely underestimating the problem space. Given there are a multitude of scientific areas focusing on human nature, and even then, they've shown difficulty on explaining each of its parts. Look, we can make assumptions given a much simpler technology, and the outcomes of our past selves. But, while the physics for those wings was pretty much good enough at the time, the aforementioned scientific areas aren't. And we know it.

williamtrask 3y ago

> there are no physical or info-theoretical limits that superhuman intelligence must require a megawatt-sized compute cluster and all the data on the internet (which obviously no human could ever see). Much of your "intelligence" is a function of natural selection. This is billions of years X bajillions of creatures in parallel, each processing tons of data at a crazy fast sampling rate in an insanely large/expensive environment (the real world). Humanity's algorithm is evolution moreso than the brain. Humans learn for a little while, start unlearning, and then die — which is an important inner for loop in the overall learning process. Taken together, there is some evidence to suggest that superhuman intelligence must require megawatt-sized compute cluster and all the data on the internet (and a lot... LOT more)

pixl97 3y ago
LLMs we're messing with are text data only, we're barely starting to eat video data in multi modal LLMs. This world doesn't lack data.
- mrtranscendence 3y ago
  I'm not sure why watching Linus Tech Tips and makeup tutorials is going to give AI a better shot at super-intelligence, but sure?
jabradoodle 3y ago
Evolved creatures are somewhat handicapped by needing to make only incremental changes from one form to the next, and needing to not be eaten by predators while doing so. That isn't strong evidence of what would be required by a well engineered system with none of those constraints. Intelligence is not the end of goal of evolution, it is a by product.