4 ms·

Systems around us and designed by us tend to have diminishing returns problem. I wonder what's the limiting factor in the architecture of the human brain should

Systems around us and designed by us tend to have diminishing returns problem. I wonder what's the limiting factor in the architecture of the human brain should we want to scale it further. How much more intelligent can the cortex get without a massive architectural shift? I like to think that our first intelligent machines will run on some very specialized hardware with, by definition, designed particular architecture. I suppose both will have many different limiting factors to how deeply can a machine reason about itself. That's why I believe there won't be a runaway effect in intelligent modelling/reasoning. It'll be a step function. Another related issue is that a new architecture which breaks the scalability limits of previous generation will produce new and distinct entities. If intelligence and self-interest often correlate, a machine might be vary of creating a better version of itself lest it be replaced.

jacknews 5y ago
> Another related issue is that a new architecture which breaks the scalability limits of previous generation will produce new and distinct entities. If intelligence and self-interest often correlate, a machine might be vary of creating a better version of itself lest it be replaced. That assumes AI will manifest as multiple individual entities. I day-dreamed about this many years ago - what are the limits to intelligence, and what possible forms can it take, are 'societies' like minds, etc? It seems to me possible that single, distributed mind might be the ultimate form of intelligence, in which case the AI would not be replacing itself, but upgrading itself. Perhaps that is already happening.
- walleeee 5y ago
  your interpretation seems reasonable somehow we persist in the belief that misaligned superintelligence is a thought experiment but as far as I can tell distributed AGI is a reality and paperclip maximizers already exist
- porkbrain 5y ago
  I agree that your version is more satisfying and I can imagine that to be the future, although I find more plausible that the first generations will be distinct and geo-spacially localized. Once the mind hive's perception of self (if that is even necessary for intelligence) is blurred sufficiently that pretty much any system can be plugged in, than the whole possibility I mentioned crumbles. > Perhaps that is already happening. Do you mean intentionally by many researcher groups who don't share it, or as a shadow society peggy-baging on the internet without anyone's attention?
  - jacknews 5y ago
    I mean that if we consider humanity as an intelligence, it is clearly in the process of upgrading itself. Of course it's a bit of a stretch to consider all of humanity as a single intelligence, certainly from our individual vantage points. It doesn't seem conscious or even that smart. But from a different vantage point, especially in time, perhaps it does. I do have a more-strongly-integrated distributed intelligence in mind (with 'components' that are less autonomous), but perhaps there are other structures that already are intelligent on different scales, that we don't yet recognize. Perhaps it is not us upgrading, but the universe itself.

jacknews 5y ago

> Another related issue is that a new architecture which breaks the scalability limits of previous generation will produce new and distinct entities. If intelligence and self-interest often correlate, a machine might be vary of creating a better version of itself lest it be replaced. That assumes AI will manifest as multiple individual entities. I day-dreamed about this many years ago - what are the limits to intelligence, and what possible forms can it take, are 'societies' like minds, etc? It seems to me possible that single, distributed mind might be the ultimate form of intelligence, in which case the AI would not be replacing itself, but upgrading itself. Perhaps that is already happening.

walleeee 5y ago
your interpretation seems reasonable somehow we persist in the belief that misaligned superintelligence is a thought experiment but as far as I can tell distributed AGI is a reality and paperclip maximizers already exist
porkbrain 5y ago
I agree that your version is more satisfying and I can imagine that to be the future, although I find more plausible that the first generations will be distinct and geo-spacially localized. Once the mind hive's perception of self (if that is even necessary for intelligence) is blurred sufficiently that pretty much any system can be plugged in, than the whole possibility I mentioned crumbles. > Perhaps that is already happening. Do you mean intentionally by many researcher groups who don't share it, or as a shadow society peggy-baging on the internet without anyone's attention?
- jacknews 5y ago
  I mean that if we consider humanity as an intelligence, it is clearly in the process of upgrading itself. Of course it's a bit of a stretch to consider all of humanity as a single intelligence, certainly from our individual vantage points. It doesn't seem conscious or even that smart. But from a different vantage point, especially in time, perhaps it does. I do have a more-strongly-integrated distributed intelligence in mind (with 'components' that are less autonomous), but perhaps there are other structures that already are intelligent on different scales, that we don't yet recognize. Perhaps it is not us upgrading, but the universe itself.