3 ms·

Despite my last name, I'm skeptical about predictions that growth will foreseeably slow down or cease entirely. I believe it was Jim Manzi who coined the concep

Despite my last name, I'm skeptical about predictions that growth will foreseeably slow down or cease entirely. I believe it was Jim Manzi who coined the concept of the "Icarus Fallacy," in which the experts of each decade invariably predict an energy crisis 30 years hence — just after they will die. Does this demonstrate that current, analogous predictions are false? Certainly lot. But Bayes theorem dictates that the similarity decreases the confidence that one should have in contemporary climate science. I have neither the time nor the inclination to remain current on cutting-edge climate science. Whenever I check in, I find that the critics seem to be getting the better of the argument. But I profess no expertise. I do, however, consider myself reasonably qualified to opine about the state of AGI research. And, despite more than a decade of trying, I have no idea when to forecast the emergence of AGI. I would not be shocked if it happened tomorrow, nor would I be surprised if it occurred 300 years from now — or not at all. I am, however, confident that the emergence of AGI will radically alter the state of the earth, thereby rendering virtually all pre-AGI predictions worthless. The population trends of Africa mean little if some Western nation unleashes a digital demi-god capable of reshaping the earth in its image. The IPCC's failure to account for this effect, which I believe in quite strongly, does not inspire confidence in the reliability of its other data. I do not, of course, mean to imply that it would be reasonable to expect an international panel on climate change to address the risks from artificial intelligence. But that is because there are so many different contingencies that it is silly to make projections so far into the future. It is not at all clear that an energy crisis is even the greatest existential threat that modern man faces.

skimpycompiler 11y ago
> I do, however, consider myself reasonably qualified to opine about the state of AGI research. And, despite more than a decade of trying, I have no idea when to forecast the emergence of AGI. I would not be shocked if it happened tomorrow, nor would I be surprised if it occurred 300 years from now — or not at all. I'd be surprised if it comes tomorrow. The leading scientists have little belief that their current methods could be easily adjusted for AGI. All of the leads in AGI currently can't even simulate the problem solving skills of C. elegans. There's not a single AGI project that accomplishes the same skills of a specialized model.
- ClintEhrlich 11y ago
  >There's not a single [public] AGI project that accomplishes >the same skills of a specialized model.
icebraining 11y ago
If, as you readily admit, the emergence of AGI would radically alter the whole state of Earth in ways that can't be predicted, how do you expect the IPCC to account for it? Essentially, all they could say would be: "If AGI doesn't happen, we predict [current predictions]. If AGI happens, who the hell knows?" You can't account for something which can't be accounted for.
- ClintEhrlich 11y ago
  That's my point. The limits on our epistomological capacity are greater than advertised. Sadly, this is not a trivial insight. The obstacles to clairvoyance privilege immediacy over wisdom, because it radically limits the number of possible permutations. The solution to uncertainty is not an obsessive focus on one discrete risk, like anthropogenic climate change. Instead, it necessitates a focus on the factors that provide the most robust protection against a broad array of risks. Objectively, the single factor with the greatest relevance to the entire body of existential threats is economic growth or, at minimum, annual economic productive capacity, for it enables effective adaptive responses to myriad crises.
  - icebraining 11y ago
    The limits on our epistomological capacity are greater than advertised. I would say that's because the obvious is assumed, so there's no need to advertise it. Nobody expects the IPCC prevision to hold if a singular event happens, so what would be the point of underlying that fact? The solution to uncertainty is not an obsessive focus on one discrete risk, like anthropogenic climate change. It's the Intergovernmental Panel on Climate Change, what do you expect them to focus on? You wrote: "The IPCC's failure to account for this effect, which I believe in quite strongly, does not inspire confidence in the reliability of its other data." But there is no reason for them to account for something that can't be accounted, and it makes no sense to criticize them for focusing on what they've been created to focus on.
peterashford 11y ago
to me this reads as "i have little faith in 99% of the expert scientific community has to say.... but here's a batshit crazy theory which I think is a dead cert"
- ClintEhrlich 11y ago
  I prefer to rely on my own understanding of the relevant science, rather than glorified opinions polls. It troubles me when this leads me to conclusions that veer from the mainstream, and I certainly include that fact when I analyze the probability of their veracity. But, try as I may, I remain convinced of my current positions, based on the available evidence.
  - peterashford 11y ago
    As do I.

skimpycompiler 11y ago

> I do, however, consider myself reasonably qualified to opine about the state of AGI research. And, despite more than a decade of trying, I have no idea when to forecast the emergence of AGI. I would not be shocked if it happened tomorrow, nor would I be surprised if it occurred 300 years from now — or not at all. I'd be surprised if it comes tomorrow. The leading scientists have little belief that their current methods could be easily adjusted for AGI. All of the leads in AGI currently can't even simulate the problem solving skills of C. elegans. There's not a single AGI project that accomplishes the same skills of a specialized model.

ClintEhrlich 11y ago
>There's not a single [public] AGI project that accomplishes >the same skills of a specialized model.

icebraining 11y ago

If, as you readily admit, the emergence of AGI would radically alter the whole state of Earth in ways that can't be predicted, how do you expect the IPCC to account for it? Essentially, all they could say would be: "If AGI doesn't happen, we predict [current predictions]. If AGI happens, who the hell knows?" You can't account for something which can't be accounted for.

ClintEhrlich 11y ago
That's my point. The limits on our epistomological capacity are greater than advertised. Sadly, this is not a trivial insight. The obstacles to clairvoyance privilege immediacy over wisdom, because it radically limits the number of possible permutations. The solution to uncertainty is not an obsessive focus on one discrete risk, like anthropogenic climate change. Instead, it necessitates a focus on the factors that provide the most robust protection against a broad array of risks. Objectively, the single factor with the greatest relevance to the entire body of existential threats is economic growth or, at minimum, annual economic productive capacity, for it enables effective adaptive responses to myriad crises.
- icebraining 11y ago
  The limits on our epistomological capacity are greater than advertised. I would say that's because the obvious is assumed, so there's no need to advertise it. Nobody expects the IPCC prevision to hold if a singular event happens, so what would be the point of underlying that fact? The solution to uncertainty is not an obsessive focus on one discrete risk, like anthropogenic climate change. It's the Intergovernmental Panel on Climate Change, what do you expect them to focus on? You wrote: "The IPCC's failure to account for this effect, which I believe in quite strongly, does not inspire confidence in the reliability of its other data." But there is no reason for them to account for something that can't be accounted, and it makes no sense to criticize them for focusing on what they've been created to focus on.

peterashford 11y ago

to me this reads as "i have little faith in 99% of the expert scientific community has to say.... but here's a batshit crazy theory which I think is a dead cert"

ClintEhrlich 11y ago
I prefer to rely on my own understanding of the relevant science, rather than glorified opinions polls. It troubles me when this leads me to conclusions that veer from the mainstream, and I certainly include that fact when I analyze the probability of their veracity. But, try as I may, I remain convinced of my current positions, based on the available evidence.
- peterashford 11y ago
  As do I.