4 ms·

I agree fully on the tendency for toy philosophical problems to devolve into bikeshedding and useless arguments over irrelevant particulars. I also agree that i

I agree fully on the tendency for toy philosophical problems to devolve into bikeshedding and useless arguments over irrelevant particulars. I also agree that ignorance of unknown harms and second-order effects deeply harms the potential good that engineered systems can do. I worry, however, that selectively-applied caution leads to equally unfavorable results in the opposite direction; to whit, it encourages sticking to the status quo. It is often complex to the point of impossibility to predict the wider effects of technological or social changes. The requirement that we understand those consequences would, if applied consistently, basically halt any technological progress. As distasteful as it is to admit it, we learn much more quickly by failure than success, and often the quickest, cheapest, and most thorough way to understand the consequences of a technology is to implement it. In other words, while I agree that applying implementing new technology without sufficient caution is often disastrous, I make note of that camel's nose "sufficient." The cost of excessive caution is that certain ideas (cars, fossil fuel energy, destructive economic models, etc) are allowed to continue far past the point where we can implement better alternatives. The cost of trying to do that is some measure of failure; there will always be some cost, and some of that measured in human lives, with trying to improve the state of the world.

walleeee 6y ago
> As distasteful as it is to admit it, we learn much more quickly by failure than success, and often the quickest, cheapest, and most thorough way to understand the consequences of a technology is to implement it. I totally agree. This is a fundamental paradox. As you state it's impossible to fully understand the consequences of any technological development. We can't learn without taking risks. > The cost of excessive caution is that certain ideas (cars, fossil fuel energy, destructive economic models, etc) are allowed to continue far past the point where we can implement better alternatives. I'm not sure about your use of "caution" here. I don't think fossil fuel extraction persists because we are too cautious (i.e., too risk-averse) to pursue alternatives. This would imply that biogeochemical realities are absent from our risk-benefit analysis, leading us to conclude burning carbon is less risky than other options. This may have been the case 50 years ago but "climate consciousness" is well within the mainstream now. I would argue instead that humans are uniquely skilled at ignoring unpleasant potentialities. This is socially useful: we coordinate our activities by genuflecting to fictions. This was a better strategy when we were incapable of terraforming the planet. Perhaps we can salvage it, but insulating ourselves from consequences as we seem so keen to do is a great way to walk off a cliff before the loss function kicks in. If you feed a system too-thoroughly sanitized data it will not form an accurate world model. > trying to improve the state of the world. I would submit that most existentially threatening technologies were born out of a desire to do just this. It seems the arc of development finds its way to entrenchment, if not the trenches, with little to no regard for inventors' intentions. As you say, perverse economic incentives don't help. I don't mean to condemn tool-making wholesale. But if we're going to build tools, imo they must be easy to dismantle and supersede. This principle is not so uncommon in software but the opposite tends to obtain more generally. An atmosphere full of carbon is not an easy thing to deconstruct, nor are stockpiles of nuclear weapons, or soils exhausted by years of improper care in pursuit of yield-maximization, or car-centric urban megalopoli.
- endominus 6y ago
  >I'm not sure about your use of "caution" here. I was alluding to caution in reference to fossil fuels specifically because nuclear power causes less deaths per megawatt than any fossil fuel (and, interestingly, hydroelectric, if only because of the catastrophic failure of the Three Gorges Dam), and the extra scrutiny it receives in comparison is thus unjustified in my opinion. Over 100 people are killed in car crashes every day in the US alone, which helps put the potential value of self-driving cars in better perspective (although, of course, the switchover to self-driving cars opens us up to the possibility of malicious actors suborning the entire fleet; a worthwhile hypothetical to consider, I think you'll agree, given the... lackadaisical information security standards both IoT and vehicle manufacturers adhere to). The example of economic models is more speculative, but I could well imagine any potential new system being derided for being unproven or too risky even as our current one encourages and rewards looting the planet and poisoning the communal well. To your point on ignoring unpleasant potentialities, I couldn't agree more. The optimists have always led the way, and as long as there is more commons to translate into immediate success, it shall be so. We are living in a time of whalefall, with a surplus of resources sufficient to support a maximally extractive lifestyle for perhaps one generation more, before it is gone forever. I have grown increasingly disillusioned with our potential to avert the coming global ecological and subsequent economic catastrophe; it seems that our Neros would rather fiddle while Rome burns. >But if we're going to build tools, imo they must be easy to dismantle and supersede. An interesting point, and one that I never considered before. My assumption is generally that for physical goods, robustness and long lifetimes are strongly favored over upgradability. Or rather, that "ease of transition away from" is basically never considered in the design or proliferation of new tools. That's a difficult point; is a meme that provides a path to a better meme more adaptive than one that locks the target in? Can it be made so?
  - walleeee 6y ago
    > I was alluding to caution in reference to fossil fuels specifically because nuclear power I follow you more clearly now- yes I agree. Maybe nuclear power gets undue scrutiny because its failures are rapid and local. I think I prefer this to an object so diffuse in space and time no one can point to it ("how can the earth be warming if it's snowing"). We are too likely to disbelieve or ignore anything that doesn't slap us in the face. A caveat: any waste disposal proposal which assumes continuity of governance or ecological predictability over any significant timescale is DOA imo. Even deep final repositories assume we can predict geologic activity half a million or more years into the future. I am not a geologist but this seems a gamble. Yes, and the situation re: self-driving cars seems similar. We entertain death behind the wheel daily but hand-wring over the particulars of hypothetical autonomous vehicle accidents. It seems there is a preference (at least in the US) for injury to come with a human face attached rather than at the banal whim of an algorithm, but fleetwide hijacking is a different story. I wonder if offline AVs are feasible. Here too the present may preclude a more desirable future: e.g., we may feel compelled to connect navigation computers to the internet to avoid roadway congestion, but in an alternate reality perhaps we did not encircle ourselves with asphalt and invite a chariot powered by the fumes of the long dead into every home. As they say, "you're not in traffic..." > We are living in a time of whalefall, with a surplus of resources sufficient to support a maximally extractive lifestyle for perhaps one generation more, before it is gone forever. I have grown increasingly disillusioned with our potential to avert the coming global ecological and subsequent economic catastrophe; it seems that our Neros would rather fiddle while Rome burns. I feel much the same way. An economist turned tree farmer I once spoke to said: "We have failed to develop cultural tools of sufficient sophistication to manage the tsunami that is our technological inventiveness." > My assumption is generally that for physical goods, robustness and long lifetimes are strongly favored over upgradability. Or rather, that "ease of transition away from" is basically never considered in the design or proliferation of new tools. Good point- I didn't consider this in depth. I agree re: physical tools. The trend toward disposability or frequent upgrades is the wrong direction. Industrial processes and "cultural tools" (i.e., social technologies or memes as you say), on the other hand, have more destructive potential than a manual implement and longer lifespans. > is a meme that provides a path to a better meme more adaptive than one that locks the target in? Can it be made so? This is a fascinating question and I don't know the answer. I'm struck by the fact that one can substitute "gene" for "meme" and with a fair bit of confidence say yes. If life had "locked the target in" prior to the oxygenation of the atmosphere presumably we would not exist today. An open question in plant biology, incidentally, is whether crop trait plasticity is itself a meta-trait we can pursue[0], instead of e.g. breeding or engineering germplasm for specific regions or conditions (although for some forms of plasticity the developmental basis is not well understood). I think of it a bit like the difference between a painstakingly programmed expert system and a neural net. We will likely need analogous reserves of sociocultural plasticity to survive on a planet growing more temperamental by the year, although the engines of financial and industrial metastasis do not appear so amenable to this. [https://www.frontiersin.org/articles/10.3389/fpls.2020.00546/full https://www.frontiersin.org/articles/10.3389/fpls.2020.00546...]

walleeee 6y ago

> As distasteful as it is to admit it, we learn much more quickly by failure than success, and often the quickest, cheapest, and most thorough way to understand the consequences of a technology is to implement it. I totally agree. This is a fundamental paradox. As you state it's impossible to fully understand the consequences of any technological development. We can't learn without taking risks. > The cost of excessive caution is that certain ideas (cars, fossil fuel energy, destructive economic models, etc) are allowed to continue far past the point where we can implement better alternatives. I'm not sure about your use of "caution" here. I don't think fossil fuel extraction persists because we are too cautious (i.e., too risk-averse) to pursue alternatives. This would imply that biogeochemical realities are absent from our risk-benefit analysis, leading us to conclude burning carbon is less risky than other options. This may have been the case 50 years ago but "climate consciousness" is well within the mainstream now. I would argue instead that humans are uniquely skilled at ignoring unpleasant potentialities. This is socially useful: we coordinate our activities by genuflecting to fictions. This was a better strategy when we were incapable of terraforming the planet. Perhaps we can salvage it, but insulating ourselves from consequences as we seem so keen to do is a great way to walk off a cliff before the loss function kicks in. If you feed a system too-thoroughly sanitized data it will not form an accurate world model. > trying to improve the state of the world. I would submit that most existentially threatening technologies were born out of a desire to do just this. It seems the arc of development finds its way to entrenchment, if not the trenches, with little to no regard for inventors' intentions. As you say, perverse economic incentives don't help. I don't mean to condemn tool-making wholesale. But if we're going to build tools, imo they must be easy to dismantle and supersede. This principle is not so uncommon in software but the opposite tends to obtain more generally. An atmosphere full of carbon is not an easy thing to deconstruct, nor are stockpiles of nuclear weapons, or soils exhausted by years of improper care in pursuit of yield-maximization, or car-centric urban megalopoli.

endominus 6y ago
>I'm not sure about your use of "caution" here. I was alluding to caution in reference to fossil fuels specifically because nuclear power causes less deaths per megawatt than any fossil fuel (and, interestingly, hydroelectric, if only because of the catastrophic failure of the Three Gorges Dam), and the extra scrutiny it receives in comparison is thus unjustified in my opinion. Over 100 people are killed in car crashes every day in the US alone, which helps put the potential value of self-driving cars in better perspective (although, of course, the switchover to self-driving cars opens us up to the possibility of malicious actors suborning the entire fleet; a worthwhile hypothetical to consider, I think you'll agree, given the... lackadaisical information security standards both IoT and vehicle manufacturers adhere to). The example of economic models is more speculative, but I could well imagine any potential new system being derided for being unproven or too risky even as our current one encourages and rewards looting the planet and poisoning the communal well. To your point on ignoring unpleasant potentialities, I couldn't agree more. The optimists have always led the way, and as long as there is more commons to translate into immediate success, it shall be so. We are living in a time of whalefall, with a surplus of resources sufficient to support a maximally extractive lifestyle for perhaps one generation more, before it is gone forever. I have grown increasingly disillusioned with our potential to avert the coming global ecological and subsequent economic catastrophe; it seems that our Neros would rather fiddle while Rome burns. >But if we're going to build tools, imo they must be easy to dismantle and supersede. An interesting point, and one that I never considered before. My assumption is generally that for physical goods, robustness and long lifetimes are strongly favored over upgradability. Or rather, that "ease of transition away from" is basically never considered in the design or proliferation of new tools. That's a difficult point; is a meme that provides a path to a better meme more adaptive than one that locks the target in? Can it be made so?
- walleeee 6y ago
  > I was alluding to caution in reference to fossil fuels specifically because nuclear power I follow you more clearly now- yes I agree. Maybe nuclear power gets undue scrutiny because its failures are rapid and local. I think I prefer this to an object so diffuse in space and time no one can point to it ("how can the earth be warming if it's snowing"). We are too likely to disbelieve or ignore anything that doesn't slap us in the face. A caveat: any waste disposal proposal which assumes continuity of governance or ecological predictability over any significant timescale is DOA imo. Even deep final repositories assume we can predict geologic activity half a million or more years into the future. I am not a geologist but this seems a gamble. Yes, and the situation re: self-driving cars seems similar. We entertain death behind the wheel daily but hand-wring over the particulars of hypothetical autonomous vehicle accidents. It seems there is a preference (at least in the US) for injury to come with a human face attached rather than at the banal whim of an algorithm, but fleetwide hijacking is a different story. I wonder if offline AVs are feasible. Here too the present may preclude a more desirable future: e.g., we may feel compelled to connect navigation computers to the internet to avoid roadway congestion, but in an alternate reality perhaps we did not encircle ourselves with asphalt and invite a chariot powered by the fumes of the long dead into every home. As they say, "you're not in traffic..." > We are living in a time of whalefall, with a surplus of resources sufficient to support a maximally extractive lifestyle for perhaps one generation more, before it is gone forever. I have grown increasingly disillusioned with our potential to avert the coming global ecological and subsequent economic catastrophe; it seems that our Neros would rather fiddle while Rome burns. I feel much the same way. An economist turned tree farmer I once spoke to said: "We have failed to develop cultural tools of sufficient sophistication to manage the tsunami that is our technological inventiveness." > My assumption is generally that for physical goods, robustness and long lifetimes are strongly favored over upgradability. Or rather, that "ease of transition away from" is basically never considered in the design or proliferation of new tools. Good point- I didn't consider this in depth. I agree re: physical tools. The trend toward disposability or frequent upgrades is the wrong direction. Industrial processes and "cultural tools" (i.e., social technologies or memes as you say), on the other hand, have more destructive potential than a manual implement and longer lifespans. > is a meme that provides a path to a better meme more adaptive than one that locks the target in? Can it be made so? This is a fascinating question and I don't know the answer. I'm struck by the fact that one can substitute "gene" for "meme" and with a fair bit of confidence say yes. If life had "locked the target in" prior to the oxygenation of the atmosphere presumably we would not exist today. An open question in plant biology, incidentally, is whether crop trait plasticity is itself a meta-trait we can pursue[0], instead of e.g. breeding or engineering germplasm for specific regions or conditions (although for some forms of plasticity the developmental basis is not well understood). I think of it a bit like the difference between a painstakingly programmed expert system and a neural net. We will likely need analogous reserves of sociocultural plasticity to survive on a planet growing more temperamental by the year, although the engines of financial and industrial metastasis do not appear so amenable to this. [https://www.frontiersin.org/articles/10.3389/fpls.2020.00546/full https://www.frontiersin.org/articles/10.3389/fpls.2020.00546...]