10 ms·

Against the Dark Forest

121 points by tobr 2y ago

mgaunard 2y ago
The dark forest hypothesis sort of existed before Liu Cixin.
- olddustytrail 2y ago
  [flagged]
- Vecr 2y ago
  In a more general form too, and one that I think might be more realistic (depending on the ultimate limits of technology): https://en.wikipedia.org/wiki/Berserker_hypothesis https://en.wikipedia.org/wiki/Berserker_hypothesis
  - TeMPOraL 2y ago
    It's not more general, it's more specific. Berserker hypothesis is basically grey goo that goes interstellar. The Dark Forest is about how lack of trust, unpredictable technological jumps, and speed of light being dog slow, force civilizations to preemptively destroy any other civilization they spot, while themselves trying to not get discovered. Berserker event is effectively an implementation detail of one of possible events in the Dark Forest.
    - Vecr 2y ago
      I disagree. I think the conditions to get a game theory style setup going are quite specific. Always "defecting" against everything (with no warning) and clearing outwards as fast as you can is probably converged upon along many paths, from many starting points, as long as the physics allows it. Like Wikipedia says, Dark Forest is for people who are limited in tech and resources. Or have a weird sense of ethics, but ask Robin Hanson how long he thinks that would last.
- sctb 2y ago
  I'm not sure how this article relates to the dark forest hypothesis, or really anything else for that matter—I can't see the thread. Can anyone summarize?
  - mgaunard 2y ago
    It doesn't relate to it very much. It's based on another article that claims that people mostly interacting in their own social bubble, fueled by algorithms which mostly isolate people further, is akin to making the Internet a dark forest.
BSOhealth 2y ago
AI, bots, and the constant need to fuel training, turns the internet forest into a place where anything a person publishes will be vampirically consumed, mimicked to see whether it can replicate whatever value could be had, and capitalized on if possible or ignored if not. It’s an interesting contrast to actual dark forest theory—-AI doesn’t want to destroy us, so we don’t need to hide for existential sake. But imagine walking outside and as soon as you do, innumerable copies of you spring up, and each action and sound you make is replicated and amplified. Like a weird Phantom Tollbooth meets Alice in Wonderland on DMT
- TeMPOraL 2y ago
  > AI, bots, and the constant need to fuel training, turns the internet forest into a place where anything a person publishes will be vampirically consumed, mimicked to see whether it can replicate whatever value could be had, and capitalized on if possible or ignored if not. That's honestly not a problem, unless someone believes they're entitled to 100% of value generated by their interactions with the world - but that's a level of greed way beyond Scrooge McDuck, and the kind of thinking that defines the dreaded "late stage capitalism". It really is an option to publish something and not care about knowing exactly who read it, and what they're doing with it.
  - throwway120385 2y ago
    Why do you think that business value is the only thing to look at here? If someone starts copying my likeness, mannerisms, writing style, etc. they could also use that to damage my reputation or harm my relations with other people. I think that those two possibilities represent irreparable harms with no associated business value.
    - TeMPOraL 2y ago
      > If someone starts copying my likeness, mannerisms, writing style, etc. they could also use that to damage my reputation or harm my relations with other people. I think that those two possibilities represent irreparable harms with no associated business value. There's 8 billion people on the planet. Unless you're a celebrity, this is not a real problem for you at this point (and if you are, it's a business problem). There's no way for a large model learning on the entirety of the Internet to somehow convert "copying likeness, mannerisms, writing style, etc." into damaging your reputation; doing that is something hyper-targeted, and at this point (and in conceivable future), there's no middle ground between "plausible deniability" and "someone targeting you specifically, which they could do just as well in pre-AI times". You're also not as unique as you think. There are many people with same mannerism, many people with same writing style, etc. Nor are those things constant over time. The flip side of not being a unique snowflake is also that anyone's contributions to the public Internet are, for purposes of AI training, worth approximately $0 on the margin, and impact the model just as much. It's in the volume of data that the patterns emerge that LLMs learn, volume too big for any person to be entitled to a meaningful part of it.
kelseyfrog 2y ago
Real talk. The disdain engineers have toward the "soft sciences"[sociology, psychology] enables them to manipulated into building systems which reproduce and reinforce social structures and social relations in ways that they would otherwise object to. To the degree that we've collectively built a dark forest, it's been in part due to this lack of multi-disciplinarianism. Humans are amazingly adept at rationalization.
- Nasrudith 2y ago
  They always keep on asserting that 'multi-disciplinarianism' (always in the form of the less practical being pushed on the more practical, never the other way around despite the obvious role in understanding reality in well, understanding reality). They never worry about being detached from reality from their distance to the objective. It reeks of the bad old days of classical philosophy where they considered estrangement from reality an intrinsic virtue as "purity of thought" and looked down upon those who dealt with banal reality. I have yet to see any actual backing for such claims. Only standard academia's toxic social knife-fighting and Kafka-traps. The specific form of bullshit being mounting social pressure as a weapon. Where where not agreeing with them and their theories means clearly you are more wrong and a worse person for not agreeing with them and thus you must agree with them! It always rung of a scam to justify mandatory electives to justify the existence of their professorships. There is no intellectual honesty to be found here, only snobbery and blind adherence to cliches.
- asacrowflies 2y ago
  It would help if those "soft sciences" had any sort of reproducible outcomes. As far as I can tell every instance of these "soft sciences" are just an arm of the geopolitical body/state that backs them...with the sole purpose of rationalizing what ever that states status quo is.
  - kiba 2y ago
    Ignorance is really not a virtue. If you suspect that you're being lied to by the consensus, feel free to read alternatives by heterodox scholars. I suspected that they're crank. It's not like I particularly agree with the way things work now, only that we shouldn't make things worse by making obviously stupid moves in the wrong direction.
cjs_ac 2y ago
I was a young child in the 90s, and I barely used the Internet then, but I remember how people talked about it. It was Cyberspace and the Information Superhighway. It looked like blue and purple grids suspended in black space. It wasn't real in the same way a pen was real. It was an alternate reality. At some point between, say, 2000 and 2012, everyone forgot that. The Web became an extension of the real world. Now, in 2024, the Web seems to have displaced the real world as the locus of public discourse. > It names an experience of paranoia and anxiety that by the end of the 2010s was widespread among people with meaningful connections between their online personas and their ability to maintain their standard of living. This kind of thing always struck me as a poor choice. The great thing about maintaining one's anonymity on the Web, or, to be more precise, adopting a collection of alternate identities, is that they can be discarded easily. A faux pas needn't follow you around, and can't follow you to where you make your living. > The last and most most dangerous weakness of the Dark Internet Forest as a frame is that it positions the broad landscape of connection as something that “we” can simply do without—and without which we will indeed feel better and be more productive. > On the level of the individual, this is true for certain values of “we”: for people who are, in any sense, established; people who already have the social status they required to succeed in their field; people whose work doesn’t depend on them needing to find (and re-find and re-find) readers or customers; people whose professional and personal networks are already strong enough to catch them if they slip; people with money. > So what about everyone else? Should people without those forms of access and capital simply forgo all the benefits afforded by access to broad networks? Through the twentieth century, as aristocracy gave way to meritocracy, people learnt how to climb the greasy pole without access to these broad networks. Social mobility is largely a consequence of the structure of the economy, and not necessarily a consequence of networking. We are often told that, 'It's not what you know, it's who you know,' but maybe the people who found other ways to the top simply aren't as effective at sharing their secrets as the shmoozers. > This all stops being an individual problem and becomes a collective one when bad products of the social internet get worse, as when platform turmoil and manipulation helps remodel the offline world in the image of the most grotesque parts of the online one. Conflict is coming. We've reached a point, I think, we're it's inevitable. I don't think it's clear exactly what the ideas underlying the conflict will be, or what factions will contest it, or in what spaces it will be contested, but there is a range to choose from. The winning move in this situation is to keep informed about events, but otherwise to keep one's head down. Don't challenge ideas you find objectionable, but agree in a mostly-disinterested way. Take it from someone with family members who still have the machete scars.
julianeon 2y ago
This article attempts to bite off a lot! I'm going to condense it down to one small part. The Dark Forest experience that's causing "mass migrations" on the Internet is that, if you venture out into social media carelessly, you're entering an aggressive and frequently unpleasant space which is close to the opposite of "fun" for the average person. Here follows a laundry list here of things which you might expect to encounter in an unpleasant online space, most of which have a kind of tabloid quality: scams, nonstop sales pitches, porn, extreme violence, celebrity worship, ideological extremism, racism, sexism, etc. And more (that list is not exhaustive). So people find that signing up for the average generic platform that welcomes everybody is a bad experience, and migrate to more niche ones. That's been the path I've followed too, all the way here.
binary132 2y ago
Maybe I’m imagining it, but I feel like there’s a shift in gears somewhere between “we need to fix networks” and “the solution is decentralized networks”. Decentralized networks are unfixable and unmoderatable — that’s why people want them. People don’t want to be corralled or have their information consumption managed (not speaking to what’s good here). The author spends so much effort condemning Facebook for not managing the network in Myanmar enough, and then turns around and suggests — unmoderatable decentralized networks without authorities in a position to manage or moderate them? I think I must be confused.
- Dylan16807 2y ago
  The article says "The assumption that "Twitter but decentralized" or "Facebook but open-source and federated" will necessarily be good—rather than differently bad—is a weak one." So yes I think you're confused.
  - binary132 2y ago
    It also says: > I don't think any centrally governed platform at global scale is capable of doing the work, even if they hired the best and sharpest people I know. Even if they put real effort into humanist upstream product design, rather than tossing loose change to trust and safety teams sent in to clean up after the fact. > > Local norms matter too much for global governance of the social internet to make sense; the flattening of global diversity to fit the norms and interests of any given American techno-culture—corporate or otherwise—is both a baldly colonial aspiration and one we should scorn for the same reason that we leave the idea of effective, monolithic, planetary-scale government—benevolent or otherwise—to underbaked science fiction. Home rule and genuine resilience both require the existence of many places, many of them at least partially interconnected. That’s what confused me, given your cited context. It doesn’t sound centralized. Things that are not centralized are decentralized, no?
    - Dylan16807 2y ago
      Decentralization is not itself the solution. Breaking up the control of servers, but still keeping everything as a global network but now with less moderation, is not the suggestion. The article is arguing for spaces that are more loosely connected but still moderated, probably more moderated.

121 points by tobr 2y ago

mgaunard 2y ago

The dark forest hypothesis sort of existed before Liu Cixin.

olddustytrail 2y ago
[flagged]
Vecr 2y ago
In a more general form too, and one that I think might be more realistic (depending on the ultimate limits of technology): https://en.wikipedia.org/wiki/Berserker_hypothesis https://en.wikipedia.org/wiki/Berserker_hypothesis
- TeMPOraL 2y ago
  It's not more general, it's more specific. Berserker hypothesis is basically grey goo that goes interstellar. The Dark Forest is about how lack of trust, unpredictable technological jumps, and speed of light being dog slow, force civilizations to preemptively destroy any other civilization they spot, while themselves trying to not get discovered. Berserker event is effectively an implementation detail of one of possible events in the Dark Forest.
  - Vecr 2y ago
    I disagree. I think the conditions to get a game theory style setup going are quite specific. Always "defecting" against everything (with no warning) and clearing outwards as fast as you can is probably converged upon along many paths, from many starting points, as long as the physics allows it. Like Wikipedia says, Dark Forest is for people who are limited in tech and resources. Or have a weird sense of ethics, but ask Robin Hanson how long he thinks that would last.
sctb 2y ago
I'm not sure how this article relates to the dark forest hypothesis, or really anything else for that matter—I can't see the thread. Can anyone summarize?
- mgaunard 2y ago
  It doesn't relate to it very much. It's based on another article that claims that people mostly interacting in their own social bubble, fueled by algorithms which mostly isolate people further, is akin to making the Internet a dark forest.

BSOhealth 2y ago

AI, bots, and the constant need to fuel training, turns the internet forest into a place where anything a person publishes will be vampirically consumed, mimicked to see whether it can replicate whatever value could be had, and capitalized on if possible or ignored if not. It’s an interesting contrast to actual dark forest theory—-AI doesn’t want to destroy us, so we don’t need to hide for existential sake. But imagine walking outside and as soon as you do, innumerable copies of you spring up, and each action and sound you make is replicated and amplified. Like a weird Phantom Tollbooth meets Alice in Wonderland on DMT

TeMPOraL 2y ago
> AI, bots, and the constant need to fuel training, turns the internet forest into a place where anything a person publishes will be vampirically consumed, mimicked to see whether it can replicate whatever value could be had, and capitalized on if possible or ignored if not. That's honestly not a problem, unless someone believes they're entitled to 100% of value generated by their interactions with the world - but that's a level of greed way beyond Scrooge McDuck, and the kind of thinking that defines the dreaded "late stage capitalism". It really is an option to publish something and not care about knowing exactly who read it, and what they're doing with it.
- throwway120385 2y ago
  Why do you think that business value is the only thing to look at here? If someone starts copying my likeness, mannerisms, writing style, etc. they could also use that to damage my reputation or harm my relations with other people. I think that those two possibilities represent irreparable harms with no associated business value.
  - TeMPOraL 2y ago
    > If someone starts copying my likeness, mannerisms, writing style, etc. they could also use that to damage my reputation or harm my relations with other people. I think that those two possibilities represent irreparable harms with no associated business value. There's 8 billion people on the planet. Unless you're a celebrity, this is not a real problem for you at this point (and if you are, it's a business problem). There's no way for a large model learning on the entirety of the Internet to somehow convert "copying likeness, mannerisms, writing style, etc." into damaging your reputation; doing that is something hyper-targeted, and at this point (and in conceivable future), there's no middle ground between "plausible deniability" and "someone targeting you specifically, which they could do just as well in pre-AI times". You're also not as unique as you think. There are many people with same mannerism, many people with same writing style, etc. Nor are those things constant over time. The flip side of not being a unique snowflake is also that anyone's contributions to the public Internet are, for purposes of AI training, worth approximately $0 on the margin, and impact the model just as much. It's in the volume of data that the patterns emerge that LLMs learn, volume too big for any person to be entitled to a meaningful part of it.

kelseyfrog 2y ago

Real talk. The disdain engineers have toward the "soft sciences"[sociology, psychology] enables them to manipulated into building systems which reproduce and reinforce social structures and social relations in ways that they would otherwise object to. To the degree that we've collectively built a dark forest, it's been in part due to this lack of multi-disciplinarianism. Humans are amazingly adept at rationalization.

Nasrudith 2y ago
They always keep on asserting that 'multi-disciplinarianism' (always in the form of the less practical being pushed on the more practical, never the other way around despite the obvious role in understanding reality in well, understanding reality). They never worry about being detached from reality from their distance to the objective. It reeks of the bad old days of classical philosophy where they considered estrangement from reality an intrinsic virtue as "purity of thought" and looked down upon those who dealt with banal reality. I have yet to see any actual backing for such claims. Only standard academia's toxic social knife-fighting and Kafka-traps. The specific form of bullshit being mounting social pressure as a weapon. Where where not agreeing with them and their theories means clearly you are more wrong and a worse person for not agreeing with them and thus you must agree with them! It always rung of a scam to justify mandatory electives to justify the existence of their professorships. There is no intellectual honesty to be found here, only snobbery and blind adherence to cliches.
asacrowflies 2y ago
It would help if those "soft sciences" had any sort of reproducible outcomes. As far as I can tell every instance of these "soft sciences" are just an arm of the geopolitical body/state that backs them...with the sole purpose of rationalizing what ever that states status quo is.
- kiba 2y ago
  Ignorance is really not a virtue. If you suspect that you're being lied to by the consensus, feel free to read alternatives by heterodox scholars. I suspected that they're crank. It's not like I particularly agree with the way things work now, only that we shouldn't make things worse by making obviously stupid moves in the wrong direction.

cjs_ac 2y ago

I was a young child in the 90s, and I barely used the Internet then, but I remember how people talked about it. It was Cyberspace and the Information Superhighway. It looked like blue and purple grids suspended in black space. It wasn't real in the same way a pen was real. It was an alternate reality. At some point between, say, 2000 and 2012, everyone forgot that. The Web became an extension of the real world. Now, in 2024, the Web seems to have displaced the real world as the locus of public discourse. > It names an experience of paranoia and anxiety that by the end of the 2010s was widespread among people with meaningful connections between their online personas and their ability to maintain their standard of living. This kind of thing always struck me as a poor choice. The great thing about maintaining one's anonymity on the Web, or, to be more precise, adopting a collection of alternate identities, is that they can be discarded easily. A faux pas needn't follow you around, and can't follow you to where you make your living. > The last and most most dangerous weakness of the Dark Internet Forest as a frame is that it positions the broad landscape of connection as something that “we” can simply do without—and without which we will indeed feel better and be more productive. > On the level of the individual, this is true for certain values of “we”: for people who are, in any sense, established; people who already have the social status they required to succeed in their field; people whose work doesn’t depend on them needing to find (and re-find and re-find) readers or customers; people whose professional and personal networks are already strong enough to catch them if they slip; people with money. > So what about everyone else? Should people without those forms of access and capital simply forgo all the benefits afforded by access to broad networks? Through the twentieth century, as aristocracy gave way to meritocracy, people learnt how to climb the greasy pole without access to these broad networks. Social mobility is largely a consequence of the structure of the economy, and not necessarily a consequence of networking. We are often told that, 'It's not what you know, it's who you know,' but maybe the people who found other ways to the top simply aren't as effective at sharing their secrets as the shmoozers. > This all stops being an individual problem and becomes a collective one when bad products of the social internet get worse, as when platform turmoil and manipulation helps remodel the offline world in the image of the most grotesque parts of the online one. Conflict is coming. We've reached a point, I think, we're it's inevitable. I don't think it's clear exactly what the ideas underlying the conflict will be, or what factions will contest it, or in what spaces it will be contested, but there is a range to choose from. The winning move in this situation is to keep informed about events, but otherwise to keep one's head down. Don't challenge ideas you find objectionable, but agree in a mostly-disinterested way. Take it from someone with family members who still have the machete scars.

julianeon 2y ago

This article attempts to bite off a lot! I'm going to condense it down to one small part. The Dark Forest experience that's causing "mass migrations" on the Internet is that, if you venture out into social media carelessly, you're entering an aggressive and frequently unpleasant space which is close to the opposite of "fun" for the average person. Here follows a laundry list here of things which you might expect to encounter in an unpleasant online space, most of which have a kind of tabloid quality: scams, nonstop sales pitches, porn, extreme violence, celebrity worship, ideological extremism, racism, sexism, etc. And more (that list is not exhaustive). So people find that signing up for the average generic platform that welcomes everybody is a bad experience, and migrate to more niche ones. That's been the path I've followed too, all the way here.

binary132 2y ago

Maybe I’m imagining it, but I feel like there’s a shift in gears somewhere between “we need to fix networks” and “the solution is decentralized networks”. Decentralized networks are unfixable and unmoderatable — that’s why people want them. People don’t want to be corralled or have their information consumption managed (not speaking to what’s good here). The author spends so much effort condemning Facebook for not managing the network in Myanmar enough, and then turns around and suggests — unmoderatable decentralized networks without authorities in a position to manage or moderate them? I think I must be confused.

Dylan16807 2y ago
The article says "The assumption that "Twitter but decentralized" or "Facebook but open-source and federated" will necessarily be good—rather than differently bad—is a weak one." So yes I think you're confused.
- binary132 2y ago
  It also says: > I don't think any centrally governed platform at global scale is capable of doing the work, even if they hired the best and sharpest people I know. Even if they put real effort into humanist upstream product design, rather than tossing loose change to trust and safety teams sent in to clean up after the fact. > > Local norms matter too much for global governance of the social internet to make sense; the flattening of global diversity to fit the norms and interests of any given American techno-culture—corporate or otherwise—is both a baldly colonial aspiration and one we should scorn for the same reason that we leave the idea of effective, monolithic, planetary-scale government—benevolent or otherwise—to underbaked science fiction. Home rule and genuine resilience both require the existence of many places, many of them at least partially interconnected. That’s what confused me, given your cited context. It doesn’t sound centralized. Things that are not centralized are decentralized, no?
  - Dylan16807 2y ago
    Decentralization is not itself the solution. Breaking up the control of servers, but still keeping everything as a global network but now with less moderation, is not the suggestion. The article is arguing for spaces that are more loosely connected but still moderated, probably more moderated.