4 ms·

It seems that the medium of writing will inevitably reach a point of too much information to process, malicious or not; while simultaneously increasing the risk

It seems that the medium of writing will inevitably reach a point of too much information to process, malicious or not; while simultaneously increasing the risk of silos. What does a post-writing civilization look like? Is it reasonable to imagine scientific progress without the burden of having to battle the computational capability of sufficiently motivated actors who can easily manipulate authored information spaces? Writing is essentially a massive attack vector to the collective human consciousness prone to Sybil attacks at an alarming saturation point in a post gpt3 society. It feels like we need something post-writing.

closedloop129 4y ago
You still have the social graph of the authors. Professors are the gatekeepers of the next generation. Like venture capital, you will have to hustle your way to the top for an introduction if you are an outsider.
- pyinstallwoes 4y ago
  The social graph of the authors is precisely what computational superiority is able to manipulate due to the computational resources to create false reputation graphs that are indistinguishable from someone’s bias for base truth.
  - closedloop129 4y ago
    How is that possible? You start with a group of people whom you trust. Then you check papers where those people are co-authors. You add everybody to your group who is on an acceptable paper. On the other hand, you degrade or even remove those who are on a bad paper. Computational resources are useless because you cannot fake papers with trusted authors. Trusted authors won't publish a bad paper because they want to keep their reputation. All an adversary can have is something like a network of fake articles. Unlike web pages, scientific papers are not opinions. The scientific core group can check if papers are relevant and true. The scientists can limit those checks to authors whom they have met in person to minimize their time investment.
    - pyinstallwoes 4y ago
      How do you know if your network is the truthful one and isn't participating in deception? And given that networks are increasingly non-physical, how do you guarantee it is the person in question? The only way reputation networks maintain durability is if they stay physically constrained and limited by physical contact. Any system built externally and allowed to represent facts and truth will be compromised due to computational superiority. It must not be a network hosted by a computer. It must run on culture and physical reality itself. Additionally agents of deception are regularly embedded in networks to encourage malicious behavior. Spies are pretty common. See: https://m-cacm.acm.org/magazines/2021/6/252840-collusion-rings-threaten-the-integrity-of-computer-science-research/fulltext https://m-cacm.acm.org/magazines/2021/6/252840-collusion-rin... I think the key bias we're contesting is that I'm arguing computational superiority allows creating networks that are indistinguishable from truthful ones because they have the resources to have the "longest version of history" to act as the authority. If the history is not there, it can be fabricated and embed enough "covert actors" to maintain a narrative. Then you also generate useful idiots who don't know they are being used to propagate misdirection or what have you. They become unable to detect any "obviously fake content" because it doesn't require fake content to conduct deception. Very often, it is real information indeed that is most able to misdirect attention.
      - closedloop129 4y ago
        >How do you know if your network is the truthful one You don't know. Evolution will decide if your seed group was good enough. >how do you guarantee it is the person in question? You can't. You have to role back if an identity is compromised >It must run on culture and physical reality itself. Yes. That's the difference between science and writing in general. Science is meant to be checked with reality. > think the key bias ... the resources to have the "longest version of history" History is secondary if you verify all statements. History should only grant preferred treatment. >Then you also generate useful idiots That's outside of the trust network. You can help them by making education easily accessible. >Very often, it is real information indeed that is most able to misdirect attention. That's when the game becomes interesting. You can also have truths that are not verifiable etc. It all depends on what the participatns want to know.
alimw 4y ago
Mathematical claims may be formalised in a proof assistant.
- pyinstallwoes 4y ago
  The burden of proof shifting to a sacred script only understood by a minority of the population seems like giving the 0.0001% all the power.

closedloop129 4y ago

You still have the social graph of the authors. Professors are the gatekeepers of the next generation. Like venture capital, you will have to hustle your way to the top for an introduction if you are an outsider.

pyinstallwoes 4y ago
The social graph of the authors is precisely what computational superiority is able to manipulate due to the computational resources to create false reputation graphs that are indistinguishable from someone’s bias for base truth.
- closedloop129 4y ago
  How is that possible? You start with a group of people whom you trust. Then you check papers where those people are co-authors. You add everybody to your group who is on an acceptable paper. On the other hand, you degrade or even remove those who are on a bad paper. Computational resources are useless because you cannot fake papers with trusted authors. Trusted authors won't publish a bad paper because they want to keep their reputation. All an adversary can have is something like a network of fake articles. Unlike web pages, scientific papers are not opinions. The scientific core group can check if papers are relevant and true. The scientists can limit those checks to authors whom they have met in person to minimize their time investment.
  - pyinstallwoes 4y ago
    How do you know if your network is the truthful one and isn't participating in deception? And given that networks are increasingly non-physical, how do you guarantee it is the person in question? The only way reputation networks maintain durability is if they stay physically constrained and limited by physical contact. Any system built externally and allowed to represent facts and truth will be compromised due to computational superiority. It must not be a network hosted by a computer. It must run on culture and physical reality itself. Additionally agents of deception are regularly embedded in networks to encourage malicious behavior. Spies are pretty common. See: https://m-cacm.acm.org/magazines/2021/6/252840-collusion-rings-threaten-the-integrity-of-computer-science-research/fulltext https://m-cacm.acm.org/magazines/2021/6/252840-collusion-rin... I think the key bias we're contesting is that I'm arguing computational superiority allows creating networks that are indistinguishable from truthful ones because they have the resources to have the "longest version of history" to act as the authority. If the history is not there, it can be fabricated and embed enough "covert actors" to maintain a narrative. Then you also generate useful idiots who don't know they are being used to propagate misdirection or what have you. They become unable to detect any "obviously fake content" because it doesn't require fake content to conduct deception. Very often, it is real information indeed that is most able to misdirect attention.
    - closedloop129 4y ago
      >How do you know if your network is the truthful one You don't know. Evolution will decide if your seed group was good enough. >how do you guarantee it is the person in question? You can't. You have to role back if an identity is compromised >It must run on culture and physical reality itself. Yes. That's the difference between science and writing in general. Science is meant to be checked with reality. > think the key bias ... the resources to have the "longest version of history" History is secondary if you verify all statements. History should only grant preferred treatment. >Then you also generate useful idiots That's outside of the trust network. You can help them by making education easily accessible. >Very often, it is real information indeed that is most able to misdirect attention. That's when the game becomes interesting. You can also have truths that are not verifiable etc. It all depends on what the participatns want to know.

alimw 4y ago

Mathematical claims may be formalised in a proof assistant.

pyinstallwoes 4y ago
The burden of proof shifting to a sacred script only understood by a minority of the population seems like giving the 0.0001% all the power.