13 ms·

Have you ever read the Lena [https://qntm.org/mmacevedo https://qntm.org/mmacevedo] short story?

• echelon 5y ago
  That was a crystal clear and haunting presentation of many of the fears I have about BCI. Here are a few more: - Whoever wins BCI wins the evolutionary arms race. They can have an infinitely scalable workforce (slave force) to build anything conceivable and rapidly outperform the competition. (They also won't need biological humans anymore, so they can easily chemically eradicate us.) - How does Democracy work in BCI? A lot of the things we take for granted break down and have no clear analog. Who decides the rules of the system? Who shuts down simulations? Who decides what simulations get certain stimuli (or tasks)? Benevolent God or dictator seem to be more stable configurations, and the latter is probably more efficient. - If physics is solvable and losslessly reversible, a future entity could grab and emulate us at any point. We could be simulations of originals right now. Or in the future subjected to infinite torture and pain without any control. Pointing this out was how I learned of the similarly themed Roko's Basilisk. I'm curious whether we'll invent AGI or BCI first. There have been more developments from the deep learning camp, but BCI seems to be gaining more interest.