3 ms·

> We just need to make sure they don't get so black box that we trust the outputs blindly Black box is probably the only way we'll ever get AGI. Our human bra

> We just need to make sure they don't get so black box that we trust the outputs blindly Black box is probably the only way we'll ever get AGI. Our human brains are so black box that we can only accept their outputs blindly. We try to poke and prod the human black box as best we can, with magnets, sound waves, and light waves. But it still generates useful output all the time, so we assume it's pretty good and generally let it be.

BirAdam 5y ago
In fairness to all of us humans, no. We do not blindly trust our brains. This explicitly how science came about, and this is also how entire branches of philosophy came about.
- newsbinator 5y ago
  We do blindly trust our brains all the time. Whenever a brain or a group of brains tells us we need chemotherapy or to start a war, with barely the slightest understanding of how the brain is wired or based on which inferences it would have come to that conclusion, we believe it. Obviously we believe only those brains that were primed with enough training data and have a decent track record of giving useful answers and making accurate-enough predictions. But we still rely on this "blindly" in the absence of their source code or even any understanding about how or why they think, or why some collections of cells reason and demonstrate intelligence, while most don't. We currently hold machine brains to a much higher standard of transparency and comprehensibility than we do human brains. Human brains can't even tell us much about themselves or show any logging, because they barely know anything about how they work or where ideas or connections come from either.
  - BirAdam 5y ago
    In general, no. If we blindly trusted our brains, we would not require things like “another set of eyes,” or the rigors of reproducibility, or business processes/protocols, or a million other things. Further more, this lack of trust in the matter between our ears is observable in the course of daily human life when such questions are asked as “why did I say that?” or “why did I do that?”. Blind trust would obviate the need for such questions. As for the supposed trust in authority that you describe, that is not blind trust in the brain. Trust in authorities is actually quite low at the moment, and was never really blind. Every election in the USA, half of the country doesn’t trust the people elected, and possibly even the election itself. More over, War is no longer declared, and people never blindly trusted the government on war anyway, hence all of the protests of all of the wars. Trusting scientists is clearly not a universal thing either, hence the need for reproducibility as well as very large minorities of people not trusting rather obviously provable stuff (like landing on the moon where nearly any ham radio operator could tell you we did). Even if trust were not low, trusting an authority is built of many things. Brain washing, propaganda, protocol/process, committees, culture, tradition, instinct, and many more things all work together to form obedience to authority… trust in the brain isn’t much of a requirement in obeisance to authority figures.
jedberg 5y ago
This is fair. We trust our own black boxes every day. You have a good point. But on the flip side, even today, a large part of the population doesn't trust people whose black boxes work faster than theirs. Now imagine a new black box that's 1000 times faster.
- kelseyfrog 5y ago
  If there is money to be made in black boxes, black boxes will prevail.
- Razengan 5y ago
  Imagine a black box that is more capable of eliminating other boxes who don’t trust it.

BirAdam 5y ago

In fairness to all of us humans, no. We do not blindly trust our brains. This explicitly how science came about, and this is also how entire branches of philosophy came about.

newsbinator 5y ago
We do blindly trust our brains all the time. Whenever a brain or a group of brains tells us we need chemotherapy or to start a war, with barely the slightest understanding of how the brain is wired or based on which inferences it would have come to that conclusion, we believe it. Obviously we believe only those brains that were primed with enough training data and have a decent track record of giving useful answers and making accurate-enough predictions. But we still rely on this "blindly" in the absence of their source code or even any understanding about how or why they think, or why some collections of cells reason and demonstrate intelligence, while most don't. We currently hold machine brains to a much higher standard of transparency and comprehensibility than we do human brains. Human brains can't even tell us much about themselves or show any logging, because they barely know anything about how they work or where ideas or connections come from either.
- BirAdam 5y ago
  In general, no. If we blindly trusted our brains, we would not require things like “another set of eyes,” or the rigors of reproducibility, or business processes/protocols, or a million other things. Further more, this lack of trust in the matter between our ears is observable in the course of daily human life when such questions are asked as “why did I say that?” or “why did I do that?”. Blind trust would obviate the need for such questions. As for the supposed trust in authority that you describe, that is not blind trust in the brain. Trust in authorities is actually quite low at the moment, and was never really blind. Every election in the USA, half of the country doesn’t trust the people elected, and possibly even the election itself. More over, War is no longer declared, and people never blindly trusted the government on war anyway, hence all of the protests of all of the wars. Trusting scientists is clearly not a universal thing either, hence the need for reproducibility as well as very large minorities of people not trusting rather obviously provable stuff (like landing on the moon where nearly any ham radio operator could tell you we did). Even if trust were not low, trusting an authority is built of many things. Brain washing, propaganda, protocol/process, committees, culture, tradition, instinct, and many more things all work together to form obedience to authority… trust in the brain isn’t much of a requirement in obeisance to authority figures.

jedberg 5y ago

This is fair. We trust our own black boxes every day. You have a good point. But on the flip side, even today, a large part of the population doesn't trust people whose black boxes work faster than theirs. Now imagine a new black box that's 1000 times faster.

kelseyfrog 5y ago
If there is money to be made in black boxes, black boxes will prevail.
Razengan 5y ago
Imagine a black box that is more capable of eliminating other boxes who don’t trust it.