4 ms·

I'm amused by the thought that the AI models are trained on human knowledge, but human knowledge doesn't contain a reliable method for determining what truth co

I'm amused by the thought that the AI models are trained on human knowledge, but human knowledge doesn't contain a reliable method for determining what truth consists of, or what is true. I don't know how an AI could embody such a method itself.

jimmygrapes 4y ago
Thus the term "consensus truth"
- altruios 4y ago
  Hey, sorry to just randomly reply: but I saw your comment on my post and could not reply. I want to improve the typography - the current feedback I'm getting so far is that the current design works well with two friends who have dyslexia, but I think that's more to do with the formatting than current typography... I have no good sense of typographical style though. And now the more relevant comment! Consensus truth can be far from objective truth as ideas don't compete based on value or truth or usefulness: but merely by how sticky, how replicatable in the brains of others.
- bloaf 4y ago
  One of the wilder possibilities of the future of AI is discovering that we as humans have some form of collective anosognosia. The AI would just repeatedly and confidently assert things that we knew to be false, and we'd go to great lengths just to give the AI the same cognitive blinders we're wearing. https://en.wikipedia.org/wiki/Anosognosia https://en.wikipedia.org/wiki/Anosognosia Right now, the AI is operating "inside the bubble" of our thoughts and so is unlikely to figure out our collective blind spot, but once one can interact with the world in more meaningful ways, we should pay really close attention to the mistakes it makes.
breck 4y ago
> but human knowledge doesn't contain a reliable method for determining what truth consists of, or what is true. Depends on how many connections a statement has. `1+1=2` has an extremely high number of connections, so it would be very hard to vary that to `1+1=3`. When you start to say things like `Daniupolomonotrofin activates the mitoyuicinain leading to weightloss`, very few connections, very easy to vary (lie).
mdp2021 4y ago
> reliable method But we have heuristic. You do not even have the "perfect hammer", for example. But "close enough" is fine.

jimmygrapes 4y ago

Thus the term "consensus truth"

altruios 4y ago
Hey, sorry to just randomly reply: but I saw your comment on my post and could not reply. I want to improve the typography - the current feedback I'm getting so far is that the current design works well with two friends who have dyslexia, but I think that's more to do with the formatting than current typography... I have no good sense of typographical style though. And now the more relevant comment! Consensus truth can be far from objective truth as ideas don't compete based on value or truth or usefulness: but merely by how sticky, how replicatable in the brains of others.
bloaf 4y ago
One of the wilder possibilities of the future of AI is discovering that we as humans have some form of collective anosognosia. The AI would just repeatedly and confidently assert things that we knew to be false, and we'd go to great lengths just to give the AI the same cognitive blinders we're wearing. https://en.wikipedia.org/wiki/Anosognosia https://en.wikipedia.org/wiki/Anosognosia Right now, the AI is operating "inside the bubble" of our thoughts and so is unlikely to figure out our collective blind spot, but once one can interact with the world in more meaningful ways, we should pay really close attention to the mistakes it makes.

breck 4y ago

> but human knowledge doesn't contain a reliable method for determining what truth consists of, or what is true. Depends on how many connections a statement has. `1+1=2` has an extremely high number of connections, so it would be very hard to vary that to `1+1=3`. When you start to say things like `Daniupolomonotrofin activates the mitoyuicinain leading to weightloss`, very few connections, very easy to vary (lie).

mdp2021 4y ago

> reliable method But we have heuristic. You do not even have the "perfect hammer", for example. But "close enough" is fine.