5 ms·

That's the problem with any superintelligence story; they are by definition hard to write without being superintelligent. As Vinge was famously told, "you can't

by gwern 4y ago

That's the problem with any superintelligence story; they are by definition hard to write without being superintelligent. As Vinge was famously told, "you can't write this story. No one can." If a chimpanzee could write a story about a human expert of any sort, the other chimpanzees wouldn't understand it: it would either be gibberish, or dumbed down to superficial analogies that give an illusion of understanding. ("Then he used his rifle -" "what's a rifle?" "it's a stick which is like throwing a rock. Anyway, he traded some bananas for it with another monkey off the Internet." "What's an Internet?" "uh...") 'Flowers' gets around it by starting with a mentally retarded protagonist and specifically trying to avoid any consequences of superintelligence beyond the emotional & social journey, so most of the story is accomplished 'on the runway', as it were, and is about everything but what he learns & does with his intelligence which is pushed into the background. You can see how it starts getting handwavy as soon as the protagonist takes off to smarter than Keys himself, and he starts having to show the progress by him simply doing ordinary-human things but much faster than a dimmer human, like rearranging the bakery for more efficiency or learning Sanskrit in a week. While it's nice to be able to read German or Sanskrit, it's not particularly useful, especially if you are interested in neuroscience; a real protagonist would be doing things Keys can't even imagine, which sound like gibberish like 'ordinary differential equations' or 'symplectic manifold'. Any societal implications are simply ignored. With 'Understand', Chiang starts with an intelligent protagonist, in a strictly realistic universe other than the superintelligence, where he's well aware there would be major societal consequences and military implications and the protagonist can't simply sit around and play with his lab mouse. So his back is against the narrative wall from the start. He cannot do his usual world-building tricks because he's both ruled out the world mechanics he is usually in a privileged position to understand impossibly well because he made them up in the first place, and because he's not smart enough to write the superintelligent character he's assigned himself. It's an interesting story, but I agree that it can't be considered his best. Because he can't write that story as well as he wants to, and no one can.

cmrdporcupine 4y ago
"the other chimpanzees wouldn't understand it: it would either be gibberish, or dumbed down" This is kind of how "Excession" by Iain M Banks is. Much of the dialogue in the book is supposed to be between the hyperintelligent AI "Minds" that are the ships in his Culture universe. It's supposed to be in a form similar to how they talk to each other. It's quite hard to read. I can never decide which of the two (gibberish or dumbed down) it is. Probably both.
- euroderf 4y ago
  The inevitable Arthur C Clarke paraphrase: "Any sufficiently advanced conversation is indistinguishable from gibberish."
myself248 4y ago
> That's the problem with any superintelligence story; they are by definition hard to write without being superintelligent. Ditto this, but for any domain specialization. A lot of fiction gives me a bad case of something which I don't have a term for but it seems like it might be the inverse of the Gell-Mann Amnesia Effect: Basically, I was fine with the story until it got technical in a way that I know well, and the technical aspects of the story were so egregiously sloppy, I could no longer suspend my disbelief. And then even the nontechnical aspects just annoyed me because I figured, if the author was such an idiot they couldn't bother to look up how lasers work or whatever, they probably got everything else wrong too. Even the parts I'm not a specialist in, which I had previously just been taking on faith, I am now forced to assume that a specialist in those things would also find them laughably bad. And the whole work is ruined for me. (To be clear: If the author would just say, "these are magic lasers" or some other form of "a wizard did it", okay that's fine, it's fiction, I'll accept that, let's see how the story develops from there. It's when they try to claim they're real-world lasers but then merrily ascribe them impossible traits, that I get brain-jammed.)
- techdragon 4y ago
  I’m glad I’m not the only one that has this problem!
- WalterBright 4y ago
  In 20,000 Leagues Under The Sea, there's a chapter where a bunch of figures about the submarine are given. I played around with the math, and discovered that the figures were accurate. Cool!
  - myself248 4y ago
    Probably not surprising at all, but my brother once ran the numbers on how much energy it would take to "fire a pound of bacon into the asteroid belt", and I don't remember the exact number but it was roughly comparable to the capacity of modern EV batteries -- a few tens of kwh. Of course Stephenson figured out how much battery a useful EV would need, before concocting a throwaway line about it. In 1992.
_qfi9 4y ago
we can know certain things about things that are smarter than us. perhaps even smarter than a post singularity intelligence, for instance, we are "computationally" equivalent to a turing machine: a tape with symbols on it and a head that reads and writes symbols to the tape according to rules. the natural generalization of a turing machine to infinity would be a hyper turing machine: an infinitely wide tape and infinitely many heads. if one were to employ some kind of godel numbering esque scheme, most (the overwhelming majority) of the symbols the hyper turing machine used in it's operation/had in it's rule lookup table would correspond to transcendental numbers, e.g. pi or e (cuz their cardinality is larger than natural numbers). The alphabet our math uses is pretty much all finite. What percent of the axioms and theorems that you know are infinitely long? Shouldn't the overwhelming majority of axioms and theorems be infinitely long (and in particular map onto an uncountable set? (e.g. transcendental numbers). So a hyper turing machine would be a supra-mathematical entity. But you can pretty much use math to deduce that.
- geysersam 4y ago
  > we are "computationally" equivalent to a turing machine: Why do you think that? We know of modes of computation different from Turing machines, quantum computers. There's nothing saying there might not be yet other kinds of computers currently undiscovered.
  - zzzzzzzza 4y ago
    actually i think we are more limited than a turing machine since even the default turing machine has a (single) infinite tape/running time. But there is a quantum turing machine/quantum lambda calculus as well. just has (probably?) a faster running time on a subset of algorithms. but essentially turing machine/quantum turing machine have the same kinds of inpus/outputs or domain/range, whereas a hyper turing machine has a bigger domain/range.
    - geysersam 4y ago
      I don't understand what makes you think humans operate anything like Turing machines to begin with. The only way I see to reach this conclusion is to assume Turing machines are all that exist, and the conclusion follows.
WalterBright 4y ago
I always kinda avoid scifi stories about super-intelligent beings. There's no way one could write about what they'd be like. But I make one exception - Brain Wave by Poul Anderson. In it, ordinary people become much more intelligent. Their speech patterns change into very few words, because a more intelligent listener would be better at inferring meaning from the context.
dmitriid 4y ago
> a real protagonist would be doing things Keys can't even imagine, which sound like gibberish Mimsy Were the Borogoves by Lewis Padgett explores this to an extent.
- MollyRealized 4y ago
  For those who are interested in the story: Lewis Padgett was a pseudonym for Henry Kuttner [1] and his wife. The short story is available in an anthology under that name, "The Last Mimsy", and a 2007 film was made under that title [3]. [1] - https://en.wikipedia.org/wiki/Lewis_Padgett https://en.wikipedia.org/wiki/Lewis_Padgett [2] - https://www.amazon.com/Last-Mimzy-Stories-Henry-Kuttner/dp/0345497554/ https://www.amazon.com/Last-Mimzy-Stories-Henry-Kuttner/dp/0... [3] - https://www.justwatch.com/us/movie/the-last-mimzy https://www.justwatch.com/us/movie/the-last-mimzy
helloworld11 4y ago
>That's the problem with any superintelligence story; they are by definition hard to write without being superintelligent. As Vinge was famously told, "you can't write this story. No one can." If a chimpanzee could write a story about a human expert of any sort, the other chimpanzees wouldn't understand it: it would either be gibberish, or dumbed down to superficial analogies that give an illusion of understanding. ("Then he used his rifle -" "what's a rifle?" "it's a stick which is like throwing a rock. Anyway, he traded some bananas for it with another monkey off the Internet." "What's an Internet?" "uh...") I'm not entirely sure if that's true. It might be, I honestly don't know, but my personal suspicion is that there's such a thing as what I'll just go ahead and call a threshold intelligence level. In other words, a level of cognitive capacity beyond which a creature that attains said level can functionally conceptualize things far above its actual ability to understand them in detail. This allows discussion and exploration of certain extremely advanced concepts well enough to form a narrative without falling into complete incomprehension. Chimpanzees might be far enough below this threshold level that even a basic story would cause them difficulty no matter how learned they are educated or bred to be, and a complex story about, say space ships or high performance aircraft would be completely outside their conception. Humans on the other hand might be able to stretch their imagination much further towards the extremes of conceptualization. However, I also think it might have its limits, namely that there are things which if explained to us would leave even the brightest humans as befuddled as a dog being lectured on internal combustion mechanics.
captn3m0 4y ago
Brandon Sanderson has a nice video on advice on how he writes characters that are smarter than him: https://www.youtube.com/watch?v=YyaC7NmPsc0 https://www.youtube.com/watch?v=YyaC7NmPsc0 (~5 minutes). tl;dw: There's different axes of intelligence, and you can setup the situation perfectly to fake the character's intelligence. You can also fake wisdom/intelligence by taking as much time as you need to come up with the smartest/best response to a situation. Doesn't apply exactly here (He has written Gods, but they are still "mostly" human except for knowing a lot more about their world), but it's good advice.

by gwern 4y ago

cmrdporcupine 4y ago

"the other chimpanzees wouldn't understand it: it would either be gibberish, or dumbed down" This is kind of how "Excession" by Iain M Banks is. Much of the dialogue in the book is supposed to be between the hyperintelligent AI "Minds" that are the ships in his Culture universe. It's supposed to be in a form similar to how they talk to each other. It's quite hard to read. I can never decide which of the two (gibberish or dumbed down) it is. Probably both.

euroderf 4y ago
The inevitable Arthur C Clarke paraphrase: "Any sufficiently advanced conversation is indistinguishable from gibberish."

myself248 4y ago

> That's the problem with any superintelligence story; they are by definition hard to write without being superintelligent. Ditto this, but for any domain specialization. A lot of fiction gives me a bad case of something which I don't have a term for but it seems like it might be the inverse of the Gell-Mann Amnesia Effect: Basically, I was fine with the story until it got technical in a way that I know well, and the technical aspects of the story were so egregiously sloppy, I could no longer suspend my disbelief. And then even the nontechnical aspects just annoyed me because I figured, if the author was such an idiot they couldn't bother to look up how lasers work or whatever, they probably got everything else wrong too. Even the parts I'm not a specialist in, which I had previously just been taking on faith, I am now forced to assume that a specialist in those things would also find them laughably bad. And the whole work is ruined for me. (To be clear: If the author would just say, "these are magic lasers" or some other form of "a wizard did it", okay that's fine, it's fiction, I'll accept that, let's see how the story develops from there. It's when they try to claim they're real-world lasers but then merrily ascribe them impossible traits, that I get brain-jammed.)

techdragon 4y ago
I’m glad I’m not the only one that has this problem!
WalterBright 4y ago
In 20,000 Leagues Under The Sea, there's a chapter where a bunch of figures about the submarine are given. I played around with the math, and discovered that the figures were accurate. Cool!
- myself248 4y ago
  Probably not surprising at all, but my brother once ran the numbers on how much energy it would take to "fire a pound of bacon into the asteroid belt", and I don't remember the exact number but it was roughly comparable to the capacity of modern EV batteries -- a few tens of kwh. Of course Stephenson figured out how much battery a useful EV would need, before concocting a throwaway line about it. In 1992.

_qfi9 4y ago

we can know certain things about things that are smarter than us. perhaps even smarter than a post singularity intelligence, for instance, we are "computationally" equivalent to a turing machine: a tape with symbols on it and a head that reads and writes symbols to the tape according to rules. the natural generalization of a turing machine to infinity would be a hyper turing machine: an infinitely wide tape and infinitely many heads. if one were to employ some kind of godel numbering esque scheme, most (the overwhelming majority) of the symbols the hyper turing machine used in it's operation/had in it's rule lookup table would correspond to transcendental numbers, e.g. pi or e (cuz their cardinality is larger than natural numbers). The alphabet our math uses is pretty much all finite. What percent of the axioms and theorems that you know are infinitely long? Shouldn't the overwhelming majority of axioms and theorems be infinitely long (and in particular map onto an uncountable set? (e.g. transcendental numbers). So a hyper turing machine would be a supra-mathematical entity. But you can pretty much use math to deduce that.

geysersam 4y ago
> we are "computationally" equivalent to a turing machine: Why do you think that? We know of modes of computation different from Turing machines, quantum computers. There's nothing saying there might not be yet other kinds of computers currently undiscovered.
- zzzzzzzza 4y ago
  actually i think we are more limited than a turing machine since even the default turing machine has a (single) infinite tape/running time. But there is a quantum turing machine/quantum lambda calculus as well. just has (probably?) a faster running time on a subset of algorithms. but essentially turing machine/quantum turing machine have the same kinds of inpus/outputs or domain/range, whereas a hyper turing machine has a bigger domain/range.
  - geysersam 4y ago
    I don't understand what makes you think humans operate anything like Turing machines to begin with. The only way I see to reach this conclusion is to assume Turing machines are all that exist, and the conclusion follows.

WalterBright 4y ago

I always kinda avoid scifi stories about super-intelligent beings. There's no way one could write about what they'd be like. But I make one exception - Brain Wave by Poul Anderson. In it, ordinary people become much more intelligent. Their speech patterns change into very few words, because a more intelligent listener would be better at inferring meaning from the context.

dmitriid 4y ago

> a real protagonist would be doing things Keys can't even imagine, which sound like gibberish Mimsy Were the Borogoves by Lewis Padgett explores this to an extent.

MollyRealized 4y ago
For those who are interested in the story: Lewis Padgett was a pseudonym for Henry Kuttner [1] and his wife. The short story is available in an anthology under that name, "The Last Mimsy", and a 2007 film was made under that title [3]. [1] - https://en.wikipedia.org/wiki/Lewis_Padgett https://en.wikipedia.org/wiki/Lewis_Padgett [2] - https://www.amazon.com/Last-Mimzy-Stories-Henry-Kuttner/dp/0345497554/ https://www.amazon.com/Last-Mimzy-Stories-Henry-Kuttner/dp/0... [3] - https://www.justwatch.com/us/movie/the-last-mimzy https://www.justwatch.com/us/movie/the-last-mimzy

helloworld11 4y ago

>That's the problem with any superintelligence story; they are by definition hard to write without being superintelligent. As Vinge was famously told, "you can't write this story. No one can." If a chimpanzee could write a story about a human expert of any sort, the other chimpanzees wouldn't understand it: it would either be gibberish, or dumbed down to superficial analogies that give an illusion of understanding. ("Then he used his rifle -" "what's a rifle?" "it's a stick which is like throwing a rock. Anyway, he traded some bananas for it with another monkey off the Internet." "What's an Internet?" "uh...") I'm not entirely sure if that's true. It might be, I honestly don't know, but my personal suspicion is that there's such a thing as what I'll just go ahead and call a threshold intelligence level. In other words, a level of cognitive capacity beyond which a creature that attains said level can functionally conceptualize things far above its actual ability to understand them in detail. This allows discussion and exploration of certain extremely advanced concepts well enough to form a narrative without falling into complete incomprehension. Chimpanzees might be far enough below this threshold level that even a basic story would cause them difficulty no matter how learned they are educated or bred to be, and a complex story about, say space ships or high performance aircraft would be completely outside their conception. Humans on the other hand might be able to stretch their imagination much further towards the extremes of conceptualization. However, I also think it might have its limits, namely that there are things which if explained to us would leave even the brightest humans as befuddled as a dog being lectured on internal combustion mechanics.

captn3m0 4y ago

Brandon Sanderson has a nice video on advice on how he writes characters that are smarter than him: https://www.youtube.com/watch?v=YyaC7NmPsc0 https://www.youtube.com/watch?v=YyaC7NmPsc0 (~5 minutes). tl;dw: There's different axes of intelligence, and you can setup the situation perfectly to fake the character's intelligence. You can also fake wisdom/intelligence by taking as much time as you need to come up with the smartest/best response to a situation. Doesn't apply exactly here (He has written Gods, but they are still "mostly" human except for knowing a lot more about their world), but it's good advice.