8 ms·

I completely agree. Nonetheless, this approach of taking a fairly complex writing prompt and simply regurgitating contextually similar text from the internet o

I completely agree. Nonetheless, this approach of taking a fairly complex writing prompt and simply regurgitating contextually similar text from the internet often yields 100% meaningful results. This suggests that our intelligence is a lot more mimetic than we'd like to believe, which is perhaps an even more unpopular opinion.

benreesman 4y ago
I tend to agree that the math gets a lot simpler if the sun doesn’t rotate around a mystical axiom that humans are “special”.
- benreesman 4y ago
  High-energy physicists are about as serious as it gets, but they’re still dragging their feet around many worlds because it has uncomfortable implications for consciousness and free will. Homo Sapiens has a serious agenda around being something other than the smartest chimpanzee.
  - alar44 4y ago
    No, they're "dragging their feet" because it's impossible to test, it's unfalsifiable.
    - benreesman 4y ago
      Superposition collapse is impossible to test by definition of not including the observer in the system. If we just admit the possibility that the math was right all along we win, the math works, and we lose “I am a unique, distinct entity.” I’m on team math.
      - IdealeZahlen 4y ago
        That's math;not physics. You need experiments in physics.
        ncallaway 4y ago
        Can you explain the experiments and physics that justifies physicists beliefs in the Copenhagen interpretation of QM?
        IdealeZahlen 4y ago
        Experiments justify 'predictions' of certain mathematical models, not 'interpretations'. As for a belief in interpretations, it's more of a philsophy, and Feynman said just 'shut up and calculate'.
        ncallaway 4y ago
        I agree that interpretations of QM are currently in the realm of philosophy, but (and, at the risk of being a fool in saying this) I disagree with Feynman when he says “shut up and calculate” [1]. I tend to lean towards Sean Carrol’s approach of the interpretation is important and we should spend real effort thinking about the different impacts of different interpretations. After all, Einstein didn’t get to general relativity just by calculating. It first required deep thought and consideration to decide what to calculate. And I’m not convinced that the interpretations will forever remain in the realm of philosophy (though, I accept I could be totally wrong about this). [1] I don’t really disagree with Feynman on this. I think it’s perfect advice for people getting started in really learning QM (which is beyond where I am!). Being very familiar with the mechanics and being able to calculate fluently are probably a good starting point for people learning QM. But I do think that it’s worth experts in the field spending some time beyond the mathematical fundamentals.
      - tempe1h1 4y ago
        > we lose “I am a unique, distinct entity. I agree with this sentiment. quantum mechanics (stemming out of statistics mechanics) start from regarding all particles as indistinguishable. IMO, this theory is more mathematics than it is physics. Given a wide enough application, humans end up ourselves as particles, essentially we are commoditifiying our individuality.
    - omnicognate 4y ago
      And because if every outcome happens the probabilities of those outcomes are meaningless, and those probabilities are the predictive content of a quantum mechanical theory. It's an "interpretation" that requires you to ditch the entire value of the thing you're "interpreting". You don't need to know anything about QM other than that it has been empirically tested and is probabilistic in nature to reject the many worlds interpretation on this basis. Each time I mention this online I state it more confidently, in the hope that some day someone will Cunningham's law me and change my mind. If I haven't committed some gross misunderstanding it's disappointing that so many physicists fall for such obvious bunk. I've already seen the attempted arguments listed at [1] and none of them are remotely convincing. [1] https://en.wikipedia.org/wiki/Many-worlds_interpretation#Probability_and_the_Born_rule https://en.wikipedia.org/wiki/Many-worlds_interpretation#Pro...
      - benreesman 4y ago
        Hey if you’ve got David Deutsch dead to rights you should book a meeting and have him hand you his Fellowship in the Royal Society. I’m not a physicist but I am an interested layman, I’m pretty sure I’ll catch wind of the Nobel Prize you’d win for proving him wrong about everything.
        omnicognate 4y ago
        This isn't some unique insight of mine. There's a section about it in the wikipedia page for MWI, which I linked in my comment. Everett himself referenced the problem and attempted, unconvincingly, to argue around it in his original paper. The mystery to me is why the obvious flaw in the idea hasn't killed it dead for so many smart people, such as David Deutsch. Instead they engage in bizarre logical contortions to try and recover the Born probabilities and some shadow of a meaning for them. If you think there's some actual merit in their attempts, listed at the wiki link I pasted, I'm all ears.
        benreesman 4y ago
        Right, it’s well known that if you’re committed to getting probabilities out of amplitudes things go off the rails. Everett himself couldn’t go the full way. Fully deterministic, you see the part of the wave function that you see. You’re entangled with the apparatus. Deutsch, and his protege Marletto go the whole way: no free will, no arrow of time, no subjective human experience at all. Now? No measurement problem. No interpretations of collapse. These people have decided that the subjective experience of observing an experiment is of secondary importance to clean math. I appreciate that “you and I don’t exist in any way we’d recognize it” is a big pill to swallow, but I find the idea that humans looking in microscopes mutates the universe a bigger pill.
  - sitkack 4y ago
    I think of humans as a lens, mirror or piece of turbulence in a high dimensional space that information about the universe flows through. Maybe the smartest chimp, but not any more or less alive than anything else. Somewhat fortunate in the ways that we get to experience existence.
skybrian 4y ago
What this experiment shows is that making stuff up off the top of your head, using memory alone, isn't as hard as it looks. But when doing actual research, we check our work against the real world. For example, that's how you get a list of real references rather than fake ones. Suppose we played a guessing game: given a title, does the Wikipedia article exist or not? You could fairly confidently say that "Apple" exists and "wjifdvq" does not, but given a plausible-looking name of a person or place that you don't recognize, you'd have a harder time. It's not a problem in practice though, because you can look it up.
- benreesman 4y ago
  I sometimes fine tune new language models on the HN comment corpus for fun. It generates comments arguing that ML models are just parrots.
  - Swizec 4y ago
    As a bird owner, they are not “just parrots”. There is clear intent to communicate. Even species who can’t say human words use distinct vocalizations for specific purposes and situations.
    - benreesman 4y ago
      I didn’t say that fine-tuned language models have enlightened opinions about living things, just that they produce comments indistinguishable to me from the GP :)
  - IIAOPSW 4y ago
    https://i.imgur.com/xRaUzUGh.jpg https://i.imgur.com/xRaUzUGh.jpg
strogonoff 4y ago
Even if a sentence is meaningful standalone in vacuum, no communication between humans ever happens in such a way. It always matters who said it and why; we never take the substance independently from the agenda and the mind behind it, the context in which that mind existed and its relationship with our own, and so on. This strikes me as a crucial part of what intelligence (whether in humans or other creatures) means. In case of an algorithm, there is no intent of its own—except the intent and the minds of humans who trained it, ran it and supplied inputs. IMO the onus is on AGI believers to prove that material world is somehow the source of consciousness or intelligence (it wouldn’t hurt defining the terms first either). Otherwise it’s a philosophical position, and while one is entitled to hold their own one is not entitled to force it onto others.
- MontyCarloHall 4y ago
  >Even if a sentence is meaningful standalone in vacuum, no communication between humans ever happens in such a way. True, although much less so for (pseudo)anonymous online communication, as we’re having here. >It always matters who said it and why; we never take the substance independently from the agenda and the mind behind it, the context in which that mind existed and its relationship with our own, and so on. I have no idea who you are, what your agenda is, or anything about how your mind operates. There is essentially zero relationship between your mind and my own. The only context I have about you is the text of your post. >This strikes me as a crucial part of what intelligence (whether in humans or other creatures) means. Totally agreed. This is why GPT-3 can do a fairly good job emulating anonymous online discourse, but cannot convincingly emulate a person we actually know.
  - strogonoff 4y ago
    Personally, the fact that we are on HN of all places supplies a lot of context and agenda and to me is a meta-filter of sorts (pardon the pun). Perhaps one relationship between our minds is that we both have found our way here. I think there is a shared motive in trying to locate truth with a hint of procrastination[0]. Let’s imagine a slightly more down-to-earth exchange and raise the stakes in a way. For example, say it is still mostly a philosophical discussion here on HN, but in which someone is pushing a position on how much potential a certain company or store of value has; would you not question (at least to yourself) whether that person is invested and seeking to profit from it in short term, which would taint the motive? Or say someone was arguing in support of a controversial policy of a particular government known for its strong control over access to information and freedom of expression; would you not wonder whether the person is in fact a citizen of that country being misled by own government (and/or motivated to support it rather than seek truth)? Yes, the substance of what they say may or may not be true independently of that context, but if we want to function socially and exhaustively validating every claim being made is not an option we have to take shortcuts, and I think we do it all the time even without realizing it. (I’m not writing that lightly since it seems similar to profiling which is ethically icky, but it is my conclusion upon introspection.) In these cases we can at least imagine possible motive mismatch (known unknowns); in case of a GPT3-like thing instead of a motive you get a scary abyss or much more obscured motives of its human creators. I can’t imagine it having no impact on how I participate in an exchange. [0] Even still, you can see how elsewhere in the thread there are warranted accusations of the motive being tainted by human exceptionality bias.

benreesman 4y ago

I tend to agree that the math gets a lot simpler if the sun doesn’t rotate around a mystical axiom that humans are “special”.

benreesman 4y ago
High-energy physicists are about as serious as it gets, but they’re still dragging their feet around many worlds because it has uncomfortable implications for consciousness and free will. Homo Sapiens has a serious agenda around being something other than the smartest chimpanzee.
- alar44 4y ago
  No, they're "dragging their feet" because it's impossible to test, it's unfalsifiable.
  - benreesman 4y ago
    Superposition collapse is impossible to test by definition of not including the observer in the system. If we just admit the possibility that the math was right all along we win, the math works, and we lose “I am a unique, distinct entity.” I’m on team math.
    - IdealeZahlen 4y ago
      That's math;not physics. You need experiments in physics.
      - ncallaway 4y ago
        Can you explain the experiments and physics that justifies physicists beliefs in the Copenhagen interpretation of QM?
        IdealeZahlen 4y ago
        Experiments justify 'predictions' of certain mathematical models, not 'interpretations'. As for a belief in interpretations, it's more of a philsophy, and Feynman said just 'shut up and calculate'.
        ncallaway 4y ago
        I agree that interpretations of QM are currently in the realm of philosophy, but (and, at the risk of being a fool in saying this) I disagree with Feynman when he says “shut up and calculate” [1]. I tend to lean towards Sean Carrol’s approach of the interpretation is important and we should spend real effort thinking about the different impacts of different interpretations. After all, Einstein didn’t get to general relativity just by calculating. It first required deep thought and consideration to decide what to calculate. And I’m not convinced that the interpretations will forever remain in the realm of philosophy (though, I accept I could be totally wrong about this). [1] I don’t really disagree with Feynman on this. I think it’s perfect advice for people getting started in really learning QM (which is beyond where I am!). Being very familiar with the mechanics and being able to calculate fluently are probably a good starting point for people learning QM. But I do think that it’s worth experts in the field spending some time beyond the mathematical fundamentals.
    - tempe1h1 4y ago
      > we lose “I am a unique, distinct entity. I agree with this sentiment. quantum mechanics (stemming out of statistics mechanics) start from regarding all particles as indistinguishable. IMO, this theory is more mathematics than it is physics. Given a wide enough application, humans end up ourselves as particles, essentially we are commoditifiying our individuality.
  - omnicognate 4y ago
    And because if every outcome happens the probabilities of those outcomes are meaningless, and those probabilities are the predictive content of a quantum mechanical theory. It's an "interpretation" that requires you to ditch the entire value of the thing you're "interpreting". You don't need to know anything about QM other than that it has been empirically tested and is probabilistic in nature to reject the many worlds interpretation on this basis. Each time I mention this online I state it more confidently, in the hope that some day someone will Cunningham's law me and change my mind. If I haven't committed some gross misunderstanding it's disappointing that so many physicists fall for such obvious bunk. I've already seen the attempted arguments listed at [1] and none of them are remotely convincing. [1] https://en.wikipedia.org/wiki/Many-worlds_interpretation#Probability_and_the_Born_rule https://en.wikipedia.org/wiki/Many-worlds_interpretation#Pro...
    - benreesman 4y ago
      Hey if you’ve got David Deutsch dead to rights you should book a meeting and have him hand you his Fellowship in the Royal Society. I’m not a physicist but I am an interested layman, I’m pretty sure I’ll catch wind of the Nobel Prize you’d win for proving him wrong about everything.
      - omnicognate 4y ago
        This isn't some unique insight of mine. There's a section about it in the wikipedia page for MWI, which I linked in my comment. Everett himself referenced the problem and attempted, unconvincingly, to argue around it in his original paper. The mystery to me is why the obvious flaw in the idea hasn't killed it dead for so many smart people, such as David Deutsch. Instead they engage in bizarre logical contortions to try and recover the Born probabilities and some shadow of a meaning for them. If you think there's some actual merit in their attempts, listed at the wiki link I pasted, I'm all ears.
        benreesman 4y ago
        Right, it’s well known that if you’re committed to getting probabilities out of amplitudes things go off the rails. Everett himself couldn’t go the full way. Fully deterministic, you see the part of the wave function that you see. You’re entangled with the apparatus. Deutsch, and his protege Marletto go the whole way: no free will, no arrow of time, no subjective human experience at all. Now? No measurement problem. No interpretations of collapse. These people have decided that the subjective experience of observing an experiment is of secondary importance to clean math. I appreciate that “you and I don’t exist in any way we’d recognize it” is a big pill to swallow, but I find the idea that humans looking in microscopes mutates the universe a bigger pill.
- sitkack 4y ago
  I think of humans as a lens, mirror or piece of turbulence in a high dimensional space that information about the universe flows through. Maybe the smartest chimp, but not any more or less alive than anything else. Somewhat fortunate in the ways that we get to experience existence.

skybrian 4y ago

What this experiment shows is that making stuff up off the top of your head, using memory alone, isn't as hard as it looks. But when doing actual research, we check our work against the real world. For example, that's how you get a list of real references rather than fake ones. Suppose we played a guessing game: given a title, does the Wikipedia article exist or not? You could fairly confidently say that "Apple" exists and "wjifdvq" does not, but given a plausible-looking name of a person or place that you don't recognize, you'd have a harder time. It's not a problem in practice though, because you can look it up.

benreesman 4y ago
I sometimes fine tune new language models on the HN comment corpus for fun. It generates comments arguing that ML models are just parrots.
- Swizec 4y ago
  As a bird owner, they are not “just parrots”. There is clear intent to communicate. Even species who can’t say human words use distinct vocalizations for specific purposes and situations.
  - benreesman 4y ago
    I didn’t say that fine-tuned language models have enlightened opinions about living things, just that they produce comments indistinguishable to me from the GP :)
- IIAOPSW 4y ago
  https://i.imgur.com/xRaUzUGh.jpg https://i.imgur.com/xRaUzUGh.jpg

strogonoff 4y ago

Even if a sentence is meaningful standalone in vacuum, no communication between humans ever happens in such a way. It always matters who said it and why; we never take the substance independently from the agenda and the mind behind it, the context in which that mind existed and its relationship with our own, and so on. This strikes me as a crucial part of what intelligence (whether in humans or other creatures) means. In case of an algorithm, there is no intent of its own—except the intent and the minds of humans who trained it, ran it and supplied inputs. IMO the onus is on AGI believers to prove that material world is somehow the source of consciousness or intelligence (it wouldn’t hurt defining the terms first either). Otherwise it’s a philosophical position, and while one is entitled to hold their own one is not entitled to force it onto others.

MontyCarloHall 4y ago
>Even if a sentence is meaningful standalone in vacuum, no communication between humans ever happens in such a way. True, although much less so for (pseudo)anonymous online communication, as we’re having here. >It always matters who said it and why; we never take the substance independently from the agenda and the mind behind it, the context in which that mind existed and its relationship with our own, and so on. I have no idea who you are, what your agenda is, or anything about how your mind operates. There is essentially zero relationship between your mind and my own. The only context I have about you is the text of your post. >This strikes me as a crucial part of what intelligence (whether in humans or other creatures) means. Totally agreed. This is why GPT-3 can do a fairly good job emulating anonymous online discourse, but cannot convincingly emulate a person we actually know.
- strogonoff 4y ago
  Personally, the fact that we are on HN of all places supplies a lot of context and agenda and to me is a meta-filter of sorts (pardon the pun). Perhaps one relationship between our minds is that we both have found our way here. I think there is a shared motive in trying to locate truth with a hint of procrastination[0]. Let’s imagine a slightly more down-to-earth exchange and raise the stakes in a way. For example, say it is still mostly a philosophical discussion here on HN, but in which someone is pushing a position on how much potential a certain company or store of value has; would you not question (at least to yourself) whether that person is invested and seeking to profit from it in short term, which would taint the motive? Or say someone was arguing in support of a controversial policy of a particular government known for its strong control over access to information and freedom of expression; would you not wonder whether the person is in fact a citizen of that country being misled by own government (and/or motivated to support it rather than seek truth)? Yes, the substance of what they say may or may not be true independently of that context, but if we want to function socially and exhaustively validating every claim being made is not an option we have to take shortcuts, and I think we do it all the time even without realizing it. (I’m not writing that lightly since it seems similar to profiling which is ethically icky, but it is my conclusion upon introspection.) In these cases we can at least imagine possible motive mismatch (known unknowns); in case of a GPT3-like thing instead of a motive you get a scary abyss or much more obscured motives of its human creators. I can’t imagine it having no impact on how I participate in an exchange. [0] Even still, you can see how elsewhere in the thread there are warranted accusations of the motive being tainted by human exceptionality bias.