3 ms·

That's two examples in one day of GPT being bad at scansion.

by throwanem 4y ago

coldtea 4y ago
Still 100x better than the average person!
- BitwiseFool 4y ago
  I read the Wikipedia article about Scansion and I still don't understand it.
  - user00012-ab 4y ago
    I asked chatgpt to explain the comment: The comment is referring to GPT's ability to scan and analyze the meter or rhythm of poetry. Scansion is the process of analyzing the poetic meter by marking the stressed and unstressed syllables in each line. The comment implies that there were two instances where GPT failed to accurately analyze the meter of two different poems in a single day. This may suggest that GPT is not proficient in scanning poetry or that it may not be the best tool for analyzing the meter of poetic works.
  - xena 4y ago
    tl;dr: when you read Shakespeare plays out loud, there's a rhythm and meter to it. The process for creating that is called scansion.
  - deleted 4y ago
    [deleted]
- topaz0 4y ago
  I'm sure you've measured this.
  - coldtea 4y ago
    I'm sure you know that one of the uses of intelligence is the ability to estimate things without measuring them - based on your actual experience and the quality of the model of the world that you have. Thankfully we can use it to know how to e.g. tie our shoes (without having formally studied knots), or to be able to tell intuitively that most people can't understand Quantum Mechanics, even if we haven't measured it. You are however welcome to do some statistics with actual samples of talking to people and having them show their skills at scansion, and prove my estimation wrong. Heck, I'm willing to bet less than 1 in 100 Americans is able to even define what "iambic pentameter" is. Feel free to prove me wrong on that one too. Except if your concern is the measurability of the skill: whether it's 100x and not just 10x or maybe just 8.345x better. In which case you've missed the point of using it as a stand-in for "much better".
    - topaz0 4y ago
      Just as we can tie our shoes without studying knots, many people can feel iambic pentameter without having any idea what the term means. Many can't, too, or can but only vaguely. Still, those people are still on par with chatgpt when it comes to scansion. I think you'd be hard-pressed to find anybody with a worse sense of meter, let alone 100x worse (again, without knowing any of the terminology etc.)
wongarsu 4y ago
Q: What are the syllables in the word President? ChatGPT: The syllables in the word "President" are "pre-si-dent", with stress on the second syllable "si". So it does know about the nessesary concepts. It can also explain scanision. It just isn't good at putting that into practise. I guess this is one example to show that GPT isn't an AGI, as much as some want it to be.
- topaz0 4y ago
  It doesn't know anything at all, it just puts probable sequences of words together, for some definition of probable.
  - wongarsu 4y ago
    It knows things in the sense of "you can ask it and it gives you a relevant and factual answer". It can also transfer knowledge to new situations to a degree. But it doesn't know everything, and there are stricter definitions of the word knowing where GPT (almost certainly) knows nothing. But just because the last layer outputs probabilities for tokens, doesn't mean you can't have intelligence in the layers above that. I don't think "it generates a chain of token pobabilies" is really disqualifying.
  - flangola7 4y ago
    The thing that doesn't know anything will still take your job.
    - topaz0 4y ago
      Nice. I never really liked working.
      - flangola7 4y ago
        While I agree, it will probably also kill us (or at least disrupt our lives in unfathomable ways). Either on the orders of someone else, in the social unrest when unemployment is 70% and basic concepts of human existence are torn to pieces, or when it decides our atoms could be used to increase its reward function value.

That's two examples in one day of GPT being bad at scansion.

coldtea 4y ago

Still 100x better than the average person!

BitwiseFool 4y ago
I read the Wikipedia article about Scansion and I still don't understand it.
- user00012-ab 4y ago
  I asked chatgpt to explain the comment: The comment is referring to GPT's ability to scan and analyze the meter or rhythm of poetry. Scansion is the process of analyzing the poetic meter by marking the stressed and unstressed syllables in each line. The comment implies that there were two instances where GPT failed to accurately analyze the meter of two different poems in a single day. This may suggest that GPT is not proficient in scanning poetry or that it may not be the best tool for analyzing the meter of poetic works.
- xena 4y ago
  tl;dr: when you read Shakespeare plays out loud, there's a rhythm and meter to it. The process for creating that is called scansion.
- deleted 4y ago
  [deleted]
topaz0 4y ago
I'm sure you've measured this.
- coldtea 4y ago
  I'm sure you know that one of the uses of intelligence is the ability to estimate things without measuring them - based on your actual experience and the quality of the model of the world that you have. Thankfully we can use it to know how to e.g. tie our shoes (without having formally studied knots), or to be able to tell intuitively that most people can't understand Quantum Mechanics, even if we haven't measured it. You are however welcome to do some statistics with actual samples of talking to people and having them show their skills at scansion, and prove my estimation wrong. Heck, I'm willing to bet less than 1 in 100 Americans is able to even define what "iambic pentameter" is. Feel free to prove me wrong on that one too. Except if your concern is the measurability of the skill: whether it's 100x and not just 10x or maybe just 8.345x better. In which case you've missed the point of using it as a stand-in for "much better".
  - topaz0 4y ago
    Just as we can tie our shoes without studying knots, many people can feel iambic pentameter without having any idea what the term means. Many can't, too, or can but only vaguely. Still, those people are still on par with chatgpt when it comes to scansion. I think you'd be hard-pressed to find anybody with a worse sense of meter, let alone 100x worse (again, without knowing any of the terminology etc.)

wongarsu 4y ago

Q: What are the syllables in the word President? ChatGPT: The syllables in the word "President" are "pre-si-dent", with stress on the second syllable "si". So it does know about the nessesary concepts. It can also explain scanision. It just isn't good at putting that into practise. I guess this is one example to show that GPT isn't an AGI, as much as some want it to be.

topaz0 4y ago
It doesn't know anything at all, it just puts probable sequences of words together, for some definition of probable.
- wongarsu 4y ago
  It knows things in the sense of "you can ask it and it gives you a relevant and factual answer". It can also transfer knowledge to new situations to a degree. But it doesn't know everything, and there are stricter definitions of the word knowing where GPT (almost certainly) knows nothing. But just because the last layer outputs probabilities for tokens, doesn't mean you can't have intelligence in the layers above that. I don't think "it generates a chain of token pobabilies" is really disqualifying.
- flangola7 4y ago
  The thing that doesn't know anything will still take your job.
  - topaz0 4y ago
    Nice. I never really liked working.
    - flangola7 4y ago
      While I agree, it will probably also kill us (or at least disrupt our lives in unfathomable ways). Either on the orders of someone else, in the social unrest when unemployment is 70% and basic concepts of human existence are torn to pieces, or when it decides our atoms could be used to increase its reward function value.