4 ms·

This looks at the first so many lyrics in each rapper's career. Aesop Rock came out with some weird stuff right off the bat. I wonder if some of these other rap

This looks at the first so many lyrics in each rapper's career. Aesop Rock came out with some weird stuff right off the bat. I wonder if some of these other rappers became more sophisticated over time. Maybe an average per song would be better, or average uniques per word, would be better.

iLoch 13y ago
I agree, perhaps the 35,000 most recent words would be better.
- danielsf 13y ago
  OP here: the challenge is that most artists' best work is in their earlier years. I'd rather have Jay-z's first album than last, ya know?
  - jeorgun 13y ago
    Would sorting by popularity, or critical acclaim, or something along those lines be a possibility?
sfrank2147 13y ago
The problem with average per song is that you "use up" words in every new song, so all things being equal each marginal song has progressively fewer new words.
- plorg 13y ago
  I bet you could get something insightful from plotting "unique words" versus "total words" - That might give a good idea of the amount of repetition over time, the length or quantity of output, and the total vocabulary.
  - danielsf 13y ago
    here's what this looks like. ugly as sin as useless for comparing rappers. http://www.mdaniels.com/vocab/scatter.png http://www.mdaniels.com/vocab/scatter.png love your other ideas – hopefully can do them later.
- jebus989 13y ago
  Strange comment, you realise that's not an inherent truth of language? Unique words per song is trivial to calculate
  - michaelt 13y ago
    If a rapper released one song using n distinct words their score would be n/1, and if they released a second song using the same set of words their score would halve, to n/2, despite the fact their demonstrated vocabulary is still n words. In fact, if their first song used n distinct words and their second used a completely distinct set of words, but the second song was shorter than the first, their score would drop. That would be unusual behaviour for a measure of vocabulary.
    - thinkpad20 13y ago
      I don't think that's what the poster meant. By "average unique words per song" I take it to mean, within each song words are only counted once, but across songs, words can be counted multiple times. So if song A had the words "I like cats" and song B had the words "I like dogs", then the average unique word count would be ((3 + 3) / 2) = 3, not ((3 + 1)/2) = 2.
      - jeorgun 13y ago
        That's definitely one solution, but it still wouldn't quite capture it. As an extreme example, if rapper A produced 100 songs, each with exactly the same lyrics, they should surely be penalized compared with rapper B producing 100 songs with no shared words— even if rapper A's average unique-words-per-song is higher than rapper B's.

iLoch 13y ago

I agree, perhaps the 35,000 most recent words would be better.

danielsf 13y ago
OP here: the challenge is that most artists' best work is in their earlier years. I'd rather have Jay-z's first album than last, ya know?
- jeorgun 13y ago
  Would sorting by popularity, or critical acclaim, or something along those lines be a possibility?

sfrank2147 13y ago

The problem with average per song is that you "use up" words in every new song, so all things being equal each marginal song has progressively fewer new words.

plorg 13y ago
I bet you could get something insightful from plotting "unique words" versus "total words" - That might give a good idea of the amount of repetition over time, the length or quantity of output, and the total vocabulary.
- danielsf 13y ago
  here's what this looks like. ugly as sin as useless for comparing rappers. http://www.mdaniels.com/vocab/scatter.png http://www.mdaniels.com/vocab/scatter.png love your other ideas – hopefully can do them later.
jebus989 13y ago
Strange comment, you realise that's not an inherent truth of language? Unique words per song is trivial to calculate
- michaelt 13y ago
  If a rapper released one song using n distinct words their score would be n/1, and if they released a second song using the same set of words their score would halve, to n/2, despite the fact their demonstrated vocabulary is still n words. In fact, if their first song used n distinct words and their second used a completely distinct set of words, but the second song was shorter than the first, their score would drop. That would be unusual behaviour for a measure of vocabulary.
  - thinkpad20 13y ago
    I don't think that's what the poster meant. By "average unique words per song" I take it to mean, within each song words are only counted once, but across songs, words can be counted multiple times. So if song A had the words "I like cats" and song B had the words "I like dogs", then the average unique word count would be ((3 + 3) / 2) = 3, not ((3 + 1)/2) = 2.
    - jeorgun 13y ago
      That's definitely one solution, but it still wouldn't quite capture it. As an extreme example, if rapper A produced 100 songs, each with exactly the same lyrics, they should surely be penalized compared with rapper B producing 100 songs with no shared words— even if rapper A's average unique-words-per-song is higher than rapper B's.