5 ms·

The NYT probably produces 10x as many words as it did in 2015, and 50x as many as 2000, and 100x as many as 1980. These charts don't give any sense of if they a

The NYT probably produces 10x as many words as it did in 2015, and 50x as many as 2000, and 100x as many as 1980. These charts don't give any sense of if they are reporting frequency divided by total words or just frequency. I suppose the KKK and AIDS examples indicate the former but it isn't clear. Additionally, the y axis seems to be from 0 to 1: least popular years vs most popular year. This means every graph has a different Y scale. Very misleading, imho. Also, it is interesting to look at some of the words in Google ngram viewer for the same period.

itishappy 2y ago
Rescaling the axis is egregious. Normalizing the max to 1 is fine, we're investigating trends not comparing frequencies, but dropping the min to 0 is not. A frequency of zero is meaningful, and it's removal destroys pretty much anything these charts could tell us. If a word occurred 199 times in 1970 and 200 times in 2022 it can look exactly the same as a word that was never used in 1970 and spiked in 2022. Edit: To expand a bit, some of this info can be gleamed from context cues, such as usage hovering close to the bottom line for many charts implying that's the zero point. But half the charts look like "bigotry," "upsetting," "discrimination," or "duties." Are these strong trends or statistical noise? Who knows!
- hammock 2y ago
  Many of those questions can be answered. The charts show absolutely frequency. The max is not normalized across the words, each chart is 0 to 1. That’s how the data comes out of google ngram viewer when you search terms one at a time, which is what this is
  - itishappy 2y ago
    They can be answered by consulting a better chart! The charts show rise and fall, nothing more. That's enough info to see peaks, but we can't tell absolute anything without values. Zero would have been an easy value to include, but alas. Google ngram viewer gives significantly more information. It actually gives absolute measurements, to start. What's going on with "duties"? It dropped by a factor of two. Can't get that info from the article. https://books.google.com/ngrams/graph?content=duties&year_start=1970&year_end=2022&corpus=en https://books.google.com/ngrams/graph?content=duties&year_st...

itishappy 2y ago

Rescaling the axis is egregious. Normalizing the max to 1 is fine, we're investigating trends not comparing frequencies, but dropping the min to 0 is not. A frequency of zero is meaningful, and it's removal destroys pretty much anything these charts could tell us. If a word occurred 199 times in 1970 and 200 times in 2022 it can look exactly the same as a word that was never used in 1970 and spiked in 2022. Edit: To expand a bit, some of this info can be gleamed from context cues, such as usage hovering close to the bottom line for many charts implying that's the zero point. But half the charts look like "bigotry," "upsetting," "discrimination," or "duties." Are these strong trends or statistical noise? Who knows!

hammock 2y ago
Many of those questions can be answered. The charts show absolutely frequency. The max is not normalized across the words, each chart is 0 to 1. That’s how the data comes out of google ngram viewer when you search terms one at a time, which is what this is
- itishappy 2y ago
  They can be answered by consulting a better chart! The charts show rise and fall, nothing more. That's enough info to see peaks, but we can't tell absolute anything without values. Zero would have been an easy value to include, but alas. Google ngram viewer gives significantly more information. It actually gives absolute measurements, to start. What's going on with "duties"? It dropped by a factor of two. Can't get that info from the article. https://books.google.com/ngrams/graph?content=duties&year_start=1970&year_end=2022&corpus=en https://books.google.com/ngrams/graph?content=duties&year_st...