4 ms·
Some discussion around applying Ngrams to your investigation: It's really easy to make ngrams say something wrong by accident. I don't think these ones turned o
by zamadatix 3y ago
Some discussion around applying Ngrams to your investigation: It's really easy to make ngrams say something wrong by accident. I don't think these ones turned out too poorly for what the conclusions were trying trying to convey but for modern terms there are two things you want to keep an eye out for or adapt for:
1) Enable case insensitivity, particularly for things like "transistor" which are rarely capitalized or "integrated circuit" but even for "should be capitalized things" like "Moore's Law" vs "Moore's law".
2) (Especially on things spanning less than a century) check the impact of smoothing. It can "smooth out" the introduction of term to appear to take off years early or hide the true peaks/variabilities because they only lasted a couple of years instead of decades like the default smoothing might be better suited for.
3) (Not as applicable here) make sure the terms are popular enough you're not just comparing noise.
4) Adjust your range so the data is properly scaled. If your term was coined around the 1970s (and comparatively uncommon prior to that)then the 1870s shouldn't be midway through your graph.
5) If various case insensitive spellings create a messy set of graph lines adding in a less popular term like "exaflops" will cause them to be merged and you can extract that line.
6) Be careful comparing things that can be pluralized or otherwise modified with things that aren't. E.g. "Moore's Law" vs "Integrated Circuit[s]?"
Combining some of these, notes like "Here is a further ngram of the ‘End of Moore’s Law’ which has clearly been a live topic of discussion since the late 1990s." actually look to be a bit off when you switch the graph to clue in on the last 30 years, set smoothing to 0, and enable case sensitivity to catch "[Ee]nd of [Mm]oore's [Ll]aw". Case insensitivity changes the shape of the humps and 2019 ends up significantly higher than the peak in the early 2000s. Shrinking the x-axis and not smoothing out the peak to be wider than it is shows there is very little in the way of the end of Moore's Law in the late 90's, only really starting to take off in 99 and not peaking until 2003.
A similar story repeats for the opening graph, the gap between variants of "Moore's Law" and variants of "Integrated Circuit" is more than double the frequency, adding in that variants of "Integrate Circuits" are even more than either and counted separately.
Overall the points turn out right enough (a few years difference in when the phrase found its first wave doesn't impact the story, nor does how frequent the use of Moore's law is vs ICs as much as the general rise of one and decline of the other) but I always want to highlight how the friendly interface of Google's Ngram viewer can easily lead you to the wrong conclusion on the data.
I'd also like to point out the above tips are neither exhaustive or universally applicable to any type of search. Sometimes you want more smoothing, sometimes you do want an exact version of a phrase, sometimes you do want to show the term was used long before the modern meaning, and sometimes there are additional things that can bite you. E.g. one I was worried about here was "Moores Law" typos excluding the ' but it turned out to not be common at all. You really don't know until you check each possibility though.
- shermantanktop 3y agoPretty much every time I’ve seen an article that used Google ngrams as a secondary anecdote, it’s a bit of introductory fluff, and I pay zero attention to the integrity of the analysis. It’s the equivalent of a journalist writing “people are saying…”.
- zamadatix 3y agoOne example being once the phrase "sweet summer child" came up with someone asking where it originated and why it was popular all of the sudden. Some were insistent it was a common phrase in recent times prior to A Game of Thrones using it and others were saying it's always been a common thing and GoT was just the latest media but not related to its modern usage. A default ngrams search will show some decent activity in the 1800s then the phrase catching on a bit in the early 90s, staying steady through the late 90s, until it took off in the late 2000s. The phrase turns out to have definitely been said in the 1800s but maybe not with the frequency the graph might hint at initially because of the infrequency of recorded material at the time. Then unsmoothing things and shrinking the date range you'll see it really didn't take off until just after 1996 where it died down pretty quickly until 2011. I.e. what initially looked like something that had been a common phrase before and into popularity in recent times turned out to be something that had been previously mentioned but not really in use then lines up exactly with the release dates of the book and movies for A Game of Thrones instead of something which slightly preceded it. This shows how easy it is to end up with two opposite conclusions by using Ngram Viewer slightly differently. Alright, enough rambling about the hazards of Ngram Viewer in a post focused on Moore's Law :p.
- pitherpather 3y ago> This shows how easy it is to end up with two opposite conclusions by using Ngram Viewer slightly differently. Hence the commonly-encountered phrase: You believed a naive ngram reference? Oh my sweet summer child.
- deleted 3y ago[deleted]
- iraqmtpizza 3y ago