4 ms·

> Using date range commands, the author claimed to compare searches for five months in 2019 to the same five months in 2020 — pre-pandemic to mid-pandemic. They

by Cenk 5y ago

> Using date range commands, the author claimed to compare searches for five months in 2019 to the same five months in 2020 — pre-pandemic to mid-pandemic. They did so not by turning to Google Trends data — the method promoted by the scholars they cited — but by inputting their search phrases with date delimiters using Google search. > They reported the number of hits Google displays at the top of the page as the number of searches made for that search string. >These numbers were exceptionally high because, well, the search phrases were not enclosed in quotation marks. All this in an article arguing for the value of tech-enabled “rapid response” research. A culture of “move fast and break things” is common in Silicon Valley, but academics typically work a bit more slowly and carefully to avoid these kinds of errors.

tgv 5y ago
Even then, if it had been executed flawlessly, you can only expect that search term frequency and violence remain correlated ceteris paribus, i.e., when all else is the same. The two time periods are clearly different, so how the hell did it even get published?
wumpus 5y ago
The number of hits reported by web-scale engines like Google and Bing are fabrications. When I was the CTO of a full-web search engine startup, I'd get an API access request every couple of months by someone wanting to use hit counts for an academic purpose. I would always them no and why; often they wanted to argue with me. http://searchengineland.com/why-google-cant-count-results-properly-53559 http://searchengineland.com/why-google-cant-count-results-pr... http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/google-result-counts-are-a-meaningless-metric.html http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/goo...
- azinman2 5y ago
  This was the case when I was at Google in 2008; I’m sure it’s even further fabricated now.
  - throwaway3699 5y ago
    Can you elaborate on what you saw?
    - azinman2 5y ago
      That it’s not based on reality at all. That would likely be far to expensive computationally for any arbitrary query to give an accurate number that isn’t particularly valuable outside of degrees of orders of magnitude, and even that is almost never looked at.
  - Shish2k 5y ago
    I’m curious, what are they based on, and is there any reason to have a number there beyond “people expect a number”?
    - azinman2 5y ago
      It’s basically a random number. Don’t extract any meaning from it.
- lakis 5y ago
  I've worked to Inktomi/Yahoo search ~2000-~2015. Any query that is accurate across the whole index is going to be very very expensive. In order to return fast results, there are caches and precomputed results in almost every level of a web search query. But an accurate count implies that you will bypass all the caches and count every single document for every single query term. That's very very expensive in both CPU and memory. We had special internal keywords that disabled caches and produced accurate results. All of them came with special warnings that querying too fast with these special options could bring down the whole search engine. A constant worry was the new employee that knew very little, learnt these keywords and then ops were dealing with crashes across ten of thousands of machines due to out of memory. As to the fabrication. The number are not random numbers. They are estimates based on what we think is the best approximation of the real numbers. We had done internal research to compute the best approximation function. Also, it's very difficult to figure out what is real and what is fake from the outside. I was involved in https://www.nytimes.com/2005/08/10/business/worldbusiness/yahoo-says-its-search-index-is-bigger-than-googles.html https://www.nytimes.com/2005/08/10/business/worldbusiness/ya... Yahoo was 100% correct on their claims. I run the queries myself. But external researchers could not verify the claims because all external queries were hitting caches. So they were trying unique queries in order to estimate the index size which end up having a lot of problems.
croes 5y ago
By that logic, every site citing the paper makes the situation worse.

by Cenk 5y ago

tgv 5y ago

Even then, if it had been executed flawlessly, you can only expect that search term frequency and violence remain correlated ceteris paribus, i.e., when all else is the same. The two time periods are clearly different, so how the hell did it even get published?

wumpus 5y ago

The number of hits reported by web-scale engines like Google and Bing are fabrications. When I was the CTO of a full-web search engine startup, I'd get an API access request every couple of months by someone wanting to use hit counts for an academic purpose. I would always them no and why; often they wanted to argue with me. http://searchengineland.com/why-google-cant-count-results-properly-53559 http://searchengineland.com/why-google-cant-count-results-pr... http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/google-result-counts-are-a-meaningless-metric.html http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/goo...

azinman2 5y ago
This was the case when I was at Google in 2008; I’m sure it’s even further fabricated now.
- throwaway3699 5y ago
  Can you elaborate on what you saw?
  - azinman2 5y ago
    That it’s not based on reality at all. That would likely be far to expensive computationally for any arbitrary query to give an accurate number that isn’t particularly valuable outside of degrees of orders of magnitude, and even that is almost never looked at.
- Shish2k 5y ago
  I’m curious, what are they based on, and is there any reason to have a number there beyond “people expect a number”?
  - azinman2 5y ago
    It’s basically a random number. Don’t extract any meaning from it.
lakis 5y ago
I've worked to Inktomi/Yahoo search ~2000-~2015. Any query that is accurate across the whole index is going to be very very expensive. In order to return fast results, there are caches and precomputed results in almost every level of a web search query. But an accurate count implies that you will bypass all the caches and count every single document for every single query term. That's very very expensive in both CPU and memory. We had special internal keywords that disabled caches and produced accurate results. All of them came with special warnings that querying too fast with these special options could bring down the whole search engine. A constant worry was the new employee that knew very little, learnt these keywords and then ops were dealing with crashes across ten of thousands of machines due to out of memory. As to the fabrication. The number are not random numbers. They are estimates based on what we think is the best approximation of the real numbers. We had done internal research to compute the best approximation function. Also, it's very difficult to figure out what is real and what is fake from the outside. I was involved in https://www.nytimes.com/2005/08/10/business/worldbusiness/yahoo-says-its-search-index-is-bigger-than-googles.html https://www.nytimes.com/2005/08/10/business/worldbusiness/ya... Yahoo was 100% correct on their claims. I run the queries myself. But external researchers could not verify the claims because all external queries were hitting caches. So they were trying unique queries in order to estimate the index size which end up having a lot of problems.

croes 5y ago

By that logic, every site citing the paper makes the situation worse.