7 ms·

I'm curious, how did they get access to location data and Baidu search volumes for just Wuhan?

andrewtbham 6y ago
Search query data Baidu’s database (http://index.baidu.com/ http://index.baidu.com/) contains logs of web and mobile search query volume in China. User confidentiality is maintained, since only the relative term frequency data is available. We obtained daily data for symptom-related searches likely associated with COVID-19 illness in Wuhan from April 2017 to May 2020. We extracted the relative search volumes of the terms “cough” and “diarrhea” using WebPlotDigitizer, v4.215.
- dillondoyle 6y ago
  wow just searching 'cough' on google trends seems to be pretty accurate just birds-eye view would be cool to read similar research using US data, and the implications on policymaking. https://trends.google.com/trends/explore?date=today%203-m&geo=US&q=cough,covid%20cough,fever https://trends.google.com/trends/explore?date=today%203-m&ge...
  - starpilot 6y ago
    This always goes up during winter, you also have to control for all the media attention that COVID-19 is getting. People are searching just to be safe even if their symptoms are weak. It might not reflect growing infections, just growing fear.
    - dillondoyle 6y ago
      The spike in the timeframe of the link I put is in mid March and the decreasing rapidly. Plus I think it's also cool at least intuitively the 'trending' states make sense - though I have no idea if they are weighted by population or not.
  - mellosouls 6y ago
    It's interesting - tho I've just plugged in China and various periods down to the last 5 years, and there's not an obvious jump beyond the seasonal ones until after December this year.
    - pstrateman 6y ago
      google in china is less than 5% of the search market
      - mellosouls 6y ago
        yeah, I considered that - but its still presumably a large enough sample for the phrases listed in the parent comment to stand out.
- hef19898 6y ago
  As interesting as this is, but my understanding was that the genetic evolution / development of the virus can be used to determine when it started spreading. Or jumped from animal to human. Or did I understand that wrong? Not saying this study is wrong (no way for me to tell), it just seems a very indirect, and thus error prone, way to look at it.
  - mellosouls 6y ago
    I've asked that question as a separate thread above, I think its an important one, at least for lay people like me. There is a suggested reason for the discrepancy (Chinese dishonesty about timelines) tho not an answer (yet) from the researchers who've commented elsewhere here or others with a strictly scientific explanation. https://news.ycombinator.com/item?id=23460742 https://news.ycombinator.com/item?id=23460742
tindjinn 6y ago
Well, considering the all-seeing Chinese government practically owns Baidu and most other Chinese brands it is not very surprising that they're able to track individual users across multiple platforms. Location data is merely one tiny point on the vast graph of data they have access to.
- Dahoon 6y ago
  But this isn't PRC but Harvard.
- beervirus 6y ago
  It's a little surprising the PRC would let this unflattering data be available.
  - adventured 6y ago
    There are a lot of points of data to scrub in such a large system, it's not surprising that they would miss numerous of them in the wild. Watch to see if this one disappears or not.
  - cameldrv 6y ago
    The official line now is that the virus didn't start in the seafood market. The bats that carry the closest viruses to SARS-CoV-2 are located 1000 miles south of Wuhan, and were in hibernation in November. For the theory that the virus was transmitted from a bat to a human in Yunnan to work, it needs to have happened earlier in the year.
    - dificilis 6y ago
      If the virus jumped from bat to some other animal 50 years ago, then the 1000 miles of distance is not hard to explain, and doesn't tell us anything about the time-scale of initial human infection. We need to find the close relative of Covid19 (TMRCA < 1 year) in a non-human host.
bingdig 6y ago
The Chinese government and Chinese firms are often willing to share tons of data with researchers. One of my professors was able to get data on cell phone locations, metadata on all text messages sent and received, and occupations of cell phone holders from the largest telecom co in China for a paper on networks and job mobility.

Search query data Baidu’s database (http://index.baidu.com/ http://index.baidu.com/) contains logs of web and mobile search query volume in China. User confidentiality is maintained, since only the relative term frequency data is available. We obtained daily data for symptom-related searches likely associated with COVID-19 illness in Wuhan from April 2017 to May 2020. We extracted the relative search volumes of the terms “cough” and “diarrhea” using WebPlotDigitizer, v4.215.

dillondoyle 6y ago
wow just searching 'cough' on google trends seems to be pretty accurate just birds-eye view would be cool to read similar research using US data, and the implications on policymaking. https://trends.google.com/trends/explore?date=today%203-m&geo=US&q=cough,covid%20cough,fever https://trends.google.com/trends/explore?date=today%203-m&ge...
- starpilot 6y ago
  This always goes up during winter, you also have to control for all the media attention that COVID-19 is getting. People are searching just to be safe even if their symptoms are weak. It might not reflect growing infections, just growing fear.
  - dillondoyle 6y ago
    The spike in the timeframe of the link I put is in mid March and the decreasing rapidly. Plus I think it's also cool at least intuitively the 'trending' states make sense - though I have no idea if they are weighted by population or not.
- mellosouls 6y ago
  It's interesting - tho I've just plugged in China and various periods down to the last 5 years, and there's not an obvious jump beyond the seasonal ones until after December this year.
  - pstrateman 6y ago
    google in china is less than 5% of the search market
    - mellosouls 6y ago
      yeah, I considered that - but its still presumably a large enough sample for the phrases listed in the parent comment to stand out.
hef19898 6y ago
As interesting as this is, but my understanding was that the genetic evolution / development of the virus can be used to determine when it started spreading. Or jumped from animal to human. Or did I understand that wrong? Not saying this study is wrong (no way for me to tell), it just seems a very indirect, and thus error prone, way to look at it.
- mellosouls 6y ago
  I've asked that question as a separate thread above, I think its an important one, at least for lay people like me. There is a suggested reason for the discrepancy (Chinese dishonesty about timelines) tho not an answer (yet) from the researchers who've commented elsewhere here or others with a strictly scientific explanation. https://news.ycombinator.com/item?id=23460742 https://news.ycombinator.com/item?id=23460742

tindjinn 6y ago

Well, considering the all-seeing Chinese government practically owns Baidu and most other Chinese brands it is not very surprising that they're able to track individual users across multiple platforms. Location data is merely one tiny point on the vast graph of data they have access to.

Dahoon 6y ago
But this isn't PRC but Harvard.
beervirus 6y ago
It's a little surprising the PRC would let this unflattering data be available.
- adventured 6y ago
  There are a lot of points of data to scrub in such a large system, it's not surprising that they would miss numerous of them in the wild. Watch to see if this one disappears or not.
- cameldrv 6y ago
  The official line now is that the virus didn't start in the seafood market. The bats that carry the closest viruses to SARS-CoV-2 are located 1000 miles south of Wuhan, and were in hibernation in November. For the theory that the virus was transmitted from a bat to a human in Yunnan to work, it needs to have happened earlier in the year.
  - dificilis 6y ago
    If the virus jumped from bat to some other animal 50 years ago, then the 1000 miles of distance is not hard to explain, and doesn't tell us anything about the time-scale of initial human infection. We need to find the close relative of Covid19 (TMRCA < 1 year) in a non-human host.

bingdig 6y ago

The Chinese government and Chinese firms are often willing to share tons of data with researchers. One of my professors was able to get data on cell phone locations, metadata on all text messages sent and received, and occupations of cell phone holders from the largest telecom co in China for a paper on networks and job mobility.