5 ms·

The wayback machine has been essential through COVID as many government just publish the numbers and data "of the day" and the only way to compare to the day be

The wayback machine has been essential through COVID as many government just publish the numbers and data "of the day" and the only way to compare to the day before is to look at IA.

makomk 6y ago
Governments aren't the only ones overwriting old information with new either. The BBC has developed an annoying habit of completely overwriting old articles about Covid-19 with new, semi-related ones, meaning that the only way to see what they were saying earlier in the pandemic is through sites like the Internet Archive. There's also at least one outright correction that only seems to exist on the Internet Archive now: https://web.archive.org/web/20200428214306/https://www.bbc.co.uk/news/health-51943612 https://web.archive.org/web/20200428214306/https://www.bbc.c... "Correction 25 April 2020: An earlier version of this article incorrectly said that France had conducted just under 140,000 tests a day by 21 April. The figure of just under 140,000 refers instead to the number of tests it had carried out weekly."
- endgame 6y ago
  Here's another one: "some of our old tweets were wrong so we just quietly deleted them lol": https://twitter.com/voxdotcom/status/1242537366620966912 https://twitter.com/voxdotcom/status/1242537366620966912
  - airstrike 6y ago
    Related https://twitter.com/nyt_diff https://twitter.com/nyt_diff
  - gxon 6y ago
    Twitter ought to have some kind of strikethrough feature. Allow users to mark that they no longer stand behind a tweet without completely deleting it.
    - tommica 6y ago
      What would be the incentive to use that feature over just deleting the tweet?
      - TeMPOraL 6y ago
        There wouldn't be, therefore tweets shouldn't be deletable after a set period of time.
        pjc50 6y ago
        Given that a British "journalist" lost her house to legal fees after libelling someone and refusing to take the tweet down, I think there are reasonable reasons to want to take down libellous, abusive, dangerous, or wrong tweets without having to sue Twitter. ("Journalist" is in quotes because, while that was her job title, she specialized in abuse, libel, smears, and calls to murder, and I'm glad she lost the libel case)
        gxon 6y ago
        I'm not sure I like this for most users who don't have any kind of following or well developed sense of what is and is not appropriate to post. I do like this a lot for high profile accounts. It seems fair that if you're verified and have a very large number of followers, you are operating at a privileged level of impact and should lose your right to delete tweets any tweets.
      - retsibsi 6y ago
        Integrity or the appearance thereof -- showing that you're not trying to cover up your mistakes.
  - underwater 6y ago
    This is necessary on Twitter. It's easy for fake or incorrect news to spread, you can't edit it and updates/self-replies are often hidden.
    - dwaltrip 6y ago
      Sure, but they should also own up to their mistake. They flat out said "It won't be a deadly epidemic".
      - fireattack 6y ago
        They made a tweet to explicitly say they deleted it because it's wrong. How is that not "owning up"?
        TeMPOraL 6y ago
        Properly owning up would include keeping the mistake up, so that people reading the apology would actually know what they're owning up to. Without the original Tweet, they might be apologizing for mistyping "2+2=4" as "2+2=5", for all you know.
pier25 6y ago
Wikipedia is a good source too, at least for the countries I've been following.
simonw 6y ago
Do you have any examples of government sites that are doing this? I have a side-hobby of setting up scrapers which pull scraped data into a git repository, precisely for this kind of thing. I'd be happy to set a few up. Some of my posts about this technique (which I call "git scraping"): https://simonwillison.net/tags/gitscraping/ https://simonwillison.net/tags/gitscraping/
- Endlessly 6y ago
  Not sure if you’re asking for: (1) US coronavirus data in general, (2) examples of sources that do not log prior days data, or (3) source that “edit” prior data without noting the edits. (4) something else If (1) this page in the table under the column “sources” links to where the data came from: https://www.worldometers.info/coronavirus/country/us/ https://www.worldometers.info/coronavirus/country/us/
  - simonw 6y ago
    2 and 3.
- jeromegv 6y ago
  Government of Ontario has been doing it. They now have some APIs that link to the number of cases per day, but for the first month of the pandemic, they were literally overwriting the page every day and when you look at github there was tons of researchers that came up with their own parser in order to save this data before it is deleted. I've also been running bash scripts every day to save an archive of some of those pages https://github.com/jeromegv/covid_data https://github.com/jeromegv/covid_data
dublinben 6y ago
For the US at least, the COVID Tracking Project records multiple screenshots per day of the official websites where those numbers are published. https://covidtracking.com/ https://covidtracking.com/

The wayback machine has been essential through COVID as many government just publish the numbers and data "of the day" and the only way to compare to the day before is to look at IA.

makomk 6y ago

Governments aren't the only ones overwriting old information with new either. The BBC has developed an annoying habit of completely overwriting old articles about Covid-19 with new, semi-related ones, meaning that the only way to see what they were saying earlier in the pandemic is through sites like the Internet Archive. There's also at least one outright correction that only seems to exist on the Internet Archive now: https://web.archive.org/web/20200428214306/https://www.bbc.co.uk/news/health-51943612 https://web.archive.org/web/20200428214306/https://www.bbc.c... "Correction 25 April 2020: An earlier version of this article incorrectly said that France had conducted just under 140,000 tests a day by 21 April. The figure of just under 140,000 refers instead to the number of tests it had carried out weekly."

endgame 6y ago
Here's another one: "some of our old tweets were wrong so we just quietly deleted them lol": https://twitter.com/voxdotcom/status/1242537366620966912 https://twitter.com/voxdotcom/status/1242537366620966912
- airstrike 6y ago
  Related https://twitter.com/nyt_diff https://twitter.com/nyt_diff
- gxon 6y ago
  Twitter ought to have some kind of strikethrough feature. Allow users to mark that they no longer stand behind a tweet without completely deleting it.
  - tommica 6y ago
    What would be the incentive to use that feature over just deleting the tweet?
    - TeMPOraL 6y ago
      There wouldn't be, therefore tweets shouldn't be deletable after a set period of time.
      - pjc50 6y ago
        Given that a British "journalist" lost her house to legal fees after libelling someone and refusing to take the tweet down, I think there are reasonable reasons to want to take down libellous, abusive, dangerous, or wrong tweets without having to sue Twitter. ("Journalist" is in quotes because, while that was her job title, she specialized in abuse, libel, smears, and calls to murder, and I'm glad she lost the libel case)
      - gxon 6y ago
        I'm not sure I like this for most users who don't have any kind of following or well developed sense of what is and is not appropriate to post. I do like this a lot for high profile accounts. It seems fair that if you're verified and have a very large number of followers, you are operating at a privileged level of impact and should lose your right to delete tweets any tweets.
    - retsibsi 6y ago
      Integrity or the appearance thereof -- showing that you're not trying to cover up your mistakes.
- underwater 6y ago
  This is necessary on Twitter. It's easy for fake or incorrect news to spread, you can't edit it and updates/self-replies are often hidden.
  - dwaltrip 6y ago
    Sure, but they should also own up to their mistake. They flat out said "It won't be a deadly epidemic".
    - fireattack 6y ago
      They made a tweet to explicitly say they deleted it because it's wrong. How is that not "owning up"?
      - TeMPOraL 6y ago
        Properly owning up would include keeping the mistake up, so that people reading the apology would actually know what they're owning up to. Without the original Tweet, they might be apologizing for mistyping "2+2=4" as "2+2=5", for all you know.

pier25 6y ago

Wikipedia is a good source too, at least for the countries I've been following.

simonw 6y ago

Do you have any examples of government sites that are doing this? I have a side-hobby of setting up scrapers which pull scraped data into a git repository, precisely for this kind of thing. I'd be happy to set a few up. Some of my posts about this technique (which I call "git scraping"): https://simonwillison.net/tags/gitscraping/ https://simonwillison.net/tags/gitscraping/

Endlessly 6y ago
Not sure if you’re asking for: (1) US coronavirus data in general, (2) examples of sources that do not log prior days data, or (3) source that “edit” prior data without noting the edits. (4) something else If (1) this page in the table under the column “sources” links to where the data came from: https://www.worldometers.info/coronavirus/country/us/ https://www.worldometers.info/coronavirus/country/us/
- simonw 6y ago
  2 and 3.
jeromegv 6y ago
Government of Ontario has been doing it. They now have some APIs that link to the number of cases per day, but for the first month of the pandemic, they were literally overwriting the page every day and when you look at github there was tons of researchers that came up with their own parser in order to save this data before it is deleted. I've also been running bash scripts every day to save an archive of some of those pages https://github.com/jeromegv/covid_data https://github.com/jeromegv/covid_data

dublinben 6y ago

For the US at least, the COVID Tracking Project records multiple screenshots per day of the official websites where those numbers are published. https://covidtracking.com/ https://covidtracking.com/