4 ms·

Fair. Can I ask why exactly you think my take is "very weird"? Your original post was exceptionally dismissive, without explanation, and your comment on FOI w

• stevage 2y ago
  Look, for starters, the stuff we're talking about covers a pretty broad spectrum. Your framing of the question about "intentionally missing" stuff suggests that you're interested in transparency-style data: data that gives you insight into the operations of the government body. And yes, if you are looking for data that might reflect poorly on the organisation, an open data portal is generally not the place to go for it. This HN item for instance, is not about that kind of data. The datasets in question tell you about the transport network, the services, the patronage, the history, all kinds of interesting stuff. So I find it "weird" that you would respond to a good-faith effort of sharing tons of information about a public transport network with this hostile approach of disparaging open data portals, and advocating instead an approach which is extremely resource-intensive for government bodies, when it's completely uncalled for. Yeah, if you want to investigate a government cover-up, or shine light on some terrible mismanagement of resources, go for your life and submit FOI requests. Your mention of having filed thousands of FOI requests suggests you have consumed many tens of thousands of hours of public servants' time, and I really hope the results justify it.
  • chaps 2y ago
    Lemme tell you a story. Years ago during the pandemic early days, a harvard epidemiology student asked me to proof-read his paper that argued that covid-19 killed more white people than any other race. The dataset he used was the Cook County Medical Examiner dataset. There was a column in there for the race information. If you're curious how it's populated, I can share with you the information. Previously, I'd FOIA'd the data and received many more columns of information including the names of the individuals who'd died which showed a very clear pattern that the race information on the open data portal was not always accurate for Hispanic-origin names. The details are complicated, and I'm happy to explain my fact checking methods, but the Harvard student's analysis was just flat wrong because it made assumptions that the race data was correct. It was not. Their response was initially along the lines of, "even if it's 50% it's still going to be true". It ended up being more like 80%, showing that people with Hispanic-origin names were significantly more likely to die of COVID-19. If you think your audience isn't academics at mega institutions who believe that open data is 100% accurate data, then you've made many incorrect assumptions and I encourage you to reconsider.
    • stevage 2y ago
      >your audience "my audience"? What makes you think I have an audience?
      • chaps 2y ago
        I hope you're a nicer person in-person, too.