5 ms·

Why aren't all non-classified, non-sensitive public data actually public by default? The time, effort and money they spent fighting the FOIA lawsuit - wouldn't

by akudha 2y ago

Why aren't all non-classified, non-sensitive public data actually public by default? The time, effort and money they spent fighting the FOIA lawsuit - wouldn't it just be easier and cheaper to just honor the request?

remexre 2y ago
The first reasonable concern that springs to mind is that correlating a few fields that are individually non-identifying in a dataset can lead to deanonymizing people; in principle, the FOIA process gives the organization being requested time to think about what needs to be masked to protect privacy.
- JadeNB 2y ago
  > in principle, the FOIA process gives the organization being requested time to think about what needs to be masked to protect privacy. Although of course that's a lot more in principle than in practice! Like rolling your own crypto, I think experience shows that, against determined de-anonymizers, there is basically nothing you can do to preserve anonymity except to severely limit the information, and the only way definite feedback you get is if you don't succeed and someone discloses the de-anonymized information that they were able to reconstruct.
- Schiendelman 2y ago
  If government has access to all those fields the data isn't private. Your argument makes me think we really SHOULD make all government data public. Then people would have an incentive to not let governments have so much of their private data.
  - remexre 2y ago
    I'm essentially comfortable with the government knowing what banks I've banked at, what homes I've lived in, and where and when I've been arrested. I'm much less comfortable with those being in a public database anyone can query.
    - Schiendelman 2y ago
      Why are you comfortable with the government knowing what banks you have banked at? Also, I hope you don't mean the federal government should know if you've been arrested. I don't think that should be available to them at all.
gleenn 2y ago
Providing public APIs to such information sounds great and I think it should be done but is also probably realistically expensive and peone to security problems. The government seems to have a very difficult time in general running all sorts of web services, it's a nightmare in some jurisdictions to even pay the government money for e.g. tickets... I had a hell of a time paying a ticket in San Diego and had to visit the the courthouse multiple times and file a paper form because my paymentment became "late" when it never showed up in their online database weeks later. I was TRYING to pay them and even going to suffer through the "convenience" fees because they outsourced the website to some crappy company and they still couldn't get it right.
- k1t 2y ago
  Nobody said anything about providing a public API. Just provide a DB dump to this one guy.
- akudha 2y ago
  Expensive compared to what? Paying a bunch of lawyers and dragging a lawsuit for five years? Don’t lawyers, judges, court personnel’s time cost money? They even refused to provide the database schema - how is providing just the schema expensive? Or hard to do?
  - marsovo 2y ago
    As I understand it, the trial was about whether or not providing the database schema is a security risk. > Your request seeks a copy of tables or columns within each table of CANVAS. The dissemination of these pieces of network information could jeopardize the security of the systems of the City of Chicago. Related: https://news.ycombinator.com/item?id=43175628 https://news.ycombinator.com/item?id=43175628
kube-system 2y ago
Determining whether or not something is sensitive is not necessarily a trivial activity, especially if you’re asking it to be done for everything that every employee creates everyday.
- vincnetas 2y ago
  Thats why you do design o DB and decide if this data is really needed to be stored, for how long, now sensitive it is etc. You don't just add data to DB as you go (well ideally).
  - kube-system 2y ago
    If only it were that easy. Only a small amount of government data is in a database, if even electronic at all. A lot of it is on paper, in Microsoft word, in emails, in a voicemail, etc. Nobody is going to write a DB schema or call a DBA before taking notes in a notebook… but that’s government data too. Example: https://home.nps.gov/katm/blogs/images/IMG_3229.jpg https://home.nps.gov/katm/blogs/images/IMG_3229.jpg
    - vincnetas 2y ago
      True, but in this case we are talking about providing DB dumps to citizens.
      - kube-system 2y ago
        That is not how I interpreted “all non-classified, non-sensitive public data”
        vincnetas 2y ago
        true. i scope of grand parent comment. i was still thinking about TFA when i commented.
potato3732842 2y ago
Because it would significantly de-legitimize the government (particularly state and local) if any and every youtube talking head and blogger could pour over it and find the objectionable things that is supported by few and only tolerated by many because of poor awareness. And that's before you get into the spurious correlations and conspiracy stuff.
- Schiendelman 2y ago
  Or it would make us look for ways to have government do fewer things.
  - warmcompress 2y ago
    About as believable as DOGE "saving" the government $105 billion.
    - Schiendelman 2y ago
      I'm not really interested in any of that. But I am interested in us having privacy from government. The last thing I want is someone like Musk getting access to my personal information, and the only way to really prevent that is to not collect it in the first place.
- idiotsecant 2y ago
  It's exhausting the number of conversations online that assume that everything in the government is nefarious, inefficient, and focused on steeling ur freedums. I have worked in private industry, I've worked in government, and I've worked in the murky world in between. By and large government employees and thoughtful, genuinely want to make things work, and are faithful stewards of the public good. They usually make less money than they could get in private industry, the job security is absolutely no longer there, and the benefits are getting worse and worse each year. They're just people doing their best. Try to be a little less breathless with your rhetoric.
  - 2y ago

by akudha 2y ago

remexre 2y ago

The first reasonable concern that springs to mind is that correlating a few fields that are individually non-identifying in a dataset can lead to deanonymizing people; in principle, the FOIA process gives the organization being requested time to think about what needs to be masked to protect privacy.

JadeNB 2y ago
> in principle, the FOIA process gives the organization being requested time to think about what needs to be masked to protect privacy. Although of course that's a lot more in principle than in practice! Like rolling your own crypto, I think experience shows that, against determined de-anonymizers, there is basically nothing you can do to preserve anonymity except to severely limit the information, and the only way definite feedback you get is if you don't succeed and someone discloses the de-anonymized information that they were able to reconstruct.
Schiendelman 2y ago
If government has access to all those fields the data isn't private. Your argument makes me think we really SHOULD make all government data public. Then people would have an incentive to not let governments have so much of their private data.
- remexre 2y ago
  I'm essentially comfortable with the government knowing what banks I've banked at, what homes I've lived in, and where and when I've been arrested. I'm much less comfortable with those being in a public database anyone can query.
  - Schiendelman 2y ago
    Why are you comfortable with the government knowing what banks you have banked at? Also, I hope you don't mean the federal government should know if you've been arrested. I don't think that should be available to them at all.

gleenn 2y ago

Providing public APIs to such information sounds great and I think it should be done but is also probably realistically expensive and peone to security problems. The government seems to have a very difficult time in general running all sorts of web services, it's a nightmare in some jurisdictions to even pay the government money for e.g. tickets... I had a hell of a time paying a ticket in San Diego and had to visit the the courthouse multiple times and file a paper form because my paymentment became "late" when it never showed up in their online database weeks later. I was TRYING to pay them and even going to suffer through the "convenience" fees because they outsourced the website to some crappy company and they still couldn't get it right.

k1t 2y ago
Nobody said anything about providing a public API. Just provide a DB dump to this one guy.
akudha 2y ago
Expensive compared to what? Paying a bunch of lawyers and dragging a lawsuit for five years? Don’t lawyers, judges, court personnel’s time cost money? They even refused to provide the database schema - how is providing just the schema expensive? Or hard to do?
- marsovo 2y ago
  As I understand it, the trial was about whether or not providing the database schema is a security risk. > Your request seeks a copy of tables or columns within each table of CANVAS. The dissemination of these pieces of network information could jeopardize the security of the systems of the City of Chicago. Related: https://news.ycombinator.com/item?id=43175628 https://news.ycombinator.com/item?id=43175628

kube-system 2y ago

Determining whether or not something is sensitive is not necessarily a trivial activity, especially if you’re asking it to be done for everything that every employee creates everyday.

vincnetas 2y ago
Thats why you do design o DB and decide if this data is really needed to be stored, for how long, now sensitive it is etc. You don't just add data to DB as you go (well ideally).
- kube-system 2y ago
  If only it were that easy. Only a small amount of government data is in a database, if even electronic at all. A lot of it is on paper, in Microsoft word, in emails, in a voicemail, etc. Nobody is going to write a DB schema or call a DBA before taking notes in a notebook… but that’s government data too. Example: https://home.nps.gov/katm/blogs/images/IMG_3229.jpg https://home.nps.gov/katm/blogs/images/IMG_3229.jpg
  - vincnetas 2y ago
    True, but in this case we are talking about providing DB dumps to citizens.
    - kube-system 2y ago
      That is not how I interpreted “all non-classified, non-sensitive public data”
      - vincnetas 2y ago
        true. i scope of grand parent comment. i was still thinking about TFA when i commented.

potato3732842 2y ago

Because it would significantly de-legitimize the government (particularly state and local) if any and every youtube talking head and blogger could pour over it and find the objectionable things that is supported by few and only tolerated by many because of poor awareness. And that's before you get into the spurious correlations and conspiracy stuff.

Schiendelman 2y ago
Or it would make us look for ways to have government do fewer things.
- warmcompress 2y ago
  About as believable as DOGE "saving" the government $105 billion.
  - Schiendelman 2y ago
    I'm not really interested in any of that. But I am interested in us having privacy from government. The last thing I want is someone like Musk getting access to my personal information, and the only way to really prevent that is to not collect it in the first place.
idiotsecant 2y ago
It's exhausting the number of conversations online that assume that everything in the government is nefarious, inefficient, and focused on steeling ur freedums. I have worked in private industry, I've worked in government, and I've worked in the murky world in between. By and large government employees and thoughtful, genuinely want to make things work, and are faithful stewards of the public good. They usually make less money than they could get in private industry, the job security is absolutely no longer there, and the benefits are getting worse and worse each year. They're just people doing their best. Try to be a little less breathless with your rhetoric.
- 2y ago