3 ms·
Why shouldn't these records be available for free, even better yet accessible thru an API? Would the 'bad apples' still be able to operate in any area of gover
by rmason 12y ago
Why shouldn't these records be available for free, even better yet accessible thru an API? Would the 'bad apples' still be able to operate in any area of government if there was true transparency?
- jbattle 12y agoI'm guessing because the records aren't all digital and there's some clerical work needed to either digitize them or maybe do some redaction or editing to make them suitable for release (e.g. I'd hope they would strip out social security numbers if collected)
- cmdrfred 12y agoThere should be a way to do that automagicly no? Anytime you see a grouping of digits like ###-###-#### black it out? I guess it all depends on how good the OCR software is.
- jdmichal 12y agoI'm not convinced. All it would take is OCR to read one of the zeroes as the letter 'O' instead, and suddenly there's your SSN out in the open. That's at least a civil lawsuit, maybe a criminal one, depending on context. (Healthcare, banking, etc. have strict laws in the US regarding the release of personal information.) In other words, the risks avoided by paying the best heuristic computer known to man -- a human -- far outweighs the cost of the resource.
- omg_ketchup 12y agoShouldn't the office releasing the SSNs (the police department) to a member of the general public (programmers in question) be held accountable for releasing sensitive information, not the person that bought it?
- jdmichal 12y agoUnsure how you derived that interpretation. Yes, of course the releasing agency would be the ones liable.
- Lawtonfogle 12y agoNope. First the dashes may not be there. It could also be just the last 4 of the SSN (which is still bad). Also other info could be present that is not easy to identify as private. For example, say the cop was caught using extreme racial slurs against a man who was being arrested after giving his child an STD via rape. It would be extremely difficult to redact enough so that you do not release any info that could identify the victim. Even as a person who knows a thing or two about looking stuff up on the internet it would be hard to redact just enough to protect the identity of the victim while not redacting too much about the police misconduct. Doing so automatically would be a strong AI sort of problem.
- taco_emoji 12y agoIt's more than just series of numbers that need to be redacted. It's why those two guys themselves have not published this data publicly: > COP considered dumping the raw data online for anyone to view, but decided against it because it would involve the publication of sensitive personal details—things like mental illness, drug addiction, or cases involving minors.
- rmxt 12y agoWhy aren't they all digital now, in 2015? Here's some brainstorming: have a precinct or two run a trial where all field notes must be taken on a department tablet. Have a field for sensitive info. This field is automatically redacted in public releases. No paper notes. Any paper must be scanned or photographed and OCR'd. Originals are kept, but digital copies must be made. My personal thoughts: I'd venture to say that all info would eventually get tossed into the "sensitive field" as a matter of a slippery slope. I also think that there's a conflict of interest in keeping notes that are "bad" over "good" (easily retrievable, shareable, and legible). It's self-serving to have physical paper notes. They can easily be "lost", "destroyed", and they only exist in a physical form: an oversight official has to come to the physical precinct house to retrieve the records.
- Lawtonfogle 12y ago>Why aren't they all digital now, in 2015? Last time I was in a county office, the application they used was extremely dated. It looked well over 15 years old. Many governments work with really dated software with a skeleton IT crew that can barely keep even that working, much less rewrite it.
- VLM 12y ago"Why aren't they all digital now" Because its easier to cover up non-digital records. Just like sometimes the boss wants to discuss things in person or over the phone rather than in storable text format. Something no one is discussing about this supposed treasure trove of SS numbers: 1) once its out there, its out there, its like pretending your email address is secret after the spammers are already using it. And after 50 zillion examples of incompetent security, all our numbers are public to absolutely everyone except the rawest of general public (like my neighbor). Anyone I wouldn't want to have access to it, already has it. 2) as per above, they paid $22500 or whatever (hopefully prorated with the low rate or providing the docs) because someone from the city is supposed to be cleaning that data out of the file already. Otherwise that shady guy who owns the money laundering err I mean upstanding and ethical payday loan biz down the street would have already purchased a list of police reports, and done his thing with the SS numbers contained. 3) the easy and cheap part is editing out 123-45-6789 regex numbers. The hard part is in the rare situation of judicial involvement (those people rarely do their job, as per the article's claims) there might be a gag order as part of an out of court settlement or similar.
- Lawtonfogle 12y agoQuite a few reasons. 1. A lot of the paperwork is all over the place. It has to be gathered, which takes time. 2. Same is true for the digital info, all over multiple databases. It takes a programmer or few to focus on pulling it (and getting all the necessary permission to do so). 3. Everything, once compiled, has to be reviewed to ensure it is on target and that any necessary info has to be redacted. The problem is that no one wants to take responsibility for a massive data dump without checking ever single word/number. This is because, while column X or form Y should only have publically available information on it, the people who processed the forms or input the data may have put information that would need to be redacted (SSNs appear in all sorts of places where they shouldn't). So the programmers will not give any sort of guarantee for the data pulls and the people pulling the physical files won't give any sort of similar guarantee for the physical files. This means the FOIA office (or what ever they are called there) has to do a line by line, word by word, number by number redaction. 4. To make this worse, many times the staff is not fully trained and has outdated equipment. FOIA has to print out files, use a black marker on both sides of the paper, and then scan it back in and print out again (and this is per policy, so the worker can be disciplined if they do anything else, even if it is more efficient). The programmers often are tasked to go through largely undocumented decade old (or older) systems. 5. Even without the political nature of the work, so even for a completely benign info request, the work is often time of low priority and for most of the staff involved it is on top of their normal day duties. Often times the staff is already underpaid and overworked, and with the info request being of low priority, it gets largely pushed aside. It would be ideal to have it all accessible to an API, but having this done would take multiple miracles that would be better spent on obtaining world peace and ending world hunger.