29 ms·

The City of Seattle Accidentally Gave Me 32M Emails for $40

516 points by bpchaps 8y ago

OrwellianChild 8y ago
I can't tell whether this is a testament to the incompetence of public IT operations or an indictment of public records keeping practice. Maybe both?
- bpchaps 8y ago
  Probably both. Or maybe just a story about a thing that happened.
- derblitzmann 8y ago
  Well, like most of Seattle's efforts, it gets fucked up, mostly because it is from Seattle. Note: I am a Seattle resident and I expect nothing less.
  - rlucas 8y ago
    Have you lived anywhere else, like, say, the northeast? Seattle is a shockingly well run municipality among American cities of size.
    - Fellshard 8y ago
      You haven't met the city council of the last several years, then...
      - zifnab06 8y ago
        Maybe I should run. I wonder if I'd stand a chance against sawant.
- btrettel 8y ago
  Also has a good example of hostile FOIA officers. I have filed about two dozen FOIA requests, and the vast majority were fine, though usually slow. Earlier this year one longstanding request of mine was rejected because they claimed the document I wanted was export controlled. Two months later I sent in an appeal where I showed that the document in question was not export controlled (I filed another FOIA with a separate agency to check), and I suggested that they lied to me. Never got an apology, though they seem to be processing the request for real now.
  - c3534l 8y ago
    Never attribute to malice that which is adequately explained by stupidity.
    - btrettel 8y ago
      I agree, but in this case stupidity can not explain their actions. The agency processing the FOIA request would get the export control status from the third party I contacted. The third party's software highlights export controlled documents with a red and highly noticeable statement. It would be difficult to believe they thought this was present when it was not. The request also had actually been transferred several times because no one believes they have the authority to release the document I requested. The other agencies had ample opportunity to reject the request for being export controlled, but none did. There are some other reasons that I will omit for brevity. This makes me think that the export controlled claim was a lie meant to kill the request. Most people would stop at what they told me, but I thought it was worth verifying.
    - anigbrowl 8y ago
      I used to abide by this until it dawned on me that malicious people often use stupid people as a cat's paw to conceal liability. A careless disregard for the truth is itself a form of dishonesty. This argument is laid out very effectively in philosopher Harry Frankfurt's delightful short book On Bullshit.
  - scoggs 8y ago
    I've requested thousands from all over the United States, big and small municipalities. The responses and individuals ranged from a small contingent of extremely professional, organized, helpful, and all around wonderful people to many and more abysmal, unorganized, uneducated, tech-illiterate, and downright incompetent folks. I can't remember how many times I'd point the person I was attempting to request the information from directly to their sample version or official space where they said they were the holders of said datas I was looking for which also provided the instructions to specifically call them to request the data. I know from my POV (as a tech worker) it may seem silly to expect anything like that from them but what I was asking for was akin to an excel spreadsheet full of information they absolutely do have and it required no legwork, no generation of new materials, no gathering of data from multiple ancient sources. It was all material / datas each individual municipality was making money, hand over fist, every month of every year -- the sample datas most municipalities had was evidence of that. I was basically asking them for the collection of the entire data, publicly traded info, and most were convinced they didn't have it. A product of laziness is what I'd chalk it up to because the individuals and municipalities that were awesome to work with and more than helpful seemed to take pride in their work and getting said data was easier than the majority of things involving interfacing with the government at any level usually winds up being like.
lsiebert 8y ago
So they accidentally included the first 256 characters for all the emails? yikes.
kristofferR 8y ago
Awesome work! I'm glad we have people willing to go through all the hassle, so we can keep our governments responsible.
daodedickinson 8y ago
It's clear they didn't have expertise to do it, and I'm tired of reading people that know way more looking down at others over it and assuming they don't want to comply. If they hiding something and malicious, the end result wouldn't have been to send way too much, but I don't see the author realizing this fast enough.
- dnbgfher 8y ago
  I'm not sure why you think the only malicious response would be to send over too little. It's a common enough scene in TV shows and the like where a malicious actor attempts to hide incriminating information in a sea of irrelevant information.
- sanotehu 8y ago
  Agreed. You're putting an overworked, underpaid public servant in a "damned if you do, damned if you don't" scenario. They complied with a far reaching request and got told their response was too far reaching? I'd quit my job if faced with a legal minefield like that, especially one not actually related to the job itself
  - setr 8y ago
    Presumably this should be considered overreaching should be considered an important though: if there’s an authorization process in play, then more information has been given out than the public servant was actually authorized to hand out. If me as a citizen starts receiving sensitive information despite only being authorized to receive insensitive info, that could easily become a significant security breach. In fact, a known security check was actually bypassed in this case: the email review, reserved for the content of the email, causing the whole problem in the first place. It seems to me imperative that they actually deliver up to the amount authorized. Ideally exactly the amount, but never more.
  - hnaccy 8y ago
    >They complied with a far reaching request and got told their response was too far reaching? He requested metadata and they sent actual email content, kind of a big difference there.
catchmeifyoucan 8y ago
I'm confused, why was he looking for this information?
- Pfhreak 8y ago
  My guess? To look for government employees whose networks are tightly coupled to a special interest or significant person/party.
- dstick 8y ago
  He wasn’t, that’s the kicker. They included it “because we have no way to filter the output”. It’s a good read top to bottom :)
- danso 8y ago
  He mentions that he had previously requested this info in Chicago. This is the relevant previous blog post: https://mchap.io/a-tale-about-requesting-chicagos-mayors-offices-phone-records.html https://mchap.io/a-tale-about-requesting-chicagos-mayors-off...
newsbinator 8y ago
> After that call, I asked my lawyer to reach out to their lawyer and was pretty much told that Seattle was approaching the problem as if they were pursuing Computer Fraud And Abuse (CFAA) charges. For information that they sent. Jiminey Cricket..
mr_vile 8y ago
This whole exercise seems more damaging than constructive, and I don't really like the author's smug tone, as if he deserves praise.
- warent 8y ago
  Disagree. Obviously there was a bug in the system, the author simply uncovered it. I'd rather have a smug white/grey hat than a malicious black hat. Now the system is all the more secure thanks to his actions.
- mistermann 8y ago
  The smug tone of the paid (with his taxes) government employees combined with their utter incompetence seems more inappropriate. I find it constructive in that once again it is demonstrated that the narrative that we're governed by rational, competent people and that we should trust and respect our government is very much a mirage, and the 180 turn in the tone of discussions suggests managerial malice running on top of the front line bureaucratic incompetence.
united893 8y ago
I find the writer to be a bit of a dick in his responses. Yes, the city IT may not be at the same level as Google engineers, but there’s no need to mock their ballpark estimates, and after the mistake there’s no need to be a jerk about it. Be forthright about the error. Consider being on the other side of this, due to a careless mistake the data for many people is exposed on a random strangers hard drive. Asking for an independent third party verification is reasonable. Bringing lawyers in the mix was also unnecessary. And if more people follow in the authors actions then the state level FOIA laws may be put at risk over the long term.
- patd 8y ago
  For the ballpark estimate, I think they just wanted for the request to go away by quoting an insanely high price.
  - united893 8y ago
    It was a reasonable ballpark. Let's say a city prosecutor was working on a organized crime case that involves the FBI and other people. Based on timing of emails this would leak the list of people working on the case, maybe informants and put them at risk. We all lost our collective shit when NSA said they're only collecting metadata. Metadata is Data.
    - Johnny555 8y ago
      Since they revised the cost for the data they actually sent him down from $33M to $56 (90 days of data at $1.25 for 2 days data), was a ballpark estimate that's over 500,000 times higher than the actual cost really reasonable?
      - jakobegger 8y ago
        The ballpark estimate was for checking the full text of each and every email (which was a misunderstanding -- the author wanted metadata only).
      - 8y ago
pnathan 8y ago
> The passive aggression is thick. From this I can accurately deduce that you really were talking to a person in Seattle. /local-in-joke. Pretty amazing story; Seattle, collectively, always tends to mean well, but so often they stumble.
edoceo 8y ago
Author of article has no background/understanding of the "sunshine" laws in effect in WA. Those laws may (do) explain a lot of why things go this way with any/all FOIA in WA. Source: 100s of FOIA requests to various WA government agencies.
- Johnny555 8y ago
  Can you provide more details? What is it about WA sunshine laws that make the government misunderstand a request, overestimate the cost of providing the requested data, and then provide data that was not requested resulting in a breach of disclosure laws?
  - edoceo 8y ago
    Remember when Shoreline had to pay out ~$500k because of mistakes they made on a FOIA request? https://www.rcfp.org/browse-media-law-resources/news/city-must-pay-538555-public-records-suit-over-e-mail-metadata https://www.rcfp.org/browse-media-law-resources/news/city-mu... It's because Washington agencies are required to cover reasonable attorneys fees for their opponents after losing open records lawsuits (one of the factors in our FOIA laws) So when Author sent the request to Seattle, they have this above cited example (and 100s of others across the the State) where a mistake could create a lawsuit the costs this loads of money. Did you know that the burden is on the agency to establish that its denial of inspection is proper? Did you know that the court could award you an amount between $5 and $100 a day for each day that access to the records was denied. So, if they don't give you everything you ask for you can sue. And it's easy (relatively) to win in WA for that because of our FOIA laws, then the agency has to pay for the lawyers and a penalty for delay of the records. For emails that means $5 * (Days of Delay) * (Number of Records). In short, if Seattle fucked up this FOIA request, denied or delayed -- that could have cost them millions of dollars. The author didn't understand that and (like a fool) blames the city and city-workers.
    - danso 8y ago
      > In short, if Seattle fucked up this FOIA request, denied or delayed -- that could have cost them millions of dollars. Sorry, but that sounds like bullshit. The Washington law provides for agencies to take reasonable time on a request, especially one a request that is complicated and broad. In fact, unlike the FOI law for federal and other states, the Washington law does not proscribe the number of days that an agency must respond by, only that they be made "promptly": http://app.leg.wa.gov/RCW/default.aspx?cite=42.56.520 http://app.leg.wa.gov/RCW/default.aspx?cite=42.56.520 The city of Shoreline did not have to pay out $500K "because of mistakes they made on a FOIA request", not according to what you posted: > The City of Shoreline will have to reimburse $438,555 to cover the plaintiffs' costs as Washington agencies are required to cover reasonable attorneys fees for their opponents after losing open records lawsuits. Shoreline also agreed last year to pay a $100,000 statutory penalty after the court found that the city violated the state public records act. They paid $438K for fighting the request for seven years. They paid an additional $100K penalty because they were have found to violated the law. They did not pay for "mistakes", at least not mistakes in good faith.
kerng 8y ago
Interesting read to learn of challenges cities have and what mistakes they make along the way, but the author comes across as defensive and quite arrogant.
quickthrower2 8y ago
The writer has fessed up to reading a lot of the emails. As evidenced by summarizing the content (e.g. cheating spouses, zabbix etc.). Wouldn't the responsible thing to do be stop reading the emails once you realise what is going on?
- 0xfffff 8y ago
  I had the same thought - he seems to have indulged in the data quite a bit to come up with such a thourough analysis.
  - JoeSmithson 8y ago
    I would love to hear his explanation for why he thought reading through them all was necessary. I'm not surprised the city requested 3rd party proof he had deleted them, he clearly demonstrated a fascination with their contents
  - danso 8y ago
    What was thorough about his analysis? He mentions a few things that could easily be detected with grep within a corpus of millions of messages.
- bpchaps 8y ago
  I probably only spent 30 minutes looking at it and used a few regular expressions to look for anything interesting. The point there was to understand the extent of the leak so that I could raise it in the intent of being taken seriously. A search for "(Fuck|Shit|Bitch)" can go a long way. For what it's worth, I used to work at an investment bank spending 30hr/week diving through logs with unix tools, so finding interesting information quickly is something I've learned to do quickly.
  - jacquesm 8y ago
    FWIW I think you should not have done that, though I understand the temptation. At the first indication that the data was not what you requested and contained more than you - or they - bargained for you should have stopped looking at it and alerted both the sender and the relevant data protection authorities in so far as those are a functioning entity where you live to tell them they have an 'accidental disclosure' on their hands. Essentially your blog post documents something that is pretty strong proof you are not able to deal with confidential information properly.
jacquesm 8y ago
Interesting dataset. Data like this can be used to identify strong links between contractors and government officials. One problem is that the metadata should have only contained anonymized entries for the email addresses of the counterparties of the Seattle.gov addresses, the article leaves this unclear. Another potential problem is that if a case of corruption or nepotism is identified that has not been passed to the authorities for review that the author suddenly finds himself in the possession of data that can be used to blackmail some fairly powerful people, in fact there might be fish at a higher than city level government in the trawl because there have to be links between Seattle officials and state officials. Yet another problem is that the addresses most likely contain the names of private individuals (including employees) as well, and I am not quite sure what to think of that but feel that the city has no business releasing that in cleartext. A better way for amateur sleuths and the city government to work together to battle corruption would be to release only anonymized data to protect the identities of the people working for the city, for instance by releasing only hashes of the email addresses, for instance a hash@hash format where the hash for all Seatle domains is released to the requester. All the relevant analysis could still be done, and if something interesting was found it could be released to law enforcement who in turn should have then used a judge to order de-anonymization of those entries they are interested in.
- rocqua 8y ago
  hash@hash alone isn't enough. Keyed hashes, with a secret key might work. The issue with hash@hash is that it is still possible to see whether a given person sent an email. Moreover, there are probably similar issues as with hashed_known_hosts as described in [1]. In short, the space of possible emails might be small enough to just brute-force search for all e-mails. [1] https://news.ycombinator.com/item?id=18082033 https://news.ycombinator.com/item?id=18082033
  - jacquesm 8y ago
    > The issue with hash@hash is that it is still possible to see whether a given person sent an email. In that case you already have their email, and you know what hash and salt were used. It's game over at that point afaic, nothing will stop you from reversing all of the email addresses. Even just seeing the graph laid out would allow you to infer who some of the players are. In general, to release such information on the assumption that it will be impossible to reverse it is irresponsible, and I would have loved for the city to recognize this and to get a judge to sign off on the release.
JoeSmithson 8y ago
I'm very surprised they gave out this information. I'm not talking about the mistake, I mean the actual request. In the UK I don't think you could even get a production order for this. Like, it's effectively getting Communications Data simultaneously against thousands of people not suspected of any crimes?? Like, do people know that by emailing their local government their email address is now free for scammers to request under FOI? Could I request this data myself, then start emailing them scam emails "I know you contacted us in June, could you call me on 555-1223 etc" This seems totally against the spirit of FOI
- jacquesm 8y ago
  Yes, you are as far as I can see correct. The request should have been rejected as overbroad and against data privacy laws (in so far as they exist), or the purpose of the request could have been verified and then they might have seen whether or not there was another way to let the requester do their work without giving them the data they requested (see another comment of mine for one suggestion).
  - wegs2 8y ago
    That's not how FOIA works. It's a good thing too. Government employees almost always fight FOIA requests. There aren't many subjective tools (e.g. overbroad) and you're certainly not required to say why you're making the request. Data privacy laws in the US are unfortunately minimal. The bigger problem comes from imbalance -- if the government and corporations have lists of names, people need them to in order to be able to work together and organize. If you don't think this information should go out in FOIA requests, the tool to accomplish that is data destruction. Government could wipe old emails once no longer relevant.
    - sverhagen 8y ago
      Commercial organizations at least are known to implement maximum allowed retention strategies, such as having their staff not keep archived email beyond three months, presumably so it doesn't embarrassingly show up when it's legally unfavorable. Not quite the same, but along the same lines.
nickysielicki 8y ago
> Seattle was approaching the problem as if they were pursuing Computer Fraud And Abuse (CFAA) charges. For information that they sent. Jiminey Cricket.. > So, I deleted the files. Isn't it great to live in a country where we have generic felonies that governments can apply to just about anything involving a computer and ruin your life? Land of the "free" and the home of the 'fraid.
- giancarlostoro 8y ago
  Where police treat you as fully guilty if you're a suspect despite you having to be thought of as innocent until proven guilty. A number of people have gone to jail because they knew not to keep their mouths shut and told cops too much that made them sound like criminals and guilty. Most people don't know where they were 2 hours ago, let alone the night of December 5th, 1957 at 2:05 AM, like SERIOUSLY?
daenz 8y ago
Somewhat related, I'm constantly shocked (maybe I shouldn't be anymore) at the tech ineptitude of cities that are supposed to be big tech hubs. I live in Seattle, and my regular tech complaint is we can't get the buses connected to an app that is accurate within +-10 minutes. I know it doesn't sound like much, but how much tech brainpower is here, and why isn't that tech shining more clearly?
- Tempest1981 8y ago
  Because private tech companies fight like mad to avoid paying taxes to public city operations? Citizens too, probably. Nobody likes taxes. And the best/brightest tech workers gravitate to the higher private salaries. It’s not a technical problem - those are easier.
  - tvanantwerp 8y ago
    I live in DC, land of the professional Fed. At the absolute highest level and after adjusting for location, the most a DC Fed could earn is $164,200. No surprise that anyone with serious technical talent--and by extension, market value--doesn't want such a job.
    - jaaames 8y ago
      True, but consider some folks are content with a 38 hour week, may have automated large chunks of their job, and find the demands of working for a big public sector organization far less than that of a similar role in a private sector tech company.
      - tvanantwerp 8y ago
        If you can manage that, sure, sounds pretty good. But I doubt that's the case most of the time. None of the Feds I've had the chance to talk to sound like they're slacking off.
      - zifnab06 8y ago
        This is something I argue with myself a lot about. I work as a software engineer for a large non-tech company. They pay well, but I could easily go to a tech company and make 20% more. Except...well, in 18 months I've never had to work more than a 40 hour week. I can totally see the appeal of a government job.
mkoryak 8y ago
A few years ago I found a random SSD on the ground while on a walk with my son. The drive contained unencrypted records which squarly fall under HIPPA. I also did the right thing and returned it to the proper owner and told them about how their mdb files were readable by anyone. The same exact thing happened. They thanked me and then their lawyers nicely asked me to clone my hard drive and sign a bunch of shit. It was not fun at all. A lot of them thought that I hacked something.
- StudentStuff 8y ago
  The type of organization that would store HIPPA encumbered data unencrypted, which based on my brief reading is not legal anymore, is not one that would operate in a reasonable (or legal) manner. Sadly, that seems to be most organizations that fall under HIPPA, compliance is a box to be checked while expending as little resources and effort as possible. How they reacted to your kind action is sad, and depressingly common. I hope you told them to pound sand, and contacted whoever the data protection authorities were in your state. There needs to be much more aggressive enforcement of HIPPA and similar data protection laws, CYA bull like you encountered should not be happening. Article I ran across: https://info.townsendsecurity.com/bid/74330/Does-HIPAA-Require-Encryption-of-Patient-Information-ePHI https://info.townsendsecurity.com/bid/74330/Does-HIPAA-Requi...
  - scarejunba 8y ago
    Guys, it's HIPAA not HIPPA.
    - thaumasiotes 8y ago
      This is an interesting case. I always pronounced HIPAA as "hee-pah". That has the advantage of approximating the spelling, but the disadvantage that it's not really a natural way for an English word to be pronounced. People in the medical field, who deal with HIPAA all the time, pronounce it as if it was spelled HIPPA. It's a short step from there to actually spelling it HIPPA.
- rocqua 8y ago
doctorless 8y ago
To me, the most interesting thing in this entire post is the following: > Funny enough, in the middle of that question, my internet died and interrupted the call for the first time in the six months I lived in that house. Odd. It came back ten minutes later, and I dialed back into the conference line, but the mood of the call pretty much 180’d. I find that when strange things happen like this, they’re hardly coincidence. Did you run a traceroute after the disconnect anywhere? Did you see an IP address change? If so, was it a significant change in the CIDR block it was within?
- jacquesm 8y ago
  That's speculative without any proof and I personally think it is a weak point in the article. I do conference calls several times per week and the number of times I've been accidentally booted out of the room are numerous. Also, once they realized they had left the room of course they would continue to discuss the case and it is obvious they had to consider all possibilities, including the recipient releasing the information to others, hence the 180.
  - bpchaps 8y ago
    Er, why the doubt? My internet completely died. My room mates were also affected.. Not sure how I can give proof.
    - jacquesm 8y ago
      Because it is just the timing that makes you say this, and I highly doubt the city of Seattle can - on a moments notice, no less - pull the plug on any residential internet connection. If true, that would be a far bigger news item than the rest of your story.
      - bpchaps 8y ago
        Who knows. It was strange for me, too.
pontifier 8y ago
I was quoted almost $200k for a similar request for emails. I was trying to investigate a shady real estate deal, and they made it as difficult as possible. I was never actually able to get the information I requested. I'm completely disgusted and fed up with corruption.
- rocqua 8y ago
  Perhaps try and sell the story with some of the investigative podcasts / blogs? An apparent cover-up gets as much attraction as uncovered corruption. (As well it should).
  - pontifier 8y ago
    I'm now convinced that there is no amount of evidence of wrongdoing that will actually harm crooked politicians. They control the narrative, and the courts. I don't want to end up bone saw murdered, and it feels more likely every day.
nzealand 8y ago
The dump includes email addresses, both government and private. That does not seem good. There is a surprisingly little spam. Either that is about to change, or spam didn't get included in the FOIA.
amingilani 8y ago
I'm simultaneously impressed and saddened by how fast the responses for these FOIA requests were proceeded by the government. And here in my country, I needed a court order to get at least an acknowledgement of my FOI request. And now I'm petition court intervention to get the FOI processed in accordance with the law.
nzjrs 8y ago
TIL there is a genre of people who call themselves FOIA nerds and who appear to be unpleasant, doing this sort of fishing for fun.
draw_down 8y ago
To look at the mess the city made, and decide to take issue with the author’s “tone”, is a very telling reaction.
petecooper 8y ago
FTA: >Seattle's first response included a bit of gobsmackery that I’ve almost become used to Brit here. I'm always amused that 'gobsmack' and its derived words are still used these days, more so across the Atlantic. Roughly translated: lost for words, typically for a short time.
marklyon 8y ago
This could have been an extremely valuable dataset for the legal community. The Enron data is currently guiding much of our machine learning validation, simply because it's available.
- HeadInTheClouds 8y ago
  As a general point, I totally agree with this. The Enron dataset released over ~15 years ago is still used by EDiscovery and other legal vendors along with other researchers. There have been a huge number of papers using this dataset and there are not many other datasets of its type or size available and despite its age is one of the best we have. If people are aware of legally released datasets with a similar size and content I would be interested to hear about them.
nickthemagicman 8y ago
Mental note to self. Instead of reporting data breaches turn it into a torrent and make it public.
atomical 8y ago
> Especially with the use of Excel, which would be useful for removing duplicates, etc. Excel can only handle about 1 million rows, right?
- philipodonnell 8y ago
  Nah its not limited except by memory anymore AFAIK.
  - atomical 8y ago
    Ah, spinning beachball time then.
notafraudster 8y ago
In case Matt Chapman is reading this -- the contact email at the bottom of the page (matt@mchap.com) is probably not correct, given that the domain mchap.com redirects to an australian photographer. The alternative is that the email address is correct and Matt is redirecting his domain to another Matt Chapman, which would be totally hilarious.
- bpchaps 8y ago
  Ah! Thanks!
khloe122 8y ago
Hey everyone out there, I'm pleased to recommend BILL, a private investigator/hacker for any hacking related job you might be interested in. I got in contact with him online when i was having problems with my cheating husband and needed help in getting evidence against him in court, He helped me hack into my husband's phone and social networks and under some hours i was seeing chats and messages and he also help retrieve deleted conversations... I''m so glad i got in contact with him, He is into various kinds of job like Facebook hacking/spying, whats-app spying, phone tracking and cloning, snap-chat, Instagram, we-chat, phone text messages, bank statements, hangouts etc.You can contact him via BILLHACKWIZARD@ GMAIL DOT COM, Put a call or text through on +1(314)833-9209, and you can also reach him via Whats-app messenger on +1(314)635-7319. I'm just a satisfied customer sharing my testimony, you should contact him to see for yourself, and pls do not forget to mention my name to him.
Markoff 8y ago
more than dollar per email address doesn't sound like very good deal to me

516 points by bpchaps 8y ago

I can't tell whether this is a testament to the incompetence of public IT operations or an indictment of public records keeping practice. Maybe both?

bpchaps 8y ago
Probably both. Or maybe just a story about a thing that happened.
derblitzmann 8y ago
Well, like most of Seattle's efforts, it gets fucked up, mostly because it is from Seattle. Note: I am a Seattle resident and I expect nothing less.
- rlucas 8y ago
  Have you lived anywhere else, like, say, the northeast? Seattle is a shockingly well run municipality among American cities of size.
  - Fellshard 8y ago
    You haven't met the city council of the last several years, then...
    - zifnab06 8y ago
      Maybe I should run. I wonder if I'd stand a chance against sawant.
btrettel 8y ago
Also has a good example of hostile FOIA officers. I have filed about two dozen FOIA requests, and the vast majority were fine, though usually slow. Earlier this year one longstanding request of mine was rejected because they claimed the document I wanted was export controlled. Two months later I sent in an appeal where I showed that the document in question was not export controlled (I filed another FOIA with a separate agency to check), and I suggested that they lied to me. Never got an apology, though they seem to be processing the request for real now.
- c3534l 8y ago
  Never attribute to malice that which is adequately explained by stupidity.
  - btrettel 8y ago
    I agree, but in this case stupidity can not explain their actions. The agency processing the FOIA request would get the export control status from the third party I contacted. The third party's software highlights export controlled documents with a red and highly noticeable statement. It would be difficult to believe they thought this was present when it was not. The request also had actually been transferred several times because no one believes they have the authority to release the document I requested. The other agencies had ample opportunity to reject the request for being export controlled, but none did. There are some other reasons that I will omit for brevity. This makes me think that the export controlled claim was a lie meant to kill the request. Most people would stop at what they told me, but I thought it was worth verifying.
  - anigbrowl 8y ago
    I used to abide by this until it dawned on me that malicious people often use stupid people as a cat's paw to conceal liability. A careless disregard for the truth is itself a form of dishonesty. This argument is laid out very effectively in philosopher Harry Frankfurt's delightful short book On Bullshit.
- scoggs 8y ago
  I've requested thousands from all over the United States, big and small municipalities. The responses and individuals ranged from a small contingent of extremely professional, organized, helpful, and all around wonderful people to many and more abysmal, unorganized, uneducated, tech-illiterate, and downright incompetent folks. I can't remember how many times I'd point the person I was attempting to request the information from directly to their sample version or official space where they said they were the holders of said datas I was looking for which also provided the instructions to specifically call them to request the data. I know from my POV (as a tech worker) it may seem silly to expect anything like that from them but what I was asking for was akin to an excel spreadsheet full of information they absolutely do have and it required no legwork, no generation of new materials, no gathering of data from multiple ancient sources. It was all material / datas each individual municipality was making money, hand over fist, every month of every year -- the sample datas most municipalities had was evidence of that. I was basically asking them for the collection of the entire data, publicly traded info, and most were convinced they didn't have it. A product of laziness is what I'd chalk it up to because the individuals and municipalities that were awesome to work with and more than helpful seemed to take pride in their work and getting said data was easier than the majority of things involving interfacing with the government at any level usually winds up being like.

lsiebert 8y ago

So they accidentally included the first 256 characters for all the emails? yikes.

kristofferR 8y ago

Awesome work! I'm glad we have people willing to go through all the hassle, so we can keep our governments responsible.

daodedickinson 8y ago

It's clear they didn't have expertise to do it, and I'm tired of reading people that know way more looking down at others over it and assuming they don't want to comply. If they hiding something and malicious, the end result wouldn't have been to send way too much, but I don't see the author realizing this fast enough.

dnbgfher 8y ago
I'm not sure why you think the only malicious response would be to send over too little. It's a common enough scene in TV shows and the like where a malicious actor attempts to hide incriminating information in a sea of irrelevant information.
sanotehu 8y ago
Agreed. You're putting an overworked, underpaid public servant in a "damned if you do, damned if you don't" scenario. They complied with a far reaching request and got told their response was too far reaching? I'd quit my job if faced with a legal minefield like that, especially one not actually related to the job itself
- setr 8y ago
  Presumably this should be considered overreaching should be considered an important though: if there’s an authorization process in play, then more information has been given out than the public servant was actually authorized to hand out. If me as a citizen starts receiving sensitive information despite only being authorized to receive insensitive info, that could easily become a significant security breach. In fact, a known security check was actually bypassed in this case: the email review, reserved for the content of the email, causing the whole problem in the first place. It seems to me imperative that they actually deliver up to the amount authorized. Ideally exactly the amount, but never more.
- hnaccy 8y ago
  >They complied with a far reaching request and got told their response was too far reaching? He requested metadata and they sent actual email content, kind of a big difference there.

catchmeifyoucan 8y ago

I'm confused, why was he looking for this information?

Pfhreak 8y ago
My guess? To look for government employees whose networks are tightly coupled to a special interest or significant person/party.
dstick 8y ago
He wasn’t, that’s the kicker. They included it “because we have no way to filter the output”. It’s a good read top to bottom :)
danso 8y ago
He mentions that he had previously requested this info in Chicago. This is the relevant previous blog post: https://mchap.io/a-tale-about-requesting-chicagos-mayors-offices-phone-records.html https://mchap.io/a-tale-about-requesting-chicagos-mayors-off...

newsbinator 8y ago

> After that call, I asked my lawyer to reach out to their lawyer and was pretty much told that Seattle was approaching the problem as if they were pursuing Computer Fraud And Abuse (CFAA) charges. For information that they sent. Jiminey Cricket..

mr_vile 8y ago

This whole exercise seems more damaging than constructive, and I don't really like the author's smug tone, as if he deserves praise.

warent 8y ago
Disagree. Obviously there was a bug in the system, the author simply uncovered it. I'd rather have a smug white/grey hat than a malicious black hat. Now the system is all the more secure thanks to his actions.
mistermann 8y ago
The smug tone of the paid (with his taxes) government employees combined with their utter incompetence seems more inappropriate. I find it constructive in that once again it is demonstrated that the narrative that we're governed by rational, competent people and that we should trust and respect our government is very much a mirage, and the 180 turn in the tone of discussions suggests managerial malice running on top of the front line bureaucratic incompetence.

united893 8y ago

I find the writer to be a bit of a dick in his responses. Yes, the city IT may not be at the same level as Google engineers, but there’s no need to mock their ballpark estimates, and after the mistake there’s no need to be a jerk about it. Be forthright about the error. Consider being on the other side of this, due to a careless mistake the data for many people is exposed on a random strangers hard drive. Asking for an independent third party verification is reasonable. Bringing lawyers in the mix was also unnecessary. And if more people follow in the authors actions then the state level FOIA laws may be put at risk over the long term.

patd 8y ago
For the ballpark estimate, I think they just wanted for the request to go away by quoting an insanely high price.
- united893 8y ago
  It was a reasonable ballpark. Let's say a city prosecutor was working on a organized crime case that involves the FBI and other people. Based on timing of emails this would leak the list of people working on the case, maybe informants and put them at risk. We all lost our collective shit when NSA said they're only collecting metadata. Metadata is Data.
  - Johnny555 8y ago
    Since they revised the cost for the data they actually sent him down from $33M to $56 (90 days of data at $1.25 for 2 days data), was a ballpark estimate that's over 500,000 times higher than the actual cost really reasonable?
    - jakobegger 8y ago
      The ballpark estimate was for checking the full text of each and every email (which was a misunderstanding -- the author wanted metadata only).
    - 8y ago

pnathan 8y ago

> The passive aggression is thick. From this I can accurately deduce that you really were talking to a person in Seattle. /local-in-joke. Pretty amazing story; Seattle, collectively, always tends to mean well, but so often they stumble.

edoceo 8y ago

Author of article has no background/understanding of the "sunshine" laws in effect in WA. Those laws may (do) explain a lot of why things go this way with any/all FOIA in WA. Source: 100s of FOIA requests to various WA government agencies.

Johnny555 8y ago
Can you provide more details? What is it about WA sunshine laws that make the government misunderstand a request, overestimate the cost of providing the requested data, and then provide data that was not requested resulting in a breach of disclosure laws?
- edoceo 8y ago
  Remember when Shoreline had to pay out ~$500k because of mistakes they made on a FOIA request? https://www.rcfp.org/browse-media-law-resources/news/city-must-pay-538555-public-records-suit-over-e-mail-metadata https://www.rcfp.org/browse-media-law-resources/news/city-mu... It's because Washington agencies are required to cover reasonable attorneys fees for their opponents after losing open records lawsuits (one of the factors in our FOIA laws) So when Author sent the request to Seattle, they have this above cited example (and 100s of others across the the State) where a mistake could create a lawsuit the costs this loads of money. Did you know that the burden is on the agency to establish that its denial of inspection is proper? Did you know that the court could award you an amount between $5 and $100 a day for each day that access to the records was denied. So, if they don't give you everything you ask for you can sue. And it's easy (relatively) to win in WA for that because of our FOIA laws, then the agency has to pay for the lawyers and a penalty for delay of the records. For emails that means $5 * (Days of Delay) * (Number of Records). In short, if Seattle fucked up this FOIA request, denied or delayed -- that could have cost them millions of dollars. The author didn't understand that and (like a fool) blames the city and city-workers.
  - danso 8y ago
    > In short, if Seattle fucked up this FOIA request, denied or delayed -- that could have cost them millions of dollars. Sorry, but that sounds like bullshit. The Washington law provides for agencies to take reasonable time on a request, especially one a request that is complicated and broad. In fact, unlike the FOI law for federal and other states, the Washington law does not proscribe the number of days that an agency must respond by, only that they be made "promptly": http://app.leg.wa.gov/RCW/default.aspx?cite=42.56.520 http://app.leg.wa.gov/RCW/default.aspx?cite=42.56.520 The city of Shoreline did not have to pay out $500K "because of mistakes they made on a FOIA request", not according to what you posted: > The City of Shoreline will have to reimburse $438,555 to cover the plaintiffs' costs as Washington agencies are required to cover reasonable attorneys fees for their opponents after losing open records lawsuits. Shoreline also agreed last year to pay a $100,000 statutory penalty after the court found that the city violated the state public records act. They paid $438K for fighting the request for seven years. They paid an additional $100K penalty because they were have found to violated the law. They did not pay for "mistakes", at least not mistakes in good faith.

kerng 8y ago

Interesting read to learn of challenges cities have and what mistakes they make along the way, but the author comes across as defensive and quite arrogant.

quickthrower2 8y ago

The writer has fessed up to reading a lot of the emails. As evidenced by summarizing the content (e.g. cheating spouses, zabbix etc.). Wouldn't the responsible thing to do be stop reading the emails once you realise what is going on?

0xfffff 8y ago
I had the same thought - he seems to have indulged in the data quite a bit to come up with such a thourough analysis.
- JoeSmithson 8y ago
  I would love to hear his explanation for why he thought reading through them all was necessary. I'm not surprised the city requested 3rd party proof he had deleted them, he clearly demonstrated a fascination with their contents
- danso 8y ago
  What was thorough about his analysis? He mentions a few things that could easily be detected with grep within a corpus of millions of messages.
bpchaps 8y ago
I probably only spent 30 minutes looking at it and used a few regular expressions to look for anything interesting. The point there was to understand the extent of the leak so that I could raise it in the intent of being taken seriously. A search for "(Fuck|Shit|Bitch)" can go a long way. For what it's worth, I used to work at an investment bank spending 30hr/week diving through logs with unix tools, so finding interesting information quickly is something I've learned to do quickly.
- jacquesm 8y ago
  FWIW I think you should not have done that, though I understand the temptation. At the first indication that the data was not what you requested and contained more than you - or they - bargained for you should have stopped looking at it and alerted both the sender and the relevant data protection authorities in so far as those are a functioning entity where you live to tell them they have an 'accidental disclosure' on their hands. Essentially your blog post documents something that is pretty strong proof you are not able to deal with confidential information properly.

jacquesm 8y ago

Interesting dataset. Data like this can be used to identify strong links between contractors and government officials. One problem is that the metadata should have only contained anonymized entries for the email addresses of the counterparties of the Seattle.gov addresses, the article leaves this unclear. Another potential problem is that if a case of corruption or nepotism is identified that has not been passed to the authorities for review that the author suddenly finds himself in the possession of data that can be used to blackmail some fairly powerful people, in fact there might be fish at a higher than city level government in the trawl because there have to be links between Seattle officials and state officials. Yet another problem is that the addresses most likely contain the names of private individuals (including employees) as well, and I am not quite sure what to think of that but feel that the city has no business releasing that in cleartext. A better way for amateur sleuths and the city government to work together to battle corruption would be to release only anonymized data to protect the identities of the people working for the city, for instance by releasing only hashes of the email addresses, for instance a hash@hash format where the hash for all Seatle domains is released to the requester. All the relevant analysis could still be done, and if something interesting was found it could be released to law enforcement who in turn should have then used a judge to order de-anonymization of those entries they are interested in.

rocqua 8y ago
hash@hash alone isn't enough. Keyed hashes, with a secret key might work. The issue with hash@hash is that it is still possible to see whether a given person sent an email. Moreover, there are probably similar issues as with hashed_known_hosts as described in [1]. In short, the space of possible emails might be small enough to just brute-force search for all e-mails. [1] https://news.ycombinator.com/item?id=18082033 https://news.ycombinator.com/item?id=18082033
- jacquesm 8y ago
  > The issue with hash@hash is that it is still possible to see whether a given person sent an email. In that case you already have their email, and you know what hash and salt were used. It's game over at that point afaic, nothing will stop you from reversing all of the email addresses. Even just seeing the graph laid out would allow you to infer who some of the players are. In general, to release such information on the assumption that it will be impossible to reverse it is irresponsible, and I would have loved for the city to recognize this and to get a judge to sign off on the release.

JoeSmithson 8y ago

I'm very surprised they gave out this information. I'm not talking about the mistake, I mean the actual request. In the UK I don't think you could even get a production order for this. Like, it's effectively getting Communications Data simultaneously against thousands of people not suspected of any crimes?? Like, do people know that by emailing their local government their email address is now free for scammers to request under FOI? Could I request this data myself, then start emailing them scam emails "I know you contacted us in June, could you call me on 555-1223 etc" This seems totally against the spirit of FOI

jacquesm 8y ago
Yes, you are as far as I can see correct. The request should have been rejected as overbroad and against data privacy laws (in so far as they exist), or the purpose of the request could have been verified and then they might have seen whether or not there was another way to let the requester do their work without giving them the data they requested (see another comment of mine for one suggestion).
- wegs2 8y ago
  That's not how FOIA works. It's a good thing too. Government employees almost always fight FOIA requests. There aren't many subjective tools (e.g. overbroad) and you're certainly not required to say why you're making the request. Data privacy laws in the US are unfortunately minimal. The bigger problem comes from imbalance -- if the government and corporations have lists of names, people need them to in order to be able to work together and organize. If you don't think this information should go out in FOIA requests, the tool to accomplish that is data destruction. Government could wipe old emails once no longer relevant.
  - sverhagen 8y ago
    Commercial organizations at least are known to implement maximum allowed retention strategies, such as having their staff not keep archived email beyond three months, presumably so it doesn't embarrassingly show up when it's legally unfavorable. Not quite the same, but along the same lines.

nickysielicki 8y ago

> Seattle was approaching the problem as if they were pursuing Computer Fraud And Abuse (CFAA) charges. For information that they sent. Jiminey Cricket.. > So, I deleted the files. Isn't it great to live in a country where we have generic felonies that governments can apply to just about anything involving a computer and ruin your life? Land of the "free" and the home of the 'fraid.

giancarlostoro 8y ago
Where police treat you as fully guilty if you're a suspect despite you having to be thought of as innocent until proven guilty. A number of people have gone to jail because they knew not to keep their mouths shut and told cops too much that made them sound like criminals and guilty. Most people don't know where they were 2 hours ago, let alone the night of December 5th, 1957 at 2:05 AM, like SERIOUSLY?

daenz 8y ago

Somewhat related, I'm constantly shocked (maybe I shouldn't be anymore) at the tech ineptitude of cities that are supposed to be big tech hubs. I live in Seattle, and my regular tech complaint is we can't get the buses connected to an app that is accurate within +-10 minutes. I know it doesn't sound like much, but how much tech brainpower is here, and why isn't that tech shining more clearly?

Tempest1981 8y ago
Because private tech companies fight like mad to avoid paying taxes to public city operations? Citizens too, probably. Nobody likes taxes. And the best/brightest tech workers gravitate to the higher private salaries. It’s not a technical problem - those are easier.
- tvanantwerp 8y ago
  I live in DC, land of the professional Fed. At the absolute highest level and after adjusting for location, the most a DC Fed could earn is $164,200. No surprise that anyone with serious technical talent--and by extension, market value--doesn't want such a job.
  - jaaames 8y ago
    True, but consider some folks are content with a 38 hour week, may have automated large chunks of their job, and find the demands of working for a big public sector organization far less than that of a similar role in a private sector tech company.
    - tvanantwerp 8y ago
      If you can manage that, sure, sounds pretty good. But I doubt that's the case most of the time. None of the Feds I've had the chance to talk to sound like they're slacking off.
    - zifnab06 8y ago
      This is something I argue with myself a lot about. I work as a software engineer for a large non-tech company. They pay well, but I could easily go to a tech company and make 20% more. Except...well, in 18 months I've never had to work more than a 40 hour week. I can totally see the appeal of a government job.

mkoryak 8y ago

A few years ago I found a random SSD on the ground while on a walk with my son. The drive contained unencrypted records which squarly fall under HIPPA. I also did the right thing and returned it to the proper owner and told them about how their mdb files were readable by anyone. The same exact thing happened. They thanked me and then their lawyers nicely asked me to clone my hard drive and sign a bunch of shit. It was not fun at all. A lot of them thought that I hacked something.

StudentStuff 8y ago
The type of organization that would store HIPPA encumbered data unencrypted, which based on my brief reading is not legal anymore, is not one that would operate in a reasonable (or legal) manner. Sadly, that seems to be most organizations that fall under HIPPA, compliance is a box to be checked while expending as little resources and effort as possible. How they reacted to your kind action is sad, and depressingly common. I hope you told them to pound sand, and contacted whoever the data protection authorities were in your state. There needs to be much more aggressive enforcement of HIPPA and similar data protection laws, CYA bull like you encountered should not be happening. Article I ran across: https://info.townsendsecurity.com/bid/74330/Does-HIPAA-Require-Encryption-of-Patient-Information-ePHI https://info.townsendsecurity.com/bid/74330/Does-HIPAA-Requi...
- scarejunba 8y ago
  Guys, it's HIPAA not HIPPA.
  - thaumasiotes 8y ago
    This is an interesting case. I always pronounced HIPAA as "hee-pah". That has the advantage of approximating the spelling, but the disadvantage that it's not really a natural way for an English word to be pronounced. People in the medical field, who deal with HIPAA all the time, pronounce it as if it was spelled HIPPA. It's a short step from there to actually spelling it HIPPA.
rocqua 8y ago

doctorless 8y ago

To me, the most interesting thing in this entire post is the following: > Funny enough, in the middle of that question, my internet died and interrupted the call for the first time in the six months I lived in that house. Odd. It came back ten minutes later, and I dialed back into the conference line, but the mood of the call pretty much 180’d. I find that when strange things happen like this, they’re hardly coincidence. Did you run a traceroute after the disconnect anywhere? Did you see an IP address change? If so, was it a significant change in the CIDR block it was within?

jacquesm 8y ago
That's speculative without any proof and I personally think it is a weak point in the article. I do conference calls several times per week and the number of times I've been accidentally booted out of the room are numerous. Also, once they realized they had left the room of course they would continue to discuss the case and it is obvious they had to consider all possibilities, including the recipient releasing the information to others, hence the 180.
- bpchaps 8y ago
  Er, why the doubt? My internet completely died. My room mates were also affected.. Not sure how I can give proof.
  - jacquesm 8y ago
    Because it is just the timing that makes you say this, and I highly doubt the city of Seattle can - on a moments notice, no less - pull the plug on any residential internet connection. If true, that would be a far bigger news item than the rest of your story.
    - bpchaps 8y ago
      Who knows. It was strange for me, too.

pontifier 8y ago

I was quoted almost $200k for a similar request for emails. I was trying to investigate a shady real estate deal, and they made it as difficult as possible. I was never actually able to get the information I requested. I'm completely disgusted and fed up with corruption.

rocqua 8y ago
Perhaps try and sell the story with some of the investigative podcasts / blogs? An apparent cover-up gets as much attraction as uncovered corruption. (As well it should).
- pontifier 8y ago
  I'm now convinced that there is no amount of evidence of wrongdoing that will actually harm crooked politicians. They control the narrative, and the courts. I don't want to end up bone saw murdered, and it feels more likely every day.

nzealand 8y ago

The dump includes email addresses, both government and private. That does not seem good. There is a surprisingly little spam. Either that is about to change, or spam didn't get included in the FOIA.

amingilani 8y ago

I'm simultaneously impressed and saddened by how fast the responses for these FOIA requests were proceeded by the government. And here in my country, I needed a court order to get at least an acknowledgement of my FOI request. And now I'm petition court intervention to get the FOI processed in accordance with the law.

nzjrs 8y ago

TIL there is a genre of people who call themselves FOIA nerds and who appear to be unpleasant, doing this sort of fishing for fun.

draw_down 8y ago

To look at the mess the city made, and decide to take issue with the author’s “tone”, is a very telling reaction.

petecooper 8y ago

FTA: >Seattle's first response included a bit of gobsmackery that I’ve almost become used to Brit here. I'm always amused that 'gobsmack' and its derived words are still used these days, more so across the Atlantic. Roughly translated: lost for words, typically for a short time.

marklyon 8y ago

This could have been an extremely valuable dataset for the legal community. The Enron data is currently guiding much of our machine learning validation, simply because it's available.

HeadInTheClouds 8y ago
As a general point, I totally agree with this. The Enron dataset released over ~15 years ago is still used by EDiscovery and other legal vendors along with other researchers. There have been a huge number of papers using this dataset and there are not many other datasets of its type or size available and despite its age is one of the best we have. If people are aware of legally released datasets with a similar size and content I would be interested to hear about them.

nickthemagicman 8y ago

Mental note to self. Instead of reporting data breaches turn it into a torrent and make it public.

atomical 8y ago

> Especially with the use of Excel, which would be useful for removing duplicates, etc. Excel can only handle about 1 million rows, right?

philipodonnell 8y ago
Nah its not limited except by memory anymore AFAIK.
- atomical 8y ago
  Ah, spinning beachball time then.

notafraudster 8y ago

In case Matt Chapman is reading this -- the contact email at the bottom of the page (matt@mchap.com) is probably not correct, given that the domain mchap.com redirects to an australian photographer. The alternative is that the email address is correct and Matt is redirecting his domain to another Matt Chapman, which would be totally hilarious.

bpchaps 8y ago
Ah! Thanks!

khloe122 8y ago

Hey everyone out there, I'm pleased to recommend BILL, a private investigator/hacker for any hacking related job you might be interested in. I got in contact with him online when i was having problems with my cheating husband and needed help in getting evidence against him in court, He helped me hack into my husband's phone and social networks and under some hours i was seeing chats and messages and he also help retrieve deleted conversations... I''m so glad i got in contact with him, He is into various kinds of job like Facebook hacking/spying, whats-app spying, phone tracking and cloning, snap-chat, Instagram, we-chat, phone text messages, bank statements, hangouts etc.You can contact him via BILLHACKWIZARD@ GMAIL DOT COM, Put a call or text through on +1(314)833-9209, and you can also reach him via Whats-app messenger on +1(314)635-7319. I'm just a satisfied customer sharing my testimony, you should contact him to see for yourself, and pls do not forget to mention my name to him.

Markoff 8y ago

more than dollar per email address doesn't sound like very good deal to me