3 ms·

Argh. I used to work a lot in open data, including a stint within the open data department of a major city government. This kind of behaviour does nothing to a

Argh. I used to work a lot in open data, including a stint within the open data department of a major city government. This kind of behaviour does nothing to advance the cause. It just perpetuates the belief that open data and FOI are massive waste of time and resources and open legal risks for no good reason. I'm also pretty surprised that people think email metadata from government is legitimate open data. Do you think it should be public how many times you have emailed government and gotten a response, and to which departments? I don't.

lbschenkel 3y ago
> I'm also pretty surprised that people think email metadata from government is legitimate open data. Do you think it should be public how many times you have emailed government and gotten a response, and to which departments? I don't. In Sweden, when you contact the government by web form or mail, they warn you that every communication is by law part of the public record (metadata and contents), and you should not include anything sensitive that you do not want divulged.
chaps 3y ago
None of this is open data. Open data is data that's released on a discretionary basis; often times a FOIA request for the same data found within open data has a legal requirement for redaction. We see this happen all the time. What's worse is that at the executive level, eg chief data officers are often forbidden by legal teams from interacting with the media and adjacent orgs. I've literally been told by a CDO that they cannot speak with me. My go-to phrase for open data is, "open data is a lie". Because at the end of the day, we have zero legal recourse to validate that what is provided through open datasets is complete, both in terms of available columns and available rows. Very rarely will it ever be explained why, or even if, information is missing. And the subsequent effects of that lead to a deep public misunderstanding of what's going on. And much of it is intentional out of explicit fear that the public will misinterpret the data. So we go through FOIA instead and we end up going through legal battles. There's a reason I've had to do ~10 FOIA suits. tl;dr: open data is nice, but its lack of rigor and accountability makes it effectively useless for anything that requires depth.
- stevage 3y ago
  Perhaps you missed the bit where I said I worked in open data for years, including inside government. They are two very different processes, yes. Open data tends to connect you with technologists and subject matter experts who want to talk about their field. FOI tends to connect you with lawyers and governance gurus who want to find reasons not to give you what you want. Around here the FOI processes are getting slower and less forthcoming, in defiance of the law, and ridiculous requests like these ones do not help make the case for their support.
  - chaps 3y ago
    I've spoken to many people inside government who work on the open data side. To be blunt: the signal to noise ratio isn't very good in terms of whether someone actually knows what they're talking about. So please consider my reluctance to take much of what is said as a product of being lied to for years, both intentionally and unintentionally. Yes, FOI processes are getting slower and yes we're going to keep suing. What other choice do we have? One thing to consider is that SO MUCH of that is a product of gov agencies simply not preparing their documents in a way that makes them FOIAable. So much can be done that simply isn't -- a significant amount of which is very intentional (eg, police records) in an intentional violation of the law. We're not trying to be dicks in all of this. We just want to know what's going on. So yours (and many others in your position)'s clear desire to think of us as adversaries rather than simply people who want to know what the fuck is going on is blinding your vision. Get out of your head as a person in government and start thinking as a person who's at least empathetic towards the frustrations of a tired public that constantly lied to, deceived, and fed communications that are perpetually weighed down by legally reviewed PR releases. We can do better, but sure as hell not with your defeated attitude.
    - lazyasciiart 3y ago
      One thing to consider is that 'government' literally constantly advertises the opportunity for you to get credentials to their system and prepare documents themselves. They will pay you to do this. > So yours (and many others in your position)'s clear desire to think of us as adversaries rather than simply people who want to know what the fuck is going on is blinding your vision. Yea, I don't work in his position or any government position, but you are being adversarial. And, whatever you're trying to do, you're being a dick in at least this comment.
colinsane 3y ago
> I'm also pretty surprised that people think email metadata from government is legitimate open data. Do you think it should be public how many times you have emailed government and gotten a response, and to which departments? I don't. on the surface i agree with you. beneath the surface, other governmental units already possess all the metadata for your emails/phone calls, it would be a weird bit of asymmetry to say it’s okay for them to have your metadata but not for you to have their metadata. it’s not a perfect mirror, for obvious reasons, but if OP went on to show that this metadata is powerful and push that neither side should be collecting that type of metadata, then that’s a single legitimate use i can think of for this kind of FOIA.
- stevage 3y ago
  >it would be a weird bit of asymmetry to say it’s okay for them to have your metadata but not for you to have their metadata. But that's not what this is. This is saying it's ok for them to give your metadata out to anyone who asks.

lbschenkel 3y ago

> I'm also pretty surprised that people think email metadata from government is legitimate open data. Do you think it should be public how many times you have emailed government and gotten a response, and to which departments? I don't. In Sweden, when you contact the government by web form or mail, they warn you that every communication is by law part of the public record (metadata and contents), and you should not include anything sensitive that you do not want divulged.

chaps 3y ago

None of this is open data. Open data is data that's released on a discretionary basis; often times a FOIA request for the same data found within open data has a legal requirement for redaction. We see this happen all the time. What's worse is that at the executive level, eg chief data officers are often forbidden by legal teams from interacting with the media and adjacent orgs. I've literally been told by a CDO that they cannot speak with me. My go-to phrase for open data is, "open data is a lie". Because at the end of the day, we have zero legal recourse to validate that what is provided through open datasets is complete, both in terms of available columns and available rows. Very rarely will it ever be explained why, or even if, information is missing. And the subsequent effects of that lead to a deep public misunderstanding of what's going on. And much of it is intentional out of explicit fear that the public will misinterpret the data. So we go through FOIA instead and we end up going through legal battles. There's a reason I've had to do ~10 FOIA suits. tl;dr: open data is nice, but its lack of rigor and accountability makes it effectively useless for anything that requires depth.

stevage 3y ago
Perhaps you missed the bit where I said I worked in open data for years, including inside government. They are two very different processes, yes. Open data tends to connect you with technologists and subject matter experts who want to talk about their field. FOI tends to connect you with lawyers and governance gurus who want to find reasons not to give you what you want. Around here the FOI processes are getting slower and less forthcoming, in defiance of the law, and ridiculous requests like these ones do not help make the case for their support.
- chaps 3y ago
  I've spoken to many people inside government who work on the open data side. To be blunt: the signal to noise ratio isn't very good in terms of whether someone actually knows what they're talking about. So please consider my reluctance to take much of what is said as a product of being lied to for years, both intentionally and unintentionally. Yes, FOI processes are getting slower and yes we're going to keep suing. What other choice do we have? One thing to consider is that SO MUCH of that is a product of gov agencies simply not preparing their documents in a way that makes them FOIAable. So much can be done that simply isn't -- a significant amount of which is very intentional (eg, police records) in an intentional violation of the law. We're not trying to be dicks in all of this. We just want to know what's going on. So yours (and many others in your position)'s clear desire to think of us as adversaries rather than simply people who want to know what the fuck is going on is blinding your vision. Get out of your head as a person in government and start thinking as a person who's at least empathetic towards the frustrations of a tired public that constantly lied to, deceived, and fed communications that are perpetually weighed down by legally reviewed PR releases. We can do better, but sure as hell not with your defeated attitude.
  - lazyasciiart 3y ago
    One thing to consider is that 'government' literally constantly advertises the opportunity for you to get credentials to their system and prepare documents themselves. They will pay you to do this. > So yours (and many others in your position)'s clear desire to think of us as adversaries rather than simply people who want to know what the fuck is going on is blinding your vision. Yea, I don't work in his position or any government position, but you are being adversarial. And, whatever you're trying to do, you're being a dick in at least this comment.

colinsane 3y ago

> I'm also pretty surprised that people think email metadata from government is legitimate open data. Do you think it should be public how many times you have emailed government and gotten a response, and to which departments? I don't. on the surface i agree with you. beneath the surface, other governmental units already possess all the metadata for your emails/phone calls, it would be a weird bit of asymmetry to say it’s okay for them to have your metadata but not for you to have their metadata. it’s not a perfect mirror, for obvious reasons, but if OP went on to show that this metadata is powerful and push that neither side should be collecting that type of metadata, then that’s a single legitimate use i can think of for this kind of FOIA.

stevage 3y ago
>it would be a weird bit of asymmetry to say it’s okay for them to have your metadata but not for you to have their metadata. But that's not what this is. This is saying it's ok for them to give your metadata out to anyone who asks.