7 ms·

Making Open and Machine Readable the New Default for Government Information

180 points by BruceM 13y ago

charonn0 13y ago
>4(b) This order shall be implemented consistent with applicable law and subject to the availability of appropriations.< >4(c) This order is not intended to, and does not, create any right or benefit, substantive or procedural, enforceable at law or in equity by any party against the United States, its departments, agencies, or entities, its officers, employees, or agents, or any other person.< (emphases mine) Don't these provisions take the teeth out of the order?
- rayiner 13y ago
  Re: 4(c) Executive orders creating causes of action is a very hairy topic. In theory, executive orders are instructions by the President to the bureaucracy to guide the executive's exercise of discretion delegated by Congress. They are not laws and generally are not construed to affect the rights of third parties by creating causes of action. Re: 4(b), every action of the executive is subject to the availability of appropriations.
- forgotAgain 13y ago
  Yeah. Just another Obama speech really. If I try I can picture him reading the order with emotion and moral righteousness. I was impressed five years ago.
BruceM 13y ago
Comments from the Sunlight Foundation: http://sunlightfoundation.com/blog/2013/05/09/open-data-executive-order-shows-path-forward/ http://sunlightfoundation.com/blog/2013/05/09/open-data-exec...
danso 13y ago
This is an unsexy directive that could be a profound game-changer for civic society. If you've ever tried to get data from a public entity and had to argue about the particulars of PDF over Excel, this kind of statement is at least a good precedent to work from...some information will still be unparsable, just as much information, period, is still withheld despite transparency laws...but this is a pretty strong statement nonetheless. Edit: I do think there's a large risk of an unintended chilling effect, as well-intentioned as this initiative. The blog post that was linked to later in this discussion talks about how defaulting to machine data will "ensure privacy"...which can only mean that there will be some process to decide which data should be machine data and which shouldn't...and, I think this will inevitably cause some datasets to just not be released at all, if there are large technical concerns in making sure the data doesn't "violate privacy" As an example, a couple years ago the Obama administration took down a public database after a doctor complained that a newspaper was able to use it to track evidence against him in a story: http://www.propublica.org/article/how-complaints-from-a-doctor-caused-the-govt-to-take-down-a-public-database http://www.propublica.org/article/how-complaints-from-a-doct... The database contained anonymized data about doctor malpractice records...but of course, if you have a doctor with a particularly long history of malpractice in a s single zip code, it's not hard to connect those anonymous records to an identity. So I think there will be some cases where an agency will decide not to put up data because of the fear that computerized analysis of it will reveal things that they don't want revealed.
- edavis 13y ago
  > If you've ever tried to get data from a public entity and had to argue about the particulars of PDF over Excel ... Or my favorite, the entities that send a PDF export of an Excel spreadsheet.
  - dmd 13y ago
    Lucky. PennDOT sent me screenshots of a spreadsheet, pasted into a doc.
    - ErikAugust 13y ago
      Oh man. That is a winner.
      - tlrobinson 13y ago
        http://thedailywtf.com/Articles/Web_0_0x2e_1.aspx http://thedailywtf.com/Articles/Web_0_0x2e_1.aspx
  - rayiner 13y ago
    They probably have a standing rule that documents go out in PDF, for the simple reason that more people can view PDF files than can view Excel files.
  - ErikAugust 13y ago
    How about a Flash site that exports CSV. That's my favorite.
    - aptwebapps 13y ago
      Isn't that sort of the opposite as the parent, in terms of machine readability?
BruceM 13y ago
The White House also posted a blog entry about this with many additional links: http://www.whitehouse.gov/blog/2013/05/09/landmark-steps-liberate-open-data http://www.whitehouse.gov/blog/2013/05/09/landmark-steps-lib...
mindcrime 13y ago
Now we need to work on getting all 50 States, and the various county/municipal governments to do likewise...
- billybob255 13y ago
  Definitely a step in the right direction, but when I worked on federal contracting I used a VM because surfing through the different departments was wading through a sea of expired SSL certificates. In terms of basic IT management Uncle Sam is way behind the times.
- rz2k 13y ago
  Sadly, the Supreme Court just (unanimously) endorsed steps in the opposite direction.[1][2] The oral arguments included outrageous statements from the usual suspects, and what sounded almost like intentionally obtuse statements about the value of open government. Alito chortled in his written opinion, that with respect to freedom of information laws in general: "There is no contention that the Nation’s unity foundered in their absence, or that it is suffering now because of the citizens-only FOIA provisions that several States have enacted".[3] [1] https://www.eff.org/deeplinks/2013/05/scotus-foils-foia-advocates-challenge-citizens-only-records-laws https://www.eff.org/deeplinks/2013/05/scotus-foils-foia-advo... [2] http://www.opengovva.org/foi-blog-list/1680-my-public-record-im-mad-as-heck-at-mcburney-opinion http://www.opengovva.org/foi-blog-list/1680-my-public-record... [3] http://www.washingtonpost.com/politics/supreme-court-says-states-may-bar-information-requests-from-nonresidents/2013/04/29/37d41134-b105-11e2-bbf2-a6f9e9d79e19_story.html http://www.washingtonpost.com/politics/supreme-court-says-st...
  - gizmo686 13y ago
    I read the Court opinion [1] (which is a fairly easy read). The situation is that "Virginia’s Freedom of Information Act (FOIA) grants Virginia citizens access to all public records, but grants no such right to non Virginians." [1] The plaintiffs claimed that this law violated their constitutional rights. The court disagreed (and cited precedent). It is not the job of the court to decide weather or not a law is a good idea. It is only their job to decide weather or not it is constitutional. [1] http://www.supremecourt.gov/opinions/12pdf/12-17_d1o2.pdf http://www.supremecourt.gov/opinions/12pdf/12-17_d1o2.pdf
forgottenpaswrd 13y ago
Specially patents. Require all blueprints to be digitally accessible, with stl or step open formats for 3D shapes and svg for 2D. I continue seeing crappy drawings in new patents. What year is it? 1891? In 1891 they were patents better formatted than today.
- riffic 13y ago
  Patents will be treated in a special manner?
adelevie 13y ago
Worth noting: The White House released the memo on GitHub[1] and used GitHub Pages to publish[2]. [1] https://github.com/project-open-data/project-open-data.github.io/blob/master/policy-memo.md https://github.com/project-open-data/project-open-data.githu... [2] http://project-open-data.github.io/policy-memo/ http://project-open-data.github.io/policy-memo/
duncan_bayne 13y ago
I wonder when the White House will be offering an RSS feed of citizens they've executed extra-judicially? Or perhaps a social network graph linking lobbyists and politicians & their staff?
dmoo 13y ago
Its also useful to have the business of government open and machine readable. In Ireland we have recently had a change in how debate information from our Parliament was published forcing a complete overhaul of the website most people use to keep track of their public representatives. http://www.thejournal.ie/kildarestreet-com-back-from-the-dead-880278-Apr2013/ http://www.thejournal.ie/kildarestreet-com-back-from-the-dea...

180 points by BruceM 13y ago

>4(b) This order shall be implemented consistent with applicable law and subject to the availability of appropriations.< >4(c) This order is not intended to, and does not, create any right or benefit, substantive or procedural, enforceable at law or in equity by any party against the United States, its departments, agencies, or entities, its officers, employees, or agents, or any other person.< (emphases mine) Don't these provisions take the teeth out of the order?

rayiner 13y ago
Re: 4(c) Executive orders creating causes of action is a very hairy topic. In theory, executive orders are instructions by the President to the bureaucracy to guide the executive's exercise of discretion delegated by Congress. They are not laws and generally are not construed to affect the rights of third parties by creating causes of action. Re: 4(b), every action of the executive is subject to the availability of appropriations.
forgotAgain 13y ago
Yeah. Just another Obama speech really. If I try I can picture him reading the order with emotion and moral righteousness. I was impressed five years ago.

BruceM 13y ago

Comments from the Sunlight Foundation: http://sunlightfoundation.com/blog/2013/05/09/open-data-executive-order-shows-path-forward/ http://sunlightfoundation.com/blog/2013/05/09/open-data-exec...

danso 13y ago

This is an unsexy directive that could be a profound game-changer for civic society. If you've ever tried to get data from a public entity and had to argue about the particulars of PDF over Excel, this kind of statement is at least a good precedent to work from...some information will still be unparsable, just as much information, period, is still withheld despite transparency laws...but this is a pretty strong statement nonetheless. Edit: I do think there's a large risk of an unintended chilling effect, as well-intentioned as this initiative. The blog post that was linked to later in this discussion talks about how defaulting to machine data will "ensure privacy"...which can only mean that there will be some process to decide which data should be machine data and which shouldn't...and, I think this will inevitably cause some datasets to just not be released at all, if there are large technical concerns in making sure the data doesn't "violate privacy" As an example, a couple years ago the Obama administration took down a public database after a doctor complained that a newspaper was able to use it to track evidence against him in a story: http://www.propublica.org/article/how-complaints-from-a-doctor-caused-the-govt-to-take-down-a-public-database http://www.propublica.org/article/how-complaints-from-a-doct... The database contained anonymized data about doctor malpractice records...but of course, if you have a doctor with a particularly long history of malpractice in a s single zip code, it's not hard to connect those anonymous records to an identity. So I think there will be some cases where an agency will decide not to put up data because of the fear that computerized analysis of it will reveal things that they don't want revealed.

edavis 13y ago
> If you've ever tried to get data from a public entity and had to argue about the particulars of PDF over Excel ... Or my favorite, the entities that send a PDF export of an Excel spreadsheet.
- dmd 13y ago
  Lucky. PennDOT sent me screenshots of a spreadsheet, pasted into a doc.
  - ErikAugust 13y ago
    Oh man. That is a winner.
    - tlrobinson 13y ago
      http://thedailywtf.com/Articles/Web_0_0x2e_1.aspx http://thedailywtf.com/Articles/Web_0_0x2e_1.aspx
- rayiner 13y ago
  They probably have a standing rule that documents go out in PDF, for the simple reason that more people can view PDF files than can view Excel files.
- ErikAugust 13y ago
  How about a Flash site that exports CSV. That's my favorite.
  - aptwebapps 13y ago
    Isn't that sort of the opposite as the parent, in terms of machine readability?

BruceM 13y ago

The White House also posted a blog entry about this with many additional links: http://www.whitehouse.gov/blog/2013/05/09/landmark-steps-liberate-open-data http://www.whitehouse.gov/blog/2013/05/09/landmark-steps-lib...

mindcrime 13y ago

Now we need to work on getting all 50 States, and the various county/municipal governments to do likewise...

billybob255 13y ago
Definitely a step in the right direction, but when I worked on federal contracting I used a VM because surfing through the different departments was wading through a sea of expired SSL certificates. In terms of basic IT management Uncle Sam is way behind the times.
rz2k 13y ago
Sadly, the Supreme Court just (unanimously) endorsed steps in the opposite direction.[1][2] The oral arguments included outrageous statements from the usual suspects, and what sounded almost like intentionally obtuse statements about the value of open government. Alito chortled in his written opinion, that with respect to freedom of information laws in general: "There is no contention that the Nation’s unity foundered in their absence, or that it is suffering now because of the citizens-only FOIA provisions that several States have enacted".[3] [1] https://www.eff.org/deeplinks/2013/05/scotus-foils-foia-advocates-challenge-citizens-only-records-laws https://www.eff.org/deeplinks/2013/05/scotus-foils-foia-advo... [2] http://www.opengovva.org/foi-blog-list/1680-my-public-record-im-mad-as-heck-at-mcburney-opinion http://www.opengovva.org/foi-blog-list/1680-my-public-record... [3] http://www.washingtonpost.com/politics/supreme-court-says-states-may-bar-information-requests-from-nonresidents/2013/04/29/37d41134-b105-11e2-bbf2-a6f9e9d79e19_story.html http://www.washingtonpost.com/politics/supreme-court-says-st...
- gizmo686 13y ago
  I read the Court opinion [1] (which is a fairly easy read). The situation is that "Virginia’s Freedom of Information Act (FOIA) grants Virginia citizens access to all public records, but grants no such right to non Virginians." [1] The plaintiffs claimed that this law violated their constitutional rights. The court disagreed (and cited precedent). It is not the job of the court to decide weather or not a law is a good idea. It is only their job to decide weather or not it is constitutional. [1] http://www.supremecourt.gov/opinions/12pdf/12-17_d1o2.pdf http://www.supremecourt.gov/opinions/12pdf/12-17_d1o2.pdf

forgottenpaswrd 13y ago

Specially patents. Require all blueprints to be digitally accessible, with stl or step open formats for 3D shapes and svg for 2D. I continue seeing crappy drawings in new patents. What year is it? 1891? In 1891 they were patents better formatted than today.

riffic 13y ago
Patents will be treated in a special manner?

adelevie 13y ago

Worth noting: The White House released the memo on GitHub[1] and used GitHub Pages to publish[2]. [1] https://github.com/project-open-data/project-open-data.github.io/blob/master/policy-memo.md https://github.com/project-open-data/project-open-data.githu... [2] http://project-open-data.github.io/policy-memo/ http://project-open-data.github.io/policy-memo/

duncan_bayne 13y ago

I wonder when the White House will be offering an RSS feed of citizens they've executed extra-judicially? Or perhaps a social network graph linking lobbyists and politicians & their staff?

dmoo 13y ago

Its also useful to have the business of government open and machine readable. In Ireland we have recently had a change in how debate information from our Parliament was published forcing a complete overhaul of the website most people use to keep track of their public representatives. http://www.thejournal.ie/kildarestreet-com-back-from-the-dead-880278-Apr2013/ http://www.thejournal.ie/kildarestreet-com-back-from-the-dea...