4 ms·
I'll remain skeptical about OMB's ability to know about all the datasets from federal agencies, because as you said, there is a wealth of government data that i
by esonderegger 12y ago
I'll remain skeptical about OMB's ability to know about all the datasets from federal agencies, because as you said, there is a wealth of government data that is "public" but obscure.
Another example is the Department of Defense's budget justification data. The PDF documents that go into the annual President's budget are made public and are easily found on agency websites. They look like this:
http://www.saffm.hq.af.mil/shared/media/document/AFD-150130-006.pdf http://www.saffm.hq.af.mil/shared/media/document/AFD-150130-...
What the public would have no way of knowing, though, is that attached to those documents are files with the extension of *.zzz. Those are really zip files, but need to be renamed because DoD computer systems tend to view anything with a zip extension as toxic. Inside those zip files are the xml files used to create the PDF document. The data is both clean and comprehensive.
I'm writing a set of scrapers that will download the publicly available PDF files and extract the XML, so if someone wanted to import them into their own database for querying, they could. I really wish that weren't necessary, though.
Edit: I should mention that those xml attachments only exist for the Procurement and RDT&E budgets, which account for about a third of total DoD spending.
- philipashlock 12y agoFor what it's worth, OMB is overseeing the Executive Order that requires agencies to compile these inventories. You can find more background at: - https://project-open-data.cio.gov/ https://project-open-data.cio.gov/ - http://www.digitalgov.gov/resources/how-to-get-your-open-data-on-data-gov/#federal-data-with-project-open-data http://www.digitalgov.gov/resources/how-to-get-your-open-dat... And you can see how agencies are currently performing at http://labs.data.gov/dashboard/offices http://labs.data.gov/dashboard/offices Can you point to some examples or links for the .zzz files you mentioned?