3 ms·

It seems like most federal and state agencies nowadays make much data - data that they are forced by law to make public - only available in pdf files. The reaso

It seems like most federal and state agencies nowadays make much data - data that they are forced by law to make public - only available in pdf files. The reason for this seems to be to prevent as much as possible use and analysis of the data, while technically complying with public disclosure requirements. It is often extremely difficult to get public data in a usable form out of most public web sites, and the ways in which this happen are so unoptimal they move past the level of mere government incompetence to intentional obfuscation.

brudgers 14y ago
Any format other than PDF and the large numbers of citizens will not have the proper tool to access it - e.g. a person using a public computer in a library. It's not malice even if it hampers outside researchers. Any format other than printed (e.g. PDF) is going to require the user to deal with the meta-data in order to extract useful information - that's just the nature of data.
- amboar 14y ago
  They could publish the data in multiple formats, PDF and something actually useful if you're worried about someone at a library not having the tools. The point is, don't cripple the data from an analysis perspective.
SilasX 14y ago
Wait, how come they release newly-public data to traders on financial markets in easily computer-parseable format like JSON, but PDFs with tables for so much of this stuff? You'd almost think it's an issue of priorities.
BrianEatWorld 14y ago
Having worked for the US government, trying to collect trade and production data from other governments and private entities, I always assumed this was the case when I received PDFs. Interestingly, interest groups were always eager to give you the data in whatever format you requested...
gamblor956 14y ago
The law requires state and federal agencies to make the data public. It does not require them to make the data useful for a specific use case. But generally, the reasons PDF are used is: (1) PDF signals (internally) that the file has been examined from sensitive, confidential, or non-public content, and no longer needs further modification; (2) PDF is a long-lived standard with plenty of free viewers; and (3) PDF guarantees print fidelity across all known viewers, which is something most office software (including online suites) can't even promise between minor version updates.

brudgers 14y ago

Any format other than PDF and the large numbers of citizens will not have the proper tool to access it - e.g. a person using a public computer in a library. It's not malice even if it hampers outside researchers. Any format other than printed (e.g. PDF) is going to require the user to deal with the meta-data in order to extract useful information - that's just the nature of data.

amboar 14y ago
They could publish the data in multiple formats, PDF and something actually useful if you're worried about someone at a library not having the tools. The point is, don't cripple the data from an analysis perspective.

SilasX 14y ago

Wait, how come they release newly-public data to traders on financial markets in easily computer-parseable format like JSON, but PDFs with tables for so much of this stuff? You'd almost think it's an issue of priorities.

BrianEatWorld 14y ago

Having worked for the US government, trying to collect trade and production data from other governments and private entities, I always assumed this was the case when I received PDFs. Interestingly, interest groups were always eager to give you the data in whatever format you requested...

gamblor956 14y ago

The law requires state and federal agencies to make the data public. It does not require them to make the data useful for a specific use case. But generally, the reasons PDF are used is: (1) PDF signals (internally) that the file has been examined from sensitive, confidential, or non-public content, and no longer needs further modification; (2) PDF is a long-lived standard with plenty of free viewers; and (3) PDF guarantees print fidelity across all known viewers, which is something most office software (including online suites) can't even promise between minor version updates.