4 ms·
> The response for all queries is formatted as a two dimensional JSON array where the first row provides column names and subsequent rows provide data values.
by jack-r-abbit 14y ago
> The response for all queries is formatted as a two dimensional JSON array where the first row provides column names and subsequent rows provide data values.
Hmmm... looking at the example of this I can't help but think there has got to be a better way. This is more just a standard CSV (first row is header, all other rows are data). Using JSON for data formatted as such is kind of a waste of JSON. If you pass that example into a JSON decoder you get an extremely more difficult to use object. Am I missing something? or is this just typical "the government doesn't do tech properly" stuff?
EDIT: sorry, I try to be less negative but sometimes it is hard. I do applaud them for at least making the info available. I don't have a use for it but if some else does then dealing with a stupid format is better than not having data at all.
EDIT2: (to clarify) I was not saying the data should just be CSV... I'm saying it is and that defeats the purpose of using JSON. They should still use JSON but with their data formatted differently. Using their example, properly closed but truncated to just to first 2 records, the PHP function json_decode() turns it into this array:
array (
0 => array (
0 => 'P0010001',
1 => 'NAME',
2 => 'state',
),
1 => array (
0 => '710231',
1 => 'Alaska',
2 => '02',
),
2 => array (
0 => '4779736',
1 => 'Alabama',
2 => '01',
),
)
I don't find that format to be very pleasant.
- tantalor 14y agoI also found this odd, but I can think of two reasons, 1. CSV is not a single well-defined format. Some people use rfc4180, but not everybody does. 2. JSON is easier to parse than CSV for JavaScript clients, i.e., browsers. On the other hand, JSON is not a subset of JavaScript.
- deleted 14y ago[deleted]
- pnathan 14y agoIt's very simple to transform that kind of data into $your_preferred_structure. It's also a very agnostic format.
- deleted 14y ago[deleted]
- bcherry 14y agoThis doesn't seem unreasonable. 1) Whether it's user-friendly or not, it's still JSON, which makes JSON-P possible (they support JSON-P). 2) It probably mirrors the way they store their data (in tables, whether SQL or Access or Excel, doesn't really matter). 3) It's more compact than traditional JSON, which means less bandwidth, which means less cost. Keep in mind that this is essentially a not-for-profit API from a not-for-profit organization with the worst possible budgeting scenario. Yes, Gzip would basically eliminate this benefit, but they don't have gzip enabled and enabling it may be difficult or impossible under whatever constraints they operate under. Also, it's entirely possible that this API has existed privately for a long, long time in a CSV format, and they simply made a minor enhancement to make it JSON and JSON-P compatible and open up the endpoints to the public.
- jack-r-abbit 14y ago#3 seems a likely reason. and like I said, having a harder to work with data structure is still better than nothing. and it is free.
- peter_l_downs 14y agoAll good explanations, but I can't help but wish they had chosen a format like: { P0010001 : [ '710231', '4779736', // ... etc. ], NAME : [ 'Alaska', 'Alabama', // ... etc. ], state: [ '02', '01, // ... etc. ], // ... etc. }
- asdfaoeu 14y agoWhy would that be better? You'd probably still have to convert it. And also that structure would be harder to convert to from a CSV format or a SQL query.
- tzs 14y agoI'm having trouble thinking of any situation where that would be more convenient or useful than the format they are using now. Where would your proposed format be an advantage?
- 14y ago
- lancefisher 14y agoFactual.com serves their data like this too. I thought it was odd at first, but it does save a lot of bytes on the wire. I just parsed it with JSON.parse() and used underscore to map to my object format. You don't really even need underscore for that. It's just a matter of mapping an array of rows to an array of objects. It's nice there is an API now.
- tzs 14y agoWith their format, it is an easy matter to mechanically turn the first item into: INSERT INTO table ('P0010001','NAME','state') VALUES (?,?,?) and then loop through the remaining items using that to insert them into a database.
- jackcviers3 14y agoOne of the talks given at fluent conf (on Google's feedback canvas screenshot system) last week talked about how moving data from object-based storage to array-based storage saved serialization time and memory use. I would guess that is why they did it this way.
- jack-r-abbit 14y agoI can't edit my original post anymore so I'll reply to myself. Having had some time to think about this I've changed my mind slightly. It seems that if you are getting various datesets with various unknown field names, you really don't have any other way to programmatically know what keys you would pull. Having the first row give the fields might make processing know datasets wonky, it is the unknown datasets that need the most help. I take back my original criticism. Thanks for all the insight.