7 ms·
United States Census Bureau APIs
- staticautomatic 9y agoLooking forward to checking this out. Scraping census data is exhausting. Just navigating the available tables is hard.
- minimaxir 9y agoWhere do you go to register for an API key?
- heyheyhey 9y agohttps://api.census.gov/data/key_signup.html https://api.census.gov/data/key_signup.html
- yeukhon 9y agoWhere do you see API key mentioned? I thought https://www.census.gov/data/developers/updates/new-discovery-tool.html https://www.census.gov/data/developers/updates/new-discovery... is good enough?
- sova 9y agohttp://api.census.gov/data/key_signup.html http://api.census.gov/data/key_signup.html or on the Discovery Tool Page
- chasedehan 9y agoStoked for this. the GUI has been terrible
- ashurov 9y agohave already been using this for a while.. great improvement and easy to use.
- jackgolding 9y agoReally hope Australia gets this some time!
- danso 9y agoFWIW, this has been around for a couple of years and has felt pretty solid, i.e. no obvious rate limit when bulk data gathering. For example, I believe that if you want to collect stats at the census tract/block level, you'd have to loop through every state, which is not a problem. County-level stats can be gathered in a single call, IIRC. The API seems to be the basis for the Census's new data platform: https://data.census.gov/ https://data.census.gov/
- duderific 9y agoInterestingly, if you click on any of the links on the "example" page [1], it seems to return results, even without an API key. Conveniently, it strips off the url param "YOUR_API_KEY_HERE" for you. Not sure what to make of the results, though. [1] https://api.census.gov/data/2010/sf1/examples.html https://api.census.gov/data/2010/sf1/examples.html
- gwright 9y agoOne reason to get an API key is that they use the associated email to send out (very low volume) messages about API "news".
- rogerhoward 9y agoI've worked with this API a fair bit since early 2016, and it's been around longer. I've got no real complaints about the API, just want to note that the data itself is.. arcane.. it'll take a bit of study and research to get up to speed on even a small subset of what's available.
- strictnein 9y agoJust to add to that: I spent a fair amount of time with this data on a side project and you really need to dedicate yourself to understanding what this data is and isn't, and how it gets updated.
- AznHisoka 9y agoIs the data out of date? or is it fairly recent (ie at least 2016)
- rogerhoward 9y agoThe main census is only every ten years, but there are other datasets updates more often, as well as extrapolated data. The issue for me wasn’t data being out of date, it’s just that the structure and content is fairly arcane if you’re not well versed in census - I assume much of the structure has been carried forward over the decades as there’s an obvious need to make sure data is comparable across different census.
- pc86 9y agoWhile awkward, "censuses" is indeed the correct plural of census.
- cepth 9y agoThe main datasets that are tied to a "geography" are the American Community Survey (ACS) and the Census. The ACS comes in 1, 3, and 5 year editions. The 1-year is the least precise, and has the smallest sample, while the 5-year uses trailing 5-year data. The tradeoff is that ACS-5 estimates are only available, IIRC, for geographies of 50,000 or more people. The ACS-1 estimates for 2016 are already released. ACS-5 will come out in December.
- japhyr 9y agoWe've recently seen the removal of some publicly-available datasets that are overseen by the federal government. Does anyone have any perspective on how stable this particular dataset is?
- danso 9y agoThe future of the Census, including 2020, is indeed in question: http://www.npr.org/sections/codeswitch/2017/07/15/536908867/could-a-census-without-a-leader-spell-trouble-in-2020 http://www.npr.org/sections/codeswitch/2017/07/15/536908867/... Maybe that means API access is one of the line-item things that could be cut when the budget isn't properly funded. But the data itself, including current datasets (which have been downloadable via bulk FTP for awhile) [0] should be pretty stable. After all, the Census is a function directly mandated by the Constitution. [0] https://www.census.gov/programs-surveys/acs/data/data-via-ftp.html https://www.census.gov/programs-surveys/acs/data/data-via-ft...
- spaceisballer 9y agoIt’s good to see the Census Bureau getting coverage. And it is true we don’t have a director, however we have a great deal of staff who have at least one to two Censuses under their belts. The real issue aside from leadership is funding. All through the planning stages we’ve had to cut important tests because of Congress and their inability to pass a budget.
- craftyguy 9y ago> The future of the Census, including 2020, is indeed in question The outgoing director doesn't seem worried.. from your link, when asked if he was worried about there being a leadership vacuum after his departure: > (laughs) No, the Census is much more than the director
- danso 9y agoMaybe I should have picked a different link that discussed the bureau's overall issues, particularly anticipated budgetary shortfalls: https://www.nytimes.com/2017/07/17/opinion/census-trump-budget-cuts.html https://www.nytimes.com/2017/07/17/opinion/census-trump-budg... Yes, the Census is more than its leader, but losing its longtime leader and seemingly not having anyone new (there is a new acting director, IIRC) in place this close to the next census seems worrying.
- dhuramas 9y agoHave been using it through R using the excellent https://github.com/walkerke/tigris https://github.com/walkerke/tigris for some geospatial data visualization. No complaints.
- cepth 9y agoThe censusapi package is also worth a look: https://github.com/hrecht/censusapi https://github.com/hrecht/censusapi In my experience, it provides much more granular access to Census API endpoints vs tigris. I.e., you can explore datasets not tied to a specific geography.
- calvinbhai 9y agoAnyone know if the APIs give info about spoken languages? I feel non hispanic immigrants are a huge variety and was wondering if this aspect is available through the API. A quick search through the API docs didn't reveal anything.
- eindiran 9y agoThere is this: https://www.census.gov/data/developers/data-sets/language-stats.html https://www.census.gov/data/developers/data-sets/language-st...
- brailsafe 9y agoMike Bostock of D3 puts this API to use in his somewhat esoteric but very impressive Command Line Cartography tutorial series. https://medium.com/@mbostock/command-line-cartography-part-1-897aa8f8ca2c https://medium.com/@mbostock/command-line-cartography-part-1...
- amyjess 9y agoI've had fun with the API. I used it to download a bunch of data that I then fed into a system to correlate various things and display information for different parts of the country at several levels. One of these days, I'll need to start it up again, set up a robots.txt to make sure the Googlebot doesn't crawl it (it has a lot of content generated on demand, including vector maps from shapefile data), and post it here.
- nulagrithom 9y agoWhen I clicked the "Mailing List" button and signed up for the newsletter I was also given the option to subscribe to a lot of government related email lists, including those local to my state. Thought that was pretty neat -- I had no idea such a thing existed.
- komali2 9y agoI did that once, years ago. Careful, the government is really good at sending an absurd volume of emails.
- godzillabrennus 9y agoAdd that to the long list absurdities government is good at.
- mooreds 9y agoI haven't had a chance to look at this yet, but I did some census data exploration a few years ago and it was ... baroque at the time. More here: http://www.mooreds.com/wordpress/archives/963 http://www.mooreds.com/wordpress/archives/963 If things haven't changed, I'd echo the comments that getting the data is only the first and in some ways easiest step. Understanding the data is much more important and a deeper process.
- dbatten 9y agoI really wish the Census would publish more normalized data sources. For example, the American Community Survey summary file is effectively a giant table with thousands of rows (representing different geographies) and thousands of columns (representing different statistics that have been calculated for those geographies). The columns just have an ID and a name that tells you what's in there, but it will be something like "median income for black or african american women with a master's degree." If you want that data point, you basically have to go searching through a bunch of docs (or one of their horrendous online interfaces) to find out the ID of the column. Instead, I'd love to have a set of tables... in each one, you'd have a geography column and then, using my above example, a race column, a gender column, an educational attainment column and a median income column. Then you could easily load that data to a database and run a SELECT query over the information to get what you need. Heck, I'd even settle for metadata that was good enough to enable me to construct this stuff myself. For example, for each column ID, tell me what variables were used to construct it and what their values were equal to. Unfortunately I've contacted the Census Bureau and they've told me such a thing doesn't exist.
- esbranson 9y agoSDMX was designed by statistical agencies for this purpose. Instead they created a non-standard API. (SDMX has a standardized REST API.) Next they will create a mobile app.