4 ms·

I'm one of the developers. This site is all about making public data about places more accessible. We have pages all the way from the national level all the way

by jrd79 12y ago

I'm one of the developers. This site is all about making public data about places more accessible. We have pages all the way from the national level all the way down to individual city blocks, and everything in between. The data itself is mostly demographics: age, sex, race & ethnicity, income, employment, and education. We separate by entity and by topic and each page has sections for the stats within the entity, comparisons of child entities, and comparisons of the entity with peer entities. Everything is extensively cross-linked, both in the maps, and in navigational lists. The data came primarily from the Census Bureau's American Community Survey, and the 2010 Census.

gajomi 12y ago
This is really great. I was pleasantly surprised to see that there was information available about household income at the "tract" level. I had searched some months ago for a data set that exposed high resolution income/poverty statistics at this resolution for Chicago, but could never find much beyond neighborhoods, or as anonymized sets. But I wasn't until now aware of the firehouse that is http://factfinder.census.gov/ http://factfinder.census.gov/. I assume the lack of an API for automating calls to your wonderfully structured data is to comply with the TOS of the census.gov site. Or is this for some other reason?
- jrd79 12y ago
  Thanks. We may build an API at some point, but it is not to do with the TOS. The raw data is so big (and we use memory mapped files, not a DB) that it is a pain from an infrastructure perspective to have available on a web box.
aw3c2 12y ago
Can you tell us a bit about the software that drives this both back- and frontend? It looks great!
- jacobn 12y ago
  The site was developed in Scala + Java, and the dev-version uses Play Framework, but the public site is statically hosted on S3 + CloudFront. Each page takes O(10 seconds) to generate and the machine that does the generation needs ~20+ GB of RAM, so for ops reasons we pregenerated all the content. (I'm one of the devs)

by jrd79 12y ago

gajomi 12y ago

This is really great. I was pleasantly surprised to see that there was information available about household income at the "tract" level. I had searched some months ago for a data set that exposed high resolution income/poverty statistics at this resolution for Chicago, but could never find much beyond neighborhoods, or as anonymized sets. But I wasn't until now aware of the firehouse that is http://factfinder.census.gov/ http://factfinder.census.gov/. I assume the lack of an API for automating calls to your wonderfully structured data is to comply with the TOS of the census.gov site. Or is this for some other reason?

jrd79 12y ago
Thanks. We may build an API at some point, but it is not to do with the TOS. The raw data is so big (and we use memory mapped files, not a DB) that it is a pain from an infrastructure perspective to have available on a web box.

aw3c2 12y ago

Can you tell us a bit about the software that drives this both back- and frontend? It looks great!

jacobn 12y ago
The site was developed in Scala + Java, and the dev-version uses Play Framework, but the public site is statically hosted on S3 + CloudFront. Each page takes O(10 seconds) to generate and the machine that does the generation needs ~20+ GB of RAM, so for ops reasons we pregenerated all the content. (I'm one of the devs)