4 ms·

I'm one of the editors on this — and would be happy to answer any questions... To address the obvious: It's a lot of data. A pixel (roughly) for each of the 25

I'm one of the editors on this — and would be happy to answer any questions... To address the obvious: It's a lot of data. A pixel (roughly) for each of the 255 industries we chose to cover the private sector, for each month, for each of the past ten years. Sadly, it may bring your browser to its knees. I've tried some tricks to try and keep things moving along smoothly, but if anyone has any brilliant perf. ideas that aren't already incorporated, I'm all ears.

devbug 12y ago
Can we get the raw data? (I assume you've collected and cleaned up the raw data from the bfs.)
- jashkenas 12y ago
  Sure. What you're asking for is ... The graphic itself is powered by CSV's of the relevant data, pub'd out here: Jobs: http://graphics8.nytimes.com/newsgraphics/2014/03/05/state-of-the-economy/42f91284773cf307436fcef82a49a9bdf176a391/data/ces.csv http://graphics8.nytimes.com/newsgraphics/2014/03/05/state-o... Wages: http://graphics8.nytimes.com/newsgraphics/2014/03/05/state-of-the-economy/42f91284773cf307436fcef82a49a9bdf176a391/data/wages.csv http://graphics8.nytimes.com/newsgraphics/2014/03/05/state-o... (Note that you'll want to adjust those wages for inflation, and those jobs for population growth, depending on what you're doing.) But our versions aren't really "cleaned up" in any meaningful way. The BLS already seasonally adjusts the jobs numbers for their CES series, which is super handy. Instead, go to the original source here: http://data.bls.gov/cgi-bin/dsrv?ce http://data.bls.gov/cgi-bin/dsrv?ce ... and walk through the wizard to pick which data points you'd like to include in your query. One big tip, once you're on the results screen, is to click on "More Formatting Options", and then choose the "Multi-series table" format. That'll give you something much easier to pipe into Excel or into a database. Enjoy!
  - devbug 12y ago
    Awesome! Thanks.
jacobolus 12y ago
Not a performance suggestion, but.. For the first few minutes, I found these charts confusing, because the horizontal axis is labeled “<- Lower Wages Industries Higher Wages ->”, but within each chart, it represents time. I’d recommend adding some kind of explanation/legend for this.
- jashkenas 12y ago
  You're quite right that we're double-coding along the X-axis (and the Y-axis as well to a lesser extent). The introduction and the "How to read this chart" tooltip should help explain that, as should the year legend within each mini-chart ... but if you have a concrete idea about where we could place additional explanation to make it clearer, let me know.
bicknergseng 12y ago
Obviously not a short term solution for this set of charts, but have you tried rendering this data on canvas? Looking at timeline, the framerate chokes whenever it has to render and recalc styles for all of those svg dom elements. A shorter term solution might be something like fabric.js's svg rasterizing http://fabricjs.com/svg-caching http://fabricjs.com/svg-caching (no promises... just a thought). I bet rendering this many points would really scream if you could use WebGL (though it would definitely be understandable if that doesn't have broad enough support for NYT). Really awesome work though... any chance you could blog about your tricks or the making of this particular article?

devbug 12y ago

Can we get the raw data? (I assume you've collected and cleaned up the raw data from the bfs.)

jashkenas 12y ago
Sure. What you're asking for is ... The graphic itself is powered by CSV's of the relevant data, pub'd out here: Jobs: http://graphics8.nytimes.com/newsgraphics/2014/03/05/state-of-the-economy/42f91284773cf307436fcef82a49a9bdf176a391/data/ces.csv http://graphics8.nytimes.com/newsgraphics/2014/03/05/state-o... Wages: http://graphics8.nytimes.com/newsgraphics/2014/03/05/state-of-the-economy/42f91284773cf307436fcef82a49a9bdf176a391/data/wages.csv http://graphics8.nytimes.com/newsgraphics/2014/03/05/state-o... (Note that you'll want to adjust those wages for inflation, and those jobs for population growth, depending on what you're doing.) But our versions aren't really "cleaned up" in any meaningful way. The BLS already seasonally adjusts the jobs numbers for their CES series, which is super handy. Instead, go to the original source here: http://data.bls.gov/cgi-bin/dsrv?ce http://data.bls.gov/cgi-bin/dsrv?ce ... and walk through the wizard to pick which data points you'd like to include in your query. One big tip, once you're on the results screen, is to click on "More Formatting Options", and then choose the "Multi-series table" format. That'll give you something much easier to pipe into Excel or into a database. Enjoy!
- devbug 12y ago
  Awesome! Thanks.

jacobolus 12y ago

Not a performance suggestion, but.. For the first few minutes, I found these charts confusing, because the horizontal axis is labeled “<- Lower Wages Industries Higher Wages ->”, but within each chart, it represents time. I’d recommend adding some kind of explanation/legend for this.

jashkenas 12y ago
You're quite right that we're double-coding along the X-axis (and the Y-axis as well to a lesser extent). The introduction and the "How to read this chart" tooltip should help explain that, as should the year legend within each mini-chart ... but if you have a concrete idea about where we could place additional explanation to make it clearer, let me know.

bicknergseng 12y ago

Obviously not a short term solution for this set of charts, but have you tried rendering this data on canvas? Looking at timeline, the framerate chokes whenever it has to render and recalc styles for all of those svg dom elements. A shorter term solution might be something like fabric.js's svg rasterizing http://fabricjs.com/svg-caching http://fabricjs.com/svg-caching (no promises... just a thought). I bet rendering this many points would really scream if you could use WebGL (though it would definitely be understandable if that doesn't have broad enough support for NYT). Really awesome work though... any chance you could blog about your tricks or the making of this particular article?