6 ms·

I downloaded the first year dataset and didn't get very far in my analysis... but this was the first result, which was kind of pretty: http://i.imgur.com/ov6K6

by eck 11y ago

I downloaded the first year dataset and didn't get very far in my analysis... but this was the first result, which was kind of pretty: http://i.imgur.com/ov6K6mt.jpg http://i.imgur.com/ov6K6mt.jpg

dankohn1 11y ago
Beautiful! Love that LaGuardia and JFK glow.
amdlla 11y ago
Where would one start learning methods of data vis like this?
- aw3c2 11y ago
  That is just each point plotted on a map. So, check out software that can make maps? QGIS is nice.
  - minimaxir 11y ago
    You don't even need dedicated mapping programs. It's just latitudes and longitudes, so any 2D plotting program would be sufficient (as long as the data can fit into memory!). I made this map awhile ago for Instagram photos using R/ggplot2: http://i.imgur.com/IvGox1f.png http://i.imgur.com/IvGox1f.png Although the country boundaries aren't strictly necessary. Since there appears to be a demand for generating these maps, I'll work on a tutorial for this NYC data set.
    - misiti3780 11y ago
      you didnt have to install anything but ggplot2 to build that. wanna share the source :)
    - aw3c2 11y ago
      You murdered Greenland .D
  - myth_buster 11y ago
    Apparently, they didn't plot it on the map but on a grid. The density of data gives the well defined contour; a clever hack!
    - aw3c2 11y ago
      It would look 99% the same if you just plotted the coordinates with some (smartly chosen) transparency
stillmotion 11y ago
I would love to buy a wall print like this.
- verelo 11y ago
  This really is beautiful and I agree, if you do not want to sell it, is there any chance you could share a higher res image?
  - eck 11y ago
    The trouble with making a higher res image is that when you make that lat/lon buckets smaller, you have fewer samples in each one, so the image gets noisier. For the best possible image you'd want to download all the years of data. The process of making it was quite simple. I zeroed a 2d array of integers, then took all the pickup/dropoff points and incremented the nearest cell. The pixel values are based on the logarithm of the counts, since otherwise everything outside midtown would be pretty much black. There are some artifacts, like the thin vertical line down the east river. I think that was because of how the data was rounded, i.e. the number of unique longitude values that map to a certain image column. I wrote this myself with a few hundred lines of C++, though I'm sure there's GIS software out there that will do all this for you with a few clicks.
    - zippzom 11y ago
      Can you explain why there is a line going to the airport? Shouldn't the airport be more of an island of light (since I imagine most people aren't getting picked up / dropped off a half mile from the airport)? Also did you overlay it onto a map? How did you get the angled effect if it's just a grid?
      - eck 11y ago
        > Can you explain why there is a line going to the airport? I assume it is because there is a fixed fare to/from JFK so drivers have little incentive to start/stop the meter at the exact pickup/dropoff location. > Also did you overlay it onto a map? No. If taxis did not pick up or drop off people on some street, that street does not appear. For example there is an area downtown where there are streets but they have had security barriers since 9/11 thus no taxis. > How did you get the angled effect if it's just a grid? None of NYCs grids are exactly north/south/east/west aligned.
myth_buster 11y ago
This is quite beautiful! I was thinking of doing something similar when I saw the post but well... Could you give some info on your stack?
misiti3780 11y ago
very nice - how did you create it?
pavel_lishin 11y ago
This would be even more beautiful if animated.
AndrewConn 11y ago
Love this. I took a few minutes and converted your image into an Illustrator/vector file. Not 100% true to your original, but pretty good. Infinitely scalable so people can print it if they want. Preview in black and white: http://i.imgur.com/OhemZK2.jpg http://i.imgur.com/OhemZK2.jpg Bonus! preview in Uber blue: http://i.imgur.com/gvFDtWN.jpg http://i.imgur.com/gvFDtWN.jpg Source AI files: https://www.dropbox.com/s/zw1oqdwhaqhqb6u/NYC%20Taxi%20Data.zip?dl=0 https://www.dropbox.com/s/zw1oqdwhaqhqb6u/NYC%20Taxi%20Data....

by eck 11y ago

I downloaded the first year dataset and didn't get very far in my analysis... but this was the first result, which was kind of pretty: http://i.imgur.com/ov6K6mt.jpg http://i.imgur.com/ov6K6mt.jpg

dankohn1 11y ago

Beautiful! Love that LaGuardia and JFK glow.

amdlla 11y ago

Where would one start learning methods of data vis like this?

aw3c2 11y ago
That is just each point plotted on a map. So, check out software that can make maps? QGIS is nice.
- minimaxir 11y ago
  You don't even need dedicated mapping programs. It's just latitudes and longitudes, so any 2D plotting program would be sufficient (as long as the data can fit into memory!). I made this map awhile ago for Instagram photos using R/ggplot2: http://i.imgur.com/IvGox1f.png http://i.imgur.com/IvGox1f.png Although the country boundaries aren't strictly necessary. Since there appears to be a demand for generating these maps, I'll work on a tutorial for this NYC data set.
  - misiti3780 11y ago
    you didnt have to install anything but ggplot2 to build that. wanna share the source :)
  - aw3c2 11y ago
    You murdered Greenland .D
- myth_buster 11y ago
  Apparently, they didn't plot it on the map but on a grid. The density of data gives the well defined contour; a clever hack!
  - aw3c2 11y ago
    It would look 99% the same if you just plotted the coordinates with some (smartly chosen) transparency

stillmotion 11y ago

I would love to buy a wall print like this.

verelo 11y ago
This really is beautiful and I agree, if you do not want to sell it, is there any chance you could share a higher res image?
- eck 11y ago
  The trouble with making a higher res image is that when you make that lat/lon buckets smaller, you have fewer samples in each one, so the image gets noisier. For the best possible image you'd want to download all the years of data. The process of making it was quite simple. I zeroed a 2d array of integers, then took all the pickup/dropoff points and incremented the nearest cell. The pixel values are based on the logarithm of the counts, since otherwise everything outside midtown would be pretty much black. There are some artifacts, like the thin vertical line down the east river. I think that was because of how the data was rounded, i.e. the number of unique longitude values that map to a certain image column. I wrote this myself with a few hundred lines of C++, though I'm sure there's GIS software out there that will do all this for you with a few clicks.
  - zippzom 11y ago
    Can you explain why there is a line going to the airport? Shouldn't the airport be more of an island of light (since I imagine most people aren't getting picked up / dropped off a half mile from the airport)? Also did you overlay it onto a map? How did you get the angled effect if it's just a grid?
    - eck 11y ago
      > Can you explain why there is a line going to the airport? I assume it is because there is a fixed fare to/from JFK so drivers have little incentive to start/stop the meter at the exact pickup/dropoff location. > Also did you overlay it onto a map? No. If taxis did not pick up or drop off people on some street, that street does not appear. For example there is an area downtown where there are streets but they have had security barriers since 9/11 thus no taxis. > How did you get the angled effect if it's just a grid? None of NYCs grids are exactly north/south/east/west aligned.

myth_buster 11y ago

This is quite beautiful! I was thinking of doing something similar when I saw the post but well... Could you give some info on your stack?

misiti3780 11y ago

very nice - how did you create it?

pavel_lishin 11y ago

This would be even more beautiful if animated.

AndrewConn 11y ago

Love this. I took a few minutes and converted your image into an Illustrator/vector file. Not 100% true to your original, but pretty good. Infinitely scalable so people can print it if they want. Preview in black and white: http://i.imgur.com/OhemZK2.jpg http://i.imgur.com/OhemZK2.jpg Bonus! preview in Uber blue: http://i.imgur.com/gvFDtWN.jpg http://i.imgur.com/gvFDtWN.jpg Source AI files: https://www.dropbox.com/s/zw1oqdwhaqhqb6u/NYC%20Taxi%20Data.zip?dl=0 https://www.dropbox.com/s/zw1oqdwhaqhqb6u/NYC%20Taxi%20Data....