10 ms·

Using Deep Learning and Google Street View to Estimate Demographic Makeup of US

145 points by phodo 10y ago

Animats 10y ago
Or you could just order that data from Experian Automotive, which has vehicle registration data from all US states.
- backpropaganda 10y ago
  But I need to get a job at one of the big tech companies, and so I need to show them I can do deep learning.
  - NTDF9 10y ago
    Doesn't matter! You'll still be asked to write perfect code, with awesome variable names, on a whiteboard, with eyes closed. Don't forget to do handstands to stand out from the crowd.
    - narrator 10y ago
      Don't forget to practice doing math like 36^5 in your head. No pencil or paper allowed!
      - draugadrotten 10y ago
        If you would need to practice that, the job is not for you.
      - logfromblammo 10y ago
        Well, let's see, 36 is 00100100 in binary. Binomial coefficients are 1 5 10 10 5 1. 2^25 + 5 * 2^22 + 10 * 2^19 + 10 * 2^16 + 5 * 2^13 + 2^10. 10 is 5 * 2, so bit shift those middle two. 5 is binary 0101, so add copies bit shifted by 2. 2^25 + 2^24 + 2^22 + 2^22 + 2^20 + 2^19 + 2^17 + 2^15 + 2^13 + 2^10. Comes out to 0011 1001 1010 1010 0100 0000 0000 0x039AA400 Which is a perfectly cromulent numeric response. If the interviewer asks what that is in decimal, they ought to be glad I don't have a pencil, because it would end up in someone's eye on my way out the door.
        tropo 10y ago
        Me as interviewer: have you heard of the XOR operator?
        logfromblammo 10y ago
        Yeah, it looks like a plus sign with a circle around it. It's not in the ASCII character set. Sometimes, programming languages will use the caret character to represent XOR for boolean types and bitwise XOR for integral types, perhaps using two asterisks for exponentiation, or leaving it as a named function call, like Math.Exp or Math.Pow. But you have to be careful there, because sometimes Math.Exp is actually the inverse of the natural log function, rather than exponentiation. If caret means XOR, then 36 ^ 5 is 0010 0001, or 33. And that is the point where the interviewer marks me as "do not hire", because they can't stand it when they can't show off how smart they are to the candidate.
- nl 10y ago
  But (I assume) that only has the home address of the car. This method can be clearly extended to do things like measuring the socio-economic features of people parked at events etc.
- Bedon292 10y ago
  It seems like the idea is to use this method places outside of the US, which may not have the same level of data available.
- tmalsburg2 10y ago
  I once attended a talk in which the speaker presented a system for inferring the orientation of a mobile device in space. This was before smart phones and the system used accelerometers, some computer vision, and fancy algorithms which were running on a server because the device didn't have enough computing power. One person in the audience asked: Why don't you simply use a compass? The answer was silence.
- xapata 10y ago
  Nah, that data is weeks old. Some states are as slow as 3 months to hand the registration data to Experian. Better to scrape classifieds ads to see what people are selling ;-)
nl 10y ago
This is a great piece of work. I wonder if it would be possible to use a similar approach but on shop signs. Intuitively there should be some kind of relationship between the type of shops in different areas, but I guess it would be much less dense data than car type. I guess the next step is to make a completely end-to-end version without the manual feature engineering.
- colinbartlett 10y ago
  There was a similar discussion[1] here about that a while back using data about different business types to assert gentrification levels. 1. https://news.ycombinator.com/item?id=10391753 https://news.ycombinator.com/item?id=10391753
Doctor_Fegg 10y ago
> We focus on the motor vehicles encountered during this journey because over 90% of American households own a motor vehicle This is a clever hack, but I sorely hope no actual policy decisions are made on the basis of a methodology that wilfully ignores the 10% who choose not to own, or can't afford, a car.
- blowski 10y ago
  Can there possibly be a methodology that is 100% correct? If you try to interview 100% of people, then the results will be out of date by the time you've reached everybody. (Plus the Hawthorne effect suggests people will start changing their answers as people comment on the questions they are asked.) So you create samples, and necessarily some of those samples will be unrepresentative because they will miss some nuance. Catch 22 - unless you interview 100% of the people, you can't know what all the nuances are. I guess the best we can hope for is a result that merges lots of different results using lots of different methodologies, and base decisions on that.
  - deleted 10y ago
    [deleted]
d--b 10y ago
Dammit AI people, wake up! This is SO backwards! AI-based profiling of anything is bound to filter off outliers. Censuses are made in the field for a reason. The goal is to gather data so that you can make statistical reasoning on it. Not the opposite!!!! Here, they just gather some car data, and infer demographic data from it. And then what? We've just created a population which matches the car sample. We can't draw ANY conclusion from that data, beyond the type of cars that are found in such and such neighborhoods.
- paganel 10y ago
  In my parts of the world (Eastern Europe) you can infer a lot more about the people inhabiting a certain area by looking at the cars they drive, I mean, compared to census and income/tax data. That's because there are lots of way of not telling the authorities how much you earn, but once it comes to purchasing a car you leave aside such niceties and you get the car that best fits your real economic status (with few exceptions).
  - d--b 10y ago
    Yes, of course there is obviously some correlation between demographics and car ownership. But this paper makes it seem that their method is good to replace a in-the-field census. Basically, replacing all demographic data that the census brings with the single factor of car ownership. In the region where the actual correlation is less than 90%, the data generated is completely useless.
    - paganel 10y ago
      > But this paper makes it seem that their method is good to replace a in-the-field census. Ah, I missed that, sorry! To be honest I only skimped through the description, for other lazy people like me here's the relevant line: > Our results suggest that automated systems for monitoring demographic trends may effectively complement labor-intensive approaches, with the potential to detect trends with fine spatial resolution, in close to real time. which I also find as a little bit crazy. Otherwise I find the project quite interesting. I quite like to lose (too much time) on GStreetView, and at the same time I developed an interest in 20-30-40-year old cars, so I wrote a small Chrome extension that allows me to locally save images of old cars that I find on my country's roads while "taking a walk" on GStreetView, along with the relevant info (lat-lng, address, and the make and model of the cars which I input by hand using said extension). I had also noticed that there's a distinct correlation between a city's economic status and the cars you can find on its streets, and I was wondering how hard would it be to do the car "recognition" using some AI-thing and try to draw some conclusions from that. Glad that someone actually did it.
stillsut 10y ago
Everyone jumps to the dystopian scifi implication. But how about lets try thinking like real technologists? Example use: I know of three small but vibrant towns in Massachusetts where my wife and I would like to open a Bed 'n Breakfast. The problem is they're too expensive. So I feed the street views of these downtowns into the AI, and filter by <$250K home price. And it returns 20 beautifully "quaint" towns that I never could have found just by traditional census / business database searching.
- supernumerary 10y ago
  In terms of jumping to the sci-fi dystopian implication, I think it's quite fun to think about and certainly part of being a real technologist...: After all, doesn't the kind of of digital truth (or digital behaviourism) epitomized by this study threaten to undermine the foundation of ideas like individual emancipation or of the common? Like let's say you move to a quaint town that the AI found for you. Who moved in and is now a member of that community, you and your wife or your digital doppelgangers? And where did you move? The town or it's statistical analog warehoused in a data-center somewhere. When a new shop opens up in the town and you find your favorite brand of wine from your previous town on the shelves ... what's going on?
- ucaetano 10y ago
  A major example: data in emerging markets. For most emerging markets, there's nothing comparable to the per-track census data in the US (and other developed countries). Some fixed-line ISPs do market assessments by driving through neighborhoods and counting the number of cars and AC units in each house, as a proxy for income, and it is unlikely that better data will be available by traditional means any time soon. I once had to compile a lot of per-city info on Indonesia, it involved a friend actually going into the national statistics office, spending several hours drinking tea with them, getting the actual raw survey data recorded into a CD (this was a couple years ago, not a couple decades) so I could eventually process it and get some results (despite the abundance of gaps and errors in the data). In other words, Street View data is more pervasive, updated and readily available than data from the national statistics office.
mempko 10y ago
Let's ignore building racist AI for a sec. This is hugely flawed because most street view data is NOT updated on a yearly bases. Many images are years old. Doing yearly door to door will get fresher data.
anonu 10y ago
From the paper on collecting Street View images: "This was done via browser emulation and requires only the latitude and longitude of each point" My question is does the TOS for Google Maps or Street View allow for this? I'm not trying to diminish the research - it is SUPER cool. I'm just thinking it would be good to have access to the dataset they captured if publicly available.
- tgebru 10y ago
  At least parts of it will be publicly available. Our goal is to make the entire code/data publicly available. We might have to provide URLs and bounding boxes/other meta data for each image. But we're trying to figure out how to do this properly.
h4nkoslo 10y ago
Weev is working on a similar project: https://youtu.be/ZMptVkyZWE4 https://youtu.be/ZMptVkyZWE4
id122015 10y ago
I want people to use technology to find what those people who drive a car into pedestrians have in common. There must be something unique about them, the media says they all have mental health issues. Suddenly since 2016 people with metal issues got access to cars and trucks.
platita 10y ago
Extracting life quality related data from streetview does not seem to be a new thing: https://hackernoon.com/machine-learning-our-cities-617ce005ba27#.eg22ymq75 https://hackernoon.com/machine-learning-our-cities-617ce005b...
misiti3780 10y ago
does anyone know how they download this data ?

145 points by phodo 10y ago

Animats 10y ago

Or you could just order that data from Experian Automotive, which has vehicle registration data from all US states.

backpropaganda 10y ago
But I need to get a job at one of the big tech companies, and so I need to show them I can do deep learning.
- NTDF9 10y ago
  Doesn't matter! You'll still be asked to write perfect code, with awesome variable names, on a whiteboard, with eyes closed. Don't forget to do handstands to stand out from the crowd.
  - narrator 10y ago
    Don't forget to practice doing math like 36^5 in your head. No pencil or paper allowed!
    - draugadrotten 10y ago
      If you would need to practice that, the job is not for you.
    - logfromblammo 10y ago
      Well, let's see, 36 is 00100100 in binary. Binomial coefficients are 1 5 10 10 5 1. 2^25 + 5 * 2^22 + 10 * 2^19 + 10 * 2^16 + 5 * 2^13 + 2^10. 10 is 5 * 2, so bit shift those middle two. 5 is binary 0101, so add copies bit shifted by 2. 2^25 + 2^24 + 2^22 + 2^22 + 2^20 + 2^19 + 2^17 + 2^15 + 2^13 + 2^10. Comes out to 0011 1001 1010 1010 0100 0000 0000 0x039AA400 Which is a perfectly cromulent numeric response. If the interviewer asks what that is in decimal, they ought to be glad I don't have a pencil, because it would end up in someone's eye on my way out the door.
      - tropo 10y ago
        Me as interviewer: have you heard of the XOR operator?
        logfromblammo 10y ago
        Yeah, it looks like a plus sign with a circle around it. It's not in the ASCII character set. Sometimes, programming languages will use the caret character to represent XOR for boolean types and bitwise XOR for integral types, perhaps using two asterisks for exponentiation, or leaving it as a named function call, like Math.Exp or Math.Pow. But you have to be careful there, because sometimes Math.Exp is actually the inverse of the natural log function, rather than exponentiation. If caret means XOR, then 36 ^ 5 is 0010 0001, or 33. And that is the point where the interviewer marks me as "do not hire", because they can't stand it when they can't show off how smart they are to the candidate.
nl 10y ago
But (I assume) that only has the home address of the car. This method can be clearly extended to do things like measuring the socio-economic features of people parked at events etc.
Bedon292 10y ago
It seems like the idea is to use this method places outside of the US, which may not have the same level of data available.
tmalsburg2 10y ago
I once attended a talk in which the speaker presented a system for inferring the orientation of a mobile device in space. This was before smart phones and the system used accelerometers, some computer vision, and fancy algorithms which were running on a server because the device didn't have enough computing power. One person in the audience asked: Why don't you simply use a compass? The answer was silence.
xapata 10y ago
Nah, that data is weeks old. Some states are as slow as 3 months to hand the registration data to Experian. Better to scrape classifieds ads to see what people are selling ;-)

nl 10y ago

This is a great piece of work. I wonder if it would be possible to use a similar approach but on shop signs. Intuitively there should be some kind of relationship between the type of shops in different areas, but I guess it would be much less dense data than car type. I guess the next step is to make a completely end-to-end version without the manual feature engineering.

colinbartlett 10y ago
There was a similar discussion[1] here about that a while back using data about different business types to assert gentrification levels. 1. https://news.ycombinator.com/item?id=10391753 https://news.ycombinator.com/item?id=10391753

Doctor_Fegg 10y ago

> We focus on the motor vehicles encountered during this journey because over 90% of American households own a motor vehicle This is a clever hack, but I sorely hope no actual policy decisions are made on the basis of a methodology that wilfully ignores the 10% who choose not to own, or can't afford, a car.

blowski 10y ago
Can there possibly be a methodology that is 100% correct? If you try to interview 100% of people, then the results will be out of date by the time you've reached everybody. (Plus the Hawthorne effect suggests people will start changing their answers as people comment on the questions they are asked.) So you create samples, and necessarily some of those samples will be unrepresentative because they will miss some nuance. Catch 22 - unless you interview 100% of the people, you can't know what all the nuances are. I guess the best we can hope for is a result that merges lots of different results using lots of different methodologies, and base decisions on that.
- deleted 10y ago
  [deleted]

d--b 10y ago

Dammit AI people, wake up! This is SO backwards! AI-based profiling of anything is bound to filter off outliers. Censuses are made in the field for a reason. The goal is to gather data so that you can make statistical reasoning on it. Not the opposite!!!! Here, they just gather some car data, and infer demographic data from it. And then what? We've just created a population which matches the car sample. We can't draw ANY conclusion from that data, beyond the type of cars that are found in such and such neighborhoods.

paganel 10y ago
In my parts of the world (Eastern Europe) you can infer a lot more about the people inhabiting a certain area by looking at the cars they drive, I mean, compared to census and income/tax data. That's because there are lots of way of not telling the authorities how much you earn, but once it comes to purchasing a car you leave aside such niceties and you get the car that best fits your real economic status (with few exceptions).
- d--b 10y ago
  Yes, of course there is obviously some correlation between demographics and car ownership. But this paper makes it seem that their method is good to replace a in-the-field census. Basically, replacing all demographic data that the census brings with the single factor of car ownership. In the region where the actual correlation is less than 90%, the data generated is completely useless.
  - paganel 10y ago
    > But this paper makes it seem that their method is good to replace a in-the-field census. Ah, I missed that, sorry! To be honest I only skimped through the description, for other lazy people like me here's the relevant line: > Our results suggest that automated systems for monitoring demographic trends may effectively complement labor-intensive approaches, with the potential to detect trends with fine spatial resolution, in close to real time. which I also find as a little bit crazy. Otherwise I find the project quite interesting. I quite like to lose (too much time) on GStreetView, and at the same time I developed an interest in 20-30-40-year old cars, so I wrote a small Chrome extension that allows me to locally save images of old cars that I find on my country's roads while "taking a walk" on GStreetView, along with the relevant info (lat-lng, address, and the make and model of the cars which I input by hand using said extension). I had also noticed that there's a distinct correlation between a city's economic status and the cars you can find on its streets, and I was wondering how hard would it be to do the car "recognition" using some AI-thing and try to draw some conclusions from that. Glad that someone actually did it.

stillsut 10y ago

Everyone jumps to the dystopian scifi implication. But how about lets try thinking like real technologists? Example use: I know of three small but vibrant towns in Massachusetts where my wife and I would like to open a Bed 'n Breakfast. The problem is they're too expensive. So I feed the street views of these downtowns into the AI, and filter by <$250K home price. And it returns 20 beautifully "quaint" towns that I never could have found just by traditional census / business database searching.

supernumerary 10y ago
In terms of jumping to the sci-fi dystopian implication, I think it's quite fun to think about and certainly part of being a real technologist...: After all, doesn't the kind of of digital truth (or digital behaviourism) epitomized by this study threaten to undermine the foundation of ideas like individual emancipation or of the common? Like let's say you move to a quaint town that the AI found for you. Who moved in and is now a member of that community, you and your wife or your digital doppelgangers? And where did you move? The town or it's statistical analog warehoused in a data-center somewhere. When a new shop opens up in the town and you find your favorite brand of wine from your previous town on the shelves ... what's going on?
ucaetano 10y ago
A major example: data in emerging markets. For most emerging markets, there's nothing comparable to the per-track census data in the US (and other developed countries). Some fixed-line ISPs do market assessments by driving through neighborhoods and counting the number of cars and AC units in each house, as a proxy for income, and it is unlikely that better data will be available by traditional means any time soon. I once had to compile a lot of per-city info on Indonesia, it involved a friend actually going into the national statistics office, spending several hours drinking tea with them, getting the actual raw survey data recorded into a CD (this was a couple years ago, not a couple decades) so I could eventually process it and get some results (despite the abundance of gaps and errors in the data). In other words, Street View data is more pervasive, updated and readily available than data from the national statistics office.

mempko 10y ago

Let's ignore building racist AI for a sec. This is hugely flawed because most street view data is NOT updated on a yearly bases. Many images are years old. Doing yearly door to door will get fresher data.

anonu 10y ago

From the paper on collecting Street View images: "This was done via browser emulation and requires only the latitude and longitude of each point" My question is does the TOS for Google Maps or Street View allow for this? I'm not trying to diminish the research - it is SUPER cool. I'm just thinking it would be good to have access to the dataset they captured if publicly available.

tgebru 10y ago
At least parts of it will be publicly available. Our goal is to make the entire code/data publicly available. We might have to provide URLs and bounding boxes/other meta data for each image. But we're trying to figure out how to do this properly.

h4nkoslo 10y ago

Weev is working on a similar project: https://youtu.be/ZMptVkyZWE4 https://youtu.be/ZMptVkyZWE4

id122015 10y ago

I want people to use technology to find what those people who drive a car into pedestrians have in common. There must be something unique about them, the media says they all have mental health issues. Suddenly since 2016 people with metal issues got access to cars and trucks.

platita 10y ago

Extracting life quality related data from streetview does not seem to be a new thing: https://hackernoon.com/machine-learning-our-cities-617ce005ba27#.eg22ymq75 https://hackernoon.com/machine-learning-our-cities-617ce005b...

misiti3780 10y ago

does anyone know how they download this data ?