17 ms·

Google Unveils Neural Network to Determine the Location of Almost Any Image

191 points by uptown 11y ago

hacker_9 11y ago
What an impressive, and extremely creepy, use of state of the art neural net technology.
ck2 11y ago
ah good, this means the data can be "poisoned" They go further and use the machine to locate images that do not have location cues, such as those taken indoors or of specific items. This is possible when images are part of albums that have all been taken at the same place. I'm going to start including pictures of the moon surface in all my albums.
- DominikR 11y ago
  > I'm going to start including pictures of the moon surface in all my albums. Sorry to say it but they're going to be able to exclude whatever you throw at it. And even if you found some way to meaningfully degrade the quality, they'll analyse what you do and find a solution for that. They are not going to suddenly throw their hands up in the air and decide to give up. It's a matter of how many resources can you throw at the problem and how many can they to counteract you.
  - kafkaesq 11y ago
    They're going to be able to exclude whatever you throw at it. How, exactly? You're talking about something that's one of the most poorly understand areas in AI at present, and making it sound like it's a solved problem.
    - DominikR 11y ago
      > How, exactly? You're talking about something that's one of the most poorly understand areas in AI at present, and making it sound like it's a solved problem. With manpower. I said that they are going to analyse whatever users do to wilfully degrade the service and will find a way to exclude that. Just like they do with search for years. Somehow they manage to keep their search relevant even though so many people constantly work on coming up with new SEO tricks that would seriously degrade this service if left unchecked. Of course I cannot know how _exactly_ they will do that, even the persons working on that service cannot know at this moment how exactly they will keep this working in the future since they do not know yet what people will try to do.
      - gleenn 11y ago
        I like your enthusiasm but look at the number of images again, and remember that Google is notorious for trying hard not to use humans. I think the statistical significance will probably overwhelm moon pictures, and that's what they'd use long before some human starts poking around for weird data.
        nickcano 11y ago
        Or they could train a network to recognize persons likely to try and poison their data sets and use it as an exclusion. You could go on to argue that those persons could also poison that data, but then they could handle it in a similar way. This goes process could theoretically go on ad infinitum.
  - gue5t 11y ago
    Exactly. Machine learning is an inequality-expanding lever just like many other forms of capital. This is one of the reasons Google et al are so dangerous.
    - SquidLord 11y ago
      This has about as much reasonable intellectual bearing as suggesting that language is an inequality expanding lever just like many other forms of capital and one of the reasons Google and others are so dangerous. This is just the Luddite fallacy repeated over and over. By this logic, we should never develop new informatic methodologies, we should never develop new forms of analysis, we should never develop new software – because of economies of scale exist. I'll the state the obvious: that's stupid. There's no way to soften that blow.
- transpy 11y ago
  This will not be of statistical significance and thus will not affect the performance of the system.
andreyk 11y ago
An impressive, if not very technically interesting, result. I wonder if it can get cooler with hierarchical sets of neural nets with ever smaller grids... An aside - I was really annoyed by "small-scale experiment shows that PlaNet reaches superhuman performance at the task of geolocating Street View scenes". It's absurd to say computers are superhuman at searching the web for instance, and though this is a much more impressive perception-based result it is still fundamentally very data intensive. Would be nice if 'superhuman' was used a bit less now that Deep Learning has already been proven to be capable of really impressive performance in perception tasks.
- amelius 11y ago
  It would be even more impressive if they could compress the images, given that the backgrounds are already known.
mey 11y ago
An extensions of? http://xkcd.com/1425/ http://xkcd.com/1425/ http://parkorbird.flickr.com/ http://parkorbird.flickr.com/
Lanari 11y ago
Years ago I was like Richard Stallman is totally crazy!, saying stuff like don't post my pictures on Facebook and such. Today his fear is really really justifiable...
- Houshalter 11y ago
  It's not anything a human can't do in theory, identifying plants and common architecture features. If you are uploading photos on facebook, it's far more likely you'll leak geographic data through the photo's metadata, your ip, and gps.
  - Lanari 11y ago
    Metadata is basically giving your data... Stallman was asking to not put any kind of his pictures on Facebook, which means your face is recognized now by Facebook, and there's plenty of things to do with that...
- mwfunk 11y ago
  I'm all in favor of credit where credit is due, but that's like saying "the Unabomber was right" because examples exist of modern technology being misused in the past 20 years. If anything, his advocacy has damaged the credibility of legitimate privacy and civil rights advocates. Surely, SURELY RMS is not one and only person you have ever heard in your entire life warn about the dangers of putting personal information online, or the dangers of government surveillance. That's all just common sense and being relatively informed and skeptical about the world and the people around you. RMS says a heck of a lot more than "don't post your pictures on Facebook", much like PETA says a lot more than just, "treat animals ethically", or Earth First says a lot more than just, "don't destroy the environment". If sea levels rise much more in my lifetime, one lesson I will not draw from it is, "Earth First was right!" If you care about this stuff, donate to the ACLU, not the FSF.
  - MoOmer 11y ago
    Really? Let's not forget the NSA revelations that occurred so recently. Thanks for reminding me to renew my fsf membership
danjoc 11y ago
It's so good, it guesses the correct continent less than 50% of the time. I bet it even gets the correct planet 100% of the time :)
- Houshalter 11y ago
  The median error is 1100 km. Which is better than that. But I agree with you that this is nothing to worry about. I skimmed through the paper and thought "wow that's really interesting it can beat humans and get results that good", but it's not so good that it's a significant leak of privacy. Like the top comments seem to be scared about. The dataset might be a little biased though. I wonder how many pictures are just of people's faces, or food, or random indoor scenes. It's much more difficult to detect a location from such an image. How good are the results if you exclude the images that don't have any clues at all? As I understand it, the algorithm can identify common types of plants unique to different regions, or architectural features, or popularity of types of cars. So an outdoor photo is more likely to be recognized. But I'm not 100% sure what features it's using. In fact the creators don't even know!
throwawaypartay 11y ago
Mark my words--this tool is going to be quickly hijacked by meme groupies who will submit dick pics and post the location results: "Check it out, I'm Mogadishu!"
joshfraser 11y ago
You could improve the accuracy by using the timestamp of when the photo was taken & looking at the amount of daylight in the photo.
- markovbling 11y ago
  Interesting idea!
- asciiiiii 11y ago
  Will this really work? Cameras are by design able to accurately control the rate (and duration) of light they let in to create an image.
  - astrange 11y ago
    For known cameras like iPhone, that shouldn't hurt since you can read the exposure time out of the EXIF.
  - Veratyr 11y ago
    If you can also get accurate white balance, you may be able to get useful information from the colour of the sky as an alternative.
- est 11y ago
  > the amount of daylight wouldn't that be much effected by weather conditions and air quality ?
  - stephenboyd 11y ago
    That would actually improve the accuracy since the image can be compared to historical weather records.
- jononor 11y ago
  Possibly also length of shadows? (sounds tricky though)
atrudeau 11y ago
Ah, the luxury of having so much labeled data. Average machine learners like us will never have access to such low hanging fruit...
- ant6n 11y ago
  There are probably average machine learners working at companies like Google
  - deleted 11y ago
    [deleted]
- jmcdowell 11y ago
  Any idea who would have "126 million images along with their accompanying Exif location data."? That seems an insane amount to me.
  - michael_h 11y ago
    Android phones that backup all the photos to Google Photos? Seems like a good place to start.
  - cfcef 11y ago
    Sounds like 'all public Google+ photo albums'. (I say public because given what Facebook reports about daily photo uploads, a cumulative 126m seems like it would be way too small for G+ if it covered all Google Docs/Photos uploads including private ones.)
  - nl 11y ago
    Google Street View
    - dooptroop 11y ago
      Yup. And we've already been tagging locations with adresses for years through that captcha.
JoeAltmaier 11y ago
We're closer to my nirvana: I go on vacation and take NO pictures, just enjoy myself, but photobomb other people constantly. When I get home, Google makes a vacation album for me, by identifying my face in all those other pictures! With knowledge of my itinerary it should be easy now!
hartator 11y ago
Do you guys have an url where we can try it out? Or you think it's like every "new" Google product from a few years where there is a lot of talking before anything.
- stephenboyd 11y ago
  Blame the headline. It's an interesting AI research paper by a team with two Google employees on it, not a product.
miyuru 11y ago
in the article there is a game to identify locations by photos. its pretty neat. https://www.geoguessr.com/ https://www.geoguessr.com/
- dazmax 11y ago
  It's a great game, but a little misleading to use as a proof of concept since presumably they trained it with google streetview data.
cafebeen 11y ago
I'm just going to leave this here: http://graphics.cs.cmu.edu/projects/im2gps/ http://graphics.cs.cmu.edu/projects/im2gps/
nl 11y ago
http://groups.csail.mit.edu/vision/SUN/ http://groups.csail.mit.edu/vision/SUN/
vedaprodarte 11y ago
In 2008, when not all of us are using smartphones, we always remind the smartphone users to protect their privacy. We told them not to upload their photos. But now, who cares? Everyone should know where I am!
matchagaucho 11y ago
“Our model uses only 377 MB, which even fits into the memory of a smartphone” Okay. But wouldn't this be more useful as a web service?
- dontreact 11y ago
  -Realtime feedback -Better privacy
acd 11y ago
Bad guys posting internet selfies are now going to get hunted by law enforcement which is a good thing.

191 points by uptown 11y ago

What an impressive, and extremely creepy, use of state of the art neural net technology.

ck2 11y ago

ah good, this means the data can be "poisoned" They go further and use the machine to locate images that do not have location cues, such as those taken indoors or of specific items. This is possible when images are part of albums that have all been taken at the same place. I'm going to start including pictures of the moon surface in all my albums.

DominikR 11y ago
> I'm going to start including pictures of the moon surface in all my albums. Sorry to say it but they're going to be able to exclude whatever you throw at it. And even if you found some way to meaningfully degrade the quality, they'll analyse what you do and find a solution for that. They are not going to suddenly throw their hands up in the air and decide to give up. It's a matter of how many resources can you throw at the problem and how many can they to counteract you.
- kafkaesq 11y ago
  They're going to be able to exclude whatever you throw at it. How, exactly? You're talking about something that's one of the most poorly understand areas in AI at present, and making it sound like it's a solved problem.
  - DominikR 11y ago
    > How, exactly? You're talking about something that's one of the most poorly understand areas in AI at present, and making it sound like it's a solved problem. With manpower. I said that they are going to analyse whatever users do to wilfully degrade the service and will find a way to exclude that. Just like they do with search for years. Somehow they manage to keep their search relevant even though so many people constantly work on coming up with new SEO tricks that would seriously degrade this service if left unchecked. Of course I cannot know how _exactly_ they will do that, even the persons working on that service cannot know at this moment how exactly they will keep this working in the future since they do not know yet what people will try to do.
    - gleenn 11y ago
      I like your enthusiasm but look at the number of images again, and remember that Google is notorious for trying hard not to use humans. I think the statistical significance will probably overwhelm moon pictures, and that's what they'd use long before some human starts poking around for weird data.
      - nickcano 11y ago
        Or they could train a network to recognize persons likely to try and poison their data sets and use it as an exclusion. You could go on to argue that those persons could also poison that data, but then they could handle it in a similar way. This goes process could theoretically go on ad infinitum.
- gue5t 11y ago
  Exactly. Machine learning is an inequality-expanding lever just like many other forms of capital. This is one of the reasons Google et al are so dangerous.
  - SquidLord 11y ago
    This has about as much reasonable intellectual bearing as suggesting that language is an inequality expanding lever just like many other forms of capital and one of the reasons Google and others are so dangerous. This is just the Luddite fallacy repeated over and over. By this logic, we should never develop new informatic methodologies, we should never develop new forms of analysis, we should never develop new software – because of economies of scale exist. I'll the state the obvious: that's stupid. There's no way to soften that blow.
transpy 11y ago
This will not be of statistical significance and thus will not affect the performance of the system.

andreyk 11y ago

An impressive, if not very technically interesting, result. I wonder if it can get cooler with hierarchical sets of neural nets with ever smaller grids... An aside - I was really annoyed by "small-scale experiment shows that PlaNet reaches superhuman performance at the task of geolocating Street View scenes". It's absurd to say computers are superhuman at searching the web for instance, and though this is a much more impressive perception-based result it is still fundamentally very data intensive. Would be nice if 'superhuman' was used a bit less now that Deep Learning has already been proven to be capable of really impressive performance in perception tasks.

amelius 11y ago
It would be even more impressive if they could compress the images, given that the backgrounds are already known.

mey 11y ago

An extensions of? http://xkcd.com/1425/ http://xkcd.com/1425/ http://parkorbird.flickr.com/ http://parkorbird.flickr.com/

Lanari 11y ago

Years ago I was like Richard Stallman is totally crazy!, saying stuff like don't post my pictures on Facebook and such. Today his fear is really really justifiable...

Houshalter 11y ago
It's not anything a human can't do in theory, identifying plants and common architecture features. If you are uploading photos on facebook, it's far more likely you'll leak geographic data through the photo's metadata, your ip, and gps.
- Lanari 11y ago
  Metadata is basically giving your data... Stallman was asking to not put any kind of his pictures on Facebook, which means your face is recognized now by Facebook, and there's plenty of things to do with that...
mwfunk 11y ago
I'm all in favor of credit where credit is due, but that's like saying "the Unabomber was right" because examples exist of modern technology being misused in the past 20 years. If anything, his advocacy has damaged the credibility of legitimate privacy and civil rights advocates. Surely, SURELY RMS is not one and only person you have ever heard in your entire life warn about the dangers of putting personal information online, or the dangers of government surveillance. That's all just common sense and being relatively informed and skeptical about the world and the people around you. RMS says a heck of a lot more than "don't post your pictures on Facebook", much like PETA says a lot more than just, "treat animals ethically", or Earth First says a lot more than just, "don't destroy the environment". If sea levels rise much more in my lifetime, one lesson I will not draw from it is, "Earth First was right!" If you care about this stuff, donate to the ACLU, not the FSF.
- MoOmer 11y ago
  Really? Let's not forget the NSA revelations that occurred so recently. Thanks for reminding me to renew my fsf membership

danjoc 11y ago

It's so good, it guesses the correct continent less than 50% of the time. I bet it even gets the correct planet 100% of the time :)

Houshalter 11y ago
The median error is 1100 km. Which is better than that. But I agree with you that this is nothing to worry about. I skimmed through the paper and thought "wow that's really interesting it can beat humans and get results that good", but it's not so good that it's a significant leak of privacy. Like the top comments seem to be scared about. The dataset might be a little biased though. I wonder how many pictures are just of people's faces, or food, or random indoor scenes. It's much more difficult to detect a location from such an image. How good are the results if you exclude the images that don't have any clues at all? As I understand it, the algorithm can identify common types of plants unique to different regions, or architectural features, or popularity of types of cars. So an outdoor photo is more likely to be recognized. But I'm not 100% sure what features it's using. In fact the creators don't even know!

throwawaypartay 11y ago

Mark my words--this tool is going to be quickly hijacked by meme groupies who will submit dick pics and post the location results: "Check it out, I'm Mogadishu!"

joshfraser 11y ago

You could improve the accuracy by using the timestamp of when the photo was taken & looking at the amount of daylight in the photo.

markovbling 11y ago
Interesting idea!
asciiiiii 11y ago
Will this really work? Cameras are by design able to accurately control the rate (and duration) of light they let in to create an image.
- astrange 11y ago
  For known cameras like iPhone, that shouldn't hurt since you can read the exposure time out of the EXIF.
- Veratyr 11y ago
  If you can also get accurate white balance, you may be able to get useful information from the colour of the sky as an alternative.
est 11y ago
> the amount of daylight wouldn't that be much effected by weather conditions and air quality ?
- stephenboyd 11y ago
  That would actually improve the accuracy since the image can be compared to historical weather records.
jononor 11y ago
Possibly also length of shadows? (sounds tricky though)

atrudeau 11y ago

Ah, the luxury of having so much labeled data. Average machine learners like us will never have access to such low hanging fruit...

ant6n 11y ago
There are probably average machine learners working at companies like Google
- deleted 11y ago
  [deleted]
jmcdowell 11y ago
Any idea who would have "126 million images along with their accompanying Exif location data."? That seems an insane amount to me.
- michael_h 11y ago
  Android phones that backup all the photos to Google Photos? Seems like a good place to start.
- cfcef 11y ago
  Sounds like 'all public Google+ photo albums'. (I say public because given what Facebook reports about daily photo uploads, a cumulative 126m seems like it would be way too small for G+ if it covered all Google Docs/Photos uploads including private ones.)
- nl 11y ago
  Google Street View
  - dooptroop 11y ago
    Yup. And we've already been tagging locations with adresses for years through that captcha.

JoeAltmaier 11y ago

We're closer to my nirvana: I go on vacation and take NO pictures, just enjoy myself, but photobomb other people constantly. When I get home, Google makes a vacation album for me, by identifying my face in all those other pictures! With knowledge of my itinerary it should be easy now!

hartator 11y ago

Do you guys have an url where we can try it out? Or you think it's like every "new" Google product from a few years where there is a lot of talking before anything.

stephenboyd 11y ago
Blame the headline. It's an interesting AI research paper by a team with two Google employees on it, not a product.

miyuru 11y ago

in the article there is a game to identify locations by photos. its pretty neat. https://www.geoguessr.com/ https://www.geoguessr.com/

dazmax 11y ago
It's a great game, but a little misleading to use as a proof of concept since presumably they trained it with google streetview data.

cafebeen 11y ago

I'm just going to leave this here: http://graphics.cs.cmu.edu/projects/im2gps/ http://graphics.cs.cmu.edu/projects/im2gps/

nl 11y ago

http://groups.csail.mit.edu/vision/SUN/ http://groups.csail.mit.edu/vision/SUN/

vedaprodarte 11y ago

In 2008, when not all of us are using smartphones, we always remind the smartphone users to protect their privacy. We told them not to upload their photos. But now, who cares? Everyone should know where I am!

matchagaucho 11y ago

“Our model uses only 377 MB, which even fits into the memory of a smartphone” Okay. But wouldn't this be more useful as a web service?

dontreact 11y ago
-Realtime feedback -Better privacy

acd 11y ago

Bad guys posting internet selfies are now going to get hunted by law enforcement which is a good thing.