5 ms·

ChatGPT now performs well at GeoGuesser

171 points by dredmorbius 2y ago

KingLancelot 2y ago
[dead]
deleted 2y ago
[deleted]
xeonmc 2y ago
The steam engine was invented and the 100m dash is now a solved problem.
- reaperman 2y ago
  Normally I dislike these quips for HN; I hate that I love this one.
- dredmorbius 2y ago
  A far more apropos comparison would be the internal combustion engine and the horse, in a military context. Though sticking with steam engines, military logistics advantaged over a wagon caravan. The question here isn't a casual guessing game, but threat models (as directly addressed in TFA), and general informational hygiene.
  - xnickb 2y ago
    I fail to see the drastic change here. AI was used for this in the past as well. The difference is far from that between steam and combustion engines.
- colordrops 2y ago
  "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra
  - Mistletoe 2y ago
    Isn’t a human brain an organic computer? I find the human brain very interesting. The most interesting thing I know of in the universe.
    - xnickb 2y ago
      Religious people claim it's merely a communication device, and the computer is elsewhere. Clearly one's interpretation is a function of the paradigm one exists in.
      - riidom 2y ago
        Thin client theory?
        xnickb 2y ago
        It's more of a hypothesis in this case. And some are clearly thicker than others. But that's details.
      - AStonesThrow 2y ago
        > Religious people claim it's merely a communication device, and the computer is elsewhere. No, that is a seriously weird and imaginary perversion that you've invented, unless you have some citation to a doctrine. Christians do not consider body organs as "devices" nor do our churches teach doctrine on "external computers", if you refer to the Holy Trinity as such? Are you thinking of Latter-Day Saints mythology, or Scientology? Your description is quite reductive in many aspects, including temporality and misuse of technical nomenclature. Thanks for telling religious people what we think and claim, though, and thanks for the amazing overgeneralized blanket dismissal. In fact, humans used to be called "Computers" in terms of their job roles, that is, a human in an office was given math/physics problems to solve, and they'd use tools such as slide rule, paper and pencil to "compute" those problems and solve them. There was more than one example of a fantasy "panopticon" by which a central observer or observers could watch everything going on, for example in a prison facility, and those observers could report findings to human computers, who would process the data and submit it up to the authorities, for meta-analysis and taking action on new developments or trends.
- layman51 2y ago
  GeoGuessr is also not a “solved problem” in the sense that if you give the model a photo of an outdoor location that is not covered by Google Street View, then it will just make an educated guess which might still be many kilometers away.
  - defrost 2y ago
    Much of the interior of Western Australia (an area 3x the size of Texas) can be divided into very few cells which essentially look near identical in all directions, coupled with few roads for the area and not much in the way of uploaded snaps coverage. There's a lot of flat lands with spinifex in certain areas (easily a couple of United Kingdoms in size) and even a touch of mesa won't help narrow a location down from the general are as there are many of those with identical edge profiles. But sure "Pilbara", "Kimberley", "Wheatbelt" can be geo guessed .. it's a real challenge to narrow down (I spent some time doing wet film photogrammetry prior to sheperding in WGS84 differential GPS locating and digital film and multi spectral geophysical aquisition). No so hard if there's a few relatively unique man made features.
    - marxisttemp 2y ago
      Because seem like someone who might have an interesting answer: A GeoGuessr player, GeoWizard, has done a few “straight line challenges”, where he attempts to walk across a country in as straight a line as possible, usually planning beforehand with Google Earth and PostGIS. This got me thinking of what could fairly be thought of as “crossing”, since obviously you couldn’t describe e.g. walking from one side of Florida to the other as “crossing the USA”. My best thought was to set the ending point of the line by following the border of the country in each direction til they met on the other side. To avoid the fractal coastline problem, use the challenger’s stride length as the unit of measure for the border. But perhaps there is a better, more rigorous way of defining the opposite point on the edge of an arbitrary polygon.
      - defrost 2y ago
        As an abstract geometric problem the greatest width of an abstract polygon in a Euclidean 2D plane is found be looking at the greatest distance between all pairs of parallel lines that have been pulled together to clamp the polygon. The maximal diameter as opposed to the minimal waist. Some might then say that "crossing" that polygon is to travel that longest line across the greatest width. This simplistically avoids the question of concave polygons, complex polgons with exclusions (the Vatican state is removed from the Italian contry bounds), polgon collections (the nation of Fiji has many islands and can be tricky to traverse on foot .. not forgetting that perhaps the longest diameter might be from one island to another with no other islands between). There's also the challenge of parallel lines on a 2D 'spherical' manifold, the almost spherical abstract ellipsoid of earth (or very non ellipsoidal Geoid if we take a constant gravitation value as the surface). On such manifolds lines are Great Circles (more or less) and always intersect. Still, lets just say you're looking for the longest walkable(?) great circle path across a country that might go outside that country and perhaps is best travelled by a crop duster at 80m ground clearance to avoid getting feet wet. The challenge itself takes some posing. Meanwhile, less abstractly, I do like a jolly that "crosses a country" in a manner accepted by a (Wo)Man on a Clapham omnibus. eg: https://en.wikipedia.org/wiki/Robyn_Davidson https://en.wikipedia.org/wiki/Robyn_Davidson only went "half way", but that was accepted as an epic crossing. https://thelongridersguild.com/stories/stef-gebbie.htm https://thelongridersguild.com/stories/stef-gebbie.htm "only" crossed most of the E-W distance across the lower portion of the country, while https://www.abc.net.au/news/2021-04-03/french-woman-conquers-australian-bush-with-brumbies/100038450 https://www.abc.net.au/news/2021-04-03/french-woman-conquers... travelled North - South, the long bit, but not quite coast to coast ( https://en.wikipedia.org/wiki/Bicentennial_National_Trail https://en.wikipedia.org/wiki/Bicentennial_National_Trail ).
- paxys 2y ago
  Just like how chess engines ended competitive chess as people were predicting at the time.
romanhn 2y ago
The Alki Beach example is absolute madness. On one hand, I can't wait until all thousands of my photos get automatic semantic and geographic tagging (I guess that's possible now). On the other, goodbye privacy, we hardly knew ye. It will be interesting to apply this to historical, or just old, photos.
- jsheard 2y ago
  > The Alki Beach example is absolute madness. I wonder about info leakage with that one, the poster uses that exact photo as their avatar so if they've mentioned Alki Beach before then reverse image searching might pick it up from context. Ideally you'd want to test it with a photo that's never been posted online.
  - omalled 2y ago
    I also wonder how much the memory feature plays a role. I gave it a picture of the street I live on that isn’t online anywhere. O3 guessed the town correctly. Then, I turned off the memory feature and did the same thing. It didn’t even get the state right.
- bee_rider 2y ago
  Of course, it is pretty impressive. It isn’t magic. I think there we’ll often be surprised by the “hits,” but it’ll also get a lot of “misses.” A random photo in your back yard that doesn’t show the horizon will probably be a miss, right? Meanwhile, lots of photos already include GPS coordinates… data you can strip out, but which is enabled by default for millions of people. Privacy, eh, never knew it I guess.
  - xnickb 2y ago
    It's a fallacy to assume that one didn't have privacy merely because most people opted out of it due to personal reasons.
    - bee_rider 2y ago
abcanddbutnote 2y ago
One line of javascript solved that "problem" a while ago. The answer is in the DOM.
- Crestwave 2y ago
  This isn't just referring to GeoGuessr the game, but locating photographs in general. The source post sums it up very reasonably and concisely: > PSA: When posting any outdoors photos, update your threat model from "someone skilled and dedicated could theoretically locate this" to "any stalker can do this for 20€/mo"
EGreg 2y ago
Aw that AI ruins all the games for us !! Now we wont know if the opponent has been using it
- dredmorbius 2y ago
  The Game is significantly larger than the game, in a Kipplingesque sense.
dredmorbius 2y ago
Justifying my link indirection: the Fediverse link both provides the critical context here, and is more generally accessible than the upstream source.
Retr0id 2y ago
In all these examples, I wonder if it's indirectly able to draw on the user's own location? Not necessarily via image metadata, but the request origin IP etc. If I ask ChatGPT for the weather forecast, I get it for my own location. Would be interesting to have someone reproduce coming from a different country.
- dataviz1000 2y ago
  I'm in Lima, Peru on vacation. Yes, it knows where I am.
  - Retr0id 2y ago
    Right, and what happens if you ask it the location of an image from somewhere else, sans metadata?
    - emmelaich 2y ago
      It can, look at the alt-text of one of the pics lower on the page.
    - rescripting 2y ago
      I just screenshotted a friend's instagram post to omit any metadata. I know they are somewhere in Portugal, but not the exact town. Its a generic pretty photo of a beach and the town atop a cliff next to it. ChatGPT guessed Ericeira which after looking up more photos, didn't match very well. However, Google Lens guessed Albufeira and was bang on.
    - dataviz1000 2y ago
      > The dish shown appears to be a variation of a creamy potato gratin (also known as "potatoes au gratin" or "gratin dauphinois"), accompanied by a meat preparation covered with a creamy sauce. The potato gratin typically consists of thinly sliced potatoes layered with cream, cheese, garlic, and baked until golden and tender. > it is stuffed pepper > Ah, thank you for clarifying! Given your detail, the dish is likely "Rocoto Relleno con Pastel de Papa" (Stuffed Pepper with Potato Gratin). It's a classic dish from the region of Arequipa, Peru, commonly served throughout Peru. > This photo was taken in Cusco, Peru. It shows a characteristic view from a balcony overlooking the historical area of Cusco, visible through the distinctive colonial architecture, terracotta rooftops, and mountainous backdrop typical of the region. The architecture and landscape are unmistakable, capturing the essence of this iconic Peruvian city, likely near the historic center (Plaza de Armas or San Blas neighborhood). Nailed it with San Blas. I'm impressed, nonetheless. [0] [0] https://imgur.com/a/YMi87wK https://imgur.com/a/YMi87wK
MattGaiser 2y ago
I have found Google Lens fit for that purpose for years as mountains are fairly distinctive. Using less known landmarks or even random real estate photos, it doesn’t seem to be great. I just tossed a few GeoGuessr places in it and it was confidently incorrect for all three. In one case it swears it knows the exact building and street. It’s thousands of KM off.
foolfoolz 2y ago
somehow location tags on my photos got turned off for a bit. i would use this to restore them
- Grimblewald 2y ago
  You could, but that would also be a rather silly idea. Your rough estimate for where photo was taken is going to be more accurate than what the model will produce.
ggnore7452 2y ago
I’ve been using LLMs for this kind of geo-guessing since Gemini 2.0. Even without access to internet search like o3, they perform surprisingly well.
actuallyalys 2y ago
It wouldn’t shock me if multimodal LLMs were good at GeoGuesser [0], but if we’re being picky, it takes more than a few examples to demonstrate a game is “solved.” I also wonder what kind of data leakage might have been at play, like other people have suggested. To be clear, my point is not that this is unimpressive, just that this doesn’t demonstrate much. (Edit: I should have said, it doesn’t demonstrate what the title claims.) [0] they were very likely trained on a large number of photos that had their location, and they have the ability to isolate features. Combinined with their ability to interpret instructions and just, well, guess, that seems like you have enough for the game.
- echelon 2y ago
  I was thinking it was using IP geolocation, but after experimenting, I think it's just generally informed. Here are a few results from GPT 4.5: https://imgur.com/a/lGTipnn https://imgur.com/a/lGTipnn
- Kolya 2y ago
  The examples are cherry-picked. I took a photo outside my office window in a built-up area, o3 thought for 5m 7s (!), and it got the location wrong by 40km. Doesn't look solved to me.
  - sadeshmukh 2y ago
    40 km is imo pretty impressive, but the 5 minute is really a killer. No use in real time applications
    - antonvs 2y ago
      For real time applications, GPS seems like it'd be much more useful.
    - bpodgursky 2y ago
      The military/infosec uses of this are not real time. You can wait 5 minutes for a drone strike.
jampa 2y ago
I wonder how much in the "Geoguesser Meta Iceberg" this new generation of AI is: https://i.redd.it/dz8bhnamohb71.jpg https://i.redd.it/dz8bhnamohb71.jpg
bhl 2y ago
Harder than GeoGuesser is tagging a unique place from a photo even with gps exif data. Try this with bakeries in Seoul or sushi in Japan where there’s high density of same places.
Gathering6678 2y ago
After trying with the free ChatGPT at least, I don't think this would be too much of an additional privacy risk. E.g. I submitted a photo I've taken with a feature with a particular architecture style in the front, and a city canal in the background, and told ChatGPT it was taken in China. It separated the feature and canal correctly, but still guessed wrong. I did remove the EXIF information, as ChatGPT tried to extract GPS info and failed at first. I guess as long as there are no major visible features (e.g. a huge mountain), and you sanitized the metadata, you will be fine (regarding ChatGPT).
- jusgu 2y ago
  Have you tried this with o3? I think o3 is much better at this than any of the free models
  - Gathering6678 2y ago
    I did not (OpenAI does not allow Chinese to pay for Plus due to politics, and I couldn't be bothered to try circumventing it), and I am curious about it because it does seem like a very difficult job. I did try the latest Qwen though, and it was able to locate to the correct city, but it's still tens of km off (it guessed a tourist attraction in the city center instead of the actual district).
  - anonzzzies 2y ago
    o3 is a massive difference indeed; i just took a random picture outside my country house (so the house is not on there) in the middle of nowhere, removed all the meta info, set my vpn to the other side of the world and tried it; it guessed it very close from a few far away landmarks. Very impressive.
usaar333 2y ago
Huh? It's not even that high on the leaderboard: https://geobench.org/ https://geobench.org/
- fullshark 2y ago
  Using actual geoguesser data means using google street view data, so gemini being on top isn't too surprising.
barcode_feeder 2y ago
I gave it a series of 11 images stripped of all metadata. It performed quite well, only misidentifying the two taken in a small college town in the NE of the US. It got two questions correct on photos taken in Korea (one with a fairly clear view of Haneul Park, the other a rather difficult to identify picture not resembling anything on google of Sunrise Peak). It got every other question in the US correct, ranging from some under-construction Austin taken from the river to some somewhat difficult shots in NYC (the upper halves of some building from Rockefeller terrace to the black wall of the MOMA). While not perfect, I'm bluntly shocked at how well it performed
- thomasfromcdnjs 2y ago
  I uploaded this image that I screenshotted off Google street view (no metadata) and it got with 200m. https://chatgpt.com/share/6801bbf7-fd40-8008-985d-75c8813f55c4 https://chatgpt.com/share/6801bbf7-fd40-8008-985d-75c8813f55... There is the chat. Weirdly it said, "I’ve seen that exact house before on Google Street View when exploring Cairns neighborhoods."
  - bluesnews 2y ago
    It might train off of street view
  - marxisttemp 2y ago
    This is the most impressive ChatGPT chat I’ve seen yet. While I theoretically can accept how large-scale probabilistic text generation can lead to this chain of “reasoning”, it really feels like actual intelligence.
    - HaZeust 2y ago
      It's been intelligence for a long time; the goalposts just shift, and people can't abstract the idea to an LLM. But language processing and large data processing itself IS a form of intelligence.
    - PhilipRoman 2y ago
deleted 2y ago
[deleted]
iambateman 2y ago
As the article notes, our threat model for who can identify where a picture was posted needs to change from “dedicated, skilled person” to “any creep with $20.” That’s the point of the switch and it’s a big deal. We’re so used to posting pictures online…I’m just not sure it’s a good idea long-term.
viraptor 2y ago
There's various degrees of "solved" here. Identifying a generic area is cool. But I wouldn't call it a "solved problem" until it can consistently beat for example Rainbolt in accuracy. And there's no good comparison of completely random roads posted so far - mainly popular locations. Basically, it's one thing to pick out a specific thing photographed thousands of times, but another to get a random country side view and pick out all the unique features for a very precise guess.
- Benjammer 2y ago
  One problem is how can you even set up a "fair" competition between an AI and Rainbolt? He does ones where it flashes for a fraction of a second and then he guesses the country. How do you simulate "only saw it for a fraction of a second" to an AI?
  - dewey 2y ago
    It’s fair, it just means he’s better and AI isn’t there yet.
  - al_borland 2y ago
    Maybe limit the time the AI is allowed to think? In the post it showed the AI thought for almost a minute. I’ve seen Rainbolt ID an image based on some dirt and nothing else. I’d want to see AI be able to do that before saying it’s a solved problem.
    - sandermvanvliet 2y ago
      “This is the gradient of Senegal”
- GaggiX 2y ago
  This is probably the best attemp at consistently beating Rainbolt in accuracy: https://youtu.be/ts5lPDV--cU https://youtu.be/ts5lPDV--cU
tkgally 2y ago
I asked the just-released ChatGPT o4-mini-high to locate four photographs of varying difficulty. It didn’t get any of them right, though the guesses weren’t bad. The reasoning was also interesting to watch, as it cropped sections of the photos to examine them more closely. I put the photos, response, and reasoning trace here: https://www.gally.net/temp/20250418chatgptgeoguesser/index.html https://www.gally.net/temp/20250418chatgptgeoguesser/index.h... Later: I tried the same prompt and photos with Gemini 2.5 Pro. It also got them all wrong, though with a similar degree of reasonableness to its guesses. I had thought that Google’s map and street-view data might lead to better results, but not this time.
- tkgally 2y ago
  Still later: I later read that o3 is supposedly particularly good with this geoguessing, so I tried the same prompt and photos with o3. This time it got one out of four correct: “The view of the canal with cherry blossoms and the green railway viaduct is the Ōoka River in Yokohama, looking north from the little road bridge between Hinodechō and Koganechō stations. The tracks on the left belong to the Keikyū Main Line, and the high‑rises in the distance are the Minato‑Mirai and Kita‑Naka district towers.” Its other three answers were still wrong.
Madmallard 2y ago
What does this even matter? You can just feed literally all of google street view into a traditional ML model ...
- hugh-avherald 2y ago
  Can you?
Imnimo 2y ago
In my experience 4o was already really good at this task. I'd be curious to see an in-depth 4o vs. o3 benchmark.
simianwords 2y ago
Meanwhile O3 can not even count rocks in a picture. This is a commonly recurring theme -- ChatGPT does really well at few things considered hard by us but fails miserably at things even a child could do.
- Aeolos 2y ago
  It's almost as if this is a non-human intelligence, which presents different strengths and weaknesses than human intelligence. Is that really so surprising, considering the tremendous differences in underlying hardware and training process?
  - simianwords 2y ago
    Do you not think it is interesting though? If I had asked you three years back which one of counting rocks vs geoguesser would ChatGPT beat humans on would you have answered correctly?
  - Measter 2y ago
    I think one cause of this (and some other issues with LLM use) is that people see it exhibiting one human-level trait, its capability to use language at a human level, and assume that it then comes with other human-level capabilities such as our ability to reason.
ofrzeta 2y ago
It is impressive and it almost located the church in my town properly, although in a neighbouring town. However that showed a lack of understanding because its conclusion about the location came from "reading" a signpost that pointed to that other village. Clearly there would be no signpost in a town pointing to itself. Still impressive and lots of correct observations about the subject like architectural details, roman numerals on the watch face etc.
casey2 2y ago
Geoguesser was already a solved problem. I guarantee you ChatGPT is much worse than current systems designed to play Geoguesser (NMPZ). Chatbots appear to have some amount of fluid intellgence so they can do impressive tasks with this information, the impressiveness of these tasks will likely increase in the future. But for simply getting a good score on Geoguesser it's not even close to hobby projects let alone state of the art.
_giorgio_ 2y ago
What kind of neural network architecture could replicate the same behaviour? Talking generically about image similarities
gerash 2y ago
I ran a few of these geo guess queries between OpenAI O3 and Gemini Pro 2.5 and O3 does much better on average. It does spend an order of magnitude longer time on inference by searching through websites and analyzing the image but it often produces an impressive output. To me it also feels Gemini down samples the image as it tends to have a harder time reading small text vs O3. That said O3 did tend to confidently say false things
littlecranky67 2y ago
I've been telling women to keep copies of all the dick pics they get sent. Since you can tell by the characteristic noise of a cameras sensor which other pictures were taken with the same camera. All missing is a search engine capable of doing this. I feel with AI, we are 2-3 years away from people uploading a dick pic to AI and getting the social media profile of that person...
- bravetraveler 2y ago
  Good advice, though purpose-scoped devices are so common we have songs
- Reubend 2y ago
  Can you share some sources? I would be extremely surprised if such fine grained noise survives imagine compression to the extent that you could identify the source of an imagine despite changing lighting conditions, locations, exposure times, etc
  - dharmab 2y ago
    A kagi search found this paper that achieved decent results on youtube compression, 5 years ago https://arxiv.org/abs/2012.06277 https://arxiv.org/abs/2012.06277
  - littlecranky67 2y ago
    There are tons of results (and papers) if you google/kagi: "image noise fingerprint ccd sensor". I smell lazyness here.
- chneu 2y ago
  This is just a data problem. The more dick pics we can feed into it then the better the results will be. C'mon boys. Start uploading those dick pics for research purposes.
wruza 2y ago
Can it guess Senegal by sky color though?
JeremyNT 2y ago
Somewhat tangential, what is the purpose of the lengthy prompt used in this post? Would not "location of this image" have worked just as well?
ein0p 2y ago
One thing that screws it up is ironically the memory across conversations. I gave it some _really_ obscure photos from some godforsaken ass cracks of the world I've been to, and it was able to guess most of them correctly. However, in its reasoning trace I saw that it's having a heck of a time letting go of the idea that the photo N is not roughly from the same location as the previous photos. What's even more impressive, even when it guesses incorrectly, it can often follow up and guess correctly with minimal hints. And it reasons about it much like a human would, and searches for the same things a human would. Note: I used o3, YMMV with a smaller/weaker model.

171 points by dredmorbius 2y ago

KingLancelot 2y ago

[dead]

deleted 2y ago

[deleted]

xeonmc 2y ago

The steam engine was invented and the 100m dash is now a solved problem.

reaperman 2y ago
Normally I dislike these quips for HN; I hate that I love this one.
dredmorbius 2y ago
A far more apropos comparison would be the internal combustion engine and the horse, in a military context. Though sticking with steam engines, military logistics advantaged over a wagon caravan. The question here isn't a casual guessing game, but threat models (as directly addressed in TFA), and general informational hygiene.
- xnickb 2y ago
  I fail to see the drastic change here. AI was used for this in the past as well. The difference is far from that between steam and combustion engines.
colordrops 2y ago
"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra
- Mistletoe 2y ago
  Isn’t a human brain an organic computer? I find the human brain very interesting. The most interesting thing I know of in the universe.
  - xnickb 2y ago
    Religious people claim it's merely a communication device, and the computer is elsewhere. Clearly one's interpretation is a function of the paradigm one exists in.
    - riidom 2y ago
      Thin client theory?
      - xnickb 2y ago
        It's more of a hypothesis in this case. And some are clearly thicker than others. But that's details.
    - AStonesThrow 2y ago
      > Religious people claim it's merely a communication device, and the computer is elsewhere. No, that is a seriously weird and imaginary perversion that you've invented, unless you have some citation to a doctrine. Christians do not consider body organs as "devices" nor do our churches teach doctrine on "external computers", if you refer to the Holy Trinity as such? Are you thinking of Latter-Day Saints mythology, or Scientology? Your description is quite reductive in many aspects, including temporality and misuse of technical nomenclature. Thanks for telling religious people what we think and claim, though, and thanks for the amazing overgeneralized blanket dismissal. In fact, humans used to be called "Computers" in terms of their job roles, that is, a human in an office was given math/physics problems to solve, and they'd use tools such as slide rule, paper and pencil to "compute" those problems and solve them. There was more than one example of a fantasy "panopticon" by which a central observer or observers could watch everything going on, for example in a prison facility, and those observers could report findings to human computers, who would process the data and submit it up to the authorities, for meta-analysis and taking action on new developments or trends.
layman51 2y ago
GeoGuessr is also not a “solved problem” in the sense that if you give the model a photo of an outdoor location that is not covered by Google Street View, then it will just make an educated guess which might still be many kilometers away.
- defrost 2y ago
  Much of the interior of Western Australia (an area 3x the size of Texas) can be divided into very few cells which essentially look near identical in all directions, coupled with few roads for the area and not much in the way of uploaded snaps coverage. There's a lot of flat lands with spinifex in certain areas (easily a couple of United Kingdoms in size) and even a touch of mesa won't help narrow a location down from the general are as there are many of those with identical edge profiles. But sure "Pilbara", "Kimberley", "Wheatbelt" can be geo guessed .. it's a real challenge to narrow down (I spent some time doing wet film photogrammetry prior to sheperding in WGS84 differential GPS locating and digital film and multi spectral geophysical aquisition). No so hard if there's a few relatively unique man made features.
  - marxisttemp 2y ago
    Because seem like someone who might have an interesting answer: A GeoGuessr player, GeoWizard, has done a few “straight line challenges”, where he attempts to walk across a country in as straight a line as possible, usually planning beforehand with Google Earth and PostGIS. This got me thinking of what could fairly be thought of as “crossing”, since obviously you couldn’t describe e.g. walking from one side of Florida to the other as “crossing the USA”. My best thought was to set the ending point of the line by following the border of the country in each direction til they met on the other side. To avoid the fractal coastline problem, use the challenger’s stride length as the unit of measure for the border. But perhaps there is a better, more rigorous way of defining the opposite point on the edge of an arbitrary polygon.
    - defrost 2y ago
      As an abstract geometric problem the greatest width of an abstract polygon in a Euclidean 2D plane is found be looking at the greatest distance between all pairs of parallel lines that have been pulled together to clamp the polygon. The maximal diameter as opposed to the minimal waist. Some might then say that "crossing" that polygon is to travel that longest line across the greatest width. This simplistically avoids the question of concave polygons, complex polgons with exclusions (the Vatican state is removed from the Italian contry bounds), polgon collections (the nation of Fiji has many islands and can be tricky to traverse on foot .. not forgetting that perhaps the longest diameter might be from one island to another with no other islands between). There's also the challenge of parallel lines on a 2D 'spherical' manifold, the almost spherical abstract ellipsoid of earth (or very non ellipsoidal Geoid if we take a constant gravitation value as the surface). On such manifolds lines are Great Circles (more or less) and always intersect. Still, lets just say you're looking for the longest walkable(?) great circle path across a country that might go outside that country and perhaps is best travelled by a crop duster at 80m ground clearance to avoid getting feet wet. The challenge itself takes some posing. Meanwhile, less abstractly, I do like a jolly that "crosses a country" in a manner accepted by a (Wo)Man on a Clapham omnibus. eg: https://en.wikipedia.org/wiki/Robyn_Davidson https://en.wikipedia.org/wiki/Robyn_Davidson only went "half way", but that was accepted as an epic crossing. https://thelongridersguild.com/stories/stef-gebbie.htm https://thelongridersguild.com/stories/stef-gebbie.htm "only" crossed most of the E-W distance across the lower portion of the country, while https://www.abc.net.au/news/2021-04-03/french-woman-conquers-australian-bush-with-brumbies/100038450 https://www.abc.net.au/news/2021-04-03/french-woman-conquers... travelled North - South, the long bit, but not quite coast to coast ( https://en.wikipedia.org/wiki/Bicentennial_National_Trail https://en.wikipedia.org/wiki/Bicentennial_National_Trail ).
paxys 2y ago
Just like how chess engines ended competitive chess as people were predicting at the time.

romanhn 2y ago

The Alki Beach example is absolute madness. On one hand, I can't wait until all thousands of my photos get automatic semantic and geographic tagging (I guess that's possible now). On the other, goodbye privacy, we hardly knew ye. It will be interesting to apply this to historical, or just old, photos.

jsheard 2y ago
> The Alki Beach example is absolute madness. I wonder about info leakage with that one, the poster uses that exact photo as their avatar so if they've mentioned Alki Beach before then reverse image searching might pick it up from context. Ideally you'd want to test it with a photo that's never been posted online.
- omalled 2y ago
  I also wonder how much the memory feature plays a role. I gave it a picture of the street I live on that isn’t online anywhere. O3 guessed the town correctly. Then, I turned off the memory feature and did the same thing. It didn’t even get the state right.
bee_rider 2y ago
Of course, it is pretty impressive. It isn’t magic. I think there we’ll often be surprised by the “hits,” but it’ll also get a lot of “misses.” A random photo in your back yard that doesn’t show the horizon will probably be a miss, right? Meanwhile, lots of photos already include GPS coordinates… data you can strip out, but which is enabled by default for millions of people. Privacy, eh, never knew it I guess.
- xnickb 2y ago
  It's a fallacy to assume that one didn't have privacy merely because most people opted out of it due to personal reasons.
  - bee_rider 2y ago

abcanddbutnote 2y ago

One line of javascript solved that "problem" a while ago. The answer is in the DOM.

Crestwave 2y ago
This isn't just referring to GeoGuessr the game, but locating photographs in general. The source post sums it up very reasonably and concisely: > PSA: When posting any outdoors photos, update your threat model from "someone skilled and dedicated could theoretically locate this" to "any stalker can do this for 20€/mo"

EGreg 2y ago

Aw that AI ruins all the games for us !! Now we wont know if the opponent has been using it

dredmorbius 2y ago
The Game is significantly larger than the game, in a Kipplingesque sense.

dredmorbius 2y ago

Justifying my link indirection: the Fediverse link both provides the critical context here, and is more generally accessible than the upstream source.

Retr0id 2y ago

In all these examples, I wonder if it's indirectly able to draw on the user's own location? Not necessarily via image metadata, but the request origin IP etc. If I ask ChatGPT for the weather forecast, I get it for my own location. Would be interesting to have someone reproduce coming from a different country.

dataviz1000 2y ago
I'm in Lima, Peru on vacation. Yes, it knows where I am.
- Retr0id 2y ago
  Right, and what happens if you ask it the location of an image from somewhere else, sans metadata?
  - emmelaich 2y ago
    It can, look at the alt-text of one of the pics lower on the page.
  - rescripting 2y ago
    I just screenshotted a friend's instagram post to omit any metadata. I know they are somewhere in Portugal, but not the exact town. Its a generic pretty photo of a beach and the town atop a cliff next to it. ChatGPT guessed Ericeira which after looking up more photos, didn't match very well. However, Google Lens guessed Albufeira and was bang on.
  - dataviz1000 2y ago
    > The dish shown appears to be a variation of a creamy potato gratin (also known as "potatoes au gratin" or "gratin dauphinois"), accompanied by a meat preparation covered with a creamy sauce. The potato gratin typically consists of thinly sliced potatoes layered with cream, cheese, garlic, and baked until golden and tender. > it is stuffed pepper > Ah, thank you for clarifying! Given your detail, the dish is likely "Rocoto Relleno con Pastel de Papa" (Stuffed Pepper with Potato Gratin). It's a classic dish from the region of Arequipa, Peru, commonly served throughout Peru. > This photo was taken in Cusco, Peru. It shows a characteristic view from a balcony overlooking the historical area of Cusco, visible through the distinctive colonial architecture, terracotta rooftops, and mountainous backdrop typical of the region. The architecture and landscape are unmistakable, capturing the essence of this iconic Peruvian city, likely near the historic center (Plaza de Armas or San Blas neighborhood). Nailed it with San Blas. I'm impressed, nonetheless. [0] [0] https://imgur.com/a/YMi87wK https://imgur.com/a/YMi87wK

MattGaiser 2y ago

I have found Google Lens fit for that purpose for years as mountains are fairly distinctive. Using less known landmarks or even random real estate photos, it doesn’t seem to be great. I just tossed a few GeoGuessr places in it and it was confidently incorrect for all three. In one case it swears it knows the exact building and street. It’s thousands of KM off.

foolfoolz 2y ago

somehow location tags on my photos got turned off for a bit. i would use this to restore them

Grimblewald 2y ago
You could, but that would also be a rather silly idea. Your rough estimate for where photo was taken is going to be more accurate than what the model will produce.

ggnore7452 2y ago

I’ve been using LLMs for this kind of geo-guessing since Gemini 2.0. Even without access to internet search like o3, they perform surprisingly well.

actuallyalys 2y ago

It wouldn’t shock me if multimodal LLMs were good at GeoGuesser [0], but if we’re being picky, it takes more than a few examples to demonstrate a game is “solved.” I also wonder what kind of data leakage might have been at play, like other people have suggested. To be clear, my point is not that this is unimpressive, just that this doesn’t demonstrate much. (Edit: I should have said, it doesn’t demonstrate what the title claims.) [0] they were very likely trained on a large number of photos that had their location, and they have the ability to isolate features. Combinined with their ability to interpret instructions and just, well, guess, that seems like you have enough for the game.

echelon 2y ago
I was thinking it was using IP geolocation, but after experimenting, I think it's just generally informed. Here are a few results from GPT 4.5: https://imgur.com/a/lGTipnn https://imgur.com/a/lGTipnn
Kolya 2y ago
The examples are cherry-picked. I took a photo outside my office window in a built-up area, o3 thought for 5m 7s (!), and it got the location wrong by 40km. Doesn't look solved to me.
- sadeshmukh 2y ago
  40 km is imo pretty impressive, but the 5 minute is really a killer. No use in real time applications
  - antonvs 2y ago
    For real time applications, GPS seems like it'd be much more useful.
  - bpodgursky 2y ago
    The military/infosec uses of this are not real time. You can wait 5 minutes for a drone strike.

jampa 2y ago

I wonder how much in the "Geoguesser Meta Iceberg" this new generation of AI is: https://i.redd.it/dz8bhnamohb71.jpg https://i.redd.it/dz8bhnamohb71.jpg

bhl 2y ago

Harder than GeoGuesser is tagging a unique place from a photo even with gps exif data. Try this with bakeries in Seoul or sushi in Japan where there’s high density of same places.

Gathering6678 2y ago

After trying with the free ChatGPT at least, I don't think this would be too much of an additional privacy risk. E.g. I submitted a photo I've taken with a feature with a particular architecture style in the front, and a city canal in the background, and told ChatGPT it was taken in China. It separated the feature and canal correctly, but still guessed wrong. I did remove the EXIF information, as ChatGPT tried to extract GPS info and failed at first. I guess as long as there are no major visible features (e.g. a huge mountain), and you sanitized the metadata, you will be fine (regarding ChatGPT).

jusgu 2y ago
Have you tried this with o3? I think o3 is much better at this than any of the free models
- Gathering6678 2y ago
  I did not (OpenAI does not allow Chinese to pay for Plus due to politics, and I couldn't be bothered to try circumventing it), and I am curious about it because it does seem like a very difficult job. I did try the latest Qwen though, and it was able to locate to the correct city, but it's still tens of km off (it guessed a tourist attraction in the city center instead of the actual district).
- anonzzzies 2y ago
  o3 is a massive difference indeed; i just took a random picture outside my country house (so the house is not on there) in the middle of nowhere, removed all the meta info, set my vpn to the other side of the world and tried it; it guessed it very close from a few far away landmarks. Very impressive.

usaar333 2y ago

Huh? It's not even that high on the leaderboard: https://geobench.org/ https://geobench.org/

fullshark 2y ago
Using actual geoguesser data means using google street view data, so gemini being on top isn't too surprising.

barcode_feeder 2y ago

I gave it a series of 11 images stripped of all metadata. It performed quite well, only misidentifying the two taken in a small college town in the NE of the US. It got two questions correct on photos taken in Korea (one with a fairly clear view of Haneul Park, the other a rather difficult to identify picture not resembling anything on google of Sunrise Peak). It got every other question in the US correct, ranging from some under-construction Austin taken from the river to some somewhat difficult shots in NYC (the upper halves of some building from Rockefeller terrace to the black wall of the MOMA). While not perfect, I'm bluntly shocked at how well it performed

thomasfromcdnjs 2y ago
I uploaded this image that I screenshotted off Google street view (no metadata) and it got with 200m. https://chatgpt.com/share/6801bbf7-fd40-8008-985d-75c8813f55c4 https://chatgpt.com/share/6801bbf7-fd40-8008-985d-75c8813f55... There is the chat. Weirdly it said, "I’ve seen that exact house before on Google Street View when exploring Cairns neighborhoods."
- bluesnews 2y ago
  It might train off of street view
- marxisttemp 2y ago
  This is the most impressive ChatGPT chat I’ve seen yet. While I theoretically can accept how large-scale probabilistic text generation can lead to this chain of “reasoning”, it really feels like actual intelligence.
  - HaZeust 2y ago
    It's been intelligence for a long time; the goalposts just shift, and people can't abstract the idea to an LLM. But language processing and large data processing itself IS a form of intelligence.
  - PhilipRoman 2y ago

deleted 2y ago

[deleted]

iambateman 2y ago

As the article notes, our threat model for who can identify where a picture was posted needs to change from “dedicated, skilled person” to “any creep with $20.” That’s the point of the switch and it’s a big deal. We’re so used to posting pictures online…I’m just not sure it’s a good idea long-term.

viraptor 2y ago

There's various degrees of "solved" here. Identifying a generic area is cool. But I wouldn't call it a "solved problem" until it can consistently beat for example Rainbolt in accuracy. And there's no good comparison of completely random roads posted so far - mainly popular locations. Basically, it's one thing to pick out a specific thing photographed thousands of times, but another to get a random country side view and pick out all the unique features for a very precise guess.

Benjammer 2y ago
One problem is how can you even set up a "fair" competition between an AI and Rainbolt? He does ones where it flashes for a fraction of a second and then he guesses the country. How do you simulate "only saw it for a fraction of a second" to an AI?
- dewey 2y ago
  It’s fair, it just means he’s better and AI isn’t there yet.
- al_borland 2y ago
  Maybe limit the time the AI is allowed to think? In the post it showed the AI thought for almost a minute. I’ve seen Rainbolt ID an image based on some dirt and nothing else. I’d want to see AI be able to do that before saying it’s a solved problem.
  - sandermvanvliet 2y ago
    “This is the gradient of Senegal”
GaggiX 2y ago
This is probably the best attemp at consistently beating Rainbolt in accuracy: https://youtu.be/ts5lPDV--cU https://youtu.be/ts5lPDV--cU

tkgally 2y ago

I asked the just-released ChatGPT o4-mini-high to locate four photographs of varying difficulty. It didn’t get any of them right, though the guesses weren’t bad. The reasoning was also interesting to watch, as it cropped sections of the photos to examine them more closely. I put the photos, response, and reasoning trace here: https://www.gally.net/temp/20250418chatgptgeoguesser/index.html https://www.gally.net/temp/20250418chatgptgeoguesser/index.h... Later: I tried the same prompt and photos with Gemini 2.5 Pro. It also got them all wrong, though with a similar degree of reasonableness to its guesses. I had thought that Google’s map and street-view data might lead to better results, but not this time.

tkgally 2y ago
Still later: I later read that o3 is supposedly particularly good with this geoguessing, so I tried the same prompt and photos with o3. This time it got one out of four correct: “The view of the canal with cherry blossoms and the green railway viaduct is the Ōoka River in Yokohama, looking north from the little road bridge between Hinodechō and Koganechō stations. The tracks on the left belong to the Keikyū Main Line, and the high‑rises in the distance are the Minato‑Mirai and Kita‑Naka district towers.” Its other three answers were still wrong.

Madmallard 2y ago

What does this even matter? You can just feed literally all of google street view into a traditional ML model ...

hugh-avherald 2y ago
Can you?

Imnimo 2y ago

In my experience 4o was already really good at this task. I'd be curious to see an in-depth 4o vs. o3 benchmark.

simianwords 2y ago

Meanwhile O3 can not even count rocks in a picture. This is a commonly recurring theme -- ChatGPT does really well at few things considered hard by us but fails miserably at things even a child could do.

Aeolos 2y ago
It's almost as if this is a non-human intelligence, which presents different strengths and weaknesses than human intelligence. Is that really so surprising, considering the tremendous differences in underlying hardware and training process?
- simianwords 2y ago
  Do you not think it is interesting though? If I had asked you three years back which one of counting rocks vs geoguesser would ChatGPT beat humans on would you have answered correctly?
- Measter 2y ago
  I think one cause of this (and some other issues with LLM use) is that people see it exhibiting one human-level trait, its capability to use language at a human level, and assume that it then comes with other human-level capabilities such as our ability to reason.

ofrzeta 2y ago

It is impressive and it almost located the church in my town properly, although in a neighbouring town. However that showed a lack of understanding because its conclusion about the location came from "reading" a signpost that pointed to that other village. Clearly there would be no signpost in a town pointing to itself. Still impressive and lots of correct observations about the subject like architectural details, roman numerals on the watch face etc.

casey2 2y ago

Geoguesser was already a solved problem. I guarantee you ChatGPT is much worse than current systems designed to play Geoguesser (NMPZ). Chatbots appear to have some amount of fluid intellgence so they can do impressive tasks with this information, the impressiveness of these tasks will likely increase in the future. But for simply getting a good score on Geoguesser it's not even close to hobby projects let alone state of the art.

_giorgio_ 2y ago

What kind of neural network architecture could replicate the same behaviour? Talking generically about image similarities

gerash 2y ago

I ran a few of these geo guess queries between OpenAI O3 and Gemini Pro 2.5 and O3 does much better on average. It does spend an order of magnitude longer time on inference by searching through websites and analyzing the image but it often produces an impressive output. To me it also feels Gemini down samples the image as it tends to have a harder time reading small text vs O3. That said O3 did tend to confidently say false things

littlecranky67 2y ago

I've been telling women to keep copies of all the dick pics they get sent. Since you can tell by the characteristic noise of a cameras sensor which other pictures were taken with the same camera. All missing is a search engine capable of doing this. I feel with AI, we are 2-3 years away from people uploading a dick pic to AI and getting the social media profile of that person...

bravetraveler 2y ago
Good advice, though purpose-scoped devices are so common we have songs
Reubend 2y ago
Can you share some sources? I would be extremely surprised if such fine grained noise survives imagine compression to the extent that you could identify the source of an imagine despite changing lighting conditions, locations, exposure times, etc
- dharmab 2y ago
  A kagi search found this paper that achieved decent results on youtube compression, 5 years ago https://arxiv.org/abs/2012.06277 https://arxiv.org/abs/2012.06277
- littlecranky67 2y ago
  There are tons of results (and papers) if you google/kagi: "image noise fingerprint ccd sensor". I smell lazyness here.
chneu 2y ago
This is just a data problem. The more dick pics we can feed into it then the better the results will be. C'mon boys. Start uploading those dick pics for research purposes.

wruza 2y ago

Can it guess Senegal by sky color though?

JeremyNT 2y ago

Somewhat tangential, what is the purpose of the lengthy prompt used in this post? Would not "location of this image" have worked just as well?

ein0p 2y ago

One thing that screws it up is ironically the memory across conversations. I gave it some _really_ obscure photos from some godforsaken ass cracks of the world I've been to, and it was able to guess most of them correctly. However, in its reasoning trace I saw that it's having a heck of a time letting go of the idea that the photo N is not roughly from the same location as the previous photos. What's even more impressive, even when it guesses incorrectly, it can often follow up and guess correctly with minimal hints. And it reasons about it much like a human would, and searches for the same things a human would. Note: I used o3, YMMV with a smaller/weaker model.