5 ms·

I gave it a series of 11 images stripped of all metadata. It performed quite well, only misidentifying the two taken in a small college town in the NE of the US

I gave it a series of 11 images stripped of all metadata. It performed quite well, only misidentifying the two taken in a small college town in the NE of the US. It got two questions correct on photos taken in Korea (one with a fairly clear view of Haneul Park, the other a rather difficult to identify picture not resembling anything on google of Sunrise Peak). It got every other question in the US correct, ranging from some under-construction Austin taken from the river to some somewhat difficult shots in NYC (the upper halves of some building from Rockefeller terrace to the black wall of the MOMA). While not perfect, I'm bluntly shocked at how well it performed

thomasfromcdnjs 1y ago
I uploaded this image that I screenshotted off Google street view (no metadata) and it got with 200m. https://chatgpt.com/share/6801bbf7-fd40-8008-985d-75c8813f55c4 https://chatgpt.com/share/6801bbf7-fd40-8008-985d-75c8813f55... There is the chat. Weirdly it said, "I’ve seen that exact house before on Google Street View when exploring Cairns neighborhoods."
- bluesnews 1y ago
  It might train off of street view
- marxisttemp 1y ago
  This is the most impressive ChatGPT chat I’ve seen yet. While I theoretically can accept how large-scale probabilistic text generation can lead to this chain of “reasoning”, it really feels like actual intelligence.
  - HaZeust 1y ago
    It's been intelligence for a long time; the goalposts just shift, and people can't abstract the idea to an LLM. But language processing and large data processing itself IS a form of intelligence.
  - PhilipRoman 1y ago
    Maybe you're right, but I think it's more likely that it had been trained on street view photos and then invented a plausible justification for the guess afterwards (which is something I often see ChatGPT do, when it easily arrives at the correct answer, but gives bullshit explanations for it).
- thomasfromcdnjs 1y ago
  This was the image: https://imgur.com/a/cCUvgDG https://imgur.com/a/cCUvgDG
- geysersam 1y ago
  > Weirdly it said, "I’ve seen that exact house before on Google Street View when exploring Cairns neighborhoods." That's slightly creepy!
  - oezi 1y ago
    The anthropomorphisation certainly is weird. But the technical aspect seems even weirder. Did OpenAI really build dedicated tools to have their models train on Google Street View? Or do they have generic technology for browsing complex sites like Street view?
    - comex 1y ago
      It’s just a hallucination, same idea as o3 claiming that it uses its laptop to mine Bitcoin: https://transluce.org/investigating-o3-truthfulness https://transluce.org/investigating-o3-truthfulness I doubt the model was trained on Street View, but even if it was, LLMs don’t retain any “memory” of how/when they were trained, so any element of truthfulness would be coincidental.
      - geysersam 1y ago
        If it's trained on street view data it's not unlikely that the model can associate a particular piece of context to street view. For example, a picture can have telltale signs that street view content has, such as blurred faces and street signs, watermarks, etc. Even if it's not directly trained on street view data it has probably encountered street view content in it's training dataset.
        oezi 1y ago
        I think the test which the OP performed (to pick a random street view and let it pinpoint it) would indicate that it has ingested some kind of information in this regard in a structured manner.
        namaria 1y ago
        The training process doesn't preserve information needed for the LLM to infer that. It cannot be anything other than nonsense that sounds plausible, which is what they do best.
CSMastermind 1y ago
I played a round of Geoguessr against it and while it did a shockingly good job compared to what I was expecting, it still lags behind even novice human players. The locations and its guesses were: Bliss, Idaho - Burns, Oregon (273 miles away) Quilleco, Biobio, Chile - Eugene, Oregon (6,411 miles away) Dettighofen, Switzerland - Mühldorf, Germany (228 miles away) Pretoria, South Africa - Johannesburg, South Africa (36 miles away) Rockhampton, Australia - Gold Coast, Australia (437 miles away)
- CSMastermind 1y ago
  Okay, I decided to benchmark a bunch of AI models with geoguessr. One round each on diverse world, here's how they did out of 25,000: Claude 3.7 Sonnet: 22,759 Qwen2.5-Max: 22,666 o3-mini-high: 22,159 Gemini 2.5 Pro: 18,479 Llama 4 Maverick: 14,316 mistral-large-latest: 10,405 Grok 3: 5,218 Deepseek R1: 0 command-a-03-2025: 0 Nova Pro: 0
  - nemo1618 1y ago
    Neat, thanks for doing this!
  - msephton 1y ago
    How does Google Lens compare?
    - CSMastermind 1y ago
      I tried it but as far as I can tell Google Lens doesn't give you a location - it just describes generally what you're looking at.
      - msephton 1y ago
        I had cause to try Google Lens today and found the location to exact address thanks to a veterinary clinic which was in the background of an image. ChatGPT got the country but wrong city.
  - 1y ago
delusional 1y ago
I gave It some photos from denmark, didn't even bother to strip the metadata. One is correctly said give of "Scandinavian vibes" every other photo was very wrong. I also gave it a photo of the french Alps, it guessed Switzerland.
Measter 1y ago
I gave o4-mini-high a cropped version of a photo I found on Facebook[0][1], and it quickly determined that this was in the UK from the road markings. It also decided that it was from a coastal city because it could see water on the horizon, which is the correct conclusion from incorrect data. There is no water, I think that's trees on a hill. It focused heavily on the spherical structure, which makes sense because it's distinctive, though it had a hard time placing it. It also decided that the building on the left was probably a shopping centre. It eventually decided that the photo was taken outside the Scottish Exhibition and Conference Centre in Glasgow. It actually generally considered Scottish locations more than others. The picture was actually taken in Plymouth (so pretty much as far from Scotland as you can get in Britain), on Charles Street looking south-east[2]. The building on the right is Drake Circus, and the one on the left is the Arts University. It actually did consider Plymouth, but decided it didn't match. [0] This image with the "university plymouth" on the left cropped out, just to make it harder: https://www.facebook.com/photo/?fbid=9719044988151697&set=gm.9558109594275124&idorvanity=2683098095109676 https://www.facebook.com/photo/?fbid=9719044988151697&set=gm... [1] https://chatgpt.com/share/68024c91-61d0-800c-99b1-fcecf0bfe88d https://chatgpt.com/share/68024c91-61d0-800c-99b1-fcecf0bfe8... [2] https://maps.app.goo.gl/3TXv2UxH5128xQjJ9 https://maps.app.goo.gl/3TXv2UxH5128xQjJ9

thomasfromcdnjs 1y ago

I uploaded this image that I screenshotted off Google street view (no metadata) and it got with 200m. https://chatgpt.com/share/6801bbf7-fd40-8008-985d-75c8813f55c4 https://chatgpt.com/share/6801bbf7-fd40-8008-985d-75c8813f55... There is the chat. Weirdly it said, "I’ve seen that exact house before on Google Street View when exploring Cairns neighborhoods."

bluesnews 1y ago
It might train off of street view
marxisttemp 1y ago
This is the most impressive ChatGPT chat I’ve seen yet. While I theoretically can accept how large-scale probabilistic text generation can lead to this chain of “reasoning”, it really feels like actual intelligence.
- HaZeust 1y ago
  It's been intelligence for a long time; the goalposts just shift, and people can't abstract the idea to an LLM. But language processing and large data processing itself IS a form of intelligence.
- PhilipRoman 1y ago
  Maybe you're right, but I think it's more likely that it had been trained on street view photos and then invented a plausible justification for the guess afterwards (which is something I often see ChatGPT do, when it easily arrives at the correct answer, but gives bullshit explanations for it).
thomasfromcdnjs 1y ago
This was the image: https://imgur.com/a/cCUvgDG https://imgur.com/a/cCUvgDG
geysersam 1y ago
> Weirdly it said, "I’ve seen that exact house before on Google Street View when exploring Cairns neighborhoods." That's slightly creepy!
- oezi 1y ago
  The anthropomorphisation certainly is weird. But the technical aspect seems even weirder. Did OpenAI really build dedicated tools to have their models train on Google Street View? Or do they have generic technology for browsing complex sites like Street view?
  - comex 1y ago
    It’s just a hallucination, same idea as o3 claiming that it uses its laptop to mine Bitcoin: https://transluce.org/investigating-o3-truthfulness https://transluce.org/investigating-o3-truthfulness I doubt the model was trained on Street View, but even if it was, LLMs don’t retain any “memory” of how/when they were trained, so any element of truthfulness would be coincidental.
    - geysersam 1y ago
      If it's trained on street view data it's not unlikely that the model can associate a particular piece of context to street view. For example, a picture can have telltale signs that street view content has, such as blurred faces and street signs, watermarks, etc. Even if it's not directly trained on street view data it has probably encountered street view content in it's training dataset.
      - oezi 1y ago
        I think the test which the OP performed (to pick a random street view and let it pinpoint it) would indicate that it has ingested some kind of information in this regard in a structured manner.
      - namaria 1y ago
        The training process doesn't preserve information needed for the LLM to infer that. It cannot be anything other than nonsense that sounds plausible, which is what they do best.

CSMastermind 1y ago

I played a round of Geoguessr against it and while it did a shockingly good job compared to what I was expecting, it still lags behind even novice human players. The locations and its guesses were: Bliss, Idaho - Burns, Oregon (273 miles away) Quilleco, Biobio, Chile - Eugene, Oregon (6,411 miles away) Dettighofen, Switzerland - Mühldorf, Germany (228 miles away) Pretoria, South Africa - Johannesburg, South Africa (36 miles away) Rockhampton, Australia - Gold Coast, Australia (437 miles away)

CSMastermind 1y ago
Okay, I decided to benchmark a bunch of AI models with geoguessr. One round each on diverse world, here's how they did out of 25,000: Claude 3.7 Sonnet: 22,759 Qwen2.5-Max: 22,666 o3-mini-high: 22,159 Gemini 2.5 Pro: 18,479 Llama 4 Maverick: 14,316 mistral-large-latest: 10,405 Grok 3: 5,218 Deepseek R1: 0 command-a-03-2025: 0 Nova Pro: 0
- nemo1618 1y ago
  Neat, thanks for doing this!
- msephton 1y ago
  How does Google Lens compare?
  - CSMastermind 1y ago
    I tried it but as far as I can tell Google Lens doesn't give you a location - it just describes generally what you're looking at.
    - msephton 1y ago
      I had cause to try Google Lens today and found the location to exact address thanks to a veterinary clinic which was in the background of an image. ChatGPT got the country but wrong city.
- 1y ago

delusional 1y ago

I gave It some photos from denmark, didn't even bother to strip the metadata. One is correctly said give of "Scandinavian vibes" every other photo was very wrong. I also gave it a photo of the french Alps, it guessed Switzerland.

Measter 1y ago

I gave o4-mini-high a cropped version of a photo I found on Facebook[0][1], and it quickly determined that this was in the UK from the road markings. It also decided that it was from a coastal city because it could see water on the horizon, which is the correct conclusion from incorrect data. There is no water, I think that's trees on a hill. It focused heavily on the spherical structure, which makes sense because it's distinctive, though it had a hard time placing it. It also decided that the building on the left was probably a shopping centre. It eventually decided that the photo was taken outside the Scottish Exhibition and Conference Centre in Glasgow. It actually generally considered Scottish locations more than others. The picture was actually taken in Plymouth (so pretty much as far from Scotland as you can get in Britain), on Charles Street looking south-east[2]. The building on the right is Drake Circus, and the one on the left is the Arts University. It actually did consider Plymouth, but decided it didn't match. [0] This image with the "university plymouth" on the left cropped out, just to make it harder: https://www.facebook.com/photo/?fbid=9719044988151697&set=gm.9558109594275124&idorvanity=2683098095109676 https://www.facebook.com/photo/?fbid=9719044988151697&set=gm... [1] https://chatgpt.com/share/68024c91-61d0-800c-99b1-fcecf0bfe88d https://chatgpt.com/share/68024c91-61d0-800c-99b1-fcecf0bfe8... [2] https://maps.app.goo.gl/3TXv2UxH5128xQjJ9 https://maps.app.goo.gl/3TXv2UxH5128xQjJ9