4 ms·

GPT-5 Thinking in ChatGPT (a.k.a. Research Goblin) is shockingly good at search

52 points by klelatti 1y ago

yellow_lead 1y ago
To me, there's nothing shocking in here. I do think that search has gotten slightly better in GPT-5, but these examples all feel like softballs to me. Also as a note to the author - a lot of links are missing spaces after them.
- simonw 1y ago
  Rats, that looks like a bug in my copy-and-paste to Substack workflow.
simonw 1y ago
This is a link to my duplicated newsletter, but the better URL would be the post itself: https://simonwillison.net/2025/Sep/6/research-goblin/ https://simonwillison.net/2025/Sep/6/research-goblin/ I just posted a follow-up about how Google's new "AI mode" is also impressive (notably different from "AI overviews" which are terrible): https://simonwillison.net/2025/Sep/7/ai-mode/ https://simonwillison.net/2025/Sep/7/ai-mode/
- card_zero 1y ago
  You should have posted the instructive failures you mention, not just the successes.
  - simonw 1y ago
    This one was meant to be illustrative of a (thought provoking) failure https://simonwillison.net/2025/Sep/6/research-goblin/#history-of-the-caverns-in-exeter-quay https://simonwillison.net/2025/Sep/6/research-goblin/#histor...
    - card_zero 1y ago
      I wondered what "something I’m certain it won’t be able to handle" would look like.
      - simonw 1y ago
        I tried "Is the pelican in her pity unique to English churches or does it show up in France too?" (Easy) followed by "Find places I can see this near nontron in France" (which I anticipated to be extremely hard): https://chatgpt.com/share/68bd9ca1-f6d0-8006-a507-c81785795789 https://chatgpt.com/share/68bd9ca1-f6d0-8006-a507-c817857957... It gave me a result that looked convincing! But I don't trust it. I very much doubt the internet has reliable, comprehensive data about instances of that piece of religious imagery in small churches around this region of France.
- dang 1y ago
  We've merged the comments* to https://news.ycombinator.com/item?id=45152284 https://news.ycombinator.com/item?id=45152284 and re-upped that one. (* except the ones that only make sense here)
- avinassh 1y ago
  Does substack let you add canonical URLs for the post? Otherwise won't you penalised by search engines for duplicated content?
  - simonw 1y ago
    I decided I am willing to take that risk.
indigodaddy 1y ago
Reposting my comment from simonw's original item that somehow didn't catch on like this item did. Pretty wild! I wonder how much high school teachers and college professors are struggling with the inevitable usage though? "Do deep internet research and thinking to present as much evidence in favor of the idea that JRR Tolkein's Lord of the Rings trilogy was inspired by Mervyn Peake's Gormenghast series." https://chatgpt.com/share/68bcd796-bf8c-800c-ad7a-51387b1e531b https://chatgpt.com/share/68bcd796-bf8c-800c-ad7a-51387b1e53...
- dang 1y ago
  I know it's well intentioned but please don't copy/paste comments across threads - it makes merging threads a pain. If you wanted, you could give us a heads-up about the split thread at hn@ycombinator.com - that way we can hopefully get around to merging them sooner.
  - indigodaddy 1y ago
    Noted, and sorry about that! Since that post/item hadn't caught on I thought it was left in the dust. But makes sense to merge them, didn't think about that happening.

52 points by klelatti 1y ago

To me, there's nothing shocking in here. I do think that search has gotten slightly better in GPT-5, but these examples all feel like softballs to me. Also as a note to the author - a lot of links are missing spaces after them.

simonw 1y ago
Rats, that looks like a bug in my copy-and-paste to Substack workflow.

simonw 1y ago

This is a link to my duplicated newsletter, but the better URL would be the post itself: https://simonwillison.net/2025/Sep/6/research-goblin/ https://simonwillison.net/2025/Sep/6/research-goblin/ I just posted a follow-up about how Google's new "AI mode" is also impressive (notably different from "AI overviews" which are terrible): https://simonwillison.net/2025/Sep/7/ai-mode/ https://simonwillison.net/2025/Sep/7/ai-mode/

card_zero 1y ago
You should have posted the instructive failures you mention, not just the successes.
- simonw 1y ago
  This one was meant to be illustrative of a (thought provoking) failure https://simonwillison.net/2025/Sep/6/research-goblin/#history-of-the-caverns-in-exeter-quay https://simonwillison.net/2025/Sep/6/research-goblin/#histor...
  - card_zero 1y ago
    I wondered what "something I’m certain it won’t be able to handle" would look like.
    - simonw 1y ago
      I tried "Is the pelican in her pity unique to English churches or does it show up in France too?" (Easy) followed by "Find places I can see this near nontron in France" (which I anticipated to be extremely hard): https://chatgpt.com/share/68bd9ca1-f6d0-8006-a507-c81785795789 https://chatgpt.com/share/68bd9ca1-f6d0-8006-a507-c817857957... It gave me a result that looked convincing! But I don't trust it. I very much doubt the internet has reliable, comprehensive data about instances of that piece of religious imagery in small churches around this region of France.
dang 1y ago
We've merged the comments* to https://news.ycombinator.com/item?id=45152284 https://news.ycombinator.com/item?id=45152284 and re-upped that one. (* except the ones that only make sense here)
avinassh 1y ago
Does substack let you add canonical URLs for the post? Otherwise won't you penalised by search engines for duplicated content?
- simonw 1y ago
  I decided I am willing to take that risk.

indigodaddy 1y ago

Reposting my comment from simonw's original item that somehow didn't catch on like this item did. Pretty wild! I wonder how much high school teachers and college professors are struggling with the inevitable usage though? "Do deep internet research and thinking to present as much evidence in favor of the idea that JRR Tolkein's Lord of the Rings trilogy was inspired by Mervyn Peake's Gormenghast series." https://chatgpt.com/share/68bcd796-bf8c-800c-ad7a-51387b1e531b https://chatgpt.com/share/68bcd796-bf8c-800c-ad7a-51387b1e53...

dang 1y ago
I know it's well intentioned but please don't copy/paste comments across threads - it makes merging threads a pain. If you wanted, you could give us a heads-up about the split thread at hn@ycombinator.com - that way we can hopefully get around to merging them sooner.
- indigodaddy 1y ago
  Noted, and sorry about that! Since that post/item hadn't caught on I thought it was left in the dust. But makes sense to merge them, didn't think about that happening.