2 ms·

People slam this for being ChatGPT but I wonder how often the references it finds on the web are more relevant than ones in this [1] style of HN [2] comment tha

People slam this for being ChatGPT but I wonder how often the references it finds on the web are more relevant than ones in this [1] style of HN [2] comment that often get to the top of this type of thread yet lack any deep analysis either. Slamming it for not validating the sources support the claim at least makes sense but isn't really specific to the ChatGPT fixation. Perhaps somewhat telling on that is the comments are focusing on how the source was found rather than debating or correcting the figures themselves. The first figure sourced seems both accurately sourced and accurately estimated. The second figure I'm having trouble finding in the linked source and the figure seems to be more based on renewable energy investment not renewable energy research. I'm also having a hard time finding any good source for current renewable energy research investment to replace that with though so I can't say much in regards to how the final conclusion fairs.

SV_BubbleTime 2y ago
The problem is that even when incorrect, typically someone intended the correct information to on a website. Where LLMs are just an inference of related words. So by your scenario, the LLM also picked up the incorrect answer on a website and included it into its inference. He isn’t downvoted for being lazy, he’s downvoted for using a language model as a fact model. Even if sometimes it does work.
- zamadatix 2y ago
  The problem has nothing to do with how you're finding your sources and everything to do with the lack of bothering to verify and understand them properly. What does it matter if you got your source list from tea leaves so long as you validate the information in the sources? Nothing replaces actually going out and verifying anything yourself (yet) but that doesn't make tools which do a first past somehow to blame for someone's laziness while tools that put no effort in at all are supposed to be just fine.

SV_BubbleTime 2y ago

The problem is that even when incorrect, typically someone intended the correct information to on a website. Where LLMs are just an inference of related words. So by your scenario, the LLM also picked up the incorrect answer on a website and included it into its inference. He isn’t downvoted for being lazy, he’s downvoted for using a language model as a fact model. Even if sometimes it does work.

zamadatix 2y ago
The problem has nothing to do with how you're finding your sources and everything to do with the lack of bothering to verify and understand them properly. What does it matter if you got your source list from tea leaves so long as you validate the information in the sources? Nothing replaces actually going out and verifying anything yourself (yet) but that doesn't make tools which do a first past somehow to blame for someone's laziness while tools that put no effort in at all are supposed to be just fine.