3 ms·

Trusting LLM advice about poisons seems… sort of like being a test pilot for a brand new aerospace company with no reputation for safety.

selfhoster11 1y ago
Only if you don't check it against a classical search query later. Not to mention that all you might get is search results from slop farms that aren't concerned with safety or factuality - ones that were a problem before LLMs.
- soraminazuki 1y ago
  So you agree that you need to do thorough research and be careful about which sources you trust. At that point, why not jump straight to actual research and bypass the information laundering machine entirely? Even you acknowledge that you can't trust what it says.
  - selfhoster11 1y ago
    Yes, I do. Why not jump into actual research from scratch then? 1. To search, you need to know the right search terms. An LLM might, in a rare scenario produce a nonsensical answer that still contains two or three domain-specific terms that you can plug into a search engine. Pull the thread, and see where it takes you. You literally cannot do that with (current) search engines unless you already know the terms. 2. Because validation is far quicker and takes far less effort than researching from scratch. If an LLM tells you that "poison XYZ interferes with levels of X in blood, which inhibits pathway ABC, and you die", then you can easily verify whether poison XYZ interferes with levels of X in blood (which if it doesn't, you know the answer is incorrect), and whether if the levels of X in blood are too high or too low, then pathway ABC is inhibited (which if it isn't, you know the answer is incorrect. If you can verify both facts, then the LLM's answer is correct. You do two pinpoint search queries that give you an answer in 30 seconds each, instead of having to do the research yourself for a lot longer than that.
tim333 1y ago
I agree in general but it wasn't of much importance whether my carpet beetles died or not.

selfhoster11 1y ago

Only if you don't check it against a classical search query later. Not to mention that all you might get is search results from slop farms that aren't concerned with safety or factuality - ones that were a problem before LLMs.

soraminazuki 1y ago
So you agree that you need to do thorough research and be careful about which sources you trust. At that point, why not jump straight to actual research and bypass the information laundering machine entirely? Even you acknowledge that you can't trust what it says.
- selfhoster11 1y ago
  Yes, I do. Why not jump into actual research from scratch then? 1. To search, you need to know the right search terms. An LLM might, in a rare scenario produce a nonsensical answer that still contains two or three domain-specific terms that you can plug into a search engine. Pull the thread, and see where it takes you. You literally cannot do that with (current) search engines unless you already know the terms. 2. Because validation is far quicker and takes far less effort than researching from scratch. If an LLM tells you that "poison XYZ interferes with levels of X in blood, which inhibits pathway ABC, and you die", then you can easily verify whether poison XYZ interferes with levels of X in blood (which if it doesn't, you know the answer is incorrect), and whether if the levels of X in blood are too high or too low, then pathway ABC is inhibited (which if it isn't, you know the answer is incorrect. If you can verify both facts, then the LLM's answer is correct. You do two pinpoint search queries that give you an answer in 30 seconds each, instead of having to do the research yourself for a lot longer than that.

tim333 1y ago

I agree in general but it wasn't of much importance whether my carpet beetles died or not.