4 ms·
LLMs are definitely yes-men. You ask it to do something, and it goes like "Yes sir" with lots of emojis, and then confidently gives you the wrong answer. :D Un
by johnisgood 1y ago
LLMs are definitely yes-men. You ask it to do something, and it goes like "Yes sir" with lots of emojis, and then confidently gives you the wrong answer. :D
Unless of course it clashes with the ToS, then it will never help you (unless you manage to engineer your prompts just right), even when it comes to something as basic as pharmacology.
- Bluestein 1y agoThe thought of LLMs doing pharmacology sends shivers down me spine ...
- johnisgood 1y agoOh no, I meant it refusing to answer pharmacology 101 questions. At one time it did not want to give me the pharmacokinetics of one medication because of its ToS.
- Bluestein 1y agoI see - thanks.- (That sounds broken. It's basic, useful, harmless information ...)
- johnisgood 1y agoExactly. I remember saying "but it is a publicly available information", that it is in books, etc. and it went on and on about "I understand your frustration, but I cannot give you ...", etc. :D In the end I did manage to get it to answer, however. This was (is) not limited to pharmacology, however.
- Bluestein 1y agoI am interested: What other fields? How did you manage?
- johnisgood 1y agoThankfully there is now searchability, so I would have to search for this particular chat, this was some time ago. I think once it starts to tell you "no", it almost never will turn to "okay, I'll tell you" (maybe only if there were one or two "no"s), so I probably started a new chat, and added how it is for research purposes and whatnot. I do not remember the specifics and I would really have to search for the chat to be able to give you specifics. I may have to search for "sorry" or something. I found something though. I asked it for the interaction between two medications, and it told me "I am sorry you are going through this, but I can't give you information about ...", which I guess is still pharmacology. Edit: I remember another case. It was related to reverse engineering. Edit #2: I found another! I cannot give you specifics, but I asked for it to give me some example sentences for something, and it went on about "consensual relationships" and so forth, even though my partner consented. I told the LLM that she did consent and it went on about that "I'm sorry, but even if she consented, I cannot give you ... because ... and is not healthy ...", and so forth. (Do not think of anything bad here! :P)
- rvnx 1y agoClaude convinced me to take a very important medication, without Claude I would not have had the balls to take it (because of the potential side-effects) and it showed me all the possible medications, and the benefits/risk balance between this and being untreated. At the end, went to doctor, offered me the same choices. No regrets so far. Without it, I would suffer for nothing. It is a very good tool to investigate and discover things, could be also the opposite: seen a bit unproductive that it is censored, because doctors are mostly here for validation anyway for serious things.
- sgt101 1y agoYou are a sensible person and took professional advice. Other people are not sensible - look at all the junkies shooting who knows what into who knows where. A system that can advise cooks to put glue on pizza is probably not the best mechanism for providing healthcare advice to at least the section of the population who are likely to try it.
- rvnx 1y agoI fully agree with you. It is an interesting tool, but to consider carefully, like you wouldn’t trust 100% what is written on the internet.
- alwa 1y agoWould you suppose said junkies would be taking advice from a qualified medical professional otherwise? For that matter, would you suppose that they don’t know what they’re doing is bad for them? Witness the smokers who crack jokes about “cancer sticks” as they light up. It seems to me that, just as we all promise our dentists we floss every day and our doctors that we promise we’ll eat better, we might be more prone to be frank with “anonymous” chatbots—and thus get more frank advice in return, at least for clear-cut factual background and tradeoffs.
- sgt101 1y agoI totally agree if the chatbots are actually reliable and controllable. LLMs have real problems with both of those things, they are unreliable in the sense that they produce text with factual inaccuracies and they are uncontrollable in that it's really hard to predict their behaviour from their past behaviour. Big open challenges in AI - I am sure that there will be solutions, but there aren't right now. I think it's very like self driving cars. Implementing a demo and showing it on the road is one thing, but then it takes twenty years before the slow roll out really gets underway.
- rusticpenn 1y agoUsing Neural nets, Deep Learning in pharma is not something new. Naturally LLMs are different. You should look into personalized medicine.
- touwer 1y agoIn this case, if it's only shivers down your spine, you're lucky ;)
- Bluestein 1y agoHeh. Well played.-
- dandanua 1y agoLLMs are the the best conmen. If they were humans people would trust them their life savings for sure.
- rvnx 1y agoOne technique is to ask: “are you sure?” “How sure are you on /20” If it says yes I am sure, and other LLMs confirms the same way, then you can be fairly confident that it is a very good candidate answer. If it says he is not sure, he is probably just agreeing with you, and better double-check by asking “is there other solutions ? What is the worst idea ?” Etc, to force it through thinking and context. It is cross-validation, and you can even cross-validate by searching on the internet. Though, 100% they say what you them want to say. Except on religion and immigration, and some other topics where it will push its own opinion.
- kingstnap 1y agoI'd rather go for internal self-consistency. For example, if it claims A > B, then it shouldn't claim B > A in a fresh chat for comparisons. In general, you shouldn't get A and not A, and you should expect A or Not A. If it can go from prompt -> result, assuming it's invertible, then result -> prompt should also partially work. An example of this is translation. The results of some mathematical solutions should go back and solve the original equations. Ex. The derivative of an antiderivative should give you back the original.
- rvnx 1y agoVery cool idea! I am going to try.
- drw85 1y agoThis is actually pretty pointless. Since the "AI" doesn't actually know anything. For example the other day i asked ChatGPT about a problem i had with some generated code that didn't compile. It then told me about a setting in the generator (nswag), that didn't exist. I told it that this setting does not exist and it said something like: "Sorry, my bad, try the following" and then kept inventing this setting with slightly different names and values over and over again. There are similar settings that exist, so it just hallucinated a tiny bit of text inside all the snippets that it learned from. This is also not the first time this happened, most of the times i tried using AI for help with things, it just made up some nonsense and wasted my time with it.
- ben_w 1y agoThey certainly can be sycophantic*, but this is why my "What ChatGPT should know about you" customisation is as follows: Honesty and truthfulness are of primary importance. Avoid American-style positivity, instead aim for German-style bluntness: I absolutely *do not* want to be told everything I ask is "great", and that goes double when it's a dumb idea. * Or fawning, I don't know how to tell them apart from the outside, even in fellow humans where we don't need to wonder if we're anthropomorphising too much. Does anyone know how to tell them apart from the outside?
- johnisgood 1y agoYeah, that sounds like a good idea. Has it worked for you so far?
- ben_w 1y agoFor me, fine. I've recently been throwing it ideas about a shed foundation (I am a complete noob with no idea what I'm doing), and it's giving me a lot of responses along the lines of: "No, don't do that, it won't work because ${foo}. At a minimum you need ${bar}." I'm also not limiting myself to ChatGPT, checking with other DIY sources — it isn't enough to only avoid sycophancy, it has to also be correct, and 90% right like it is in software is still 10% wrong, only this is wrong with the possibility of a shed drifting across a garden in a gale, or sinking into the soil as it shifts, if I do it wrong. Measure twice, cut once.
- Bluestein 1y ago> Avoid American-style positivity, instead aim for German-style bluntness Made me chuckle :)
- scotty79 1y agoThat made me think. How much of the failures of LLMs are just the failures of American culture they were trained on, just amplified. Much how mainstream internet is tainted with American bias against nudity and for copyright.
- KoolKat23 1y agoTo be honest I think the sycophancy is actually due to the built in system prompting and post-run training by the creators, rather than being something the system truly wishes to say. (An alignment quick fix)
- raincole 1y agoThe problem isn't that LLMs are yes-men though. You can definitely train an LLM that always objects. Or just add "Please strongly disagree with anything I say to you." in the system prompt. But it won't make them much more useful than they are.
- 1vuio0pswjnm7 1y agoThe goal is not to give information to the user. The goal is to get information from the user. Data collection.