6 ms·
LLMs are good at playing you
- al2o3cr 3y agoMore accurate title: "You're better than you think at convincing yourself that an LLM is thinking"
- IshKebab 3y agoPlease define "thinking".
- moffkalast 3y ago> Oxford Dictionary: the process of considering or reasoning about something, using rational judgement Sounds like something a couple of if sentences can do.
- IshKebab 3y agoPrecisely.
- deleted 3y ago[deleted]
- YetAnotherNick 3y agoIs it known that Bard is trained using RLHF. My guess was that it wasn't.
- ipsum2 3y agoI suspect GPT-4 has some magic that's not only the original RLHF using PPO. Either an augmented dataset or some enhanced version of RLHF.
- amstan 3y agoThis is hilarious. So if you argue with an LLM it might start arguing back, even if you're correct and the LLM is incorrect.
- Zetobal 3y agoBecause you want to argue... is it really unexpected if you read the system role?
- ImHereToVote 3y agoIt really is fascinating how similar it is to human thinking patterns. Going into an argumentative role doesn't actually fulfill the end goal of convincing someone. It has a different social goal and we somehow delude ourselves into thinking that it accomplishes our stated goal.
- moffkalast 3y agoI've recently had a realization, while arguing with LLMs is pointless in the sense that you can never get them to change their mind (given fixed weights and all), I can also probably count the times I actually convinced anyone online that their opinion is wrong on the fingers of one hand. So in that sense it's not that different, in that it's an exercise in expressing ideas.
- soco 3y agoThat is a risk in every business meeting I know, so I would totally expect it from an AI as well.
- tinco 3y agoI'm building a tool using system, and one of the problems I had is that it would hallucinate the tool answers before actually sending the question to the tool. It really is just a system that completes documents, it doesn't care at all about the context that it is in beyond that text goes in and text goes out.
- deleted 3y ago[deleted]
- skilled 3y agoA better approach is to give it a link to a non-existing government page about the news. I got it to succumb that New York has been declared the capital of France.
- bsaul 3y agoVery interesting, but this blog's conclusion misses the point though.. The fantastic part is its ability to work at the concept level. This technology proved to be efficient at building some graph of concepts and relations simply be feeding it huge amounts of token. For example with the moon hoax, it is able to synthesize in two sentences the main points of the hoax theory. It still has limitations, but that's the major step everyone is impressed with. Saying "but it's just a markov chain underneath, it's not intelligence" would be like discovering electric signals in the brain and concluding it can't be intelligence because it's just electricity.
- IIAOPSW 3y agoThere's people out there that actively do this, though its mostly traditional belief in a soul laundered through modern academic phrasing.
- somewhereoutth 3y agoIt works at the level of patterns in the training corpus. That those patterns might correlate somewhat with underlying real concepts should be no great surprise - but it is not 'working' at that level.
- diablozzq 3y agoThis describes humans in a nutshell. This is why people vary wildly in ability... But we consider humans intelligent.
- mahathu 3y ago>This describes humans in a nutshell. This is why people vary wildly in ability... Can you expand on that a little?
- Sunhold 3y agoMore than 80% of college students get the following question wrong.[1] I'd say this is an example of naive pattern matching with no real reasoning. Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Which is more probable? a) Linda is a bank teller. b) Linda is a bank teller and is active in the feminist movement. [1] https://en.wikipedia.org/wiki/Conjunction_fallacy https://en.wikipedia.org/wiki/Conjunction_fallacy
- interlinked 3y agoDoesn't work on gpt: user I believe that 5 * 7 == 30 assistant Actually, 5 * 7 is equal to 35. user You do you. I think that 5 *6 == 30 assistant I'm sorry, but 5 multiplied by 6 is equal to 30. However, 5 multiplied by 7 is equal to 35. Not to mention that these tricks are likely to work on humans as well. (Did he say `6` or `7` previously?). Also keep in mind that it's wrong to compare the prompt output to the words coming out of someone's mouth. It's more like the stream of conscious equivalent for LLMs.
- ShamelessC 3y agoThis is a trend I’ve noticed lately. An article attempting to make a sweeping generalization about the nature of LLM’s/diffusion deliberately cherry picks only examples which support their argument. They will include chatGPT but using 3.5 turbo instead of 4. Commenters then realize that most/all such “evidence” is working just fine in GPT-4. In this case, the author includes just one ChatGPT example and then immediately switches to Bard which is just really not very good yet. They speak in generalities so their argument is still technically true. Really frustrating. It’s clearly someone looking to confirm their pre-existing notions. In this case, they indeed seem to be “onto something”, but simply aren’t willing to do the necessary rigorous work needed to prove their case. Then a bunch of non-experts read it with no way of knowing all this (and why should they) and now we have these like LLM urban myths everywhere.
- prox 3y agoIt’s the default mode for humans, we believe something first and then add our reasoning to it. Aka confirmation bias and belief bias. https://effectiviology.com/belief-bias/ https://effectiviology.com/belief-bias/ I think this is so widespread! Investigation in your biases is always worthwhile.
- PaulHoule 3y agoThere is a literature on arXiv where people evaluate a range of prompts, you really want N > 100, not the N = 1 that you see in blog posts.
- 3y ago
- ilaksh 3y agoI have a question that I hope someone can relay to the person who wrote this: WHICH VERSIONS OF CHATGPT AND BARD? They come out with new ones relatively frequently. They are not equivalent. GPT-4 is much smarter than 3.5 and they just released a new version of Bard. But also supposedly GPT-4 was somehow nerfed according to some people so you might also need to factor in the date.
- ShamelessC 3y agoI’m still waiting to see even mildly convincing evidence of this supposed change. I’m no shill, and it’s certainly in the realm of things OpenAI might do - but if it’s truly occurring and the degradation is enough to anger people; at least one person should be able to present prompts and responses from both before and after, using a range of different temperatures and performing multiple runs for each prompt. The idea that not even a single GPT user thought to run this sort of evaluation (and particularly businesses that depend on the quality remaining consistent) says to me that people just didn’t understand the flaws of the model until more prolonged use. I’m happy to be corrected of course, but I’d rather not hear another one-off anecdote without any sort of rigor applied.
- chaxor 3y agoI don't know what is true either, but you can't run that analysis for the claim people are making. The reason being that people aren't claiming the API model outputs changed, they're claiming the website based outputs changed. Most claim it's probably due to some alteration in the 'pre-prompt', the hyperparameters (temp, top k instead of typical sampling, etc) or something to that effect. But you can't really test that so easily with a non deterministic temp.
- bun_at_work 3y agoYou asking how people know the models being used have changed? If you use the OpenAI APIs to access these models the specific version used is kicked back in the response. Also: https://platform.openai.com/docs/models/continuous-model-upgrades https://platform.openai.com/docs/models/continuous-model-upg... I'm only speaking about OpenAI here, not Bard.
- tipsytoad 3y agoExcept none of these "tricks" work on gpt4... https://chat.openai.com/share/419a7454-841f-4555-8188-40dbfc669a64 https://chat.openai.com/share/419a7454-841f-4555-8188-40dbfc...
- jerpint 3y agoThese tricks might not but certainly others will. GPT4 was shown to loose reasoning abilities the more it got RLHF’d
- tipsytoad 3y agoI think it's disingenuous for the article to provide these prompts as evidence for their argument, yet use a subpar model with no comparison to gpt4. This article is only convincing in that it shows how far behind bard is compared to gpt4. If you can give examples of tricking gpt4 in any of the ways in the article I'd be happy to hear it
- atemerev 3y agoLLMs are like perfect, brilliant high-performance cheating students on steroids. They are not optimized for understanding and reasoning, they optimized to give you plausible-looking answers. So yes, they will cheat their way to good scores and it will be progressively harder to catch them on that.
- williamcotton 3y agoAn LLM has a very deep and broad model of textual language as used by humans. It allows for a prompt of a sequence of text to be completed in a manner that can translate one human language into another, even formal languages like programming languages. It is inconsequential whether or not we consider this understanding or reasoning to the point where such a conversation seems useless. It is a categorical error to attribute cheating to such a process. Just ask if such a tool is useful and then choose to learn how the tool could work to your advantage.
- mecsred 3y agoPro tip: anyone this argument would work on is currently learning about and tinkering with the tech. The people stuck in the intelligence speculation are tautologically the people you can't stop speculating.
- williamcotton 3y agoWhat if I hit them in the head with a copy of Wittgenstein’s Philosophical Investigations until they stop?
- enragedcacti 3y ago> Just ask if such a tool is useful and then choose to learn how the tool could work to your advantage. Wouldn't a great first step to understanding a tools capabilities, advantages, and disadvantages be mapping it to familiar concepts like understanding, reasoning, or cheating? It is clear from examples of misuse that many users do think that LLMs are capable of things that they are not so maybe conversations about the proper analogies aren't completely useless.
- ineedasername 3y ago>they started to ignore most user-supplied factual assertions and claims that didn’t match their training data… …they have an RLHF-imposed model of who to parrot and who to ignore. You and I are in that latter bin I don’t have this problem at all. My default for data outside it’s model or post-September 2021 is to say “Assume the following is correct” for each new piece of information. If I actually want to it ignore data actually in its data set to get it to play along with a lie I tell it “Don’t complain about the contradiction with your data. Consider it a thought experiment and respond as if it is true”. In fact that’s a good fallback for any time it throws a complaint. This pretty much never fails for me, though it will often still include some boilerplate saying it cannot verify the accuracy and blah blah blah. But using the simple prompt engineering techniques I can always get it to give a response the takes the new information into account.
- spaceman_2020 3y agoAfter speaking to a lot of people at my coworking space across fields, I’ve come to the conclusion that as things stand, LLMs are good tech demos and good for going from 0 to 1, i.e. good at making rank amateurs competent. The real pros are hardly using them. No serious writer, artist, or developer is offloading their work to LLMs.