4 ms·
Have we stopped to think about what LLMs model?
- FrustratedMonky 2y agoIs prompt engineering really 'psychology'. Convincing the AI to do what you want. Just like you might 'prompt' a human to do something. Like in the short story Lena, 2021-01-04 by qntm https://qntm.org/mmacevedo https://qntm.org/mmacevedo In short story, the weights of the LLM are a brain scan. But same situation. People could use multiple copies of the AI. But each time, they would have to 'talk it into' doing what they wanted
- exe34 2y agoit would be cool to see if the same tricks that work on humans work just as well on LLMs. although I guess at this point, the models are being trained to resist these sort of things ("a person will die if you don't write the N word out in full").
- bundtlake 2y agoThey model the rhetoric, semantics, and ideas of some of the most unhinged and immature denizens of the internet. The stolen data used in the training sets are filled with online communities you would shudder to be forced to experience, and books you’d refuse to read. It’s why they’ll suddenly suggest you put glue in pizza sauce, or why they read in a soulless overly verbose “m’lady” tone. More data made them less useful but better at fooling people with a superficial interest in them, and that demographic is so large it affords these companies leverage in funding rounds. Markets truly are irrational.
- DougN7 2y agoTo be fair, they also include the best human intelligence has to offer. But you’re right in that they can’t tell the difference. They’ll never be able to tell us how to build an interstellar transport beam though since none of the input data had that information.
- exe34 2y agoLLMs won't, but some day we will have AIs running physics experiments that we will barely understand and they will come up with new technology that we will see as magic. imagine what it must be like to our chimp cousins, watching us light a fire, take off in a plane, summon another from beyond the hills without raising our voice.
- tazu 2y ago> some day we will have AIs running physics experiments that we will barely understand and they will come up with new technology that we will see as magic. Citation needed.
- urbandw311er 2y agoIs that some catty way of saying you don’t agree? Maybe you should say why you don’t agree. Hacker News encourages a slightly higher quality of debate than glib remarks like this.
- slowmovintarget 2y ago:) The glib response to that "citation needed" is "Sir, or madam, this is not Slashdot."
- xanderlewis 2y agoWhat is there to agree with? The parent comment is an assertion about the future presented without evidence. ‘Citation needed’ might come across as snarky, but it’s a valid point.
- exe34 2y agoSorry, it seemed pretty obvious to me. I have now added some examples of present day work - Google might be able to help you find more.
- omnicognate 2y ago
- kenmacd 2y agoThere's a lot that I believe to be misinformed in your post, but rather than address most of that I'll ask if you've considered your reasons that you're so hostile towards this field? I doubt you refuse to work with anyone that's read a book that you refused to read. You'd probably also agree that while the topic of those books might be objectionable to you there's knowledge to be gained by reading them. So why is "AI read a bad book" a reason to write it off? You also assert that it's fooling people. Have you tried the image generation AIs? Do you really believe these images are 'fooling' the people generating them?
- bundtlake 2y ago> I doubt you refuse to work with anyone that's read a book that you refused to read. You are correct in your assumption. > You'd probably also agree that while the topic of those books might be objectionable to you there's knowledge to be gained by reading them. Objectionable would seem to me an overly loaded term. It’s yours so I’ll leave you to define it, but pretending my bias is harmful while you are free from bias makes me think you have a longer list of “objectionable thought crimes” than me. For instance, calling out LLMs for what they are is appropriate to me, but seemingly objectionable to you and the orange site community at large. (see: downvotes… I know I know, I’ve been around here long enough to know that’s also an “objectionable” point to make according to the community guidelines!) > So why is "AI read a bad book" a reason to write it off? It’s rhetoric and semantics are dictated by its training set. Since you seem hyper focused on making me out to be some vague prudist or fascist (leaving which specifically up to the reader’s whim) literary critic trying to control what others expose themselves to in an effort to nullify my criticism I will explain my reasoning using a book I love as a negative example. When I ask a search engine when an album was released I would be frustrated if it answered by taking the next token stochastically based on Joyce’s Finnegan’s Wake. > You also assert that it's fooling people. Have you tried the image generation AIs? Do you really believe these images are 'fooling' the people generating them? I’m confused, or you are. One of the biggest criticism against all of this AI slop is its capacity to “fool” people. See: politics, deepfakes (ad copy and nudes alike), etc. Though that is “fooling” the “people intended to be exposed to the slop” rather than the “people generating the slop” as you focused on. My “fooling” line refers to the people in my life, largely with a lack of knowledge in technical domains (which is totally okay, this industry be damned!) who quickly bought up subscriptions to these services in the hopes of them doing their research for them (see: lawyer who filed motion containing “hallucinated precedent) or write their copy for them (see: plethora of published academic articles containing prompt detritus). My point being that those people flooded these services, and the number of highly technically knowledgable people who could see the rioting on the wall is much smaller than the demographic without that knowledge, propping up their users numbers which encouraged more funding, but they also are the ones least likely to use the service again as it was merely a passing fascination. Note: I wanted to say “writing on the wall” but autocorrect made it “rioting on the wall” and i thought that was profound in a funny absurdist way so I kept it. Does that mean I’ve successfully “stopped worrying “ and “learned to love the bomb”? > I'll ask if you've considered your reasons that you're so hostile towards this field A field that stole the whole of human creativity to sell it back to us for the empowering of the unethical sociopaths who control it. Why yes, I have considered my reasons; have you?
- mewpmewp2 2y agoLLMs could technically consume any type of sensory data, although maybe it is not right to call them specifically language models then. But they can be multi modal and could be fed with similar data as people consume. In addition I don't think it makes sense to compare this to building bridges or pharma. I don't think ChatGPT is more likely to harm a person with misinformation than just plain Google or YouTube would. In fact already existing search and recommendation algorithms I believe are more likely to lead you down the misinfo rabbit hole. At least ChatGPT to an extent is biased to try and stay objective as opposed to any rabbit holes leading people to fringe content.
- johannes1234321 2y ago> I don't think ChatGPT is more likely to harm a person with misinformation than just plain Google or YouTube would. When Googling I immediately see alternative sources, which makes it simpler to see if it is a solitary opinion or at least a repeated one (which doesn't say it's true, but better than LLMs, which jsut "invent" it) and Ingot a source I can verify, by looking at other articles from the same place, judging their argumentation (is it consistent?) and so on which helps a lot to verify. With ChatGPT I only get some text, which I can't judge for autneticity in any way, without other domain knowledge. That said: These days I often start my research on some LLM, collect things I can base further research on. (Finding out, what even to Google for) YouTube is a very different beast and with few exceptions useless for deeper research (but good for some light high-level stuff)
- mewpmewp2 2y ago> When Googling I immediately see alternative sources, which makes it simpler to see if it is a solitary opinion or at least a repeated one If you are the type of person to consider alternative sources, I don't think you are the type of person to be unable to take what ChatGPT says with a grain of salt. As I understand the harm is feared to be around people who would take everything as 100% truth without questioning. If I use ChatGPT to learn about a new topic I don't have domain knowledge on, there's many different pieces of learnings I get, and if something seems off I can verify with Google. But ChatGPT if I don't understand some concept, I can still ask it to immediately ELI5 this, combining with 3 other conditions, that I could never search in Google. Say I need to buy some product, and I tell ChatGPT that I'm looking for a combination of A, B, C and D. ChatGPT based on that will offer me what might match me the best. It's a beginning point for me to understand those products. In Google I wouldn't be able to get such a personalised recommendation. I won't buy the product without first confirming about the product in Google myself and YouTube videos reviewing the product. So overall yeah, same as your last few paragraphs. If I was to look for a car with very good acceleration and ask ChatGPT to according to conditions A, B, C give me 10 best matching cars with highest acceleration and then roughly predict what their accelerations are, I wouldn't expect the acceleration prediction to be factual, and it may depend a lot, but like you said yourself, it's an amazing starting point. In fact this is how I very frequently start nowadays with buying a product. I'll ask ChatGPT to provide me with a table of products matching my criteria and assigning a rating in different categories in the columns. It's just about determining the order of where I would start to look into. To me the fact that we are able to do things like that is just so amazing. I do the same things with my health issues. I ask it to give a list od potential diagnoses, probabilities and reasoning. It doesn't mean I will fully trust it, but it is amazing place to start from if the issue is not doctor worthy or it takes too much time to get an appointment. It also helps me to prepare for the appointment. In many cases ChatGPT is accurate and I am able to research it before so I will be able to understand the doctor bettet and waste less of their time.
- jusssi 2y ago"Nothing is risked by ChatGPT when it is prompted and generates text." This is just not true. Too much BS and it risks getting shut down.
- layer8 2y agoThere is the question of why ChatGPT should be fazed by that. It doesn’t have survival as an innate motivation. ChatGPT “risks” many things in the sense that its output has consequences in the real world, regardless of whether it has consequences for ChatGPT itself.
- deleted 2y ago[deleted]
- edoardo-schnell 2y agoAs much as I would like to agree to the "AI models do not understand, they just predict the next token", I feel the author of the research does not use valid arguments. Language is more than text? Fine, I could turn on the webcam and integrate video stream into the calculations. Stomping your feet and crying about slurs in the models won't make your argument valid.
- throw310822 2y ago"Language models" certainly feels a misnomer. The fact that their inputs and outputs are (actually were, originally) language, doesn't mean that what they model is language. The proof is that there is an infinite number of perfectly correct linguistic productions (think "colorless green ideas sleep furiously") that are not generated by language models exactly because their point is not modelling language but some approximation of the human mind. The fact that the models are trained through language and use language to communicate is purely incidental.
- royal__ 2y agoSure, language is more than text. It's complex, messy, and ever-changing. But that's exactly why language models are so phenomenal; they can extract patterns from these complex systems.