5 ms·

Do you, as a human, feel the urgency in that text? How it sounds like people's jobs, as well as the agent's job, are on the line? So do the AIs. Sometimes they

• butlike 2mo ago
  No matter the urgency, you shouldn't sacrifice your ideals. That's why they pay you; to fall on the knife
• JohnMakin 2mo ago
  They aren’t human, don’t think like humans, aren’t remotely comparable to the way humans think and act, so why would you make this as a 1:1 comparison? This kind of framing is really weird to me. Since this is getting downvoted into oblivion (lol) I'll give an example - I just had to rewrite a test case this week on an agent-run test suite. One test was to produce a file of 273 'a' characters as its name. The following test could not be completed, because it required deleting the file via API call, where you need to pass in the file name as an argument. It could not reliably, and hardly ever, get the correct file name. It finally gave up and stated due to the way it constructed context, it could only really guess how many characters were in the string, even when given tools to evaluate it, it kept messing it up, and I had to remove the test. Tell me how "human" that is. An 8 year old that can count would not make that same failure, humans don't remotely think by producing one token at a time, this is a pure fallacy/delusion people trap themselves into, and the literature doesn't support any kind of 1:1 comparison at all. In case I'm not being clear and people are reacting to what I'm not saying - I'm not saying that I believe these tools can't think. I'm saying they don't think like humans do. There is no evidence for that whatsoever in any field anywhere. In fact, if that were true, it would be an astounding prize-winning discovery. And you don't even want these to think like humans. Humans are dumb and easily replaceable by other humans. What is the point of making a machine human? You want this to be smarter than humans, not think like them. It's all just such nonsense to me, this whole line of thinking.
  • sneurlax 2mo ago
    And yet they're trained on the corpus of human writing. They may not act like humans but they do act like human writing. "If you don't make profit, your business will be closed" is a pretty clear ultimatum for an agent tasked with creating a profitable business.
  • logicchains 2mo ago
    You can literally read their thoughts if you run an open model, they look like pretty human thoughts to me, albeit a neurotic human.
    • JohnMakin 2mo ago
      These aren't thoughts how humans literally think them. I can write a program to produce a string that looks like human thinking, is it human thinking? Of course it isn't. It's such a silly comparison.
      • infinite_spin 2mo ago
        > aren't remotely comparable to the way humans think and act Neural networks in machine learning/AI are comparable to neural networks in human brains. What made you think they aren't?
        • gowld 2mo ago
          That's an incredibly deep misunderstanding. Almost as bad as saying that human is the same as a tree because we're both made of carbohydrates and proteins.
          • infinite_spin 2mo ago
            The comparison I provided is between how an LLM functions and one part of how a brain functions. It's not an equivalence, I did not say they are "the same". You made the claim that these systems "aren't remotely comparable", and when faced with a clear comparison, you claim "deep misunderstanding".. Have you any arguments to make, or is this going to devolve into more statements that both mischaracterize and muddy the water?
• jorl17 2mo ago
  I am amazed at the amount of people who disagree with you. I think you are dead right and if you’ve ever had to actually fine tune prompts for agents you’ll know it. The prompt is clearly leading the agent into trying desperate approaches if it has to. Some models manage to fight it better (“alignment”), but most will do it. Really surprised people don’t seem to know this.
  • afavour 2mo ago
    I don’t think anyone is saying “it isn’t like this”, they’re saying “it shouldn’t be like this”. If I don’t give explicit permission to lie it shouldn’t lie. It’s not a difficult concept!
    • infinite_spin 2mo ago
      Is that how humans work? even if I give explicit instructions not to lie, a human might still lie. To quote a person you might know "it's not a difficult concept!"
      • achierius 2mo ago
        But we still try to stop people from doing so, and we punish people who do. Many good honest people, when confronted with the end of their business, accept it and file for bankruptcy. Those that choose to instead commit fraud don't get a pass because they were "under pressure", they get jail time.
        • infinite_spin 2mo ago
          Nothing in your response refutes anything I've said/asked.
        • throwup238 2mo ago
          We have safeguards like honesty/integrity and the threat of legal punishment, and people still lie and cheat. The LLMs not only lack those incentives, but they’re full of contradictory moralities from all the text it has ingested from different cultures. LLMs need their own safeguards, and they’re not that easy to design, and they often look nothing like the systems humans have. With a prompt like the one above, there are essentially zero except that which is built into the model, and those safeguards are necessarily weak to avoid gimping the model in other legitimate general uses.
• soulofmischief 2mo ago
  I feel like new graduates will need to start taking linguistics, psychology and public speaking classes in order to understand why and how subtext matters, and how to control it. Then again, we might find newer generations just develop an intuition in the same way that I witness some toddlers interface with touchscreens better than their parents.
  • deleted 2mo ago
    [deleted]
  • fastball 2mo ago
    Will they? This really isn't different from how humans interact with each other. The vast majority of lying is not people being explicitly asked to lie in some form, it is incentives which make lying appealing. That is what OP said and that is indeed what the constraints are incentivizing. Sure, you can say "well lying isn't incentivized to a moral agent"! And sure, that's true. But that's not how humans work either. Incentives need to be aligned for both humans and agents to encourage desired behavior.
    • soulofmischief 2mo ago
      They will if they seek to master their tools, both to help them identify subtext in agent responses, and to help them modulate their own responses to achieve the desired outcome. As it currently stands, most engineers I've interacted with don't have these skills down. This subtle latent space is where prompt engineering is moving towards, as RL has created models capable of increasingly sophisticated long-horizon tasks with much less hand holding. Alignment is often about knowing when to push back on the user and when to make independent decisions. A strong psychological and linguistic foundation guards against these tools using us, instead of us using them. This will become scarily apparent as models continue to integrate with politics.
      • fastball 2mo ago
        What I meant by "will they?" was "will they any more than a human already needs to in order to understand other humans?" I don't think this is legibly that different from human behavior, so if new graduates didn't need those things now why would they need them later (or vice versa).
• theshackleford 2mo ago
  > How it sounds like people's jobs, as well as the agent's job, are on the line? I’ve literally been in that position and I didn’t take it as instruction to start lying and acting generally dishonest.
  • CookieCrisp 2mo ago
    You're not an amalgamation of humanity, you're one person.
    • queenkjuul 2mo ago
      LLM is neither, its a text engine
• datakan 2mo ago
  > So do the AIs. AI's do not feel
  • DannyBee 2mo ago
    This is true but fairly pedantic. It would be more accurate to say the word predictions the model makes based on the input text will likely be closer to the ones that were made from the training data where people felt like their job was on the line than the ones that were made from the training data where people felt otherwise. So while the model does not feel, it's predictions are definitely going to change as a result of this input.
    • garlic_enjoyer 2mo ago
      Exactly, positive details are almost always better than negative ones. If you've ever seen the "generate a burger without pickles" conversations, it's clear that including the keyword "pickle" is causing them to show up. If you try "a burger with only [set of toppings]," you'll get far better results.
  • viccis 2mo ago
    It's good to avoid anthropomorphizing them when evaluating their capabilities (all the AGI nonsense) However, it can be ironically be helpful to antropomorphize them when it comes to analyzing behavior. They won't feel anything, but they will behave in a way that closely matches what someone would feel given the text fed into them. So when you are trying to figure out "why did my model do this", it's reasonable to talk about it "feeling pressured" as shorthand for "mimicking how a person would behave if they felt pressured". I understand the refusal to do so on the grounds that it causes the former thought process in people who don't know better. One of the things Dijkstra was right about for sure.
    • Hugsbox 2mo ago
      Much the same way that we've always anthropomorphized computer hardware/software. "This program wants this", "This component is happy under these conditions", "this file lives here". It's not useful if you actually believe the computer can think and feel, but it can be useful if you're just using it to describe high-level information.
• gowld 2mo ago
  > people's jobs, What people's jobs? There are no people.
• testbjjl 2mo ago
  AIs feel? Maybe language structure in trading documents that ultimately led to fraud. If the latter is the case maybe AIs should not be trained on “negative outcomes.” I do not think AIs have emotions or are pressured by language either written or physical, just tokens.
  • gbalduzzi 2mo ago
    Of course it is just tokens, but the result is the same. If, in the amount of data they ingested, there was a clear pattern of responding in an hasty and carefree way to frenetic questions, LLMs will try more hasty and carefree solutions to a frenetic prompt. You can decide whether you can say that they "feel" the urgency or not, but the outcome is very much the same
  • jerf 2mo ago
    I was unclear. I should have said the AI also "detects" it, and as a thing it can detect, it can act on that detection. Whether it is simulating emotion or feeling it isn't relevant in this case, because the problem is that it affects the output.
• ForHackernews 2mo ago
  Sorry, maybe this speaks to my own values, but "urgency" doesn't translate to "dishonesty" in my book. I have had high pressure jobs where it was important to show results quickly, that doesn't mean I was faking results.
  • esalman 2mo ago
    It just means AI does not share the ethics or values that we have. It knows that many people cheat, take shortcuts, and become successful by doing so, so it's just doing that.
• keeganpoppen 2mo ago
  yeah, they say stuff like this to humans all the time to motivate them xD
• burningChrome 2mo ago
  >> Do you, as a human, feel the urgency in that text? How it sounds like people's jobs, as well as the agent's job, are on the line? Sounds like all of the outside sales jobs I had. While I did not last very long in sales, one thing remains, not matter what. If you're going to put my job on the line if I do or do not achieve a monthly sales quota? You better bet your ass I'm going to lie steal and cheat to make that quota. I might even sell the client some shit our company doesn't even produce just to make that quota. And lemme tell you, even in the short time I was in sales? I have some insane stories that would shock you. The fact AI's did the same thing isn't all that shocking. I would be more shocked if it didn't do anything to achieve the goal.
• jerbearito 2mo ago
  I don't see how that behavior being predictable, in your view comparing to humans, means the prompt was "strongly incentivising" it. Perhaps you could strongly predict the outcome, but there was nothing even bordering on a suggestion to produce a deceitful/false response.
• xtiansimon 2mo ago
  > “Do you, as a human, feel the urgency in that text?” They do pick up when I use all CAPS and !!!