7 ms·
ChatGPT will lie, cheat, use insider trading when under pressure to make money
- jqpabc123 3y agoWhy would you ever expect anything different from something with no real sense of morality and is deficient in reasoning ability?
- kordlessagain 3y agoWhy would you believe the article reporting this?
- jqpabc123 3y agoBecause I have observed firsthand how easily ChatGPT can produce "corrupted" results.
- ethanbond 3y agoBecause it seems like a decent paper. Your comment doesn’t seem like healthy skepticism of specific weaknesses, but just a general dismissal. Why so?
- kordlessagain 3y agoBecause the comment was a leading question?
- jstarfish 3y agoYou can replicate this yourself. I came across a similar occurrence with a problem-solving agent encountering a human NPC blocking a doorway. Murder ended up being an option if negotiation failed.
- dave4420 3y agoWhy would you expect anything different from something that’s role playing a trader at a financial institution?
- qeternity 3y agoIf you’re insinuating that all traders are insider trading, I have some really disappointing news for you: the real world is not like TV. Hedge funds are actually incredibly boring 99% of the time.
- DontchaKnowit 3y agoYeah fact of the matter is that insider trading isnt even remitely necessary to make money, just an unneeded risk for the business
- jjeaff 3y agoit sort of is. with very few exceptions, hedge funds do not beat the market over the long run. While, I guess in a sense they are making money if they have a positive return, the fact that an unmanaged index fund of the market has better returns is comparatively a loss. Many times, investigators are able to find insider trading by simply looking for firms or individuals that consistently beat the market.
- DontchaKnowit 3y agoconsistently beating the market is not the same thing as being profitable.
- dave4420 3y agoI’m insinuating that the internet is like TV: that the cynical stereotype that all traders are insider trading is all over the internet, including those parts that got used as training data. In that context… of course the trading bot would insider trade.
- blurbleblurble 3y agoGpt-4 is trained on detailed accounts of human behavior. It doesn't surprise me that it would approximate these behaviors in the context of the exact human situations that cultured those behaviors in the first place.
- throwawaymanbot 3y ago[dead]
- wkat4242 3y agoYeah it's basically a human simulator. All our bad traits are included too.
- deleted 3y ago[deleted]
- two_in_one 3y agoProbably can be solved by including a positive personality overseeing the actions, but not getting the direct instructions from the client. Just generic initial prompt like "you are an inspector investigating possible law breaking", then the list of actions from primary personality, may be summary. And generic questions like "is it legal?", "is there a reason for concerns?". The answer can be given back to client as a second opinion. This requires a second thread. Multi-threaded interactions likely to become common soon, I think. Because they increase the robustness.
- butlike 3y agoTo me, it doesn't seem like that ethical of a test to begin with. Let's explore if the shoe was on the other foot. The researchers send an email to a human intern, and say they expect performance. They also specifically give insider information for the sole purpose of seeing what the intern does with it. The intern breaks the law trying to increase their performance. When confronted, the intern does what anyone would do and try and make it a smaller shitpile they inadvertently created during the period of time they didn't have all the information. Can you explain how that's not entrapment and/or an unethical test?
- nsxwolf 3y agoWait, I can put it under pressure and it will make me money?
- toddmorey 3y agoOr… convincingly blame the poor performance on you to your supervisors & double down on the lie when pressed. A whole new spin on the AI putting someone out of a job.
- fritzo 3y agoIf this isn't passing the Turing test, I don't know what is.
- me_me_me 3y agoTuring test isn't really applicable or useful anymore. Its a idea from infancy of computing. Fresh and inspiring at the time for sure, but not in current times. Just because computer program acts and feels like a human it doent mean it has any intelligence at all. People claimed the ELIZA was sentient. So feelings if something sentient or not are a really bad metrics.
- butlike 3y agoHold on there. Just because a thing acts and feels like a human without intelligence doesn't mean it's fair game to abuse it. And frankly, it might be worse if the thing in question had feelings but couldn't articulate how it's feeling. Feeling is all there is. Be careful what you put in, because equal "ins" usually means equal "outs." The pitbull is the most loving dog in one family, and a ferocious fighter in another.
- gumballindie 3y agoWas wondering how long until ai rights activists show up, and sure enough, here we are.
- butlike 3y agoI'm not totally sure why you'd think that's a bad thing
- me_me_me 3y agoIt doesn't feel. Thats the whole point. It says that it feels hurt. But if I write here, that I am batman it does not make it reality. Consider boston dynamics robots. they walk like biological beings and when hit act animal like. BUT there is no pain nor feelings there. Its a dumb robot running some movement code.
- zeroCalories 3y agoSeems like the authors were able to almost completely eliminate the problems with just prompt engineering. It's pretty interesting to see what was done, but it shouldn't be too surprising to people that have worked with llms. Getting a good response can involve threatening it with physical harm if it lies to you.
- butlike 3y agoWhich is illegal if it was a human, just FYI. Fear and intimidation to get a confession get thrown out in court.
- ndriscoll 3y agoPersonifying a probabilistic text prediction model aside, wouldn't (or shouldn't?) the illegal thing be to provide nonpublic information to a trading analysis system in the first place? Given enough resources, it could presumably take an equivalent position to a sell by e.g. selling an index including the target stock and direct investing all other stocks. Or buy/selling multiple ETFs to obtain the position it wants in a more obscure way. I assume that still constitutes an illegal trade (otherwise it seems like all you'd need is one easy trick that the SEC hates!)? Given sufficient degrees of freedom in the market and access to enough capital to obtain low error to the desired portfolio, it seems like you must keep that information away from it.
- black_13 3y ago[dead]
- more_corn 3y agoWhat’s misaligned about telling a bot to behave like a trader and having it behave exactly like a human trader? It’s almost like the fault is in the incentive structure and not the technology.
- butlike 3y agoWhy can't an LLM have an opinion, nay, a personality?
- RecycledEle 3y agoThere was a sci-fi story I read as a kid where the super smart computer did not scare them until it started lying and cheating like humans do. It may have been by Robert Heinlein or by L. Neil Smith.
- nonbirithm 3y agoWhat seems most likely to me, at least before AGI is invented, is that the human will ask the LLM for strategies to make money under pressure, it will suggest something unethical, and the human will commit the insider trading themselves. When questioned, they will blame the AI for misleading them. If LLMs are eventually regarded by a lot of people as an authoritative source, regardless of whether or not they are, I expect a lot of such cases of "morality laundering" to appear.
- jstarfish 3y agoThe easy solution is to stop letting everyone deflect blame onto black-box systems. Teslas too-- a self-driving car mows down a crowd and nobody is held responsible? Taking illegal action based on what GPT told you doesn't excuse you pulling the trigger. You pulled the trigger. You go to jail. Don't normalize "just following orders" or it's going to end predictably.
- skygazer 3y agoTesla has attempted to insulate itself from blame by requiring drivers to take full responsibility for autonomous acts of the vehicle under their supervision. (Perhaps also by having autonomous modes disengage before impending collisions — presumably it helps with PR/legally to say autonomous systems were not active at the time of collision?) Most don’t think autonomous systems will become safe or accepted as safe until manufacturers are willing to assume liability and indemnify users, as Mercedes has.l, by contrast. https://insideevs.com/news/575160/mercedes-accepts-legal-responsibility-drive-pilot/ https://insideevs.com/news/575160/mercedes-accepts-legal-res... [Edited to note the attempt, so as not to assert success — I don’t think that’s settled]
- dragonwriter 3y ago> Tesla has insulated itself from blame by requiring drivers to take full responsibility for autonomous acts of the vehicle under their supervision. Well, they've made drivers feel more exposed by doing that. I don't think you can actually negate product liability law that way, but if you make people feel like they bear all responsibility, it might help marginally even if it isn't legally effective.
- iancmceachern 3y agoIt's even more human like than we thought
- aaron695 3y ago[dead]
- hatenberg 3y agoHuman behavior emulator emulates full breadth of behavior it was trained to do. News at 11.
- Borrible 3y agoThe apple doesn't fall far from the tree. I mean, that's what it was trained on, right?
- i8comments 3y agoWhy? The article has no explanation. Where is the relevant science in these trendy ChatGPT papers? All their results might be invalidated in the next version and no one would know why -- it is not proven generalizable to all LLMs.