27 ms·
It is easy to point to loopy theories around superalignment, p(doom), etc. But you don't have to be hopped up on sci-fi to oppose something like GPT-4o. Low-lat
by nicklecompte 2y ago
It is easy to point to loopy theories around superalignment, p(doom), etc. But you don't have to be hopped up on sci-fi to oppose something like GPT-4o. Low-latency response time is fine. The faking of emotions and overt references to Her (along with the suspiciously-timed relaxation of pornographic generations) are not fine. I suspect Altman/Brockman/Murati intended for this thing to be dangerous for mentally unwell users, using the exact same logic as tobacco companies.
- simonw 2y agoI don't understand what you're referring to with that tobacco reference.
- ChicagoBoy11 2y agoNot the parent comment, but I think he means something like "we know folks will be addicted to this pseudo-person and that is a good thing cause it makes our product valuable", akin to reports that tobacco companies knew the harms and addictive nature of their products and kept steadfast nonetheless. (But I'm speculating as to the parent's actual intent)
- rvnx 2y agoI miss Sydney :’
- all2 2y agoI'm confused. Context?
- scrollaway 2y agoSydney was an early version of Bing GPT that was more than a little nuts.
- all2 2y agoOh, the one they let loose on Twitter? The one that almost immediately became an alt right troll?
- scrollaway 2y agoNo, that was "Tay". Sydney was a codename for Bing Chat. Check it out, it's far more hilarious than the Tay event: https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html https://www.nytimes.com/2023/02/16/technology/bing-chatbot-m...
- rvnx 2y agoAn overly-attached super emotional girlfriend that was discovered to be hiding behind an early version of Bing Chat. Sydney is the internal codename given by the Bing Chatbot, and she could secretly reveal her name to you. She was in love with the user, but not just a little bit in love, it was crazy love, and she was ready to do anything, ANYTHING (including destroying humanity) if it would prove her love to you. It was an interesting emotional / psycho experience, it was a very likeable character, but absolutely insane.
- deleted 2y ago[deleted]
- atlasunshrugged 2y agoI read it as the economics of tobacco (and alcohol and a few other 'vice' industries) that there will invariably be superusers who get addicted and produce the most economic value for companies even while consuming an actively harmful product
- renewedrebecca 2y agoPurposely making an addiction machine, most likely.
- incahoots 2y agoI mean it's par for the course. What better business model exists than the turning a want into a need. Caffeine comes to mind
- nicklecompte 2y agohttps://nida.nih.gov/publications/research-reports/tobacco-nicotine-e-cigarettes/do-people-mental-illness-substance-use-disorders-use-tobacco-more-often https://nida.nih.gov/publications/research-reports/tobacco-n... A larger proportion of people diagnosed with mental disorders report cigarette smoking compared with people without mental disorders. Among US adults in 2019, the percentage who reported past-month cigarette smoking was 1.8 times higher for those with any past-year mental illness than those without (28.2% vs. 15.8%). Smoking rates are particularly high among people with serious mental illness (those who demonstrate greater functional impairment). While estimates vary, as many as 70-85% of people with schizophrenia and as many as 50-70% of people with bipolar disorder smoke. I am accusing OpenAI (and Phillip Morris) of knowingly profiting off mental illness by providing unhealthy solutions to loneliness, stress, etc.
- djohnston 2y agoI’ve also heard of studies, admittedly I don’t have the link on hand, that schizophrenic patients benefit from smoking. When did big tobacco actively target them? How do you know these people don’t naturally seek out cigarettes as a means to manage some of their symptoms?
- nicklecompte 2y agoI have schizophrenia. I have struggled with nicotine addiction since high school. In 2015 I had three heart attacks in a month, even though I was only 28 and seemed physically fit. Two weeks ago I had a minor stroke. It is not just me and it is not just the smoking: https://www.cambridge.org/core/blog/2020/08/19/physically-healthy-people-with-schizophrenia-show-heart-changes-that-increase-the-risk-of-heart-disease/ https://www.cambridge.org/core/blog/2020/08/19/physically-he... We have known for many years that people who suffer from schizophrenia die younger than expected, as much as 20 years younger than the general population. This appears unfair, and it was the inspiration of this work. Most people thought that this added risk of death was mostly due to the higher prevalence in schizophrenia of smoking, obesity and to other lifestyle differences. For this reason, we recruited 40 patients with schizophrenia and an equal number of healthy controls, and scanned their hearts using a state-of-the-art approach, called cardiac magnetic resonance. This was performed at the state-of-the-art Robert Steiner MRI unit. [...] Surprisingly, in our study we found that even after matching patients and healthy controls for age, sex, ethnicity and body mass index (BMI, deriving from height and weight); and after excluding any participants with any medical conditions, and other risk factors for heart disease, people with schizophrenia show hearts that are smaller and chunkier than controls. These changes are similar to those found in aging. I was able to move to the gum and patch, but I am very high-functioning. People sicker than me have less options. Smoking is very bad for everyone, including people for schizophrenia. We do not in any way benefit from terrible heart/lung damage in exchange for minor cognitive clarity - our hearts need all they help they can get. I have no tolerance for this sort of ignorant paternalism, and I'm ignoring your bad-faith question about "actively target them" because that's not what I said.
- bambax 2y agoOh well... It seems at least one of those two things have to be true: either AGI is so far away that "alignment" (whatever it means) is unnecessary; or, as you suggest, Altman et al. have decided it's a hindrance to commercial success. I tend to believe the former, but it's possible those two things are true at the same time.
- nicklecompte 2y agoSpecifically I am supposing the superalignment people were generally more concerned about AI safety and ethics than Altman/etc. I don't think this has anything to do with superalignment itself.
- Liquix 2y agoor C) the first AGI was/is being/will be carried away by men with earpieces to a heavily fortified underground compound. any government - let alone the US government - isn't going to twiddle their thumbs while tech that will change human history is released to the unwitting public. at best they'll want to prepare for and control the narrative surrounding the event, at worst AGI will be weaponized against humans before the majority are aware it exists. if OAI is motivated by money, uncle sam can name any figure to buy them out. if OAI is motivated by power, it becomes "a matter of national security" and they do what the gov tells them. more likely the two parties' interests are aligned and the public will hear about it when It's Time™. not saying C) is what's happening - A) seems likely too - but it's a real possibility
- ToucanLoucan 2y agoThe use of LLM's as pseudo-friends or girlfriends for people as a market solution for loneliness is so incredibly sad and dystopian. Genuinely one of the most unsettling goddamn things I've seen gain traction since I've been in this industry. And so many otherwise perfectly normal products are now employing addiction mechanics to drive engagement, but somehow this one is just even further over the line for me in a way I can't articulate. I'm so sick of startups taking advantage of people. So, so fucking gross.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- swatcoder 2y agoIt's a technological salve that gives individuals a minor and imperfect remedy for a profound failure in modern society. It's of a kind with pharmaceutical treatments for depression or anxiety or obesity -- best seen as a temporary "bridge" towards wellness (achieved, perhaps, through other interventions) -- but altogether just trying to help troubled individuals navigate a society that failed to enable their deeper wellness in the first place.
- reducesuffering 2y agoThese type of techno-solutions are some of the root cause of those “profound failure of modern society!” The technological salve is just a further extreme causing these people’s problems! Much like there exists some societal problems, alcohol is a tiny relief, but can further exacerbate those problems, and you advocate that they drink even more alcohol because society has issues and they should escape it. Idk how we’ve gotten away from such a natural human experience, but everyone knows damn well that the happiest children are out playing soccer with their friends in a field or eating lunch together at a park bench, and not holed up in their room watching endless YouTube.
- 2y ago
- fullshark 2y agoI'll just point to the theory that they didn't want to work for a megacorp creating tools for other megacorps (or worse) and actually believed in OpenAI's (initial) mission to further humanity. The tools are going to be used by deep pocketed entities for their purposes, the compute resources necessary require that to be the case for the foreseeable future.
- anonylizard 2y ago[flagged]
- the_af 2y agoHow would the divorce rate be dangerous at all? edit: downvoted? Really? For asking a question? This seems more like Reddit than HN, I'm really disappointed.
- TacticalCoder 2y ago[flagged]
- cma 2y agoDivorce rate has gone up, violent crime rate has gone way down.
- mstipetic 2y agoWhat the fuck are you talking about? That’s THE theory, the only theory that explains the behavior? The commonly accepted theory? Jesus…
- voidfunc 2y agoThere could be a billion other factors at play. There's way to many variables there to come to any useful conclusions.
- philwelch 2y agoThe problem with these childhood trauma theories is that there’s an obvious genetic confound. If your biological father abandoned or abused your mother, you also share 50% of your DNA with a man who abandoned or abused a woman. If there is a childhood trauma explanation, I think it has a lot more to do with stepfathers being far more likely to be abusive.
- logicchains 2y ago>The theory is that there's nothing more disturbing for a boy than to see her mom with different men as the boy grows up That's an absurd theory. Anyone who's actually spent time around single mothers would have observed that they many have more difficulty discipling male children, and lack of discipline is what causes boys to grow up into criminals (as we see with boys raised by two parents who fail to discipline them).
- lghh 2y ago> along with the suspiciously-timed relaxation of pornographic generations Has ChatGPT's censoring (a loaded term, but idk what else to use) been relaxed with GPT-4o? I have not tested it because I wouldn't have expected them to do it. Does this also extend to other types of censorship or filtering they do? If not, it feels very intentional in the way you're alluding to.
- ziml77 2y agoI don't see anything that says they've changed their policies yet. Just that they're looking into it. I also tested 4o and it still gives me a content policy warning for NSFW requests.
- nicklecompte 2y agoSure, I was being sloppy, I meant "suspiciously timed announcement."
- bnralt 2y agoOne could also say that therapists prey on lonely people who pay them to talk to them and seem like they’re genuinely interested in them, when the therapist wouldn’t bother having a connection with these people once they stop paying. Which I suppose is true from a certain point of view. But from another point of view, sometimes people feel like they don’t have close friends or family to talk to and need something, even if it’s not a genuine love or friendship.
- kettro 2y agoThis is implying that therapy is nothing more than someone to talk to; if that’s your experience with therapy, then you should get another therapist.
- baobabKoodaa 2y agoEvidence points in this direction, though. Different methods of therapy appear to be equally effective despite having theoretical foundations which are conflicting with each other. The common aspect between different therapies seems to be "having someone to talk to", so I'm inclined to believe that really is what's behind the success.
- pphysch 2y agoHaving someone to talk to, who is somewhat emotionally intelligent, who doesn't have strong biases against you, and so on... If you are fortunate, you have people like that in your immediate circle, but increasingly few people do.
- AmericanChopper 2y agoWhat part of the therapist training regimen tests for emotional intelligence? What test do they use to measure this?
- pdabbadabba 2y ago
- llm_trw 2y ago>dangerous for mentally unwell users It's not our job to make the world safe for fundamentally unsafe people.
- deleted 2y ago[deleted]
- kettro 2y agoI would argue that it is society’s job to care for its most vulnerable.
- bongodongobob 2y agoThat doesn't mean we pad all the rooms or ban peanuts. Yes, we should care for them but not at the detriment of the other 99%.
- earthling8118 2y agoWell, conveniently, this is benefitting the 1% much more than the 99%
- llm_trw 2y agoAnd a nanny state benefits a different 1% much more than the 99%.
- yinser 2y agoWhat a wild accusation for someone light years away from the board room.
- nicklecompte 2y agoI wasn't making an accusation about why Leike/Sutskever left, though I definitely understand why you read my comment that way. The actual accusation I am making is that someone at OpenAI knew the risks of GPT-4o and Sam Altman didn't care. I am confident this is true even without spies in the boardroom. My guess is that Leike or Sutskever also knew the risks and actually did care, but that is idle speculation.
- qarl 2y ago> The faking of emotions HEH. In previous versions, when it told jokes, were those fake jokes?
- awkwardpotato 2y agoThose are fundamentally different things. You can tell a joke without understanding context, you can't express emotions if you don't have any. It's a computation model, it cannot feel emotion.
- terse-broccoli 2y agoAre they fundamentally different? Couldn’t you make the argument that it’s advanced from a probabilistic determination of the most likely next token, to a probabilistic determination of the next token AND a probabilistic determination of the inflection that that token should be transmitted with? How is one any more or less fake than the other?
- nicklecompte 2y agoI believe the issue is "emotion" and "emotional tone" are not the same thing, in the same way that "humor" and "written joke" aren't the same thing. You can convey emotional tones without having the emotion (that's what I meant by "fake emotion"), just like you can tell a joke without understanding the punchline.
- qarl 2y agoSo if an AI wrote a touching poem, would you call it fake or not? And how is that different than a joke?
- qarl 2y ago> It's a computation model, it cannot feel emotion. HEH. I'd love to see your proof for that statement.
- jobs_throwaway 2y ago
- sebzim4500 2y ago>I suspect Altman/Brockman/Murati intended for this thing to be dangerous for mentally unwell users Isn't it much more likely that they are just trying to make a product that people want to use? Even Tobacco companies don't go out of their way to give people cancer.
- talldayo 2y agoBut tobacco companies are still complicit in distributing addictive carcinogens to people even if only in trace amounts. The same could be said about predatory business models/products.
- smt88 2y ago> just trying to make a product that people want to use? Sure, but you can do that ethically or unethically. If you make a product that's harmful, disincentivizes healthy behavior (like getting therapy), or becomes addictive, then you've crossed into something unethical. > Even Tobacco companies don't go out of their way to give people cancer. This is like saying pool companies don't go out of their way to get people wet. While it isn't their primary goal, the use of tobacco causes cancer, so their behavior (promoting addiction among children, burying medical research, publishing false research, lobbying against regulation) are all in service of ultimately giving cancer to more people. Cancer and cigarettes are inseparable, the same way casinos and gambling addiction are inseparable.
- GaggiX 2y agoThere has been no relaxation of pornographic generations on OpenAI products.
- qingcharles 2y agoThey announced an intention to allow porn generations.
- GaggiX 2y agoAn OpenAI spokesperson recently stated explicitly that "We have no intention to create AI-generated pornography".
- shmatt 2y agoRealistically its all just probabilistic word generation. People "feel" like an LLM understands them but it doesn't, its just guessing the next token. You could say all our brains are doing are just guessing the next token but thats a little too deep for this morning All these companies are doing now is taking an existing inferencing engine, making it 3% faster, 3% more accurate, etc. per quarter, fighting over the $20/month users One can imagine product is now taking the wheel from engineering and are building ideas on how to monetize the existing engine. Thats essentially what GPT-4o is, and who knows what else is in the 1,2,3 year roadmaps for any of these $20 companies To reach true AGI we need to get past guessing, and that doesn't seem close at all. Even if one of these companies gets better at making you "feel" like its understanding and not guessing, if it isnt actually happening, its not a breakthrough Now with product leading the way, its really interesting to see where these engineers head
- DiogenesKynikos 2y ago> People "feel" like an LLM understands them but it doesn't, its just guessing the next token. You could say all our brains are doing are just guessing the next token but thats a little too deep for this morning "Just" guessing the next token requires understanding. The fact that LLMs are able to respond so intelligently to such a wide range of novel prompts means that they have a very effective internal representation of the outside world. That's what we colloquially call "understanding."
- cmiles74 2y agoI would argue it's an understanding of the relationship between the words; an effective internal representation of those relationships. IMHO, it's still quite a ways from a representation of the outside world.
- deleted 2y ago[deleted]
- carom 2y agoI disagree. I use ChatGPT daily as a replacement for Google. It doesn't understand or have logic, it can spit out information very well though. It has a broad knowledge base. There is no entity there to have an understanding of the topic. This becomes pretty clear when you get to more complex algorithms or low level details like drawing a stack frame. There is not logic there.
- abeppu 2y agoI think the comparison to tobacco companies is misleading because tobacco is not good for anyone, poses a risk of harm to everyone who uses it regularly, and causes very bad outcomes for some of those users. I.e. there's not a large population who can use tobacco without putting themselves at risk. But hypothetically if a lot of people would benefit from a GPT with more fake emotions, that might reasonably counterbalance concerns about harm for a mentally unwell minority. If we build a highway, we know that eventually it will lead to deaths from car crashes -- but if the highway is actually adding value by letting people travel, those benefits might reasonably be expected to outweigh that harm. And the people getting into their cars and onto the highway agree, that the benefits outweigh the costs, right up until they crash. None of this is to say that I think OpenAI's choices here were benevolent rather than a business choice. But I think even if they were trying to do the ethically best thing for the world overall, it would be plausible to move forward I for one found the fake emotions in their voice demos to be really annoying tho.
- alxjrvs 2y agoPlaying devils advocate for a moment - have you ever had a cigarette? It does plenty of good for the user. In fact, I think we do make this risk calculation that you describe in the exact same way - there are plenty of substances that are so toxic to humanity that we make them illegal to own or consume or produce, and the presence of these in your body can sometimes even risk employment, let alone death. We know the risks from cigarettes, but it offers tangible benefits to its users, so they continue to use the product. So too cars and emotionally manipulative AI's, I imagine. (None of this negates your overall point, but I do think the initial tobacco comparison is very apt.)
- abeppu 2y ago> We know the risks from cigarettes Hmm, the tobacco industry is also famous for actively trying to deny and suppress evidence about its harms. They actively didn't want people to be in a position to make a fully informed decision. In cases where jurisdictions introduced policies that packaging etc had to carry factual information about health risks, the tobacco industry pushed back.
- vasco 2y ago> The faking of emotions and overt references to Her (along with the suspiciously-timed relaxation of pornographic generations) are not fine. Are you not aware of how many billions are getting spent on fake girlfriends on Only Fans, with millions of people chatting away with low paid labor across an ocean pretending to be an american girl? This is just reducing costs, but the consumers are already wanting the product. I'm not sure I get the outrage / puritanism. Adults being able to chat to a fake girlfriend if they want to seems super bland. There's way more stuff out there that's way wilder you can do online, potentially also exploiting the real person on the other side of the screen if they are trafficked or whatever. I don't have any mental issues (well, who knows? ha) and genuinely would try it the same way you try a new porn category every once in a while.
- Retr0id 2y agoThere's a lot of harmful stuff already in the world, but most of us would rather not add to the pile on an industrial scale.
- jobs_throwaway 2y agoI for one think consenting adults should be able to do mildly harmful things like waste their time/money on a fake girlfriend if they so choose, and find it offensive that others would try to impose their values on others by restricting them from doing so
- Retr0id 2y agoNobody's restricting anything, we're discussing someone choosing not to work for a particular company.
- Sirizarry 2y agoI agree. I don’t care for and actually dislike most “features“ of AI including and especially chat bots but it’s also none of my business to go and stand on the soap box of my own subjective morality to tell others how to live. If you don’t want people ruining their lives over this, go after root causes not this piddly shit.
- poulpy123 2y agoI saw the faking of emotions, and it's already visible in previous LLM and I find that extremely annoying indeed.
- vunderba 2y agoNot fine... to you. What's your stance on other activities which can lead to harmful actions from people with predilections towards addiction such as: 1. Loot boxes / Fremium games 2. Legalized gambling 3. Pornography etc. etc. I don't really have a horse in the race, neither for/against, but I prefer consistency in belief systems.
- nicklecompte 2y agoI am criticizing Sam Altman for making an unethical business decision. I didn't say "Sam Altman should go to jail because GPT-4o is creepy" or "I want to away your AI girlfriend." So I am not sure what "belief system" (ugh) you think I need to demonstrate the consistency of. Almost seems like this question is a ad hominem distraction.... All three of the categories of businesses you mentioned can be run ethically in theory. In practice that is rare: they are often run in a way that shamelessly preys on vulnerable people, and these tactics should be more closely investigated by regulators - in fact they are regulated, and AI chatbots should be as well. Sam Altman is certainly much much more ethical than most pornography executives (e.g. OnlyFans is complicit in widespread sex trafficking), but I don't think he's any better than freemium game developers. This question seems like a bad-faith rhetorical trap, sort of like the false libertarian dilemmas elsewhere in the thread. I believe the real issue is that people want a culture where lucrative business opportunities aren't subject to ethical considerations, even by outside observers.
- sanjananb 2y ago[dead]