I have spilled a lot of ink about my thoughts on the vast majority of “AI ethics” work done at frontier LLM labs already, but indulge me as I do it again. Whil

I have spilled a lot of ink about my thoughts on the vast majority of “AI ethics” work done at frontier LLM labs already, but indulge me as I do it again. While some consideration for extreme scenarios can have merit, I don't think the industry is paying attention to the most pressing issues of our time. Especially this early and given (admittedly my personal) doubts that LLMs could ever attain what one could consider intelligence, the focus on doomsday scenarios and preventing models outputting information that has been freely accessible on the internet for decades is simply not the best use of our limited attention. These topics take up an obscene percentage in system cards, public reporting, and discussions on regulation by thought leaders, partly because this does benefit companies by increasing investment ("we are so close to Super-Mega-Hyper-AGI that we have to keep the LLM from nuking us, if you invest with us we'll crack that in a trifle"[0]) and keeping regulators from focusing on more immediate concerns that could affect profits. Current day ethical issues like copyright owner compensation, misinformation at scale, and how these tools affect people’s psyche do not get nearly enough focus. We have already had multiple people die [1] due to improper guardrails. We appear to have people isolating themselves by replacing (or no longer seeking) companionship with LLM output. And we have seen attempts to pass off model generated content as real in order to influence political reporting [2]. These are real issues we see right now, yet the industry seems unwilling to fully wrestle with how we could address them. In this context, I actually saw GPT-5 as a step in the right direction. I had (naive, I admit) hopes this signaled OpenAI shifting toward tangible, current day concerns, complementing Anthropic’s more grounded publications that put an emphasis on real world tasks a professional user may encounter (Heck, their Claude 4 system card looked into LLM tool calls when asked to be bold and making value judgements which lead to them being criticized for "snitching", when multiple models appear to do the same [4]). Researchers putting some focus on down the lime issues is fine. I am not against that, maybe it'll be of great value in a hypothetical future, but “super alignment” and terminator level scenarios mustn't be the only things considered. GPT-5, again, seemed to be designed in a way that considered users mental health and the issues that arise from overly agreeable output better than most, even though it wasn't reported as a focus for them. I found that a good step. Then again, my experience with GPT-5 and Horizon Alpha Beta seemed different from most either way. The latter did unimaginably worse in my personal testing, appallingly I maintain, while the former was far more impressive to me than what the common sentiment seems to be, especially in dealing with extensive context, handling slight (intentional) tool call changes that weren't previously provided to the model and longer term task coherence. Regardless of raw performance, GPT-5 being closer to o3 or o4 than to GPT-4o in subjective agreeableness seemed like a good development for reducing some of the harm these models may cause to certain susceptible users psyches, especially if this development continues. We have already seen a subset of users largely or even entirely seizing to seek out human companionship, which is likely to affect them in not yet fully understood ways in the long term. As models advance this may expand to an ever increasing fraction of the user base and in my opinion needs to be of concern to the entire industry. If OpenAI now walks this back, I will be severely disappointed. I would also be surprised if there weren't precise internal insights about which users are most impacted and how, similar to how cigarette and gambling companies have long known who drives profit and at what cost, while fighting regulation. If this continues and we see more people turning to LLMs for companionship whilst isolating themselves, I could see a similar trajectory in a few decades, or given the pace, in just a few years. Essentially, a lot of people suffering in the future due to a lack of regulatory action in the present. Then again, even if all frontier LLM labs take a moral stance on this, there may be others who fill that demand and, of course, local models are always an option. So maybe the impact of this technology on certain users psyche has already become a future public healthcare expenditure we cannot prevent, which would be very depressing indeed. I want to add that I wouldn't be surprised if a large number of researchers didn't argue for similar behind closed doors, but the conversation is unfortunately dominated by a mix of annoyingly loud, yet not very grounded in reality [4] folks, alongside a group of investors and company leaders who keep using the former to, as mentioned, justify hype-bubble valuations and draw attention away from actually impactful regulation. [0] https://www.windowscentral.com/artificial-intelligence/openai-chatgpt/sam-altman-is-afraid-of-openais-gpt-5-creation https://www.windowscentral.com/artificial-intelligence/opena... [1] https://www.reuters.com/investigates/special-report/meta-ai-chatbot-death/ https://www.reuters.com/investigates/special-report/meta-ai-... and https://www.nbcwashington.com/investigations/moms-lawsuit-blames-14-year-old-sons-suicide-on-ai-relationship/3967878/ https://www.nbcwashington.com/investigations/moms-lawsuit-bl... [2] https://www.theguardian.com/us-news/2025/aug/07/chris-cuomo-alexandria-ocasio-cortez-deepfake https://www.theguardian.com/us-news/2025/aug/07/chris-cuomo-... and https://www.reuters.com/fact-check/video-does-not-show-ukraines-president-zelenskiy-belly-dancing-2024-01-10/ https://www.reuters.com/fact-check/video-does-not-show-ukrai... [3] https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686f4f3b2ff47.pdf https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686... and https://snitchbench.t3.gg https://snitchbench.t3.gg [4] https://xkcd.com/1450/ https://xkcd.com/1450/ and https://en.wikipedia.org/wiki/Pascal%27s_wager https://en.wikipedia.org/wiki/Pascal%27s_wager