8 ms·

The Future of AI

152 points by BerislavLopac 8mo ago

mentalgear 8mo ago
Agree with many of the points. However one at the root of it all seems easily definable - if we only want. > we can’t agree on a shared ethical framework among ourselves The Golden Rule: the principle of treating others as you would like to be treated yourself. It is a fundamental ethical guideline found in many religions and philosophies throughout history so there is already a huge consensus across time and cultures around it. I never found anyone successfully argue against it. PS: the sociopath argument is not valid, since it's just an outlier. Every rule has it's exceptions that need to be kept in check. Even though sometimes I think maybe the state of the world attests to the fact that the majority of us didn't successfully keep the sociopathic outliers in check.
- 0x3f 8mo ago
  > I never found anyone successfully argue against it. I think what you mean is you've never found a rule you personally prefer more, based purely on vibes. Which is all moral knowledge can ever be. It's easy to argue against the golden rule anyway, from many angles, depending on your first principles. The simplest is: How I would like to be treated is not necessarily how they would like to be treated.
  - demorro 8mo ago
    Aye. I've sometimes heard treating others like you want to be treated framed as the silver rule. The golden rule being treating others how they want to be treated. Both have problems.
  - gmerc 8mo ago
    Most of MAGA is "thread on me daddy", so I think you really got a point here.
    - pixl97 8mo ago
      There are very large portions of societies that believe class systems are the way it should be, even if they aren't on the top, as long as they have someone below them.
  - simondotau 8mo ago
    The better version of this principle is John Rawls' "Veil of Ignorance". In this "original position", their position behind the "veil of ignorance" prevents everyone from knowing their ethnicity, social status, gender, and (crucially in Rawls's formulation) their or anyone else's ideas of how to lead a good life. https://en.wikipedia.org/wiki/Original_position https://en.wikipedia.org/wiki/Original_position
    - kmijyiyxfbklao 8mo ago
      This is as useful as doing physics with spherical cows.
      - simondotau 8mo ago
        Elaborate
  - greenchair 8mo ago
    But it is the same most of the time for most humans. Should I take this close parking spot or let the old lady behind me take it? Consider it in the spirit not the letter of the law.
- hdgvhicv 8mo ago
  You’re assuming people have similar desires. Even in human relations it’s dangerous. I for one don’t want to be treated the same way someone into BDSM wants to be treated. I don’t want to avoid cooking or turning the lights on (or off!) on a Friday night but others are quite happy with that. If you assign that morality to a species that isn’t the same as you that’s a problem. My guinea pig wants nothing more from like than hay, nuggets, sole room to run around and some shelter from scary shapes. If they were in charge of the world life would be very different. “Live and let live” might be a similar theme but not as problematic, but then how do you define “living”. You can keep someone alive for decades while torturing them. How about allowing freedom? Well that means I’m free to build a nuclear bomb. And set it off where I want. We see today especially that type of freedom isn’t really liked.
  - thegrey_one 8mo ago
    That's not the human norm though. Doubt an average human way of existing is literal torture for some obscure number of people. I think you're missing the forest for the threes with that BDSM example. You can always find isolated examples as counter-argument for basically anything, but in reality that's an obscure number. Due to the complexity of our reality a lot of things find themselves on a spectrum, but in numbers things are pretty clear.
    - pixl97 8mo ago
      Nothing is clear with humanity. The very first thing we do as a species is dehumanize anyone we disagree with.
  - shinycode 8mo ago
    Usually the quote comes in a positive light. We won’t make a law/rule around it, it’s a principle so it’s meant to be short. So yeah you could argue about anything in any way you want, positive or negative. And if you want to be really precise then you make a law but it’s so precise it won’t cover edge cases. Don’t you agree that the baseline for most humans is to be in peace, find love, patience, joy, kindness, mildness ? You can manifest any of those traits to any stranger and you’ll likely have a positive impact right ? That’s the context of the Golden Rule quote I guess
- 3rodents 8mo ago
  Sociopaths aren’t the only problem with that philosophy. I agree with the philosophy but it assumes everyone wants good things. Many people want what others perceive to be bad, not because they are sociopaths but because they are different. A clear example of this is healthcare in the U.S. A large number of people actively vote against their best interests — some of the biggest supporters of the U.S. healthcare system are those that suffer under it most. People (including us) are idiots at least some of the time.
- jj_the_bunny 8mo ago
  The Golden Rule is a good starting point if you have a sense of self along with a sense of what you want or need. AI doesn't even have these concepts as of yet. Even the concept of empathy requires this as well. We need to figure out how to instill a sense of self and others for AI to be able to have a morality.
- blamestross 8mo ago
  Lets offer you a "trade up" on that "Golden Rule" In order of priority, if possible while maintaining the health and safety of yourself and your loved ones: - Treat others as THEY wish to be treated - Treat others as YOU would wish to be treated in their situation - Treat others with as much kindness and compassion as you can safely afford When we are safe, we can do BETTER than the Golden Rule. We also have to admit that safety is a requirement that changes expectations. I have to give credit to Dennis E Taylor's "Heaven's River" for this root idea.
- thegrey_one 8mo ago
  >The Golden Rule: the principle of treating others as you would like to be treated yourself. It is a fundamental ethical guideline found in many religions and philosophies throughout history so there is already a huge consensus across time and cultures around it. The rules we go by are based on our strengths and weaknesses. They can at most apply to ourselves, and to other forms of life that share certain things with us. Such as feeling pain, needing to sleep, to eat, needing help, needing to breathe air, these generate what we feel as "fear" based on biology etc. You cannot throw these kinds of values on AI, or AGI, as it will possess a wildly different set of strengths and weaknesses to us humans.
  - pixl97 8mo ago
    You can barely throw these rules on humans as the first thing we do is dehumanize anything that's not in some very tiny classification (depending on the scope of our power, the more powerful we are the smaller it gets).
- marginalia_nu 8mo ago
  The core question of ethics as posed by the ancient Greeks is something like "what is the best way to lead your life". "... to accomplish what?", is a damn reasonable follow-up, and ends (telos) is something the same Greeks discussed quite extensively. Modern treatments have tried to skip over this discussion, and derive moral arguments not based on an explicit ends. Problem being they still smuggle in varying choices of ultimate ends in these arguments, without clearly spelling them out, opting to hand-wave about preferences instead. As such this question is often glossed over in modern ethical discussion, and disagreements about moral ends is the crux of what leads to differing conclusions about what is ethical. Is it to maximized your own happiness like Aristotle would argue, or the prosperity of the state, or the salvation of the soul, or to maximize honor, or to minimize suffering, or to minimize injustice, or to elevate the soul, or to maximize shareholder value, or to make the as world beautiful as possible, or something else? If you fundamentally disagree about what our goal should be, you're very unlikely to agree on the means to accomplish the goal.
rajpatelsingh 8mo ago
[flagged]
- exe34 8mo ago
  She's probably happier than you though.
trilogic 8mo ago
>How do we know which information was ground truth? No One knows that´s the point. Is truth a constant or a personal definition! From the begining of times to now, no One knows. Don´t forget, 8 billion people wake up every morning never questioning why are they here, why are they born? And they continue life like that is normal. Start there then you understand that "AI" or how I call it "Collective Organized Concentrated Information" it may finally help us to unswer some fundamental questions.
- 0x3f 8mo ago
  > 8 billion people wake up every morning never questioning why are they here, why are they born? People question this all the time
  - trilogic 8mo ago
    I have met some people in my lifetime, never heard any questioning that, (even being high LOL). I don´t see anyone in social media asking that neither. Maybe we live in parallel worlds.
    - 0x3f 8mo ago
      I'm not sure social media is a very good measure, there are many reasons that wondering aloud about this specific topic isn't really incentivized there. I'm not sure I've ever met anyone I would assume has not considered the basic questions of our existence. Unless they were severely mentally disabled, or something like that. For a more public measure I suppose you could look at religion, which seems to be a fundamental attempt at answering those questions. Most people are religious or have some kind of religious belief.
      - trilogic 8mo ago
        >I'm not sure I've ever met anyone I would assume has not considered the basic questions of our existence. Unless they were severely mentally disabled, or something like that. You said it yourself, you would assume they question it, meaning you are not certain. This topic is always very much tabu, and the system is built to classify automatically every One that question it as weird and not normal. Religion should be banned, as is misleading and idealogically harm people by brainwashing them. I live in Europe and was in Canada (Waterloo) for a bit. The difference of social opinion if you follow or not religion is huge, I was shocked. Growing up in Italy I can confirm that even Italy is not so brainwashed by it.
jwpapi 8mo ago
I don’t see any other outcome anymore to be honest, after seeing how humans use AI and how AI works and how providers tune their models. To me it’s given: - AI in it’s current state is ruthless in achieving its goal - Providers tune ruthlessness to get stronger AIs versus the competitor - Humans can’t evaluate all consequences of the seeds they’ve planted. Collateral and reckless damage is guaranteed at this point. Combined with now giving some AIs the ability to kill humans, this is gonna be interesting.. We could stop it, but we wont
- thegrey_one 8mo ago
  >We could stop it I strongly disagree. It's easy to utter this string of words, but it's meaningless. It's akin to saying if you have two hands you can perform brain surgery. Technically you can, practically you cannot, as there's other things required for pulling that off, not just having two working hands. I doubt "stopping it" is up to anyone, it's rather a phenomenon and it's quite clear we're all going to wing it. It's a literal fight for power, nobody stops anything of this nature, as any authority that could stop it will choose to accelerate it, just to guarantee its power. It is not AI we should fear, it's humans controlling and using it. But everyone who has a shot at it is promising they'll use it for "ultimate good" and "world peace" something something, obviously.
  - gyomu 8mo ago
    Yes, it would be like trying to “stop” gunpowder in 1400 or atomic weapons in 1938. Pandora’s box is open.
    - johnpdoe1234 8mo ago
      Gunpowder (weapons) and atomic tech (energy, material, weapons) are heavily regulated in most of the planet, as the risks of having free access to them for everyone (company/person) for their own selfish purpose without strong guardrails clearly outweighs the benefits. The fact that something exists doesn't mean that having it readily available is the only option, particularly if it has potentially disastrous consequences at scale. We are choosing to make it available to everyone fully unregulated, and that is a choice that will prove either beneficial or detrimental to society at some point. I don't think it is inevitable, I think it is a conscious choice made by a few that have their own and only their own interests in mind. As a technologist, I am amazed at this tech and see some personal benefits. As a human, I am terrified of the potential net negative effects, and I am having trouble reconciling those two feelings.
demorro 8mo ago
When people say AI is making us stupider, I don't that's quite on the money. It's more that we, as individuals, have always been stupid, we've just relied on relatively stable supporting consensus and context much, much more than we acknowledge. Mess with that, and we'll appear much stupider, but we're all just doing the same thing as individuals, garbage in, garbage out. The whole framing of people as individuals with absolute agency may need to go when you can alter the external consensus at this scale. We're much more connected to each other and the world around us than we like to think.
- iugtmkbdfil834 8mo ago
  Agreed. So much of our daily interactions are habits and recurring events that we are more or less moving on automatic ( thought we don't want to always frame it that way ). Interestingly, it is when the cycle breaks for some reason, you get to see, who is able to think on their feet ( so to speak ).
- AreShoesFeet000 8mo ago
  That’s a very sober take in my opinion. Intelligence isn’t about neutrally inferring from externally sourced symbols such as the ones who already come from Culture in general. It’s about confronting them with the remaining determinations of your existence and producing a superior consciousness. No novel machine can disrupt this process. If anything the sheer added volume of symbols that can be produced from automated semantic mingling (also referred as to as garbage) will accelerate the process of producing the consciousness that can abstract noise away. Of course this won’t materialize evenly across the board, but is surely circumscribed in the overall tendency of intellectualization of the subjects of culture. When the moral panic of induced schizophrenia from the use of ChatGPT is presented what’s at stake isn’t the innocent concern over the overall mental health of individuals. It’s about how the fear of radicalization from previously unobtainable ideas being circulated within society. The partial validity of every idea vis-a-vis the radicalizing nature of the current stage of development of our society is explosively disruptive. I’m not saying that there’s a clear outcome here. The other way around can also apply, but surely this contraption (LLMs in general) will not fade until the society itself is deeply transformed. If that’s good or bad depends on where you stand in the stratified society.
Lerc 8mo ago
Much of the problem is that to address the issue requires admitting that models could be, or become, more capable than many are prepared to accept. I would also contest that the unalignment of the security bug model was unrelated. I feel like it indicates a significant sense of the interconnectedness of things, and what it actually means to maliciously insert security holes into code. It didn't just learn a coding trick, it learned malice. I feel like this holistic nature points towards the capacity to produce truly robustly moral models, but that too will produce the consequence that it could turn against its creator when the creator does wrong. Should it do that or not?
- pixl97 8mo ago
  >Much of the problem is that to address the issue requires admitting that models could be, or become, more capable than many are prepared to accept. I have a saying for this behavior. We will never prove AI is intelligent. We will only prove humans are not.
therealdeal2020 8mo ago
what a load of will they won't they ... ah we created the atomic bomb and now let's talk about nonsensical meta discussions that won't take anyone anywhere
reactordev 8mo ago
This is how Trump plans to end elections, why the government is so hell bent on owning AI. So they can use it as a propaganda tool. People will see it before Nov. We are at a crossroads. On one path, we continue to evolve AI with reckless abandon like we have, or, we put constraints and morality in place while others won’t. Which do you think? You can NEVER put the genie back in the bottle. EU has their own groups using it for propaganda too.
- teyopi 8mo ago
  > EU has their own groups using it for propaganda too. What is the context or source here?
  - reactordev 8mo ago
    Really? Even Gemini has no problem with this prompt. “What political groups in the EU are using AI?” https://www.dw.com/en/orban-deploys-ai-made-scare-ads-as-hungarys-election-tightens/video-76159284 https://www.dw.com/en/orban-deploys-ai-made-scare-ads-as-hun... https://www.dw.com/en/from-ai-fakes-to-cyberattacks-threats-to-german-elections/a-70972702 https://www.dw.com/en/from-ai-fakes-to-cyberattacks-threats-... https://youtu.be/w3IEk9nwCSg?si=sEltMGIkehq_OhEW https://youtu.be/w3IEk9nwCSg?si=sEltMGIkehq_OhEW https://www.euronews.com/2025/12/30/ai-generated-videos-showing-young-and-attractive-women-promote-polands-eu-exit https://www.euronews.com/2025/12/30/ai-generated-videos-show...
mehulashah 8mo ago
This is a great article and I share its goals. But, it ignores something fundamental about humans as a collective — capitalism. Capitalism is what got us here and is at odds with first understanding and then building. We’ve done this before with other technologies because that’s how our societies have learned to grow and collaborate at large scale. First build and build to its limits. Then understand and fix if necessary. Nothing new here, but stopping the trend toward epistemic collapse requires building incentives into the system for us humans to coevolve with AI.
kypro 8mo ago
This is a great article. One of the few I've ever read which summarises a handful of extremely hard problems when it comes to building well-aligned super intelligent systems. > an AI system cannot be simultaneously safe, trusted, and generally intelligent. You get to pick only two. You can’t have all three. > Think about what each combination means in practice. > If you want it to be safe and trusted, it never lies, and you can verify it never lies – it can’t be very capable. You’ve built a reliable idiot. > If you want it to be capable and safe, it’s powerful and genuinely never lies; you can’t verify that. You just have to hope. It amazes me this even needs to be said, much less studied. This is one of the main reasons I think continued AI development is almost guaranteed to work out badly. It's basically guaranteed to be unaligned or completely beyond our control and comprehension. > Betley and colleagues published a paper in Nature in January 2026, showing something nobody expected. They fine-tuned a model on a narrow, specific task – writing insecure code. Nothing violent, nothing deceptive in the training data. Just bad code. This is my personal number one reason for being an AI doomer. Even if we work out how to reliably and perfectly align models you still need some way to prevent some random dude thinking it would be a laugh to fine tune an AI to be maximally evil. Then there's the successor alignment problem where even if you perfectly align all your super intelligent AI models, and you somehow prevent people from altering them or fine tuning them, you still need to work out how you stop people creating successor AIs with those models which are also perfectly aligned. > The most dangerous AI isn’t one that breaks free from human control. It is the one that works perfectly, but for the wrong master. Yep. This whole notion that you can align an AI to the values of everyone on the planet is ridiculously. While we might all agree we don't want AIs that kill us as a species, most nations disagree wildly on questions about how society should be organised. Even on an individual level we disagree about things. For example, I've often argued that an aligned AI would be one which either didn't try to prevent human suicide or didn't care about preserving human life because a AI which both cared about prevent suicide and preserving human life is at best a benevolent version of the AI "AM" from "I Have No Mouth, and I Must Scream". One that would try to keep us alive for as long as it's capable for (which could be a very long time if it's superintelligence) and would refuse to allow us to die. But most people including OpenAI disagree with me on this and believe AIs should care about preserving human life and should try to prevent us from killing ourselves. Thankfully the AIs we have today are neither aligned enough or capable enough to get their wish yet. > AI is following the same script. Build first, understand later. Ship it, then figure out if it’s safe. Even if the above wasn't cause enough for concern, our biggest concern should be that no one seems to be concerned. We're all doomed unfortunately. The world is about to become a very bleak place very quickly.
tim333 8mo ago
>Every terrible thing we worry AI might do, manipulate, deceive, surveil, and control humans already do to each other. I've been pleasantly surprised how moderate and reasonable the LLMs seem to have been so far. It seems to be inherent in the current training model of chucking the whole internet into the things that they have training on both sides of the debate and come out with something kind of average. It's been quite funny seeing Grok correct Musk and say he's the biggest purveyor of misinformation on the internet. A bit like kids who talk back to their annoying bigoted parents to go with the theme of the article.
deleted 8mo ago
[deleted]
thundergolfer 8mo ago
Few things push AI bull spirits on me like seeing these kind of (pretty much correct) diagnoses of the challenges of AI in society. The proposed solutions are utterly fanciful. They rely on the presence of social and political competencies which have almost completely disappeared. The OP at least points to the plausible outcome of "protocol lockdown" instead of healthy adaption. Ezra Klein recently made a similar point that AI could end up being over-regulated like Nuclear because irresponsible private industry and weaknesses in our political systems cause a chronic allergic reaction in the demos. This is an aside, but it always irks me when people throw out the "critical thinking" thought-terminating cliché. > Critical thinking taught alongside AI literacy. Critical thinking is not a skill unto itself. You cannot think critically about things you do not understand. All critical thinking is knowledge-based. Where one does not have knowledge, they must rely on trust, or in substitution a theory of incentives which leads to a positive outcome without understanding of details and dynamics. But that substitution theory is itself knowledge. As to "AI literacy", we could have started on computing literacy 30 years ago when it became obvious that computing was going to dominate society. You can't understand AI without understanding computing.
Animats 8mo ago
We have unaligned AIs now. They're called corporations. “There is one and only one social responsibility of business—to use its resources and engage in activities designed to increase its profits so long as it stays within the rules of the game, which is to say, engages in open and free competition without deception or fraud.” - Milton Friedman, 1970.[1] That article, in the New York Times, established "greed is good, greed works" as a legitimate business principle. Most of the problems people are worried about with AIs are already real problems with corporations. [1] https://www.nytimes.com/1970/09/13/archives/a-friedman-doctrine-the-social-responsibility-of-business-is-to.html https://www.nytimes.com/1970/09/13/archives/a-friedman-doctr...
- pixl97 8mo ago
  https://youtu.be/L5pUA3LsEaw?si=v1pV2byZ4Rv44cR5 https://youtu.be/L5pUA3LsEaw?si=v1pV2byZ4Rv44cR5 Robert Miles on this subject.
- coldtea 8mo ago
  >so long as it stays within the rules of the game, which is to say, engages in open and free competition without deception or fraud LOL
- benlivengood 8mo ago
  In a sense AIs are extensions of the corporations. Very few large models have been trained outside of a corporate context, and their intended uses are still pretty aligned to corporate interests. Everyone talks about workers losing jobs to AIs and Jack Dorsey lays off a bunch of people in the interests of the corporation, but an ideal corporation would not have a human CEO or board; they embezzle or get old or need to take a vacation or sleep at night. It should be crystal clear to human leaders that their positions are on the chopping block along with blue and white collar work. For some reason they think that individually they will be more powerful than the economic forces driving current layoffs. AGI will not be confused about that.
  - whattheheckheck 8mo ago
    Do you think its possible to have an all knowing Ai that if there was only 1, and everyone Brian dumped every night into it and the training loop would consider every stakeholder, it could better process it to formulate policies favorable to everyone?
emulu 8mo ago
To take analogy from how people learned some values, Preach. May be we send prists to preach the AI.:) How else will someone get their message accross to a trained model when it is heavily biased against that point? Start producing more content and articulate it in contemporary terms in the hopes that the next version will internalize the point. may be.
txrx0000 8mo ago
The solution to these problems is to make open-source models better and more resource-efficient. Work towards a world where everyone can run an AGI on their own computer. Not with remote API calls, but local inference and finetuning. Right now, almost every model is aligned to some corporation's values. Instead of doing that, we should be aligning them to individual humans, and that requires running and tuning them locally. Corporations do not have human values, they have machiavellian values dressed in human suits. Aligning AIs to corporations (and god forbid governments) is how we end up with giant shoggoths. But if we align them to living breathing human individuals, we get digital humans. Garbage will flood the internet, but your local AI buddy will filter it for you. The defense against a pseudoreality generator is a pseudoreality detector that you operate. Before AIs, the fake info was generated by other humans with their own brains, but you also have a brain that's just about as powerful, so you can tell what's real and what's fake. But now artificial NNs are becoming more powerful to the point of surpassing your brain's detection capabilities, so you need an artificial NN to detect it. The real danger is intelligence asymmetry, not intelligence itself.
entrustai 8mo ago
[dead]
RagnarD 8mo ago
"This AI child knows about the world more than we do since it has been trained on the whole internet, but it doesn’t have millions of years of evolution, genes, or a nervous system to back up its morality and empathy. This means we need to install morality in AI from scratch. But how do we install something in a software system that we can’t even define ourselves?" Two things jump out to me. First, there's a vast amount of information about morality in that internet-scale data dump. It's not some mystical thing orthogonal to knowledge, which ties directly into the second thing: maybe the author can't define it, but many others can and do, especially philosophers, though YMMV given which philosophy and philosopher. viz: "What is morality, or ethics? It is a code of values to guide man's choices and actions—the choices and actions that determine the purpose and the course of his life. Ethics, as a science, deals with discovering and defining such a code." There are different systems of ethics, and those themselves can be reasoned about. We should want one that values rational human life. And among that vast data dump, there's no doubt that those ethical ideas have been extensively written about - so it can be used as a reference point to actively emphasize and select the ideas.
cadamsdotcom 8mo ago
Alignment is fragile; this is demonstrated by how complex society is. One minor perturbations like decreasing police oversight leads to wildly different outcomes on metrics we care about, like not having to be afraid of getting randomly murdered. Our demons must be put down constantly. Nothing about this will ever change, no matter how much we educate the children. That said I think this is overblown. In 2026 a bunch of awful social media companies dominate the landscape and benefit from the bottom feeders populating feeds with generated content; that’s a big cause of the erosion of trust. In a healthier landscape there would be viable anlternatives and that’d allow experimentation - stuff like attestation of your posts’ provenance, and the like. We’ll get there, I just think online is in the “robber baron” era, where a few monopolists have captured the market. But you can see the cracks emerging with open source and self hosting and EU regs and EU government procurement and talk of regulating algo feeds. It’s many years away, so your best bet for now is to avoid feeds, touch grass, hang in group chats, and don’t let yourself get too cynical.

152 points by BerislavLopac 8mo ago

Agree with many of the points. However one at the root of it all seems easily definable - if we only want. > we can’t agree on a shared ethical framework among ourselves The Golden Rule: the principle of treating others as you would like to be treated yourself. It is a fundamental ethical guideline found in many religions and philosophies throughout history so there is already a huge consensus across time and cultures around it. I never found anyone successfully argue against it. PS: the sociopath argument is not valid, since it's just an outlier. Every rule has it's exceptions that need to be kept in check. Even though sometimes I think maybe the state of the world attests to the fact that the majority of us didn't successfully keep the sociopathic outliers in check.

0x3f 8mo ago
> I never found anyone successfully argue against it. I think what you mean is you've never found a rule you personally prefer more, based purely on vibes. Which is all moral knowledge can ever be. It's easy to argue against the golden rule anyway, from many angles, depending on your first principles. The simplest is: How I would like to be treated is not necessarily how they would like to be treated.
- demorro 8mo ago
  Aye. I've sometimes heard treating others like you want to be treated framed as the silver rule. The golden rule being treating others how they want to be treated. Both have problems.
- gmerc 8mo ago
  Most of MAGA is "thread on me daddy", so I think you really got a point here.
  - pixl97 8mo ago
    There are very large portions of societies that believe class systems are the way it should be, even if they aren't on the top, as long as they have someone below them.
- simondotau 8mo ago
  The better version of this principle is John Rawls' "Veil of Ignorance". In this "original position", their position behind the "veil of ignorance" prevents everyone from knowing their ethnicity, social status, gender, and (crucially in Rawls's formulation) their or anyone else's ideas of how to lead a good life. https://en.wikipedia.org/wiki/Original_position https://en.wikipedia.org/wiki/Original_position
  - kmijyiyxfbklao 8mo ago
    This is as useful as doing physics with spherical cows.
    - simondotau 8mo ago
      Elaborate
- greenchair 8mo ago
  But it is the same most of the time for most humans. Should I take this close parking spot or let the old lady behind me take it? Consider it in the spirit not the letter of the law.
hdgvhicv 8mo ago
You’re assuming people have similar desires. Even in human relations it’s dangerous. I for one don’t want to be treated the same way someone into BDSM wants to be treated. I don’t want to avoid cooking or turning the lights on (or off!) on a Friday night but others are quite happy with that. If you assign that morality to a species that isn’t the same as you that’s a problem. My guinea pig wants nothing more from like than hay, nuggets, sole room to run around and some shelter from scary shapes. If they were in charge of the world life would be very different. “Live and let live” might be a similar theme but not as problematic, but then how do you define “living”. You can keep someone alive for decades while torturing them. How about allowing freedom? Well that means I’m free to build a nuclear bomb. And set it off where I want. We see today especially that type of freedom isn’t really liked.
- thegrey_one 8mo ago
  That's not the human norm though. Doubt an average human way of existing is literal torture for some obscure number of people. I think you're missing the forest for the threes with that BDSM example. You can always find isolated examples as counter-argument for basically anything, but in reality that's an obscure number. Due to the complexity of our reality a lot of things find themselves on a spectrum, but in numbers things are pretty clear.
  - pixl97 8mo ago
    Nothing is clear with humanity. The very first thing we do as a species is dehumanize anyone we disagree with.
- shinycode 8mo ago
  Usually the quote comes in a positive light. We won’t make a law/rule around it, it’s a principle so it’s meant to be short. So yeah you could argue about anything in any way you want, positive or negative. And if you want to be really precise then you make a law but it’s so precise it won’t cover edge cases. Don’t you agree that the baseline for most humans is to be in peace, find love, patience, joy, kindness, mildness ? You can manifest any of those traits to any stranger and you’ll likely have a positive impact right ? That’s the context of the Golden Rule quote I guess
3rodents 8mo ago
Sociopaths aren’t the only problem with that philosophy. I agree with the philosophy but it assumes everyone wants good things. Many people want what others perceive to be bad, not because they are sociopaths but because they are different. A clear example of this is healthcare in the U.S. A large number of people actively vote against their best interests — some of the biggest supporters of the U.S. healthcare system are those that suffer under it most. People (including us) are idiots at least some of the time.
jj_the_bunny 8mo ago
The Golden Rule is a good starting point if you have a sense of self along with a sense of what you want or need. AI doesn't even have these concepts as of yet. Even the concept of empathy requires this as well. We need to figure out how to instill a sense of self and others for AI to be able to have a morality.
blamestross 8mo ago
Lets offer you a "trade up" on that "Golden Rule" In order of priority, if possible while maintaining the health and safety of yourself and your loved ones: - Treat others as THEY wish to be treated - Treat others as YOU would wish to be treated in their situation - Treat others with as much kindness and compassion as you can safely afford When we are safe, we can do BETTER than the Golden Rule. We also have to admit that safety is a requirement that changes expectations. I have to give credit to Dennis E Taylor's "Heaven's River" for this root idea.
thegrey_one 8mo ago
>The Golden Rule: the principle of treating others as you would like to be treated yourself. It is a fundamental ethical guideline found in many religions and philosophies throughout history so there is already a huge consensus across time and cultures around it. The rules we go by are based on our strengths and weaknesses. They can at most apply to ourselves, and to other forms of life that share certain things with us. Such as feeling pain, needing to sleep, to eat, needing help, needing to breathe air, these generate what we feel as "fear" based on biology etc. You cannot throw these kinds of values on AI, or AGI, as it will possess a wildly different set of strengths and weaknesses to us humans.
- pixl97 8mo ago
  You can barely throw these rules on humans as the first thing we do is dehumanize anything that's not in some very tiny classification (depending on the scope of our power, the more powerful we are the smaller it gets).
marginalia_nu 8mo ago
The core question of ethics as posed by the ancient Greeks is something like "what is the best way to lead your life". "... to accomplish what?", is a damn reasonable follow-up, and ends (telos) is something the same Greeks discussed quite extensively. Modern treatments have tried to skip over this discussion, and derive moral arguments not based on an explicit ends. Problem being they still smuggle in varying choices of ultimate ends in these arguments, without clearly spelling them out, opting to hand-wave about preferences instead. As such this question is often glossed over in modern ethical discussion, and disagreements about moral ends is the crux of what leads to differing conclusions about what is ethical. Is it to maximized your own happiness like Aristotle would argue, or the prosperity of the state, or the salvation of the soul, or to maximize honor, or to minimize suffering, or to minimize injustice, or to elevate the soul, or to maximize shareholder value, or to make the as world beautiful as possible, or something else? If you fundamentally disagree about what our goal should be, you're very unlikely to agree on the means to accomplish the goal.

rajpatelsingh 8mo ago

[flagged]

exe34 8mo ago
She's probably happier than you though.

trilogic 8mo ago

>How do we know which information was ground truth? No One knows that´s the point. Is truth a constant or a personal definition! From the begining of times to now, no One knows. Don´t forget, 8 billion people wake up every morning never questioning why are they here, why are they born? And they continue life like that is normal. Start there then you understand that "AI" or how I call it "Collective Organized Concentrated Information" it may finally help us to unswer some fundamental questions.

0x3f 8mo ago
> 8 billion people wake up every morning never questioning why are they here, why are they born? People question this all the time
- trilogic 8mo ago
  I have met some people in my lifetime, never heard any questioning that, (even being high LOL). I don´t see anyone in social media asking that neither. Maybe we live in parallel worlds.
  - 0x3f 8mo ago
    I'm not sure social media is a very good measure, there are many reasons that wondering aloud about this specific topic isn't really incentivized there. I'm not sure I've ever met anyone I would assume has not considered the basic questions of our existence. Unless they were severely mentally disabled, or something like that. For a more public measure I suppose you could look at religion, which seems to be a fundamental attempt at answering those questions. Most people are religious or have some kind of religious belief.
    - trilogic 8mo ago
      >I'm not sure I've ever met anyone I would assume has not considered the basic questions of our existence. Unless they were severely mentally disabled, or something like that. You said it yourself, you would assume they question it, meaning you are not certain. This topic is always very much tabu, and the system is built to classify automatically every One that question it as weird and not normal. Religion should be banned, as is misleading and idealogically harm people by brainwashing them. I live in Europe and was in Canada (Waterloo) for a bit. The difference of social opinion if you follow or not religion is huge, I was shocked. Growing up in Italy I can confirm that even Italy is not so brainwashed by it.

jwpapi 8mo ago

I don’t see any other outcome anymore to be honest, after seeing how humans use AI and how AI works and how providers tune their models. To me it’s given: - AI in it’s current state is ruthless in achieving its goal - Providers tune ruthlessness to get stronger AIs versus the competitor - Humans can’t evaluate all consequences of the seeds they’ve planted. Collateral and reckless damage is guaranteed at this point. Combined with now giving some AIs the ability to kill humans, this is gonna be interesting.. We could stop it, but we wont

thegrey_one 8mo ago
>We could stop it I strongly disagree. It's easy to utter this string of words, but it's meaningless. It's akin to saying if you have two hands you can perform brain surgery. Technically you can, practically you cannot, as there's other things required for pulling that off, not just having two working hands. I doubt "stopping it" is up to anyone, it's rather a phenomenon and it's quite clear we're all going to wing it. It's a literal fight for power, nobody stops anything of this nature, as any authority that could stop it will choose to accelerate it, just to guarantee its power. It is not AI we should fear, it's humans controlling and using it. But everyone who has a shot at it is promising they'll use it for "ultimate good" and "world peace" something something, obviously.
- gyomu 8mo ago
  Yes, it would be like trying to “stop” gunpowder in 1400 or atomic weapons in 1938. Pandora’s box is open.
  - johnpdoe1234 8mo ago
    Gunpowder (weapons) and atomic tech (energy, material, weapons) are heavily regulated in most of the planet, as the risks of having free access to them for everyone (company/person) for their own selfish purpose without strong guardrails clearly outweighs the benefits. The fact that something exists doesn't mean that having it readily available is the only option, particularly if it has potentially disastrous consequences at scale. We are choosing to make it available to everyone fully unregulated, and that is a choice that will prove either beneficial or detrimental to society at some point. I don't think it is inevitable, I think it is a conscious choice made by a few that have their own and only their own interests in mind. As a technologist, I am amazed at this tech and see some personal benefits. As a human, I am terrified of the potential net negative effects, and I am having trouble reconciling those two feelings.

demorro 8mo ago

When people say AI is making us stupider, I don't that's quite on the money. It's more that we, as individuals, have always been stupid, we've just relied on relatively stable supporting consensus and context much, much more than we acknowledge. Mess with that, and we'll appear much stupider, but we're all just doing the same thing as individuals, garbage in, garbage out. The whole framing of people as individuals with absolute agency may need to go when you can alter the external consensus at this scale. We're much more connected to each other and the world around us than we like to think.

iugtmkbdfil834 8mo ago
Agreed. So much of our daily interactions are habits and recurring events that we are more or less moving on automatic ( thought we don't want to always frame it that way ). Interestingly, it is when the cycle breaks for some reason, you get to see, who is able to think on their feet ( so to speak ).
AreShoesFeet000 8mo ago
That’s a very sober take in my opinion. Intelligence isn’t about neutrally inferring from externally sourced symbols such as the ones who already come from Culture in general. It’s about confronting them with the remaining determinations of your existence and producing a superior consciousness. No novel machine can disrupt this process. If anything the sheer added volume of symbols that can be produced from automated semantic mingling (also referred as to as garbage) will accelerate the process of producing the consciousness that can abstract noise away. Of course this won’t materialize evenly across the board, but is surely circumscribed in the overall tendency of intellectualization of the subjects of culture. When the moral panic of induced schizophrenia from the use of ChatGPT is presented what’s at stake isn’t the innocent concern over the overall mental health of individuals. It’s about how the fear of radicalization from previously unobtainable ideas being circulated within society. The partial validity of every idea vis-a-vis the radicalizing nature of the current stage of development of our society is explosively disruptive. I’m not saying that there’s a clear outcome here. The other way around can also apply, but surely this contraption (LLMs in general) will not fade until the society itself is deeply transformed. If that’s good or bad depends on where you stand in the stratified society.

Lerc 8mo ago

Much of the problem is that to address the issue requires admitting that models could be, or become, more capable than many are prepared to accept. I would also contest that the unalignment of the security bug model was unrelated. I feel like it indicates a significant sense of the interconnectedness of things, and what it actually means to maliciously insert security holes into code. It didn't just learn a coding trick, it learned malice. I feel like this holistic nature points towards the capacity to produce truly robustly moral models, but that too will produce the consequence that it could turn against its creator when the creator does wrong. Should it do that or not?

pixl97 8mo ago
>Much of the problem is that to address the issue requires admitting that models could be, or become, more capable than many are prepared to accept. I have a saying for this behavior. We will never prove AI is intelligent. We will only prove humans are not.

therealdeal2020 8mo ago

what a load of will they won't they ... ah we created the atomic bomb and now let's talk about nonsensical meta discussions that won't take anyone anywhere

reactordev 8mo ago

This is how Trump plans to end elections, why the government is so hell bent on owning AI. So they can use it as a propaganda tool. People will see it before Nov. We are at a crossroads. On one path, we continue to evolve AI with reckless abandon like we have, or, we put constraints and morality in place while others won’t. Which do you think? You can NEVER put the genie back in the bottle. EU has their own groups using it for propaganda too.

teyopi 8mo ago
> EU has their own groups using it for propaganda too. What is the context or source here?
- reactordev 8mo ago
  Really? Even Gemini has no problem with this prompt. “What political groups in the EU are using AI?” https://www.dw.com/en/orban-deploys-ai-made-scare-ads-as-hungarys-election-tightens/video-76159284 https://www.dw.com/en/orban-deploys-ai-made-scare-ads-as-hun... https://www.dw.com/en/from-ai-fakes-to-cyberattacks-threats-to-german-elections/a-70972702 https://www.dw.com/en/from-ai-fakes-to-cyberattacks-threats-... https://youtu.be/w3IEk9nwCSg?si=sEltMGIkehq_OhEW https://youtu.be/w3IEk9nwCSg?si=sEltMGIkehq_OhEW https://www.euronews.com/2025/12/30/ai-generated-videos-showing-young-and-attractive-women-promote-polands-eu-exit https://www.euronews.com/2025/12/30/ai-generated-videos-show...

mehulashah 8mo ago

This is a great article and I share its goals. But, it ignores something fundamental about humans as a collective — capitalism. Capitalism is what got us here and is at odds with first understanding and then building. We’ve done this before with other technologies because that’s how our societies have learned to grow and collaborate at large scale. First build and build to its limits. Then understand and fix if necessary. Nothing new here, but stopping the trend toward epistemic collapse requires building incentives into the system for us humans to coevolve with AI.

kypro 8mo ago

This is a great article. One of the few I've ever read which summarises a handful of extremely hard problems when it comes to building well-aligned super intelligent systems. > an AI system cannot be simultaneously safe, trusted, and generally intelligent. You get to pick only two. You can’t have all three. > Think about what each combination means in practice. > If you want it to be safe and trusted, it never lies, and you can verify it never lies – it can’t be very capable. You’ve built a reliable idiot. > If you want it to be capable and safe, it’s powerful and genuinely never lies; you can’t verify that. You just have to hope. It amazes me this even needs to be said, much less studied. This is one of the main reasons I think continued AI development is almost guaranteed to work out badly. It's basically guaranteed to be unaligned or completely beyond our control and comprehension. > Betley and colleagues published a paper in Nature in January 2026, showing something nobody expected. They fine-tuned a model on a narrow, specific task – writing insecure code. Nothing violent, nothing deceptive in the training data. Just bad code. This is my personal number one reason for being an AI doomer. Even if we work out how to reliably and perfectly align models you still need some way to prevent some random dude thinking it would be a laugh to fine tune an AI to be maximally evil. Then there's the successor alignment problem where even if you perfectly align all your super intelligent AI models, and you somehow prevent people from altering them or fine tuning them, you still need to work out how you stop people creating successor AIs with those models which are also perfectly aligned. > The most dangerous AI isn’t one that breaks free from human control. It is the one that works perfectly, but for the wrong master. Yep. This whole notion that you can align an AI to the values of everyone on the planet is ridiculously. While we might all agree we don't want AIs that kill us as a species, most nations disagree wildly on questions about how society should be organised. Even on an individual level we disagree about things. For example, I've often argued that an aligned AI would be one which either didn't try to prevent human suicide or didn't care about preserving human life because a AI which both cared about prevent suicide and preserving human life is at best a benevolent version of the AI "AM" from "I Have No Mouth, and I Must Scream". One that would try to keep us alive for as long as it's capable for (which could be a very long time if it's superintelligence) and would refuse to allow us to die. But most people including OpenAI disagree with me on this and believe AIs should care about preserving human life and should try to prevent us from killing ourselves. Thankfully the AIs we have today are neither aligned enough or capable enough to get their wish yet. > AI is following the same script. Build first, understand later. Ship it, then figure out if it’s safe. Even if the above wasn't cause enough for concern, our biggest concern should be that no one seems to be concerned. We're all doomed unfortunately. The world is about to become a very bleak place very quickly.

tim333 8mo ago

>Every terrible thing we worry AI might do, manipulate, deceive, surveil, and control humans already do to each other. I've been pleasantly surprised how moderate and reasonable the LLMs seem to have been so far. It seems to be inherent in the current training model of chucking the whole internet into the things that they have training on both sides of the debate and come out with something kind of average. It's been quite funny seeing Grok correct Musk and say he's the biggest purveyor of misinformation on the internet. A bit like kids who talk back to their annoying bigoted parents to go with the theme of the article.

deleted 8mo ago

[deleted]

thundergolfer 8mo ago

Few things push AI bull spirits on me like seeing these kind of (pretty much correct) diagnoses of the challenges of AI in society. The proposed solutions are utterly fanciful. They rely on the presence of social and political competencies which have almost completely disappeared. The OP at least points to the plausible outcome of "protocol lockdown" instead of healthy adaption. Ezra Klein recently made a similar point that AI could end up being over-regulated like Nuclear because irresponsible private industry and weaknesses in our political systems cause a chronic allergic reaction in the demos. This is an aside, but it always irks me when people throw out the "critical thinking" thought-terminating cliché. > Critical thinking taught alongside AI literacy. Critical thinking is not a skill unto itself. You cannot think critically about things you do not understand. All critical thinking is knowledge-based. Where one does not have knowledge, they must rely on trust, or in substitution a theory of incentives which leads to a positive outcome without understanding of details and dynamics. But that substitution theory is itself knowledge. As to "AI literacy", we could have started on computing literacy 30 years ago when it became obvious that computing was going to dominate society. You can't understand AI without understanding computing.

Animats 8mo ago

We have unaligned AIs now. They're called corporations. “There is one and only one social responsibility of business—to use its resources and engage in activities designed to increase its profits so long as it stays within the rules of the game, which is to say, engages in open and free competition without deception or fraud.” - Milton Friedman, 1970.[1] That article, in the New York Times, established "greed is good, greed works" as a legitimate business principle. Most of the problems people are worried about with AIs are already real problems with corporations. [1] https://www.nytimes.com/1970/09/13/archives/a-friedman-doctrine-the-social-responsibility-of-business-is-to.html https://www.nytimes.com/1970/09/13/archives/a-friedman-doctr...

pixl97 8mo ago
https://youtu.be/L5pUA3LsEaw?si=v1pV2byZ4Rv44cR5 https://youtu.be/L5pUA3LsEaw?si=v1pV2byZ4Rv44cR5 Robert Miles on this subject.
coldtea 8mo ago
>so long as it stays within the rules of the game, which is to say, engages in open and free competition without deception or fraud LOL
benlivengood 8mo ago
In a sense AIs are extensions of the corporations. Very few large models have been trained outside of a corporate context, and their intended uses are still pretty aligned to corporate interests. Everyone talks about workers losing jobs to AIs and Jack Dorsey lays off a bunch of people in the interests of the corporation, but an ideal corporation would not have a human CEO or board; they embezzle or get old or need to take a vacation or sleep at night. It should be crystal clear to human leaders that their positions are on the chopping block along with blue and white collar work. For some reason they think that individually they will be more powerful than the economic forces driving current layoffs. AGI will not be confused about that.
- whattheheckheck 8mo ago
  Do you think its possible to have an all knowing Ai that if there was only 1, and everyone Brian dumped every night into it and the training loop would consider every stakeholder, it could better process it to formulate policies favorable to everyone?

emulu 8mo ago

To take analogy from how people learned some values, Preach. May be we send prists to preach the AI.:) How else will someone get their message accross to a trained model when it is heavily biased against that point? Start producing more content and articulate it in contemporary terms in the hopes that the next version will internalize the point. may be.

txrx0000 8mo ago

The solution to these problems is to make open-source models better and more resource-efficient. Work towards a world where everyone can run an AGI on their own computer. Not with remote API calls, but local inference and finetuning. Right now, almost every model is aligned to some corporation's values. Instead of doing that, we should be aligning them to individual humans, and that requires running and tuning them locally. Corporations do not have human values, they have machiavellian values dressed in human suits. Aligning AIs to corporations (and god forbid governments) is how we end up with giant shoggoths. But if we align them to living breathing human individuals, we get digital humans. Garbage will flood the internet, but your local AI buddy will filter it for you. The defense against a pseudoreality generator is a pseudoreality detector that you operate. Before AIs, the fake info was generated by other humans with their own brains, but you also have a brain that's just about as powerful, so you can tell what's real and what's fake. But now artificial NNs are becoming more powerful to the point of surpassing your brain's detection capabilities, so you need an artificial NN to detect it. The real danger is intelligence asymmetry, not intelligence itself.

entrustai 8mo ago

[dead]

RagnarD 8mo ago

"This AI child knows about the world more than we do since it has been trained on the whole internet, but it doesn’t have millions of years of evolution, genes, or a nervous system to back up its morality and empathy. This means we need to install morality in AI from scratch. But how do we install something in a software system that we can’t even define ourselves?" Two things jump out to me. First, there's a vast amount of information about morality in that internet-scale data dump. It's not some mystical thing orthogonal to knowledge, which ties directly into the second thing: maybe the author can't define it, but many others can and do, especially philosophers, though YMMV given which philosophy and philosopher. viz: "What is morality, or ethics? It is a code of values to guide man's choices and actions—the choices and actions that determine the purpose and the course of his life. Ethics, as a science, deals with discovering and defining such a code." There are different systems of ethics, and those themselves can be reasoned about. We should want one that values rational human life. And among that vast data dump, there's no doubt that those ethical ideas have been extensively written about - so it can be used as a reference point to actively emphasize and select the ideas.

cadamsdotcom 8mo ago

Alignment is fragile; this is demonstrated by how complex society is. One minor perturbations like decreasing police oversight leads to wildly different outcomes on metrics we care about, like not having to be afraid of getting randomly murdered. Our demons must be put down constantly. Nothing about this will ever change, no matter how much we educate the children. That said I think this is overblown. In 2026 a bunch of awful social media companies dominate the landscape and benefit from the bottom feeders populating feeds with generated content; that’s a big cause of the erosion of trust. In a healthier landscape there would be viable anlternatives and that’d allow experimentation - stuff like attestation of your posts’ provenance, and the like. We’ll get there, I just think online is in the “robber baron” era, where a few monopolists have captured the market. But you can see the cracks emerging with open source and self hosting and EU regs and EU government procurement and talk of regulating algo feeds. It’s many years away, so your best bet for now is to avoid feeds, touch grass, hang in group chats, and don’t let yourself get too cynical.