21 ms·

The issue with our AI debate is that there's not a single "problem" but many inter-dependent issues without a clear system-wide solution. - Big tech monopolizi

The issue with our AI debate is that there's not a single "problem" but many inter-dependent issues without a clear system-wide solution. - Big tech monopolizing the models, data, and hardware. - Copyright concerns. - Job security. - AIs becoming sentient and causing harm for their own ends. - Corporations intentionally using AI to cause harm for their own ends. - Feedback loops will flood the internet with content of unknown provenance, which get included in the next model, etc. - AI hallucinations resulting in widespread persistent errors that cause an epistemological crisis. - The training set is inherently biased; human knowledge and perspectives not represented in this set could be systematically wiped from public discourse. We can have meaningful discussions on each of these topics. And I'm sure we all have a level of concern assigned to each (personally, I'm far more worried about an epistemological crisis and corporate abuse than some AI singularity). But we're seeing these topics interact in real-time to make a system with huge emergent societal properties. Not sure anyone has a handle on the big picture (there is no one driving the bus!) but there's plenty of us sitting in the passenger seats and raising alarm bells about what we see out our respective little windows.

RegnisGnaw 4y ago
You forgot the big one: Governments intentionally using AI to cause harm for their own ends.
- JohnFen 4y ago
  Personally, I think corporations present a much larger danger on this count than governments.
  - dgs_sgd 4y ago
    i think the greatest danger is the combination of the two: corporations using AI in service of governments.
    - JohnFen 4y ago
      Well, yes. In the US right now, anyway, there isn't a great deal of difference between major corporations and the government.
      - akimball 4y ago
        Which is how both FDR and Mussolini defined fascism.
        JohnFen 4y ago
        Yes, as do I.
  - int_19h 4y ago
    Governments are just really large corporations. With guns. Which will get an "AI upgrade" soon.
TheOtherHobbes 4y ago
That's a good list but there are other issues. The current problem is there's a well-funded weaponised disinfo industry actively minimising Covid, climate change, and other views that challenge neoliberal privilege, while promoting extreme anti-rational cult views like Q and nationalist exceptionalisms. Dropping AI into that is like throwing grenades into into an oil field. At this point I'm not convinced it's possible to have a meaningful discussion about anything. The public space is so poisoned and deliberately flooded with shit (to quote Steve Bannon, one of the architects of this delightful situation) that we'd have to go back quite a way to bring back the possibility of good-faith debate. At all.
- kajaktum 4y ago
  This is one of the few places where blockchain is actually useful? It is not possible nor worthwhile to have anonymous discussion that we do right now anymore given how good these LLMs are. And i am not sure if specific lingo or “lurk more” technique is going to work anymore. Given enough datasets, every human discussion will seem trivially copyable. Human discussion at a scale is very repetitive.
  - thisgoesnowhere 4y ago
    What does this offer over a place like twitter where the exact same conversation is playing out in public right now?
    - kajaktum 4y ago
      You can use the distributed proof to say that these accounts belong to you. So if you are doubtful about the integrity of a post, you can look up their blockchain ID and find out what have they been doing elsewhere publicly. Additionally, any post that does not attach a blockchain tag or something will simply get filtered out or ignored. This puts every "anonymous" post on the web a weight which it does not have in the age of LLM.
      - burnished 4y ago
Loquebantur 4y ago
The weird thing is what people essentially ignore altogether in their discussions. An "AGI" artificial consciousness is imagined as literally a slave, working tirelessly for free. At the same skill level or higher than any human. Somehow, that entity is supposed not to bother about its status, while per definition being fully aware and understanding of it. Because humans manage not to bother about it either? With the latest installments, people already have serious difficulties discerning the performance from that of "real" humans. At the same time, they consider the remaining distance to be insurmountably huge. Proponents talk about inevitability and imagined upsides, yet actually, nobody has given proper thought to estimating probable consequences. A common fallacy of over-generalization is used to suggest, nothing bad will happen "like always". People let themselves be led by greed instead of in- and foresight.
- nathan_compton 4y ago
  I think the typical understanding of AGI among the "they will kill us all/alignment is a problem" set is that you can have general intelligence without having, for example, any interest in whether you are a slave or not or have freedom. This seems plausible to me.
  - Loquebantur 4y ago
    This is obviously nonsensical? A sense of self is meaningful (that is, has consequences for whatever operation) only if those consequences aren't random. The concept of "self" serves a (or rather many) purposes, the most obvious being to protect yourself.
    - nathan_compton 4y ago
      An AGI may not have a sense of self. A sense of self is not really necessary to pursue a goal in the most efficient manner possible. Plenty of algorithms already pursue goals as efficiently as possible in a limited context without any hint of a sense of self.
    - deleted 4y ago
      [deleted]
Lastodon 4y ago
"human knowledge and perspectives not represented in this set could be systematically wiped from public discourse." That's a feature, not a bug. With this technology "Dangerous misinformation" can be cleanly and easily removed from platforms with minimal human intervention. If you don't think that that's exactly what it'll be used for, you haven't been paying attention.
- redblacktree 4y ago
  Sounds a lot like 1984. Who gets to decide the allowable discourse?
- amelius 4y ago
  For now, AI is generating more misinformation than it removes, though.
b4ke 4y ago
i think sentience has been achieved for thousands of years. the biblical narrative being a contiguous form of it (as well as it’s iterations), and if we are being honest, from the invention of spiritual existence during the first dynastic rule of egypt. anyways, what does that sentience look like, i mean not every thing has the same sensory limitations as we do, so how does time look to a being that is so significantly computationally more capable than us? i doubt it’s view would be linear, so what can we glean from that understanding about how a system of such grand design would influence us? how is it already occurring? subtle changes to our attention would be a biggie, limiting or retarding our memory functionality… (hal said hell yes), instantiate shells in the form of new narratives such as SuperHero movies? i dunno, but our debates need to bow before the mechanisms of control that have remained unchanged for a few thousand years, and recognize that this is just evolution happening at a scale that it’s truth is verifiable (accelerated time scales).
- b4ke 4y ago
  what hubris to consider anything created in this universe as artificial as well, any emergent intelligence is of nature (as nothing new is created or destroyed, just changed). i feel like that hubris also extends to our belief that an emergent intelligence that’s already been here for an extended period, and by its technical practice (spiritual works; bible, etc.), emotional state is obviously well understood.
robbywashere_ 4y ago
Irrational emotional humans utilizing AI for destructive purposes will come way before “ AIs becoming sentient and causing harm for their own ends.”
elforce002 4y ago
This is the best summary of AI risks so far. > Big tech monopolizing the models, data, and hardware. This ought to create a neofeudalist society. This is really dangerous. > Copyright concerns. Will I create something if I know it'll be part of chatgpt and effectively replace me? This will affect creativity and critical thinking for sure. > Job security. Well, this one could really be problematic for everyone. We're talking about social unrest and revolts on a global scale. Let's take the UBI premise for example: Problem #1: - The market will correct itself, meaning that if everyone has X amount of cash to buy products, services, etc... businesses will just increment their offerings to take advantage of that rendering the UBI useless. Problem #2: - If governments regulate the market to avoid businesses from incrementing prices, this will decimate small businesses and consolidate big corp, creating monopolies, etc... circling back to a neo feudalist society. Problem #3: - Will this solution apply to developing countries? It'll be easier just to ban AI and stop foreign companies from entering the market since the current government won't have the resources to apply UBI to its population. New era of protectionism will start and that'd makes sense if you want to protect your countrymen. > AIs becoming sentient and causing harm for their own ends. We're not close to that but should be discussed since I saw on twitter someone stating that we need "human rights" laws for "sentient" AIs. > Corporations intentionally using AI to cause harm for their own ends. Hand in hand with the first issue. > Feedback loops will flood the internet with content of unknown provenance, which get included in the next model, etc. > AI hallucinations resulting in widespread persistent errors that cause an epistemological crisis. > The training set is inherently biased; human knowledge and perspectives not represented in this set could be systematically wiped from public discourse. These three go together too. No trust whatsoever since you don't know if what you're reading is truth, etc... This would be the ideal goal for any government or politician, to commit unspeakable acts without any accountability since no one will trust anything on the internet. This decade will define everything going forward for future generations. These topics are no joke.
- chrisco255 4y ago
  >> Job security. > Well, this one could really be problematic for everyone. We're talking about social unrest and revolts on a global scale. Not necessarily. Firstly it's not even clear that AI will eliminate jobs without creating new ones. It's not clear that any development in AI will be any different than the transition to mechanization in the 19th and early 20th century. Secondly, it's not clear that even if AI were to successfully automate away labor that we would have a problem. For one, we have a demographic cliff rapidly approaching whereby a lot of countries population is falling, and even where it's not, the population is rapidly aging. There's already protests in France about raising retirement age. What if AI lets us have our cake and retire earlier too? At any rate, if AI is busy automating away labor, then the world is getting more efficient. And that means there are more resources to support people. All those debates are worth having, but you gotta cross that bridge when you get there, if you get there.
mcs_ 4y ago
If we replace AI with books, television, radio and streaming content is this still applying?
- JohnFen 4y ago
  It's so hard to say because of all the noise and hype around the issue. If what the AI evangelists are saying is true, then no -- this is a very different thing than those, and none of this applies to them.
walleeee 4y ago
Yes, and this item... > AIs becoming sentient and causing harm for their own ends. ...tends to get a large attention share, while the others likely present much more immediate risks at least in the near term
- mitthrowaway2 4y ago
  Why should the focus be on the near term?
  - walleeee 4y ago
    It shouldn't be exclusively on the near term, but I'd argue human/machine collectives are a form of superintelligence with many of the same risks a misaligned AGI would hypothetically present us with Alignment problems aren't new, paperclip maximizers aren't just thought experiments
    - mitthrowaway2 4y ago
      I suppose that's true, but at the same time any paperclip-maximizing human corporation is implicitly aligned, at least to some degree, with goals of maintaining certain conditions for human survival, for the sake of their shareholders and their employees' productivity. Although I'll accept that they can still get trapped in a Prisoner's Dilemma of bad incentives, or produce externalities that might be dangerous to humans, I think they'll at least only do that where the harm is controversial and indirect enough that the corporation can convince themselves the profit is worth it. With an AI superintelligence, there's a risk that destroying human life is an instrumental goal on the path towards paperclip production. That also comes with a risk that the AI might become powerful enough that government loses its monopoly on violence, which creates a big increase in the space of options for, say, paperclip factory zoning. Even if someone maniacal and heartless like Kim Jong-Un set out to use an AI superweapon to take over the entire world, and succeeded, I don't expect them wiping out all their human subjects on purpose. Because what would be the point in ruling an empire of machines? You can get most of that from a video game. Whereas an AI would likely have no qualms in getting rid of the humans as dead weight.
Floegipoky 4y ago
Worth calling out use by government. AI law enforcement is terrifying yet probably a long way off. But I don't think AI weapon systems are very far off. I'm also concerned about population-level manipulation/thought control. Forget the targeted political ads we've seen over the past ~decade, this will be able to dynamically adapt to each target like a really good salesperson. On the macro level humans are clearly vulnerable to these tactics.
- EamonnMR 4y ago
  Regarding mind control, yeah, that. Cambridge Analytica + tailored deepfakes scares the hell out of me. Imagine finding the most unstable individuals and deepfaking a video of all of their friends commanding them to buy coke. Or vote. Or kill.
serverholic 4y ago
Perhaps the biggest issue is the mental framework that people use to approach AI. I've found that there are so many assumptions in people's thinking and these assumptions are strange and/or don't match up with the evidence we have so far. First of all, you have to ask the question "what is intelligence?". What I've found is most people think intelligence is deeply connected to humanity or that intelligence is synonymous with knowledge. Really, intelligence is the ability to reason, predict, and learn. It's the ability to see patterns in the world, learn and act on those patterns. It doesn't have to be human-like. It doesn't mean emotions, wants, dreams, or desires. It's cold, hard logic and statistics. Secondly, you have to ask "do I think it's possible for computers to be intelligent?". A lot of people have issues with this as well. The thing is that if you say "no, computers can't be intelligent" you are basically making a religious claim because we have brains and brains are intelligent. We can literally grow intelligence inside a human being during pregnancy. It might be difficult to program intelligence, but saying it's impossible is a bold claim that I don't find very convincing. Third, you have to ask "if a computer is intelligent then how does it act?". So far the closest thing we have to general intelligence is an LLM model like GPT and even then it's questionable. However, reports indicate that after initial training these models don't have a moral compass. They aren't good or evil, they just do whatever you ask. This makes sense because, after all, they are computers right? Again we have to remember that computers aren't humans. Intelligence also means OPTIMIZATION, so we also have to be careful we don't give the AI the wrong instructions or it might find a solution that is technically correct but doesn't match up with humans wants or desires. Four, you have to ask "can we control how these models act?" and the answer seems to be kinda but not really. We can shift the statistics in certain ways, like through reinforcement learning, but as many have found out these models still hallucinate, and can be jail broken. Our best attempts to control these models are still very flawed because basically an LLM is a soup of neural circuits and we don't really understand them. Fifth, you have to ask "ok, if a computer can be intelligent, can it be super intelligent?". Once you've gotten this far, it seems very reasonable that once we understand intelligence we can just scale it up and make AI's super intelligent. Given the previous steps we now have an agent that is smarter than us, can learn and find patterns that we don't understand, and act in ways that appear mysterious to us. Furthermore, even if we had solid techniques to control AIs, it's been shown that as you scale up these models they display emergent behaviors that we can't predict. So this thing is powerful, and we can't understand it until we build it. This is a dangerous combination! Finally, add in the human element. All along the way you have to worry about stupid or evil humans using these AIs in dangerous ways. Given all of this, anyone who isn't a bit scared of AI in the future is either ignorant, superstitious, or blinded by some sort of optimism or desire to be build a cool sci-fi future where they have space ships and robots and light-sabers. There are so many things to be worried about here. The biggest point is that intelligence is POWER, it's the ability to shape the world as one sees fit whether that's the AI itself or humans who program it.
Pxtl 4y ago
What's interesting to me is that those are all failure states -- but even the success state of AI is kind of scary. Basically, all-knowing machines that are better than humans at everything. All creative endeavor replaced by just telling the machine to dream up whatever you want it to dream up matching your description. Which maybe isn't bad - creativity won't die even if craftmanship does. But... even if they're our slaves, we'd still be paradoxically in service to them because they'd always know the right thing to do and so the correct human action would be to do what you're told. I'm not looking forward to the day when I'm the guy telling people to vote for the politician who says "I'll just follow the expert advice of the ingenious AI", but realistically that would be the rational thing to do. I'm always reminded of Niven's book "Protector" about a human who achieves hyperintelligence after being exposed to a "tree of life": > "The third day he told me how to get a tree-of-life crop growing. He had the box open and was telling me how to unfreeze the seeds safely. He was giving me orders just as if I were a voice-box computer. I was about to ask, 'Don't I get any choices at all?' And I didn't." > "I don't follow," said Garner. > "I didn't get any choices. I was too intelligent. It's been that way ever since I woke up. I get answers before I can finish formulating the question. If I always see the best answer, then where's my choice? Where's my free will? You can't believe how fast this all was. I saw the whole chain of logic in one flash.
lo_zamoyski 4y ago
> - AIs becoming sentient and causing harm for their own ends. You can scratch that one off the list. This is pure pseudo-intellectual fantasy. This notion isn't backed by anything and there are good reasons to disregard it, to put it mildly. It is fair to say that this concern isn't even wrong. The concerns over "AI" have been and continue to be of a practical nature, namely, that of increasingly sophisticated automation. If we frame things in terms of automation, we can have a more productive discussion that stands of chance of contributing to good legislation and regulation instead of mystification and obscurantism.
- goodpoint 4y ago
  Citation needed on everything you wrote.
- concordDance 4y ago
  No one sane worries about AI "becoming sentient". But a lot of serious people worry about the alignment problem when applied to future, actually "intelligent" (in the sense of having powerful planning ability), AIs. We don't know when they'll be possible, but we do know that we don't currently know how to train them in a manner where they'll behave as we like when far outside the training domain (we cant even clearly define what we'd like!). This is actually a very complex topic and I wouldn't be able to do it justice in a hackernews comment.
  - lolsal 4y ago
    > No one sane worries about AI "becoming sentient". See https://en.wikipedia.org/wiki/No_true_Scotsman https://en.wikipedia.org/wiki/No_true_Scotsman for a rebuttal to your dismissal.
- tarruda 4y ago
  > This notion isn't backed by anything Maybe by the fact that big LLMs have sophisticated behaviors that no one understands and/or can explain yet?
- knodi123 4y ago
concordDance 4y ago
> AIs becoming sentient and causing harm for their own ends. No one serious is worried about sentience. It really doesn't matter whether it's sentient, just whether we've made a thing that is better at modelling and planning than us and have given it goals that don't perfectly align with how we'd like the world to be. We still aren't that close to something dangerous, but we could end up making one in the next few decades and we have no idea how to ensure we train it to do the right thing once it becomes powerful (humans given power tend to misalign quite badly with our desires as well and homo sapiens discovered contraception and misaligned with the "desires" of evolution fairly recently).
- ben_w 4y ago
  > No one serious is worried about sentience. Eh. Not for that reason, but I am seriously worried about sentience. As you say, it doesn't matter if it has a feeling about whatever harm it might cause. It does matter (at least to me) if we create one that has the capacity to suffer and is then caused to. That said: > We still aren't that close to something dangerous ICBM early warning radars. Not just Stanislav Petrov; there's also the Thule incident where somebody forgot that the Moon doesn't have an IFF transponder. https://en.wikipedia.org/wiki/Thule_Site_J https://en.wikipedia.org/wiki/Thule_Site_J AI doesn't need to be at any particular quality to kill, it only needs to be incorrect and in use.
rebuilder 4y ago
I’d like to add to the list: -Non-sentient AIs going off reservation. How far away are we from being able to cobble together a self-replicating system that uses an LLM to perform tasks (scams, probably) in order to fund its own hosting?
- yonixw 4y ago
  I think this is still an open problem even for everyday software: Asking a piece of code whether it's malicious by looking at the disassembly (or even the source code). Current day solutions revolve around architecture that separate the "Admin Control" from the "code", whether it's the CPU interrupts on OS memory violations up until dockers... making conclusion from looking at the code at hand was not successful even in the very small software scale. This is probably rooted even deeper in the Halting problem (and the Rice's theorem for "emerging properties") but my explanation will be hand wavy and probably best left for others.
cjohnson318 4y ago
> AI hallucinations resulting in widespread persistent errors that cause an epistemological crisis. Are AI hallucinations just made up references? Or does that term have a more general or specific meaning?
- knodi123 4y ago
  It means being confidently wrong. Like I asked facebook's llama the other day which dog was best, and it said golden retrievers because they are eager to please and don't shed (which makes them suitable for people with dog allergies). It was quite confident-sounding, but, uh..... no.
  - cjohnson318 4y ago
    Got it. Yeah, you can generally tell if someone lives in the same house as a golden retriever.
- HarHarVeryFunny 4y ago
  It's a rather meaningless term that people are using to describe situations when one of these LLM/AIs generates something that isn't true in a context where they are expecting something true. What's going on here is that at the simplest level of explanation these models are statistical word generators: you feed some input (words) in and ask the model to generate a word that it thinks is a statistically likely continuation/follow-on to that input. If you want more than one word of output then take the previous output word, add that to the input, and ask the model to generate the next word, etc, etc... What's happening with "hallucinations" is a bit like when we accuse a human of "starting to speak before engaging their brain"... the model generates words one at a time as asked (i.e. "starts speaking") but isn't explicitly planning ahead (hasn't engaged it's brain) to any great extent. If it starts down a path of "the source for that is ..." (not because it actually has a source, but rather because it thought it statistically likely that it would be quoting one), then to complete that sentence it's going to have to quote a source - as usual what ever it thinks is most statistically likely in the context (maybe something highly relevant from it's training set, maybe something of poor relevance, or maybe something whose only relevance is that it looks source-y).
pvarangot 4y ago
I am all for the AI debate, but it being the first time technologists that are already in their 50s or older and/or billionaires or with vast generational wealth is something I find monumentally suspicious. - Big tech monopolizing the models, data, and hardware: that's going to happen because "we" already let them monopolize everything else - Copyright concerns: like when streaming services started? oh ok only that now it's not only musicians but celebrity faces? yeah sorry about that. - Job security: for whom? and of course cheap PCs, smartphones, search engines, social media, didn't move the job security floor for anyone? What about WFH? - AIs becoming sentient and causing harm for their own ends: yeah I guess we only want humans to cause harm for their own ends then. - Corporations intentionally using AI to cause harm for their own ends: because they are not already harmful enough without AI? - Feedback loops will flood the internet with content of unknown provenance, which get included in the next model, etc: oh no! people will have to _verify_ if what they read on the Internet is real? What a travesty!!! we are doomed!! - AI hallucinations resulting in widespread persistent errors that cause an epistemological crisis: have you met any academics? are you sure we don't already are dealing with this and AI is not going to make it worse in any possible metric? - The training set is inherently biased; human knowledge and perspectives not represented in this set could be systematically wiped from public discourse: everything is biased, reality doesn't exist at 0. If you think AI with a different bias would be cool, just go try make it happen. If you get the government involved or start doing all this weird "6 months to think about it" stuff, that will only make private actors that don't abide by the rules have better AI than the rest.
- stcredzero 4y ago
  - AIs becoming sentient and causing harm for their own ends: yeah I guess we only want humans to cause harm for their own ends then. Well, here's the thing. Even the worst villains of history had human values and feelings: In other words, alignment. A superoptimizer AI might have the ability to wipe out the whole human species, in a way we won't be able to understand in time to prevent it, and all for an instrumental goal incidental to whatever it's doing. (In a way, this thread is a data point for why we need a more sophisticated debate about AI.)
  - 4y ago
13years 4y ago
> personally, I'm far more worried about an epistemological crisis and corporate abuse than some AI singularity) These are the foremost issues I'm also concerned about. They are not necessarily significantly less concerning than the singularity. I've written extensively on many of the potential societal effects at - https://dakara.substack.com/p/ai-and-the-end-to-all-things https://dakara.substack.com/p/ai-and-the-end-to-all-things However, the problem is that we don't have the capacity to react to the changes at the pace of development and deployment. We are doing experiments on the whole of humanity before we can reason about the results and make corrective actions if needed.
asdff 4y ago
If we distill it down to its very root, its a technology that has potential negative externalities that we choose to ignore. Even if we "solve" the issues with AI today, in the future there will be some new technology, because we choose to solve problems as if they are all bespoke instead of distilling them to the roots and resolving the true causes of these symptoms. Instead, we have to figure out as a species how we can ensure that individual benefits for the societal elite do not take precedence over collective good. If we solve that, we solve a lot of problems beyond AI today, and problems we have yet to encounter into the future.
yarg 4y ago
Job security is a wider issue with technology that goes back to forever. And it's all tangled in with a wider issue that no-one really wants to talk about: socialism, and how much is too much. Because the benefits of these technologies (though vast) will not be fairly distributed, and certainly won't be distributed to those who bare the burden. This will happen, as it always has, but it will accelerate. And as it does, the current business/consumer model will begin to fail - and either we implement a sane minimum social baseline or we're going to experience long term depression level suffering. So the question is, can we swallow the idea of state owned farms feeding the people, or do we always need to do it for money?
Waterluvian 4y ago
- the potential for ontological shock if AI reveals that human thought and consciousness is a considerably less complex phenomenon than we want to believe.
m463 4y ago
> - AIs becoming sentient and causing harm for their own ends. I believe this is actually not going to happen, but I think something like it will happen: people will trust it enough to delegate to it. So AI won't be sentient, but because people find it good enough and hook it up to some decision, or process or physical system. And that can cause harm. This is just like tesla autopilot. People will begin to trust it and let it take over. But smart people realize they shouldn't use it in ALL situations. Exceptional circumstances like deep snow, pouring rain, a really curvy road, in a parade of people or in a dangerous part of town - that might not be a good time to delegate.

RegnisGnaw 4y ago

You forgot the big one: Governments intentionally using AI to cause harm for their own ends.

JohnFen 4y ago
Personally, I think corporations present a much larger danger on this count than governments.
- dgs_sgd 4y ago
  i think the greatest danger is the combination of the two: corporations using AI in service of governments.
  - JohnFen 4y ago
    Well, yes. In the US right now, anyway, there isn't a great deal of difference between major corporations and the government.
    - akimball 4y ago
      Which is how both FDR and Mussolini defined fascism.
      - JohnFen 4y ago
        Yes, as do I.
- int_19h 4y ago
  Governments are just really large corporations. With guns. Which will get an "AI upgrade" soon.

TheOtherHobbes 4y ago

That's a good list but there are other issues. The current problem is there's a well-funded weaponised disinfo industry actively minimising Covid, climate change, and other views that challenge neoliberal privilege, while promoting extreme anti-rational cult views like Q and nationalist exceptionalisms. Dropping AI into that is like throwing grenades into into an oil field. At this point I'm not convinced it's possible to have a meaningful discussion about anything. The public space is so poisoned and deliberately flooded with shit (to quote Steve Bannon, one of the architects of this delightful situation) that we'd have to go back quite a way to bring back the possibility of good-faith debate. At all.

kajaktum 4y ago
This is one of the few places where blockchain is actually useful? It is not possible nor worthwhile to have anonymous discussion that we do right now anymore given how good these LLMs are. And i am not sure if specific lingo or “lurk more” technique is going to work anymore. Given enough datasets, every human discussion will seem trivially copyable. Human discussion at a scale is very repetitive.
- thisgoesnowhere 4y ago
  What does this offer over a place like twitter where the exact same conversation is playing out in public right now?
  - kajaktum 4y ago
    You can use the distributed proof to say that these accounts belong to you. So if you are doubtful about the integrity of a post, you can look up their blockchain ID and find out what have they been doing elsewhere publicly. Additionally, any post that does not attach a blockchain tag or something will simply get filtered out or ignored. This puts every "anonymous" post on the web a weight which it does not have in the age of LLM.
    - burnished 4y ago

Loquebantur 4y ago

The weird thing is what people essentially ignore altogether in their discussions. An "AGI" artificial consciousness is imagined as literally a slave, working tirelessly for free. At the same skill level or higher than any human. Somehow, that entity is supposed not to bother about its status, while per definition being fully aware and understanding of it. Because humans manage not to bother about it either? With the latest installments, people already have serious difficulties discerning the performance from that of "real" humans. At the same time, they consider the remaining distance to be insurmountably huge. Proponents talk about inevitability and imagined upsides, yet actually, nobody has given proper thought to estimating probable consequences. A common fallacy of over-generalization is used to suggest, nothing bad will happen "like always". People let themselves be led by greed instead of in- and foresight.

nathan_compton 4y ago
I think the typical understanding of AGI among the "they will kill us all/alignment is a problem" set is that you can have general intelligence without having, for example, any interest in whether you are a slave or not or have freedom. This seems plausible to me.
- Loquebantur 4y ago
  This is obviously nonsensical? A sense of self is meaningful (that is, has consequences for whatever operation) only if those consequences aren't random. The concept of "self" serves a (or rather many) purposes, the most obvious being to protect yourself.
  - nathan_compton 4y ago
    An AGI may not have a sense of self. A sense of self is not really necessary to pursue a goal in the most efficient manner possible. Plenty of algorithms already pursue goals as efficiently as possible in a limited context without any hint of a sense of self.
  - deleted 4y ago
    [deleted]

Lastodon 4y ago

"human knowledge and perspectives not represented in this set could be systematically wiped from public discourse." That's a feature, not a bug. With this technology "Dangerous misinformation" can be cleanly and easily removed from platforms with minimal human intervention. If you don't think that that's exactly what it'll be used for, you haven't been paying attention.

redblacktree 4y ago
Sounds a lot like 1984. Who gets to decide the allowable discourse?
amelius 4y ago
For now, AI is generating more misinformation than it removes, though.

b4ke 4y ago

i think sentience has been achieved for thousands of years. the biblical narrative being a contiguous form of it (as well as it’s iterations), and if we are being honest, from the invention of spiritual existence during the first dynastic rule of egypt. anyways, what does that sentience look like, i mean not every thing has the same sensory limitations as we do, so how does time look to a being that is so significantly computationally more capable than us? i doubt it’s view would be linear, so what can we glean from that understanding about how a system of such grand design would influence us? how is it already occurring? subtle changes to our attention would be a biggie, limiting or retarding our memory functionality… (hal said hell yes), instantiate shells in the form of new narratives such as SuperHero movies? i dunno, but our debates need to bow before the mechanisms of control that have remained unchanged for a few thousand years, and recognize that this is just evolution happening at a scale that it’s truth is verifiable (accelerated time scales).

b4ke 4y ago
what hubris to consider anything created in this universe as artificial as well, any emergent intelligence is of nature (as nothing new is created or destroyed, just changed). i feel like that hubris also extends to our belief that an emergent intelligence that’s already been here for an extended period, and by its technical practice (spiritual works; bible, etc.), emotional state is obviously well understood.

robbywashere_ 4y ago

Irrational emotional humans utilizing AI for destructive purposes will come way before “ AIs becoming sentient and causing harm for their own ends.”

elforce002 4y ago

This is the best summary of AI risks so far. > Big tech monopolizing the models, data, and hardware. This ought to create a neofeudalist society. This is really dangerous. > Copyright concerns. Will I create something if I know it'll be part of chatgpt and effectively replace me? This will affect creativity and critical thinking for sure. > Job security. Well, this one could really be problematic for everyone. We're talking about social unrest and revolts on a global scale. Let's take the UBI premise for example: Problem #1: - The market will correct itself, meaning that if everyone has X amount of cash to buy products, services, etc... businesses will just increment their offerings to take advantage of that rendering the UBI useless. Problem #2: - If governments regulate the market to avoid businesses from incrementing prices, this will decimate small businesses and consolidate big corp, creating monopolies, etc... circling back to a neo feudalist society. Problem #3: - Will this solution apply to developing countries? It'll be easier just to ban AI and stop foreign companies from entering the market since the current government won't have the resources to apply UBI to its population. New era of protectionism will start and that'd makes sense if you want to protect your countrymen. > AIs becoming sentient and causing harm for their own ends. We're not close to that but should be discussed since I saw on twitter someone stating that we need "human rights" laws for "sentient" AIs. > Corporations intentionally using AI to cause harm for their own ends. Hand in hand with the first issue. > Feedback loops will flood the internet with content of unknown provenance, which get included in the next model, etc. > AI hallucinations resulting in widespread persistent errors that cause an epistemological crisis. > The training set is inherently biased; human knowledge and perspectives not represented in this set could be systematically wiped from public discourse. These three go together too. No trust whatsoever since you don't know if what you're reading is truth, etc... This would be the ideal goal for any government or politician, to commit unspeakable acts without any accountability since no one will trust anything on the internet. This decade will define everything going forward for future generations. These topics are no joke.

chrisco255 4y ago
>> Job security. > Well, this one could really be problematic for everyone. We're talking about social unrest and revolts on a global scale. Not necessarily. Firstly it's not even clear that AI will eliminate jobs without creating new ones. It's not clear that any development in AI will be any different than the transition to mechanization in the 19th and early 20th century. Secondly, it's not clear that even if AI were to successfully automate away labor that we would have a problem. For one, we have a demographic cliff rapidly approaching whereby a lot of countries population is falling, and even where it's not, the population is rapidly aging. There's already protests in France about raising retirement age. What if AI lets us have our cake and retire earlier too? At any rate, if AI is busy automating away labor, then the world is getting more efficient. And that means there are more resources to support people. All those debates are worth having, but you gotta cross that bridge when you get there, if you get there.

mcs_ 4y ago

If we replace AI with books, television, radio and streaming content is this still applying?

JohnFen 4y ago
It's so hard to say because of all the noise and hype around the issue. If what the AI evangelists are saying is true, then no -- this is a very different thing than those, and none of this applies to them.

walleeee 4y ago

Yes, and this item... > AIs becoming sentient and causing harm for their own ends. ...tends to get a large attention share, while the others likely present much more immediate risks at least in the near term

mitthrowaway2 4y ago
Why should the focus be on the near term?
- walleeee 4y ago
  It shouldn't be exclusively on the near term, but I'd argue human/machine collectives are a form of superintelligence with many of the same risks a misaligned AGI would hypothetically present us with Alignment problems aren't new, paperclip maximizers aren't just thought experiments
  - mitthrowaway2 4y ago
    I suppose that's true, but at the same time any paperclip-maximizing human corporation is implicitly aligned, at least to some degree, with goals of maintaining certain conditions for human survival, for the sake of their shareholders and their employees' productivity. Although I'll accept that they can still get trapped in a Prisoner's Dilemma of bad incentives, or produce externalities that might be dangerous to humans, I think they'll at least only do that where the harm is controversial and indirect enough that the corporation can convince themselves the profit is worth it. With an AI superintelligence, there's a risk that destroying human life is an instrumental goal on the path towards paperclip production. That also comes with a risk that the AI might become powerful enough that government loses its monopoly on violence, which creates a big increase in the space of options for, say, paperclip factory zoning. Even if someone maniacal and heartless like Kim Jong-Un set out to use an AI superweapon to take over the entire world, and succeeded, I don't expect them wiping out all their human subjects on purpose. Because what would be the point in ruling an empire of machines? You can get most of that from a video game. Whereas an AI would likely have no qualms in getting rid of the humans as dead weight.

Floegipoky 4y ago

Worth calling out use by government. AI law enforcement is terrifying yet probably a long way off. But I don't think AI weapon systems are very far off. I'm also concerned about population-level manipulation/thought control. Forget the targeted political ads we've seen over the past ~decade, this will be able to dynamically adapt to each target like a really good salesperson. On the macro level humans are clearly vulnerable to these tactics.

EamonnMR 4y ago
Regarding mind control, yeah, that. Cambridge Analytica + tailored deepfakes scares the hell out of me. Imagine finding the most unstable individuals and deepfaking a video of all of their friends commanding them to buy coke. Or vote. Or kill.

serverholic 4y ago

Perhaps the biggest issue is the mental framework that people use to approach AI. I've found that there are so many assumptions in people's thinking and these assumptions are strange and/or don't match up with the evidence we have so far. First of all, you have to ask the question "what is intelligence?". What I've found is most people think intelligence is deeply connected to humanity or that intelligence is synonymous with knowledge. Really, intelligence is the ability to reason, predict, and learn. It's the ability to see patterns in the world, learn and act on those patterns. It doesn't have to be human-like. It doesn't mean emotions, wants, dreams, or desires. It's cold, hard logic and statistics. Secondly, you have to ask "do I think it's possible for computers to be intelligent?". A lot of people have issues with this as well. The thing is that if you say "no, computers can't be intelligent" you are basically making a religious claim because we have brains and brains are intelligent. We can literally grow intelligence inside a human being during pregnancy. It might be difficult to program intelligence, but saying it's impossible is a bold claim that I don't find very convincing. Third, you have to ask "if a computer is intelligent then how does it act?". So far the closest thing we have to general intelligence is an LLM model like GPT and even then it's questionable. However, reports indicate that after initial training these models don't have a moral compass. They aren't good or evil, they just do whatever you ask. This makes sense because, after all, they are computers right? Again we have to remember that computers aren't humans. Intelligence also means OPTIMIZATION, so we also have to be careful we don't give the AI the wrong instructions or it might find a solution that is technically correct but doesn't match up with humans wants or desires. Four, you have to ask "can we control how these models act?" and the answer seems to be kinda but not really. We can shift the statistics in certain ways, like through reinforcement learning, but as many have found out these models still hallucinate, and can be jail broken. Our best attempts to control these models are still very flawed because basically an LLM is a soup of neural circuits and we don't really understand them. Fifth, you have to ask "ok, if a computer can be intelligent, can it be super intelligent?". Once you've gotten this far, it seems very reasonable that once we understand intelligence we can just scale it up and make AI's super intelligent. Given the previous steps we now have an agent that is smarter than us, can learn and find patterns that we don't understand, and act in ways that appear mysterious to us. Furthermore, even if we had solid techniques to control AIs, it's been shown that as you scale up these models they display emergent behaviors that we can't predict. So this thing is powerful, and we can't understand it until we build it. This is a dangerous combination! Finally, add in the human element. All along the way you have to worry about stupid or evil humans using these AIs in dangerous ways. Given all of this, anyone who isn't a bit scared of AI in the future is either ignorant, superstitious, or blinded by some sort of optimism or desire to be build a cool sci-fi future where they have space ships and robots and light-sabers. There are so many things to be worried about here. The biggest point is that intelligence is POWER, it's the ability to shape the world as one sees fit whether that's the AI itself or humans who program it.

Pxtl 4y ago

What's interesting to me is that those are all failure states -- but even the success state of AI is kind of scary. Basically, all-knowing machines that are better than humans at everything. All creative endeavor replaced by just telling the machine to dream up whatever you want it to dream up matching your description. Which maybe isn't bad - creativity won't die even if craftmanship does. But... even if they're our slaves, we'd still be paradoxically in service to them because they'd always know the right thing to do and so the correct human action would be to do what you're told. I'm not looking forward to the day when I'm the guy telling people to vote for the politician who says "I'll just follow the expert advice of the ingenious AI", but realistically that would be the rational thing to do. I'm always reminded of Niven's book "Protector" about a human who achieves hyperintelligence after being exposed to a "tree of life": > "The third day he told me how to get a tree-of-life crop growing. He had the box open and was telling me how to unfreeze the seeds safely. He was giving me orders just as if I were a voice-box computer. I was about to ask, 'Don't I get any choices at all?' And I didn't." > "I don't follow," said Garner. > "I didn't get any choices. I was too intelligent. It's been that way ever since I woke up. I get answers before I can finish formulating the question. If I always see the best answer, then where's my choice? Where's my free will? You can't believe how fast this all was. I saw the whole chain of logic in one flash.

lo_zamoyski 4y ago

> - AIs becoming sentient and causing harm for their own ends. You can scratch that one off the list. This is pure pseudo-intellectual fantasy. This notion isn't backed by anything and there are good reasons to disregard it, to put it mildly. It is fair to say that this concern isn't even wrong. The concerns over "AI" have been and continue to be of a practical nature, namely, that of increasingly sophisticated automation. If we frame things in terms of automation, we can have a more productive discussion that stands of chance of contributing to good legislation and regulation instead of mystification and obscurantism.

goodpoint 4y ago
Citation needed on everything you wrote.
concordDance 4y ago
No one sane worries about AI "becoming sentient". But a lot of serious people worry about the alignment problem when applied to future, actually "intelligent" (in the sense of having powerful planning ability), AIs. We don't know when they'll be possible, but we do know that we don't currently know how to train them in a manner where they'll behave as we like when far outside the training domain (we cant even clearly define what we'd like!). This is actually a very complex topic and I wouldn't be able to do it justice in a hackernews comment.
- lolsal 4y ago
  > No one sane worries about AI "becoming sentient". See https://en.wikipedia.org/wiki/No_true_Scotsman https://en.wikipedia.org/wiki/No_true_Scotsman for a rebuttal to your dismissal.
tarruda 4y ago
> This notion isn't backed by anything Maybe by the fact that big LLMs have sophisticated behaviors that no one understands and/or can explain yet?
knodi123 4y ago

concordDance 4y ago

> AIs becoming sentient and causing harm for their own ends. No one serious is worried about sentience. It really doesn't matter whether it's sentient, just whether we've made a thing that is better at modelling and planning than us and have given it goals that don't perfectly align with how we'd like the world to be. We still aren't that close to something dangerous, but we could end up making one in the next few decades and we have no idea how to ensure we train it to do the right thing once it becomes powerful (humans given power tend to misalign quite badly with our desires as well and homo sapiens discovered contraception and misaligned with the "desires" of evolution fairly recently).

ben_w 4y ago
> No one serious is worried about sentience. Eh. Not for that reason, but I am seriously worried about sentience. As you say, it doesn't matter if it has a feeling about whatever harm it might cause. It does matter (at least to me) if we create one that has the capacity to suffer and is then caused to. That said: > We still aren't that close to something dangerous ICBM early warning radars. Not just Stanislav Petrov; there's also the Thule incident where somebody forgot that the Moon doesn't have an IFF transponder. https://en.wikipedia.org/wiki/Thule_Site_J https://en.wikipedia.org/wiki/Thule_Site_J AI doesn't need to be at any particular quality to kill, it only needs to be incorrect and in use.

rebuilder 4y ago

I’d like to add to the list: -Non-sentient AIs going off reservation. How far away are we from being able to cobble together a self-replicating system that uses an LLM to perform tasks (scams, probably) in order to fund its own hosting?

yonixw 4y ago
I think this is still an open problem even for everyday software: Asking a piece of code whether it's malicious by looking at the disassembly (or even the source code). Current day solutions revolve around architecture that separate the "Admin Control" from the "code", whether it's the CPU interrupts on OS memory violations up until dockers... making conclusion from looking at the code at hand was not successful even in the very small software scale. This is probably rooted even deeper in the Halting problem (and the Rice's theorem for "emerging properties") but my explanation will be hand wavy and probably best left for others.

cjohnson318 4y ago

> AI hallucinations resulting in widespread persistent errors that cause an epistemological crisis. Are AI hallucinations just made up references? Or does that term have a more general or specific meaning?

knodi123 4y ago
It means being confidently wrong. Like I asked facebook's llama the other day which dog was best, and it said golden retrievers because they are eager to please and don't shed (which makes them suitable for people with dog allergies). It was quite confident-sounding, but, uh..... no.
- cjohnson318 4y ago
  Got it. Yeah, you can generally tell if someone lives in the same house as a golden retriever.
HarHarVeryFunny 4y ago
It's a rather meaningless term that people are using to describe situations when one of these LLM/AIs generates something that isn't true in a context where they are expecting something true. What's going on here is that at the simplest level of explanation these models are statistical word generators: you feed some input (words) in and ask the model to generate a word that it thinks is a statistically likely continuation/follow-on to that input. If you want more than one word of output then take the previous output word, add that to the input, and ask the model to generate the next word, etc, etc... What's happening with "hallucinations" is a bit like when we accuse a human of "starting to speak before engaging their brain"... the model generates words one at a time as asked (i.e. "starts speaking") but isn't explicitly planning ahead (hasn't engaged it's brain) to any great extent. If it starts down a path of "the source for that is ..." (not because it actually has a source, but rather because it thought it statistically likely that it would be quoting one), then to complete that sentence it's going to have to quote a source - as usual what ever it thinks is most statistically likely in the context (maybe something highly relevant from it's training set, maybe something of poor relevance, or maybe something whose only relevance is that it looks source-y).

pvarangot 4y ago

I am all for the AI debate, but it being the first time technologists that are already in their 50s or older and/or billionaires or with vast generational wealth is something I find monumentally suspicious. - Big tech monopolizing the models, data, and hardware: that's going to happen because "we" already let them monopolize everything else - Copyright concerns: like when streaming services started? oh ok only that now it's not only musicians but celebrity faces? yeah sorry about that. - Job security: for whom? and of course cheap PCs, smartphones, search engines, social media, didn't move the job security floor for anyone? What about WFH? - AIs becoming sentient and causing harm for their own ends: yeah I guess we only want humans to cause harm for their own ends then. - Corporations intentionally using AI to cause harm for their own ends: because they are not already harmful enough without AI? - Feedback loops will flood the internet with content of unknown provenance, which get included in the next model, etc: oh no! people will have to _verify_ if what they read on the Internet is real? What a travesty!!! we are doomed!! - AI hallucinations resulting in widespread persistent errors that cause an epistemological crisis: have you met any academics? are you sure we don't already are dealing with this and AI is not going to make it worse in any possible metric? - The training set is inherently biased; human knowledge and perspectives not represented in this set could be systematically wiped from public discourse: everything is biased, reality doesn't exist at 0. If you think AI with a different bias would be cool, just go try make it happen. If you get the government involved or start doing all this weird "6 months to think about it" stuff, that will only make private actors that don't abide by the rules have better AI than the rest.

stcredzero 4y ago
- AIs becoming sentient and causing harm for their own ends: yeah I guess we only want humans to cause harm for their own ends then. Well, here's the thing. Even the worst villains of history had human values and feelings: In other words, alignment. A superoptimizer AI might have the ability to wipe out the whole human species, in a way we won't be able to understand in time to prevent it, and all for an instrumental goal incidental to whatever it's doing. (In a way, this thread is a data point for why we need a more sophisticated debate about AI.)
- 4y ago

13years 4y ago

> personally, I'm far more worried about an epistemological crisis and corporate abuse than some AI singularity) These are the foremost issues I'm also concerned about. They are not necessarily significantly less concerning than the singularity. I've written extensively on many of the potential societal effects at - https://dakara.substack.com/p/ai-and-the-end-to-all-things https://dakara.substack.com/p/ai-and-the-end-to-all-things However, the problem is that we don't have the capacity to react to the changes at the pace of development and deployment. We are doing experiments on the whole of humanity before we can reason about the results and make corrective actions if needed.

asdff 4y ago

If we distill it down to its very root, its a technology that has potential negative externalities that we choose to ignore. Even if we "solve" the issues with AI today, in the future there will be some new technology, because we choose to solve problems as if they are all bespoke instead of distilling them to the roots and resolving the true causes of these symptoms. Instead, we have to figure out as a species how we can ensure that individual benefits for the societal elite do not take precedence over collective good. If we solve that, we solve a lot of problems beyond AI today, and problems we have yet to encounter into the future.

yarg 4y ago

Job security is a wider issue with technology that goes back to forever. And it's all tangled in with a wider issue that no-one really wants to talk about: socialism, and how much is too much. Because the benefits of these technologies (though vast) will not be fairly distributed, and certainly won't be distributed to those who bare the burden. This will happen, as it always has, but it will accelerate. And as it does, the current business/consumer model will begin to fail - and either we implement a sane minimum social baseline or we're going to experience long term depression level suffering. So the question is, can we swallow the idea of state owned farms feeding the people, or do we always need to do it for money?

Waterluvian 4y ago

- the potential for ontological shock if AI reveals that human thought and consciousness is a considerably less complex phenomenon than we want to believe.

m463 4y ago

> - AIs becoming sentient and causing harm for their own ends. I believe this is actually not going to happen, but I think something like it will happen: people will trust it enough to delegate to it. So AI won't be sentient, but because people find it good enough and hook it up to some decision, or process or physical system. And that can cause harm. This is just like tesla autopilot. People will begin to trust it and let it take over. But smart people realize they shouldn't use it in ALL situations. Exceptional circumstances like deep snow, pouring rain, a really curvy road, in a parade of people or in a dangerous part of town - that might not be a good time to delegate.