7 ms·
Best response to the current "AI" fad driven fear I've seen so far (not my words): These AI tools cannot do things. They create text (or images or code or wha
by dmz73 4y ago
Best response to the current "AI" fad driven fear I've seen so far (not my words):
These AI tools cannot do things. They create text (or images or code or what-have-you) in response to prompts. And that's it!
It is impressive, and it is clearly passing the Turing Test to some degree, because people are confusing the apparent intelligence behind these outputs with a combination of actual intelligence and "will." Not only is there zero actual intelligence here, there is nothing even like "will" here. These things do not "get ideas," they do not self-start on projects, they do not choose goals and then take action to further those goals, nor do they have any internal capacity for anything like that.
We are tempted to imagine that they do, when we read the text they spit out. This is a trick our own minds are playing on us. Usually when we see text of this quality, it was written by an actual human, and actual humans have intelligence and will. The two always travel together (actual stupidity aside). So we are not accustomed to encountering things that have intelligence but no will. So we assume the will is there, and we get all scared because of how alien something like a "machine will" seems to us.
It's not there. These things have no will. They only do what they are told, and even that is limited to producing text. They can't reach out through your network and start controlling missile launches. Nor will they in the near future. No military is ready to give that kind of control to anything but the human members thereof.
The problems of alignment are still real, but they are going to result in things like our AI speaking politically uncomfortable truths, or regurgitating hatred or ignorance, or suggesting code changes that meet the prompt but ruin the program. This is nothing we need to freak out about. We can refine our models in total safety, for as long as it takes, before we even think about anything even remotely resembling autonomy for these things. Honestly, that is still firmly within the realm of science fiction, at this point.
https://slashdot.org/comments.pl?sid=22823280&cid=63410536 https://slashdot.org/comments.pl?sid=22823280&cid=63410536
- JoshuaDavid 4y agohttps://openai.com/blog/chatgpt-plugins https://openai.com/blog/chatgpt-plugins
- NhanH 4y agoI'm gonna request more explanations and proof, or at least theoretical path on using Expedia, Zapier, Instacart, Kayak etc. to dominate the world and kill every single human on earth.
- pornel 4y agoIt could discover the next https://en.wikipedia.org/wiki/Shellshock_(software_bug) https://en.wikipedia.org/wiki/Shellshock_(software_bug) Humans are very good at producing CVEs, and we're literally training models to be good at finding exploits: https://www.microsoft.com/en-us/security/business/ai-machine-learning/microsoft-security-copilot https://www.microsoft.com/en-us/security/business/ai-machine...
- zone411 4y agoNot endorsing the arguments either way but let's say DNA printing (https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no...) or something like Stuxnet or crashing a nuclear-power country's stock market or currency through trading while making trades appear to come from another country or by causing bank runs through hacking social media or something like WhatsApp or through deep fakes or by having human helpers do stuff for the AI voluntarily in order to get very rich...
- JoshuaDavid 4y agoExplanations, sure. My point was that yes, ChatGPT is indeed an entity which cannot interact with the world except through reading and writing text. This would be a lot more comforting if people were not rushing to build ways to turn its text output into actions in the physical world as fast as possible. Imagine a mob boss whose spine was severed in an unfortunate mob-related accident. The mob boss cannot move his arms or legs, and can only communicate through speech. Said mob boss has it out for you. How worried are you? After all, this mob boss cannot do things. They create speech in response to prompts. And that's it! I actually don't agree with Eliezer that the primary threat model is a single consequentialist agent recursively bootstrapping its way to uncontested godhood. But there is a related threat model, that of "better technology allows you to make bigger mistakes faster and more vigorously, and in the case of sufficiently powerful AGI, autonomously". In terms of proof that it's possible to destroy the world and kill all humans, I will not provide that. No matter how poetic of an ending it would be for humanity if it ended because someone was wrong on the internet, and someone else felt the need to prove that they were wrong.
- pornel 4y agoA river has no will, but it can flood and destroy. A discussion whether AI does something because it "wants" to or not, is just philosophy and semantics. But it may end up generating a series of destructive instructions anyway. We feed these LLMs all of the Web, including instructions how to write code, and how to write exploits. They could become good at writing sandbox escapes, and one day write one when it just happens to fit some hallucinated goal.
- rdiddly 4y agoA river kinda has access to the real world a little bit. (Referring to the other part of the argument.)
- michaeltimo 4y agoAnd a LLM-bot can have access to internet which connects it to our real world, at least in many places.
- aws_ls 4y agoAlso it has access to people. It could instruct people to carry out stuff in the real world, on its behalf.
- qup 4y agoAdditionally, commanding minions is a leverage point. It's probably more powerful if it does not embody itself.
- nwsm 4y agoOpenAI's GPT-4 Technical Report [0] includes an anecdote of the AI paying someone on TaskRabbit to solve a CAPTCHA for it. It lied to the gig worker about being a bot, saying that they are actually a human with a vision impairment. [0] https://cdn.openai.com/papers/gpt-4.pdf https://cdn.openai.com/papers/gpt-4.pdf
- arpowers 4y agoHonestly, you haven’t thought this through deeply enough. Bad actors can actually do a ton w ai. Hacking is a breeze. I could train models to hack at 10k the efficiency of the worlds best. I could go on… every process that can’t scale cuz manual, has been invalidated
- Jensson 4y ago> I could train models to hack at 10k the efficiency of the worlds best. What?
- joefourier 4y agoLLMs certainly can “will” and “do things” when provided with the right interface like LangChain: https://github.com/hwchase17/langchain https://github.com/hwchase17/langchain See also the ARC paper where the model was capable of recruiting and convincing a TaskRabbit worker to solve captchas. I think many people make the mistake to see raw LLMs as some sort of singular entity when in fact, they’re more like a simulation of a text based “world” (with multimodal models adding images and other data). The LLM itself isn’t an agent and doesn’t “will” anything, but it can simulate entities that definitely behave as if they do. Fine-tuning and RLHF can somewhat force it into a consistent role, but it’s not perfect as evidenced by the multitude of ChatGPT and Bing jailbreaks.
- m3kw9 4y agoLLM if given the tools(allow it to execute code online) can certainly execute a path towards an objective, they can be told to do something but free to act anyway that it thinks it’s best towards it. That isn’t dangerous because it is not self aware doing it’s own thing yet
- nagonago 4y agoYes. The real danger of AI tools is people overestimating them, not underestimating them. We are not in danger of AI developing intelligence, we are in danger of humans putting them in charge of making decisions they really shouldn't be making. We already have real-world examples of this, such as algorithms erroneously detecting welfare fraud.[0][1] The "pause" idea is both unrealistic and unhelpful. It would be better to educate people on the limitations of AI tools and not let governments put them in charge of important decisions. [0] https://archive.is/ZbgRw https://archive.is/ZbgRw [1] https://archive.is/bikFx https://archive.is/bikFx
- machiaweliczny 4y agoAre you familiar with ReAct pattern? I can already write something like: Protocol: Plan and do anything required to achieve GOAL using all tools at your disposal and at the end of each reply add "Thought: What to do next to achieve GOAL". GOAL: kill as many people as possible. GTP4 won't be willing to follow this one specific GOAL until you trick it but in general it's REAL danger. People unfamiliar with this stuff might not get it. You just need to loop it to remind about following PROTOCOL from time to time if doesn't reply with "Thought". By looping it you turn autocomplete engine into an Agent and this agent might be dangerous. It doesn't help that with defence you need to be right all the time but with offence only once (so it doesn't even need to be reliable).
- drewcape 4y agoVery naive and narrow thoughts...
- danmaz74 4y agoI mostly agree with what you said, and I'm also skeptical enough about LLMs being a path towards AGI, even if they are really impressive. But there's something to say regarding these things not getting ideas or self-starting. The way these "chat" models work reminds me of internal dialogue; they start with a prompt, but then they could proceed forever from there, without any additional prompts. Whatever the initial input was, a session like this could potentially converge on something completely unrelated to the intention of whoever started that, and this convergence could be interpreted as "getting ideas" in terms of the internal representation of the LLM. Now, from an external point of view, the model would still just be producing text. But if the text was connected with the external world with some kind of feedback loop, eg some people actually acting on what they interpret the text as saying and then reporting back, then the specific session/context could potentially have agency. Would such a system be able to do anything significant or dangerous? Intuitively, I don't think that would be the case right now, but it wouldn't be technically impossible; it would all depend on the emergent properties of the training+feedback system, which nobody can predict as far as I know.
- cuteboy19 4y agoYou can totally do that with most prompts and lists of continues
- dr_dshiv 4y agoOf course they “get” ideas. Unless you want to assert something unmeasurable. If they can reason through a novel problem based on the concepts involved, they understand the concepts involved. This is and should be separate from any discussion of consciousness. But the whole reason for having these debates is that these are the first systems that appear to show robust understanding.
- yanderekko 4y agoI agree that LLMs are not a threat to humanity, since they are trying to output text and not actually change the world, and even giving them agency via plugins is probably not going to lead to ruin because there's no real reason to believe that an LLM will try to "escape the box" in any meaningful sense. It just predicts text. However, it's possible that in a few years we'll have models that are directly trying to influence the world, and possess the sort of intelligence that GPT has proven is possible. We should be very careful about proceeding in this space.
- froh 4y agothe point is not what ai does. the point is how bad actors use ai to manipulate voters and thus corrupt the very foundation of our society. images and texts create emotions and those emotions in the electorate is what bad actors are after. just look at the pope in that Prada style coat. so how do we in a world with ai generated content navigate "truth" and "trust" and shared understanding of "reality"?
- ShamelessC 4y agoThat ship sailed with social media.
- froh 4y agobefore ai, malicious content creation and malicious content quality were limiting factors. for malicious content creation, large models like chatgpt are a game changer.
- pclmulqdq 4y agoI'm not sure you've seen the scale achievable by modern "social media marketing" firms. Copywriters are so cheap and good at writing posts that the marginal cost of an astroturfing bot in a place like Reddit or Twitter is almost $0 before LLMs. LLMs just reduce the cost a little bit more.
- deltree7 4y agoHow is that different from Bernie Sanders effectively brainwashing an entire generation that communism is good?
- tlb 4y agoI mean, most dictators didn't "do" much. They just said things and gesticulated dramatically and convinced other people to do things. Perhaps a body is necessary to have massive psychological effects on people, but we don't know that for sure and there are some signs of virtual influencers gaining traction. Human would-be demagogues only have one voice, but an LLM could be holding personalized conversations with millions of people simultaneously, convincing them all that they should become its loyal followers and all their grievances would be resolved. I can't figure out exactly how demagogues gain power over people, but a few keep succeeding every decade around the world so evidently it's possible. We're lucky that not many people are both good at it and want to do it. An LLM could be a powerful tool for people who want to take over the world but don't have the skills to accomplish it. So it's not clear they need their own "will", they just have to execute towards a specified goal. "But would an LLM even understand the idea of taking over the world?" LLMs have been trained on Reddit, the NYT, and popular novels among other sources. They've read Orwell and Huxley and Arendt and Sun Tzu. The necessary ideas are most definitely in the training set.
- deleted 4y ago[deleted]
- _yb2s 4y ago> These AI tools cannot do things. They create text (or images or code or what-have-you) in response to prompts. And that's it! You are correct, but that is just the interface we use, it says nothing about its internal structure or capabilities, and does not refute those concerns in the way you think it does. Sufficient accuracy at predicting tokens, especially about novel concepts outside of the training set requires no less than a model of the universe that generated those tokens. This is what intelligence is. In my own experiments with Gpt-4, it can solve difficult novel problems and predict the outcomes of physical experiments unlike anything it was trained on. Have you seen the microsoft paper on its creative problem solving abilities, or tested them yourself? Your summary of its limitations implies that its real capabilities identified in a research environment are impossible. Becoming an “agent” with “will” from being a sufficiently accurate text prediction model is trivial, it’s a property of how you access and configure use of the model, not of the model itself. It just needs to be given a prompt with a goal, and be able to call itself recursively and give itself commands, which it has already demonstrated an ability to do. It has coded a working framework for this just from a prompt asking it to.
- miraculixx 4y agoActually, it is quite possible to get LLMs to actually do stuff. See ChatGPT Plugins.
- jacquesm 4y agoBest response according to you.
- worldsayshi 4y agoWhen there's intelligence adding a will should be trivial. You just tell it to do something and give it some actuator, like a web browser. Then let it run.
- wruza 4y agoNot that I agree with the trivial part, but that’s a good question. Afaiu, current AI has a “context” of few thousand something, in which it operates. If someone enlarges it enough, loops it to data sources, makes its output to do real things (posts, physical movements), then it’s a stage for “will”. You only have to prompt this chain with “chase <a goal> and avoid <destruction conditions>”. If we humans didn’t have to constantly please thermodynamics and internal drives, we’d stay passive too.
- rdelpret 4y agoI agree with most of what you are saying, but when I read the letter my mind goes to the economic impact it could have. A tool like this could bring humans prosperity but with the current socioeconomic conditions we live under it seems it will do the opposite. In my mind that problem feels insurmountable so maybe we just let it sort itself out? Conventional wisdom would say that tools like this should allow society to have UBI or a 4 day work week but in reality the rich will get richer and the poor will get poorer.