11 ms·

Really concerning to me that people find the luddite argument so persuasive and that it gets so much play in the press. The crux of the argument from the outsid

Really concerning to me that people find the luddite argument so persuasive and that it gets so much play in the press. The crux of the argument from the outside "ethicists" quoted in the article is something like. "This new piece of technology might be dangerous and we don't fully understand it, so we should not poke at it or allow people to study it." Maybe there's just something about my personality that is deeply at odds with this sentiment, but it's also about the lack of testable predictions coming from people like this. Their position could be taken about literally anything with the same logical justification. It's a political and emotional stance masquerading as a technical or scientific process.

trention 4y ago
Here is one prediction: Because of language models, the amount of fake news online will increase by an order of magnitude (at least) before this decade ends. And there is no interpretation of that development as anything else but a net negative. Another more broad prediction: In a decade, the overall influence of language models on our society will be universally seen as a net negative.
- xmprt 4y ago
  What if that's the push that brings people out into the physical world where they don't have to deal with all this crap online.
  - carschno 4y ago
    Slightly less optimistic, but perhaps more realistic thought: what if that's the push that makes people validate their (internet) sources? Seems like it might become clear that random web pages are just automatically generated content. If you really want to learn something credible at all, you'll really have to be more specific about your sources than "the internet".
    - ninkendo 4y ago
      Yeah, that was gonna be my contrarian HN hot-take too. Basically if it becomes really obvious some day that basically all online news publication is noise written by computers, maybe people will stop actually trusting it so much?
- NoMAD76 4y ago
  Fake news and AI generated news are kinda all over the place for a good amount of time. It's faster and cheaper to have AI write news from a press release. My prediction is that in the next 10y we will really struggle to determine between fake-people and real-human. There will be an explosion of fake-identities posting more and more human-like. But I'm not Nostradamus so I could be very very off here.
  - shon 4y ago
    You’re probably right but it will be an arms race with ever more sophisticated mitigation techniques being deployed to filter. I’d say Neil Stephenson has a pretty good take on what this might look like in his recent book: Fall, where in everyone has a “feed” and those that or more savvy/wealthy have better editors (AI, etc) of their feed.
    - NoMAD76 4y ago
      It's all about having the right tools, but I wonder how long can we "beat the machine" :)
  - paganel 4y ago
    It depends. On forums like this it would basically take a machine that would pass the Turing test in order not to be seen as an AI in any "meaningful" conversation that it might join (so, not just a comment posted as a reply here and there). And even if the powers that be manage to get those future AI bots to post stuff that will very much resemble what we now post in here, it is my belief that the uncanny valley will be, in the end, impossible to pass (in fact that's one of the main motifs of many of Asimov's books when it comes to robots).
  - jerf 4y ago
    I'm not particularly convinced by the Dead Internet Theory [1] as of 2022, in the sense that it is completely true right now. But I am convinced it is building around us, and even now, the correct frame for the question isn't whether it is true, but how true it is. There is too many entities with too much reason to build it for it not to be happening. And the nature of it is such that it doesn't need to be one entity doing it for one unified reason; dozens, hundreds can all be participating & fighting with each other on different levels and the sum total of all that is to put it together that much more quickly. You know, the PGP web of trust idea may yet take off all these decades later, not because we need a web of trust to send 100.0000000% safely encrypted messages to each other to protect from governments, but because we need a web of trust just to know who the real humans are. [1]: https://forum.agoraroad.com/index.php?threads/dead-internet-theory-most-of-the-internet-is-fake.3011/ https://forum.agoraroad.com/index.php?threads/dead-internet-...
- l33t2328 4y ago
  I have already generated fake news-esque things with gpt-3 to send to friends. A lot of the outputs look incredibly genuine. We live in interesting times.
- notahacker 4y ago
  Cost per word of fake news is already very low though, and humans are much better at tailoring it to the appropriate audience and agenda (and not just restating stuff that's already out there that might actually be true) GPT type models are much better suited to low effort blogspam, and whilst that's not a good thing, they produce better blogspam than existing blogspamming techniques. I think we underestimate how bad the Internet already is, and at worst text generated by AI is simply going to reflect that.
  - NoMAD76 4y ago
    It's about being 1st to publish it. It is mainly use during live press conferences. No human can snap a photo (at least), write a short update in a dedicated article, and so on... all in 1s. Been there (as an independent press member years ago), simply you cannot beat that.
    - notahacker 4y ago
      First to publish matters, but GPT-3 is neither necessary nor sufficient to achieve that. If you're producing fake news related to a press conference, speed of content generation is entirely unimportant because you don't have to wait for the press conference to start to write the fake article/update/tweet. If you care about fidelity to the message of the press conference, I don't see many situations in which a human who has anticipated the likely message(s) of the conference and therefore has pre-drafted paragraphs about "striking a conciliatory tone" and "confirmed the reversal of the policy" ready to select and combine with a choice quote or two as soon as they're uttered isn't significantly better (and as quick as) GPT-type models prompted by the speech-to-text of the press conference. Sure, more reliable publications will want to stick more validation and at least wait for the conference to finish before summarising its message steps in the way, but those apply to bots as well as journalists (or not, for the publications that prioritise being first over being right).
      - NoMAD76 4y ago
        You have a solid point, but I wasn't talking abut summarizing or excerpt from a press release (those are anyway handed as press-kits before with all NDA agreements and so on). Real human journalists have a delay of about 1m before making a short tweet. Funny (or not), something similar was in the "live update" article page in less than 10s. Including photo(s). I was on quite a lot of tech-conferences/live events and earned a decent living then as an independent tech journalist (but then I got bored and really it was a 1-man-show). Another personal observation (from field), that was not happening prior to 2010-2012, the years we all got Siri, Cortana.. You can make the dots and dashes.
- jerf 4y ago
  That has already happened. manimino linked me to this great page on another thread on HN a few days ago: https://cookingflavr.com/should-you-feed-orioles-all-summer/ https://cookingflavr.com/should-you-feed-orioles-all-summer/ But consider that a particularly easy to detect version of the problem. Click some of the other links on that site and have a look. I especially suggest you click something on a topic you know nothing about. I've been hitting these sites in searches accidentally more and more over the past few months. Goodness help you if you don't realize it's totally fake; some of what I've seen is dangerous, like, bad electrical advice being blithely generated by whichever exact transformer variant is spewing that stuff.
  - Vetch 4y ago
    There is an economic incentive to detect machine generated output and curate trusted sites since the feedback loop of training models on an unfiltered internet of mostly generated junk output will eventually lead the models into converging on a useless degenerate state.
    - shon 4y ago
      Useless Degenerate State... I think I just found the name of my new band!
    - tonypace 4y ago
      But here it is in our field of view. This model has very limited look ahead. I don't think we can rely on that in the future.
- bufferoverflow 4y ago
  People don't get their news from random AI-generated blogs. The actually bad consequence is SEO spam of high quality. You can now generate a hundred articles a minute in any topic.
  - uni_rule 4y ago
    We are already seeing a lot of fucked up SEO spam rising to the top these days. IMO it might actually start picking at Google's market share because prior to this the product actually seemed infallible to the average layman.
- textcortex 4y ago
  I would not worry about that too much. There are already models that can predict if its transformer generated or not. At the same time google started penalizing transformer generated text; https://youtu.be/EZWx05O6Hss https://youtu.be/EZWx05O6Hss
  - hhmc 4y ago
    It doesn't seem like there's any real guarantee that the 'generated text detectors' will outpace the 'generated text generators'
- brian_cloutier 4y ago
  What is the issue with fake news? Language models can now pass as human in many situations but there are already billions of humans capable of writing fake news, this isn't a new capability. We have already created mechanisms for deciding which voices to trust and no matter how good language models get they will not be able to prevent you from visiting economist.com
  - texaslonghorn5 4y ago
    I think the issue could be volume (and also that the vast majority of humans aren't actively exercising their ability to write fake news at present). Also that language models might be far more convincing.
  - Viliam1234 4y ago
    > there are already billions of humans capable of writing fake news You have to pay them, and most of them are not very good at writing. Even with a big budget, you get a limited number of good articles per day. If you can make writing fake news 100x cheaper, and then just throw everything at social networks and let people sort out the most viral stuff, that can change the game. Also, computers can be faster. If something new happens today and hundred articles are written about it, a computer can quickly process them and generate hundred more articles on the same topic, than a group of humans would. (Many humans can do the writing in parallel, but each of them needs to read individually the things they want to react to.)
    - Vetch 4y ago
      The vast majority of generators of fake news today are from Content Mills, Copy Paste Mills and general SEO spammers. Political misinformation is the other big generator. The economics of it and not "ethical" gate keeping is what will affect their output. Realistically, normal people don't have the ability to coordinate hordes of proxied IPs to attack social networks requiring account sign ups and custom feeds. The value of exercising critical thinking, checking trusted curated sources, information hygiene, recognizing and avoiding manipulation tactics in news and ads will have to go up. The internet is already almost entirely unreliable without requiring any fancy AI. The listed skills will be necessary regardless any increase of politically manipulative content, advertisements or product misrepresentations.
- mgraczyk 4y ago
  Alternative hypothesis for which I have at least as much evidence: Because of large language models, detecting Fake News becomes trivial and cheap. Building and doing inference on language models is too expensive for most attackers, so they give up. Only well financed state actors are capable of disseminating fake news, and they are no better at it than they are today because content generation is not a bottleneck.
  - deleted 4y ago
    [deleted]
- radford-neal 4y ago
  "Because of language models, the amount of fake news online will increase by an order of magnitude (at least) before this decade ends. And there is no interpretation of that development as anything else but a net negative." That's not at all clear. You're assuming people will continue to give credence to random stuff they read. But once fake AI-generated content is common, people will surely become less trusting. The end result could easily be that fewer people than before believe fake news is real. Presumably, fewer people will believe real news too, but the result could still be net positive.
  - joe_the_user 4y ago
    Bogus content has been available in abundance online for a while and people often are credulous of it still. But still, you have to assume that the cost of creating fake news is the primary limit to it's appearance in front of people to claim AI will seriously have an impact and that's not at all obvious.
- mortenjorck 4y ago
  I keep seeing this prediction, but have yet to see a convincing argument as to how this content is supposed to circumvent existing trust networks. News outlets are nothing without a track record. People trust names they recognize. You can spin up as many CMS instances, domains, and social media profiles for fake news as you want, but without a history shared with its core audience, all the language models in the world aren't going to convince anyone but the most credulous when the content is coming from unfamiliar sources.
  - ramblenode 4y ago
    Most people now get their news from downstream social media, not from the newsroom. You don't have to sway opinion with fake articles, just a sea of astroturf that supports some opinion and creates the illusion of a crowd, which has traditionally been the main signal for credibility of an idea.
  - NoMAD76 4y ago
    Our grand-grand-granfathers generation believed in most of the things because, you know "I heard it in the church". Our grand-grandfathers generation believed in most of the things because, you know "I read it on the newspaper". Our grandfathers generation believed in most of the things because, you know "I've heard it on the radio". Our parents generation believed in most of the things because, you know "I've seen it on TV". Our generation believed in most of the things because, you know "I've read/seen/watch it on social media". I don't know about the next generations(s), the only thing in common for the above mentions was that they were made by a real-human.
  - vintermann 4y ago
    Well, how is it done today? Twitter is instructive. You have mostly-automated accounts that like, retweet and reply to content from a specific perspective. Often it is an incendiary perspective, and usually it's a perspective underrepresented in regular news, because that's more useful for getting people to hit like and follow on the account's post. Once the account has gained enough real followers, they can carefully start to push payload content, the content the operator really cares about. This drives polarisation. There's this idea that sinister foreign adversaries are trying to "spread chaos", but I don't buy it. I think it's merely a by-product of building an audience. Russia (to pick an example) doesn't care about BLM, anti-BLM, US culture wars or the Assange case. Rather, it cares about exactly the things you'd expect it to care about: sanctions, conflicts it is involved in, allies etc. (For that matter, ad-driven media does much the same. Gawker before its demise had really gone all-in on "outrage bait" stories.) They're going to start with the most credulous, maybe, but they'll build confidence in the same way as everyone else who's an unknown at start.
- jstummbillig 4y ago
  > And there is no interpretation of that development as anything else but a net negative. Sure there is: An inevitability. I am hoping we will increasingly turn attention towards how to handle it (although I am relatively certain that's already going on at rapidly growing scale at fb, google and openai). I could see updated legislation doing a lot of heavy lifting – strict rules against automated mass-disinformation – but more action is certainly going to be required.
  - trention 4y ago
    The fact that something is "an inevitability" (which is definitely not the case here) has nothing to do with whether it's a net negative or not. This is like saying "you should classify this as blue" and then a response like "no, it's heavy".
    - jstummbillig 4y ago
      In actuality, it's more like saying "you can only classify this as blue", being awkwardly wrong with the premise, proceeding to not see reason and then get cute.
- Aeolos 4y ago
  This has already happened, to a large extent. Search for most any topic on google, e.g. recipes, and the first two pages will be chock-full of AI-generated copy.
- wruza 4y ago
  Plot twist, people stop relying on news’ blogs’ meanings and create a system of decision making that is another level and actually works for them and not for the most sound speech talkers. You learn to swim by getting wet.
  - tonypace 4y ago
    In the shadow end, typically. You can easily stand up and breathe.
- AmericanChopper 4y ago
  Hot take: The majority of information consumed by people is compromised of low quality opinions, misrepresentations, and outright lies. It has always been this way, always will be this way, and every single person who clutches at their pearls over this fact also believes things would be improved by having the correct people control access to information.
drcode 4y ago
The remaining great apes remain alive primarily due to our pity for their plight- They were once the highest IQ species in the world, but no more Probably, we will be in the same situation relatively soon: And there is little reason to expect the AI systems to have the same pity Sorry I can't set up a double-blind, testable, peer-reviewed study to help convince you of this
guelo 4y ago
The attitude that I have trouble understanding is "A company spent millions of dollars researching and developing a new technology, they must make it available to me or else they are evil."
- ipaddr 4y ago
  Spent millions on tech that could be a net negative for society. Keeping details secret makes people think they are evil because that's how coverups happen.
amelius 4y ago
One day, these people will be right! (And then we know one solution to the Fermi paradox.)
rglover 4y ago
Read Ted Kaczynski's (yes, that one) "Industrial Society and Its Future" with a neutral mind and you will start to understand why it's compelling.
dr-detroit 4y ago
phphphphp 4y ago
History has shown that humans are terrible judges of the outcomes of our behaviour; your belief that we can confidently understand the risks of anything through experimentation might work in theory but hasn’t been borne out in practice. Extremists exist at both ends of the spectrum and serve to balance each other out: without people positing the worst-case scenarios, the people positing the best-case scenarios would run full steam ahead without any consideration for what could happen. Perhaps if the proponents of (various flavours of) AI were doing careful experimentation and iteratively working towards a better understanding, then maybe the loud voices against it would be less valuable, but as we’ve seen through the last 20 years, progress in technology is being made without a second thought for the consequences — and what Facebook are doing here is a bare minimum, so it’s reasonable for proponents to be somewhat cynical about the long term consequence.
adamsmith143 4y ago
That's not the argument at all. Rather it's that the technology is progressing so fast and could become dangerous far faster than we can make it safe. Therefore it's worth seriously thinking about the risks that scenario poses. Stopping research or further work completely is A potential solution but given the monetary investments involved it's extremely unlikely it will be implemented. There are lots of very serious people seriously looking at these issues and to dismiss them as simple luddites is frankly insulting.
- mgraczyk 4y ago
  But nobody is failing to take the risks seriously. The people who actually work on these models understand the risks far better than the outside ethicists. I work on ML in research. Reading their work is like listening to a drunk relative at Thanksgiving ranting about Bill Gates putting nanobots in vaccines. It's completely uninformed pseudoscience that comes from a place of strong political bias. For example Timnit's "parrots" paper confused training with inference and GPUs with TPUs, making specific quantitative estimates that were off by orders of magnitude. If she had talked to a single person working on large language models, she would have recognized the error. But these people work in a bubble where facts don't matter and identify politics is everything.
  - trention 4y ago
    There are enough people criticizing both current language models and the overall "quest" towards AGI that come from a non-political (unless you subscribe to an aristotelian everything-is-politics) perspective. I personally don't think any of the companies with significant AI research is actually doing anything meaningful in terms of safety. Also, from their public "utterings", it's quite clear to me that both Altman and Hassabis (not to mention Lecun) don't actually care about safety or consequences.
    - mgraczyk 4y ago
      > I personally don't think any of the companies with significant AI research is actually doing anything meaningful in terms of safety I assume this is just speculation on your part? Do you have any reason to make that claim? I personally know multiple people doing this full time at large tech companies. I can give you some examples of serious safety oriented criticism of large language models to contrast with what plays out in the press and amongst "ethicists". It's well understood that one can generate so-called "adversarial examples" for image classifiers. These adversarial examples can be chosen so that to a human they look like thing A, but the model classifies them as thing B with high probability. Methods of finding these adversaries are well understood. Methods of preventing them from being problematic are less developed but rapidly advancing. For language models, the situation is much worse. I don't know of any effective way to prevent a large language model from being searched for adversarial inputs, trivially. That is, an attacker could find inputs from large curated input spaces that cause the model to output a specific, desired sequence. For example, an attacker with access to the model weights could probably find an innocuous looking input that causes the model to output "kill yourself". Is this a risk that AI researchers are aware of? Yes, of course. But the difference between AI researchers and "ethicists" is that AI researchers understand the implications of the risk and will work on mitigations. "Ethicists" do not care about mitigating risk, and they don't care that the people who build the models already understand them and are comfortable with them.
jonas21 4y ago
It's much easier to sit on the sidelines and come up with reasons why you shouldn't do something new than it is to actually do it. Some people have figured this out and built careers on it. This wouldn't be a problem, except that this opposition eventually becomes their professional identity - they derive prestige from being the person who is fighting against whatever. So even after researchers address their concerns, they have to ignore the progress or move the goalposts so they can keep on opposing it.
- eszaq 4y ago
  This doesn't seem like a bad thing to me. In the same way we have public defenders who professionally defend scoundrels, it seems good to have people who professionally critique new technologies. I'm old enough to remember the naive optimism around the internet in the 2000s. "The long tail", "cognitive surplus", "one laptop per child", Creative Commons, the Arab "Spring", breathless Youtube videos about how social media is gonna revolutionize society for the better, etc. Hardly anyone forecasted clickbait, Trump tweets, revenge porn, crypto scams, or social media shaming. If we had a few professional critics who were incentivized to pour cold water on the whole deal, or at least scan the horizon for potential problems, maybe things would've turned out better.
skybrian 4y ago
It seems like there is a difference between "let's release it for researchers to investigate" and "let's release it for the general public to use and abuse, including all the script kiddies and malware authors and 4chan trolls and spammer sites out there." Unfortunately, that difference can only be maintained through some kind of gatekeeping. I like to try out the new algorithms, but I'm mostly just playing, and I don't see how they make it available to me without letting any random troll use it.
api 4y ago
This is the hydrogen bomb of propaganda. Imagine assigning every single living human being a dedicated 24/7 con artist to follow them around and convince them of something. That's what will soon be possible if not already. It will be intimate con artistry at scale driven by big data, a massive DDOS attack on human cognition and our ability to conduct any form of honest discourse. What hustlers, trolls, and completely amoral companies will do is bad enough. Now throw in state sponsored intelligence agencies, propaganda farms, militaries, special interest groups, and political parties. Usually I'm anything but a luddite, but with this I can't help but think of many more evil uses than good ones. It doesn't help that the principal business models of the (consumer) Internet seem to center around surveillance, advertising, propaganda, and addictive forms of entertainment (like slot-machine-like mobile games) designed to suck money or time out of people. Lesser but also very bad concerns include: the end of useful search engines due to a deluge of continuously learning adversarial SEO spam, the collapse of pretty much any open online forum due to same, and addictive "virtual friend" / "virtual relationship partner" hyper-sophisticated chatbots that hook vulnerable lonely people and then empty their bank accounts in various ways. I really don't fear AI itself. I fear what human beings will do with AI.
- Konohamaru 4y ago
  > Imagine assigning every single living human being a dedicated 24/7 con artist to follow them around and convince them of something. This sounds like traditional Christian teaching of the role of demons.
  - api 4y ago
    We invented Satan and his minions to make people click ads.
- mgraczyk 4y ago
  > It will be intimate con artistry at scale driven by big data, a massive DDOS attack on human cognition and our ability to conduct any form of honest discourse. This is an unfounded fear. For one thing, if the value in doing this is high then it's already cheap enough to be practical. The Chinese govt can pay millions of people to do this to dozens of people each. They basically do this already, for specific topics and issues. LLMs won't significantly move the needle here. Second, are you proposing that attempts to stop Facebook from releasing models will somehow slow down or stop the Chinese, US, or Russian governments? What's the goal, to buy us 6 months? I would much rather the technology be out in the open for everyone to research and understand vs accessible only to state actors or huge tech companies.
humanistbot 4y ago
> "ethicists" > It's a political and emotional stance masquerading as a technical or scientific process. I don't think you understand what ethics is.
- vintermann 4y ago
  I use to say that academic ethics is the study of what you can get away with. If you want guidance on being good, you need a saint, not an ethicist.
armchairhacker 4y ago
My understanding is that when companies say “we aren’t releasing this / limited access / restrictions / for ethical reasons” they really mean “we aren’t releasing this because a) it’s expensive to run these models, b) it was even more expensive to create them and we might be able to profit, and c) maybe it’s bad for our ethics, which affects our funding and relations, and also, ethics.”
godmode2019 4y ago
Its a business play, They are asking to be regulated because they have finished writing their models. With regulation it will be harder for up and coming models to gain traction. Its getting so much coverage because its paid press, I read about it in my newspaper BEFORE tech YouTube and here.
JohnHaugeland 4y ago
The movie Star Trek 2 has done tremendous damage to peoples' understanding of what a luddite was. In modern terms, luddite wanted a tax on automation to support loss from tradesworkers who were too old to retrain. It's similar to when bankers wanted a pension buyout at the introduction of the atm. Luddites were pro technology, not anti. They just wanted wage distribution of the productivity gains. They're basically early unions.
dalbasal 4y ago
So... I'm with you on being dispositionally pro-tech ad anti-luddite but.... >> lack of testable predictions coming from people like this I think this is a disingenuous line of argument, more often than not. Popperian science is great, but it is not everything. The majority of our opinions, knowledge and conclusions are not based on testable hypotheses and falsifiable statements. Take the sentence "One person should not have absolute power." It's not a falsifiable statement, or scientific in other ways. It's based on anecdote and folk wisdom, not science. I agree with you. And, I think we need to argue back. But, don't argue meta. Meet the arguments head on.

trention 4y ago

Here is one prediction: Because of language models, the amount of fake news online will increase by an order of magnitude (at least) before this decade ends. And there is no interpretation of that development as anything else but a net negative. Another more broad prediction: In a decade, the overall influence of language models on our society will be universally seen as a net negative.

xmprt 4y ago
What if that's the push that brings people out into the physical world where they don't have to deal with all this crap online.
- carschno 4y ago
  Slightly less optimistic, but perhaps more realistic thought: what if that's the push that makes people validate their (internet) sources? Seems like it might become clear that random web pages are just automatically generated content. If you really want to learn something credible at all, you'll really have to be more specific about your sources than "the internet".
  - ninkendo 4y ago
    Yeah, that was gonna be my contrarian HN hot-take too. Basically if it becomes really obvious some day that basically all online news publication is noise written by computers, maybe people will stop actually trusting it so much?
NoMAD76 4y ago
Fake news and AI generated news are kinda all over the place for a good amount of time. It's faster and cheaper to have AI write news from a press release. My prediction is that in the next 10y we will really struggle to determine between fake-people and real-human. There will be an explosion of fake-identities posting more and more human-like. But I'm not Nostradamus so I could be very very off here.
- shon 4y ago
  You’re probably right but it will be an arms race with ever more sophisticated mitigation techniques being deployed to filter. I’d say Neil Stephenson has a pretty good take on what this might look like in his recent book: Fall, where in everyone has a “feed” and those that or more savvy/wealthy have better editors (AI, etc) of their feed.
  - NoMAD76 4y ago
    It's all about having the right tools, but I wonder how long can we "beat the machine" :)
- paganel 4y ago
  It depends. On forums like this it would basically take a machine that would pass the Turing test in order not to be seen as an AI in any "meaningful" conversation that it might join (so, not just a comment posted as a reply here and there). And even if the powers that be manage to get those future AI bots to post stuff that will very much resemble what we now post in here, it is my belief that the uncanny valley will be, in the end, impossible to pass (in fact that's one of the main motifs of many of Asimov's books when it comes to robots).
- jerf 4y ago
  I'm not particularly convinced by the Dead Internet Theory [1] as of 2022, in the sense that it is completely true right now. But I am convinced it is building around us, and even now, the correct frame for the question isn't whether it is true, but how true it is. There is too many entities with too much reason to build it for it not to be happening. And the nature of it is such that it doesn't need to be one entity doing it for one unified reason; dozens, hundreds can all be participating & fighting with each other on different levels and the sum total of all that is to put it together that much more quickly. You know, the PGP web of trust idea may yet take off all these decades later, not because we need a web of trust to send 100.0000000% safely encrypted messages to each other to protect from governments, but because we need a web of trust just to know who the real humans are. [1]: https://forum.agoraroad.com/index.php?threads/dead-internet-theory-most-of-the-internet-is-fake.3011/ https://forum.agoraroad.com/index.php?threads/dead-internet-...
l33t2328 4y ago
I have already generated fake news-esque things with gpt-3 to send to friends. A lot of the outputs look incredibly genuine. We live in interesting times.
notahacker 4y ago
Cost per word of fake news is already very low though, and humans are much better at tailoring it to the appropriate audience and agenda (and not just restating stuff that's already out there that might actually be true) GPT type models are much better suited to low effort blogspam, and whilst that's not a good thing, they produce better blogspam than existing blogspamming techniques. I think we underestimate how bad the Internet already is, and at worst text generated by AI is simply going to reflect that.
- NoMAD76 4y ago
  It's about being 1st to publish it. It is mainly use during live press conferences. No human can snap a photo (at least), write a short update in a dedicated article, and so on... all in 1s. Been there (as an independent press member years ago), simply you cannot beat that.
  - notahacker 4y ago
    First to publish matters, but GPT-3 is neither necessary nor sufficient to achieve that. If you're producing fake news related to a press conference, speed of content generation is entirely unimportant because you don't have to wait for the press conference to start to write the fake article/update/tweet. If you care about fidelity to the message of the press conference, I don't see many situations in which a human who has anticipated the likely message(s) of the conference and therefore has pre-drafted paragraphs about "striking a conciliatory tone" and "confirmed the reversal of the policy" ready to select and combine with a choice quote or two as soon as they're uttered isn't significantly better (and as quick as) GPT-type models prompted by the speech-to-text of the press conference. Sure, more reliable publications will want to stick more validation and at least wait for the conference to finish before summarising its message steps in the way, but those apply to bots as well as journalists (or not, for the publications that prioritise being first over being right).
    - NoMAD76 4y ago
      You have a solid point, but I wasn't talking abut summarizing or excerpt from a press release (those are anyway handed as press-kits before with all NDA agreements and so on). Real human journalists have a delay of about 1m before making a short tweet. Funny (or not), something similar was in the "live update" article page in less than 10s. Including photo(s). I was on quite a lot of tech-conferences/live events and earned a decent living then as an independent tech journalist (but then I got bored and really it was a 1-man-show). Another personal observation (from field), that was not happening prior to 2010-2012, the years we all got Siri, Cortana.. You can make the dots and dashes.
jerf 4y ago
That has already happened. manimino linked me to this great page on another thread on HN a few days ago: https://cookingflavr.com/should-you-feed-orioles-all-summer/ https://cookingflavr.com/should-you-feed-orioles-all-summer/ But consider that a particularly easy to detect version of the problem. Click some of the other links on that site and have a look. I especially suggest you click something on a topic you know nothing about. I've been hitting these sites in searches accidentally more and more over the past few months. Goodness help you if you don't realize it's totally fake; some of what I've seen is dangerous, like, bad electrical advice being blithely generated by whichever exact transformer variant is spewing that stuff.
- Vetch 4y ago
  There is an economic incentive to detect machine generated output and curate trusted sites since the feedback loop of training models on an unfiltered internet of mostly generated junk output will eventually lead the models into converging on a useless degenerate state.
  - shon 4y ago
    Useless Degenerate State... I think I just found the name of my new band!
  - tonypace 4y ago
    But here it is in our field of view. This model has very limited look ahead. I don't think we can rely on that in the future.
bufferoverflow 4y ago
People don't get their news from random AI-generated blogs. The actually bad consequence is SEO spam of high quality. You can now generate a hundred articles a minute in any topic.
- uni_rule 4y ago
  We are already seeing a lot of fucked up SEO spam rising to the top these days. IMO it might actually start picking at Google's market share because prior to this the product actually seemed infallible to the average layman.
textcortex 4y ago
I would not worry about that too much. There are already models that can predict if its transformer generated or not. At the same time google started penalizing transformer generated text; https://youtu.be/EZWx05O6Hss https://youtu.be/EZWx05O6Hss
- hhmc 4y ago
  It doesn't seem like there's any real guarantee that the 'generated text detectors' will outpace the 'generated text generators'
brian_cloutier 4y ago
What is the issue with fake news? Language models can now pass as human in many situations but there are already billions of humans capable of writing fake news, this isn't a new capability. We have already created mechanisms for deciding which voices to trust and no matter how good language models get they will not be able to prevent you from visiting economist.com
- texaslonghorn5 4y ago
  I think the issue could be volume (and also that the vast majority of humans aren't actively exercising their ability to write fake news at present). Also that language models might be far more convincing.
- Viliam1234 4y ago
  > there are already billions of humans capable of writing fake news You have to pay them, and most of them are not very good at writing. Even with a big budget, you get a limited number of good articles per day. If you can make writing fake news 100x cheaper, and then just throw everything at social networks and let people sort out the most viral stuff, that can change the game. Also, computers can be faster. If something new happens today and hundred articles are written about it, a computer can quickly process them and generate hundred more articles on the same topic, than a group of humans would. (Many humans can do the writing in parallel, but each of them needs to read individually the things they want to react to.)
  - Vetch 4y ago
    The vast majority of generators of fake news today are from Content Mills, Copy Paste Mills and general SEO spammers. Political misinformation is the other big generator. The economics of it and not "ethical" gate keeping is what will affect their output. Realistically, normal people don't have the ability to coordinate hordes of proxied IPs to attack social networks requiring account sign ups and custom feeds. The value of exercising critical thinking, checking trusted curated sources, information hygiene, recognizing and avoiding manipulation tactics in news and ads will have to go up. The internet is already almost entirely unreliable without requiring any fancy AI. The listed skills will be necessary regardless any increase of politically manipulative content, advertisements or product misrepresentations.
mgraczyk 4y ago
Alternative hypothesis for which I have at least as much evidence: Because of large language models, detecting Fake News becomes trivial and cheap. Building and doing inference on language models is too expensive for most attackers, so they give up. Only well financed state actors are capable of disseminating fake news, and they are no better at it than they are today because content generation is not a bottleneck.
- deleted 4y ago
  [deleted]
radford-neal 4y ago
"Because of language models, the amount of fake news online will increase by an order of magnitude (at least) before this decade ends. And there is no interpretation of that development as anything else but a net negative." That's not at all clear. You're assuming people will continue to give credence to random stuff they read. But once fake AI-generated content is common, people will surely become less trusting. The end result could easily be that fewer people than before believe fake news is real. Presumably, fewer people will believe real news too, but the result could still be net positive.
- joe_the_user 4y ago
  Bogus content has been available in abundance online for a while and people often are credulous of it still. But still, you have to assume that the cost of creating fake news is the primary limit to it's appearance in front of people to claim AI will seriously have an impact and that's not at all obvious.
mortenjorck 4y ago
I keep seeing this prediction, but have yet to see a convincing argument as to how this content is supposed to circumvent existing trust networks. News outlets are nothing without a track record. People trust names they recognize. You can spin up as many CMS instances, domains, and social media profiles for fake news as you want, but without a history shared with its core audience, all the language models in the world aren't going to convince anyone but the most credulous when the content is coming from unfamiliar sources.
- ramblenode 4y ago
  Most people now get their news from downstream social media, not from the newsroom. You don't have to sway opinion with fake articles, just a sea of astroturf that supports some opinion and creates the illusion of a crowd, which has traditionally been the main signal for credibility of an idea.
- NoMAD76 4y ago
  Our grand-grand-granfathers generation believed in most of the things because, you know "I heard it in the church". Our grand-grandfathers generation believed in most of the things because, you know "I read it on the newspaper". Our grandfathers generation believed in most of the things because, you know "I've heard it on the radio". Our parents generation believed in most of the things because, you know "I've seen it on TV". Our generation believed in most of the things because, you know "I've read/seen/watch it on social media". I don't know about the next generations(s), the only thing in common for the above mentions was that they were made by a real-human.
- vintermann 4y ago
  Well, how is it done today? Twitter is instructive. You have mostly-automated accounts that like, retweet and reply to content from a specific perspective. Often it is an incendiary perspective, and usually it's a perspective underrepresented in regular news, because that's more useful for getting people to hit like and follow on the account's post. Once the account has gained enough real followers, they can carefully start to push payload content, the content the operator really cares about. This drives polarisation. There's this idea that sinister foreign adversaries are trying to "spread chaos", but I don't buy it. I think it's merely a by-product of building an audience. Russia (to pick an example) doesn't care about BLM, anti-BLM, US culture wars or the Assange case. Rather, it cares about exactly the things you'd expect it to care about: sanctions, conflicts it is involved in, allies etc. (For that matter, ad-driven media does much the same. Gawker before its demise had really gone all-in on "outrage bait" stories.) They're going to start with the most credulous, maybe, but they'll build confidence in the same way as everyone else who's an unknown at start.
jstummbillig 4y ago
> And there is no interpretation of that development as anything else but a net negative. Sure there is: An inevitability. I am hoping we will increasingly turn attention towards how to handle it (although I am relatively certain that's already going on at rapidly growing scale at fb, google and openai). I could see updated legislation doing a lot of heavy lifting – strict rules against automated mass-disinformation – but more action is certainly going to be required.
- trention 4y ago
  The fact that something is "an inevitability" (which is definitely not the case here) has nothing to do with whether it's a net negative or not. This is like saying "you should classify this as blue" and then a response like "no, it's heavy".
  - jstummbillig 4y ago
    In actuality, it's more like saying "you can only classify this as blue", being awkwardly wrong with the premise, proceeding to not see reason and then get cute.
Aeolos 4y ago
This has already happened, to a large extent. Search for most any topic on google, e.g. recipes, and the first two pages will be chock-full of AI-generated copy.
wruza 4y ago
Plot twist, people stop relying on news’ blogs’ meanings and create a system of decision making that is another level and actually works for them and not for the most sound speech talkers. You learn to swim by getting wet.
- tonypace 4y ago
  In the shadow end, typically. You can easily stand up and breathe.
AmericanChopper 4y ago
Hot take: The majority of information consumed by people is compromised of low quality opinions, misrepresentations, and outright lies. It has always been this way, always will be this way, and every single person who clutches at their pearls over this fact also believes things would be improved by having the correct people control access to information.

drcode 4y ago

The remaining great apes remain alive primarily due to our pity for their plight- They were once the highest IQ species in the world, but no more Probably, we will be in the same situation relatively soon: And there is little reason to expect the AI systems to have the same pity Sorry I can't set up a double-blind, testable, peer-reviewed study to help convince you of this

guelo 4y ago

The attitude that I have trouble understanding is "A company spent millions of dollars researching and developing a new technology, they must make it available to me or else they are evil."

ipaddr 4y ago
Spent millions on tech that could be a net negative for society. Keeping details secret makes people think they are evil because that's how coverups happen.

amelius 4y ago

One day, these people will be right! (And then we know one solution to the Fermi paradox.)

rglover 4y ago

Read Ted Kaczynski's (yes, that one) "Industrial Society and Its Future" with a neutral mind and you will start to understand why it's compelling.

dr-detroit 4y ago

phphphphp 4y ago

History has shown that humans are terrible judges of the outcomes of our behaviour; your belief that we can confidently understand the risks of anything through experimentation might work in theory but hasn’t been borne out in practice. Extremists exist at both ends of the spectrum and serve to balance each other out: without people positing the worst-case scenarios, the people positing the best-case scenarios would run full steam ahead without any consideration for what could happen. Perhaps if the proponents of (various flavours of) AI were doing careful experimentation and iteratively working towards a better understanding, then maybe the loud voices against it would be less valuable, but as we’ve seen through the last 20 years, progress in technology is being made without a second thought for the consequences — and what Facebook are doing here is a bare minimum, so it’s reasonable for proponents to be somewhat cynical about the long term consequence.

adamsmith143 4y ago

That's not the argument at all. Rather it's that the technology is progressing so fast and could become dangerous far faster than we can make it safe. Therefore it's worth seriously thinking about the risks that scenario poses. Stopping research or further work completely is A potential solution but given the monetary investments involved it's extremely unlikely it will be implemented. There are lots of very serious people seriously looking at these issues and to dismiss them as simple luddites is frankly insulting.

mgraczyk 4y ago
But nobody is failing to take the risks seriously. The people who actually work on these models understand the risks far better than the outside ethicists. I work on ML in research. Reading their work is like listening to a drunk relative at Thanksgiving ranting about Bill Gates putting nanobots in vaccines. It's completely uninformed pseudoscience that comes from a place of strong political bias. For example Timnit's "parrots" paper confused training with inference and GPUs with TPUs, making specific quantitative estimates that were off by orders of magnitude. If she had talked to a single person working on large language models, she would have recognized the error. But these people work in a bubble where facts don't matter and identify politics is everything.
- trention 4y ago
  There are enough people criticizing both current language models and the overall "quest" towards AGI that come from a non-political (unless you subscribe to an aristotelian everything-is-politics) perspective. I personally don't think any of the companies with significant AI research is actually doing anything meaningful in terms of safety. Also, from their public "utterings", it's quite clear to me that both Altman and Hassabis (not to mention Lecun) don't actually care about safety or consequences.
  - mgraczyk 4y ago
    > I personally don't think any of the companies with significant AI research is actually doing anything meaningful in terms of safety I assume this is just speculation on your part? Do you have any reason to make that claim? I personally know multiple people doing this full time at large tech companies. I can give you some examples of serious safety oriented criticism of large language models to contrast with what plays out in the press and amongst "ethicists". It's well understood that one can generate so-called "adversarial examples" for image classifiers. These adversarial examples can be chosen so that to a human they look like thing A, but the model classifies them as thing B with high probability. Methods of finding these adversaries are well understood. Methods of preventing them from being problematic are less developed but rapidly advancing. For language models, the situation is much worse. I don't know of any effective way to prevent a large language model from being searched for adversarial inputs, trivially. That is, an attacker could find inputs from large curated input spaces that cause the model to output a specific, desired sequence. For example, an attacker with access to the model weights could probably find an innocuous looking input that causes the model to output "kill yourself". Is this a risk that AI researchers are aware of? Yes, of course. But the difference between AI researchers and "ethicists" is that AI researchers understand the implications of the risk and will work on mitigations. "Ethicists" do not care about mitigating risk, and they don't care that the people who build the models already understand them and are comfortable with them.

jonas21 4y ago

It's much easier to sit on the sidelines and come up with reasons why you shouldn't do something new than it is to actually do it. Some people have figured this out and built careers on it. This wouldn't be a problem, except that this opposition eventually becomes their professional identity - they derive prestige from being the person who is fighting against whatever. So even after researchers address their concerns, they have to ignore the progress or move the goalposts so they can keep on opposing it.

eszaq 4y ago
This doesn't seem like a bad thing to me. In the same way we have public defenders who professionally defend scoundrels, it seems good to have people who professionally critique new technologies. I'm old enough to remember the naive optimism around the internet in the 2000s. "The long tail", "cognitive surplus", "one laptop per child", Creative Commons, the Arab "Spring", breathless Youtube videos about how social media is gonna revolutionize society for the better, etc. Hardly anyone forecasted clickbait, Trump tweets, revenge porn, crypto scams, or social media shaming. If we had a few professional critics who were incentivized to pour cold water on the whole deal, or at least scan the horizon for potential problems, maybe things would've turned out better.

skybrian 4y ago

It seems like there is a difference between "let's release it for researchers to investigate" and "let's release it for the general public to use and abuse, including all the script kiddies and malware authors and 4chan trolls and spammer sites out there." Unfortunately, that difference can only be maintained through some kind of gatekeeping. I like to try out the new algorithms, but I'm mostly just playing, and I don't see how they make it available to me without letting any random troll use it.

api 4y ago

This is the hydrogen bomb of propaganda. Imagine assigning every single living human being a dedicated 24/7 con artist to follow them around and convince them of something. That's what will soon be possible if not already. It will be intimate con artistry at scale driven by big data, a massive DDOS attack on human cognition and our ability to conduct any form of honest discourse. What hustlers, trolls, and completely amoral companies will do is bad enough. Now throw in state sponsored intelligence agencies, propaganda farms, militaries, special interest groups, and political parties. Usually I'm anything but a luddite, but with this I can't help but think of many more evil uses than good ones. It doesn't help that the principal business models of the (consumer) Internet seem to center around surveillance, advertising, propaganda, and addictive forms of entertainment (like slot-machine-like mobile games) designed to suck money or time out of people. Lesser but also very bad concerns include: the end of useful search engines due to a deluge of continuously learning adversarial SEO spam, the collapse of pretty much any open online forum due to same, and addictive "virtual friend" / "virtual relationship partner" hyper-sophisticated chatbots that hook vulnerable lonely people and then empty their bank accounts in various ways. I really don't fear AI itself. I fear what human beings will do with AI.

Konohamaru 4y ago
> Imagine assigning every single living human being a dedicated 24/7 con artist to follow them around and convince them of something. This sounds like traditional Christian teaching of the role of demons.
- api 4y ago
  We invented Satan and his minions to make people click ads.
mgraczyk 4y ago
> It will be intimate con artistry at scale driven by big data, a massive DDOS attack on human cognition and our ability to conduct any form of honest discourse. This is an unfounded fear. For one thing, if the value in doing this is high then it's already cheap enough to be practical. The Chinese govt can pay millions of people to do this to dozens of people each. They basically do this already, for specific topics and issues. LLMs won't significantly move the needle here. Second, are you proposing that attempts to stop Facebook from releasing models will somehow slow down or stop the Chinese, US, or Russian governments? What's the goal, to buy us 6 months? I would much rather the technology be out in the open for everyone to research and understand vs accessible only to state actors or huge tech companies.

humanistbot 4y ago

> "ethicists" > It's a political and emotional stance masquerading as a technical or scientific process. I don't think you understand what ethics is.

vintermann 4y ago
I use to say that academic ethics is the study of what you can get away with. If you want guidance on being good, you need a saint, not an ethicist.

armchairhacker 4y ago

My understanding is that when companies say “we aren’t releasing this / limited access / restrictions / for ethical reasons” they really mean “we aren’t releasing this because a) it’s expensive to run these models, b) it was even more expensive to create them and we might be able to profit, and c) maybe it’s bad for our ethics, which affects our funding and relations, and also, ethics.”

godmode2019 4y ago

Its a business play, They are asking to be regulated because they have finished writing their models. With regulation it will be harder for up and coming models to gain traction. Its getting so much coverage because its paid press, I read about it in my newspaper BEFORE tech YouTube and here.

JohnHaugeland 4y ago

The movie Star Trek 2 has done tremendous damage to peoples' understanding of what a luddite was. In modern terms, luddite wanted a tax on automation to support loss from tradesworkers who were too old to retrain. It's similar to when bankers wanted a pension buyout at the introduction of the atm. Luddites were pro technology, not anti. They just wanted wage distribution of the productivity gains. They're basically early unions.

dalbasal 4y ago

So... I'm with you on being dispositionally pro-tech ad anti-luddite but.... >> lack of testable predictions coming from people like this I think this is a disingenuous line of argument, more often than not. Popperian science is great, but it is not everything. The majority of our opinions, knowledge and conclusions are not based on testable hypotheses and falsifiable statements. Take the sentence "One person should not have absolute power." It's not a falsifiable statement, or scientific in other ways. It's based on anecdote and folk wisdom, not science. I agree with you. And, I think we need to argue back. But, don't argue meta. Meet the arguments head on.