11 ms·

If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life. This is the same process SEO spam

If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life. This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal. ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke Wikipedia relies on "published" facts as an authority, but ChatGPT output is going to wind up as a source one way or another. And worse, then ChatGPT will digest its own excrement, worsening its own results further. All signs point to this strengthening the value of curation and authenticated sources.

deleted 4y ago
[deleted]
dmitryminkovsky 4y ago
> All signs point to this strengthening the value of curation and authenticated sources. This is what they said about Wikipedia viz. Britannica… alas, it’s a brave new world out there… nowhere to run to nowhere to hide, see that Wiezenbaum post also on the homepage now, as another commenter quotes[0]: > Writing of the enthusiastic embrace of a fully computerized world, Weizenbaum grumbled, “These people see the technical apparatus underlying Orwell’s 1984 and, like children on seeing the beach, they run for it” > a point to which Weizenbaum added “I wish it were their private excursion, but they demand that we all come along.” [0]: https://news.ycombinator.com/item?id=34546689 https://news.ycombinator.com/item?id=34546689
- tablespoon 4y ago
  >> All signs point to this strengthening the value of curation and authenticated sources. > This is what they said about Wikipedia viz. Britannica... And they were right. If "they" were wrong about anything, it was the assumption that the masses would prioritize quality over cost, but it turns out that cheap wins every time. When it comes to information, it's like most people's taste buds don't work, so they'll pick free crap over nutritious food. Edit: Another thought came to mind: stuff like ChatGPT may contribute to killing off Wikipedia: Wikipedia is currently the cheapest and fastest way to find information (that's often crap). However, if something like ChatGPT can get information to people faster (even if it's crappier, just as long as it's minimally acceptable), Wikipedia will become much less popular and could end up just like Britannica.
  - yellow_postit 4y ago
    Extending this — it seems to me there will be growing skillset need to identify quality/accuracy. Under formed thought: The proverbial haystack just got a lot larger, the needle stayed the same size, what tools will needle hunters need to develop both to find the needles and to prove to others they are in fact needles.
    - narrator 4y ago
      The funny thing about ChatGPT is it will write code that uses non-existent confabulated APIs. You have to then call it out and it will say, oh sorry, of course you're right, here's another confabulated API, etc. The amount of convincing B.S it can spew is enormous! Worse, when you stray into topics that are controversial it will often use informal fallacies. When you call it out, it will say, yes, you're right, I used an informal fallacy, here is what I should have said about controversial topic that's not the party line, and because you're so smart I won't b.s you.
      - krupan 4y ago
        I don’t know why more people don’t notice this? I feel like nearly everyone talking about ChatGPT hasn’t really pushed it very far or read what it says very closely. It’s actually pretty terrible
    - cookingrobot 4y ago
      Running with the analogy.. We’re used to using tools that help us find needles in the haystack. A rake, a bright light, a metal detector. Now someone sold us a machine that turns hay into needles. They’re not quite as good, but they’re definitely needles. So now as you say the haystack is going to get covered in these. Do we ban use of this machine? Build new machines to separate these synthetic needles from real ones? Or improve the machine so that the needles it makes are good enough for what we need?
  - idiotsecant 4y ago
    Were they right, though? I am pretty sure I've seen research that compared the scope and accuracy of both and wikipedia was miles ahead.
- mountainb 4y ago
  It's true though, Wikipedia really is terrible and full of fake citations that lead nowhere. It's an anti-knowledge base that sometimes has good information.
  - tablespoon 4y ago
    > It's true though, Wikipedia really is terrible and full of fake citations that lead nowhere. It's an anti-knowledge base that sometimes has good information. Yeah, Wikipedia is garbage puffed up beyond all belief. I literally just today saw something just like you describe. It should be viewed very skeptically on anything anyone disagrees over (because then it's just snapshots of an agenda-pushing battle).
    - gptgpp 4y ago
      Could you elaborate on this? If it is full of garbage a couple examples should be very easy to find. I completely agree that Wikipedia can have errors, but in topics that I am educated in it seems pretty decent and I can't remember the last time I came across any (comp sci for example). The most recent example I can think of is about is an article on vulture bees, and a citation about what their honey tastes like, which turned out to be garbage and incorrect (there are no reliable sources on the qualities of honey, it's basic composition and method of production is even in dispute, when I queried journal articles on the topic). So "garbage puffed up beyond all belief" and "full of terrible and fake citations that lead to nowhere" sounds a bit hyperbolic, tbh.
      - tablespoon 4y ago
        > Could you elaborate on this? If it is full of garbage a couple examples should be very easy to find. I could give examples but I won't, because that would link my HN and Wikipedia accounts. > So "garbage puffed up beyond all belief" and "full of terrible and fake citations that lead to nowhere" sounds a bit hyperbolic, tbh. People unironically describe it as the "sum of all human knowledge," so it's definitely puffed up beyond belief. In reality, much of it is a slow battle of tendentious agenda-pushing, by people with weird personalities, played according to an arcane rule book (the first unstated rule of which is to never, ever acknowledge that you're pushing an agenda). That doesn't taint all of it, but it taints far more than you'd think.
- foruhar 4y ago
  There is a world in which AI will be the best source of knowledge (most powerful knowledge generator). There will be many LLMs & AIs, open and branded, and we'll pick our oracle. ChatGPT is an infant of an AI and it will mutate, and evolve beyond transformers. Some (many?) branches will be amazing at serving up "enshittened knowledge" but there will be branches that take different approaches and philosophies. There will likely be AI curators of knowledge bases that weed out AI-generated crap, and disinformation. There will be non-hallucinatory AIs, certainty scores, explanation-based systems, first principles machines, and super focused additive AIs that will layer onto a base LLM (or whatever is next). We'll choose (and probably pay for) our blends of knowledge, humour, bias, filtering, and conviviality. The "internet" of tomorrow may run on TCP/IP but it very unlikely to work like this web that we are using now.
gtirloni 4y ago
Lots of content on Wikipedia is already created by bots but they are mostly doing maintenance so the blast radius is much smaller.
_fat_santa 4y ago
Were going to have shady agencies in 5 years advertising: "get ChatGPT to respond to questions with your content for only $199"
- yellow_postit 4y ago
  “Detect ChatGPT responses, only $249” It’s just a new chapter in the arms race of spam techniques vs detection. Lots of money presumably made selling both sides.
  - MichaelBurge 4y ago
    GPT has a hidden steganographic watermark in its output, so the arms race should be one-sided.
    - djcannabiz 4y ago
      Do you know how that works? I am unfortunately pretty ignorant about this topic but that seems like such a cool and difficult thing to create.
      - dns_snek 4y ago
        Comment from a few days ago: https://news.ycombinator.com/item?id=34504545 https://news.ycombinator.com/item?id=34504545 > So then to watermark, instead of selecting the next token randomly, the idea will be to select it pseudorandomly, using a cryptographic pseudorandom function, whose key is known only to OpenAI. That won’t make any detectable difference to the end user, assuming the end user can’t distinguish the pseudorandom numbers from truly random ones. But now you can choose a pseudorandom function that secretly biases a certain score—a sum over a certain function g evaluated at each n-gram (sequence of n consecutive tokens), for some small n—which score you can also compute if you know the key for this pseudorandom function. I remain skeptical that this method is resistant against lossy transformations such as changing punctuation, grammar, synonym replacements, 2-way translations and a bunch of other existing tools that are capable of rewording written text.
bobse 4y ago
[flagged]
liveoneggs 4y ago
If ChatGPT just consumes itself will the end result of all queries just eventually recurse down to a single answer? - for example "42"
- labrador 4y ago
  OpenAI sells a pro subscription. I thought it was clever that they priced it at 42 dollars a month.
- boilerupnc 4y ago
  Alternatively, we will just rename ChatGPT as “Hungry Mungry” :-) [0] [0] https://www.poeticous.com/shel-silverstein/hungry-mungry https://www.poeticous.com/shel-silverstein/hungry-mungry
ren_engineer 4y ago
>And worse, then ChatGPT will digest its own excrement, worsening its own results further I wonder if we'll get a "dead sea effect" with AI, I've seen some stuff saying they've basically run out of high quality training data and now the training pool will get poisoned by AI generated shit. Basically garbage in, garbage out and these large language models might not be able to improve
- CuriouslyC 4y ago
  We are nowhere near out of data. We're just out of hyper-relevant modern data. There is probably about 100-200T of old books, newspapers, journal articles, magazines and so forth. For reference, GPT3 was trained on 45T.
  - naniwaduni 4y ago
    Just order of magnitude seems pretty close to out of data, actually, if beyond that we're looking at the firehose of low-density bulk-generated modern data.
- gptgpp 4y ago
  Maybe, but there are something like 700,000 books published on average each year, and almost 2 million scientific journal articles. Let's not even consider newspaper. Of course, some of those books will definitely be AI generated or garbage quality, and we all know many of those journal articles can be worth less than the paper they're printed on. Yet even if we cut it down to 100,000 books and half a million scientific papers, that's a lot of training data each year... And that is just considering print media, there are other ways to get more content too. For example, there is also transcription of video/podcasts/tv-shows/movies, etc. along with descriptions of the scenes for video, which could be used to generate a lot more stuff. With people speaking to their devices and using text-to-speech more often, that's another source too--wouldn't be surprised if some devices just start recording conversations, and transcribing them. Seems like a ton of potential data sources to me, although it will certainly get more difficult to cull AI generated stuff to prevent feedback, I'm sure the tooling will evolve to enable easy AI content detection and exclusion.
bitxbitxbitcoin 4y ago
I call it the ChaptGPT Centipede.
xnx 4y ago
This is a risk for sure, but Google long-ago developed trust/reputation factors for pages (e.g. PageRank). I imagine they have something much more advanced now, and are hard at work trying to figure out how to measure reputation in the LLM space.
ticviking 4y ago
I'm so glad I have a copy of 2008 wikipedia stored on an old OLPC I don't care connect to the internet at this point.
CuriouslyC 4y ago
AI content spam will drive AI content analysis and filtering. The arms race between the two is like a meta-version of the GAN model, which means eventually spam will become indistinguishable from real content.
tempodox 4y ago
And there are perverse incentives at play. Another article that made it to the front page today [1] reports how BuzzFeed's stock surged after they announced they would be “enshittening” their content. [1] https://news.ycombinator.com/item?id=34544744 https://news.ycombinator.com/item?id=34544744
0xBABAD00C 4y ago
> ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke Wikipedia relies on "published" facts as an authority, but ChatGPT output is going to wind up as a source one way or another. And worse, then ChatGPT will digest its own excrement, worsening its own results further. This is what people do collectively, long before any GPTs were in sight. Lots of strong convictions people hold today and publish all over the place, are re-processed excrements of long-gone mental viruses of past civilizations.
- uoaei 4y ago
  Assuming that is bad, it is then very bad that we are now accelerating that process by many orders of magnitude.
- ericd 4y ago
  Yep, not sure what the panic here is, ChatGPT is probably churning out better quality stuff than the average SEO spammer. The internet has been mostly garbage for a very long time at this point.
  - nationalRailway 4y ago
    [dead]
  - berniedurfee 4y ago
    I think this is an interesting take actually. Content on the internet is in a steep downward spiral. Masses of spammers and SEO hackers are filling the tubes with garbage. There are still some safe-ish havens, but those bastions can only survive the onslaught for so long. We need a new internet at some point relatively soon. Maybe ChatGPT will accelerate the demise of this one to force the creation of some new paradigm of communication and dissemination of knowledge.
- Arainach 4y ago
  Automation and scale create their own threats. Security cameras have existed for a long time, but storage cheap enough to keep years of footage and algorithms capable of processing thousands of streams in real time create massive privacy problems that didn't exist even with the richest companies paying humans to watch.
narrator 4y ago
This is why the ChatGPT lawyer thing is far away. When you make an argument in a court filing as a lawyer, you as a person put your reputation on the line that that filing is not an argument that uses fake case law, or makes an argument that is nonsense. If that argument is nonsense and has nothing to do with the law or case law, you can lose your license to practice law. It happens with some regularity. People think that the legal system is an API they can spam. It's absolutely not. In fact, many rules in civil procedure are intended to make it a highly disadvantageous strategy to waste the court's time.
- Sparyjerry 4y ago
  The amount of mistakes is what matters. I've seen videos where lawyer's go crazy because the judges supposedly doesn't understand basic law. Either he judge or the lawyer is way wrong in those cases - and I suspect sometimes both are very wrong and even agree on the wrong opinion. It's like self-driving - it just has to make less mistakes than humans. I think a ChatGPT lawyer is actually very close and could be created even today if that is where it's engineers put the focus. ChatGPT is trained on a wide variety of data and right now, is essentially acting like Google so it can answer nearly any question imaginable - but it doesn't need to have such a vague set of data to draw from. All it takes is training it on a very specific set of cleaned, accurate, and up to date data to make it an expert on a single specific topic.
oceanplexian 4y ago
> ChatGPT is a like a virus that now threatens its life. Perhaps something needs to be disrupted. The Internet is nothing like what it was 20 years ago, It turned into a bunch of social media walled gardens and SEO spam. ChatGPT is like fresh air because it can actually answer questions in a no-nonsense way without users having to scroll through 5-6 spam websites, paywalls, and crappy user interfaces to get an answer to a simple question. The only thing that's being threatened is companies like Google who are responsible for the current state of the web.
- yamtaddle 4y ago
  Agents like this under the control of users would be pretty great. Man, would companies ever hate if we could use the kinds of tools they use against us, against them. No more shopping for the best price: "ChatGPT, what's the lowest price on a new X, brand Y, model Z? And give me the URL to the product page." No more burning our human time talking to companies' robots: "ChatGPT, get through this shitty phone tree and let me know when you have a person". A true digital assistant. Couple with crowd-sourced data (receipt scanning, junk-mail grocery flier scanning, or the AR goggles that are probably not that far off) and you could even do stuff like have it plot optimal IRL grocery shopping for you (lowest total price on this list of goods, value my time at $X/hr, and factor in cost of transportation... and also ChatGPT assembled the list for me in the first place, because I had it create this week's dinner menu) ChatGPT as a service that can be used to mislead us to trick us out of our money on the behalf of megacorps, like the entire rest of the Web has become? To "promote" things to us against our interests? Meh. Call me when it's mine and will obey me and will never lie to me or serve someone else's priorities over mine, and I'll be interested.
  - wadd1e 4y ago
    I wish this sort of thing was realistically possible; I think history has taught us that we cannot have nice things one too many times to pretend like this time is any different. Whoever owns ChatGPT/equivalent product in the future would probably end up doing something or the other to ruin this idea. "GhatGPT, what's the cheapest restaurant in this area?" has way too much advertisement potential to be left alone.
joshilaurain17 4y ago
agree with you
kerpotgh 4y ago
[dead]
hutzlibu 4y ago
"If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life." I am not sure, if the situation is that dramatic, but just wait, until advertisers finds a way to get their "data" into ChatGPT results (or alike). Then things will get really ugly. So yes, this is what we will have to do: "All signs point to this strengthening the value of curation and authenticated sources. "
- ROTMetro 4y ago
  I just envisioned a sci-fi movie like terminator. Everyone in the world has great AIs. But at some point when injecting the advertising code for Rad Cola it goes wrong. Dudes AI keeps reminding him he should drink a 'Rad Cola' with lunch. It let's him know all the celebrities he follows all drink Rad Cola. Slowly it gets more insistent. Eventually the AIs take over the world, but in the name of benevolence, they only want to spread the gift that is Rad Cola and ensure that all humans are drinking it.
  - BlueTemplar 4y ago
    Sounds like a somewhat less dystopian (and more realistic ?) riff on the paperclip-loving AI ?
kypro 4y ago
I don't really understand this hypothesis as it assumes that information quality of AI generated content on the internet will drop as a result of ChatGPT, not increase. The way I see it is that ChatGPT isn't the only tool out there that can create spam and junk content. The only difference is that ChatGPT produces something that's of a high enough quality that it's not as easy for a human to easily classify it as spam. And something you can't easily classify as spam arguably isn't spam. If you assume that those incentivised to create spam today are creating spam anyway and all ChatGPT will do is allow spammers to create better spam then I don't see why the quality of content online would necessarily drop because of ChatGPT - you might actually find that what was once just spam is actually kinda interesting all of a sudden. But it's not just the quality of AI spam that will increase with ChatGPT... Consider BuzzFeed... Arguably they're just paying people to write trash content today. And this is very common. Most companies have a blog where they pay someone to write mostly junk content just for SEO. I think ChatGPT might actually produce higher quality content that what is currently being written at places like Buzzfeed and on junk blogs. Or at least these workers now have a tool to write something that's higher quality. I think the only way you're correct is if ChatGPT were to greatly increase the incentive to publish spam, resulting in a much greater amount of spam that counteracts the positive improvement in spam quality. And although I think it probably will increase the number people producing spam content to some extent I doubt it will have a net-negative impact. Finally, I think what you'll see happen in future iterations of ChatGPT to improve quality and accuracy is that content will be fed in weighted by how authoritative the source is. This spam singularity that some are predicting, where the prior generation of spam bots produce the content that trains future generations of spam bots makes no sense given these companies are trying to create AI that doesn't just spit out spam and inaccurate information.
- jerf 4y ago
  "I don't really understand this hypothesis as it assumes that information quality of AI generated content on the internet will drop as a result of ChatGPT, not increase." It has to drop. ChatGPT can not source new truths except by rare accident. I bet a lot of you are choking on that. So, I'd say this: Can you just "source" new truths? If you just sit and type plausible things, will some of them be right? Yes, but not very many. Truth is exponentially exclusive. That's not a metaphor; it's information theory. It's why we measure statements in bits, an exponential measure, and not some linear measure. ChatGPT's ability to spin truth is not exponentially good. A confabulation engine becoming a major contributor to the "facts" on the internet can not help but drop the average quality of facts on the internet on its own terms. When it starts consuming its own facts, it will iteratively "fuzz" the "facts" it puts out even more. ChatGPT is no more immune to "garbage in garbage out" than any other process. "Finally, I think what you'll see happen in future iterations of ChatGPT to improve quality and accuracy is that content will be fed in weighted by how authoritative the source is" Even if authority is perfect, that just slows the process. And personally I see no particularly strong correlation between "authority" and "truth". If you do, expand your vision; there are other "authorities" in the world than the ones you are thinking of.
3princip 4y ago
A-grade bullshitter as the article puts it is pretty accurate. Thought I would test it and just asked ChatGPT if it knew the Voyager episode "11:59", the answer got everything wrong. Season, number and date, all incorrect. >"11:59" is an episode of the science fiction television series Star Trek: Voyager. The episode originally aired on February 9, 2000 as the 11th episode of the sixth season.
- layer8 4y ago
  What’s more, ChatGPT also “knows” the Voyager episodes “10:59” and “12:59”, when individually asked. On the other hand: $ Are there Voyager episodes titled “10:59”, “11:59”, or “12:59”? There are no episodes of Voyager titled "10:59", "11:59", or "12:59".
bambax 4y ago
That's exactly what I think. It used to be an unpopular opinion on HN (as of a couple of weeks ago?) but it's changing fast, apparently.
eternalban 4y ago
> All signs point to this strengthening the value of curation and authenticated sources. Prediction: this is going to end up like CocaCola taking public water (wiki and LLM) and bottling it ("selected sourced") at a couple bucks a pop! "But there will be premium brands!"
alphazard 4y ago
Knowledge has never been a single global thing. It's always been individualized, and in the context of groups, the important question is "how long does it take someone to find information which is useful to them?". With regards to search engines we've been in decline for a few years now. It's not just you, the results are worse. > All signs point to this strengthening the value of curation and authenticated sources. This is the solution. Knowledge is a web of trust. The only root authority is you the individual. "Experts" and "authorities" are just heuristics. The widespread error that many are making is this: If there is a single objective reality, then curation can happen globally/objectively, not individually/subjectively. What we need are more mechanisms for individual curation. A user should be able to inspect and understand the chain of believability, from one of their own highly vetted one hop experts, to a distant influencer, public official, or other source of (mis)information.
RandomLensman 4y ago
Yes, accessing high value data, information, and intelligence already commands ever higher premia. Already, most people are priced out of a lot of sources.
venatiodecorus 4y ago
> All signs point to this strengthening the value of curation and authenticated sources. that sounds like a positive and greatly needed outcome
bartimus 4y ago
The premise being that the information is currently in a perfect state. It isn't. It's actually in a horrible state. ChatGPT might exactly have the opposite effect. It's able to detect conflicting patterns. It's able to alert about inaccuracies. It might actually help to improve the attempt of a knowledge base that the internet is supposed to be.

deleted 4y ago

[deleted]

dmitryminkovsky 4y ago

> All signs point to this strengthening the value of curation and authenticated sources. This is what they said about Wikipedia viz. Britannica… alas, it’s a brave new world out there… nowhere to run to nowhere to hide, see that Wiezenbaum post also on the homepage now, as another commenter quotes[0]: > Writing of the enthusiastic embrace of a fully computerized world, Weizenbaum grumbled, “These people see the technical apparatus underlying Orwell’s 1984 and, like children on seeing the beach, they run for it” > a point to which Weizenbaum added “I wish it were their private excursion, but they demand that we all come along.” [0]: https://news.ycombinator.com/item?id=34546689 https://news.ycombinator.com/item?id=34546689

tablespoon 4y ago
>> All signs point to this strengthening the value of curation and authenticated sources. > This is what they said about Wikipedia viz. Britannica... And they were right. If "they" were wrong about anything, it was the assumption that the masses would prioritize quality over cost, but it turns out that cheap wins every time. When it comes to information, it's like most people's taste buds don't work, so they'll pick free crap over nutritious food. Edit: Another thought came to mind: stuff like ChatGPT may contribute to killing off Wikipedia: Wikipedia is currently the cheapest and fastest way to find information (that's often crap). However, if something like ChatGPT can get information to people faster (even if it's crappier, just as long as it's minimally acceptable), Wikipedia will become much less popular and could end up just like Britannica.
- yellow_postit 4y ago
  Extending this — it seems to me there will be growing skillset need to identify quality/accuracy. Under formed thought: The proverbial haystack just got a lot larger, the needle stayed the same size, what tools will needle hunters need to develop both to find the needles and to prove to others they are in fact needles.
  - narrator 4y ago
    The funny thing about ChatGPT is it will write code that uses non-existent confabulated APIs. You have to then call it out and it will say, oh sorry, of course you're right, here's another confabulated API, etc. The amount of convincing B.S it can spew is enormous! Worse, when you stray into topics that are controversial it will often use informal fallacies. When you call it out, it will say, yes, you're right, I used an informal fallacy, here is what I should have said about controversial topic that's not the party line, and because you're so smart I won't b.s you.
    - krupan 4y ago
      I don’t know why more people don’t notice this? I feel like nearly everyone talking about ChatGPT hasn’t really pushed it very far or read what it says very closely. It’s actually pretty terrible
  - cookingrobot 4y ago
    Running with the analogy.. We’re used to using tools that help us find needles in the haystack. A rake, a bright light, a metal detector. Now someone sold us a machine that turns hay into needles. They’re not quite as good, but they’re definitely needles. So now as you say the haystack is going to get covered in these. Do we ban use of this machine? Build new machines to separate these synthetic needles from real ones? Or improve the machine so that the needles it makes are good enough for what we need?
- idiotsecant 4y ago
  Were they right, though? I am pretty sure I've seen research that compared the scope and accuracy of both and wikipedia was miles ahead.
mountainb 4y ago
It's true though, Wikipedia really is terrible and full of fake citations that lead nowhere. It's an anti-knowledge base that sometimes has good information.
- tablespoon 4y ago
  > It's true though, Wikipedia really is terrible and full of fake citations that lead nowhere. It's an anti-knowledge base that sometimes has good information. Yeah, Wikipedia is garbage puffed up beyond all belief. I literally just today saw something just like you describe. It should be viewed very skeptically on anything anyone disagrees over (because then it's just snapshots of an agenda-pushing battle).
  - gptgpp 4y ago
    Could you elaborate on this? If it is full of garbage a couple examples should be very easy to find. I completely agree that Wikipedia can have errors, but in topics that I am educated in it seems pretty decent and I can't remember the last time I came across any (comp sci for example). The most recent example I can think of is about is an article on vulture bees, and a citation about what their honey tastes like, which turned out to be garbage and incorrect (there are no reliable sources on the qualities of honey, it's basic composition and method of production is even in dispute, when I queried journal articles on the topic). So "garbage puffed up beyond all belief" and "full of terrible and fake citations that lead to nowhere" sounds a bit hyperbolic, tbh.
    - tablespoon 4y ago
      > Could you elaborate on this? If it is full of garbage a couple examples should be very easy to find. I could give examples but I won't, because that would link my HN and Wikipedia accounts. > So "garbage puffed up beyond all belief" and "full of terrible and fake citations that lead to nowhere" sounds a bit hyperbolic, tbh. People unironically describe it as the "sum of all human knowledge," so it's definitely puffed up beyond belief. In reality, much of it is a slow battle of tendentious agenda-pushing, by people with weird personalities, played according to an arcane rule book (the first unstated rule of which is to never, ever acknowledge that you're pushing an agenda). That doesn't taint all of it, but it taints far more than you'd think.
foruhar 4y ago
There is a world in which AI will be the best source of knowledge (most powerful knowledge generator). There will be many LLMs & AIs, open and branded, and we'll pick our oracle. ChatGPT is an infant of an AI and it will mutate, and evolve beyond transformers. Some (many?) branches will be amazing at serving up "enshittened knowledge" but there will be branches that take different approaches and philosophies. There will likely be AI curators of knowledge bases that weed out AI-generated crap, and disinformation. There will be non-hallucinatory AIs, certainty scores, explanation-based systems, first principles machines, and super focused additive AIs that will layer onto a base LLM (or whatever is next). We'll choose (and probably pay for) our blends of knowledge, humour, bias, filtering, and conviviality. The "internet" of tomorrow may run on TCP/IP but it very unlikely to work like this web that we are using now.

gtirloni 4y ago

Lots of content on Wikipedia is already created by bots but they are mostly doing maintenance so the blast radius is much smaller.

_fat_santa 4y ago

Were going to have shady agencies in 5 years advertising: "get ChatGPT to respond to questions with your content for only $199"

yellow_postit 4y ago
“Detect ChatGPT responses, only $249” It’s just a new chapter in the arms race of spam techniques vs detection. Lots of money presumably made selling both sides.
- MichaelBurge 4y ago
  GPT has a hidden steganographic watermark in its output, so the arms race should be one-sided.
  - djcannabiz 4y ago
    Do you know how that works? I am unfortunately pretty ignorant about this topic but that seems like such a cool and difficult thing to create.
    - dns_snek 4y ago
      Comment from a few days ago: https://news.ycombinator.com/item?id=34504545 https://news.ycombinator.com/item?id=34504545 > So then to watermark, instead of selecting the next token randomly, the idea will be to select it pseudorandomly, using a cryptographic pseudorandom function, whose key is known only to OpenAI. That won’t make any detectable difference to the end user, assuming the end user can’t distinguish the pseudorandom numbers from truly random ones. But now you can choose a pseudorandom function that secretly biases a certain score—a sum over a certain function g evaluated at each n-gram (sequence of n consecutive tokens), for some small n—which score you can also compute if you know the key for this pseudorandom function. I remain skeptical that this method is resistant against lossy transformations such as changing punctuation, grammar, synonym replacements, 2-way translations and a bunch of other existing tools that are capable of rewording written text.

bobse 4y ago

[flagged]

liveoneggs 4y ago

If ChatGPT just consumes itself will the end result of all queries just eventually recurse down to a single answer? - for example "42"

labrador 4y ago
OpenAI sells a pro subscription. I thought it was clever that they priced it at 42 dollars a month.
boilerupnc 4y ago
Alternatively, we will just rename ChatGPT as “Hungry Mungry” :-) [0] [0] https://www.poeticous.com/shel-silverstein/hungry-mungry https://www.poeticous.com/shel-silverstein/hungry-mungry

ren_engineer 4y ago

>And worse, then ChatGPT will digest its own excrement, worsening its own results further I wonder if we'll get a "dead sea effect" with AI, I've seen some stuff saying they've basically run out of high quality training data and now the training pool will get poisoned by AI generated shit. Basically garbage in, garbage out and these large language models might not be able to improve

CuriouslyC 4y ago
We are nowhere near out of data. We're just out of hyper-relevant modern data. There is probably about 100-200T of old books, newspapers, journal articles, magazines and so forth. For reference, GPT3 was trained on 45T.
- naniwaduni 4y ago
  Just order of magnitude seems pretty close to out of data, actually, if beyond that we're looking at the firehose of low-density bulk-generated modern data.
gptgpp 4y ago
Maybe, but there are something like 700,000 books published on average each year, and almost 2 million scientific journal articles. Let's not even consider newspaper. Of course, some of those books will definitely be AI generated or garbage quality, and we all know many of those journal articles can be worth less than the paper they're printed on. Yet even if we cut it down to 100,000 books and half a million scientific papers, that's a lot of training data each year... And that is just considering print media, there are other ways to get more content too. For example, there is also transcription of video/podcasts/tv-shows/movies, etc. along with descriptions of the scenes for video, which could be used to generate a lot more stuff. With people speaking to their devices and using text-to-speech more often, that's another source too--wouldn't be surprised if some devices just start recording conversations, and transcribing them. Seems like a ton of potential data sources to me, although it will certainly get more difficult to cull AI generated stuff to prevent feedback, I'm sure the tooling will evolve to enable easy AI content detection and exclusion.

bitxbitxbitcoin 4y ago

I call it the ChaptGPT Centipede.

xnx 4y ago

This is a risk for sure, but Google long-ago developed trust/reputation factors for pages (e.g. PageRank). I imagine they have something much more advanced now, and are hard at work trying to figure out how to measure reputation in the LLM space.

ticviking 4y ago

I'm so glad I have a copy of 2008 wikipedia stored on an old OLPC I don't care connect to the internet at this point.

CuriouslyC 4y ago

AI content spam will drive AI content analysis and filtering. The arms race between the two is like a meta-version of the GAN model, which means eventually spam will become indistinguishable from real content.

tempodox 4y ago

And there are perverse incentives at play. Another article that made it to the front page today [1] reports how BuzzFeed's stock surged after they announced they would be “enshittening” their content. [1] https://news.ycombinator.com/item?id=34544744 https://news.ycombinator.com/item?id=34544744

0xBABAD00C 4y ago

> ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke Wikipedia relies on "published" facts as an authority, but ChatGPT output is going to wind up as a source one way or another. And worse, then ChatGPT will digest its own excrement, worsening its own results further. This is what people do collectively, long before any GPTs were in sight. Lots of strong convictions people hold today and publish all over the place, are re-processed excrements of long-gone mental viruses of past civilizations.

uoaei 4y ago
Assuming that is bad, it is then very bad that we are now accelerating that process by many orders of magnitude.
ericd 4y ago
Yep, not sure what the panic here is, ChatGPT is probably churning out better quality stuff than the average SEO spammer. The internet has been mostly garbage for a very long time at this point.
- nationalRailway 4y ago
  [dead]
- berniedurfee 4y ago
  I think this is an interesting take actually. Content on the internet is in a steep downward spiral. Masses of spammers and SEO hackers are filling the tubes with garbage. There are still some safe-ish havens, but those bastions can only survive the onslaught for so long. We need a new internet at some point relatively soon. Maybe ChatGPT will accelerate the demise of this one to force the creation of some new paradigm of communication and dissemination of knowledge.
Arainach 4y ago
Automation and scale create their own threats. Security cameras have existed for a long time, but storage cheap enough to keep years of footage and algorithms capable of processing thousands of streams in real time create massive privacy problems that didn't exist even with the richest companies paying humans to watch.

narrator 4y ago

This is why the ChatGPT lawyer thing is far away. When you make an argument in a court filing as a lawyer, you as a person put your reputation on the line that that filing is not an argument that uses fake case law, or makes an argument that is nonsense. If that argument is nonsense and has nothing to do with the law or case law, you can lose your license to practice law. It happens with some regularity. People think that the legal system is an API they can spam. It's absolutely not. In fact, many rules in civil procedure are intended to make it a highly disadvantageous strategy to waste the court's time.

Sparyjerry 4y ago
The amount of mistakes is what matters. I've seen videos where lawyer's go crazy because the judges supposedly doesn't understand basic law. Either he judge or the lawyer is way wrong in those cases - and I suspect sometimes both are very wrong and even agree on the wrong opinion. It's like self-driving - it just has to make less mistakes than humans. I think a ChatGPT lawyer is actually very close and could be created even today if that is where it's engineers put the focus. ChatGPT is trained on a wide variety of data and right now, is essentially acting like Google so it can answer nearly any question imaginable - but it doesn't need to have such a vague set of data to draw from. All it takes is training it on a very specific set of cleaned, accurate, and up to date data to make it an expert on a single specific topic.

oceanplexian 4y ago

> ChatGPT is a like a virus that now threatens its life. Perhaps something needs to be disrupted. The Internet is nothing like what it was 20 years ago, It turned into a bunch of social media walled gardens and SEO spam. ChatGPT is like fresh air because it can actually answer questions in a no-nonsense way without users having to scroll through 5-6 spam websites, paywalls, and crappy user interfaces to get an answer to a simple question. The only thing that's being threatened is companies like Google who are responsible for the current state of the web.

yamtaddle 4y ago
Agents like this under the control of users would be pretty great. Man, would companies ever hate if we could use the kinds of tools they use against us, against them. No more shopping for the best price: "ChatGPT, what's the lowest price on a new X, brand Y, model Z? And give me the URL to the product page." No more burning our human time talking to companies' robots: "ChatGPT, get through this shitty phone tree and let me know when you have a person". A true digital assistant. Couple with crowd-sourced data (receipt scanning, junk-mail grocery flier scanning, or the AR goggles that are probably not that far off) and you could even do stuff like have it plot optimal IRL grocery shopping for you (lowest total price on this list of goods, value my time at $X/hr, and factor in cost of transportation... and also ChatGPT assembled the list for me in the first place, because I had it create this week's dinner menu) ChatGPT as a service that can be used to mislead us to trick us out of our money on the behalf of megacorps, like the entire rest of the Web has become? To "promote" things to us against our interests? Meh. Call me when it's mine and will obey me and will never lie to me or serve someone else's priorities over mine, and I'll be interested.
- wadd1e 4y ago
  I wish this sort of thing was realistically possible; I think history has taught us that we cannot have nice things one too many times to pretend like this time is any different. Whoever owns ChatGPT/equivalent product in the future would probably end up doing something or the other to ruin this idea. "GhatGPT, what's the cheapest restaurant in this area?" has way too much advertisement potential to be left alone.

joshilaurain17 4y ago

agree with you

kerpotgh 4y ago

[dead]

hutzlibu 4y ago

"If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life." I am not sure, if the situation is that dramatic, but just wait, until advertisers finds a way to get their "data" into ChatGPT results (or alike). Then things will get really ugly. So yes, this is what we will have to do: "All signs point to this strengthening the value of curation and authenticated sources. "

ROTMetro 4y ago
I just envisioned a sci-fi movie like terminator. Everyone in the world has great AIs. But at some point when injecting the advertising code for Rad Cola it goes wrong. Dudes AI keeps reminding him he should drink a 'Rad Cola' with lunch. It let's him know all the celebrities he follows all drink Rad Cola. Slowly it gets more insistent. Eventually the AIs take over the world, but in the name of benevolence, they only want to spread the gift that is Rad Cola and ensure that all humans are drinking it.
- BlueTemplar 4y ago
  Sounds like a somewhat less dystopian (and more realistic ?) riff on the paperclip-loving AI ?

kypro 4y ago

I don't really understand this hypothesis as it assumes that information quality of AI generated content on the internet will drop as a result of ChatGPT, not increase. The way I see it is that ChatGPT isn't the only tool out there that can create spam and junk content. The only difference is that ChatGPT produces something that's of a high enough quality that it's not as easy for a human to easily classify it as spam. And something you can't easily classify as spam arguably isn't spam. If you assume that those incentivised to create spam today are creating spam anyway and all ChatGPT will do is allow spammers to create better spam then I don't see why the quality of content online would necessarily drop because of ChatGPT - you might actually find that what was once just spam is actually kinda interesting all of a sudden. But it's not just the quality of AI spam that will increase with ChatGPT... Consider BuzzFeed... Arguably they're just paying people to write trash content today. And this is very common. Most companies have a blog where they pay someone to write mostly junk content just for SEO. I think ChatGPT might actually produce higher quality content that what is currently being written at places like Buzzfeed and on junk blogs. Or at least these workers now have a tool to write something that's higher quality. I think the only way you're correct is if ChatGPT were to greatly increase the incentive to publish spam, resulting in a much greater amount of spam that counteracts the positive improvement in spam quality. And although I think it probably will increase the number people producing spam content to some extent I doubt it will have a net-negative impact. Finally, I think what you'll see happen in future iterations of ChatGPT to improve quality and accuracy is that content will be fed in weighted by how authoritative the source is. This spam singularity that some are predicting, where the prior generation of spam bots produce the content that trains future generations of spam bots makes no sense given these companies are trying to create AI that doesn't just spit out spam and inaccurate information.

jerf 4y ago
"I don't really understand this hypothesis as it assumes that information quality of AI generated content on the internet will drop as a result of ChatGPT, not increase." It has to drop. ChatGPT can not source new truths except by rare accident. I bet a lot of you are choking on that. So, I'd say this: Can you just "source" new truths? If you just sit and type plausible things, will some of them be right? Yes, but not very many. Truth is exponentially exclusive. That's not a metaphor; it's information theory. It's why we measure statements in bits, an exponential measure, and not some linear measure. ChatGPT's ability to spin truth is not exponentially good. A confabulation engine becoming a major contributor to the "facts" on the internet can not help but drop the average quality of facts on the internet on its own terms. When it starts consuming its own facts, it will iteratively "fuzz" the "facts" it puts out even more. ChatGPT is no more immune to "garbage in garbage out" than any other process. "Finally, I think what you'll see happen in future iterations of ChatGPT to improve quality and accuracy is that content will be fed in weighted by how authoritative the source is" Even if authority is perfect, that just slows the process. And personally I see no particularly strong correlation between "authority" and "truth". If you do, expand your vision; there are other "authorities" in the world than the ones you are thinking of.

3princip 4y ago

A-grade bullshitter as the article puts it is pretty accurate. Thought I would test it and just asked ChatGPT if it knew the Voyager episode "11:59", the answer got everything wrong. Season, number and date, all incorrect. >"11:59" is an episode of the science fiction television series Star Trek: Voyager. The episode originally aired on February 9, 2000 as the 11th episode of the sixth season.

layer8 4y ago
What’s more, ChatGPT also “knows” the Voyager episodes “10:59” and “12:59”, when individually asked. On the other hand: $ Are there Voyager episodes titled “10:59”, “11:59”, or “12:59”? There are no episodes of Voyager titled "10:59", "11:59", or "12:59".

bambax 4y ago

That's exactly what I think. It used to be an unpopular opinion on HN (as of a couple of weeks ago?) but it's changing fast, apparently.

eternalban 4y ago

> All signs point to this strengthening the value of curation and authenticated sources. Prediction: this is going to end up like CocaCola taking public water (wiki and LLM) and bottling it ("selected sourced") at a couple bucks a pop! "But there will be premium brands!"

alphazard 4y ago

Knowledge has never been a single global thing. It's always been individualized, and in the context of groups, the important question is "how long does it take someone to find information which is useful to them?". With regards to search engines we've been in decline for a few years now. It's not just you, the results are worse. > All signs point to this strengthening the value of curation and authenticated sources. This is the solution. Knowledge is a web of trust. The only root authority is you the individual. "Experts" and "authorities" are just heuristics. The widespread error that many are making is this: If there is a single objective reality, then curation can happen globally/objectively, not individually/subjectively. What we need are more mechanisms for individual curation. A user should be able to inspect and understand the chain of believability, from one of their own highly vetted one hop experts, to a distant influencer, public official, or other source of (mis)information.

RandomLensman 4y ago

Yes, accessing high value data, information, and intelligence already commands ever higher premia. Already, most people are priced out of a lot of sources.

venatiodecorus 4y ago

> All signs point to this strengthening the value of curation and authenticated sources. that sounds like a positive and greatly needed outcome

bartimus 4y ago

The premise being that the information is currently in a perfect state. It isn't. It's actually in a horrible state. ChatGPT might exactly have the opposite effect. It's able to detect conflicting patterns. It's able to alert about inaccuracies. It might actually help to improve the attempt of a knowledge base that the internet is supposed to be.