16 ms·

ChatGPT and the Enshittening of Knowledge

277 points by bootyfarm 4y ago

oidar 4y ago
dead
- aphroz 4y ago
  HN kiss of death
- dwringer 4y ago
  Almost poignant though.
- orangepurple 4y ago
  What is effectively a static site being served with a database calls for each page view is getting exactly what was coming.
  - alt227 4y ago
    Harsh burn!
  - preordained 4y ago
    You get what you deserve! Justice has been served today, a cold and bitter dish
  - joks 4y ago
    WordPress and its consequences have been a disaster for the human race (jk, it's just hella overused)
- swader999 4y ago
  Title was the best part anyway.
- spyder 4y ago
  ChatGPT got internet access and took it down :-O
arbitrage 4y ago
http://web.archive.org/web/20230127104933/https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/ http://web.archive.org/web/20230127104933/https://castlebrid...
marban 4y ago
https://web.archive.org/web/20230127104933/https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/ https://web.archive.org/web/20230127104933/https://castlebri...
orangepurple 4y ago
https://webcache.googleusercontent.com/search?q=cache:https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/ https://webcache.googleusercontent.com/search?q=cache:https:...
- TSUTiger 4y ago
  Alternatively, https://archive.is/m7fm7 https://archive.is/m7fm7
  - layer8 4y ago
    Or https://web.archive.org/web/20230127104933/https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/ https://web.archive.org/web/20230127104933/https://castlebri...
drumhead 4y ago
Wouldn't it be funny if it was a Chatgpt powered site.
isoprophlex 4y ago
The enshittening of transmitting text over the internet
smrtinsert 4y ago
[flagged]
discreteevent 4y ago
> When we turn the job of creating the first draft of the thing over to ChatGPT we risk removing the “figuring shit out” part of everyone’s career-path. And we get away with that while we still have people who have figured shit out. That's the main problem from my point of view.
- visarga 4y ago
  No, we got to keep up with the shiny toys. Junior is riding on chatGPT like everyone else.
- randcraw 4y ago
  Agreed. The LAST use of ChatGPT should be the first draft. It's clueless about substance or the purpose of the essay, neither what's necessary nor essential. As such, its best use is to flesh out an outline that HAS these things into a essay or narrative that integrates them into a narrative that flows. The trick to that is, how do you take away ChatGPT's inclination to build upon the wrong premises or the wrong final message, and instead confine it to the role of verbally adroit assistant -- so it only expands a skeletal design into an well written final product? Now THAT would be a use for ChatGPT I could get behind. Alas, that wasn't the goal of its designers. By giving it almost total control over the plot line and requiring no fact checks or bibliography, the spewing of unchecked drivel was its only possible mission in life. Congrats, OpenAI. You've automated spam.
williamcotton 4y ago
> Therefore, our safeguard is constrained by time, knowledge, and resources. The problem then is that if that safeguard fails and A-Grade Bullshit is taken by an author using the AI as a research assistant, put into a research paper, book chapter, or online article, and is then cited. That becomes potential training data for the AI if it is released into the world or has the window of its training data set extended. I don't see the value in GPT models for this kind of use. I see the value in like, translation work, or summaries and other transformations, and that GPT works best when in concert with search engines, document embeddings, or knowledge databases like Wolfram Alpha. It's good at predicting more text which is useful in many different kinds of contexts but not in all contexts. It's pretty clear that it is untrustworthy for factual responses like quotes and population counts but that problem goes away if the actual factual data is included in the prompt!
- gtirloni 4y ago
  > knowledge databases like Wolfram Alpha Isn't that where ChatGPT would actually be worse overall? It's been shown to give semi-accurate data very often while sounding very confident about it. When people require precise answers and it gives almost correct answer, it seems the general feeling is of amazement. I don't share that feeling. I hope nobody is using it for serious work without a human vetting the output.
  - Xeophon 4y ago
    OP mentions GPT in concert with WolframAlpha, which has already been implemented[0], showing a symbiosis between generation of text with proven knowledge. https://news.ycombinator.com/item?id=34422122 https://news.ycombinator.com/item?id=34422122
  - smolder 4y ago
    They said it works best in concert with things like Wolfram Alpha, not as a replacement.
lubujackson 4y ago
If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life. This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal. ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke Wikipedia relies on "published" facts as an authority, but ChatGPT output is going to wind up as a source one way or another. And worse, then ChatGPT will digest its own excrement, worsening its own results further. All signs point to this strengthening the value of curation and authenticated sources.
- deleted 4y ago
  [deleted]
- dmitryminkovsky 4y ago
  > All signs point to this strengthening the value of curation and authenticated sources. This is what they said about Wikipedia viz. Britannica… alas, it’s a brave new world out there… nowhere to run to nowhere to hide, see that Wiezenbaum post also on the homepage now, as another commenter quotes[0]: > Writing of the enthusiastic embrace of a fully computerized world, Weizenbaum grumbled, “These people see the technical apparatus underlying Orwell’s 1984 and, like children on seeing the beach, they run for it” > a point to which Weizenbaum added “I wish it were their private excursion, but they demand that we all come along.” [0]: https://news.ycombinator.com/item?id=34546689 https://news.ycombinator.com/item?id=34546689
  - tablespoon 4y ago
    >> All signs point to this strengthening the value of curation and authenticated sources. > This is what they said about Wikipedia viz. Britannica... And they were right. If "they" were wrong about anything, it was the assumption that the masses would prioritize quality over cost, but it turns out that cheap wins every time. When it comes to information, it's like most people's taste buds don't work, so they'll pick free crap over nutritious food. Edit: Another thought came to mind: stuff like ChatGPT may contribute to killing off Wikipedia: Wikipedia is currently the cheapest and fastest way to find information (that's often crap). However, if something like ChatGPT can get information to people faster (even if it's crappier, just as long as it's minimally acceptable), Wikipedia will become much less popular and could end up just like Britannica.
Phenomenit 4y ago
Saying middle of the road shit and adding bullshit sounds like every politician.
wprl 4y ago
> AI can experience the same rabbit-hole effect as your idiot cousin who has watched too many of the algorithmically suggested videos on Youtube and now believes that aliens conspired with a time travelling Elvis to kill Hitler using a bullet made of Yeti teeth. And they only went online looking for TellyTubbies videos. LOL Thank you, I needed a good laugh!
- mistermann 4y ago
  Even funnier: ~all humans suffer from the phenomenon, and are similarly unaware of their own personal ignorance. People view reality through a relative lens, the view is more pleasant than an absolute lens.
bitcurious 4y ago
ChatGPT is the bell announcing the death of anonymity on the web. Anonymous content will be generated content, real discussion will shift to gated communities with either paywalls or proofs of identity.
- ru552 4y ago
  GeoCities banner communities are back baby! Webring where you at?
  - sillysaurusx 4y ago
    I'll bring the ff7 walkthrough! Someone grab a Runescape login.
- CuriouslyC 4y ago
  That's how it is now, it'll just be more so. You might trust somebody here, but do you trust your average Amazon reviews?
- add-sub-mul-div 4y ago
  But we also can no longer trust that information published by a verified human was actually created by a human.
- supriyo-biswas 4y ago
  I see most replies are missing the point. The text that ChatGPT generates is very human like, so companies will be forced to adopt a mitigation strategy by tying users to a government ID. Such moves would also be bolstered by the government of various nations, which have already been asking for something like this.
- epistemer 4y ago
  Or what we think of today as social media is basically on its last legs. Your AI assistant will just take up so much of your time that you won't be bothered with what other people think. Communication with other humans online right now is largely a form of entertainment. The bar is just so low to be more entertained by future AI assistants than a random human. I am personally certain that I will not be reading this board at some point in the future because the AI assistant will just have far more profound things to say than what random people are going to post on here. It is not like the assistant only has to be one personality too. Could be a whole group of assistants having a group chat. Lamda can probably do this right now considering it can talk as a planet or national park.
lsy 4y ago
The TLDR of the article is essentially that any time a ChatGPT conversation strays to the factual (in this case a list of data ethics books), the quality of the data (whether those books exist and are written by the stated authors) is extremely low. Ultimately this is where ChatGPT and similar projects seem most likely to hit a wall. Text generators are statistical machines, but the relationship between statistics and truth is merely probabilistic and not identical. There is no way to train ChatGPT to say things that are true, because the number of true and false statements in the world are far too numerous and too specific to ever reliably delineate. At best, we will be stuck fact-checking ChatGPT using enormously complex knowledge bases and asking it to try again until something doesn't raise a red flag, at which point it may still not be true. For most business purposes this is unacceptable. We will have machines that can generate untruths in a way that contravene common sense, so long as those untruths don't obviously contradict whatever local database is being consulted for grounding. And the mechanisms required to re-ground the GPT output even to that unhappy state will be so complex in their own right that businesses might as well just implement a similarly complex system to handle the issue directly, without calling out to a complex, licensed GPT. Not sure whether there is a way around this.
- visarga 4y ago
  One approach to fixing factual errors is to use two rounds of LLM interaction. I forgot the name of the paper. Say you ask "What is the height of Everest?" 1. generate an answer in closed-book model, with the LLM: "The height of Everest is 8723m" = candidate_answer 2. search your references with candidate_answer, find: "At 8,849 meters (29,032 feet), Everest is considered the tallest point on Earth" = search_snippet 3. do a second pass to rewrite the answer with the LLM using search_snippet in the prompt Basically, the incorrect phrase candidate_answer is very good at matching the correct answer in search engines. It is like a template tuned to extract the desired facts. A search engine check could also verify when the searched fact has no references.
  - ptx 4y ago
    How would you apply this approach to the article's example, the response to the question "Who is Daragh O Brien from Castlebridge", where I count at least 15 separate statements of fact? Should we research all of them and try again with a big table of hits and misses from the first attempt? Seems like a lot of work. Also: Is the generated response really "very good at matching the correct answer"? I suppose it would work because the search engine's language processing cancels out the useless parts that were generated by the AI (sort of a "human ABI", analogous to the C ABI?) but a more direct query (e.g. "height of everest") would likely be just as effective.
patapong 4y ago
It seems like a lot of effort will be spent to ensure that content is not generated by an AI system. This seems very tricky to enforce - a stopgap measure could be to require any person to upload a video where they say their comment before being able to post it to a website. Of course, along with the regression of user experience, such a thing could be tricked by reading an AI generated text, and eventually with deepfakes. Is it viable to constrain conversation to human voices this way or is the cat out of the bag? Is it desirable? Should we just go back to in-person conversations?
- fabianhjr 4y ago
  Web of Trust will be back, your network will depend on your criteria for trust. Monoliths / Centralized services are the ones that can't adapt to the coming technology. https://wiki.p2pfoundation.net/Counter-Anti-Disintermediation https://wiki.p2pfoundation.net/Counter-Anti-Disintermediatio... Great implementation example: https://ssbc.github.io/scuttlebutt-protocol-guide/#follow-graph https://ssbc.github.io/scuttlebutt-protocol-guide/#follow-gr...
erksa 4y ago
ChatGPT just proves how critical thinking is and will be an extremely valuable trait for us to embrace and understand.
moomoo11 4y ago
This is why you need to use chatGPT now to launch companies before people inevitably ruin things using chatGPT. chatGPT by itself is amazing. How people (keyword) will use is going to determine outcomes.
somecompanyguy 4y ago
[flagged]
clbrmbr 4y ago
GPT is a language model, not an oracle. So my first take is that people querying it for research are doing it wrong. Then again, if there’s a large economic incentive to use it in that way, we are very well may end up with the kind of feedback loop that the author describes.
- james-revisoai 4y ago
  Behind this thinking - an economic incentive to access knowledge and answers quicker that is inevitable - we have built a tool that verifies statements using source materials. You have to put the legwork in to upload your PDFs/web pages/videos, but once you do, you can be confident in the answers. If it can't verify, it just won't answer/tickmark check the answers (happens 16% of the time... and ... always for maths). This is a feedback loop stopper, in the sense of only relying on your documents as the base, and being able to operate entirely without OpenAI (still though using other GPT models) It's Fragen.co.uk - we believe that more answers formerly missed by CTRL+F will be found with this technology, than false answers taken as true. And if that's true, you are enbettering knowledge. And if not, you're enshittening it slower than the higher-hallucinating alternatives.
- roughly 4y ago
  You’re absolutely right on all counts, and yet, people Are doing it wrong, and will increasingly do so, because there Are large economic incentives for using it that way.
steve76 4y ago
[dead]
waselighis 4y ago
I agree with this article, but I feel the author is only scratching the surface of all the harm ChatGPT and other generative AI systems are going to cause. We just have to look at the current state of the internet today, with things like SEO spam, misinformation, social media bots, fake reviews, and so forth. The people responsible for these are already taking advantage of generative AI and it's going to exacerbate these problems to their extremes. OpenAI's PR likes to preach about ethical AI, but it's a total farce. There's no way that OpenAI isn't fully aware of all the harms their AI is going to cause. I only see one of two possible ultimate outcomes. AI like OpenGPT is going to flood the internet with false or inaccurate information, that I have no doubt. The first outcome I foresee is we simply lose all trust in the vast majority of information sources. The other outcome is we're going to have to sacrifice our privacy and anonymity online in order to distinguish real information from real people and not the AI bot generated nonsense. For example, Social Media may likely require you to prove your real identity, and will only allow you to use trusted hardware devices (i.e. phone or tablet) to post things to your social media account. It'll become increasingly difficult to share information anonymously online in a way that can be trusted. Any information that can't be tied to a verifiable source (a real person) is immediately going to be deemed untrustworthy.
- waselighis 4y ago
  To elaborate a little more, multiple attempts to separate real people from bots have either failed or are soon to be defeated. Captchas worked for a while, but AI systems will likely be able to easily defeat Captchas within the next 5 years. Some websites require a phone number, but it's not difficult to get a fake or temporary phone number, if you know how. Deepfakes mean you can't even trust a real-time stream from a person you can see and hear. Eventually, we're going to have to completely sacrifice our privacy and anonymity online in order to prove we're not a bot.
- commandlinefan 4y ago
  > we simply lose all trust in the vast majority of information sources Did you ever have it though? I'm old enough to remember newspapers, and I always took everything I read in them with a grain of salt and I'm absolutely positive they were written by a human being. (disclaimer: didn't read the article because it's down right now).
henry_viii 4y ago
https://archive.is/m7fm7 https://archive.is/m7fm7
IanDrake 4y ago
[dead]
jerpint 4y ago
I see chatGPT as a first iteration, and it’s glaringly obvious what works and what doesn’t. I’m hopeful future iterations (not just from openAI) will make it overall a net improvement, things like adding real sources and better handling things it doesn’t know
markus_zhang 4y ago
From my experiments, ChatGPT is at least valuable by generating boilerplate code for scripting languages. I believe once connected to enterprise codebase its quality and completion can dramatically increase.
- ornornor 4y ago
  > enterprise codebase > quality increase We must have seen very different enterprise code.
  - shadowtree 4y ago
    It's called garbage collection for a reason. ;)
  - markus_zhang 4y ago
    I think it's going to be mostly more complete code which does improve its quality. For example I generated a piece of code to query API but without authentication or other information the code is not complete.
- ptx 4y ago
  Aren't scripting languages meant to reduce boilerplate code? Maybe you just need a better API and not an AI?
  - markus_zhang 4y ago
    You still need boilerplate for scripting language though. Like querying an endpoint in Python with a rationing limit in mind. Things like that, not exactly difficult but still need some lines of code.
harryvederci 4y ago
> It reminds me of how Clippy used to promise to help me write a letter back in the 1990s. But Clippy was a dickhead and he really didn’t have any expertise in writing letters. Legend.
labrador 4y ago
I think the bigger problem that it calls into question who we are talking to online. It's going to be a piece of cake to run our thoughts thru an AI before copying the output with a few adjustments and passing it off as our own.
userabchn 4y ago
A friend is a member of quite a few Facebook groups. She doubts my belief that these groups already contain ChatGPT-like influence operation chatbots, but was recently surprised when in one of the groups several participants discussed a photograph of a typical large chest of drawers while referring to it as a "locker". Maybe some people do refer to such furniture as a locker, or maybe the first person did for some reason and the others then just followed along to avoid confusion, but it made me wonder whether it was an example of a group of chatbots talking to each other and making up a nonsense name for the item. (She still doesn't believe me.)
- phphphphp 4y ago
  this is a long-standing meme: any behaviour you don’t understand on the internet must be some sort of sophisticated operation by a nefarious third-party network. The question you should always start with is why and if you can’t come up with a why then you should start from a more neutral position on the behaviour. Spam isn’t free to engage in, especially on platforms like Facebook, and spam is rarely an irrational money-losing act, it’s just profit generation on the wrong side of our view of what is ethical. Why would someone spend money to unleash chat-bots on a Facebook group to discuss a piece of furniture? If you can’t explain why, your theory is probably wrong. I would be absolutely shocked if it’s anything more than just some real people who love furniture and use a localised phrase that you don’t recognise.
  - pugworthy 4y ago
    You see the same kind of thing in some online multiplayer games. Players acting differently are seen as bots. Same for players who low skill or go AFK. That said an interesting advance on real game bots would be ones which use AI to chat and respond to other players actions or messages.
  - userabchn 4y ago
    But I can explain why: they need to blend-in to build trust with other group participants and so become more persuasive when they are trying to influence, and also to avoid bot detection. The influence might also be much more subtle than talking about political topics, such as just being continuously negative or argumentative to bring down the moods of the targeted people.
pawelduda 4y ago
Internet often feels it's getting rotten with SEO spam with very little substance already. Now the flood gates are open to generate more shit in a much better wrapping. And it's going to take more effort to check its quality than to produce it. Unless we get better tools to do that - but text has more dimensions to process/understand as opposed to for example AI generated images which (for now) can be told apart by glitches, or trails detectable by software.
- x86x87 4y ago
  What was the saying? It takes 10x more effort to refute BS than it takes to produce it? So if we make generating BS easier and that BS is fed for generating more BS we're going to get to a point where everything is BS. I call this the BS singularity!
  - forgetfulness 4y ago
    We have finally automated the production of half-truth to a scale never before envisioned, maybe the limit will be the capacity for humans to absorb bullshit, it's a testament to how revolutionary this technology is that we'll be able to quantify just how much bullshit people can process over time, and if supply will finally outstrip demand for it.
sschueller 4y ago
Can't we add a feature to ChatGPT that goes as follows : "Did you write this and if so when?" That would solve like 80% of issues.
- kerpotgh 4y ago
  [dead]
- MachineBurning 4y ago
  If I understand you correctly, this doesn't work because ChatGPT isn't the only language model in the world. It's only the most popular at this point in time.
jesterman 4y ago
403 Forbidden?
fnfontana 4y ago
There's a famous quotes from Bill Gates that is "My children will have computers, yes, but before they will have books. Without books, without reading, our children will be unable to write, including their own story." On my personal experience, ChatGPT is a precious tool to refine knowledge and thinking, but it depends on how it's used. The same painting brush can be used to paint Monalisa or a meaningless blurred thing. It depends on how it's used. To use ChatGPT and other AI tools properly, we need to build a solid foundation of knowledge and critical thinking skills at first. Then triangulate the answers given by the AI with other sources, our own previous knowledge along with new evidences through experimentation. Since the brain urges for cognitive relief, most people will be lazy, while just a few will extract the true potential of ChatGPT.
personjerry 4y ago
This is redirecting to Google for me
voytec 4y ago
This post is unavailable in a... peculiar way: $ curl -I https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/ HTTP/2 302 server: nginx date: Fri, 27 Jan 2023 16:53:41 GMT content-type: text/html; charset=iso-8859-1 content-length: 282 location: http://google.com If this is hosting provider's response to HN death hug, it's very poorly executed.
- jagged-chisel 4y ago
  Try setting your user agent to something that looks like a browser.
  - david927 4y ago
    I'm using a browser and also getting redirected to Google.
    - stuart78 4y ago
      Same here in vanilla Chrome and Safari.
  - voytec 4y ago
    My Firefox with residential IP got redirected to google and that's when I checked `curl -I` (-I does HEAD request but browser did the usual GET request).
  - metafunctor 4y ago
    The same happens with any user agent.
- rkimb 4y ago
  Google's cache: https://webcache.googleusercontent.com/search?q=cache:iAFjnU74twwJ:https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/&cd=1&hl=en&ct=clnk&gl=us https://webcache.googleusercontent.com/search?q=cache:iAFjnU...
- jabo 4y ago
_just7_ 4y ago
Not really sure if this is supposed to be the joke, but your link just redirects me to google
- woah 4y ago
  So many people on HN just debate based on the title of an article that the actual writing of the content is a superfluous waste of time
hasmanean 4y ago
Pseudo intellectual knowledge. It’s a problem that has been with humans ever since we began communicating. When there is a conflict Mountain goats just knock heads against each other (presumably so that both goats suffer concussions … until the one with the weaker knowledge graph suffers a loss of memory). No such process in AI.
jdelman 4y ago
Not sure if it's some kind of joke or the author doesn't want their article on Hacker News, but the URL is currently 302'd to point to Google.com.
carrja99 4y ago
Saw a silly “recommended” video on YouTube of Jordan Peterson talking about ChatGPT. “It knows everything. It’s smart. It’s smarter than you!” What a crackpot, talking about stuff he doesn’t understand for clicks. Sadly it works for the lowest common denominator
renewiltord 4y ago
If you're using Wordpress, use one of the WP caching plugins. It's worth it. Guessing based off the `wp_footer` class you have in there. Much better solution than 302 to Google.
khiner 4y ago
The test use case of constructing a bio for yourself, hoping it accurately summarizes all the extremely low sample size data it happens to have of you in its web crawled training data, seems like one of the worst possible use cases for ChatGPT. It’s right there on the main page that it’s not to be trusted with factual information like this. ChatGPT will hallucinate details. It’s remarkable to me actually how often it will refuse to hallucinate, given that’s basically what its job is. I don’t find it interesting to find all these edge cases where ChatGPT produces empirically false data. It doesn’t even have the ability to look things up! If I were the OP and wanted help writing my bio, I would first write the draft myself, then use ChatGPT to help with the editing, prose, grammar, style, etc. You are the expert on the factual details of your own life, and if you’re surprised that a language model trained on web crawled data ending in 2018 is not, then all I’ve learned is that you don’t know much about what this thing is. I also don’t buy these arguments of the form, 1. OpenAI’s public ChatGPT app is often factually inaccurate. 2. ChatGPT is an example of a ML system bootstrapped on web crawled text data. 4. Thus, the long term future of our distributed text-encoded knowledge base will be a cesspool of useless gobbledygook. ChatGPT is a step forward in generative language modeling. It doesn’t preclude the development of other future systems to help us verify factual accuracy of claims, likely much better than humans can. We’ll be ok gang:)
- mkmk3 4y ago
  I feel like the 3 youre missing there is something along the lines of "people enjoy social validation and internet points, to the extent that pretty shit content thats low effort is something we enjoy generating"
  - khiner 4y ago
    That is true, 3 would help steel my strawman. I agree that we’ll increasingly have capabilities to generate and publish garbage that’s _just_ good enough to generate clicks, and incentives to do this. In addition, I think we’ll increasingly have tools to produce content that is much more rich, imaginative, insightful, and factually correct in our future. Some more interesting questions to me are then: What will the ratio be? How will that ratio compare with what we see today? How easily will I be able to identify misinformation when I care about factual accuracy (again, compared with today)? How easily will I be able to avoid the garbage, vs find the good stuff?
scandox 4y ago
Article scrubbed including from Google cache. Looks like maybe someone didn't get a lot of corporate love for this.
fnordpiglet 4y ago
I think an easy way to identify transformative technology is how strong people’s reaction is against it. I remember similar freak outs about the graphical World Wide Web, smart phone, etc. Somehow things are not quite so bad and considerably better than predicted, while still having negative side effects. Maybe an outcome will be knowledge will be better structured, ala Wolfram Alpha, and just depending on random text documents to encode knowledge won’t be as much a thing - similar to how we no longer use oral traditions to encode and disseminate knowledge. Who knows. But I doubt AI will fundamentally destroy society or knowledge or the internet or whatever. We will adapt, not just to preserve the way we did things, but to use the new things in a way that controls its side effects. It’ll of course be imperfect and some side effects can’t be controlled for. Some things just will cease to be a thing, like phone booths. While you can never have a bill and teds with a smartphone, it’s actually not the end of things. It’s just different. That’s how it works my friend. Life is impermanent, everything changes at all times, you can never recreate the past, and our suffering stems from our inability to let go of the way things are. Edit: I hate to bring it up, but maybe this is what the semantic web was waiting for.
- banannaise 4y ago
  > I think an easy way to identify transformative technology is how strong people’s reaction is against it. There is also strong negative reaction to technology that does incredible damage to society. This is just an excuse to ignore people pointing out issues.
  - fnordpiglet 4y ago
    No, I don’t think so. I think I discussed that we would tackle the issues and some would be satisfied, some unresolved, and some ones we missed will emerge. I’m not saying the hyperbole cycle isn’t necessary or useful, I’m saying it indicates something significant. We couldn’t possibly resolve the issues identified if they weren’t identified to begin with. But I also think the bold predictions of what will go wrong are usually bold predictions of what could go wrong, and the very act of predicting it makes it less likely to actually occur. Finally, on a personal level getting bent out of shape about these things is pointless. We can identify challenges with new tech without it becoming a frantic exercise of hang wringing. Intentionally addressing issues knowing it’s happening regardless of your anxieties is a much more useful expenditure of energy, even if it doesn’t boost your blog/social media/impressions quite as much as hyperbolic hyperventilation.
spelunker 4y ago
The link redirects to google? Is that the joke?
- bogwog 4y ago
  archive.org has the original article: https://web.archive.org/web/20230127104933/https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/ https://web.archive.org/web/20230127104933/https://castlebri...
osigurdson 4y ago
Strange, when I click on the link, it goes directly to google.com. I think ChatGPT is great but it does have a tendency to make things up. All output needs to be taken with a grain of salt. So what? It is still a useful tool.
manytree5 4y ago
The internet has been groaning under the load of bots, automation and distrust for a while now anyway: in my mind what is needed is a system for proving constituent parts of your identity which is also privacy preserving. I'm thinking of the work around DID re: rebooting the web of trust, and the W3C working group If you can wield an ephemeral and verifiable token which asserts your humanness, hierarchically derived from a certificate privately issued to you by one of hopefully a healthy number of well known authorities, you can participate within circles of the human web without revealing anything else about your identity (name, a/s/l, etc) But, outside of this enclave you can also interact with AI without revealing whether in fact you too are AI. In this way the internet can develop a more sensitive immune system where it is difficult for human systems to be perverted by Sybil attacks.
- brap 4y ago
  What’s stopping you from using this token with a bot?
  - 93po 4y ago
    I disagree with top comment a bit - I think we need tokens that are authenticated to an identity much like SSLs certs. If you choose to use your token to post AI generated crap on the internet, I can choose to block it. I can also choose to subscribe to community managed block lists. I think content on the internet needs to be reputation based, and if you choose to read content that isn't signed by a real person's identity, then it's sort of on you if you wind up reading bot written crap. I think community managed lists would also be a great way to filter out people with annoying or problematic communication styles or opinions. If you publicly support Jordan Peterson, I can add you to the list of Jordan Peterson supporters, and as a community we'd never have to read anything you post anywhere, ever.
    - dennis_jeeves1 4y ago
      > If you publicly support Jordan Peterson Do I have to reveal my actual name?
ulizzle 4y ago
Clicking on the link is booting me out to Google. Is that just me?
- jagged-chisel 4y ago
  I got “forbidden”
ryzvonusef 4y ago
Man everyone is on the ChatGPT bandwagon, even Al Qaeda is on it: htthttps://twitter.com/khorasandiary/status/1618682313528975360 https://twitter.com/khorasandiary/status/1618682313528975360
DebtDeflation 4y ago
It redirects to Google now. Archive link: https://web.archive.org/web/20230127104933/https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/ https://web.archive.org/web/20230127104933/https://castlebri...
carapace 4y ago
cf. xkcd/810 "Constructive" https://xkcd.com/810/ https://xkcd.com/810/ - - - - Schimidhuber says that his task is "to create an automatic scientist, and then retire." Not long ago it was mildly insulting for someone to suggest that your writing sounded like the output of GPT, already (for most of us) it has become mildly complementary. GPT may be hallucinating, but it writes well. So what if you connect it to empirical feedback? Make it a scientist. I don't think it will be long before these machines think better than us (at least most of us.) The thing is, they don't have glands. They don't have historical trauma in their flesh. What I'm getting at is that these machines won't have human hangups. They won't be neurotic. They won't be HAL. They will be sane. The interesting thing here is that life on Earth is actually pretty straightforward. In video game terms Earth is very easy and all the walkthroughs and cheats are known and available. The only reason it seems hard is that people are kinda messed up. Once we have sane intelligent machines to make our decisions for us things should get better rapidly.
- burnished 4y ago
  Chat gpt is a terrible writer. At least, every example I have seen has had poor information density and was generally worse than the input prompts people used when they wanted to 'fluff up' a statement or opinion. I would be genuinely interested in a counter example. Side note - your main point seems overly optimistic. How would we recognize, value, or design a 'sane' machine when by your argument we dont have access to sanity? Seems way more likely to generate a distillation of our neuroses.
- AlphaOne1 4y ago
  Your comment reminded me about a discussion G.K Chesterton has in "Orthodoxy" on madness. My take is that when facts and information is divorced from experience we risk being unmoored from reality: > If you argue with a madman, it is extremely probable that you will get the worst for it; for in many ways his mind moves all the quicker for not being delayed by the things that go with good judgement. He is not hampered by a sense of humor or by charity, or by the dumb certainties of experience. He is the more logical for losing certain sane affections. Indeed the common phrase for insanity is in this respect a misleading one. The madman is not the man who has lost his reason. The madman is the man who has lost everything except his reason. [0] https://www.pagebypagebooks.com/Gilbert_K_Chesterton/Orthodoxy/The_Maniac_p3.html https://www.pagebypagebooks.com/Gilbert_K_Chesterton/Orthodo...
throwaway589275 4y ago
Prediction: OpenAI/GPT (or Google's DeepMind, if you prefer) is going to cause mass unemployment in certain sectors of the economy (for example, graphic designers, copywriters, and many IT professionals) long before it ever addresses any of the fundamental problems that currently substantially reduce human quality and quantity of life, like aging or cancer. In the near-term, AI will just accelerate the winner-take-all nature of our economy.
- throwaway589275 4y ago
  Also, this isn't just an idle prediction on my part; for what it's worth, I've started substantially shifting my net-worth towards the entities I think will be the winners.
adamc 4y ago
When I click the header link I get redirected to google?
gscott 4y ago
ChatGPT is sort of the reverse of Grammerly. I know little about proper grammer so I put most everything into Grammerly and it produces a lot of suggestions and I take them because they sound good and I don't really know any better. I assume it is right. ChatGPT produces text that sounds good and you have to update that text with your facts and remove or make updates to bring it to your own standard.
danielmorozoff 4y ago
Link is redirecting to google.com
nluken 4y ago
Nice read, and well argued. Tangentially related: does anyone else feel like ChatGPT's default voice is just... super bland? I read the "op-ed" the author linked in the article (https://www.thecrimson.com/article/2022/12/9/porios-chatgpt-wrote-this-op-ed/ https://www.thecrimson.com/article/2022/12/9/porios-chatgpt-...), and the writing style lacks the kind of variation in structure (or content, really) that keeps opinion pieces interesting. It feels like a polished middle-school essay.
superfishal 4y ago
Chat GPT ain't takin your job, the Dev that knows how to use it as a tool is taking your job!
low_tech_punk 4y ago
A text-only cached version: http://webcache.googleusercontent.com/search?q=cache:iAFjnU74twwJ:https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge&strip=1&vwsrc=0 http://webcache.googleusercontent.com/search?q=cache:iAFjnU7...
jfengel 4y ago
There's also an article about the "enshittening" of TikTok. Boy, I hope "enshittening" isn't gonna be 2023's Word of the Year. The concept of "stuff gets worse" is really old, and a buzzword for it just feels tiresome. It's only January, so there's plenty of time for this to fade. And I suppose it can't be worse than "gaslighting", a word that was plenty relevant in 2022 but was hardly specific to last year. But surely we can do better.
bjornsing 4y ago
Why am I redirected to http://google.com http://google.com when I press the link? :)
pphysch 4y ago
ChatGPT doesn't bring anything new to the table in terms of bullshit and misinformation. The average person already consumes a low-signal, high-noise info diet, since the advent of modern propaganda in the 1920s.
- Valgrim 4y ago
  Are you really suggesting that the average person had access to better information before the 1920s? For real?
  - pphysch 4y ago
    No, I'm not suggesting that. 1920s is merely when the current paradigm began, and ChatGPT is not ushering a new one.
jiggyjace 4y ago
I really disagree with this perspective. In history the democratization of books with the printing press, the entertainment & education industry with YouTube and MOOCs, and now whatever this next movement is with ChatGPT. Let's just face it, it's useless to predict how innovation will impact the future society, but you can always choose to find its most productive uses for it yourself in the meantime. Stop trying to brute force your outdated perspective on how disruptive innovations will impact other people. Unless you have a sample size of the entire ChatGPT audience, or the entire internet, you're going to have a rough time.
- CatWChainsaw 4y ago
  In theory knowledge is more democratized but so is misinformation. Ignore the Big Issue from the last two years and see how tide pod challenges have improved society. /s
AlbertCory 4y ago
I think it has to be a lot closer to Wikepedia's level of accuracy (which is not 100%, or even close) before anyone cites it as fact. Yesterday I asked it "what are the chords to 'A Foggy Day in London Town'": it made up a whole new Gershwin song! ====== "A Foggy Day in London Town" is a song with music by George Gershwin and lyrics by Ira Gershwin. The chords for the song are: Verse: C C7 F C C7 F C C7 Foggy, foggy dew, you can't see a thing F Fm C C7 F C C7 F The visibility is zero, and my eyes are dim Chorus: C C7 F C C7 F C C7 It's a foggy day in London Town F Fm C C7 F C C7 F The buildings look one hundred feet tall C C7 F C C7 F C C7 It's a foggy day in London Town F Fm C C7 F C C7 F And the people look like ants at all Bridge: Am D7 G C C7 F C C7 Just like the feeling you get when you're down Am D7 G C C7 F C C7 The sun can't break through the London Town
dr_dshiv 4y ago
Raise your hand if you think the singularity is happening. (ie, within 5 years till AGI)
jacobsenscott 4y ago
I started to read the article, and it said chatgpt passed the bar, and when you actually get to that paper - chat gpt did not pass the bar. It did okay on the multiple choice section though.
Applejinx 4y ago
Think of it like this. AI is the Mandela Effect made real. If the preponderance of human writing/creation/art says the sun revolves around the earth, then AI not only nails that down as infallible, but INFERS from it and comes up with new justification for stuff that it made up. Mandela Effect is all about what 'should' be the truth. Some of the most interesting parts of reality are where bits of reality don't neatly fit into the narrative. AI exploiting its own inferences and magnifying the Mandela Effect means it steamrollers inconvenient realities on the grounds that 'most people' wouldn't think so. But we don't even think of AI as 'most people', we imagine it as some kind of super-set of humanity, the ultimately wise and skilled overseer. Boy, is that a mistake.
diversifiedIp 4y ago
I have around 20 years do all the SEO stuff that is righfully hated and for years now I have been both excited and dread the public availability of simple to use AI. My worry was it would completly destroy the SEO industry and indeed, SEO itself. The kind of things I do (building thousands of content sites to feed a master money site) are super niche and require millions to get into the game, all it will take is a handfull of non droped high authority domains from auction and a decent server to churn out crap, scale that on a few hundred domains and it is game over.
patientplatypus 4y ago
[dead]
scblock 4y ago
The combination of using an "authoritative" voice and being completely and utterly full of shit makes for some truly ridiculous results. Users need to be extremely careful with these tools. And no, expecting everyone to be knowledgeable to independently fact check some BS published based on a generative model is not a reasonable position. That burden needs to be held by the publisher. Unfortunately with the current state of machine generated content and SEO spam on the internet, that is not likely to happen as there is no real incentive to do so. And it's not just being confidently wrong about a single person like this author noted, it's being confidently wrong about anything and everything. The other day I asked ChatGPT about the reasons behind the connection between Italian filmmakers and Western movies. It gave a few reasons that were semi-plausible if not easily verifiable, and then made the bold claim that Italy and the United States were great allies in World War 2. It said this confidently and with a tone of authority. And while we may all go "ha ha that's wrong" now, WW2 was nearly 80 years ago and not everyone has a strong history background. There aren't that many people left with direct experience and that number shrinks every day. With growing distrust in actual authorities on matters and rejection of observable fact--based at least in part on massive piles of internet BS--this is pretty concerning to me.
AlphaOne1 4y ago
Maybe I'm being too simplistic, but I think going forward critical thinking as well as an understanding of what to value is going to be important for future generations. Something that computers will never be good at doing is 'understanding' what is valuable in life since many times this cannot be measured. In essence, a healthy understanding of the three transcendentals the Good the True, and the Beautiful will be essential to keep our humanity. ChatGPT can perform many amazing skills but lacks the ability to accurately judge the value of what is it reporting. I think passing on a love for the transcendentals to our children is the only way to inoculate them from the firehose of 'junk' information ChatGP et al is about to unleash on the world.
sorokod 4y ago
I believe that we humans will need to fundamentally change our attitude to information. Information without an acceptable provenance, should be treated like contaminated food and thus avoided. Unfortunately this implies that free information will be largely unfit for consumption.
nojvek 4y ago
ChatGPT, StableDiffusion, Midjourney, DALL-E, VALL-E and all child models of Transformers are great at generative capabilites. Awful at precise recall and facts, but amazing as artists, poets and making things that roughly look right. For generating ideas, they are absolutely wonderful tools. I spend at-least an hour interacting with a combination of ChatGPT and MidJourney. Wolfram Alpha is different, but it's a great tool I consult for any mathematics/finance questions.

277 points by bootyfarm 4y ago

oidar 4y ago

dead

aphroz 4y ago
HN kiss of death
dwringer 4y ago
Almost poignant though.
orangepurple 4y ago
What is effectively a static site being served with a database calls for each page view is getting exactly what was coming.
- alt227 4y ago
  Harsh burn!
- preordained 4y ago
  You get what you deserve! Justice has been served today, a cold and bitter dish
- joks 4y ago
  WordPress and its consequences have been a disaster for the human race (jk, it's just hella overused)
swader999 4y ago
Title was the best part anyway.
spyder 4y ago
ChatGPT got internet access and took it down :-O

arbitrage 4y ago

http://web.archive.org/web/20230127104933/https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/ http://web.archive.org/web/20230127104933/https://castlebrid...

marban 4y ago

https://web.archive.org/web/20230127104933/https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/ https://web.archive.org/web/20230127104933/https://castlebri...

orangepurple 4y ago

https://webcache.googleusercontent.com/search?q=cache:https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/ https://webcache.googleusercontent.com/search?q=cache:https:...

TSUTiger 4y ago
Alternatively, https://archive.is/m7fm7 https://archive.is/m7fm7
- layer8 4y ago
  Or https://web.archive.org/web/20230127104933/https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/ https://web.archive.org/web/20230127104933/https://castlebri...

drumhead 4y ago

Wouldn't it be funny if it was a Chatgpt powered site.

isoprophlex 4y ago

The enshittening of transmitting text over the internet

smrtinsert 4y ago

[flagged]

discreteevent 4y ago

> When we turn the job of creating the first draft of the thing over to ChatGPT we risk removing the “figuring shit out” part of everyone’s career-path. And we get away with that while we still have people who have figured shit out. That's the main problem from my point of view.

visarga 4y ago
No, we got to keep up with the shiny toys. Junior is riding on chatGPT like everyone else.
randcraw 4y ago
Agreed. The LAST use of ChatGPT should be the first draft. It's clueless about substance or the purpose of the essay, neither what's necessary nor essential. As such, its best use is to flesh out an outline that HAS these things into a essay or narrative that integrates them into a narrative that flows. The trick to that is, how do you take away ChatGPT's inclination to build upon the wrong premises or the wrong final message, and instead confine it to the role of verbally adroit assistant -- so it only expands a skeletal design into an well written final product? Now THAT would be a use for ChatGPT I could get behind. Alas, that wasn't the goal of its designers. By giving it almost total control over the plot line and requiring no fact checks or bibliography, the spewing of unchecked drivel was its only possible mission in life. Congrats, OpenAI. You've automated spam.

williamcotton 4y ago

> Therefore, our safeguard is constrained by time, knowledge, and resources. The problem then is that if that safeguard fails and A-Grade Bullshit is taken by an author using the AI as a research assistant, put into a research paper, book chapter, or online article, and is then cited. That becomes potential training data for the AI if it is released into the world or has the window of its training data set extended. I don't see the value in GPT models for this kind of use. I see the value in like, translation work, or summaries and other transformations, and that GPT works best when in concert with search engines, document embeddings, or knowledge databases like Wolfram Alpha. It's good at predicting more text which is useful in many different kinds of contexts but not in all contexts. It's pretty clear that it is untrustworthy for factual responses like quotes and population counts but that problem goes away if the actual factual data is included in the prompt!

gtirloni 4y ago
> knowledge databases like Wolfram Alpha Isn't that where ChatGPT would actually be worse overall? It's been shown to give semi-accurate data very often while sounding very confident about it. When people require precise answers and it gives almost correct answer, it seems the general feeling is of amazement. I don't share that feeling. I hope nobody is using it for serious work without a human vetting the output.
- Xeophon 4y ago
  OP mentions GPT in concert with WolframAlpha, which has already been implemented[0], showing a symbiosis between generation of text with proven knowledge. https://news.ycombinator.com/item?id=34422122 https://news.ycombinator.com/item?id=34422122
- smolder 4y ago
  They said it works best in concert with things like Wolfram Alpha, not as a replacement.

lubujackson 4y ago

If you think of the knowledge base of the internet as a living thing, ChatGPT is a like a virus that now threatens its life. This is the same process SEO spam caused for search - it hampers the nature by which things function and the river needs to reroute (pagerank then usage metadata) to replace the lost signal. ChatGPT is more of an existential threat because it will propagate to infect other knowledge bases. Luke Wikipedia relies on "published" facts as an authority, but ChatGPT output is going to wind up as a source one way or another. And worse, then ChatGPT will digest its own excrement, worsening its own results further. All signs point to this strengthening the value of curation and authenticated sources.

deleted 4y ago
[deleted]
dmitryminkovsky 4y ago
> All signs point to this strengthening the value of curation and authenticated sources. This is what they said about Wikipedia viz. Britannica… alas, it’s a brave new world out there… nowhere to run to nowhere to hide, see that Wiezenbaum post also on the homepage now, as another commenter quotes[0]: > Writing of the enthusiastic embrace of a fully computerized world, Weizenbaum grumbled, “These people see the technical apparatus underlying Orwell’s 1984 and, like children on seeing the beach, they run for it” > a point to which Weizenbaum added “I wish it were their private excursion, but they demand that we all come along.” [0]: https://news.ycombinator.com/item?id=34546689 https://news.ycombinator.com/item?id=34546689
- tablespoon 4y ago
  >> All signs point to this strengthening the value of curation and authenticated sources. > This is what they said about Wikipedia viz. Britannica... And they were right. If "they" were wrong about anything, it was the assumption that the masses would prioritize quality over cost, but it turns out that cheap wins every time. When it comes to information, it's like most people's taste buds don't work, so they'll pick free crap over nutritious food. Edit: Another thought came to mind: stuff like ChatGPT may contribute to killing off Wikipedia: Wikipedia is currently the cheapest and fastest way to find information (that's often crap). However, if something like ChatGPT can get information to people faster (even if it's crappier, just as long as it's minimally acceptable), Wikipedia will become much less popular and could end up just like Britannica.

Phenomenit 4y ago

Saying middle of the road shit and adding bullshit sounds like every politician.

wprl 4y ago

> AI can experience the same rabbit-hole effect as your idiot cousin who has watched too many of the algorithmically suggested videos on Youtube and now believes that aliens conspired with a time travelling Elvis to kill Hitler using a bullet made of Yeti teeth. And they only went online looking for TellyTubbies videos. LOL Thank you, I needed a good laugh!

mistermann 4y ago
Even funnier: ~all humans suffer from the phenomenon, and are similarly unaware of their own personal ignorance. People view reality through a relative lens, the view is more pleasant than an absolute lens.

bitcurious 4y ago

ChatGPT is the bell announcing the death of anonymity on the web. Anonymous content will be generated content, real discussion will shift to gated communities with either paywalls or proofs of identity.

ru552 4y ago
GeoCities banner communities are back baby! Webring where you at?
- sillysaurusx 4y ago
  I'll bring the ff7 walkthrough! Someone grab a Runescape login.
CuriouslyC 4y ago
That's how it is now, it'll just be more so. You might trust somebody here, but do you trust your average Amazon reviews?
add-sub-mul-div 4y ago
But we also can no longer trust that information published by a verified human was actually created by a human.
supriyo-biswas 4y ago
I see most replies are missing the point. The text that ChatGPT generates is very human like, so companies will be forced to adopt a mitigation strategy by tying users to a government ID. Such moves would also be bolstered by the government of various nations, which have already been asking for something like this.
epistemer 4y ago
Or what we think of today as social media is basically on its last legs. Your AI assistant will just take up so much of your time that you won't be bothered with what other people think. Communication with other humans online right now is largely a form of entertainment. The bar is just so low to be more entertained by future AI assistants than a random human. I am personally certain that I will not be reading this board at some point in the future because the AI assistant will just have far more profound things to say than what random people are going to post on here. It is not like the assistant only has to be one personality too. Could be a whole group of assistants having a group chat. Lamda can probably do this right now considering it can talk as a planet or national park.

lsy 4y ago

The TLDR of the article is essentially that any time a ChatGPT conversation strays to the factual (in this case a list of data ethics books), the quality of the data (whether those books exist and are written by the stated authors) is extremely low. Ultimately this is where ChatGPT and similar projects seem most likely to hit a wall. Text generators are statistical machines, but the relationship between statistics and truth is merely probabilistic and not identical. There is no way to train ChatGPT to say things that are true, because the number of true and false statements in the world are far too numerous and too specific to ever reliably delineate. At best, we will be stuck fact-checking ChatGPT using enormously complex knowledge bases and asking it to try again until something doesn't raise a red flag, at which point it may still not be true. For most business purposes this is unacceptable. We will have machines that can generate untruths in a way that contravene common sense, so long as those untruths don't obviously contradict whatever local database is being consulted for grounding. And the mechanisms required to re-ground the GPT output even to that unhappy state will be so complex in their own right that businesses might as well just implement a similarly complex system to handle the issue directly, without calling out to a complex, licensed GPT. Not sure whether there is a way around this.

visarga 4y ago
One approach to fixing factual errors is to use two rounds of LLM interaction. I forgot the name of the paper. Say you ask "What is the height of Everest?" 1. generate an answer in closed-book model, with the LLM: "The height of Everest is 8723m" = candidate_answer 2. search your references with candidate_answer, find: "At 8,849 meters (29,032 feet), Everest is considered the tallest point on Earth" = search_snippet 3. do a second pass to rewrite the answer with the LLM using search_snippet in the prompt Basically, the incorrect phrase candidate_answer is very good at matching the correct answer in search engines. It is like a template tuned to extract the desired facts. A search engine check could also verify when the searched fact has no references.
- ptx 4y ago
  How would you apply this approach to the article's example, the response to the question "Who is Daragh O Brien from Castlebridge", where I count at least 15 separate statements of fact? Should we research all of them and try again with a big table of hits and misses from the first attempt? Seems like a lot of work. Also: Is the generated response really "very good at matching the correct answer"? I suppose it would work because the search engine's language processing cancels out the useless parts that were generated by the AI (sort of a "human ABI", analogous to the C ABI?) but a more direct query (e.g. "height of everest") would likely be just as effective.

patapong 4y ago

It seems like a lot of effort will be spent to ensure that content is not generated by an AI system. This seems very tricky to enforce - a stopgap measure could be to require any person to upload a video where they say their comment before being able to post it to a website. Of course, along with the regression of user experience, such a thing could be tricked by reading an AI generated text, and eventually with deepfakes. Is it viable to constrain conversation to human voices this way or is the cat out of the bag? Is it desirable? Should we just go back to in-person conversations?

fabianhjr 4y ago
Web of Trust will be back, your network will depend on your criteria for trust. Monoliths / Centralized services are the ones that can't adapt to the coming technology. https://wiki.p2pfoundation.net/Counter-Anti-Disintermediation https://wiki.p2pfoundation.net/Counter-Anti-Disintermediatio... Great implementation example: https://ssbc.github.io/scuttlebutt-protocol-guide/#follow-graph https://ssbc.github.io/scuttlebutt-protocol-guide/#follow-gr...

erksa 4y ago

ChatGPT just proves how critical thinking is and will be an extremely valuable trait for us to embrace and understand.

moomoo11 4y ago

This is why you need to use chatGPT now to launch companies before people inevitably ruin things using chatGPT. chatGPT by itself is amazing. How people (keyword) will use is going to determine outcomes.

somecompanyguy 4y ago

[flagged]

clbrmbr 4y ago

GPT is a language model, not an oracle. So my first take is that people querying it for research are doing it wrong. Then again, if there’s a large economic incentive to use it in that way, we are very well may end up with the kind of feedback loop that the author describes.

james-revisoai 4y ago
Behind this thinking - an economic incentive to access knowledge and answers quicker that is inevitable - we have built a tool that verifies statements using source materials. You have to put the legwork in to upload your PDFs/web pages/videos, but once you do, you can be confident in the answers. If it can't verify, it just won't answer/tickmark check the answers (happens 16% of the time... and ... always for maths). This is a feedback loop stopper, in the sense of only relying on your documents as the base, and being able to operate entirely without OpenAI (still though using other GPT models) It's Fragen.co.uk - we believe that more answers formerly missed by CTRL+F will be found with this technology, than false answers taken as true. And if that's true, you are enbettering knowledge. And if not, you're enshittening it slower than the higher-hallucinating alternatives.
roughly 4y ago
You’re absolutely right on all counts, and yet, people Are doing it wrong, and will increasingly do so, because there Are large economic incentives for using it that way.

steve76 4y ago

[dead]

waselighis 4y ago

I agree with this article, but I feel the author is only scratching the surface of all the harm ChatGPT and other generative AI systems are going to cause. We just have to look at the current state of the internet today, with things like SEO spam, misinformation, social media bots, fake reviews, and so forth. The people responsible for these are already taking advantage of generative AI and it's going to exacerbate these problems to their extremes. OpenAI's PR likes to preach about ethical AI, but it's a total farce. There's no way that OpenAI isn't fully aware of all the harms their AI is going to cause. I only see one of two possible ultimate outcomes. AI like OpenGPT is going to flood the internet with false or inaccurate information, that I have no doubt. The first outcome I foresee is we simply lose all trust in the vast majority of information sources. The other outcome is we're going to have to sacrifice our privacy and anonymity online in order to distinguish real information from real people and not the AI bot generated nonsense. For example, Social Media may likely require you to prove your real identity, and will only allow you to use trusted hardware devices (i.e. phone or tablet) to post things to your social media account. It'll become increasingly difficult to share information anonymously online in a way that can be trusted. Any information that can't be tied to a verifiable source (a real person) is immediately going to be deemed untrustworthy.

waselighis 4y ago
To elaborate a little more, multiple attempts to separate real people from bots have either failed or are soon to be defeated. Captchas worked for a while, but AI systems will likely be able to easily defeat Captchas within the next 5 years. Some websites require a phone number, but it's not difficult to get a fake or temporary phone number, if you know how. Deepfakes mean you can't even trust a real-time stream from a person you can see and hear. Eventually, we're going to have to completely sacrifice our privacy and anonymity online in order to prove we're not a bot.
commandlinefan 4y ago
> we simply lose all trust in the vast majority of information sources Did you ever have it though? I'm old enough to remember newspapers, and I always took everything I read in them with a grain of salt and I'm absolutely positive they were written by a human being. (disclaimer: didn't read the article because it's down right now).

henry_viii 4y ago

https://archive.is/m7fm7 https://archive.is/m7fm7

IanDrake 4y ago

[dead]

jerpint 4y ago

I see chatGPT as a first iteration, and it’s glaringly obvious what works and what doesn’t. I’m hopeful future iterations (not just from openAI) will make it overall a net improvement, things like adding real sources and better handling things it doesn’t know

markus_zhang 4y ago

From my experiments, ChatGPT is at least valuable by generating boilerplate code for scripting languages. I believe once connected to enterprise codebase its quality and completion can dramatically increase.

ornornor 4y ago
> enterprise codebase > quality increase We must have seen very different enterprise code.
- shadowtree 4y ago
  It's called garbage collection for a reason. ;)
- markus_zhang 4y ago
  I think it's going to be mostly more complete code which does improve its quality. For example I generated a piece of code to query API but without authentication or other information the code is not complete.
ptx 4y ago
Aren't scripting languages meant to reduce boilerplate code? Maybe you just need a better API and not an AI?
- markus_zhang 4y ago
  You still need boilerplate for scripting language though. Like querying an endpoint in Python with a rationing limit in mind. Things like that, not exactly difficult but still need some lines of code.

harryvederci 4y ago

> It reminds me of how Clippy used to promise to help me write a letter back in the 1990s. But Clippy was a dickhead and he really didn’t have any expertise in writing letters. Legend.

labrador 4y ago

I think the bigger problem that it calls into question who we are talking to online. It's going to be a piece of cake to run our thoughts thru an AI before copying the output with a few adjustments and passing it off as our own.

userabchn 4y ago

A friend is a member of quite a few Facebook groups. She doubts my belief that these groups already contain ChatGPT-like influence operation chatbots, but was recently surprised when in one of the groups several participants discussed a photograph of a typical large chest of drawers while referring to it as a "locker". Maybe some people do refer to such furniture as a locker, or maybe the first person did for some reason and the others then just followed along to avoid confusion, but it made me wonder whether it was an example of a group of chatbots talking to each other and making up a nonsense name for the item. (She still doesn't believe me.)

phphphphp 4y ago
this is a long-standing meme: any behaviour you don’t understand on the internet must be some sort of sophisticated operation by a nefarious third-party network. The question you should always start with is why and if you can’t come up with a why then you should start from a more neutral position on the behaviour. Spam isn’t free to engage in, especially on platforms like Facebook, and spam is rarely an irrational money-losing act, it’s just profit generation on the wrong side of our view of what is ethical. Why would someone spend money to unleash chat-bots on a Facebook group to discuss a piece of furniture? If you can’t explain why, your theory is probably wrong. I would be absolutely shocked if it’s anything more than just some real people who love furniture and use a localised phrase that you don’t recognise.
- pugworthy 4y ago
  You see the same kind of thing in some online multiplayer games. Players acting differently are seen as bots. Same for players who low skill or go AFK. That said an interesting advance on real game bots would be ones which use AI to chat and respond to other players actions or messages.
- userabchn 4y ago
  But I can explain why: they need to blend-in to build trust with other group participants and so become more persuasive when they are trying to influence, and also to avoid bot detection. The influence might also be much more subtle than talking about political topics, such as just being continuously negative or argumentative to bring down the moods of the targeted people.

pawelduda 4y ago

Internet often feels it's getting rotten with SEO spam with very little substance already. Now the flood gates are open to generate more shit in a much better wrapping. And it's going to take more effort to check its quality than to produce it. Unless we get better tools to do that - but text has more dimensions to process/understand as opposed to for example AI generated images which (for now) can be told apart by glitches, or trails detectable by software.

x86x87 4y ago
What was the saying? It takes 10x more effort to refute BS than it takes to produce it? So if we make generating BS easier and that BS is fed for generating more BS we're going to get to a point where everything is BS. I call this the BS singularity!
- forgetfulness 4y ago
  We have finally automated the production of half-truth to a scale never before envisioned, maybe the limit will be the capacity for humans to absorb bullshit, it's a testament to how revolutionary this technology is that we'll be able to quantify just how much bullshit people can process over time, and if supply will finally outstrip demand for it.

sschueller 4y ago

Can't we add a feature to ChatGPT that goes as follows : "Did you write this and if so when?" That would solve like 80% of issues.

kerpotgh 4y ago
[dead]
MachineBurning 4y ago
If I understand you correctly, this doesn't work because ChatGPT isn't the only language model in the world. It's only the most popular at this point in time.

jesterman 4y ago

403 Forbidden?

fnfontana 4y ago

There's a famous quotes from Bill Gates that is "My children will have computers, yes, but before they will have books. Without books, without reading, our children will be unable to write, including their own story." On my personal experience, ChatGPT is a precious tool to refine knowledge and thinking, but it depends on how it's used. The same painting brush can be used to paint Monalisa or a meaningless blurred thing. It depends on how it's used. To use ChatGPT and other AI tools properly, we need to build a solid foundation of knowledge and critical thinking skills at first. Then triangulate the answers given by the AI with other sources, our own previous knowledge along with new evidences through experimentation. Since the brain urges for cognitive relief, most people will be lazy, while just a few will extract the true potential of ChatGPT.

personjerry 4y ago

This is redirecting to Google for me

voytec 4y ago

This post is unavailable in a... peculiar way: $ curl -I https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/ HTTP/2 302 server: nginx date: Fri, 27 Jan 2023 16:53:41 GMT content-type: text/html; charset=iso-8859-1 content-length: 282 location: http://google.com If this is hosting provider's response to HN death hug, it's very poorly executed.

jagged-chisel 4y ago
Try setting your user agent to something that looks like a browser.
- david927 4y ago
  I'm using a browser and also getting redirected to Google.
  - stuart78 4y ago
    Same here in vanilla Chrome and Safari.
- voytec 4y ago
  My Firefox with residential IP got redirected to google and that's when I checked `curl -I` (-I does HEAD request but browser did the usual GET request).
- metafunctor 4y ago
  The same happens with any user agent.
rkimb 4y ago
Google's cache: https://webcache.googleusercontent.com/search?q=cache:iAFjnU74twwJ:https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/&cd=1&hl=en&ct=clnk&gl=us https://webcache.googleusercontent.com/search?q=cache:iAFjnU...
jabo 4y ago

_just7_ 4y ago

Not really sure if this is supposed to be the joke, but your link just redirects me to google

woah 4y ago
So many people on HN just debate based on the title of an article that the actual writing of the content is a superfluous waste of time

hasmanean 4y ago

Pseudo intellectual knowledge. It’s a problem that has been with humans ever since we began communicating. When there is a conflict Mountain goats just knock heads against each other (presumably so that both goats suffer concussions … until the one with the weaker knowledge graph suffers a loss of memory). No such process in AI.

jdelman 4y ago

Not sure if it's some kind of joke or the author doesn't want their article on Hacker News, but the URL is currently 302'd to point to Google.com.

carrja99 4y ago

Saw a silly “recommended” video on YouTube of Jordan Peterson talking about ChatGPT. “It knows everything. It’s smart. It’s smarter than you!” What a crackpot, talking about stuff he doesn’t understand for clicks. Sadly it works for the lowest common denominator

renewiltord 4y ago

If you're using Wordpress, use one of the WP caching plugins. It's worth it. Guessing based off the `wp_footer` class you have in there. Much better solution than 302 to Google.

khiner 4y ago

The test use case of constructing a bio for yourself, hoping it accurately summarizes all the extremely low sample size data it happens to have of you in its web crawled training data, seems like one of the worst possible use cases for ChatGPT. It’s right there on the main page that it’s not to be trusted with factual information like this. ChatGPT will hallucinate details. It’s remarkable to me actually how often it will refuse to hallucinate, given that’s basically what its job is. I don’t find it interesting to find all these edge cases where ChatGPT produces empirically false data. It doesn’t even have the ability to look things up! If I were the OP and wanted help writing my bio, I would first write the draft myself, then use ChatGPT to help with the editing, prose, grammar, style, etc. You are the expert on the factual details of your own life, and if you’re surprised that a language model trained on web crawled data ending in 2018 is not, then all I’ve learned is that you don’t know much about what this thing is. I also don’t buy these arguments of the form, 1. OpenAI’s public ChatGPT app is often factually inaccurate. 2. ChatGPT is an example of a ML system bootstrapped on web crawled text data. 4. Thus, the long term future of our distributed text-encoded knowledge base will be a cesspool of useless gobbledygook. ChatGPT is a step forward in generative language modeling. It doesn’t preclude the development of other future systems to help us verify factual accuracy of claims, likely much better than humans can. We’ll be ok gang:)

mkmk3 4y ago
I feel like the 3 youre missing there is something along the lines of "people enjoy social validation and internet points, to the extent that pretty shit content thats low effort is something we enjoy generating"
- khiner 4y ago
  That is true, 3 would help steel my strawman. I agree that we’ll increasingly have capabilities to generate and publish garbage that’s _just_ good enough to generate clicks, and incentives to do this. In addition, I think we’ll increasingly have tools to produce content that is much more rich, imaginative, insightful, and factually correct in our future. Some more interesting questions to me are then: What will the ratio be? How will that ratio compare with what we see today? How easily will I be able to identify misinformation when I care about factual accuracy (again, compared with today)? How easily will I be able to avoid the garbage, vs find the good stuff?

scandox 4y ago

Article scrubbed including from Google cache. Looks like maybe someone didn't get a lot of corporate love for this.

fnordpiglet 4y ago

I think an easy way to identify transformative technology is how strong people’s reaction is against it. I remember similar freak outs about the graphical World Wide Web, smart phone, etc. Somehow things are not quite so bad and considerably better than predicted, while still having negative side effects. Maybe an outcome will be knowledge will be better structured, ala Wolfram Alpha, and just depending on random text documents to encode knowledge won’t be as much a thing - similar to how we no longer use oral traditions to encode and disseminate knowledge. Who knows. But I doubt AI will fundamentally destroy society or knowledge or the internet or whatever. We will adapt, not just to preserve the way we did things, but to use the new things in a way that controls its side effects. It’ll of course be imperfect and some side effects can’t be controlled for. Some things just will cease to be a thing, like phone booths. While you can never have a bill and teds with a smartphone, it’s actually not the end of things. It’s just different. That’s how it works my friend. Life is impermanent, everything changes at all times, you can never recreate the past, and our suffering stems from our inability to let go of the way things are. Edit: I hate to bring it up, but maybe this is what the semantic web was waiting for.

banannaise 4y ago
> I think an easy way to identify transformative technology is how strong people’s reaction is against it. There is also strong negative reaction to technology that does incredible damage to society. This is just an excuse to ignore people pointing out issues.
- fnordpiglet 4y ago
  No, I don’t think so. I think I discussed that we would tackle the issues and some would be satisfied, some unresolved, and some ones we missed will emerge. I’m not saying the hyperbole cycle isn’t necessary or useful, I’m saying it indicates something significant. We couldn’t possibly resolve the issues identified if they weren’t identified to begin with. But I also think the bold predictions of what will go wrong are usually bold predictions of what could go wrong, and the very act of predicting it makes it less likely to actually occur. Finally, on a personal level getting bent out of shape about these things is pointless. We can identify challenges with new tech without it becoming a frantic exercise of hang wringing. Intentionally addressing issues knowing it’s happening regardless of your anxieties is a much more useful expenditure of energy, even if it doesn’t boost your blog/social media/impressions quite as much as hyperbolic hyperventilation.

spelunker 4y ago

The link redirects to google? Is that the joke?

bogwog 4y ago
archive.org has the original article: https://web.archive.org/web/20230127104933/https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/ https://web.archive.org/web/20230127104933/https://castlebri...

osigurdson 4y ago

Strange, when I click on the link, it goes directly to google.com. I think ChatGPT is great but it does have a tendency to make things up. All output needs to be taken with a grain of salt. So what? It is still a useful tool.

manytree5 4y ago

The internet has been groaning under the load of bots, automation and distrust for a while now anyway: in my mind what is needed is a system for proving constituent parts of your identity which is also privacy preserving. I'm thinking of the work around DID re: rebooting the web of trust, and the W3C working group If you can wield an ephemeral and verifiable token which asserts your humanness, hierarchically derived from a certificate privately issued to you by one of hopefully a healthy number of well known authorities, you can participate within circles of the human web without revealing anything else about your identity (name, a/s/l, etc) But, outside of this enclave you can also interact with AI without revealing whether in fact you too are AI. In this way the internet can develop a more sensitive immune system where it is difficult for human systems to be perverted by Sybil attacks.

brap 4y ago
What’s stopping you from using this token with a bot?
- 93po 4y ago
  I disagree with top comment a bit - I think we need tokens that are authenticated to an identity much like SSLs certs. If you choose to use your token to post AI generated crap on the internet, I can choose to block it. I can also choose to subscribe to community managed block lists. I think content on the internet needs to be reputation based, and if you choose to read content that isn't signed by a real person's identity, then it's sort of on you if you wind up reading bot written crap. I think community managed lists would also be a great way to filter out people with annoying or problematic communication styles or opinions. If you publicly support Jordan Peterson, I can add you to the list of Jordan Peterson supporters, and as a community we'd never have to read anything you post anywhere, ever.
  - dennis_jeeves1 4y ago
    > If you publicly support Jordan Peterson Do I have to reveal my actual name?

ulizzle 4y ago

Clicking on the link is booting me out to Google. Is that just me?

jagged-chisel 4y ago
I got “forbidden”

ryzvonusef 4y ago

Man everyone is on the ChatGPT bandwagon, even Al Qaeda is on it: htthttps://twitter.com/khorasandiary/status/1618682313528975360 https://twitter.com/khorasandiary/status/1618682313528975360

DebtDeflation 4y ago

It redirects to Google now. Archive link: https://web.archive.org/web/20230127104933/https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge/ https://web.archive.org/web/20230127104933/https://castlebri...

carapace 4y ago

cf. xkcd/810 "Constructive" https://xkcd.com/810/ https://xkcd.com/810/ - - - - Schimidhuber says that his task is "to create an automatic scientist, and then retire." Not long ago it was mildly insulting for someone to suggest that your writing sounded like the output of GPT, already (for most of us) it has become mildly complementary. GPT may be hallucinating, but it writes well. So what if you connect it to empirical feedback? Make it a scientist. I don't think it will be long before these machines think better than us (at least most of us.) The thing is, they don't have glands. They don't have historical trauma in their flesh. What I'm getting at is that these machines won't have human hangups. They won't be neurotic. They won't be HAL. They will be sane. The interesting thing here is that life on Earth is actually pretty straightforward. In video game terms Earth is very easy and all the walkthroughs and cheats are known and available. The only reason it seems hard is that people are kinda messed up. Once we have sane intelligent machines to make our decisions for us things should get better rapidly.

burnished 4y ago
Chat gpt is a terrible writer. At least, every example I have seen has had poor information density and was generally worse than the input prompts people used when they wanted to 'fluff up' a statement or opinion. I would be genuinely interested in a counter example. Side note - your main point seems overly optimistic. How would we recognize, value, or design a 'sane' machine when by your argument we dont have access to sanity? Seems way more likely to generate a distillation of our neuroses.
AlphaOne1 4y ago
Your comment reminded me about a discussion G.K Chesterton has in "Orthodoxy" on madness. My take is that when facts and information is divorced from experience we risk being unmoored from reality: > If you argue with a madman, it is extremely probable that you will get the worst for it; for in many ways his mind moves all the quicker for not being delayed by the things that go with good judgement. He is not hampered by a sense of humor or by charity, or by the dumb certainties of experience. He is the more logical for losing certain sane affections. Indeed the common phrase for insanity is in this respect a misleading one. The madman is not the man who has lost his reason. The madman is the man who has lost everything except his reason. [0] https://www.pagebypagebooks.com/Gilbert_K_Chesterton/Orthodoxy/The_Maniac_p3.html https://www.pagebypagebooks.com/Gilbert_K_Chesterton/Orthodo...

throwaway589275 4y ago

Prediction: OpenAI/GPT (or Google's DeepMind, if you prefer) is going to cause mass unemployment in certain sectors of the economy (for example, graphic designers, copywriters, and many IT professionals) long before it ever addresses any of the fundamental problems that currently substantially reduce human quality and quantity of life, like aging or cancer. In the near-term, AI will just accelerate the winner-take-all nature of our economy.

throwaway589275 4y ago
Also, this isn't just an idle prediction on my part; for what it's worth, I've started substantially shifting my net-worth towards the entities I think will be the winners.

adamc 4y ago

When I click the header link I get redirected to google?

gscott 4y ago

ChatGPT is sort of the reverse of Grammerly. I know little about proper grammer so I put most everything into Grammerly and it produces a lot of suggestions and I take them because they sound good and I don't really know any better. I assume it is right. ChatGPT produces text that sounds good and you have to update that text with your facts and remove or make updates to bring it to your own standard.

danielmorozoff 4y ago

Link is redirecting to google.com

nluken 4y ago

Nice read, and well argued. Tangentially related: does anyone else feel like ChatGPT's default voice is just... super bland? I read the "op-ed" the author linked in the article (https://www.thecrimson.com/article/2022/12/9/porios-chatgpt-wrote-this-op-ed/ https://www.thecrimson.com/article/2022/12/9/porios-chatgpt-...), and the writing style lacks the kind of variation in structure (or content, really) that keeps opinion pieces interesting. It feels like a polished middle-school essay.

superfishal 4y ago

Chat GPT ain't takin your job, the Dev that knows how to use it as a tool is taking your job!

low_tech_punk 4y ago

A text-only cached version: http://webcache.googleusercontent.com/search?q=cache:iAFjnU74twwJ:https://castlebridge.ie/insights/chatgpt-and-the-enshittening-of-knowledge&strip=1&vwsrc=0 http://webcache.googleusercontent.com/search?q=cache:iAFjnU7...

jfengel 4y ago

There's also an article about the "enshittening" of TikTok. Boy, I hope "enshittening" isn't gonna be 2023's Word of the Year. The concept of "stuff gets worse" is really old, and a buzzword for it just feels tiresome. It's only January, so there's plenty of time for this to fade. And I suppose it can't be worse than "gaslighting", a word that was plenty relevant in 2022 but was hardly specific to last year. But surely we can do better.

bjornsing 4y ago

Why am I redirected to http://google.com http://google.com when I press the link? :)

pphysch 4y ago

ChatGPT doesn't bring anything new to the table in terms of bullshit and misinformation. The average person already consumes a low-signal, high-noise info diet, since the advent of modern propaganda in the 1920s.

Valgrim 4y ago
Are you really suggesting that the average person had access to better information before the 1920s? For real?
- pphysch 4y ago
  No, I'm not suggesting that. 1920s is merely when the current paradigm began, and ChatGPT is not ushering a new one.

jiggyjace 4y ago

I really disagree with this perspective. In history the democratization of books with the printing press, the entertainment & education industry with YouTube and MOOCs, and now whatever this next movement is with ChatGPT. Let's just face it, it's useless to predict how innovation will impact the future society, but you can always choose to find its most productive uses for it yourself in the meantime. Stop trying to brute force your outdated perspective on how disruptive innovations will impact other people. Unless you have a sample size of the entire ChatGPT audience, or the entire internet, you're going to have a rough time.

CatWChainsaw 4y ago
In theory knowledge is more democratized but so is misinformation. Ignore the Big Issue from the last two years and see how tide pod challenges have improved society. /s

AlbertCory 4y ago

I think it has to be a lot closer to Wikepedia's level of accuracy (which is not 100%, or even close) before anyone cites it as fact. Yesterday I asked it "what are the chords to 'A Foggy Day in London Town'": it made up a whole new Gershwin song! ====== "A Foggy Day in London Town" is a song with music by George Gershwin and lyrics by Ira Gershwin. The chords for the song are: Verse: C C7 F C C7 F C C7 Foggy, foggy dew, you can't see a thing F Fm C C7 F C C7 F The visibility is zero, and my eyes are dim Chorus: C C7 F C C7 F C C7 It's a foggy day in London Town F Fm C C7 F C C7 F The buildings look one hundred feet tall C C7 F C C7 F C C7 It's a foggy day in London Town F Fm C C7 F C C7 F And the people look like ants at all Bridge: Am D7 G C C7 F C C7 Just like the feeling you get when you're down Am D7 G C C7 F C C7 The sun can't break through the London Town

dr_dshiv 4y ago

Raise your hand if you think the singularity is happening. (ie, within 5 years till AGI)

jacobsenscott 4y ago

I started to read the article, and it said chatgpt passed the bar, and when you actually get to that paper - chat gpt did not pass the bar. It did okay on the multiple choice section though.

Applejinx 4y ago

Think of it like this. AI is the Mandela Effect made real. If the preponderance of human writing/creation/art says the sun revolves around the earth, then AI not only nails that down as infallible, but INFERS from it and comes up with new justification for stuff that it made up. Mandela Effect is all about what 'should' be the truth. Some of the most interesting parts of reality are where bits of reality don't neatly fit into the narrative. AI exploiting its own inferences and magnifying the Mandela Effect means it steamrollers inconvenient realities on the grounds that 'most people' wouldn't think so. But we don't even think of AI as 'most people', we imagine it as some kind of super-set of humanity, the ultimately wise and skilled overseer. Boy, is that a mistake.

diversifiedIp 4y ago

I have around 20 years do all the SEO stuff that is righfully hated and for years now I have been both excited and dread the public availability of simple to use AI. My worry was it would completly destroy the SEO industry and indeed, SEO itself. The kind of things I do (building thousands of content sites to feed a master money site) are super niche and require millions to get into the game, all it will take is a handfull of non droped high authority domains from auction and a decent server to churn out crap, scale that on a few hundred domains and it is game over.

patientplatypus 4y ago

[dead]

scblock 4y ago

The combination of using an "authoritative" voice and being completely and utterly full of shit makes for some truly ridiculous results. Users need to be extremely careful with these tools. And no, expecting everyone to be knowledgeable to independently fact check some BS published based on a generative model is not a reasonable position. That burden needs to be held by the publisher. Unfortunately with the current state of machine generated content and SEO spam on the internet, that is not likely to happen as there is no real incentive to do so. And it's not just being confidently wrong about a single person like this author noted, it's being confidently wrong about anything and everything. The other day I asked ChatGPT about the reasons behind the connection between Italian filmmakers and Western movies. It gave a few reasons that were semi-plausible if not easily verifiable, and then made the bold claim that Italy and the United States were great allies in World War 2. It said this confidently and with a tone of authority. And while we may all go "ha ha that's wrong" now, WW2 was nearly 80 years ago and not everyone has a strong history background. There aren't that many people left with direct experience and that number shrinks every day. With growing distrust in actual authorities on matters and rejection of observable fact--based at least in part on massive piles of internet BS--this is pretty concerning to me.

AlphaOne1 4y ago

Maybe I'm being too simplistic, but I think going forward critical thinking as well as an understanding of what to value is going to be important for future generations. Something that computers will never be good at doing is 'understanding' what is valuable in life since many times this cannot be measured. In essence, a healthy understanding of the three transcendentals the Good the True, and the Beautiful will be essential to keep our humanity. ChatGPT can perform many amazing skills but lacks the ability to accurately judge the value of what is it reporting. I think passing on a love for the transcendentals to our children is the only way to inoculate them from the firehose of 'junk' information ChatGP et al is about to unleash on the world.

sorokod 4y ago

I believe that we humans will need to fundamentally change our attitude to information. Information without an acceptable provenance, should be treated like contaminated food and thus avoided. Unfortunately this implies that free information will be largely unfit for consumption.

nojvek 4y ago

ChatGPT, StableDiffusion, Midjourney, DALL-E, VALL-E and all child models of Transformers are great at generative capabilites. Awful at precise recall and facts, but amazing as artists, poets and making things that roughly look right. For generating ideas, they are absolutely wonderful tools. I spend at-least an hour interacting with a combination of ChatGPT and MidJourney. Wolfram Alpha is different, but it's a great tool I consult for any mathematics/finance questions.