6 ms·

In the past, you had to put a lot of effort to produce a text which seemed to be high quality, especially when you knew nothing about the subject. By the look o

• roenxi 2y ago
  > While the professional looking text could have been already wrong, the likelihood was smaller... I don't criticise you for it, because that strategy is both rational and popular. But you never checked the accuracy of your information before so you have no way of telling if it has gotten more or less accurate with the advent of AI. You were testing for whether someone of high social intelligence wanted you to believe what they said rather than if what they said was true.
  • dietr1ch 2y ago
    I guess the complaint is about losing this proxy to gain some assurance for little cost. We humans are great at figuring out the least amount of work that's good enough. Now we'll need to be fully diligent, which means more work, and also there'll be way more things to review.
    • roenxi 2y ago
      I'd argue people clearly don't care about the truth at all - they care about being part of a group and that is where it ends. It shows up in things like critical thinking being a difficult skill acquired slowly vs social proof which humans just do by reflex. Makes a lot of sense, if there are 10 of us and 1 of you it doesn't matter how smartypants you may be when the mob forms. AI does indeed threaten people's ability to identify whether they are reading work by a high status human and what the group consensus is - and that is a real problem for most people. But it has no bearing on how correct information was in the past vs will be in the future. Groups are smart but they get a lot of stuff wrong in strategic ways (it is almost a truism that no group ever identifies itself or its pursuit of its own interests as the problem).
      • Jensson 2y ago
        > I'd argue people clearly don't care about the truth at all Plenty of people care about the truth in order to get advantages over the ignorant. Beliefs aren't just about fitting in a group, they are about getting advantages and making your life better, if you know the truth you can make much better decisions than those who are ignorant. Similarly plenty of people try to hide the truth in order to keep people ignorant so they can be exploited.
        • rendall 2y ago
          > if you know the truth you can make much better decisions than those who are ignorant There are some fallacious hidden assumptions there. One is that "knowing the truth" equates to better life outcomes. I'd argue that history shows more often than not that what one knows to be true best align with prevailing consensus if comfort, prosperity and peace is one's goal, even if that consensus is flat out wrong. The list is long of lone geniuses who challenged the consensus and suffered. Galileo, Turing, Einstein, Mendel, van Gogh, Darwin, Lovelace, Boltzmann, Gödel, Faraday, Kant, Poe, Thoreau, Bohr, Tesla, Kepler, Copernicus, et. al. all suffered isolation and marginalization of some degree during their lifetimes, some unrecognized until after their death, many living in poverty, many actively tormented. I can't see how Turing, for instance, had a better life than the ignorant who persecuted him despite his excellent grasp of truth.
          • Jensson 2y ago
            You are thinking too big, most of the time the truth is whether a piece of food is spoiled or not etc, and that greatly affects your quality of life. Companies would love to keep you ignorant here so they can sell you literal shit, so there are powerful forces wanting to keep you ignorant, and today those powerful forces has way stronger tools than ever before working to keep you ignorant.
          • roenxi 2y ago
            Socrates is also a big name. Never forget.
      • 2y ago
    • wlesieutre 2y ago
      There’s not enough time in the day to go on a full bore research project about every sentence I read, so it’s not physically possible to be “fully diligent.” The best we can hope for is prioritizing which things are worth checking. But even that gets harder because you go looking for sources and now those are increasingly likely to be LLM spam.
      • Terr_ 2y ago
        Traditionally, humans have addressed the imbalance between energy-to-generate and energy-to-validate by building another system on top, such as one which punishes fraudsters or at least allows other individuals to efficiently disassociate from them. Unfortunately it's not clear how this could be adapted to the internet and international commerce without harming some of the open-ness aspects we'd like to keep.
    • BehindBlueEyes 2y ago
      My new cheap proxy to save mental cost: pay to search on kagi, sort results by tracker count. My hope is fewer trackers correlates with lower incentives to seo spam. This may change but seems to work decently for now.
  • quietbritishjim 2y ago
    How do you "check the accuracy of your information" if all the other reliable-sounding sources could also be AI generated junk? If it's something in computing, like whether something compiles, you can sometimes literally check for yourself, but most things you read about are not like that.
  • cutemonster 2y ago
    Interesting points! Doesn't sound impossible with an AI that's wrong less often than an average human author (if the AIs training data was well curated). I suppose a related problem is that we can't know if the human who posted the article, actually agrees with it themselves. (Or if they clicked "Generate" and don't actually care, or even have different opinions)
  • glenstein 2y ago
    >But you never checked the accuracy of your information before so They didn't say that and that's not a fair or warranted extrapolation. They're talking about a heuristic that we all use, as a shorthand proxy that doesn't replace but can help steer the initial navigation in the selection of reliable sources, which can be complemented with fact checking (see the steelmanning I did there?). I don't think someone using that heuristic can be interpreted as tantamount to completely ignoring facts, which is a ridiculous extrapolation. I also think is misrepresents the lay of the land, which is that in the universe of nonfiction writing, I don't think that there's a fire hose of facts and falsehoods indistinguishable in tone. I think there's in fact a reasonably high correlation between the discernible tone of impersonal professional and credible information, which, again (since this seems to be a difficult sticking point) doesn't mean that the tone substitutes for the facts which still need to be verified. The idea that information and misinformation are tonally indistinguishable is, in my experience, only something believed by post-truth "do you own research" people who think there are equally valid facts in all directions. There's not, for instance, a Science Daily of equally sciency sounding misinformation. There's not a second different IPCC that publishes a report with thousands of citations which are all wrong, etc. Misinformation is out there but it's not symmetrical, and understanding that it's not symmetrical is an important aspect of information literacy. This is important because it goes to their point, which is that something has changed, in the advent of LLMS. That symmetry may be coming, and it's precisely the fact that it wasn't there before that is pivotal.
  • SoftTalker 2y ago
    In the past, with a printed book or journal article, it was safe to assume that an editor had been involved, to some degree or another challenging claimed facts, and the publisher also had an interest in maintaining their reputation by not publishing poorly researched or outright false information. You would also have reviewers reading and reacting to the book in many cases. All of that is gone now. You have LLMs spitting their excrement directly onto the web without so much as a human giving it a once-over.
    • Eisenstein 2y ago
      I suggest you look into how many things were published without such scrutiny, because they sold.
• ookdatnog 2y ago
  Writing a text of decent quality used to constitute proof of work. This is now no longer the case, and we haven't adapted to this assumption becoming invalid. For example, when applying to a job, your cover letter used to count as proof of work. The contents are less important than the fact that you put some amount of effort in it, enough to prove that you care about this specific vacancy. Now this basic assumption has evaporated, and job searching has become a meaningless two-way spam war, where having your AI-generated application selected from hundreds or thousands of other AI-generated applications is little more than a lottery.
  • bitexploder 2y ago
    This. I am very picky about how I use ML still, but it is unsurpassed as a virtual editor. It can clean up grammar and rephrase things in a very light way, but it gives my prose the polish I want. The thing is, I am a very decent writer. I wrote professionally for 18 years as a part of my job delivering reports of high quality as my work product. So, it really helps that I know exactly what “good” looks like by my standards. ML can clean things up so much faster than I can and I am confident my writing is organic still, but it can fix up small issues, find mistakes, etc very quickly. A word change here or there, some punctuation, that is normal editing. It is genuinely good at light rephrasing as well, if you have some idea of what intent you want. When it becomes obvious, though, is when people let the LLM do the writing for them. The job search bit is definitely rough. Referrals, references, and actual accomplishments may become even more important.
    • gtirloni 2y ago
      As usual, LLMs are an excellent tool when you already have a decent understanding of the field you're interested in using them in. Which is not the case of people posting in social media or creating their first programs. That's where the dullness and noise come from. The noise ground has been elevated 100x by LLMs. It was already bad before but it's accelerated the trend. So, yes, we should have never been trusting anything online but before LLMs we could rely on our brains to quickly identify the bad. Nowadays, it's exhausting. Maybe we need a LLM trained on spotting LLMs. This month, I, with decades of experience, used Claude Dev as an experiment to create a small automation tool. After countless manual fixes, it finally worked and I was happy. Until I gave thr whole thing a decent look again and realized what a piece of garbage I had created. It's exhausting to be on the lookout for these situations. I prefer to think things through myself, it's a more rewarding experience with better end results anyway.
• diggan 2y ago
  > By the look of text and the usage of the words, you could tell how professional the writer was and you had some confidence that the writer knew something about the subject How did you know this unless you also had the same or more knowledge than the author? It would seem to me we are as clueless now as before about how to judge how skilled a writer is without requiring to already posses that very skill ourselves.
• gizmo 2y ago
  I think you overestimate the value of things looking professional. The overwhelming majority of books published every year are trash, despite all the effort that went into research, writing, and editing them. Most news is trash. Most of what humanity produces just isn't any good. An top expert in his field can leave a typo-riddled comment in a hurry that contains more valuable information than a shelf of books written on the subject by lesser minds. AIs are good at writing professional looking text because it's a low bar to clear. It doesn't require much intelligence or expertise.
  • bitexploder 2y ago
    I think you underestimate how high that bar is, but I will grant that it isn’t that high. It can be a form of sophistry all of its own. Still, it is a difficult skill to write clearly, simply, and without a lot of extravagant words.
  • herval 2y ago
    > AIs are good at writing professional looking text because it's a low bar to clear. It doesn't require much intelligence or expertise. AIs are getting good at precisely imitating your voice with a single sample as reference, or generating original music, or creating video with all sorts of impossible physics and special effects. By your rationale, nothing “requires much intelligence or expertise”, which is patently false (even for text writing)
    • gizmo 2y ago
      My point is that writing a good book is vastly more difficult than writing a mediocre book. The distance between incoherent babble and a mediocre book is smaller than the distance between a mediocre book and a great book. Most people can write professional looking text just by putting in a little bit of effort.
• ImHereToVote 2y ago
  So content produced by think tanks was credible by default, since think tanks are usually very well funded. Interesting perspective
• factormeta 2y ago
  >In the past, you had to put a lot of effort to produce a text which seemed to be high quality, especially when you knew nothing about the subject. By the look of text and the usage of the words, you could tell how professional the writer was and you had some confidence that the writer knew something about the subject. Now, that is completely removed. There is no easy filter anymore. That is pretty much true also for other media, such as audio and video. Before digital stuff become mainstream pics are developed in the darkroom, and film are actually cut with scissors. A lot of effort are put into producing the final product. AI has really commoditized for many brain related tasks. We must realize the fragile nature of digital tech and still learn how to do these by ourselves.
• jackthetab 2y ago
  > While the professional looking text could have been already wrong, the likelihood was smaller, since you usually needed to know something at least in order to write convincing text. https://en.wikipedia.org/wiki/Michael_Crichton#Gell-Mann_amnesia_effect https://en.wikipedia.org/wiki/Michael_Crichton#Gell-Mann_amn...
• mewpmewp2 2y ago
  Although presently at least it's still quite obvious when something is written by AI.
  • chilli_axe 2y ago
    it's obvious when text has been produced by chatGPT with the default prompt - but there's probably loads of text on the internet which doesn't follow AI's usual prose style that blends in well.
    • mewpmewp2 2y ago
      Even when I try some other variation of prompts or writing styles there's always this sense of "perfectness", with all paragraph lengths being too perfect, length and the style of it being like that.
• mewpmewp2 2y ago
  Although, there were already before tons of "technical influencers" before that who excelled at writing, but didn't know deeply what they were writing about. They give a superficially smart look, but really they regurgitate without deep understanding.
• TuringNYC 2y ago
  >> While the professional looking text could have been already wrong, the likelihood was smaller, since you usually needed to know something at least in order to write convincing text. ...or...the likelihood of text being really wrong pre-LLMs was worse because you needed to be a well-capitalized player to pay your thoughts into public discourse. Just look at our global conflicts and you see how much they are driven by well-planned lobbying, PR, and...money. That is not new.