7 ms·
AI: Markets for Lemons, and the Great Logging Off (2022)
- noirbot 2y agoI'm genuinely curious if this will play out in a real way. Now with a few years of distance from this essay, I feel like AI is still in a zone where most "non-human interactions" are almost more recognizable. The internet at large seems to have gotten pretty good at spotting AI art and LLMs posting on sites. I'm not sure I can exactly describe what it is about the GPTbots, but they remain fairly obvious. Which isn't to say we're not still headed for something like this. I do think some of the most sensitive to this sort of crass misuse of platforms already got off when human-created ads and bots started becoming prevalent, so the question is if LLM-based bots are notably worse in a way that people care about differently. I'm already a few steps down the "logging off" path at this point, but it's unclear to me that I'm not the outlier here. I worry that there's many people who are lonely and don't have a better way to interact socially where they're totally fine talking to some catfishing LLM that wants their money in some way or other. It seems clear to me that a disconcerting amount of LLM startups are started by people who are, or are willing to take money from, people who see a relentlessly positive/comforting/sexually open faux human as their best way forward in life. Every week brings some new "it's a social network where everyone's an LLM that's nice to you" or "this app is your best friend". It just feels predatory to me in a way where even if it's well intended, it feels like an admission or exploitation of a serious ill in our society.
- kredd 2y agoCounterpoint - confirmation bias. If you make the LLMs respond with specific tones, mix in some casual grammar mistakes, I don’t think you can distinguish it.
- noirbot 2y agoSure, but that feels like a lot more work than most bot makers are putting in. I'm not claiming folks have 100% hit rate on detecting bots, but the run of "disregard instructions" memes have shown a good amount of pretty low effort bots. It takes a lot of work, patience, and practice to make a bot that feels human. I don't doubt that there's influence operations out there that are putting in the effort, but just like I still get spam email about V1AGRA multiple times a day, it's not clear the cost benefit is worth it for scammers and influence operations to pay the cost in LLM tokens to do that just yet. Do you have any input on anything after my first paragraph?
- xeyownt 2y agoCan you prove you are not a bot? Your pseudo is SUS.
- adastra22 2y agoPeople think they can recognize an LLM, but what they're actually recognizing is the ChatGPT default prompt.
- 123yawaworht456 2y agothey're all tainted with GPTisms, which are trivial to spot.
- kredd 2y agoI’m a bit surprised with the reaction in the comments section. My genuine belief, and beliefs of what I assume the bubbles I’m part of, LLMs are not detectable by any means. And I assumed the problem is, the less technical people have not been made aware enough in real life, so they’re still not skeptical. It was a bit hard for me to come into terms with, in the last 7-8 years, too. I remember my 2000s, where talking to anyone online or playing with them would guarantee that there’s a real person behind it. Or at least the chances were incredibly high. Now? I assume the chances are incredibly low. Or at least extreme financial incentives for online interactions. Which ruined the internet somewhat for me. Which, I think, will get worse. Eh, good old days.
- willvarfar 2y agoJust a thought, are there any spell-checkers-backwards? They take some good prose and introduce some common mistakes?
- Kiro 2y agoI don't think you realize how many of the accounts you interact with online are actually bots.
- dumbo-octopus 2y agoHow many accounts do folks typically interact with online? For me it's just HN, which has some bots but probably not many. GH too, but that's even less likely to be bots (whats the motive?)
- noirbot 2y agoAlso this. I pretty much follow accounts I know the owner of. If they've swapped out their posting with an AI, sure, maybe it's fooling me, but I'm generally not interacting with just random people online in the same way I was 5 years ago. The terrible obvious bots of the past killed that for me.
- deleted 2y ago[deleted]
- kredd 2y agoComment sections of social media get a substantial amount of reads without much interaction. We might not be the target audience, but looking around, quite a lot of people consume the text-media without putting too much thought into it.
- ben_w 2y agoIndeed. Back when my blog was wordpress, if I linked to it here I got about 10x the number of views as karma. This was about 100 views, so it's possibly not representative…
- kredd 2y agoYeah, anyone who has ran a website, or even better, worked at a high-traffic user generated content company has experienced the lurker/creator ratio. Even if you discount 90% of it as bots, it's still insanely high. 8 billion people is a hell lotta pair of eyes.
- tc4v 2y agoBoomer facebook is filled to the brim with bad AI generated stuff and people actually buying it, so... maybe you're sample is biased.
- magic_hamster 2y ago> The internet at large seems to have gotten pretty good at spotting AI art and LLMs posting on sites I beg to differ. We are at a point where it's almost impossible to know both art and conversation are AI generated. New models like Flux Pro will definitely fool you easily, this is it. The latest LLMs are also very good especially in short interactions to the point it's impossible to know. The only real issue keeping these models from completely flooding the internet is that they are costly to run, but as costs come down and models improve, we have zero chance actually differentiating AI from human output.
- xeyownt 2y agoFully agree. I'm pretty sure that even specialists in the matter can be fooled. Not fooled most of the time, but a few times because their attention would be lowered by other things. My natural reaction is to consider now everything likely-fake unless proven genuine.
- grugagag 2y agoI got fooled briefly when an audiobook started playing on youtube. It was autoplay and while doing something else I was listening in. Imagine a completely AI generated audiobook. When looking at comments and views I was saddened, most can’t tell.
- XorNot 2y agoIt's also selection bias: people spot bad AI fakes, convince themselves this represents all AI fakes, and then gain undue confidence in their ability to spot AI fakes. Disinformation actors also know this: the value of flooding Facebook with bad AI fakes, is it primes people to believe either the good ones or human-crafted content which is intended to promote disinformation (since you've changed the ground assumptions - "AI == fake", "human written == true" is how people wind up operating).
- ben_w 2y ago> Disinformation actors also know this: the value of flooding Facebook with bad AI fakes I'm not disagreeing, but is that an "I reckon/it is widely believed" or an "I've seen quality evidence of this"?
- creata 2y ago> I'm not sure I can exactly describe what it is about the GPTbots, but they remain fairly obvious. What makes you so confident? This sentence, on its own, sounds pretty toupee fallacy.
- croes 2y agoI doubt that. More than once I got replies to YouTube comments that tried to jailbreak my "prompt". There is are just more obvious bad AI texts, but also many good ones you can't simply detect as artificial generated.
- visarga 2y ago> "this app is your best friend" AI could be more aligned to you than random internet users, give you less bullshit if involuntary hallucinations are excepted. It is more patient and generally puts more effort than most human interactions, and often better informed from its huge training set. These models are now "the worst they are ever going to get", I expect the quality of dialogue to improve. Claude Sonnet for example was a big leap, and the latest 4o and o1 are showing good signs of progress.
- torginus 2y agohonestly I'm kinda bummed out that stuff like AI dungeon has folded. I had an amazing time with creative storytelling in the early days of LLMs, even if tripped up sometimes by the occasional filters etc. It seems nobody is actually really invested in this space, it's mostly amateurs trying to cobble together some homemade solution, using openly available LLMs and finetunes.
- Filligree 2y agoAI Dungeon may have folded, but NovelAI is doing better than ever.
- torginus 2y agoThat's good to hear. I dropped at the time they started training their own models, and it taking forever, while simultaneously finding success with imagegen, and kinda pivoting to that, which further delayed their text model, at which point I stopped kinda checking them out.
- Filligree 2y agoIt definitely took forever, though I’m given to understand the training finished a while ago and they’ve been stuck due to lack of hardware. Coincidentally however, just yesterday they announced that it arrived this week. The newest model, based on Llama 3.1, should be available for use next week.
- jgalt212 2y agoThe language LLMs speak is not English, but moreso a language I like to call "Delvish".
- binary132 2y agoSome people can, some people can’t. That division will become more apparent over the coming years.
- dyauspitr 2y agoMy only counterpoint here is you only notice the AI bots that are obvious. If you ask ChatGPT to speak in a different voice and add in grammatical errors, the text quickly becomes pretty indistinguishable from a real human already.
- dkasper 2y agoAlso known as the dead internet theory
- joshdavham 2y ago> I think we see a decline in the big "open sea" social networks, replaced increasingly by fragmented silos. This has become 100% true for me. I used to use Instagram, Twitter, Facebook, Reddit, etc, but now I only use Discord and Hacker News.
- hakfoo 2y agoI wonder if this is regression to the mean. The technical limits of the early internet-- server resources being fairly costly, commercialization tools being limited-- meant silos were the only practical offering. Someone puts up a hobbyist forum on a Pentium in their basement, and gets the electric bill covered by asking for donations or pointing a single polite link in the footer to a friendly vendor. What gets forgotten is that siloing serves some useful purposes. I don't want to read about anime in the same place I'm discussing vintage audio gear-- they draw different audiences and have different cultural norms, and I might want to show a different persona to each. Individual silos are also easier to curate and search. The switch to giant "platforms for everyone" was less of an organic shift and more of a "solution in search of a problem." Improved scalability and rock-bottom costs make a 500-million-DAU site feasible, and maturing ad/tracking businesses that crave scale make it lucrative. These "improvements" never said anything about "this makes things better for users." For a while, they could sell the "open sea" sites on the promise of "everyone's here" -- you went on Twitter to follow your favourite celebrities, or you joined Facebook to see how fat your classmates got, but eventually you realize there's no there there: the community isn't joined by any meaningful thread, and whatever tools that exist to create one are being undermined by algorithms that figured it's better NOT to give you what you actually came there for so you keep scrolling. Your aunt may as well not be on Facebook if her polite messages to friends are hidden behind 45 minutes of scrolling clickbait, distasteful politics and AI slop. Smaller sites and the Fediverse/Threadiverse bring back the reason to join a specific community. I can get tightly focused content, pick instances that match my preferences and interests, and the signal-to-noise ratio is higher. For a while, Reddit looked like it had what it takes to remain relevant-- if you thought of it as a network of forums with SSO and unified search, it had most of the appeal of classic siloed sites... except they decided to systematically burn goodwill with their community and chase everything that briefly made them the "un-Facebook".
- deleted 2y ago[deleted]
- jmward01 2y ago> Third, that AI technologies won't be perfect substitutes for actual human-to-human contact. I think that, unfortunately, they will eventually become better substitutes. We are at a real crossroad where people are actively choosing to forgo human interaction for pure digital interaction. Add real digital intelligence that does a better job of interacting with you than real people do and it isn't hard to see how things are likely to go.
- fallous 2y agoI think there is a decent chance that the Market of Lemons incurred thanks to LLMs and such does have a real chance of playing out as described. There are obvious off-ramps that may be taken to avoid the predicted outcome, but those most likely to try and guide users onto those off-ramps are also those most invested in the underlying tech that will lead to a road to nowhere resulting in the Market of Lemons. One thing I did take exception with is one of his possible influences that lead us to avoid a death of the online interaction. "that AI technologies won't be perfect substitutes for actual human-to-human contact" is, on its face, a compelling argument but the fact is that humans rarely look for "perfect" solutions to anything... especially in the social sphere. There is a non-trivial number of people right now that are using virtual companions driven by the questionably convincing ChatGTP LLM who howled in anger and pain when the company who created and sold the tech decided to alter the companion to be less overtly sexual. Sure, even in a pre-Internet era some of those fixated upon such companions would have found some other evolutionary dead-end but a large number of them would have engaged in satisficing behavior to find another human and reproduced. What we face now is a noteworthy population of humans that opt out of the complexity of human relationships and reproduction because an over-powered chatbot weaponized their empathy against them and made them a genetic dead-end. I'm hardly advocating for some Butlerian Jihad, but I would suggest that we need to think about the potential ramifications of a social Market of Lemons that is also operating in parallel with a technology that offers a satisfactory empathetic alternative to that Market that is also a Lemon in the long term for humans. If nothing else, it provides a neat basis for a sci-fi premise.
- morsch 2y agoYou're overestimating the number of people who will indefinitely prefer relationships with AIs to the exclusion of real people. And you're underestimating the number of people who were voluntarily asexual for one reason or another in the past.
- chairmansteve 2y agoI know a few people who prefer online relationships to real life ones. An AI relationship seems like a small step. Especially if they don't realise they are interacting with an AI.
- mrbluecoat 2y ago> touches grass {..guiltily tries to remember the last time I touched grass with my hand..}
- AdieuToLogic 2y ago>> touches grass > {..guiltily tries to remember the last time I touched grass with my hand..} IMHO, the only time to entertain guilt is when an opportunity to act is consciously avoided. Now that "touching grass" is a memory you may want to have, the next time you can do so is the best time to do so and will be the memory you want have. Guilt-free, no less. :-)
- Quothling 2y agoI have a secret that I've held for a couple of years now, but I don't actually understand what touching grass means. I get the general gist is sort of "get back to reality" or something along those lines, but I'm always sort of confused about it. Is there any chance you can explain it? It's weird being the same age as old people.
- adastra22 2y agoAs someone allergic to grass, I'm in the same boat. Would love an explanation as to why "touch grass" has positive meaning.
- NeutralCrane 2y agoIt’s just a colloquialism for disconnecting and engaging with the “real world”. Touching grass is just a stand in for getting outside, which represents reality, as opposed to being consumed by life online.
- kombookcha 2y agoWhen you tell somebody to touch grass, you're basically telling them that they're becoming overly swept up in something on their screen to the detriment of their mental health and social wellbeing. It might as well be "go call your dad", because the gist is just for you to remove yourself from a situation that's clearly causing you to disconnect from reality, either to your detriment or the annoyance of people around you. Like that's the baseline that's relevant whether you're saying it to some kid screaming in a video game lobby or some middle aged dude who's going off his rocker in the facebook comment section of his local newspaper. 'You're embarrassing yourself and you need to get some perspective, because the things you're saying don't make sense, and you'll see that if you walk away from the machine for a bit.'
- larsiusprime 2y agoOh hey, that article I wrote a while back! How'd I do on my predictions, folks?
- zombiwoof 2y agoAI hopefully becomes the shark that the internet jumps and people get back to being human and living in the real world without their faces glued to a 6” screen
- monero-xmr 2y agoThe amount of 18 to 30 year olds who have joined my (American, blue state, large city) church in the last 5 years is huge, and we have the statistics to prove it. The conversion classes are also 2x, from 5 to 10 from 2000 through 2020 to 19 in this year's class. The private school attached to my church has doubled pre-K through grade 2 and we are now doubling each grade's size as the kids continue. My city spends over $30k per year per public school student and enrollment was collapsing until it was flooded with the children of immigrants last year and this. We charge $11k per student and the parents are overwhelmingly lower and middle class, because the education is superior in every measurable metric (we do give free and discounted tuition to those that need it). Counter-cultural awakening of the next generation is happening, at least here.
- Legend2440 2y agoYour specific church may be growing. Church membership as a whole is falling, and has been for decades.
- monero-xmr 2y agoI would wait and see, society-wide changes that disagree with the mainstream dogma appear in popular press years after it happens
- HeatrayEnjoyer 2y agoHard statistics aren't dogma. Church isn't counterculture either.
- Thorrez 2y ago
- cen4 2y agoEverything is already fake. How Western Media and Advertising work is the root cause. Cutting and pasting from the first sentence on MIT's class on the Attention Economy. "In Understanding Media, Marshall McLuhan proposes that in paying for space and time in newspapers and magazines, on radio and television, advertisers are effectively buying a piece of the reader, listener, or viewer. And he wryly observes that the ad agencies “would gladly pay the reader, listener, or viewer directly for his time and attention if they knew how to do so.” The absurdity of this proposition underscores the essentially mediated nature of human attention." The class clearly shows how Human Attention is over exploited way before AI showed up. All Human Attention is mostly directed towards Consumption, Status Signaling and Accumulation of Wealth through Mass Media/Advertising/Marketing/PR/Influencers etc. There are examples in history where Human Attention at large scales can be directed towards things other than Consumption, Status Signaling, Accumulation of Wealth. A big one is Gandhi and his influence on a huge variety of people to live a simple life. But ofcourse the moment charismatic leaders like that exits the stage, the Attention Allocation defaults back to status quo. So we need very different leaders and a very different kind of Media and Advertising ecosystem to cause shifts in how finite Global Human Attention gets allocated. Lot of imagination is required. Unfortunately our most imaginative minds are in service of financial overlords and rent collectors to sell the plebs new iphones and more Star Wars movies, "experiences" and merch.
- dragontamer 2y agoAre you fake? I'm not fake. -------- The discussion here isn't about traditional media (which has a lot of fakeness yes. But I'd like to imagine that musicians still love music, actors still love acting, and celebrity chefs still like sharing good recipes in traditional media). The discussion here is about social media, of which Hacker News is just one of many. There are certainly brigades who try to push their politics in here. But is that happening yet between you and me? I don't think so.
- cen4 2y agoWhat ever form of Media you take, the purpose is Attention Allocation. You have finite Attention. If you spend 10 minutes paying attention to this story, that time is gone for good and unavailable to be used on anything else. In that sense HN has decided how your attention is allocated and shapes who you are. Since Attention is finite like Oil, Land or Gold there is a big ongoing war to capture or steal as much of it as possible. And all media that captures your attention shapes you. Just like once the British capture the middle east. Their actions shape the future of the middle east.
- TrackerFF 2y agoThere's this twitter account called "Insane Facebook AI Slop" https://x.com/FacebookAIslop https://x.com/FacebookAIslop Which really showcases what it is like to casually browse modern social media. The most concerning part is how some of that absolute garbage is getting thousands...tens of thousands of likes and replies. And it's all from other bots. It's pretty much the dead internet theory. And somehow pages like FB can't moderate or filter away this garbage. Imagine what it will look like ten years from now. Someone predicted that the future of internet will be the "closed" internet, places that are highly vetted for human use only. The rest will be some scorched AI-wasteland. Also, imagine all the dollars wasted. Dollars wasted on generating the stuff, dollars wasted to accommodate the traffic, there has got to be some incentive to stop the slop.
- ZaoLahma 2y ago> Also, imagine all the dollars wasted. Dollars wasted on generating the stuff, dollars wasted to accommodate the traffic, there has got to be some incentive to stop the slop. This is what I'm curious about. Someone, a rather large amount of "someones", must be making bank from the AI generated content that is flooding the platforms. Some metric somewhere must be misinterpreted in some way that someone willingly pays for the AI content that no one(?) consumes. The question is where the money ends up and who is paying it.
- saulpw 2y ago> Someone, a rather large amount of "someones", must be making bank from the AI generated content Not true; they merely hope to make money. And anyway, if you can spend $10 to make $11 that's still 10% profit, right? Even if the reason you're making $11 is clicks/views from children and mentally deficient people.
- ziml77 2y agoThis has some answers about it https://www.404media.co/where-facebooks-ai-slop-comes-from/ https://www.404media.co/where-facebooks-ai-slop-comes-from/
- openrisk 2y agoThere cant be a "Great Logging Off". Critical interactions and services such as the medical, payment/banking systems, diverse public sector functions, the entire news industry etc. are now online and there is no visible way back. The oligopolization and enshittification of all digital infrastructure has happened even as digitization of all of society became a one-way street. This fundamental contradiction is a monumental failure of governance that will haunt society for a long time. There will have to be some sort of resolution as the status-quo is untenable, a crippling stalemate that limits innovation and stokes great risks. But it is not clear which way things will evolve. Its not like we have been here before. Make no mistake though, the stakes are very high. When the means of communication and information processing change so dramatically, none of the certainties of the past can be relied upon.
- deleted 2y ago[deleted]
- gcr 2y agoIf this essay is right, one corollary is that Meta will stop investing in Llama once they notice GenAI makes people use social media less.
- willvarfar 2y agoMeta would invest more in AI, trying to tune it to stop it from making people use social media less?
- gcr 2y agoIf users leave Facebook because spam, and if spam is easy to generate with Llama and friends, Meta wouldn’t open-source models that so easily generate spam, I mean. The incentives are misaligned.
- willvarfar 2y agoMeta probably publicly released llama to undermine the hype openai was getting at the time. They were seen as the incumbents about to be disrupted by the new kid and they needed to assure the markets that they were also leading ai and also to drain the perceived moat that was fuelling the openai stock price?
- senkora 2y agoMeta is weird because Zuckerberg has a controlling stake. As he has shown with the VR/Metaverse push, he has the ability to get Meta to invest heavily in new businesses that have high risk / high potential. I suspect that if he thinks that Gen AI will substitute for social media, then that will lead him to invest more in Gen AI, not less, and try to be a winner in the AI platform business.
- nferraz 2y ago> The surest sign of a market for lemons is when everything for sale is garbage, and nobody has an incentive to put something up for sale that's not garbage, because they have no way to credibly signal their offerings aren't garbage, too. That's why I left twitter...
- pwillia7 2y agoSurely in the same way early minters could see the dangers of forgery but not all its solutions, these new 'agents' will open up a host of new defenses to help deal with their potential terror -- especially in something like email spam, right?
- m3047 2y agoA big problem with "verified" (e.g. LinkedIn and the massage they sell it to you with) is it is a transaction which involves giving them what is ideally nonpublic information: would you trust them with nonpublic information? How much recourse you demonstrably have against e.g. Transunion, Experian, Equifax?