4 ms·
I'm a huge Neal Stephenson fan. Cryptonomicon is to this day one of my all-time favorite books. Years ago now I read Anathem. It wasn't as good but it had some
by cletus 2y ago
I'm a huge Neal Stephenson fan. Cryptonomicon is to this day one of my all-time favorite books. Years ago now I read Anathem. It wasn't as good but it had some really interesting ideas.
One such idea was how the Internet was filled with garbage by all these agents (which were implied or stated were AI, I can't recall). They would subtly change things to be wrong. Why? Essentially to sell you a solution to this that filters out all the crap.
Currently we rely a lot on altruism for much of the information on the Internet (eg Wikipedia). AI agents will get harder and harder to differentiate from actual humans making Wikipedia edits. I don't think we're that far away from human-vs-AI Wikipedia edit wars.
I really wonder how much human knowledge will be destroyed by (intentional or otherwise) AI vandalism in the future.
- pavel_lishin 2y ago> “Early in the Reticulum-thousands of years ago-it became almost useless because it was cluttered with faulty, obsolete, or downright misleading information,” Sammann said. > “Crap, you once called it,” I reminded him. > “Yes-a technical term. So crap filtering became important. Businesses were built around it. Some of those businesses came up with a clever plan to make more money: they poisoned the well. They began to put crap on the Reticulum deliberately, forcing people to use their products to filter that crap back out. They created syndevs whose sole purpose was to spew crap into the Reticulum. But it had to be good crap.” > “What is good crap?” Arsibalt asked in a politely incredulous tone. > “Well, bad crap would be an unformatted document consisting of random letters. Good crap would be a beautifully typeset, well-written document that contained a hundred correct, verifiable sentences and one that was subtly false. It’s a lot harder to generate good crap. At first they had to hire humans to churn it out. They mostly did it by taking legitimate documents and inserting errors-swapping one name for another, say. But it didn’t really take off until the military got interested.” > “As a tactic for planting misinformation in the enemy’s reticules, you mean,” Osa said. “This I know about. You are referring to the Artificial Inanity programs of the mid-First Millennium A.R.” > “Exactly!” Sammann said. “Artificial Inanity systems of enormous sophistication and power were built for exactly the purpose Fraa Osa has mentioned. In no time at all, the praxis leaked to the commercial sector and spread to the Rampant Orphan Botnet Ecologies. Never mind. The point is that there was a sort of Dark Age on the Reticulum that lasted until my Ita forerunners were able to bring matters in hand.”
- greendestiny_re 2y ago100 good sentences and 1 subtly wrong one? That makes for an extraordinary text! I suppose the idea sounded better in the writer's head but it would not be nearly as bad as implied.
- mvdtnz 2y agoI'm re-reading Anathem now (my favourite book). Something I noticed on my second reading is Stephenson describes Sammann's physical appearance exactly once in the book and I must have missed it the first time around. I always pictured the Ita as wearing elaborate robes with hoods darkening their faces due to their secretive nature. But in fact he describes Sammann as looking basically just like Gilfoyle from Silicon Valley, including the way he dresses. Which is amazing given the roots of the Ita (he describes the word as coming from Information Technology and the meaning of the A is lost to time, but it's obvious to a 20th century Earth-born reader it comes from IT Administrator). There are so many delightful details in Anathem, it's well worth a second reading.
- pavel_lishin 2y agoI don't think I remember a description of Sammann's clothes, aside from a hat. I always pictured them as wearing something similar to what Orthodox Jews wear, except in darker colors.
- gamepsys 2y agoThe same idea is explored in more details in his later book 'Fall; or Dodge in Hell' where AI generated individualized content radicalizes parts of the population and convinces them to believe blatantly false facts. It's a decently large B plot in the novel that ties into some of the larger themes.
- soco 2y agoWe didn't need AI for that, but it surely helps a lot making it easier possible.
- visarga 2y agoWe can always rely on pre-2022 data. But I guess what you're saying is plausible, I see it becoming like the adversarial game of virus vs immune system, a constant arms race. We got to build our immunity. On the one hand AI can churn bullshit on command, on the other hand training on large datasets tends to cancel out many errors across the corpus. All the more reason to use local models and curated feeds from now on. Local LLMs can clean / firewall the bad stuff, and follow our guidance. They will be like the new anti-virus software. I've predicted early in 2023 that in the future operating systems and browsers will all sport a small LLM that will ensure we don't get abused by the internet and provide a "room of our own", where we have total privacy. It's already a reality.
- tivert 2y ago> We can always rely on pre-2022 data. No we can't. How many websites from 1998 survive today, in a form you can actually find (e.g. not the Wayback Machine)? In ten or twenty years, most pre-2022 data will be inaccessible.
- 93po 2y agoIt's interesting how often I have to look at timestamps on content on the internet these days. I assume anything after 2022 that has the slightest whiff of ChatGPT wordiness is probably slop