8 ms·
On the non-use of AI in my writing process
- ainch 2mo ago> It would be foolish to deny the effectiveness of image recognizers based on generalized adversarial networks (GANs), the key neural network technology underlying LLMs I could be misreading this, but I hope the author doesn't think GANs are used in LLMs. They are cool, though.
- chpatrick 2mo agoYeah, it's hard to take the rest seriously.
- pasquinelli 2mo agomaybe you're looking for an excuse to ignore what's being said.
- llm_nerd 2mo ago[flagged]
- camillomiller 2mo ago[flagged]
- sigilsack 2mo ago[dead]
- llm_nerd 2mo ago[flagged]
- dgellow 2mo agoYou‘re both contributing negatively to the discourse by doing that sport team fighting…
- camillomiller 2mo agoOf course, quality of the discussion above all else, including destroying the world and the fabric of society via a financial grifting scheme that is commoditizing free knowledge.
- deleted 2mo ago[deleted]
- scarmig 2mo agoThings that are vaguely GAN-shaped might play some role in post training. Though no one would call them GANs or identify them as the key underlying technology.
- bonoboTP 2mo agoThere is RLHF with a model that's trained to emulate a human evaluator, but the evaluator is not really trained jointly with the main model to adapt to its distribution and tell it from real text. Though I'm sure there are some niche cases when this is done. But definitely not a prominent thing.
- threethirtytwo 2mo agoHe hallucinated. This is how you verify non AI nowadays. AI is so good that if you see an hallucination as obviously wrong like this one it’s a sign it’s written by a human.
- sxp 2mo agoYeah. Ironically, if he used an LLM to proofread his work, it would have told him that GAN's are generative not "generalized". That LLMs are primarily built on Transformers rather than GANs. And that they're more famous for image generation rather than image recognition.
- FeepingCreature 2mo agoAnd that GANs haven't been relevant in image generation since 2021.
- bonoboTP 2mo agoTransformers are an architecture and GANs are a training method for the architecture. There are GAN Transformers.
- nullstyle 2mo agoExamples for us less educated folk?
- Alpha3031 2mo agoJiang et al. (2021) TransGAN: https://dl.acm.org/doi/10.5555/3540261.3541391 https://dl.acm.org/doi/10.5555/3540261.3541391 In general probably not much of a stretch to get rid of convolutions or recurrence by replacing with attention and see if it works hence the title of the original transformers paper.
- bonoboTP 2mo agoAnalogy "it's not GAN, it's transformer!" - - "it's not a recursive implementation, it's object oriented!" if you are more familiar with CS. Or "it's not 4-wheel-drive, it's diesel!", if familiar with cars.
- andy99 2mo agoIt’s unfortunate - he’s an author, it would have been fine to stick with an authors perspective that LLMs can’t write (which is true), as well as the copyright stuff (which I don’t agree with but he certainly has standing to give an opinion on). But he’s made the error of trying to come at it from a technical perspective, when he clearly knows nothing about that side of things, which discredits the rest.
- magicalist 2mo ago> But he’s made the error of trying to come at it from a technical perspective Huh? That quote appears to be the entirety of the "technical perspective" of the post and is an aside from his larger points that you have blessed as "fine". Literally nothing in the rest of the post relies on that incorrect statement. Let me quibble with what is discredited here, given the entirety of your point is built upon an error.
- red75prime 2mo agoHere's another technical point: > They're word-association mechanisms with no embodiment and no way to associate the text vectors they manipulate with real-world phenomena. This is wrong too. RLVR grounds foundational models in reality.
- Alpha3031 2mo agoAren't RLVR signals typically based off formal systems not natural phenomena? The formal sciences are certainly useful for producing tools used in natural science, but it's not entirely clear they alone are sufficient to associate text to natural (real-world) phenomena. Honestly, the pretraining and RLHF are probably more tied to the world than RLVR, for all that RLVR might be useful (maybe even more useful) for making the model perform better in certain tasks (such as working with formally defined systems).
- azakai 2mo agoIf you want a more concrete example, then LLMs are also trained on visual data these days, which means they do have access to the world in an important way. This directly contradicts the blogpost's claim that LLMs have > no way to associate the text vectors they manipulate with real-world phenomena. Historically, that LLMs were text-only used to be a major argument for why they "lack access to meaning", see the Stochastic Parrot paper and the Octopus paper that it references. But even the authors of those papers have (grudgingly) conceded that the argument no longer holds due to multimodality.
- Apocryphon 2mo agoFrom the comments: > I will concede that using LLMs in software development is a different category from fiction, insofar as the languages and APIs are much smaller and more tightly constrained and in software you actively don't want long-range dependencies between elements (something you often do want in fiction, i.e. put a gun on the mantlepiece in part 1 of the book, pull the trigger in part 3).
- bonoboTP 2mo agoIt can definitely plan such things. Because agentic systems don't have to output a book all at once. The mainstream still lives 1-2 years in the past. Agents can plan out a narrative draft, write it out section by section, maybe out of order, then tweak it, plant clues if it wants to, then trigger it in another chapter file etc. Then read the whole thing, reconsider, etc. It can iterate. I'm not saying it will be great literature, but the problem isn't the inability to create long range references. My bigger point would be this: arguing from such mistaken technical ideas, when not having any idea about the tools is a sign that people's real problem is none of those technical limitations, but Western culture has lost the vocabulary to express their true sentiment. They are grasping for words but can't reach for "soul" and other religious terms because that would sound ridiculous in a secular modern world. But they clearly want to express that and we'd be ahead if we could discuss in such actual terms. Would you want the machine to write literature if, hypothetically it could write indistinguishably from any great writer? I guess not. Because there is something mysterious and important about the human spirit. (the same vocabulary-inaccessibility is plaguing other culture war issues too). People will rather talk about fake water consumption problems than touch the topic of the human soul or spirit.
- dgellow 2mo agoWhat are you talking about… You can use the word soul without believing in a religious concept of soul. It’s done all the time in secular societies. It’s even a very common term used to describe the problem with LLM generated content, that it lacks soul or human spirit, it lacks the human meaning, intent
- aaron695 2mo ago[dead]
- sxp 2mo agoWhile Stross is currently a neo-Luddite who doesn't believe in the Singularity, his book Accelerando is probably the best work of Singularitarian fiction ever written. It was written in the 2000s, starts in the 2010s, and covers the various decades of this century. I highly recommend it for anyone who wants to know how weird this century will be. It's also available for free online. [1] It's one of my favorite books and the main reason I'm e/acc and many other pro-AI people love it because we view it as utopian sci-fi rather than the dystopian world Stross invented. This might be the best case of the Torment Nexus meme [2] in action. 1. https://www.antipope.org/charlie/blog-static/fiction/accelerando/accelerando-intro.html https://www.antipope.org/charlie/blog-static/fiction/acceler... 2. https://en.wikipedia.org/wiki/Torment_Nexus https://en.wikipedia.org/wiki/Torment_Nexus
- bananaflag 2mo agoHe had already disappointed me in 2011 with this post https://www.antipope.org/charlie/blog-static/2011/06/reality-check-1.html https://www.antipope.org/charlie/blog-static/2011/06/reality...
- amanaplanacanal 2mo agoWhy is that disappointing? Everything he said there seems perfectly reasonable.
- ben_w 2mo agoA large part of that goes via arguing about consciousness. Starts of with a quite defensible: First: super-intelligent AI is unlikely because, if you pursue Vernor's program, you get there incrementally by way of human-equivalent AI, and human-equivalent AI is unlikely. But the moment he hits that (parenthetical) paragraph about consciousness, he blends human capabilities with human-like consciousness. If human-like capabilities required human-like consciousness, early models (like GPT-4) passing all those exams would suggest those models being "as conscious as" a university student. (My inner Douglas Adams wants to invert this, but the best I can do is "The Xarloxians did not consider university students to be conscious beings; their reasoning being…", which you may recognise as a riff on Babel fish and the non-existence of God).
- FL33TW00D 2mo agoCrazy how forward thinking this guy was in 2005 vs today.
- forgetfreeman 2mo agoCrazy to assume that an individual of vision has so thoroughly dropped the plot simply because their position is at odds with tech industry orthodoxy. If anything that should be an indication to more closely examine your assumptions.
- shimman 2mo agoAnyone that is against American big tech immediately makes me sympathetic to their side. They'd have to do a lot wrong to destroy such good will.
- forgetfreeman 2mo agoI'm with you. If the last 30 years have proven anything it's that enthusiasm for tech industry hype is at best historically illiterate.
- deleted 2mo ago[deleted]
- deleted 2mo ago[deleted]
- keeda 2mo agoI just left this comment: https://news.ycombinator.com/item?id=49137863 https://news.ycombinator.com/item?id=49137863 -- tl;dr I suspect it's his view of the Tech industry and Capitalism that is coloring his views, rather than an objective evaluation of state-of-the-art LLM technology.
- card_zero 2mo agoSpeaking as a fucker, I think the use of em-dashes has been pretentious since the 1890s. The rise of typewriters destroyed em-dashes already. The LLMs subsequently created an accidental parody of sophisticated writing.
- cindyllm 2mo ago[dead]
- tptacek 2mo agoWhat's pretentious about them? They express a distinctive rhythm in written English. What's the the punctuation you would use instead?
- card_zero 2mo agoIt's like saying I am intimately involved in the printer's art. This is usually not true, unless you're involved with a very niche publisher who still has a box of sorts and inky fingers. Hyphen-minus all the way.
- exe34 2mo agoNo it doesn't. I happened to have read "design and typography in easy steps" as a (bored) child of the 90s, but that's the extent of my "printer's art" knowledge. I don't understand the celebration of ignorance myself. There are lots of things I don't know how to do or how to use, but I'd never dream of complaining because somebody else does.
- tptacek 2mo agoDude's an artist. He should do what his artistic intuition tells him to. I'm a software developer. I'll follow my own experience. It'll all work out.
- esperent 2mo agoFor some value of "work out" anywhere between "human utopia that spreads across the stars" to "we destroy everything in a ball of nuclear fire and only microbes survive", yes, it will. The universe will keep on going either way.
- sodapopcan 2mo ago"Whatever it is, it's progress, and we're doing it!"
- nullstyle 2mo agoThe only way out is through. Unironically
- sodapopcan 2mo agoDon't I know it.
- sodapopcan 2mo agoHe says exactly this in the comments on his page. This article is specifically about writing/art.
- forgetfreeman 2mo agoRoughly a third of the market is digging a $1T-a-year-deep-hole with <5yr amortization schedule on all of it and passing around the same $100B like it's in all of their bank accounts simultaneously. Yeah, shit's going to work out for sure, in all of the same ways that flying a plane into the side of a mountain has a clear ending.
- 2mo ago
- sxp 2mo ago> I'd quite like a tool (running entirely locally on my own hardware, with no cloud service and no copyright-thieving grifters making bank on it via subscription fees) that digests a manuscript and derives a scene-by-scene timeline, that I could then query interactively and use to plan my next round of edits. Being able to map out where and when each protagonist and minor character shows up, and see a frequency distribution heat map of names in the manuscript, would be useful. Interestingly enough, I do this with various books I'm reading. E.g, I'm currently rereading Accelerando and had Claude generate a wiki-like timeline of key events, characters, and salient plot points. That makes it easier to jump around when I want to re-read a section and grok a plot thread that is scattered across chapters. Ironically, it also exposes inconsistencies (or "hallucinations" as some might call them) in the text because the author didn't have an AI proofread the text.
- derektank 2mo agoAre you able to share? Both your process and the specific timeline for Accelerando. I would love to build one for A Fire Upon the Deep
- sxp 2mo agoI'm not going to share because https://www.antipope.org/charlie/blog-static/fiction/accelerando/accelerando.html https://www.antipope.org/charlie/blog-static/fiction/acceler... has a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 License. But I've paid for my copy of the book so I can do what I want with it. If you want to build your own, you can feed this DESIGN.md to your favorite clankie: https://pastebin.com/huRkGnba https://pastebin.com/huRkGnba
- bitexploder 2mo agoYou could build a tool like that and on a MBP with 48+ GB of RAM all of that can happen locally in terms of keeping your content locally. I am sort of an outliner and planner and I heavily use AI when writing documents for work. I don't see why fiction / prose would be any different? Indexing your content with a vector RAG and having a little 27B model locally could do everything he wants with some help from a big model to implement it all. It sounds like a weekend of coding for a functional prototype to me.
- LogicFailsMe 2mo agoI don't think it ever gets old to restate that the singularity and all of the gradiose promises of a glorious 21st century have last mile problems. But this post seems to mostly channel Harlan Ellison. And while he was amazing in his prime, with some absolutely legendary rants, he didn't age well in the end. I do like his challenge to make LLMs useful to himself and to other authors though. Someone needs to make that happen and nearly exactly how he described it.
- ben_w 2mo agoI'd quite like a tool (running entirely locally on my own hardware, with no cloud service and no copyright-thieving grifters making bank on it via subscription fees) that digests a manuscript and derives a scene-by-scene timeline, that I could then query interactively and use to plan my next round of edits. Being able to map out where and when each protagonist and minor character shows up, and see a frequency distribution heat map of names in the manuscript, would be useful. I'm sure I've read about Hollywood scriptwriters having tools like this. (Or is this just the Gell-Mann amnesia effect striking again?)
- LogicFailsMe 2mo agoGenAI tools getting folded into workflows is IMO the endgame here. Using an LLM to review a story in progress and search for plot holes, contradictions and everything else he described seems like it would be really useful to me in the same way coding agents are great for diagnosing gnarly config and container issues. In my own use case, asking the LLM to find a way to close out a song verse when I can't find the right words has been awesome. That seems like an advance on the toolchain described here: https://www.thewritersforhire.com/11-great-organization-tools-for-writers/ https://www.thewritersforhire.com/11-great-organization-tool...
- visarga 2mo ago[flagged]
- andai 2mo agoIt kinda looks like we can only train AI on dead people's data. I wouldn't be too upset about that. They write better anyway.
- tim333 2mo agoShould maybe rather than can?
- andai 2mo ago> They're word-association mechanisms with no embodiment and no way to associate the text vectors they manipulate with real-world phenomena. Doesn't most of this also apply to a guy living in The Matrix?
- ben_w 2mo agoYes, but also for humans who learn of distant lands by reading (books or news, just so long as it's reading). And also these models have been associating with real-world phenomena from the first moment their training data did, and also those text vectors are (to varying degrees) associated with corresponding image vectors in multimodal models. Of course, the Plato's cave critique would still be valid.
- ericpauley 2mo agoSearle* strikes again! * https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
- satvikpendem 2mo agoAlso, AI effect: https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect
- scarmig 2mo agoEmbodiment doesn't exist, even in regular humans. We do not have direct access to reality; we have sensory inputs that are much lower bandwidth than one might expect but correlate with external events, and our brain uses them to form rich world models that are usefully predictive. Most of the world we experience is just in our head.
- logicallee 2mo agoI get what you're saying (mostly based on low bandwidth from sensory organs as opposed to direct access to reality), but your conclusion that we therefore don't have embodiment goes very far. It would be like saying planes don't really fly, since they fly by wire and only have limited inputs and outputs, rather than direct access to reality itself. Well, yeah, they fly using sensors rather than knowing reality itself, but they're still flying. Humans still obviously have embodiment.
- Fricken 2mo agoNot being able to see the forest for the trees I suppose is a prerequisite for success in Silicon Valley, because it one had even a fraction of Stross's perspective on the matter they'd be in a different line of work.
- keeda 2mo agoThis is very interesting coming from an author in whose writing autonomous, super-powerful AIs have been a common theme. Consider his book Accelerando (which I'll take the opportunity to plug again, especially as he's made available for free here: http://www.accelerando.org/fiction/accelerando/accelerando.html http://www.accelerando.org/fiction/accelerando/accelerando.h...) Not only did I find it quite engaging and thought-provoking, it is also proving rather prescient, and even helpful in decoding some of the things that are happening today. For instance, in the very first chapter the protagonist spawns agents to go research something in the background and report back to him. And then a year ago, I randomly became curious about a rather involved topic (how fast could we feasibly replace all human labor with robotics), but I did not want to spend time researching so I outsourced it to Google Deep Research which churned away for almost half an hour and came back with a 30 page report with 49 citations via "actual internet searches for verifiable sources." (If you're curious about the conclusion: not for a very long time partly due to critical supply chain constraints.) After I went through the report, it suddenly struck me: my agent may have executed in a GPU cloud instead of a cybernetic brain, but holy crap I had literally just lived a SciFi scene! A scene that I did not expect to experience in my lifetime! Which is why I found TFA a bit unexpected. TFA says a few things that I would disagree with. Like, no, AI is not a "stochastic parrot" and I'd assume he'd be primed to realize it. And AI is not competing with authors -- other authors with AI are, and using AI trained on real text to aid writing has been a thing since the days of red squiggly lines in word processors, which TFA even acknowledges. If you see his last few comments (https://news.ycombinator.com/user?id=cstross https://news.ycombinator.com/user?id=cstross) it's clear he's pretty negative on the tech industry and Capitalism, which I tend to agree with. I can see how that could color his thinking. That may also mean he's avoided LLMs to the extent that he is not aware what the frontier models have become. I would encourage him to put aside his distate and re-engage with them deeply; maybe he'd be at least a little bit excited to see some of his writing turn out to be prophetic in some good ways besides the bad.
- lukeschlather 2mo agoStross complains about people from China hammering his blog and stealing his work, but I feel like China releasing all these free models is a lot more defensible. Yeah, they're pirating a bunch of stuff and stuffing it into a blender but they're giving away the resulting soup for anyone who's hungry. (And though running it on your own hardware is a stiff proposition, I would bet Kimi K3 can figure out the scene-by-scene timeline thing.)