4 ms·

I remain hopeful that some day someone will train an LLM which is tolerable to people who take this stance (which I respect, much like I respect food vegetarian

by simonw 6mo ago

I remain hopeful that some day someone will train an LLM which is tolerable to people who take this stance (which I respect, much like I respect food vegetarians despite not being one myself). I've been tracking models trained entirely on out-of-copyright data, for example. I've not yet seen one of those which appears generally useful and didn't chuck in a scrape of the web or get fine-tuned on examples generated by a non-vegetarian model. Andrej Karpathy can train a GPT-2 class model for less than $80 now, so at least the environmental cost of training may drop to a point that it's acceptable to LLM vegetarians: https://twitter.com/karpathy/status/2017703360393318587 https://twitter.com/karpathy/status/2017703360393318587 Why do I care? This post is a great example. If you're a professor of computer science I really want you to be able to tinker with this fascinating class of models without violating your principles. UPDATE: Huh, speaking of potentially vegetarian models, I just saw https://talkie-lm.com/introducing-talkie https://talkie-lm.com/introducing-talkie on the HN homepage https://news.ycombinator.com/item?id=47927903 https://news.ycombinator.com/item?id=47927903 I've explored I different out-of-copyright trained model Mr Chatterbox before but found it to have been mildly corrupted through the help of synthetic conversation pairs from Haiku and GPT-4o-mini - https://simonwillison.net/2026/Mar/30/mr-chatterbox/ https://simonwillison.net/2026/Mar/30/mr-chatterbox/ Talkie isn't entirely pure either though: "Finally, we did another round of supervised fine-tuning, this time on rejection-sampled multi-turn synthetic chats between Claude Opus 4.6 and talkie, to smooth out persistent rough edges in its conversational abilities."

infotainment 6mo ago
> Andrej Karpathy can train a GPT-2 class model for less than $80 now, so at least the environmental cost of training may drop to a point that it's acceptable to LLM vegetarians: https://twitter.com/karpathy/status/2017703360393318587 https://twitter.com/karpathy/status/2017703360393318587 I suspect that even if you reduced the cost of training or any other real world metric, the goalposts would immediately move. It seems to me that it has never been about those things, but simply about the feeling of superiority one can attain by eschewing something seen as trending.
- WatchDog 6mo ago
  It's that but also the narcissistic injury caused by seeing an LLM practice the craft you have spent your life trying to perfect.
  - deleted 6mo ago
    [deleted]
strange_quark 6mo ago
I don't get why it's so hard for you and others in this comment section to understand why people hate AI so much because it's not just the theft and environmental destruction. A college professor, especially one at a liberal arts school, is obviously not going to like something that enables you to outsource your thinking and steals your agency. I think that's a perfectly valid viewpoint; maybe talk to someone without STEM-brain who lives outside of SF for once.
- simonw 6mo ago
  I've recently been amplifying this excellent piece about that by Nilay Patel https://www.theverge.com/podcast/917029/software-brain-ai-backlash-databases-automation https://www.theverge.com/podcast/917029/software-brain-ai-ba... I don't need computer science professors to like LLMs, but I still want them to be able to poke at them with a stick without feeling like they are violating their principles regarding energy usage and unlicensed training data.
  - strange_quark 6mo ago
    > I don't need computer science professors to like LLMs, but I still want them to be able to poke at them with a stick without feeling like they are violating their principles regarding energy usage and unlicensed training data. Why? Language models are interesting from a technical perspective, but so are tons of areas of CS. There's nothing inherently virtuous about using an LLM.
    - simonw 6mo ago
      I think LLMs are the most fascinating new piece of computer science to come along in at least the past decade. The academic field of computer science pretty much started as an exploration into whether machines could be built that could understand human language. The Turing test dates back to Turing!
      - strange_quark 6mo ago
        > I think LLMs are the most fascinating new piece of computer science to come along in at least the past decade. Agree to disagree. > The academic field of computer science pretty much started as an exploration into whether machines could be built that could understand human language. No? CS started as an offshoot of applied mathematics and physics. The study of formal logic, algorithms, digital circuits, etc. predates Turing by centuries. Hell, even the Turing machine predates the Turing test by a couple decades.

by simonw 6mo ago

infotainment 6mo ago

> Andrej Karpathy can train a GPT-2 class model for less than $80 now, so at least the environmental cost of training may drop to a point that it's acceptable to LLM vegetarians: https://twitter.com/karpathy/status/2017703360393318587 https://twitter.com/karpathy/status/2017703360393318587 I suspect that even if you reduced the cost of training or any other real world metric, the goalposts would immediately move. It seems to me that it has never been about those things, but simply about the feeling of superiority one can attain by eschewing something seen as trending.

WatchDog 6mo ago
It's that but also the narcissistic injury caused by seeing an LLM practice the craft you have spent your life trying to perfect.
- deleted 6mo ago
  [deleted]

strange_quark 6mo ago

I don't get why it's so hard for you and others in this comment section to understand why people hate AI so much because it's not just the theft and environmental destruction. A college professor, especially one at a liberal arts school, is obviously not going to like something that enables you to outsource your thinking and steals your agency. I think that's a perfectly valid viewpoint; maybe talk to someone without STEM-brain who lives outside of SF for once.

simonw 6mo ago
I've recently been amplifying this excellent piece about that by Nilay Patel https://www.theverge.com/podcast/917029/software-brain-ai-backlash-databases-automation https://www.theverge.com/podcast/917029/software-brain-ai-ba... I don't need computer science professors to like LLMs, but I still want them to be able to poke at them with a stick without feeling like they are violating their principles regarding energy usage and unlicensed training data.
- strange_quark 6mo ago
  > I don't need computer science professors to like LLMs, but I still want them to be able to poke at them with a stick without feeling like they are violating their principles regarding energy usage and unlicensed training data. Why? Language models are interesting from a technical perspective, but so are tons of areas of CS. There's nothing inherently virtuous about using an LLM.
  - simonw 6mo ago
    I think LLMs are the most fascinating new piece of computer science to come along in at least the past decade. The academic field of computer science pretty much started as an exploration into whether machines could be built that could understand human language. The Turing test dates back to Turing!
    - strange_quark 6mo ago
      > I think LLMs are the most fascinating new piece of computer science to come along in at least the past decade. Agree to disagree. > The academic field of computer science pretty much started as an exploration into whether machines could be built that could understand human language. No? CS started as an offshoot of applied mathematics and physics. The study of formal logic, algorithms, digital circuits, etc. predates Turing by centuries. Hell, even the Turing machine predates the Turing test by a couple decades.