2 ms·
How is it that LLMs launder consciousness but don't launder ownership? The author's main point seems to be that bits are colored[1], and that the process of LL
by kelseyfrog 3y ago
How is it that LLMs launder consciousness but don't launder ownership?
The author's main point seems to be that bits are colored[1], and that the process of LLM training doesn't "bleach models". In other words, ownership persists through training in a way that generated text mixes ownership rather than creating a new, ownable, artifact.
They also seem to have (extrapolating the theme) a concept of "smelly" bits. "They can also communicate with a person in a way that resembles actual conversation." ie: words have a "conscious"[smelly] category imbued upon them if they were generated by human or "unconscious"[unscented] if computer generated them.
This evokes a sense that "while they use words and form sentences like conscious people, they can't hold a conversation." Furthermore, this seems to be stated as an ontological fact, rather than say a derived and grounded in behavior such as, "They aren't currently capable of holding conversation because the output distribution doesn't match output distributions of human conversations."
These two views, that models don't bleach but they preserve smell, are fundamentally incompatible and diametrically opposed. How is it that color[ownership] persists through model training, but smell[consciousness] does not? We don't know - the author doesn't say. Jim just states both as fact without grounding either in rationale or justification.
Sure, we can take them as axiomatic, but then what's the point of the article then? There's a reoccurring disappointment in property/consciousness articles from tech-types who write on ownership and consciousness really aren't adding anything creative to the conversation. This is an extremely basic critique of the piece that the author should have thought through before hitting the publish button.
1. https://ansuz.sooke.bc.ca/entry/23 https://ansuz.sooke.bc.ca/entry/23