4 ms·

That’s an excellent observation. I struggled with this too and because I am fond of defective analogies this made me think of the meaning of a personal note to

That’s an excellent observation. I struggled with this too and because I am fond of defective analogies this made me think of the meaning of a personal note to, say, a lover. Does it matter by whom or indeed what it was written? The experience of reading occurs solely in the recipient’s mind. Surely it does not affect him/her differently if said note was produced by a romantic slot machine. Before I meander towards my confused and shaky conclusion, let me clear one thing out of the way: I did not mean to imply that mechanically generated sequences are meaningless. They absolutely carry meaning, because meaning is ultimately created by us. We are meaning-creation machines. We can find meaning in tea-leaves. Surely tea-leaves carry meaning, but sadly not the will-of-the-gods-kind. Although that doesn’t stop us from thinking they do. What I am tentatively suggesting is that the meaning of the lover’s note is as much found in the experience of the relationship they share as it is found in the symbols on the paper. The symbols reference that experience. Because it is ultimately a human that reads and interprets I think failure to take into account the origin of the material will be the source of a subtle and highly impactful, but inexorable error not dissimilar to that of the tasseographer: what you see doesn’t mean what you think it does. Language used to be a domain dominated by humans only. We are wired for it and biased by it. If it sounds intelligent, it is intelligent, right? This bias could make us defer to the machine quicker than we should. What’s the difference between GPT4 generating a good piece of code and a human? Maybe I should shut up now. I am open to interesting reading suggestions by the way.

namaria 3y ago
The difference is in the temporal dimension of the message. Generated text and code are simply statistically likely to convey enough meaning for the purpose at hand. That's all good and dandy until you think about the repercussions of communication. A person facing a problem and finding a solution encodes a message. This message ideally conveys context and purpose. The situation changes over time and we need to adapt our systems and behaviors. So we can go back to the message and re-work it. If there is no inherent message, if it is just a byte array likely to get a desired result in a specific context, you are losing traction of mind over reality and letting an awful lot of noise into our delicate systems. Isn't letting accidental complexity aka noise into our systems bad? If you don't know the purpose and the context of the message, you already let that happen, and it will only get worse as the subsequent modifications to the systems described by the code or behaviors enacted due to the message starts to become an opaque box due to the fact that there is no meaningful blueprint behind the code or the texts underlying it.
- SanderNL 3y ago
  Yes, the repercussions of this vacuousness are what interest me most. The underlying “systems” and “behaviors” that gave birth to the message are prior to any language model, but there, pre-language, is exactly where all changes happen. These changes then percolate to the top, so to speak, as language. I believe you captured this beautifully. Then you get a bunch of humans that interpret this soulless message, but fail to comprehend the inherent emptiness sufficiently. I’d say that’s not good, but I have no solution either.
  - namaria 3y ago
    I see very little difference between letting LLMs create language that people and computer systems are expected to act upon and using oracles/'revealed' wisdom/sacred texts which dictate what we ought to want, decide and do. Letting inscrutable systems write the rules is just bad. We have 300 years of humanism building a base for our shared reasoning skills to enlighten the way and giving that up for the convenience of letting noise and chance guide us seems like going backwards.

namaria 3y ago

The difference is in the temporal dimension of the message. Generated text and code are simply statistically likely to convey enough meaning for the purpose at hand. That's all good and dandy until you think about the repercussions of communication. A person facing a problem and finding a solution encodes a message. This message ideally conveys context and purpose. The situation changes over time and we need to adapt our systems and behaviors. So we can go back to the message and re-work it. If there is no inherent message, if it is just a byte array likely to get a desired result in a specific context, you are losing traction of mind over reality and letting an awful lot of noise into our delicate systems. Isn't letting accidental complexity aka noise into our systems bad? If you don't know the purpose and the context of the message, you already let that happen, and it will only get worse as the subsequent modifications to the systems described by the code or behaviors enacted due to the message starts to become an opaque box due to the fact that there is no meaningful blueprint behind the code or the texts underlying it.

SanderNL 3y ago
Yes, the repercussions of this vacuousness are what interest me most. The underlying “systems” and “behaviors” that gave birth to the message are prior to any language model, but there, pre-language, is exactly where all changes happen. These changes then percolate to the top, so to speak, as language. I believe you captured this beautifully. Then you get a bunch of humans that interpret this soulless message, but fail to comprehend the inherent emptiness sufficiently. I’d say that’s not good, but I have no solution either.
- namaria 3y ago
  I see very little difference between letting LLMs create language that people and computer systems are expected to act upon and using oracles/'revealed' wisdom/sacred texts which dictate what we ought to want, decide and do. Letting inscrutable systems write the rules is just bad. We have 300 years of humanism building a base for our shared reasoning skills to enlighten the way and giving that up for the convenience of letting noise and chance guide us seems like going backwards.