4 ms·

It should do what language does... Language is a phenomenon in, at least, one type of animal. It allows animals to coordinate with each other in a shared envir

It should do what language does... Language is a phenomenon in, at least, one type of animal. It allows animals to coordinate with each other in a shared environment; it describes their internal and external states; etc. etc. Language is a real phenomenon in the world that, like gravity, can be studied. It isnt abstract. NLP models of language arent models of language. Theyre cheap imitations which succeed only to fool language users in local highly specific situations.

nl 5y ago
> NLP models of language arent models of language. Do you actually know what a NLP Language Model refers to? It literally is a model of the language - it predicts the likelihood of the next word(s) given a set of prior word(s). It seems you think people just throw some data at a neural network and then go wow. It's not like that at all - the field of NLP grew out of linguistics study and has deep roots in that field.
- mjburgess 5y ago
  That's not a model of language. Language is a communicative activity between language users, who do things with words, with each other. What you're talking about is ignoring the entire empirical context of langauge, as a real-world phenomenon, and modelling is purely formal characteristics as recorded post-facto. This will always just produce a system which cannot use langauge, but will only ever appear to within highly constrained -- essentially illusory -- contexts. Its the difference between a system which makes a film by "predicting the next frame", and a making a film by recording actual events that you are directing. A prediction of a "next frame" is always therefore just going to be a symptom of the frames before it. When I point a camera at something new, eg., an automobile in c. 1900 -- i will record a film that has never been recorded before. And likewise, with words: we are always in genuinely unquie unprecedented situations. And what we *do with words*, is speak about those situations *to others* who are in them with us... we aim to coordinate, move, and so on with words. To model *language* isnt to model words, nor text, nor to predict words or text. It is to be a speaker here in the world with us, using language to do *what language does*. No model of the regularities of text will ever produce a language-user. Language isnt a regularity, like the frames of a film -- its a suit of capacities which are responsive to the world, and enable language users to navigate it.
  - nl 5y ago
    Until you can make quantifiable predictions of behaviour that you want to see it sounds like your objections are philosophical rather than scientific. > A prediction of a "next frame" is always therefore just going to be a symptom of the frames before it. But the physical appearance of the automobile itself was absolutely influenced by what went before - they were called "horseless carriages" after the appearance after all. And NLP Language Models can produce genuinely original and unique writing. This is a poem a large LM wrote for me: The sceptered isle Hath felt the breath of Britain, Longer than she cares to remember. Now are her champion arms outstared, Her virgin bosom stained with battle's gore. Lords and nobles, courtiers and commons, All stand abashed; the multitudinous rout Scatter their fears in every direction; Except their courage, which, to be perfect, Must be all directed to the imminent danger Which but now struck like a comet; and they feel The blow is imminent > we aim to coordinate, move, and so on with words. https://say-can.github.io/ https://say-can.github.io/ "Robots ground large language models in reality by acting as their eyes and hands while LLMs help robots execute long, abstract language instructions"
    - mjburgess 5y ago
      Chopping up sequences of film, and stiching them together, based on their prior similarity isn't making a movie -- and that's all you have here. People wrote poetry -- *for the first time* -- to say something about their own environment, that they are present in. All you have here is a system which has remembered a compressed representation of these poems and stiches them together to fool you. It really is a kind of proto-psychosis to think this machine has written a poem. It has generated the text of a poem. > quantifiable predictions of behaviour that you want to see This is trivial. I ask the machine a large number of ordinary questions, eg., "what do you think about what i'm wearing?", "what would it take to change your mind on whether murder is justified?", "do you think you'd like new york?", "could you pass me the salt?", etc. -- a trivial infinity of questions lifted from the daily life of language users. The machine cannot answer any of those questions. All it will do is generate some text on the occasion that the machine sees that text. This isn't an answer. That isnt the question. The question isnt "summarise a million documents and report an on-average plausible answer to these questions". When I ask a person any of those questions, if they did that, they wouldnt be answering them. This is trivial to observe. These systems are just taking modes() of subsets of historical data. That's just what they are. The appearence of their using language is an illusion To use language is to have something to say, to wish to talk about something. When i say, "I liked the movie!" I am not summarising a million reviews and finding an average sentence. I am thinking about my experience of the movie, and generating a public sharable "text" that aims to communicate what i actually think. *THAT* is language. Language is your intention to speak *ABOUT* something, and the capacity to generate a public shared set of words which communicate what you are talking about. Any process which begins *without anything to say* cannot ever reach langauge as a capacity. Langauge, as a capacity, begins by being in the world. No summary of the public statmenets of past speakers has anything to do with being in the world; and having things to say. Chopping that up and stiching it together is a trick. And this is trivial to show empirically. It is only by having absolutely no study of langauge use can anyone claim that text documents have anything ot do with it. IT's mumbohjumbo.

nl 5y ago

> NLP models of language arent models of language. Do you actually know what a NLP Language Model refers to? It literally is a model of the language - it predicts the likelihood of the next word(s) given a set of prior word(s). It seems you think people just throw some data at a neural network and then go wow. It's not like that at all - the field of NLP grew out of linguistics study and has deep roots in that field.

mjburgess 5y ago
That's not a model of language. Language is a communicative activity between language users, who do things with words, with each other. What you're talking about is ignoring the entire empirical context of langauge, as a real-world phenomenon, and modelling is purely formal characteristics as recorded post-facto. This will always just produce a system which cannot use langauge, but will only ever appear to within highly constrained -- essentially illusory -- contexts. Its the difference between a system which makes a film by "predicting the next frame", and a making a film by recording actual events that you are directing. A prediction of a "next frame" is always therefore just going to be a symptom of the frames before it. When I point a camera at something new, eg., an automobile in c. 1900 -- i will record a film that has never been recorded before. And likewise, with words: we are always in genuinely unquie unprecedented situations. And what we *do with words*, is speak about those situations *to others* who are in them with us... we aim to coordinate, move, and so on with words. To model *language* isnt to model words, nor text, nor to predict words or text. It is to be a speaker here in the world with us, using language to do *what language does*. No model of the regularities of text will ever produce a language-user. Language isnt a regularity, like the frames of a film -- its a suit of capacities which are responsive to the world, and enable language users to navigate it.
- nl 5y ago
  Until you can make quantifiable predictions of behaviour that you want to see it sounds like your objections are philosophical rather than scientific. > A prediction of a "next frame" is always therefore just going to be a symptom of the frames before it. But the physical appearance of the automobile itself was absolutely influenced by what went before - they were called "horseless carriages" after the appearance after all. And NLP Language Models can produce genuinely original and unique writing. This is a poem a large LM wrote for me: The sceptered isle Hath felt the breath of Britain, Longer than she cares to remember. Now are her champion arms outstared, Her virgin bosom stained with battle's gore. Lords and nobles, courtiers and commons, All stand abashed; the multitudinous rout Scatter their fears in every direction; Except their courage, which, to be perfect, Must be all directed to the imminent danger Which but now struck like a comet; and they feel The blow is imminent > we aim to coordinate, move, and so on with words. https://say-can.github.io/ https://say-can.github.io/ "Robots ground large language models in reality by acting as their eyes and hands while LLMs help robots execute long, abstract language instructions"
  - mjburgess 5y ago
    Chopping up sequences of film, and stiching them together, based on their prior similarity isn't making a movie -- and that's all you have here. People wrote poetry -- *for the first time* -- to say something about their own environment, that they are present in. All you have here is a system which has remembered a compressed representation of these poems and stiches them together to fool you. It really is a kind of proto-psychosis to think this machine has written a poem. It has generated the text of a poem. > quantifiable predictions of behaviour that you want to see This is trivial. I ask the machine a large number of ordinary questions, eg., "what do you think about what i'm wearing?", "what would it take to change your mind on whether murder is justified?", "do you think you'd like new york?", "could you pass me the salt?", etc. -- a trivial infinity of questions lifted from the daily life of language users. The machine cannot answer any of those questions. All it will do is generate some text on the occasion that the machine sees that text. This isn't an answer. That isnt the question. The question isnt "summarise a million documents and report an on-average plausible answer to these questions". When I ask a person any of those questions, if they did that, they wouldnt be answering them. This is trivial to observe. These systems are just taking modes() of subsets of historical data. That's just what they are. The appearence of their using language is an illusion To use language is to have something to say, to wish to talk about something. When i say, "I liked the movie!" I am not summarising a million reviews and finding an average sentence. I am thinking about my experience of the movie, and generating a public sharable "text" that aims to communicate what i actually think. *THAT* is language. Language is your intention to speak *ABOUT* something, and the capacity to generate a public shared set of words which communicate what you are talking about. Any process which begins *without anything to say* cannot ever reach langauge as a capacity. Langauge, as a capacity, begins by being in the world. No summary of the public statmenets of past speakers has anything to do with being in the world; and having things to say. Chopping that up and stiching it together is a trick. And this is trivial to show empirically. It is only by having absolutely no study of langauge use can anyone claim that text documents have anything ot do with it. IT's mumbohjumbo.