> There is nothing about: part of speech categories, relative clauses, morphology This is an interesting and deep question. I believe that there are multiple w

> There is nothing about: part of speech categories, relative clauses, morphology This is an interesting and deep question. I believe that there are multiple ways to tell the story of a phenomenon. Using POS categories or verb tenses helps us to organize and learn about the medium. It is important for humans to know what a verb tense - in theory and practice - is, to be able to form not just grammatical sentences, but also ones, that express things clearer. However, there is no rule that says that these concepts must be learned first before you can do anything else. In a sense, I see the (neural) machines taking a step back. Whereas earlier attempts at language modeling took into account a lot of linguistic concepts, maybe even tailored to a specific human language, the more end-to-end approaches ignore these things and get better results. What can this tell you about the concepts? They are certainly useful and have been established by a scientific community. The machine, as it reads not just tens or hundreds of sentences but billions, creates a mathematical model, that happens to have aspects, that have very useful real world sides to them. A machine that learns a language model - or that learns translation, can teach us new things about words, and ultimately also about the world. I believe this is a beautiful aspect of the current neural renaissance. ---- Kanerva in a preface to Geometry and Meaning writes: > We think of some sciences, by their very nature, as being more mathematical than others. We even refer to physics and other heavily mathematical sciences as "hard sciences," as if to imply that social sciences and humanities are somehow soft or easy. Another way to see this picture, however, is that mathematics itself is hard and that it would be applied first to sciences that themselves are relatively easy. [...] It matters greatly that the mathematics be appropriate. [...] Mathematics is a tool for coping with complexity. [...] This brings us to the substance of [this] book: the exploration of mathematics that would be appropriate for describing concepts and meaning. The branch of mathematics that has traditionally been associated with meaning is logic. Logic is also discussed here, yet the main emphasis is on identifying mathematical spaces that would lend to modeling of meaning. (http://www.puttypeg.net/book/chapters/foreword.html http://www.puttypeg.net/book/chapters/foreword.html)