5 ms·

This happens everywhere. Let me tell you a story about who studies language and how. Linguistics studies language right? It has a few hundred year history of d

This happens everywhere. Let me tell you a story about who studies language and how. Linguistics studies language right? It has a few hundred year history of doing so seriously. But once AI people wanted to also have a look linguists became unhappy. They didn't like the idea of someone else saying something about their problem with a different set of tools. Those tools don't provide explanations that they approve of. Even the thought of measuring things over a dataset to check if your linguistic theories work made them bristle with Chomsky famously rejecting the idea of evidence at scale. So AI people created natural language processing. Their own field to study the same problem. One in a while someone tried to do something to bridge the gap which led to the famous "every time I fire a linguist my performance goes up". Many people in NLP see linguists as outdated fossils. Most would probably vote to just shut down linguistics departments I suspect. Then something funny happened. The LLM people did to NLP what they did to linguistics. And the NLP community rejected them. Because LLMs don't provide an explanation that they approve of. Recently just like in linguistics the President of ACL, the main NLP organization, said recently that NLP is not about machine learning! The absurdity of that statement just boggles the mind. So the LLM people now have their own conference and field. A lot of people cannot accept that things must change over time. They want to do research the way they did it 10 or 40 years ago. So they create an immune response. It's human nature. As someone who doesn't feel this way and enjoys moving forward, I'm really sad about what's happened to NLP in the past few years. It would be nice if we could stop this rift. But it won't happen with the current hate filled President. I stopped sending any work there until they reform. Just the cycle of academic life. I wonder if science would move faster if we could select for people who are willing to change with the times.

mjburgess 2y ago
The goal of physics isn't to produce video games -- that a video game can generate a frame using unphysical formulae and unphysical processes (eg., rasterisation) does not invalidate physics. There's linguistics which produces descriptive theories of the practice of language. There's "computational linguistics" which aims to build discrete algorithmic models of language. Both of these are interested in explanations of actual physical phenomena. Machine Learning is largely an engineering discipline which, like video games, can generate output using any method which will convince a human user. Indeed, often more radically, ML works by replaying variations of past convincing output rather than have any explicit simulation involved at all. Thus there's no sense in which LLMs supercede linguistics, nor somehow computational linguistics superseding this. This would be like claiming rasterisation means we should retire optics and the physics of light.
- HPsquared 2y ago
  That's a nice dividing line, science and engineering are different for sure. But science makes use of engineering knowledge and engineering makes use of scientific knowledge.
- light_hue_1 2y ago
  That's like saying that it's not astronomy if I'm using a radio telescope, only optical telescopes are real astronomy. It makes zero sense. The tools are irrelevant. We don't have physics with and without ml journals. It's like saying AlphaFold isn't biology because it uses ml. It's absurd One goal of linguistics is to produce an account of how language works. If you understand how something works you can reproduce it. This was always a core part of linguistics. Just look at Chomsky's work. Or at what counts as evidence in a linguistics paper. It is a model that "explains" some feature of language. NLP does exactly the same thing as linguistics. It produces explanations and makes predictions about language. But it does so with ML. I think you're confused about optics and rasterization. There are countless ways to relate the two. Rasterization is not unphysical, no more so than say, finite element methods are unphysical. Heck physics is in the name of PBR. It's an approximation. And a fine one that people actually use in both science and games.
  - mjburgess 2y ago
    My master's project was applying non-gradient-based ML methods to parameter optimisation in quantum metrology -- I'm aware that the reskin on curve-fitting algorithms called "Machine Learning" may be used as part of science. We've had taylor series approximations to functions for 300 years. But insofar as we're talking about LLMs, and most products you can use, we're talking about an engineering use. Engineers build systems that "perform" as you say explanatory accuracy is not a kind of engineering performance. It would equally well be said that, for a video game studio, "the more physicists I fired, the more the games perform better" -- for, of course, physicists have never studied 'making a video-game image convincing', and their techniques are best run on supercomputers, since they'd build explanatory simulations with explanatory accuracy. Video game developers do not do this. I have developed video games also, so I know. Convincingly placing pixels on a screen as-if governed by the laws of physics does not require actually simulating them. It requires generating frames from a "point of view" that never expose the absense of real physics, or approximations thereof. The world isnt made of triangles, and you cannot clip thru solid objects. The goal of a person making an LLM is similar to the goal of a video game developer, these are engineering goals: to give a user an experience which they will pay for. Now, a 2d 80s video-game has more actual physics in it than an LLM has models of language use, but still, this is irrelevant to the goals of these creators. A linguist could analyse an LLM as the conditional probability structure of the english language, as captured largely recent electronic texts -- but this structure is partly what precisely linguists and others are trying to explain. This is why all curve-fitting to historical data isn't in itself science: it is a restatement of the very target of explanation, the data. The job of scientific fields is to explain that curve. Its explotation by engineers is explanation-free. It isnt a science, and replaces no science.
GPerson 2y ago
There’s no single forward direction in science. We’re in a machine learning hype cycle. That doesn’t mean we must fire everyone in every other department.

mjburgess 2y ago

The goal of physics isn't to produce video games -- that a video game can generate a frame using unphysical formulae and unphysical processes (eg., rasterisation) does not invalidate physics. There's linguistics which produces descriptive theories of the practice of language. There's "computational linguistics" which aims to build discrete algorithmic models of language. Both of these are interested in explanations of actual physical phenomena. Machine Learning is largely an engineering discipline which, like video games, can generate output using any method which will convince a human user. Indeed, often more radically, ML works by replaying variations of past convincing output rather than have any explicit simulation involved at all. Thus there's no sense in which LLMs supercede linguistics, nor somehow computational linguistics superseding this. This would be like claiming rasterisation means we should retire optics and the physics of light.

HPsquared 2y ago
That's a nice dividing line, science and engineering are different for sure. But science makes use of engineering knowledge and engineering makes use of scientific knowledge.
light_hue_1 2y ago
That's like saying that it's not astronomy if I'm using a radio telescope, only optical telescopes are real astronomy. It makes zero sense. The tools are irrelevant. We don't have physics with and without ml journals. It's like saying AlphaFold isn't biology because it uses ml. It's absurd One goal of linguistics is to produce an account of how language works. If you understand how something works you can reproduce it. This was always a core part of linguistics. Just look at Chomsky's work. Or at what counts as evidence in a linguistics paper. It is a model that "explains" some feature of language. NLP does exactly the same thing as linguistics. It produces explanations and makes predictions about language. But it does so with ML. I think you're confused about optics and rasterization. There are countless ways to relate the two. Rasterization is not unphysical, no more so than say, finite element methods are unphysical. Heck physics is in the name of PBR. It's an approximation. And a fine one that people actually use in both science and games.
- mjburgess 2y ago
  My master's project was applying non-gradient-based ML methods to parameter optimisation in quantum metrology -- I'm aware that the reskin on curve-fitting algorithms called "Machine Learning" may be used as part of science. We've had taylor series approximations to functions for 300 years. But insofar as we're talking about LLMs, and most products you can use, we're talking about an engineering use. Engineers build systems that "perform" as you say explanatory accuracy is not a kind of engineering performance. It would equally well be said that, for a video game studio, "the more physicists I fired, the more the games perform better" -- for, of course, physicists have never studied 'making a video-game image convincing', and their techniques are best run on supercomputers, since they'd build explanatory simulations with explanatory accuracy. Video game developers do not do this. I have developed video games also, so I know. Convincingly placing pixels on a screen as-if governed by the laws of physics does not require actually simulating them. It requires generating frames from a "point of view" that never expose the absense of real physics, or approximations thereof. The world isnt made of triangles, and you cannot clip thru solid objects. The goal of a person making an LLM is similar to the goal of a video game developer, these are engineering goals: to give a user an experience which they will pay for. Now, a 2d 80s video-game has more actual physics in it than an LLM has models of language use, but still, this is irrelevant to the goals of these creators. A linguist could analyse an LLM as the conditional probability structure of the english language, as captured largely recent electronic texts -- but this structure is partly what precisely linguists and others are trying to explain. This is why all curve-fitting to historical data isn't in itself science: it is a restatement of the very target of explanation, the data. The job of scientific fields is to explain that curve. Its explotation by engineers is explanation-free. It isnt a science, and replaces no science.

GPerson 2y ago

There’s no single forward direction in science. We’re in a machine learning hype cycle. That doesn’t mean we must fire everyone in every other department.