4 ms·

The problem with research conducted in this way is that there's no theory of operation of the model, except the implicit one "it works like how I expect". And

The problem with research conducted in this way is that there's no theory of operation of the model, except the implicit one "it works like how I expect". And then there's no hypothesis test against predictions made based on this theory, ie., no one tries to disconfirm the model works that way. It is usually trivial to disconfirm. ML models almost always work some other way, usually exploiting coincidental statistical information in the training set. This turns the whole show into a superstitious activity, in which we think google answers our "local restaurants" query because it knows where I live (rather than say, using an IP location or one of 100 other methods). If you present a ML model as "capable of x", you're required to disprove that claim as best you can, and present the occasions where it failed.

xyzzy123 4y ago
It might not be rigorous science but it's a cracker of a blog post. It shows a surprising ability of language models, code is provided and plenty of mystery is left for the reader... which can be investigated by almost anyone for tens of dollars.
- mjburgess 4y ago
  But is it an ability? I can't tell. What would a useless "random" algorithm find operating on these graphs as inputs? What would a useful one find? What would that be "useful" for? Is this system much closer to the useless or the useful? We cant tell. Without any explanation, we're left reading this blog post as a magic trick in the literal sense. Consider the rate at which it "uses edges that don't exist" -- this makes it not a graph traversal algorithm. So it is just exploting syntax in the input whose representation has some "graphlike" qualities which can be statistically exploited. Why is that interesting? I dont know. Is this a good system?
  - xyzzy123 4y ago
    I don't know! That's why it's fun. (EDIT: I can safely say this isn't practically useful... but it's interesting in the same way as watching a cat try to fill out a tax return. I'm not surprised the cat does it badly, I'm surprised that it seems to be doing it at all, even if it's really a trick).
YorkshireSeason 4y ago
When Newton described the laws of gravity, now often referred to as Newtonian physics, did he explain what gravity was, in other words, did he have a "theory of operation of the model, except the implicit one"? No, au contraire! He gave laws that took gravity as a black box, but worked very well allowing us to predict things "like how I expect". It has been argued that giving up on explaining what gravity really is, and instead focussing on being able to predict its workings, has been one of Newton's great insights. One might argue that we still don't understand what gravity and the remaining fundamental forces (electromagnetic, strong and weak forces) really are, we still treat them as a black box, just like modern ML research mostly treat transformers (and similar) as a black box. That's normal science. Indeed, what else can you do? The fact that "it works like how I expect" and that there is a clear trajectory of working more and more "like how I expect" is a strong indicator of scientific progress.
- mjburgess 4y ago
  The semantics for the mathematical model /are/ the theory of how it works. M = mass, F = force etc. st., the model claims we are in a world of masses and forces, and so on. And this model can be disconfirmed. You're confusing two issues here. Nowhere in the whole of science are models presented without an explanatory semantics. Almost nowhere in the whole of ML, are they. Here it is trivial to disconfirm even the authors interpretation of the NNs 'finding' of solutions, by noting it is often using edges that don't exist. This whatever it is doing, we can immediately show it isn't traversing a graph. The author seems to wilfully opted for the superstitious interpretation
visarga 4y ago
> ML models almost always work some other way , usually exploiting coincidental statistical information in the training set. That's true for models trained on small datasets, but GPT-3's dataset was anything but superficial, and on top of it we consider this task to be OOD, thus the amazement.
- Der_Einzige 4y ago
  Sorry, you and the posters who think that this task wasn't in the training data are wrong. I'm pretty sure nearly everything, including textual representations of graph problems, are found in the training data. "The internet is vast and infinite" - Motoko Kusanagi at the end of the original ghost in the shell...

xyzzy123 4y ago

It might not be rigorous science but it's a cracker of a blog post. It shows a surprising ability of language models, code is provided and plenty of mystery is left for the reader... which can be investigated by almost anyone for tens of dollars.

mjburgess 4y ago
But is it an ability? I can't tell. What would a useless "random" algorithm find operating on these graphs as inputs? What would a useful one find? What would that be "useful" for? Is this system much closer to the useless or the useful? We cant tell. Without any explanation, we're left reading this blog post as a magic trick in the literal sense. Consider the rate at which it "uses edges that don't exist" -- this makes it not a graph traversal algorithm. So it is just exploting syntax in the input whose representation has some "graphlike" qualities which can be statistically exploited. Why is that interesting? I dont know. Is this a good system?
- xyzzy123 4y ago
  I don't know! That's why it's fun. (EDIT: I can safely say this isn't practically useful... but it's interesting in the same way as watching a cat try to fill out a tax return. I'm not surprised the cat does it badly, I'm surprised that it seems to be doing it at all, even if it's really a trick).

YorkshireSeason 4y ago

When Newton described the laws of gravity, now often referred to as Newtonian physics, did he explain what gravity was, in other words, did he have a "theory of operation of the model, except the implicit one"? No, au contraire! He gave laws that took gravity as a black box, but worked very well allowing us to predict things "like how I expect". It has been argued that giving up on explaining what gravity really is, and instead focussing on being able to predict its workings, has been one of Newton's great insights. One might argue that we still don't understand what gravity and the remaining fundamental forces (electromagnetic, strong and weak forces) really are, we still treat them as a black box, just like modern ML research mostly treat transformers (and similar) as a black box. That's normal science. Indeed, what else can you do? The fact that "it works like how I expect" and that there is a clear trajectory of working more and more "like how I expect" is a strong indicator of scientific progress.

mjburgess 4y ago
The semantics for the mathematical model /are/ the theory of how it works. M = mass, F = force etc. st., the model claims we are in a world of masses and forces, and so on. And this model can be disconfirmed. You're confusing two issues here. Nowhere in the whole of science are models presented without an explanatory semantics. Almost nowhere in the whole of ML, are they. Here it is trivial to disconfirm even the authors interpretation of the NNs 'finding' of solutions, by noting it is often using edges that don't exist. This whatever it is doing, we can immediately show it isn't traversing a graph. The author seems to wilfully opted for the superstitious interpretation

visarga 4y ago

> ML models almost always work some other way , usually exploiting coincidental statistical information in the training set. That's true for models trained on small datasets, but GPT-3's dataset was anything but superficial, and on top of it we consider this task to be OOD, thus the amazement.

Der_Einzige 4y ago
Sorry, you and the posters who think that this task wasn't in the training data are wrong. I'm pretty sure nearly everything, including textual representations of graph problems, are found in the training data. "The internet is vast and infinite" - Motoko Kusanagi at the end of the original ghost in the shell...