7 ms·

DeepMind’s AI helps untangle the mathematics of knots

136 points by bryan0 5y ago

vanusa 5y ago
One technique in particular, called saliency maps, turned out to be especially helpful. It is often used in computer vision to identify which parts of an image carry the most-relevant information. Saliency maps pointed to knot properties that were likely to be linked to each other, and generated a formula that seemed to be correct in all cases that could be tested. Lackenby and Juhász then provided a rigorous proof that the formula applied to a very large class of knots2. Kewl, and not to discount the result but sounds like ... pattern matching, right? So why do the editors of all these journals keep feeding off the ML == AI meme? Surely they know better by now.
- gmadsen 5y ago
  sounds more like pattern finding. and if you break down human cognition, it isn't much different..
  - vanusa 5y ago
    Yep - that's the whole point, the only point. And again, I thought it was pretty simple and generally known.
- Jensson 5y ago
  Because pattern matching is the only significant new thing in AI research the past few decades. Every time they apply pattern matching to something new we will get articles like this and AI optimists will come and say that soon AI will take our jobs. Pattern matched images, pattern matched texts, reverse pattern matching to generate uncanny images and articles, pattern matched gameplay states etc.
  - version_five 5y ago
    Deep learning is pattern finding, not pattern matching. That is the difference. There is a business narrative that's hyping up ML/AI, there's lots of retrospective tracing of modern ML's roots back to something older, but that doesn't change the fact that there has been a big explosion of demonstrated advances opened up by modern approaches over the past 10 years. Edit: just to expand a bit, the "intelligence" in modern AI is in the training, not inference. I think people see inference in a neural network as pattern matching or some kind of filter, and that's basically true, and "dumb" if you like. But learning the weights is a whole different thing. Stochastic Gradient Descent et al are using often only a comparative few examples to learn a pattern and embed it in a set of weights in a way that can generalize despite being highly underdetermined. It's not general intelligence, but it's a much different thing than the casual dismissals people like to post, usually directed at the inference part as if the weights just magically appeared
    - vanusa 5y ago
      Deep learning is pattern finding, not pattern matching. "Pattern matching" obviously includes "pattern finding" already, in common parlance. By "deep learning" one means, in effect, "hierarchical pattern finding" (which is basically what "feature" or "representation" learning means to a lay person). But still, at the end of the day ... pattern finding. Or that is to say: "just ML", not AI.
      - rackjack 5y ago
        I'm pretty sure they meant "pattern finding" as "discovering new patterns that can be used for matching", not "finding a pattern that matches an existing known one".
        Jensson 5y ago
        But currently humans do that "pattern finding". If you want it to learn to recognize animals you give it thousands of images of different animals in all sorts of poses, angles and environments and tells it what those animals are, basically the "pattern" is created by the humans who compiles the dataset with enough examples to catch every situation, and then the program uses that pattern to find matches in other images. However if you want a human to recognize animals it is enough to show this picture and then they will be able to recognize most of these animals in other photos and poses, humans creates the pattern from the image and don't just rely on having tons of data: https://i.pinimg.com/originals/35/78/47/35784708f8cc9ef2345cf0670816a317.png https://i.pinimg.com/originals/35/78/47/35784708f8cc9ef2345c... Edit: In some cases you can write a program to explore a space. For example, you can write a program that plays board games randomly and notes the outcome, that is a data generator. Then you hook that to a pattern recognizer powered by a data centre, and you now have a state of the art gameplay AI. It isn't that simple, since writing the driver and feedback loop is extremely hard work that an AI can't do, humans has to do it.
- atty 5y ago
  I get that when you boil it down to pattern matching it sounds less impressive, but we are starting to see superhuman pattern matching algorithms, and algorithms that display logic through advanced “pattern matching”. And it’s quite obvious now that pattern matching is far more useful than pure rule based systems for any problem of sufficient complexity. And AI is a very broad field, including pattern matching (supervised ML or otherwise), logic/rule based systems, etc. Is your complaint that they use AI/ML interchangeably? Because in this case, while AI is a more term for the technology than ML, it is not a strictly incorrect title.
  - vanusa 5y ago
    Is your complaint that they use AI/ML interchangeably? Yes, that's all I'm saying. And it seems like such a known point that they're not the same and not interchangeable, or so I thought (that I'm kind of astonished at the downvotes at my original comment). Because in this case, while AI is a more term for the technology than ML, it is not a strictly incorrect title. I get what you're saying at the product level -- and the fact that the vast bulk of the public subjected to these technologies couldn't tell you the difference, nor could they begin to care. But to practitioners, the basic facts remain: ML ≠ AI, it's a proper subset, and we're doing the public a genuine disservice (and arguably causing substantial harm) by pretending to tell them that we're making good progress developing AI as distinct from hypercharged ML, that any day now they'll have self driving cars ... and all that crap the industry has basically been telling people.
- TulliusCicero 5y ago
  > So why do the editors of all these journals keep feeding off the ML == AI meme? Your comment is a good example of AI meaning whatever we haven't solved yet. Playing chess well is AI until we have programs that can regularly defeat professionals, then it suddenly doesn't count. Accurately labeling the contents of a photo is AI until we have services that do this for anyone, and then it's not AI anymore. Pretty soon self-driving cars will also not count as AI, because we'll have solved self-driving and therefore it's not hard enough anymore.
  - vanusa 5y ago
    Your comment is a good example of AI meaning whatever we haven't solved yet. Yes, I'm aware of the Rodney Brooks quote. I think it's kind of glib and misses the mark. Take chess, for example. When algorithms were developed to beat humans at chess -- it wasn't, like the quote implies, out of some sense of envy or schadenfreude that people said, "Oh, but that's not real AI". But rather -- from the intrinsic features of the system that first met this challenge. Which, at the end of the day, were basically about generalized tree search. Okay, there was more to it than that -- but in 3 word or less, that's gist of the AlphaZero algorithm, as described in the original paper. So people said "Oh -- it turns out you don't need human-like cognition to beat humans at chess/go/shogi. You just need good enough tree search, and some decent hardware." Pretty soon self-driving cars will also not count as AI, because we'll have solved self-driving and therefore it's not hard enough anymore. That's one of the weird things about the current moment in AI: people keep talking about self-driving cars as if they're a straight-line extrapolation the breakthroughs we've seen so far (give or take a few details). They're not. If you just think about if for a minute: nearly all of the challenges met thus far -- taking the two you just mentioned as particularly shining examples -- as significant as they have been, have been pretty darn one-dimensional. "Here's a function; optimize it" (basically). The reason self-driving cars are floundering at the moment is that the intrinsic problem they're dealing with is an entirely different beast from "solve this symbolic game faster than a human" or "label these photos better than a roomful of monkees". It is vastly more complex. And the emerging consensus (whatever some companies may say to the contrary) is that solution is not lurking just around the corner, and not coming our way anytime soon -- certainly not "pretty soon".
aerodog 5y ago
surprised Erik Demaine wasn't part of this
deleted 5y ago
[deleted]
agnosticmantis 5y ago
See [0] for a critical review of (the popularization of) this paper. As usual, there’s something real here but it’s way overhyped because everyone involved benefits from over-playing the role of deep learning here. [0:https://arxiv.org/abs/2112.04324 https://arxiv.org/abs/2112.04324] Edit: the critical review isn’t about the technical content of the paper per se, but rather the hyped news around it.
- version_five 5y ago
  There seems to be a lot of animosity towards deep learning, to the point that it comes off as jealousy - I mean read the other long thread in this discussion. I don't mean to say the criticisms in the article you linked are not valid, just that there are a lot of people ready to jump on any failure or exaggeration related to ML, giving it far more scrutiny than most research disciplines. It gets more money than most as well so maybe that's ok. And I know it gets overhyped in business contexts and media. But there are research preprints. That said, my observations about this story was that one one hand, everyone talks about how AI/ML will change everything and has lots of applications. But this article strikes my as clearly as a story of a solution in search of a problem, that had to go deep into a branch of theoretical math to find a place where ML could make a demonstrable impact. I know this application is not the only thing ML is good at (for context I own a machine learning consulting company and am bullish on the business applications of ML). But when you look at deepmind and the money that's gone into it, between the often cited "data centre cooling optimization" and this, they are not exactly having a massive impact, even as they do a lot of cool stuff.
  - Jensson 5y ago
    > There seems to be a lot of animosity towards deep learning, to the point that it comes off as jealousy This happens every time people post research here. Any article about new battery tech will have people saying "What about all the other new battery techs that never went anywhere? This is just a prototype, it wont revolutionize the industry", and they are right. The difference is that when people say those things in a thread about ML, then AI optimists who thinks that these things are evidence that the singularity is near starts to argue that no, this time is truly different! And they do that every single time, AI optimists have believed that computers will take over any day now for 50 years. If those AI optimists didn't post so much then those threads wouldn't grow.
dukeofdoom 5y ago
Is there a reason why things get tangled so easily. Like it seems like its the way of the universe. I recently tried fly fishing. I spend 1/3 of the time trying to untangle my fishing line.
- Jensson 5y ago
  Probably entropy. If you randomize fishing line positions almost all of them would be tangled. Now putting the fishing line in a box isn't the same thing as randomizing it, but the point of entropy is that small random changes will move the state towards higher entropy, meaning that the random changes ought to move the line towards being tangled. Of course this assumes that a tangled line has higher entropy for some definition of entropy. Proving/structuring that would be too much work though, but if solved could make for a nice paper. Hmm, since it would be such a nice paper I looked it up, and apparently people have already studied these things: https://www.npr.org/sections/bryantpark/2008/01/knot_theory_why_everything_tan.html https://www.npr.org/sections/bryantpark/2008/01/knot_theory_...
  - dukeofdoom 5y ago
    Thanks for the great reply! Fascinating
- bamboozled 5y ago
  Almost never happens to me, but it did when learning. So I think the reason it’s happening is you. All due respect. Edit: I’ll try be more constructive. There’s a good chance you’re looping your fly back over the line during your cast crating knots and tangles. I do agree stuff tangles easily but I don’t find it to be the case with fly lines in general.
  - dukeofdoom 5y ago
    I'm definitely doing that. I'm using an amazon rod and reel combo deal by maxcatch, and it came with a very fine fishing line at the tip of the leader. So the wind picks it up, and loops it around the tip of the rod sometimes. But I also had snags inside the reel. And snags and tangles around bushes, and tree limbs.
oAlbe 5y ago
There's a pretty good Numberphile Podcast episode[0] about this. It doesn't go deep into mathematical details of the topic, it's more of an overview of the current state of things and an explanation of ways in which ML can help coming up with new theories/conjectures. [0] https://www.numberphile.com/podcast/deep-mind https://www.numberphile.com/podcast/deep-mind
sarosh 5y ago
The pre-print referred to in this article itself is here: https://arxiv.org/abs/2111.15323 https://arxiv.org/abs/2111.15323
zmachinaz 5y ago
There has been work of similar spirit going on since at least 2017 in mathematical/theoretical physics. With similar limited importance. But the ideas are floating around already for a long time. But if DeepMind gives it a go they make a big fuzz about it, and of course they even do not need to cite previous works ...

136 points by bryan0 5y ago

One technique in particular, called saliency maps, turned out to be especially helpful. It is often used in computer vision to identify which parts of an image carry the most-relevant information. Saliency maps pointed to knot properties that were likely to be linked to each other, and generated a formula that seemed to be correct in all cases that could be tested. Lackenby and Juhász then provided a rigorous proof that the formula applied to a very large class of knots2. Kewl, and not to discount the result but sounds like ... pattern matching, right? So why do the editors of all these journals keep feeding off the ML == AI meme? Surely they know better by now.

gmadsen 5y ago
sounds more like pattern finding. and if you break down human cognition, it isn't much different..
- vanusa 5y ago
  Yep - that's the whole point, the only point. And again, I thought it was pretty simple and generally known.
Jensson 5y ago
Because pattern matching is the only significant new thing in AI research the past few decades. Every time they apply pattern matching to something new we will get articles like this and AI optimists will come and say that soon AI will take our jobs. Pattern matched images, pattern matched texts, reverse pattern matching to generate uncanny images and articles, pattern matched gameplay states etc.
- version_five 5y ago
  Deep learning is pattern finding, not pattern matching. That is the difference. There is a business narrative that's hyping up ML/AI, there's lots of retrospective tracing of modern ML's roots back to something older, but that doesn't change the fact that there has been a big explosion of demonstrated advances opened up by modern approaches over the past 10 years. Edit: just to expand a bit, the "intelligence" in modern AI is in the training, not inference. I think people see inference in a neural network as pattern matching or some kind of filter, and that's basically true, and "dumb" if you like. But learning the weights is a whole different thing. Stochastic Gradient Descent et al are using often only a comparative few examples to learn a pattern and embed it in a set of weights in a way that can generalize despite being highly underdetermined. It's not general intelligence, but it's a much different thing than the casual dismissals people like to post, usually directed at the inference part as if the weights just magically appeared
  - vanusa 5y ago
    Deep learning is pattern finding, not pattern matching. "Pattern matching" obviously includes "pattern finding" already, in common parlance. By "deep learning" one means, in effect, "hierarchical pattern finding" (which is basically what "feature" or "representation" learning means to a lay person). But still, at the end of the day ... pattern finding. Or that is to say: "just ML", not AI.
    - rackjack 5y ago
      I'm pretty sure they meant "pattern finding" as "discovering new patterns that can be used for matching", not "finding a pattern that matches an existing known one".
      - Jensson 5y ago
        But currently humans do that "pattern finding". If you want it to learn to recognize animals you give it thousands of images of different animals in all sorts of poses, angles and environments and tells it what those animals are, basically the "pattern" is created by the humans who compiles the dataset with enough examples to catch every situation, and then the program uses that pattern to find matches in other images. However if you want a human to recognize animals it is enough to show this picture and then they will be able to recognize most of these animals in other photos and poses, humans creates the pattern from the image and don't just rely on having tons of data: https://i.pinimg.com/originals/35/78/47/35784708f8cc9ef2345cf0670816a317.png https://i.pinimg.com/originals/35/78/47/35784708f8cc9ef2345c... Edit: In some cases you can write a program to explore a space. For example, you can write a program that plays board games randomly and notes the outcome, that is a data generator. Then you hook that to a pattern recognizer powered by a data centre, and you now have a state of the art gameplay AI. It isn't that simple, since writing the driver and feedback loop is extremely hard work that an AI can't do, humans has to do it.
atty 5y ago
I get that when you boil it down to pattern matching it sounds less impressive, but we are starting to see superhuman pattern matching algorithms, and algorithms that display logic through advanced “pattern matching”. And it’s quite obvious now that pattern matching is far more useful than pure rule based systems for any problem of sufficient complexity. And AI is a very broad field, including pattern matching (supervised ML or otherwise), logic/rule based systems, etc. Is your complaint that they use AI/ML interchangeably? Because in this case, while AI is a more term for the technology than ML, it is not a strictly incorrect title.
- vanusa 5y ago
  Is your complaint that they use AI/ML interchangeably? Yes, that's all I'm saying. And it seems like such a known point that they're not the same and not interchangeable, or so I thought (that I'm kind of astonished at the downvotes at my original comment). Because in this case, while AI is a more term for the technology than ML, it is not a strictly incorrect title. I get what you're saying at the product level -- and the fact that the vast bulk of the public subjected to these technologies couldn't tell you the difference, nor could they begin to care. But to practitioners, the basic facts remain: ML ≠ AI, it's a proper subset, and we're doing the public a genuine disservice (and arguably causing substantial harm) by pretending to tell them that we're making good progress developing AI as distinct from hypercharged ML, that any day now they'll have self driving cars ... and all that crap the industry has basically been telling people.
TulliusCicero 5y ago
> So why do the editors of all these journals keep feeding off the ML == AI meme? Your comment is a good example of AI meaning whatever we haven't solved yet. Playing chess well is AI until we have programs that can regularly defeat professionals, then it suddenly doesn't count. Accurately labeling the contents of a photo is AI until we have services that do this for anyone, and then it's not AI anymore. Pretty soon self-driving cars will also not count as AI, because we'll have solved self-driving and therefore it's not hard enough anymore.
- vanusa 5y ago
  Your comment is a good example of AI meaning whatever we haven't solved yet. Yes, I'm aware of the Rodney Brooks quote. I think it's kind of glib and misses the mark. Take chess, for example. When algorithms were developed to beat humans at chess -- it wasn't, like the quote implies, out of some sense of envy or schadenfreude that people said, "Oh, but that's not real AI". But rather -- from the intrinsic features of the system that first met this challenge. Which, at the end of the day, were basically about generalized tree search. Okay, there was more to it than that -- but in 3 word or less, that's gist of the AlphaZero algorithm, as described in the original paper. So people said "Oh -- it turns out you don't need human-like cognition to beat humans at chess/go/shogi. You just need good enough tree search, and some decent hardware." Pretty soon self-driving cars will also not count as AI, because we'll have solved self-driving and therefore it's not hard enough anymore. That's one of the weird things about the current moment in AI: people keep talking about self-driving cars as if they're a straight-line extrapolation the breakthroughs we've seen so far (give or take a few details). They're not. If you just think about if for a minute: nearly all of the challenges met thus far -- taking the two you just mentioned as particularly shining examples -- as significant as they have been, have been pretty darn one-dimensional. "Here's a function; optimize it" (basically). The reason self-driving cars are floundering at the moment is that the intrinsic problem they're dealing with is an entirely different beast from "solve this symbolic game faster than a human" or "label these photos better than a roomful of monkees". It is vastly more complex. And the emerging consensus (whatever some companies may say to the contrary) is that solution is not lurking just around the corner, and not coming our way anytime soon -- certainly not "pretty soon".

aerodog 5y ago

surprised Erik Demaine wasn't part of this

deleted 5y ago

[deleted]

agnosticmantis 5y ago

See [0] for a critical review of (the popularization of) this paper. As usual, there’s something real here but it’s way overhyped because everyone involved benefits from over-playing the role of deep learning here. [0:https://arxiv.org/abs/2112.04324 https://arxiv.org/abs/2112.04324] Edit: the critical review isn’t about the technical content of the paper per se, but rather the hyped news around it.

version_five 5y ago
There seems to be a lot of animosity towards deep learning, to the point that it comes off as jealousy - I mean read the other long thread in this discussion. I don't mean to say the criticisms in the article you linked are not valid, just that there are a lot of people ready to jump on any failure or exaggeration related to ML, giving it far more scrutiny than most research disciplines. It gets more money than most as well so maybe that's ok. And I know it gets overhyped in business contexts and media. But there are research preprints. That said, my observations about this story was that one one hand, everyone talks about how AI/ML will change everything and has lots of applications. But this article strikes my as clearly as a story of a solution in search of a problem, that had to go deep into a branch of theoretical math to find a place where ML could make a demonstrable impact. I know this application is not the only thing ML is good at (for context I own a machine learning consulting company and am bullish on the business applications of ML). But when you look at deepmind and the money that's gone into it, between the often cited "data centre cooling optimization" and this, they are not exactly having a massive impact, even as they do a lot of cool stuff.
- Jensson 5y ago
  > There seems to be a lot of animosity towards deep learning, to the point that it comes off as jealousy This happens every time people post research here. Any article about new battery tech will have people saying "What about all the other new battery techs that never went anywhere? This is just a prototype, it wont revolutionize the industry", and they are right. The difference is that when people say those things in a thread about ML, then AI optimists who thinks that these things are evidence that the singularity is near starts to argue that no, this time is truly different! And they do that every single time, AI optimists have believed that computers will take over any day now for 50 years. If those AI optimists didn't post so much then those threads wouldn't grow.

dukeofdoom 5y ago

Is there a reason why things get tangled so easily. Like it seems like its the way of the universe. I recently tried fly fishing. I spend 1/3 of the time trying to untangle my fishing line.

Jensson 5y ago
Probably entropy. If you randomize fishing line positions almost all of them would be tangled. Now putting the fishing line in a box isn't the same thing as randomizing it, but the point of entropy is that small random changes will move the state towards higher entropy, meaning that the random changes ought to move the line towards being tangled. Of course this assumes that a tangled line has higher entropy for some definition of entropy. Proving/structuring that would be too much work though, but if solved could make for a nice paper. Hmm, since it would be such a nice paper I looked it up, and apparently people have already studied these things: https://www.npr.org/sections/bryantpark/2008/01/knot_theory_why_everything_tan.html https://www.npr.org/sections/bryantpark/2008/01/knot_theory_...
- dukeofdoom 5y ago
  Thanks for the great reply! Fascinating
bamboozled 5y ago
Almost never happens to me, but it did when learning. So I think the reason it’s happening is you. All due respect. Edit: I’ll try be more constructive. There’s a good chance you’re looping your fly back over the line during your cast crating knots and tangles. I do agree stuff tangles easily but I don’t find it to be the case with fly lines in general.
- dukeofdoom 5y ago
  I'm definitely doing that. I'm using an amazon rod and reel combo deal by maxcatch, and it came with a very fine fishing line at the tip of the leader. So the wind picks it up, and loops it around the tip of the rod sometimes. But I also had snags inside the reel. And snags and tangles around bushes, and tree limbs.

oAlbe 5y ago

There's a pretty good Numberphile Podcast episode[0] about this. It doesn't go deep into mathematical details of the topic, it's more of an overview of the current state of things and an explanation of ways in which ML can help coming up with new theories/conjectures. [0] https://www.numberphile.com/podcast/deep-mind https://www.numberphile.com/podcast/deep-mind

sarosh 5y ago

The pre-print referred to in this article itself is here: https://arxiv.org/abs/2111.15323 https://arxiv.org/abs/2111.15323

zmachinaz 5y ago

There has been work of similar spirit going on since at least 2017 in mathematical/theoretical physics. With similar limited importance. But the ideas are floating around already for a long time. But if DeepMind gives it a go they make a big fuzz about it, and of course they even do not need to cite previous works ...