3 ms·

Geometry of the restricted Boltzmann machine

13 points by chromophore 17y ago

jey 17y ago
More than six people on here are able to comprehend this? I'm not one of them. Never cared much for "Algebraic Geometry" nor "Metric Geometry". An introduction to the theory behind Boltzmann machines would be much more relevant here, imo. Maybe something like this: http://learning.cs.toronto.edu/~hinton/absps/pdp7.pdf http://learning.cs.toronto.edu/~hinton/absps/pdp7.pdf
- jerf 17y ago
  "We study this graphical model from the perspectives of algebraic statistics and tropical geometry, starting with the observation that its Zariski closure is a Hadamard power of the first secant variety of the Segre variety of projective lines." Look, if you don't know what that is right away just from the abstract I don't know how to help you... ... what with me having no clue either. I've always disliked mathematics' proclivity for naming things after people. I don't have any better solution, but I still don't like it. Even things I do understand look like gibberish if it has three or four proper names like that.
  - deleted 17y ago
    [deleted]
- chromophore 17y ago
  I am reading this these days. I think you are right about this not being here. Sorry. Will keep in mind.
  - bravura 17y ago
    I agree this that article is of very limited interest. A lot of people in my lab study RBMs and I'm not sure I would even forward this article to them.
  - jey 17y ago
    I don't blame you at all for submitting it. I'm just confused about who upvoted it and why.
    - chromophore 17y ago
      LOL. The paper is very recent. And extremely impregnable for someone like me. I was actually hoping for some comments on it by people who work on RBMs. Like I got some good comments on a ML gallery I posted. Sometimes comments help clear up things, and even a little clearing up is good IMHO. I think they did because they didn't have a look at the abstract, but read it partially by title only. And maybe you know, thought it was something introductory on RBM's ? I mean RBMs are extremely pretty. And I am just beginning to appreciate that.
      - slackenerny 17y ago
        There is seldom much interesting for ML person in this paper. (disclaimer: I haven't made much sense of its goal either. This reminds me of a saying “Write for lay audience, so maybe specialists will understand; Write for specialists, then noone will”) It's just Bern Strumfels running his research program, which is somewhat obscure even for algebraic geometrers. He is an expert on algorithms of algebraic geometry and their applications e.g. to optimization, and whatnot. There was a thematic school on that, videos should be buried somewhere here, http://www.ima.umn.edu/2006-2007/ http://www.ima.umn.edu/2006-2007/. This may be interesting line of research—for a mathematician—or ML person willing to learn stacks of pure mathematics leading to fluency with at least the book by Cox, Little and O'Shea just to make sense of labels on buttons.
        chromophore 17y ago
        Hmmm, Okay! Enough said. Why can't there be much interest? :) My old professor (who is an authority with RBMs) linked me up and asked me to see and make sure I know certain things about it.Let me give it some months and report back to him. I basically work with SVMs, it's his suggestion to study RBMs (which I know already) and sent me a number of papers. I am not sure why he sent me this one (it's just a month old). This one to be frank looked pretty nice to me, and I downloaded a number of other papers to study it. Even if I give up. I'll be glad to learn some of the math involved. I'd rather go with his opinion. Thanks. The aim of my submission (as is the aim of most my submissions) was actually to get some comments on it, and links to some resources. Not what a person should learn and what not. I'll know better what to do and why. Given the wide pool of people on HN, I was hoping to get something interesting, say some comments ON the paper. Even one helpful comment would have done.
newsdog 17y ago
Wow - so e e cummings and charlie the cockroach are now a professor. Seriously - nice work.
jasonmorton 17y ago
Just came across this discussion. I'm one of the authors, and if anyone is interested, I'd be happy to answer any questions about the paper (jason@math.stanford.edu). It originated in me hearing from Andrew Ng and other folks in the ML community about how powerful deep belief nets were, and wanting to quantify that power more explicitly. The first step, for mathematicians, is to understand what a RBM can represent. As you point out, the paper is written for a math journal and so emphasizes certain things. I presented a poster from a more ML perspective at the ICML workshops. Cheers!

13 points by chromophore 17y ago

jey 17y ago

More than six people on here are able to comprehend this? I'm not one of them. Never cared much for "Algebraic Geometry" nor "Metric Geometry". An introduction to the theory behind Boltzmann machines would be much more relevant here, imo. Maybe something like this: http://learning.cs.toronto.edu/~hinton/absps/pdp7.pdf http://learning.cs.toronto.edu/~hinton/absps/pdp7.pdf

jerf 17y ago
"We study this graphical model from the perspectives of algebraic statistics and tropical geometry, starting with the observation that its Zariski closure is a Hadamard power of the first secant variety of the Segre variety of projective lines." Look, if you don't know what that is right away just from the abstract I don't know how to help you... ... what with me having no clue either. I've always disliked mathematics' proclivity for naming things after people. I don't have any better solution, but I still don't like it. Even things I do understand look like gibberish if it has three or four proper names like that.
- deleted 17y ago
  [deleted]
chromophore 17y ago
I am reading this these days. I think you are right about this not being here. Sorry. Will keep in mind.
- bravura 17y ago
  I agree this that article is of very limited interest. A lot of people in my lab study RBMs and I'm not sure I would even forward this article to them.
- jey 17y ago
  I don't blame you at all for submitting it. I'm just confused about who upvoted it and why.
  - chromophore 17y ago
    LOL. The paper is very recent. And extremely impregnable for someone like me. I was actually hoping for some comments on it by people who work on RBMs. Like I got some good comments on a ML gallery I posted. Sometimes comments help clear up things, and even a little clearing up is good IMHO. I think they did because they didn't have a look at the abstract, but read it partially by title only. And maybe you know, thought it was something introductory on RBM's ? I mean RBMs are extremely pretty. And I am just beginning to appreciate that.
    - slackenerny 17y ago
      There is seldom much interesting for ML person in this paper. (disclaimer: I haven't made much sense of its goal either. This reminds me of a saying “Write for lay audience, so maybe specialists will understand; Write for specialists, then noone will”) It's just Bern Strumfels running his research program, which is somewhat obscure even for algebraic geometrers. He is an expert on algorithms of algebraic geometry and their applications e.g. to optimization, and whatnot. There was a thematic school on that, videos should be buried somewhere here, http://www.ima.umn.edu/2006-2007/ http://www.ima.umn.edu/2006-2007/. This may be interesting line of research—for a mathematician—or ML person willing to learn stacks of pure mathematics leading to fluency with at least the book by Cox, Little and O'Shea just to make sense of labels on buttons.
      - chromophore 17y ago
        Hmmm, Okay! Enough said. Why can't there be much interest? :) My old professor (who is an authority with RBMs) linked me up and asked me to see and make sure I know certain things about it.Let me give it some months and report back to him. I basically work with SVMs, it's his suggestion to study RBMs (which I know already) and sent me a number of papers. I am not sure why he sent me this one (it's just a month old). This one to be frank looked pretty nice to me, and I downloaded a number of other papers to study it. Even if I give up. I'll be glad to learn some of the math involved. I'd rather go with his opinion. Thanks. The aim of my submission (as is the aim of most my submissions) was actually to get some comments on it, and links to some resources. Not what a person should learn and what not. I'll know better what to do and why. Given the wide pool of people on HN, I was hoping to get something interesting, say some comments ON the paper. Even one helpful comment would have done.

newsdog 17y ago

Wow - so e e cummings and charlie the cockroach are now a professor. Seriously - nice work.

jasonmorton 17y ago

Just came across this discussion. I'm one of the authors, and if anyone is interested, I'd be happy to answer any questions about the paper (jason@math.stanford.edu). It originated in me hearing from Andrew Ng and other folks in the ML community about how powerful deep belief nets were, and wanting to quantify that power more explicitly. The first step, for mathematicians, is to understand what a RBM can represent. As you point out, the paper is written for a math journal and so emphasizes certain things. I presented a poster from a more ML perspective at the ICML workshops. Cheers!