9 ms·

Graph-Powered Machine Learning at Google

371 points by tim_sw 10y ago

zitterbewegung 10y ago
Anyone know if they are going to release code in Tensorflow that works with public data? I have been using the inbox reply feature in inbox and its very useful.
- akssri 10y ago
  These algorithms can't really make much use of Tensorflow.
harigov 10y ago
This is really cool. The underlying graph optimization algorithm seems similar to how graphical models are trained. Is that correct? Can someone please help me understand the difference?
- throawaybay 10y ago
  I mean, it should be.. you are learning a probability distribution over the nodes and then performing some kind of inference at eval time.
- deleted 10y ago
  [deleted]
- fritzo 10y ago
  Yes, label propagation is very similar to belief propagation on a Gaussian Markov random field (a particularly simple type of graphical model). To be precise this is 'inference' on the graphical model, rather than 'training' or 'learning'. See the author's 2016 JMLR paper for more details: http://www.jmlr.org/proceedings/papers/v51/ravi16.pdf http://www.jmlr.org/proceedings/papers/v51/ravi16.pdf
nicolewhite 10y ago
I'm a big fan of the smart replies in Allo and Inbox so this was a fun read. I did something similar in grad school where I and some other students manually labeled a handful of sentences and then used graph-based semi-supervised learning to label the rest for the purposes of using it as a training dataset. It would be neat to hear what technology they used for the graph-based learning; perhaps Cayley? We used Python's igraph at the time but it was pretty slow. It would also be interesting to try this in Neo4j.
bnchrch 10y ago
While there is a lot to get excited about with ML both as a consumer and as a software developer I can't help feeling a pang of sadness. Big data and in this case the relationship (graph) between big data points are whats needed to make great ML/AI products. By nature the only companies that will ever have access to this data in a meaningful way are going to be larger companies: Google, Amazon, Apple, ect. Because of this I worry that small upstarts may never be able to compete on these type of products in a viable way as the requirements to build these features are so easily defensible by the larger incumbents. I hope this is not the case but I'm getting less and less optimistic when I see articles like this.
- gordon_freeman 10y ago
  It depends on the vision and products too imo. Tesla is much smaller car company than Honda. But Musk just announced that they have collected 222m Autopilot miles from Tesla cars. I assume they have been feeding all that data to ML/AI systems to create fully autonomous vehicles (Level 4). Now I don't think Honda has designed its cars in such a way that allows it to collect data from its Honda Sensing features while the cars are driven.
- notliketherest 10y ago
  Yes, and it's a land grab for those domains with untapped data and potential for ML/AI related products. New startups that can identify these untapped areas will succeed.
- AndrewKemendo 10y ago
  I wrote about this a few months ago [1] The one thing that makes me feel slightly better about this is the recently announced AmaGoogFaceSoft partnership on training data open sourcing [2]. I say slightly because they still control the pipes and what is released - so they'll never likely release the good data, also cause it would probably cause privacy issues if they did. It's incumbent on us as the little guys and girls to make new data pipes through innovative products. [1] https://medium.com/@andrewkemendo/the-ai-revolution-will-be-different-from-the-microprocessor-revolution-cb417d324203#.qzp87ojum https://medium.com/@andrewkemendo/the-ai-revolution-will-be-... [2]http://www.partnershiponai.org/ http://www.partnershiponai.org/
yalogin 10y ago
Unrelated but none of the google blog links open up on safari in iOS. The link opens but text doesn't show.
- lis 10y ago
  This is probably due to your ad blocker. Same here. The page doesn't work if you disable the tracking. After disabling Focus it's working.
  - protomikron 10y ago
    That this is the case and explanation is unacceptable (not that I blame the parent, but I would like to add the statement, as we might otherwise forget, that information could be available without the tracking-mafia).
mark_l_watson 10y ago
Great stuff. I have been using machine learning since the late 1980s. Semi supervised learning is new to me, and it is exciting to be able to mix labeled and unlabeled data.
BinRoo 10y ago
The algorithm is well known. The main contribution is that they were able to scale up the algorithm to a huge graph. Google doesn't go into much detail about that because it's proprietary. So whats the point of this blog post? Oh yea, free Allo/Inbox advertisement.
prohor 10y ago
Is there implementation released maybe? Similar like they did with TensorFlow.
sandGorgon 10y ago
is there something in Expander that makes this happen... or can something like Apache Spark be used as well ?
b3n 10y ago
As a software engineer who knows nothing about ML, where's the best place to learn?
onetwo12 10y ago
I see this as a knn approach in which the distance is a function of the strength of the edge between the vertices. In the retrofitting paper cited in the comments there is a process of smoothing, that is feeding back the message or information to update the states of the graph (example in the modern ai book). It doesn't seem anything new.

371 points by tim_sw 10y ago

Anyone know if they are going to release code in Tensorflow that works with public data? I have been using the inbox reply feature in inbox and its very useful.

akssri 10y ago
These algorithms can't really make much use of Tensorflow.

harigov 10y ago

This is really cool. The underlying graph optimization algorithm seems similar to how graphical models are trained. Is that correct? Can someone please help me understand the difference?

throawaybay 10y ago
I mean, it should be.. you are learning a probability distribution over the nodes and then performing some kind of inference at eval time.
deleted 10y ago
[deleted]
fritzo 10y ago
Yes, label propagation is very similar to belief propagation on a Gaussian Markov random field (a particularly simple type of graphical model). To be precise this is 'inference' on the graphical model, rather than 'training' or 'learning'. See the author's 2016 JMLR paper for more details: http://www.jmlr.org/proceedings/papers/v51/ravi16.pdf http://www.jmlr.org/proceedings/papers/v51/ravi16.pdf

nicolewhite 10y ago

I'm a big fan of the smart replies in Allo and Inbox so this was a fun read. I did something similar in grad school where I and some other students manually labeled a handful of sentences and then used graph-based semi-supervised learning to label the rest for the purposes of using it as a training dataset. It would be neat to hear what technology they used for the graph-based learning; perhaps Cayley? We used Python's igraph at the time but it was pretty slow. It would also be interesting to try this in Neo4j.

bnchrch 10y ago

While there is a lot to get excited about with ML both as a consumer and as a software developer I can't help feeling a pang of sadness. Big data and in this case the relationship (graph) between big data points are whats needed to make great ML/AI products. By nature the only companies that will ever have access to this data in a meaningful way are going to be larger companies: Google, Amazon, Apple, ect. Because of this I worry that small upstarts may never be able to compete on these type of products in a viable way as the requirements to build these features are so easily defensible by the larger incumbents. I hope this is not the case but I'm getting less and less optimistic when I see articles like this.

gordon_freeman 10y ago
It depends on the vision and products too imo. Tesla is much smaller car company than Honda. But Musk just announced that they have collected 222m Autopilot miles from Tesla cars. I assume they have been feeding all that data to ML/AI systems to create fully autonomous vehicles (Level 4). Now I don't think Honda has designed its cars in such a way that allows it to collect data from its Honda Sensing features while the cars are driven.
notliketherest 10y ago
Yes, and it's a land grab for those domains with untapped data and potential for ML/AI related products. New startups that can identify these untapped areas will succeed.
AndrewKemendo 10y ago
I wrote about this a few months ago [1] The one thing that makes me feel slightly better about this is the recently announced AmaGoogFaceSoft partnership on training data open sourcing [2]. I say slightly because they still control the pipes and what is released - so they'll never likely release the good data, also cause it would probably cause privacy issues if they did. It's incumbent on us as the little guys and girls to make new data pipes through innovative products. [1] https://medium.com/@andrewkemendo/the-ai-revolution-will-be-different-from-the-microprocessor-revolution-cb417d324203#.qzp87ojum https://medium.com/@andrewkemendo/the-ai-revolution-will-be-... [2]http://www.partnershiponai.org/ http://www.partnershiponai.org/

yalogin 10y ago

Unrelated but none of the google blog links open up on safari in iOS. The link opens but text doesn't show.

lis 10y ago
This is probably due to your ad blocker. Same here. The page doesn't work if you disable the tracking. After disabling Focus it's working.
- protomikron 10y ago
  That this is the case and explanation is unacceptable (not that I blame the parent, but I would like to add the statement, as we might otherwise forget, that information could be available without the tracking-mafia).

mark_l_watson 10y ago

Great stuff. I have been using machine learning since the late 1980s. Semi supervised learning is new to me, and it is exciting to be able to mix labeled and unlabeled data.

BinRoo 10y ago

The algorithm is well known. The main contribution is that they were able to scale up the algorithm to a huge graph. Google doesn't go into much detail about that because it's proprietary. So whats the point of this blog post? Oh yea, free Allo/Inbox advertisement.

prohor 10y ago

Is there implementation released maybe? Similar like they did with TensorFlow.

sandGorgon 10y ago

is there something in Expander that makes this happen... or can something like Apache Spark be used as well ?

b3n 10y ago

As a software engineer who knows nothing about ML, where's the best place to learn?

onetwo12 10y ago

I see this as a knn approach in which the distance is a function of the strength of the edge between the vertices. In the retrofitting paper cited in the comments there is a process of smoothing, that is feeding back the message or information to update the states of the graph (example in the modern ai book). It doesn't seem anything new.