5 ms·

TensorFlow: Machine Learning on Heterogeneous Distributed Systems (2015) [pdf]

103 points by jxub 8y ago

godelmachine 8y ago
I just had a quick glance through the paper and felt they didn't worry too much about explaining the hardware architecture.
- ucaetano 8y ago
  Are you mixing this up with TPUs? Tensorflow is HW independent, as it is just a SW library.
  - godelmachine 8y ago
    I think you are right. Sorry, my bad! I remember the TPU paper Google had released last year (2017) circa March/ April, on performance evaluation of TPU's. Got a bit messed up.
chewxy 8y ago
As the primary author of Gorgonia[0], thank you for this. I've never read this. It's a fascinating read about the thought processes that go into creating TensorFlow. One particularly interesting thing is that it looks like Tensorflow started earlier than I thought (I've always thought Gorgonia predated Tensorflow (I started a few months before giving this talk[1], but only announced it one year later[2])). 3 years later, TensorFlow is super feature packed, and I'm still trying to drive adoption for my lib. Oh well. [0] https://github.com/gorgonia/gorgonia https://github.com/gorgonia/gorgonia [1] https://speakerdeck.com/chewxy/deep-learning-in-go-or-shennanigans-with-matrices https://speakerdeck.com/chewxy/deep-learning-in-go-or-shenna... [2] https://speakerdeck.com/chewxy/gorgonia-primitives-for-building-neural-networks-in-go https://speakerdeck.com/chewxy/gorgonia-primitives-for-build...
sja 8y ago
I remember this guy! I put together some notes and annotations for it when it came out. Some people found them useful back then- I'll leave them here in case other people can make use of them. https://github.com/samjabrahams/tensorflow-white-paper-notes https://github.com/samjabrahams/tensorflow-white-paper-notes
yablak 8y ago
One of the authors here. Working on TensorFlow has been quite a trip! Here's a followup paper submitted to Usenix: https://www.usenix.org/system/files/conference/osdi16/osdi16-abadi.pdf https://www.usenix.org/system/files/conference/osdi16/osdi16... And here's a very recent publication detailing just the flow control aspects (conditionals, while loops): https://dl.acm.org/citation.cfm?id=3190551 https://dl.acm.org/citation.cfm?id=3190551

103 points by jxub 8y ago

I just had a quick glance through the paper and felt they didn't worry too much about explaining the hardware architecture.

ucaetano 8y ago
Are you mixing this up with TPUs? Tensorflow is HW independent, as it is just a SW library.
- godelmachine 8y ago
  I think you are right. Sorry, my bad! I remember the TPU paper Google had released last year (2017) circa March/ April, on performance evaluation of TPU's. Got a bit messed up.

chewxy 8y ago

As the primary author of Gorgonia[0], thank you for this. I've never read this. It's a fascinating read about the thought processes that go into creating TensorFlow. One particularly interesting thing is that it looks like Tensorflow started earlier than I thought (I've always thought Gorgonia predated Tensorflow (I started a few months before giving this talk[1], but only announced it one year later[2])). 3 years later, TensorFlow is super feature packed, and I'm still trying to drive adoption for my lib. Oh well. [0] https://github.com/gorgonia/gorgonia https://github.com/gorgonia/gorgonia [1] https://speakerdeck.com/chewxy/deep-learning-in-go-or-shennanigans-with-matrices https://speakerdeck.com/chewxy/deep-learning-in-go-or-shenna... [2] https://speakerdeck.com/chewxy/gorgonia-primitives-for-building-neural-networks-in-go https://speakerdeck.com/chewxy/gorgonia-primitives-for-build...

sja 8y ago

I remember this guy! I put together some notes and annotations for it when it came out. Some people found them useful back then- I'll leave them here in case other people can make use of them. https://github.com/samjabrahams/tensorflow-white-paper-notes https://github.com/samjabrahams/tensorflow-white-paper-notes

yablak 8y ago

One of the authors here. Working on TensorFlow has been quite a trip! Here's a followup paper submitted to Usenix: https://www.usenix.org/system/files/conference/osdi16/osdi16-abadi.pdf https://www.usenix.org/system/files/conference/osdi16/osdi16... And here's a very recent publication detailing just the flow control aspects (conditionals, while loops): https://dl.acm.org/citation.cfm?id=3190551 https://dl.acm.org/citation.cfm?id=3190551