5 ms·

It is not a good idea to compile TensorFlow by your own unless you really need it (for example for TensorFlow serving). Python packages are the way to go.

by jorgemf 10y ago

jacquesm 10y ago
I have to because I'm trying to use TensorBox which does not play well with the regular version of tensorflow that you can get pre-compiled. See this issue: https://github.com/TensorBox/TensorBox/issues/100 https://github.com/TensorBox/TensorBox/issues/100 and https://github.com/TensorBox/TensorBox/issues/102 https://github.com/TensorBox/TensorBox/issues/102 So then we're full-circle and installing from pip which doesn't work :( sigh. Anyway, I'll get it to work, somehow.
- jorgemf 10y ago
  Those issues are related with TensorFlow <1.0. It wasn't an stable release. Try 1.0.1.
aseipp 10y ago
This is a bit unfortunate, in a real sense. I mean, I already build enough software, so I'm not sad on missing out. But here's the thing: TensorFlow actually installed great on Windows and it took less than 10 minutes to get running, once I had Python3 installed, even with GPU support. Even worked awesome in VS Code, out of the box, with autocomplete in the python mode. Even a baby like me got started easily. But it's a bit disappointing to hear that the build system is something of a nightmare, if I ever wanted to contribute myself. There's always plenty of things to help with, I don't care about the cutting edge of machine learning (I'm happy to submit docs, examples, etc)... Then again, the TF people can't just nerd around on their build system, for dorks like me to maybe write some patches every once in a while. Always great to make it easier, though.
- jacquesm 10y ago
  For me the issue with such things is not so much the hassle as much as it is long term stability of an environment that I come to depend on. If I can't reproduce it from archived sources then there is a good chance that at some point in the future my stuff will suddenly and in-explicably stop working after some minor system update. And python has a huge problem with this anyway. (Or, to put it probably more accurately, I have a huge problem with python in this way, historically my python code has had a relatively short shelf life compared to my C code (or even my PHP code)).
- jorgemf 10y ago
  > But it's a bit disappointing to hear that the build system is something of a nightmare I know... I am building it for TensorFlow Serving and it is pain in the ass. I will help but I have no idea how to do it, only find workarounds to compile specific commits.
- steren 10y ago
  Hi, I am working on Bazel, the build tool used by Tensorflow. Could you please share in more details the problems you encountered with Bazel? If you have specific questions, I encourage you to post on StackOverflow with the bazel and tensorflow tags, both the Bazel and Tensorflow teams are monitoring and answering questions there.
deleted 10y ago
[deleted]
peterbonney 10y ago
I disagree - I think you are well-served to compile on your own unless you know you don't need it, e.g. if you are just trying it out to learn how it works. The standard build uses a "least common denominator" Intel instruction set (SSE4), but the odds are extremely high that the machine on which you're running tensorflow supports 4.2. Building from source allows you to use the most up-to-date instruction set (the default configuration script at https://www.tensorflow.org https://www.tensorflow.org will do it automatically). I've seen dramatic (>50%) reductions in processing time on test scripts by building from source. Note that those tests were built primarily for my own education, not for benchmarking. But the speedup was so dramatic that I couldn't help but notice and probe a little deeper. YMMV depending on the particular application, whether you are using GPU computing (I am not), etc.
- matt4077 10y ago
  I don't quite understand how you performance-obsessed enough to care about the compiler options they use – but not run Tensorflow on a GPU. Even mobile GPUs offer a 5x speedup at least.
  - peterbonney 10y ago
    As I said above, if you are constrained to run on a machine without a supported GPU then every little bit helps.
- jorgemf 10y ago
  >The standard build uses a "least common denominator" Intel instruction set (SSE4) TensorFlow is not meant for the CPU. If you want to do something serious you have to move to the GPU, which it is at least 8 times faster than your optimized CPU with an average GPU.
  - peterbonney 10y ago
    For those of us waiting on OpenCL support because we don't have an Nvidia GPU, CPU will have to do for now.

It is not a good idea to compile TensorFlow by your own unless you really need it (for example for TensorFlow serving). Python packages are the way to go.

by jorgemf 10y ago

It is not a good idea to compile TensorFlow by your own unless you really need it (for example for TensorFlow serving). Python packages are the way to go.

jacquesm 10y ago

I have to because I'm trying to use TensorBox which does not play well with the regular version of tensorflow that you can get pre-compiled. See this issue: https://github.com/TensorBox/TensorBox/issues/100 https://github.com/TensorBox/TensorBox/issues/100 and https://github.com/TensorBox/TensorBox/issues/102 https://github.com/TensorBox/TensorBox/issues/102 So then we're full-circle and installing from pip which doesn't work :( sigh. Anyway, I'll get it to work, somehow.

jorgemf 10y ago
Those issues are related with TensorFlow <1.0. It wasn't an stable release. Try 1.0.1.

aseipp 10y ago

This is a bit unfortunate, in a real sense. I mean, I already build enough software, so I'm not sad on missing out. But here's the thing: TensorFlow actually installed great on Windows and it took less than 10 minutes to get running, once I had Python3 installed, even with GPU support. Even worked awesome in VS Code, out of the box, with autocomplete in the python mode. Even a baby like me got started easily. But it's a bit disappointing to hear that the build system is something of a nightmare, if I ever wanted to contribute myself. There's always plenty of things to help with, I don't care about the cutting edge of machine learning (I'm happy to submit docs, examples, etc)... Then again, the TF people can't just nerd around on their build system, for dorks like me to maybe write some patches every once in a while. Always great to make it easier, though.

jacquesm 10y ago
For me the issue with such things is not so much the hassle as much as it is long term stability of an environment that I come to depend on. If I can't reproduce it from archived sources then there is a good chance that at some point in the future my stuff will suddenly and in-explicably stop working after some minor system update. And python has a huge problem with this anyway. (Or, to put it probably more accurately, I have a huge problem with python in this way, historically my python code has had a relatively short shelf life compared to my C code (or even my PHP code)).
jorgemf 10y ago
> But it's a bit disappointing to hear that the build system is something of a nightmare I know... I am building it for TensorFlow Serving and it is pain in the ass. I will help but I have no idea how to do it, only find workarounds to compile specific commits.
steren 10y ago
Hi, I am working on Bazel, the build tool used by Tensorflow. Could you please share in more details the problems you encountered with Bazel? If you have specific questions, I encourage you to post on StackOverflow with the bazel and tensorflow tags, both the Bazel and Tensorflow teams are monitoring and answering questions there.

deleted 10y ago

[deleted]

peterbonney 10y ago

I disagree - I think you are well-served to compile on your own unless you know you don't need it, e.g. if you are just trying it out to learn how it works. The standard build uses a "least common denominator" Intel instruction set (SSE4), but the odds are extremely high that the machine on which you're running tensorflow supports 4.2. Building from source allows you to use the most up-to-date instruction set (the default configuration script at https://www.tensorflow.org https://www.tensorflow.org will do it automatically). I've seen dramatic (>50%) reductions in processing time on test scripts by building from source. Note that those tests were built primarily for my own education, not for benchmarking. But the speedup was so dramatic that I couldn't help but notice and probe a little deeper. YMMV depending on the particular application, whether you are using GPU computing (I am not), etc.

matt4077 10y ago
I don't quite understand how you performance-obsessed enough to care about the compiler options they use – but not run Tensorflow on a GPU. Even mobile GPUs offer a 5x speedup at least.
- peterbonney 10y ago
  As I said above, if you are constrained to run on a machine without a supported GPU then every little bit helps.
jorgemf 10y ago
>The standard build uses a "least common denominator" Intel instruction set (SSE4) TensorFlow is not meant for the CPU. If you want to do something serious you have to move to the GPU, which it is at least 8 times faster than your optimized CPU with an average GPU.
- peterbonney 10y ago
  For those of us waiting on OpenCL support because we don't have an Nvidia GPU, CPU will have to do for now.