4 ms·
While this is cool I have a general question to people here: How do other people here handle tensorflow as a python dependency? People use it like it's any oth
by rjzzleep 6y ago
While this is cool I have a general question to people here:
How do other people here handle tensorflow as a python dependency? People use it like it's any other dependency, but it frequently adds unnamed amounts of dependencies, creates conflicts every now and then, is just massively huge and for the longest time of its 1.x existence was constantly breaking some PEP's causing problems with third party tools. It's kind of poison to custom docker files. How do you guys handle this situation? This isn't really my responsibility but whenever I deal with ML engineers doing tensorflow stuff I feel like I'm immediately also a plumber.
- JosephRedfern 6y agoI tend to use Anaconda where possible, makes such dependencies easier to manage (but still not trivial... though you can change cuda runtimes with Anaconda, the actual driver is a system-wide dependency, of course).
- GistNoesis 6y agoPython Virtual-env You create one new virtual environment for each different project, and you throw them away and build them up again once a problem appear.
- 6ak74rfy 6y agoYep, always use Python virtual environments to manage dependencies. Here are my notes around that: https://ketanvijayvargiya.com/notes/39-python-setup/ https://ketanvijayvargiya.com/notes/39-python-setup/
- nunja 6y agoVirtual envs are very practical for this as it does not mess up with the global python env. Also for inference / production, I just export my models to ONNX format which is more easy to work with thanks the available runtimes: ONNX Runtime for C++ or ONNX.js.
- mark_l_watson 6y agoOn my MacBook, I create a separate conda (Anaconda) environment for TensorFlow. A few years ago I bought a System76 laptop with a 1080 GPU. I rely on the System76 update system to keep Python + CUDA + TensorFlow all up to date. I manually install PyTorch for when I need it. This works well because I only use this laptop for deep learning.
- ccmonnett 6y agoHave you dealt with this in production? One of our data scientists suggested pulling in TF as a dependency into a mainline prod service just yesterday and there were some suspicions from ops people that it could cause trouble but nothing concrete. Would appreciate your perspective on that if you don't mind!