8 ms·

Deep Learning and Free Software

101 points by pilooch 8y ago

gaius 8y ago
I kinda think this is a non-issue. If you are working at the level of Keras then the back-end is pluggable. You can use a fancy GPU relying on a binary blob, you can use PlaidML to use any random GPU, you can use CPU as a last resort. The Keras code is all the same, you just set the backend you want in an environment variable.
stared 8y ago
People not sharing weights (or sharing but on non-open licenses) is an issue. Other things are not an issue (and from the tone of this post is clear that they repeat misheard things). First, you can load things into your network regardless if you use CPU or GPU. And if needed, people can write GPU code for other architectures/ Second, inference (prediction) is fast. Yeah, it may be not useful for real-time applications on CPU (like: self-driving cars) but for detecting one object it can be fast (see e.g. https://transcranial.github.io/keras-js/#/resnet50 https://transcranial.github.io/keras-js/#/resnet50). Third, using things like TensorFlow.js you get GPU acceleration with any GPU card, not only nVidia. It is not nearly as fast, but still faster than Python + CPU. There are real-time demos such as https://experiments.withgoogle.com/collection/ai/move-mirror/view/mirror https://experiments.withgoogle.com/collection/ai/move-mirror.... Side note: I just start https://inbrowser.ai/ https://inbrowser.ai/ for tutorials and open source templates for using fully frontend AI.
sanxiyn 8y ago
In the specific case of Leela Zero, there is no NVIDIA lockin whatsoever. Leela Zero uses OpenCL so that people with AMD GPU can use it. Deep learning research code can require NVIDIA; end-user game software mostly can't.
jsty 8y ago
'In fact, they are probably not redistributable unless all the training data is supplied, since the GPL's definition of "source code" is the "preferred form for modification". For a pretrained neural network that is the training data'. To me, this is the crux of the matter. If the view is taken that software distributed under a "free-software" compatible license is non-free without the ability to obtain all training sets for any models, there's going to be huge difficulty in incorporating ML into free software in many cases. Since many datasets aren't able to be redistributed freely (e.g. licensing, legal, cost), that's an enormous advantage for non-free offerings. A possible route around this might be 'community curated' datasets, where contributors freely license their data in the same way as we do code. It'd be interesting to see if they come with an analog of the AGPL - ie. models trained on this dataset must be released to the user as source. (This might already exist?)
- sanxiyn 8y ago
  In the specific case of Leela Zero, entire training dataset, all 3 gigabytes of it, is available for download at https://leela.online-go.com/zero/ https://leela.online-go.com/zero/
  - NhanH 8y ago
    The above link actually only has the raw sgf files (game records). The actual training data (output of MCTS search) is here: https://leela.online-go.com/training/ https://leela.online-go.com/training/
- bicubic 8y ago
  I think an ML model is not free if training dataset is not provided. You can't inspect it for vulnerabilities and you can't modify it in the common interpretation of the word. Whatever difficulties this view presents, this is the only view that makes sense.
  - caenorst 8y ago
    What makes you say that you can't modify a model ? And can't inspect it for vulnerabilities ?
    - protomikron 8y ago
      Because it's nearly impossible? It's like saying you can modify and audit Microsoft Windows or other proprietary software because you can write and read the bytes of its object code.
      - caenorst 8y ago
        That's not even an argument. Here is one against the claims. Modification: There have been repeatedly successful case of transfer learning where you take a pretrained deep neural network and train on a new dataset (even very small). Vulnerability: you dont need the training dataset to test vulnerabilities, you just either very specific input (that you probably gonna have to create yourself because they are so specific that they are not even in the dataset) or using one input that you corrupt for attacking. And as far as I know it doesn't sound harder than making test for regular algorithms.
protomikron 8y ago
This is totally an issue and I am glad there is some emerging discussion. It's true that practically you can run SGD on a CPU but it is really not feasible as your training time might blow up by a factor of 50 and it is not unusual to train a network for a week using nvidia GPUs (and their proprietary drivers and frameworks like cudnn). At this point in time you just can't do some of the DL stuff without proprietary nvidia tech.
infinity0 8y ago
Debian packager of Leela Zero here. Most of the questions raised in the thread were basically irrelevant to Leela Zero, which does everything "correct" from the point-of-view of a strict interpretation of free software: - open freely licensed data set - open freely licensed training code The only issue relevant for user software freedom, is that the results of the training process can't be reproduced easily. I was irritated at the thread because 90% of it was theorising about some alternative situation where the data set was not free, or the training code was not free. That is not the situation we have with Leela Zero, we can skip that discussion and focus on what's actually at issue, i.e. the reproduction / verification of the training process.
- moviuro 8y ago
  That's really just the reproductible build issue 2.0 I suppose no one could just lend you the infrastructure needed to rebuild the model (sponsors, or otherwise supporters of Debian?)
- chubot 8y ago
  What are the issues with reproducing the training process? It requires too many computing resources (which might require non-free cloud services), or it requires non-free software like GPU drivers, or both?
  - infinity0 8y ago
    For Leela Zero it's basically just the computing resources, the software is done using an abstract but openly-specified GPU execution system OpenCL, similar (as far as I understand) to an instruction set architecture (ISA). Whether the abstract execution system is implemented in a proprietary or free way is a separate issue. For example these days it's hard to execute x86 instructions without running some sort of proprietary microcode and/or management engine. For comparison, OpenCL has many competing implementations, including the free open source MESA graphics drivers.
- deleted 8y ago
  [deleted]
- 8y ago
xvilka 8y ago
NVIDIA is the major obstacle to FOSS for a decades already. So they keep their spirit even now. Mainline Tensorflow, for example, still doesn't support OpenCL.
- stochastic_monk 8y ago
  And OpenCL was recently deprecated by OSX, wasn’t it? Edit: yes. Discussion on hn: https://news.ycombinator.com/item?id=17231593 https://news.ycombinator.com/item?id=17231593
  - dragandj 8y ago
    So what? High-end GPU are used in servers and desktops, which almost exclusively run Linux in this space. The really relevant thing here is hardware (Nvidia vs AMD) and software - Nvidia drivers + CUDA (+ Nvidia's OpenCL) vs AMD drivers + OpenCL. So far Nvidia does excellent technical work with their hardware + their out of the box CUDA toolkit + support in every major deep learning library. The drawback is that they keep everything closed and they charge a huge premium. AMD has great hardware at good prices but the software side is non-existent and the Linux support goes from bad to worse. Nvidia is the one having absolute control here and they choose to squeeze the market because they can.
    - stochastic_monk 8y ago
      And Nvidia isn’t the only company to do so. Intel had a similar monopoly and squeezed it for all it was worth, while providing the best available scientific computing libraries. I can’t blame them, but I would certainly prefer it if there was a way to generically dispatch optimal code for heterogeneous hardware.
      - gnufx 8y ago
        The advantages of MKL (or does that mean something else?) are over-sold. Indeed, I understand the relevant primitives in this context were driven by Pabst's free implementations in libxsmm. I don't know whether "dispatch optimal code for heterogeneous hardware" means chose the SIMD instructions or offload appropriately, but what's missing in whichever case it is? Intel's free software release seems to me to be rather in their favour compared with other vendors.
jxub 8y ago
So... deep learning is just creating black-box platform-specific behaviours that can't be shared and thus threaten the OSS model. Is the era of computers + internet = freedom and knowledge-sharing over? Perhaps I'm over-dramatising but this is not something to be okay with.
black_puppydog 8y ago
Fundamentally, the large-scale collection of (often user generated) data to train models puts a more and more power into the hands of those doing the collection. Since giving away this data is mostly neither possible or desirable legally, nor in the interest of the dataset owner, there is a tradeoff between wanting to learn from data, and preserving privacy of the people generating/described by the data. That is not to say that science is not trying, like for example this paper: https://dl.acm.org/citation.cfm?id=2813687 https://dl.acm.org/citation.cfm?id=2813687
JanisL 8y ago
Its good to see some discussion about this. I'm currently maintaining an open source automated speech transcription library (https://github.com/persephone-tools/persephone https://github.com/persephone-tools/persephone) that's being used in some research and we are extremely concerned about enabling reproducible research. Because the system is neural net based (Tensorflow) we have gone to effort beyond just open sourcing the code to make sure that people can rerun experiments. Reading this and the comments makes me aware that there might be sources of nondeterminism or other issues I may not be aware of. What issues should we be aware of for making sure that people can replicate results that were trained earlier? Are there any resources that discuss how people have gone about doing this? Any advice would be much appreciated
- nlowell 8y ago
  I think distributing the weights will give you guaranteed reproducibility. If your users intend on retraining the network, another idea would be to set performance expectations on a validation set within ballpark. So you could tell them, "we got 90% accuracy on this dataset, if you re-train and get below 80 you've probably made a mistake somewhere." The scary thing to me would be the very small test cases where maybe different trained neural nets end up having a lot of variance because there is barely more than noise to learn.
  - JanisL 8y ago
    Distributing the weights seems to be the safest method, something I'll have to look into a bit more. It's a bit disturbing that training with the same data and same parameters could lead to different weights and therefore different accuracies but it's better to know that this is the case than not.
- sdenton4 8y ago
  My understanding (though it may be out of date) is that reproducibility is tough, at least in part because of closed gpu drivers, and in part because the big frameworks aren't really designed for it (possibly because it's pointless given the driver situation). On the framework side, you'd want good ways to ensure that random seeds are well distributed across a cluster of workers (and be able to restore state of a machine goes down). Also need to ensure that workers consume training data in a fixed order. This will slow you down, since you've got to wait for the slow poke every time, before updating weights.
tree_of_item 8y ago
People keep saying it's "not feasible" to do deep learning on a CPU. Is that actually true? I'm thinking of papers like this one[0] from Uber. that make the claim that CPU training is very much feasible, in this case with a cluster of 10 machines, each with 72~ CPUs. There's a blog post about someone recreating the paper here[1]. Sure, you can't do it on ONE CPU, but the point is to have a cluster of CPUs. It's not the case that you're forced to use nVidia's proprietary stuff in order to do deep learning. [0]: https://arxiv.org/abs/1712.06567 https://arxiv.org/abs/1712.06567 [1]: https://towardsdatascience.com/paper-repro-deep-neuroevolution-756871e00a66 https://towardsdatascience.com/paper-repro-deep-neuroevoluti...
- blt 8y ago
  It's definitely feasible and widely used for reinforcement learning on low dimensional systems, where the neural networks are small and the simulator is more expensive than backprop. On other hand, deep Q learning from pixels on Atari is practically impossible without GPUs.
  - tree_of_item 8y ago
    Huh? I just showed you reinforcement learning from pixels on Atari with CPU...
- gnufx 8y ago
  For what it's worth, Cori http://www.nersc.gov/users/data-analytics/data-analytics-2/deep-learning/ http://www.nersc.gov/users/data-analytics/data-analytics-2/d... claimed 15 PF https://arxiv.org/pdf/1708.05256.pdf https://arxiv.org/pdf/1708.05256.pdf

101 points by pilooch 8y ago

I kinda think this is a non-issue. If you are working at the level of Keras then the back-end is pluggable. You can use a fancy GPU relying on a binary blob, you can use PlaidML to use any random GPU, you can use CPU as a last resort. The Keras code is all the same, you just set the backend you want in an environment variable.

stared 8y ago

People not sharing weights (or sharing but on non-open licenses) is an issue. Other things are not an issue (and from the tone of this post is clear that they repeat misheard things). First, you can load things into your network regardless if you use CPU or GPU. And if needed, people can write GPU code for other architectures/ Second, inference (prediction) is fast. Yeah, it may be not useful for real-time applications on CPU (like: self-driving cars) but for detecting one object it can be fast (see e.g. https://transcranial.github.io/keras-js/#/resnet50 https://transcranial.github.io/keras-js/#/resnet50). Third, using things like TensorFlow.js you get GPU acceleration with any GPU card, not only nVidia. It is not nearly as fast, but still faster than Python + CPU. There are real-time demos such as https://experiments.withgoogle.com/collection/ai/move-mirror/view/mirror https://experiments.withgoogle.com/collection/ai/move-mirror.... Side note: I just start https://inbrowser.ai/ https://inbrowser.ai/ for tutorials and open source templates for using fully frontend AI.

sanxiyn 8y ago

In the specific case of Leela Zero, there is no NVIDIA lockin whatsoever. Leela Zero uses OpenCL so that people with AMD GPU can use it. Deep learning research code can require NVIDIA; end-user game software mostly can't.

jsty 8y ago

'In fact, they are probably not redistributable unless all the training data is supplied, since the GPL's definition of "source code" is the "preferred form for modification". For a pretrained neural network that is the training data'. To me, this is the crux of the matter. If the view is taken that software distributed under a "free-software" compatible license is non-free without the ability to obtain all training sets for any models, there's going to be huge difficulty in incorporating ML into free software in many cases. Since many datasets aren't able to be redistributed freely (e.g. licensing, legal, cost), that's an enormous advantage for non-free offerings. A possible route around this might be 'community curated' datasets, where contributors freely license their data in the same way as we do code. It'd be interesting to see if they come with an analog of the AGPL - ie. models trained on this dataset must be released to the user as source. (This might already exist?)

sanxiyn 8y ago
In the specific case of Leela Zero, entire training dataset, all 3 gigabytes of it, is available for download at https://leela.online-go.com/zero/ https://leela.online-go.com/zero/
- NhanH 8y ago
  The above link actually only has the raw sgf files (game records). The actual training data (output of MCTS search) is here: https://leela.online-go.com/training/ https://leela.online-go.com/training/
bicubic 8y ago
I think an ML model is not free if training dataset is not provided. You can't inspect it for vulnerabilities and you can't modify it in the common interpretation of the word. Whatever difficulties this view presents, this is the only view that makes sense.
- caenorst 8y ago
  What makes you say that you can't modify a model ? And can't inspect it for vulnerabilities ?
  - protomikron 8y ago
    Because it's nearly impossible? It's like saying you can modify and audit Microsoft Windows or other proprietary software because you can write and read the bytes of its object code.
    - caenorst 8y ago
      That's not even an argument. Here is one against the claims. Modification: There have been repeatedly successful case of transfer learning where you take a pretrained deep neural network and train on a new dataset (even very small). Vulnerability: you dont need the training dataset to test vulnerabilities, you just either very specific input (that you probably gonna have to create yourself because they are so specific that they are not even in the dataset) or using one input that you corrupt for attacking. And as far as I know it doesn't sound harder than making test for regular algorithms.

protomikron 8y ago

This is totally an issue and I am glad there is some emerging discussion. It's true that practically you can run SGD on a CPU but it is really not feasible as your training time might blow up by a factor of 50 and it is not unusual to train a network for a week using nvidia GPUs (and their proprietary drivers and frameworks like cudnn). At this point in time you just can't do some of the DL stuff without proprietary nvidia tech.

infinity0 8y ago

Debian packager of Leela Zero here. Most of the questions raised in the thread were basically irrelevant to Leela Zero, which does everything "correct" from the point-of-view of a strict interpretation of free software: - open freely licensed data set - open freely licensed training code The only issue relevant for user software freedom, is that the results of the training process can't be reproduced easily. I was irritated at the thread because 90% of it was theorising about some alternative situation where the data set was not free, or the training code was not free. That is not the situation we have with Leela Zero, we can skip that discussion and focus on what's actually at issue, i.e. the reproduction / verification of the training process.

moviuro 8y ago
That's really just the reproductible build issue 2.0 I suppose no one could just lend you the infrastructure needed to rebuild the model (sponsors, or otherwise supporters of Debian?)
chubot 8y ago
What are the issues with reproducing the training process? It requires too many computing resources (which might require non-free cloud services), or it requires non-free software like GPU drivers, or both?
- infinity0 8y ago
  For Leela Zero it's basically just the computing resources, the software is done using an abstract but openly-specified GPU execution system OpenCL, similar (as far as I understand) to an instruction set architecture (ISA). Whether the abstract execution system is implemented in a proprietary or free way is a separate issue. For example these days it's hard to execute x86 instructions without running some sort of proprietary microcode and/or management engine. For comparison, OpenCL has many competing implementations, including the free open source MESA graphics drivers.
deleted 8y ago
[deleted]
8y ago

xvilka 8y ago

NVIDIA is the major obstacle to FOSS for a decades already. So they keep their spirit even now. Mainline Tensorflow, for example, still doesn't support OpenCL.

stochastic_monk 8y ago
And OpenCL was recently deprecated by OSX, wasn’t it? Edit: yes. Discussion on hn: https://news.ycombinator.com/item?id=17231593 https://news.ycombinator.com/item?id=17231593
- dragandj 8y ago
  So what? High-end GPU are used in servers and desktops, which almost exclusively run Linux in this space. The really relevant thing here is hardware (Nvidia vs AMD) and software - Nvidia drivers + CUDA (+ Nvidia's OpenCL) vs AMD drivers + OpenCL. So far Nvidia does excellent technical work with their hardware + their out of the box CUDA toolkit + support in every major deep learning library. The drawback is that they keep everything closed and they charge a huge premium. AMD has great hardware at good prices but the software side is non-existent and the Linux support goes from bad to worse. Nvidia is the one having absolute control here and they choose to squeeze the market because they can.
  - stochastic_monk 8y ago
    And Nvidia isn’t the only company to do so. Intel had a similar monopoly and squeezed it for all it was worth, while providing the best available scientific computing libraries. I can’t blame them, but I would certainly prefer it if there was a way to generically dispatch optimal code for heterogeneous hardware.
    - gnufx 8y ago
      The advantages of MKL (or does that mean something else?) are over-sold. Indeed, I understand the relevant primitives in this context were driven by Pabst's free implementations in libxsmm. I don't know whether "dispatch optimal code for heterogeneous hardware" means chose the SIMD instructions or offload appropriately, but what's missing in whichever case it is? Intel's free software release seems to me to be rather in their favour compared with other vendors.

jxub 8y ago

So... deep learning is just creating black-box platform-specific behaviours that can't be shared and thus threaten the OSS model. Is the era of computers + internet = freedom and knowledge-sharing over? Perhaps I'm over-dramatising but this is not something to be okay with.

black_puppydog 8y ago

Fundamentally, the large-scale collection of (often user generated) data to train models puts a more and more power into the hands of those doing the collection. Since giving away this data is mostly neither possible or desirable legally, nor in the interest of the dataset owner, there is a tradeoff between wanting to learn from data, and preserving privacy of the people generating/described by the data. That is not to say that science is not trying, like for example this paper: https://dl.acm.org/citation.cfm?id=2813687 https://dl.acm.org/citation.cfm?id=2813687

JanisL 8y ago

Its good to see some discussion about this. I'm currently maintaining an open source automated speech transcription library (https://github.com/persephone-tools/persephone https://github.com/persephone-tools/persephone) that's being used in some research and we are extremely concerned about enabling reproducible research. Because the system is neural net based (Tensorflow) we have gone to effort beyond just open sourcing the code to make sure that people can rerun experiments. Reading this and the comments makes me aware that there might be sources of nondeterminism or other issues I may not be aware of. What issues should we be aware of for making sure that people can replicate results that were trained earlier? Are there any resources that discuss how people have gone about doing this? Any advice would be much appreciated

nlowell 8y ago
I think distributing the weights will give you guaranteed reproducibility. If your users intend on retraining the network, another idea would be to set performance expectations on a validation set within ballpark. So you could tell them, "we got 90% accuracy on this dataset, if you re-train and get below 80 you've probably made a mistake somewhere." The scary thing to me would be the very small test cases where maybe different trained neural nets end up having a lot of variance because there is barely more than noise to learn.
- JanisL 8y ago
  Distributing the weights seems to be the safest method, something I'll have to look into a bit more. It's a bit disturbing that training with the same data and same parameters could lead to different weights and therefore different accuracies but it's better to know that this is the case than not.
sdenton4 8y ago
My understanding (though it may be out of date) is that reproducibility is tough, at least in part because of closed gpu drivers, and in part because the big frameworks aren't really designed for it (possibly because it's pointless given the driver situation). On the framework side, you'd want good ways to ensure that random seeds are well distributed across a cluster of workers (and be able to restore state of a machine goes down). Also need to ensure that workers consume training data in a fixed order. This will slow you down, since you've got to wait for the slow poke every time, before updating weights.

tree_of_item 8y ago

People keep saying it's "not feasible" to do deep learning on a CPU. Is that actually true? I'm thinking of papers like this one[0] from Uber. that make the claim that CPU training is very much feasible, in this case with a cluster of 10 machines, each with 72~ CPUs. There's a blog post about someone recreating the paper here[1]. Sure, you can't do it on ONE CPU, but the point is to have a cluster of CPUs. It's not the case that you're forced to use nVidia's proprietary stuff in order to do deep learning. [0]: https://arxiv.org/abs/1712.06567 https://arxiv.org/abs/1712.06567 [1]: https://towardsdatascience.com/paper-repro-deep-neuroevolution-756871e00a66 https://towardsdatascience.com/paper-repro-deep-neuroevoluti...

blt 8y ago
It's definitely feasible and widely used for reinforcement learning on low dimensional systems, where the neural networks are small and the simulator is more expensive than backprop. On other hand, deep Q learning from pixels on Atari is practically impossible without GPUs.
- tree_of_item 8y ago
  Huh? I just showed you reinforcement learning from pixels on Atari with CPU...
gnufx 8y ago
For what it's worth, Cori http://www.nersc.gov/users/data-analytics/data-analytics-2/deep-learning/ http://www.nersc.gov/users/data-analytics/data-analytics-2/d... claimed 15 PF https://arxiv.org/pdf/1708.05256.pdf https://arxiv.org/pdf/1708.05256.pdf