4 ms·

> This open source release supports single machines and mobile devices. Can someone clarify if "single machines" means the Apache license only applies to singl

by aconz2 11y ago

> This open source release supports single machines and mobile devices. Can someone clarify if "single machines" means the Apache license only applies to single machines and not distributed systems? Sidenote: I wish every open source project had a release video.

sudhirj 11y ago
There's a note on the site about being able to move computation to a process on a different machine, but it does mention copying the model. If I understand correctly, I think models tend to be quite small, though. Does it actually make sense to train a model simultaneously on different machines? I can understand running the same model on lots of machines, of course.
- aconz2 11y ago
  I was interested in whether the license only permits a single machine.
  - Sanddancer 11y ago
    Nope, nothing in the license prohibiting it. Seems more a case that they just haven't released the glue for it to run on multiple machines, but hope to have it added one way or another.
- prajit 11y ago
  Yes, there are two types of parallelism: model and data. Data parallelism is simply training the the model on multiple computers with different minibatches, and aggregating the gradients. Model parallelism is hosting different parts of the model on different computers. Of course, these two parallelisms can be combined. A good explanation is the One Weird Trick paper: http://arxiv.org/abs/1404.5997 http://arxiv.org/abs/1404.5997
- dr_zoidberg 11y ago
  If there is randomness in some part of the training process (and most surely there will be), then it makes sense. Different trainings may end up in different local minimums/maximums, so there is value in running multiple trainings simultaneously in different machines.
vonmoltke 11y ago
> Can someone clarify if "single machines" means the Apache license only applies to single machines and not distributed systems? I'm not aware that there is a way to do that and still legitimately call it the Apache license. I took the statement to mean that the source does not have the capability of parallelizing across machine boundaries.
tdaltonc 11y ago
Is it common for code to open sourced, but only for use on single machines?
- davmre 11y ago
  The distributed part of the system probably depends on other components of Google's proprietary infrastructure, e.g., for cluster job scheduling. So it's a bit of work to tear that out and replace with something that's externally usable. I can't tell whether they've committed to doing that, but I hope they do.

by aconz2 11y ago

sudhirj 11y ago

There's a note on the site about being able to move computation to a process on a different machine, but it does mention copying the model. If I understand correctly, I think models tend to be quite small, though. Does it actually make sense to train a model simultaneously on different machines? I can understand running the same model on lots of machines, of course.

aconz2 11y ago
I was interested in whether the license only permits a single machine.
- Sanddancer 11y ago
  Nope, nothing in the license prohibiting it. Seems more a case that they just haven't released the glue for it to run on multiple machines, but hope to have it added one way or another.
prajit 11y ago
Yes, there are two types of parallelism: model and data. Data parallelism is simply training the the model on multiple computers with different minibatches, and aggregating the gradients. Model parallelism is hosting different parts of the model on different computers. Of course, these two parallelisms can be combined. A good explanation is the One Weird Trick paper: http://arxiv.org/abs/1404.5997 http://arxiv.org/abs/1404.5997
dr_zoidberg 11y ago
If there is randomness in some part of the training process (and most surely there will be), then it makes sense. Different trainings may end up in different local minimums/maximums, so there is value in running multiple trainings simultaneously in different machines.

vonmoltke 11y ago

> Can someone clarify if "single machines" means the Apache license only applies to single machines and not distributed systems? I'm not aware that there is a way to do that and still legitimately call it the Apache license. I took the statement to mean that the source does not have the capability of parallelizing across machine boundaries.

tdaltonc 11y ago

Is it common for code to open sourced, but only for use on single machines?

davmre 11y ago
The distributed part of the system probably depends on other components of Google's proprietary infrastructure, e.g., for cluster job scheduling. So it's a bit of work to tear that out and replace with something that's externally usable. I can't tell whether they've committed to doing that, but I hope they do.