4 ms·

Now supporting https://github.com/allenai/tango https://github.com/allenai/tango

by gardenfelder 4y ago

Mathnerd314 4y ago
which is a build script that only integrates with torch. So they've switched to plain torch.
- lgessler 4y ago
  Did AllenNLP ever support any other engine? For the past couple years at least I think they've only supported PyTorch
- schmmd 4y ago
  When we started AllenNLP, PyTorch was just starting to emerge as a competitor to Tensorflow and we made the difficult decision to support PyTorch. In hindsight this was a great decision as the majority of top research is done in PyTorch today. Tango primarily supports PyTorch, but unlike AllenNLP, is flexible enough to support other deep learning libraries as well. For example, we're adding support for JAX so we can easily leverage TPUs.
  - linspace 4y ago
    For what I've seen Tango is a general dag/pipeline that happens to have some facilities for PyTorch. I don't see any deep learning specific. You could execute sklearn or whatever.
    - marvinalone 4y ago
      Maybe we need to re-work the docs if the DAG aspects stick out to you so much. The main functionality is the cache. If you have a complex experiment, you can still write the code as if all the steps were fast, and let them be slow only the first time you run it. The DAG stuff is also nice, but less important. That said, you could execute sklearn. If that's what your experiment needs, it's the right thing to do. This is why it gives us the flexibility to also support Jax: https://github.com/allenai/tango/pull/313 https://github.com/allenai/tango/pull/313 The DL-specific stuff is in the components we supply. Like the trainer, dataset handling stuff, file formats, and increasingly, https://github.com/allenai/catwalk https://github.com/allenai/catwalk.
- marvinalone 4y ago
  AllenNLP has only ever supported Torch. At the moment, Tango only supports Torch as well, but Jax support is well underway. And yeah, Tango is a lot like a build script. In fact, I used to manage my experiments with Makefiles. Tango is better though. Results don't have to be single files, and they don't have to live in one filesystem either, so I can run the GPU-heavy parts of my experiments on one machine, and the CPU-heavy parts on another. The way you version your code is better than what Makefiles can do. You have actual control beyond file modification time. And of course, there is the whole Python integration stuff.

Now supporting https://github.com/allenai/tango https://github.com/allenai/tango

Mathnerd314 4y ago

which is a build script that only integrates with torch. So they've switched to plain torch.

lgessler 4y ago
Did AllenNLP ever support any other engine? For the past couple years at least I think they've only supported PyTorch
schmmd 4y ago
When we started AllenNLP, PyTorch was just starting to emerge as a competitor to Tensorflow and we made the difficult decision to support PyTorch. In hindsight this was a great decision as the majority of top research is done in PyTorch today. Tango primarily supports PyTorch, but unlike AllenNLP, is flexible enough to support other deep learning libraries as well. For example, we're adding support for JAX so we can easily leverage TPUs.
- linspace 4y ago
  For what I've seen Tango is a general dag/pipeline that happens to have some facilities for PyTorch. I don't see any deep learning specific. You could execute sklearn or whatever.
  - marvinalone 4y ago
    Maybe we need to re-work the docs if the DAG aspects stick out to you so much. The main functionality is the cache. If you have a complex experiment, you can still write the code as if all the steps were fast, and let them be slow only the first time you run it. The DAG stuff is also nice, but less important. That said, you could execute sklearn. If that's what your experiment needs, it's the right thing to do. This is why it gives us the flexibility to also support Jax: https://github.com/allenai/tango/pull/313 https://github.com/allenai/tango/pull/313 The DL-specific stuff is in the components we supply. Like the trainer, dataset handling stuff, file formats, and increasingly, https://github.com/allenai/catwalk https://github.com/allenai/catwalk.
marvinalone 4y ago
AllenNLP has only ever supported Torch. At the moment, Tango only supports Torch as well, but Jax support is well underway. And yeah, Tango is a lot like a build script. In fact, I used to manage my experiments with Makefiles. Tango is better though. Results don't have to be single files, and they don't have to live in one filesystem either, so I can run the GPU-heavy parts of my experiments on one machine, and the CPU-heavy parts on another. The way you version your code is better than what Makefiles can do. You have actual control beyond file modification time. And of course, there is the whole Python integration stuff.