3 ms·

I have attempted to do something similar in 2017 [1]. A couple of issues I noticed: * The field is evolving quite rapidly, and so most options will require a l

by mmq 8y ago

I have attempted to do something similar in 2017 [1]. A couple of issues I noticed: * The field is evolving quite rapidly, and so most options will require a lot of configuration, which IMO is not suitable for a declarative approach. * It's hard to debug, eventually you will need to dive into the code at some point. * Extending the library would mean touching the code anyways. One major difference here, is that Ludwig tries to expose one interface, that for instance users without python knowledge can use, and inside a company, some machine learning engineers can extend the tool and do support by debugging other users use-cases. I think at some point such approach can be useful for a very limited mature use-cases. [1]: https://github.com/polyaxon/polyaxon-lib/blob/master/examples/configs_examples/yaml_configs/alexnet_flower17.yml https://github.com/polyaxon/polyaxon-lib/blob/master/example...

w4nderlust 8y ago
Your configuration file looks really good! Why did you stop pursuing it? To your points: * That may be true, but I think it also depends on the level of abstraction. Fon instance in Ludwig the encoder, although configurable, are pretty monolitic. It's a trade off with flexibility, in Caffe or in your configuration file you are super flexible in specifying each single operation, while in Ludwig that is abstracted from you, but the advantage is that, as long as you trust the encoder implementation, those encoders mimic papers / state of the art models and require much less configuration to be written in order to run (in many cases if you are happy with the default parameters you don't have to configure them at all). So if the field moves and a new text encoder comes in, one can easily add a new encoder to Ludwig too. It's a dangerous game to play catch up, but hopefully releasing it as open source and making it easy to extend may help spontaneous contributions from the community. * debugging is an issue, that is true, I answered to another post about that, but again, it's a matter of tradeoffs. Debugging is kind of a nightmare in SQL too for instance, or is in TensorFlow (even if the tfdbg improved things a bit). * extending requires coding, that is also true, but if you have an idea for a new encoder for instance, all you have to do is implement a function that takes a tensor of rank k ad input and output a tensor of rank z as output, and all the rest (preprocessing, training loop, distributed training, etc.) comes for free, which kind of a nice value proposition imho and let you focus on the model rather than all the rest. Thanks for the interesting discussion!

by mmq 8y ago

w4nderlust 8y ago

Your configuration file looks really good! Why did you stop pursuing it? To your points: * That may be true, but I think it also depends on the level of abstraction. Fon instance in Ludwig the encoder, although configurable, are pretty monolitic. It's a trade off with flexibility, in Caffe or in your configuration file you are super flexible in specifying each single operation, while in Ludwig that is abstracted from you, but the advantage is that, as long as you trust the encoder implementation, those encoders mimic papers / state of the art models and require much less configuration to be written in order to run (in many cases if you are happy with the default parameters you don't have to configure them at all). So if the field moves and a new text encoder comes in, one can easily add a new encoder to Ludwig too. It's a dangerous game to play catch up, but hopefully releasing it as open source and making it easy to extend may help spontaneous contributions from the community. * debugging is an issue, that is true, I answered to another post about that, but again, it's a matter of tradeoffs. Debugging is kind of a nightmare in SQL too for instance, or is in TensorFlow (even if the tfdbg improved things a bit). * extending requires coding, that is also true, but if you have an idea for a new encoder for instance, all you have to do is implement a function that takes a tensor of rank k ad input and output a tensor of rank z as output, and all the rest (preprocessing, training loop, distributed training, etc.) comes for free, which kind of a nice value proposition imho and let you focus on the model rather than all the rest. Thanks for the interesting discussion!