5 ms·
Great work -- very exciting to see a Rust ML framework! Dist-Belief (TensorFlow's predecessor) was oriented around layers (as Leaf is) but TensorFlow (like The
by rryan 11y ago
Great work -- very exciting to see a Rust ML framework!
Dist-Belief (TensorFlow's predecessor) was oriented around layers (as Leaf is) but TensorFlow (like Theano) is graph/flow-based rather than layer-based. I think in particular the layer-based approach makes it awkward to represent more complicated models since not every model can be decomposed into backward/forward passes.
Do you worry that by adopting a layer-based paradigm you are boxing yourself in? A graph-based approach does seem more flexible (and distributable across machines).
- hobofan 11y agoI am very optimistic concerning our plans to make Leaf work in a distributed fashion. It's not quite finalized yet, but the rough outline consists of abstracting other parts of the Network away as a special type of layer and using capnproto + zeromq for transportation. Sadly I haven't come across a good implementation of model parallelism to learn from, so that might become a bit tricky. I think there is still a lot of room for improvement for layer-based networks, but even if the graph-based approach will become more prelevant I think the Rust trait system gives us great tools to migrate and reuse a lot from our layer based approach.
- DjvuLee 11y agoI just curious about is there any good Actor model library in Rust, just like Akka for Scala/Java, so there is a better model for transportation.