5 ms·

Show HN: Juggernaut – Experimental Neural Network in Rust

54 points by afshinmeh 9y ago

amelius 9y ago
I'm a bit confused. How useful is this if: - Rust cannot compile to the GPU - Neural network programs are usually not large and therefore do not need the type safety that Rust offers - All cool neural network research is done on Keras/Tensorflow, so developing on that platform gives access to new algorithms automatically - Scripting in Python is virtually at least as fast as anything else because you can use Tensorflow which uses the GPU
- mi_lk 9y ago
  > All cool neural network research is done on Keras/Tensorflow Add in PyTorch if you are really talking about nn research.
- yorwba 9y ago
  > - Neural network programs are usually not large and therefore do not need the type safety that Rust offers TensorFlow Fold [1] uses static type checking at model compilation time, presumably because it is meant to consume structured data, and needs to know what the opaque tensors it passes around are supposed to represent. I don't fully comprehend that type system, and debugging type errors was annoying (huge stacktrace full of passing type info around), but after I managed to imitate it well enough to hack up my own blocks, I didn't investigate further. A language-level type system like Rust's would probably have made things easier for me. EDIT: I just remembered that there's also Leaf [2], which claims to be among the faster neural network libraries (although their comparison is a bit outdated). Evidently there are people seriously pursuing machine learning in Rust (still uses CUDA for GPU acceleration, though). [1] https://github.com/tensorflow/fold https://github.com/tensorflow/fold [2] https://github.com/autumnai/leaf https://github.com/autumnai/leaf
  - hobofan 9y ago
    Author of Leaf here. Yes, Leaf has been dead for more then a year. There recently has been an effort by some community members to pick up development of Leaf under a new name: https://github.com/spearow https://github.com/spearow
- pjmlp 9y ago
  > Rust cannot compile to the GPU Rust might not be able to compile to the GPU today, but any language can target the GPU, it is only a matter of tooling support.
- hobofan 9y ago
  > All cool neural network research is done on Keras/Tensorflow, so developing on that platform gives access to new algorithms automatically I disagree with that assessment. The implementations that come with research are rarely of a quality that they can just be picked up and used. Yes, the community is bigger with Python frameworks, which gives you quicker access to the new stuff, but the effort is about the same as writing a new implementation in another language.
  - liuliu 9y ago
    It doesn't matter. Most of the time, especially with Tensorflow, you will have a static computation graph somewhere. Nothing (well, implementation-defined) prevent you compile that into CoreML / Caffe / Caffe2 or just Tensorflow lean version to get deployed. PyTorch is a bit different because it supports dynamic graph.
  - danieldk 9y ago
    which gives you quicker access to the new stuff, but the effort is about the same as writing a new implementation in another language Not really. You can just serialize a Tensorflow graph, freeze the variables as constants, and then load and run the graph using Tensorflow's small and convenient C API. There are bindings to the C API for multiple languages (e.g. Rust and Go). This is how I use neural networks in my Rust (and formerly Go) programs: I just build and train the graph in Python and then use it in Rust. Newer versions of Tensorflow also have the XLA compiler, which compiles a graph to executable code that you can link directly into an application. I haven't tried this approach yet, since the C API serves me well, but it looks promising.
- kersny 9y ago
  #1 is not strictly true, although it's not necessarily useful yet: https://github.com/japaric/nvptx#nvptx https://github.com/japaric/nvptx#nvptx
yorwba 9y ago
I'm confused by several of the API choices in the example. Why is the training set part of the network? I would have expected it to be a parameter to the train() function. Same for the activation function, shouldn't this be a property of the layer rather than fixed for the network as a whole? I get that this is just in the early stages and more for learning than anything else, but it doesn't seem very well thought-out IMO.
- afshinmeh 9y ago
  Thanks for your suggestions, yorwba. I do see what you mean about the API and I will work on this. For now, if you want to train a model, you need to pass data to the NN struct not train method but as you mentioned, probably it is better to pass it to the train method. Sorry for the confusion and thanks for your comment!
shadowmint 9y ago
There's nothing to see here. Trivial NN implementations are a dime a dozen, and this one is no different. It's just a partial work in progress; it's not a 'Show HN'; it's just a few hundred lines of toy code. ...and that's the same feedback it got on /r/rust last week. I don't see why it's turned up here now. (Just as a baseline, at this point, if you can't use your NN implementation to at least do a basic classifier on MNIST, its probably not worth showing people)
- overcast 9y ago
  Basically because it's ticked off all the buzzword bingo that works on HN. Unfortunately that's how it goes to front page.
  - afshinmeh 9y ago
    No not really no! I'm trying to gather some information and feedbacks about my project. I don't care (at this stage of the project) to be on the first page of HN or not, as long as I have some good feedbacks to improve it. Thanks anyway!
- afshinmeh 9y ago
  thanks for your suggestions, shadowmint. I tried to mention it is "experimental", not a production ready NN. I'm working on adding more features to this project but for sure, it is not production ready at all and is not a competitor of Tensorflow or Leaf.
  - shadowmint 9y ago
    Calling this an experimental neutral network is like me calling an opengl demo that renders a few flat shaded polygons an 'experimental 3d engine'. It's not. I'm sorry if that sounds harsh, but that's what you have. There's no point to showing off something this raw: all you'll get is negative feedback about the limited features it has and negative feedback from people who try to use it for something and find that you can't actually do anything with it. Just hold onto it until it can actually do something useful with it; and as I said in my previous post, I'm not talking some 'abstract' sort of useful thing. The MNIST dataset is the basic standard task for these frameworks; my advice is focus 100% on being able to classify that dataset, and then you'll have something to show.
bmh100 9y ago
What is your goal in writing this project?
blahman2 9y ago
Keep going! I really like being able to follow projects that start small, as opposed to 'here is my 10000 line toy project. That being said, you will probably get some flak, probably because of the insane amounts of rust evangelism people on hn have had to deal with

54 points by afshinmeh 9y ago

I'm a bit confused. How useful is this if: - Rust cannot compile to the GPU - Neural network programs are usually not large and therefore do not need the type safety that Rust offers - All cool neural network research is done on Keras/Tensorflow, so developing on that platform gives access to new algorithms automatically - Scripting in Python is virtually at least as fast as anything else because you can use Tensorflow which uses the GPU

mi_lk 9y ago
> All cool neural network research is done on Keras/Tensorflow Add in PyTorch if you are really talking about nn research.
yorwba 9y ago
> - Neural network programs are usually not large and therefore do not need the type safety that Rust offers TensorFlow Fold [1] uses static type checking at model compilation time, presumably because it is meant to consume structured data, and needs to know what the opaque tensors it passes around are supposed to represent. I don't fully comprehend that type system, and debugging type errors was annoying (huge stacktrace full of passing type info around), but after I managed to imitate it well enough to hack up my own blocks, I didn't investigate further. A language-level type system like Rust's would probably have made things easier for me. EDIT: I just remembered that there's also Leaf [2], which claims to be among the faster neural network libraries (although their comparison is a bit outdated). Evidently there are people seriously pursuing machine learning in Rust (still uses CUDA for GPU acceleration, though). [1] https://github.com/tensorflow/fold https://github.com/tensorflow/fold [2] https://github.com/autumnai/leaf https://github.com/autumnai/leaf
- hobofan 9y ago
  Author of Leaf here. Yes, Leaf has been dead for more then a year. There recently has been an effort by some community members to pick up development of Leaf under a new name: https://github.com/spearow https://github.com/spearow
pjmlp 9y ago
> Rust cannot compile to the GPU Rust might not be able to compile to the GPU today, but any language can target the GPU, it is only a matter of tooling support.
hobofan 9y ago
> All cool neural network research is done on Keras/Tensorflow, so developing on that platform gives access to new algorithms automatically I disagree with that assessment. The implementations that come with research are rarely of a quality that they can just be picked up and used. Yes, the community is bigger with Python frameworks, which gives you quicker access to the new stuff, but the effort is about the same as writing a new implementation in another language.
- liuliu 9y ago
  It doesn't matter. Most of the time, especially with Tensorflow, you will have a static computation graph somewhere. Nothing (well, implementation-defined) prevent you compile that into CoreML / Caffe / Caffe2 or just Tensorflow lean version to get deployed. PyTorch is a bit different because it supports dynamic graph.
- danieldk 9y ago
  which gives you quicker access to the new stuff, but the effort is about the same as writing a new implementation in another language Not really. You can just serialize a Tensorflow graph, freeze the variables as constants, and then load and run the graph using Tensorflow's small and convenient C API. There are bindings to the C API for multiple languages (e.g. Rust and Go). This is how I use neural networks in my Rust (and formerly Go) programs: I just build and train the graph in Python and then use it in Rust. Newer versions of Tensorflow also have the XLA compiler, which compiles a graph to executable code that you can link directly into an application. I haven't tried this approach yet, since the C API serves me well, but it looks promising.
kersny 9y ago
#1 is not strictly true, although it's not necessarily useful yet: https://github.com/japaric/nvptx#nvptx https://github.com/japaric/nvptx#nvptx

yorwba 9y ago

I'm confused by several of the API choices in the example. Why is the training set part of the network? I would have expected it to be a parameter to the train() function. Same for the activation function, shouldn't this be a property of the layer rather than fixed for the network as a whole? I get that this is just in the early stages and more for learning than anything else, but it doesn't seem very well thought-out IMO.

afshinmeh 9y ago
Thanks for your suggestions, yorwba. I do see what you mean about the API and I will work on this. For now, if you want to train a model, you need to pass data to the NN struct not train method but as you mentioned, probably it is better to pass it to the train method. Sorry for the confusion and thanks for your comment!

shadowmint 9y ago

There's nothing to see here. Trivial NN implementations are a dime a dozen, and this one is no different. It's just a partial work in progress; it's not a 'Show HN'; it's just a few hundred lines of toy code. ...and that's the same feedback it got on /r/rust last week. I don't see why it's turned up here now. (Just as a baseline, at this point, if you can't use your NN implementation to at least do a basic classifier on MNIST, its probably not worth showing people)

overcast 9y ago
Basically because it's ticked off all the buzzword bingo that works on HN. Unfortunately that's how it goes to front page.
- afshinmeh 9y ago
  No not really no! I'm trying to gather some information and feedbacks about my project. I don't care (at this stage of the project) to be on the first page of HN or not, as long as I have some good feedbacks to improve it. Thanks anyway!
afshinmeh 9y ago
thanks for your suggestions, shadowmint. I tried to mention it is "experimental", not a production ready NN. I'm working on adding more features to this project but for sure, it is not production ready at all and is not a competitor of Tensorflow or Leaf.
- shadowmint 9y ago
  Calling this an experimental neutral network is like me calling an opengl demo that renders a few flat shaded polygons an 'experimental 3d engine'. It's not. I'm sorry if that sounds harsh, but that's what you have. There's no point to showing off something this raw: all you'll get is negative feedback about the limited features it has and negative feedback from people who try to use it for something and find that you can't actually do anything with it. Just hold onto it until it can actually do something useful with it; and as I said in my previous post, I'm not talking some 'abstract' sort of useful thing. The MNIST dataset is the basic standard task for these frameworks; my advice is focus 100% on being able to classify that dataset, and then you'll have something to show.

bmh100 9y ago

What is your goal in writing this project?

blahman2 9y ago

Keep going! I really like being able to follow projects that start small, as opposed to 'here is my 10000 line toy project. That being said, you will probably get some flak, probably because of the insane amounts of rust evangelism people on hn have had to deal with