4 ms·

Flare: Clojure Dynamic Neural Net Library

148 points by ghosthamlet 9y ago

jgh 9y ago
Very cool. As someone who wants to begin learning how to use ML and who enjoys clojure as a hobby language...this is great. Gives me an excuse to use Clojure more :) I hope you decide to continue working on it!
- aria 9y ago
  I'll try! I've got a full time management job so this was a fun way to code again. The library is pretty small and I think easy to follow if you want to learn
deleted 9y ago
[deleted]
nickbauman 9y ago
This is for Dynamic Neural Nets, which do not need back propagation. The benchmarks are amazing.
- stochastic_monk 9y ago
  What is the optimization algorithm for these if not backpropagation? This page does mention the use of hyperparameters for optimization and compares itself to an adaptive stochastic gradient descent method, which makes me think that it is using backpropagation.
  - nickbauman 9y ago
    Ber ber ber, I think you're right. I was assuming a Cascade-Correlation DNN.
  - aria 9y ago
    Definitely doing the standard back-prop algorithm. Check out the code, it's pretty straightforward and the whole lib is about 2,500 lines
visarga 9y ago
Needs more GPU.
- nickbauman 9y ago
  True, I think it's on the "wish list", no?
- aria 9y ago
  Author here. This is coming up. Will do OpenCL first and then CUDA.
aria 9y ago
Author here, happy to answer any questions!
- ghosthamlet 9y ago
  Thinks for your great lib in both great Clojure and ML. Do you know whether this Clojure lib can do Dynamic net: https://github.com/thinktopic/cortex https://github.com/thinktopic/cortex? you didn't compare it in your blog.
  - aria 9y ago
    Yes this does dynamic nets, the model is the same as PyTorch, you build a graph for each input(s). I think Cortex is in the Keras-style of model (only layer abstractions and not "dynamic" in the sense we talk about neural nets) and isn't dynamic, but I could be missing something. The examples seem to all be in that mold: https://github.com/thinktopic/cortex/blob/master/examples/catsdogs-classification/src/catsdogs/training.clj https://github.com/thinktopic/cortex/blob/master/examples/ca...
thom 9y ago
This made me chuckle. If anybody remembers the dim and distant past of various singularitarian mailing lists on the internet, you might remember Eliezer Yudkowsky's theoretical 'annotative' language Flare[1]. It was one of the early steps on the (what was then quite straight in some people's minds) line to AGI. As for this, possibly worthwhile for the authors to look into Dragan Djuric's work, he's done some really fast matrix stuff on Clojure[2]. 1: http://yudkowsky.net/obsolete/plan.html#tech_comp_flare http://yudkowsky.net/obsolete/plan.html#tech_comp_flare 2: https://github.com/uncomplicate/neanderthal https://github.com/uncomplicate/neanderthal
- thom 9y ago
  And I read the article and see Neantherthal's being used (stupidly thought it wasn't given a comment downthread).

148 points by ghosthamlet 9y ago

jgh 9y ago

Very cool. As someone who wants to begin learning how to use ML and who enjoys clojure as a hobby language...this is great. Gives me an excuse to use Clojure more :) I hope you decide to continue working on it!

aria 9y ago
I'll try! I've got a full time management job so this was a fun way to code again. The library is pretty small and I think easy to follow if you want to learn

deleted 9y ago

[deleted]

nickbauman 9y ago

This is for Dynamic Neural Nets, which do not need back propagation. The benchmarks are amazing.

stochastic_monk 9y ago
What is the optimization algorithm for these if not backpropagation? This page does mention the use of hyperparameters for optimization and compares itself to an adaptive stochastic gradient descent method, which makes me think that it is using backpropagation.
- nickbauman 9y ago
  Ber ber ber, I think you're right. I was assuming a Cascade-Correlation DNN.
- aria 9y ago
  Definitely doing the standard back-prop algorithm. Check out the code, it's pretty straightforward and the whole lib is about 2,500 lines

visarga 9y ago

Needs more GPU.

nickbauman 9y ago
True, I think it's on the "wish list", no?
aria 9y ago
Author here. This is coming up. Will do OpenCL first and then CUDA.

aria 9y ago

Author here, happy to answer any questions!

ghosthamlet 9y ago
Thinks for your great lib in both great Clojure and ML. Do you know whether this Clojure lib can do Dynamic net: https://github.com/thinktopic/cortex https://github.com/thinktopic/cortex? you didn't compare it in your blog.
- aria 9y ago
  Yes this does dynamic nets, the model is the same as PyTorch, you build a graph for each input(s). I think Cortex is in the Keras-style of model (only layer abstractions and not "dynamic" in the sense we talk about neural nets) and isn't dynamic, but I could be missing something. The examples seem to all be in that mold: https://github.com/thinktopic/cortex/blob/master/examples/catsdogs-classification/src/catsdogs/training.clj https://github.com/thinktopic/cortex/blob/master/examples/ca...

thom 9y ago

This made me chuckle. If anybody remembers the dim and distant past of various singularitarian mailing lists on the internet, you might remember Eliezer Yudkowsky's theoretical 'annotative' language Flare[1]. It was one of the early steps on the (what was then quite straight in some people's minds) line to AGI. As for this, possibly worthwhile for the authors to look into Dragan Djuric's work, he's done some really fast matrix stuff on Clojure[2]. 1: http://yudkowsky.net/obsolete/plan.html#tech_comp_flare http://yudkowsky.net/obsolete/plan.html#tech_comp_flare 2: https://github.com/uncomplicate/neanderthal https://github.com/uncomplicate/neanderthal

thom 9y ago
And I read the article and see Neantherthal's being used (stupidly thought it wasn't given a comment downthread).