9 ms·
Yanni – An artificial neural network for Erlang
- JamesUtah07 9y agoLooking forward to a Elixir wrapper for this.
- skrebbel 9y agoOr just call it directly.
- pselbert 9y agoThere isn't any need for that. Hex is becoming the de facto package manager for the Erlang ecosystem. You can easily install this as a dependency and call it from your Elixir code.
- JamesUtah07 9y agoOh sweet, I wasn't aware of this.
- toisanji 9y agoI wonder if you could make computation run faster by configuring the network to cut up the work into gpu vs non gpu work and have each node efficiently process the work and then have the results reassembled.
- fizixer 9y ago> When one is coming from Erlang, and sees an artificial neural network diagram/topology, one cannot help but see the 1-to-1 mapping between the Erlang concurrency model, and neural networks; it’s almost eerie. Is that some kind of a joke? A quick google image search for 'Erlang concurrency models' [1] reveals nothing that has an eerie similarity to a NN. The only thing noteworthy is that there are graph diagrams spread here and there. (Please tell me it is not true that you are clueless about a graph being the most fundamental construct in computer science, something that would pop up in pretty much every CS topic in one form or the other.) [1] https://www.google.com/search?q=Erlang+concurrency+model&tbm=isch https://www.google.com/search?q=Erlang+concurrency+model&tbm...
- JamesUtah07 9y agoIt's a amazing how minimal prior knowledge and a quick google search totally trumps someone who's spent years working in a field.
- fizixer 9y agoEver heard of the term 'argument from authority' [1] ? [1] https://en.wikipedia.org/wiki/Argument_from_authority https://en.wikipedia.org/wiki/Argument_from_authority P.S.: Ignoring the blatant fact that there is not a shred of authority apparent in the hipster, grey-font, less-than-500-word, author-less, zero-references, blogpost.
- walterstucco 9y agoAre you serious?
- deleted 9y ago[deleted]
- Cieplak 9y agoMy favorite ML book is still the Handbook of Neuroevolution Through Erlang. A bit pricey but you can borrow my copy if you're in SoMa.
- oomkiller 9y agoI have been thinking about picking this one up for a while, do you think it would be helpful for learning ML in general, or should I start somewhere else?
- Cieplak 9y agoIt's definitely a very niche book; I would definitely focus on statistics and Bayesian methods before delving into genetic algorithms for evolving neural networks.
- salimmadjd 9y agoInteresting. Can you provide more details please. Why do you recommend this book? What sets it a part from other books? What's the best thing you like about? What is your own background (skills, education)? Only asking to make a buying decision given the price.
- Cieplak 9y agoIn many ways it's a companion to this codebase: https://github.com/CorticalComputer/DXNN2 https://github.com/CorticalComputer/DXNN2
- deleted 9y ago[deleted]
- lgleason 9y agoThey should use this as the photo for the website ;) https://upload.wikimedia.org/wikipedia/en/b/b1/Nightbird_%28Yanni_album%29.jpg https://upload.wikimedia.org/wikipedia/en/b/b1/Nightbird_%28...
- fenollp 9y agoIt uses `array` (somewhat mutable Erlang structure) and NOTP (no idea how it makes code "zippy" and the repo [1] does not explain anything... it seems to be bypassing the normal way modules are loaded?). I am unsure why anyone would use Erlang for number crunching. Training neural nets is basically just multiplying big matrices. I was hoping this project would come up with an interesting approach (how about using SIMD on the binary comprehensions that can use it? now that would be cool) but performance / memory usage does not seem to be looked at here. It is naive / uneducated to think that "Erlang’s multi-core support" + distributedness will enable many things for you. How does the VM scale on 32, 64 threads? Have you tried making a cluster of 50+ VMs? Unfortunately Erlang Solutions Ltd.'s marketing has hyped many. I am not against projects like these, I am just looking for reasons behind the choices made. [1]: https://bitbucket.org/nato/notp/src https://bitbucket.org/nato/notp/src
- JamesUtah07 9y agoIn erlang you an interop with c++ libraries using NIFs so maybe the author down line will move to heavy matrix operations to a NIF
- fenollp 9y agoIn Erlang you cannot run non-BEAM code for more than ~10ms or your VM will crash. GEMM will be hard to use this way...
- striking 9y agoYou can use the dirty scheduler feature in 17.10 to get around the limit (because it creates unmanaged threads). See https://github.com/vinoski/bitwise https://github.com/vinoski/bitwise
- jerf 9y agoBut honestly, you're better off with just Don't Do That. It's not what Erlang is meant to do. It's a suitable language to coordinate number crunching if that floats your boat, but it is not a suitable language for the actual crunching. Some people still seem to get very upset when someone proposes that some langauge is not suitable for some use, but there aren't any languages that are the best for everything. The languages in which I would want to write heavy-duty number crunching code will be nowhere near as good as Erlang at writing high-concurrency, high-reliability server code. Also, to avoid a second post, contrary to apparently popular belief it is not possible to just make up for slow code by bringing in a lot of CPUs. Number crunching in Erlang is probably a bare minimum of 50x slower than an optimized implementation, it could easily be 500x slower if it can't use SIMD, and could be... well... more-or-less arbitrarily slower if you're trying to use Erlang to do something a GPU ought to be doing. 5-7 orders of magnitude are not out of the question there. You can't make up even the 50x delta very easily by just throwing more processors at the problem, let alone those bigger numbers.
- artur_makly 9y agoand it also plays a sexy clarinet??
- deleted 9y ago[deleted]
- slezakattack 9y agoThis is a neat idea but it would be great if there was a bit more substance to the post. Do we have any performance benchmarks? Why would I consider it a strong contender? Stating "multi-core support" to me is not necessarily scaling. I'm in no way an expert, but I work in Erlang in my day job and just glancing at the repo, this solution can't possibly be performant. A) Erlang is slow at math. B) Arrays don't have O(1) access(ETS tables might be able to help with this). C) You can't scale this solution with more Erlang nodes(without some additional work). I really like Erlang and want to evangelize it but I don't think this is a good way of doing it. I only see this as a neat toy but not a selling point for using Erlang.. As a side note: I noticed the repo has a feature note about adding NIF's for performance bottlenecks (native C code for Erlang to talk to). If you end up writing C code, then what are you gaining from Erlang?
- Ono-Sendai 9y agoIf you use one Erlang node or whatever per neuron, it's gonna be slow as f*ck.
- pselbert 9y agoIn Erlang you usually run one "node" per machine, though you can call between them with transparent RPC. This library uses one "process" per neuron for concurrency. Processes are extremely lightweight and entirely unrelated to system processes or threads.
- Ono-Sendai 9y agoRunning one process per neuron is going to be extremely slow.
- btbuildem 9y agoErlang's inter-process messaging is ridiculously optimized. Processes are extremely low-weight, it costs approximately nothing to start and stop them. This is one of the core strength of Erlang. Running one process per neuron would actually be a very efficient way to do it.
- mononcqc 9y agoIt should still be pretty slow compared to just doing the neural net propagation as matrix operations
- ramchip 9y agoI'm quite sure that would be a grossly inefficient approach. Sending a message is expensive in Erlang, less so than in other languages, but it's still very large compared to a few math operations. It's a common mistake to use processes to represent objects [1]. The recent article from Discord [2] also mentioned "Sending messages between Erlang processes was not as cheap as we expected, and the reduction cost — Erlang unit of work used for process scheduling — was also quite high. We found that the wall clock time of a single send/2 call could range from 30μs to 70us due to Erlang de-scheduling the calling process." [1] http://theerlangelist.com/article/spawn_or_not http://theerlangelist.com/article/spawn_or_not [2] https://blog.discordapp.com/scaling-elixir-f9b8e1e7c29b https://blog.discordapp.com/scaling-elixir-f9b8e1e7c29b
- bitmapbrother 9y agoI'm not sure how I feel about a ML project using the name of one of the greatest instrumentalists in history.
- mulmen 9y agoYeah, that's more of a web framework thing.
- brandonhsiao 9y agoWhat is the 1-to-1 mapping between the Erlang concurrency model and neural networks?
- btbuildem 9y agoEach node in a NN can be represented as an Erlang actor / process. They communicate via messages.
- deleted 9y ago[deleted]
- hartator 9y agoIsn't Erlang good for concurrency but bad at math performance?
- tormeh 9y agoYes, as many have pointed out. It's ideal for coordinating a cluster doing machine learning, but ANN code itself should not be in Erlang.
- digitalzombie 9y ago> It's ideal for coordinating a cluster doing machine learning, but ANN code itself should not be in Erlang. It's funny that's what they did with disco (http://discoproject.org/ http://discoproject.org/). Python + Erlang = Hadoop 1.0 like.
- buttershakes 9y agoI really wish there was a way to do inline optimized code, i.e a gen_server that transparently wraps another language without having to get into nif / external servers. Basically an abstraction that hides all of that cruft and builds optimized gen_servers for doing number crunching or heavy processing. Maybe I'm just being lazy. :)
- pmarreck 9y agoerlexec (or its elixir wrapper) might be worth a try: https://github.com/saleyn/erlexec https://github.com/saleyn/erlexec
- buttershakes 9y agoI've used this, it's an awesome project, but it's basically just pipes all the way down. I'm thinking of something closer to a NIF. I think Saleyn's c++ node code might be the closest thing for a lower level language.
- nukifw 9y agoNice work ! At Dernier Cri, we begon a similar work : https://github.com/derniercri/multilayer-perceptron https://github.com/derniercri/multilayer-perceptron But we were far less advanced than you!
- mehh 9y agoEach neutron as an individual process is indeed interesting but no so much if your using backprop for the training algorithm as it doesn't really fit the paradigm.
- brian_herman 9y agoWith the latest revision I keep on getting this error: src/yanni_trainer.erl:78: type array() undefined It was introduced in revision 20. 3 files updated, 0 files merged, 1 files removed, 0 files unresolved [bherman@archy yanni]$ hg update 19 [bherman@archy yanni]$ make rm -f notp notp.boot rm -fr ebin mkdir ebin erlc -o ebin src/*.erl deps/*/src/*.erl src/yanni_lib.erl:77: Warning: random:uniform/0: the 'random' module is deprecated; use the 'rand' module instead src/yanni_trainer.erl:112: Warning: random:uniform/0: the 'random' module is deprecated; use the 'rand' module instead deps/*/src/*.erl: no such file or directory make: *** [Makefile:6: default] Error 1 [bherman@archy yanni]$ hg update 20 3 files updated, 0 files merged, 0 files removed, 0 files unresolved [bherman@archy yanni]$ make rm -f notp notp.boot rm -fr ebin mkdir ebin erlc -o ebin src/*.erl deps/*/src/*.erl src/yanni_lib.erl:77: Warning: random:uniform/0: the 'random' module is deprecated; use the 'rand' module instead src/yanni_trainer.erl:78: type array() undefined src/yanni_trainer.erl:118: Warning: random:uniform/0: the 'random' module is deprecated; use the 'rand' module instead make: *** [Makefile:6: default] Error 1 [bherman@archy yanni]$ make edit: formatting
- ramchip 9y agoI wonder if the author is running an ancient Erlang? The code looks very old school, with the lack of maps, no rebar, deprecated functions, etc. From Erlang release notes: The pre-defined types array/0, dict/0, digraph/0, gb_set/0, gb_tree/0, queue/0, set/0, and tid/0 have been deprecated. They will be removed in Erlang/OTP 18.0. Instead the types array:array/0, dict:dict/0, digraph:graph/0, gb_set:set/0, gb_tree:tree/0, queue:queue/0, sets:set/0, and ets:tid/0 can be used. (Note: it has always been necessary to use ets:tid/0.)