8 ms·
Gorgonia: a library like Theano or TensorFlow, mainly written in Go
- thr0waway1239 10y agoI think Microsoft should take some initiatives to help bring more of these projects into the .NET ecosystem. C# is a fantastic language, and is generally missing out on a lot of this ML stuff. (Although maybe it conflicts with their desire to profit from it?)
- dkarapetyan 10y agoThere are no conflicts. The latest strategy across the board is to give away everything for free and then run it on the cloud. In that sense microsoft has a lot to gain actually by offering a solution in .net and then making it dead simple to deploy to azure. Software and hardware are now a commodity and all the big players know it. So the only way to make money is to add value some other way like making it dead simple to deploy tensor flow graphs in gce.
- mark_l_watson 10y agoI agree with you on Microsofts/Googles/Amazon's cloud business strategy. In spite of the apparent good profits Amazon is making, I question how profitable this business will be long term. I expect large gross sales and small profits because of intense competition.
- chewxy 10y agoMicrosoft has CNTK - https://github.com/Microsoft/CNTK https://github.com/Microsoft/CNTK Which surprisingly is one of the few I didn't refer to when implementing Gorgonia
- pjmlp 10y agoBesides the CNTK referred on a sibling comment, I imagine Alea GPU (http://quantalea.com/ http://quantalea.com/) could be used for such purposes.
- gtani 10y agoFixed up GPU work I haven't thought about using golang to feed data to a GTX 1080 or similar, but before i do that, i'd like some info on the above, and a quick google shows that there may not be a complete wrapper for the Cuda Driver or Runtime API's like pyCuda.
- chewxy 10y agoHi, So I wrote my own CUDA wrapper. Turns out there was a bug. I'm not sure where and how, but it worked for the earlier versions of Gorgonia but not this release. You may also note that a bunch of assembly for the float32 tensor are missing... they seem to be affected by the same bug, so what I'm doing now is I'm planning to rewrite from scratch
- gtani 10y agoDid you reach out to Nvidia? It seems like they should be willing to help with putting out tested/documented bindings for more languagues. Also, where in the repo is your wrapper? (I don't have enough knowledge to debug s.t. like that, just curious how they work)
- marmaduke 10y agoWhy not OpenCL? With AVX2, it's quite a performer even on Xeons.
- 10y ago
- tristanz 10y agoThe tensorflow team is working on a go wrapper as well: https://github.com/tensorflow/tensorflow/issues/10#issuecomment-245687757 https://github.com/tensorflow/tensorflow/issues/10#issuecomm...
- chewxy 10y agoHi, I wrote Gorgonia: https://github.com/chewxy/gorgonia https://github.com/chewxy/gorgonia If there are any questions I'm available to answer them
- IshKebab 10y agoHow does it handle recurrent connections. I really like Microsoft's approach with CNTK - they use a DSL called 'BrainScript' to define the network, and it's basically a computational network (hence then name), but there are special operations like x = PastValue(y) and y = FutureValue(x) for forward and backward recurrent connections. It even automatically only unrolls the recurrent part of the network. In contrast all of the other NN systems I've used have premade layers like LSTM and GRU, and to be honest their methods are so hacky I haven't really been able to work out how to do custom RNNs in any other system. How does Gorgonia handle it?
- chewxy 10y agoThe trick, IMO, is to manually unroll them. So what I do is something like this: func (m *model) costFn() (cost *Node, n int, err error) { var prev *recurrentOutput for ...{ // build your recurrent graph here } } func (m *model) runOne() (err error) { cost, n err := costFn() g := m.g.SubgraphRoots(cost) machine := NewLispMachine(g) if err = machine.RunAll(); err != nil{ return } } The LispMachine type allows for rapid prototyping. Then once that's all done and stuff, you would probably have figured out what the size of your graph is and you can then create a TapeMachine that have the sizes you defined (though I have removed all the bits of the TapeMachine that made it turing complete, so it may be a bit hard to do that).
- PeterisP 10y agoManual unrolling fails as soon as the number of repetitions is not fixed (e.g. NLP tasks where it matches number of tokens in a sentence); so you want the looping to happen after the graph has been made, preferably on the GPU.
- thinxer 10y agoI don't see the point of writing a machine learning package in Go. Go is designed for system programming, which is the opposite of scripting. In scientific computing, interactivity is an important feature. Softwares like MATLAB and IPython(now Jupyter) become popular because of this. Go doesn't have a nice REPL like IPython, and it never will. Besides, Go doesn't allow operator overload, which leads to verbose programs. Compare `c = a @ b` with `c := Must(Mul(a, b))`, the former Python statement is much cleaner than the Go one. Usually, researchers use MATLAB, R, and Python for prototyping and training. The final model will be put into production by using something like TensorFlow Serving or by hand written C++ for performance. Even if the final product uses Go for serving, the Go program mostly runs the model via cgo or RPC. Write in C/C++, and wrap the library in Python and Go is a much saner option. TensorFlow and MXNet are both good examples.
- chewxy 10y agoI agree with you in general. However, I contend that people are lazy. Part of the reason why I wrote Gorgonia was because I spent waay too long trying to deploy theano on the cloud (this was circa 2 years ago). Also, thanks for informing me about MXNet. I've never seen that before. Kinda cool.
- pjmlp 10y ago> I don't see the point of writing a machine learning package in Go. If nothing else, to prove a point that there are solid alternatives to C and C++, in memory safe languages.
- solidsnack9000 10y agoIf the point is feasibility, Java has shown us that for a long time...unless we consider the null pointer evidence of something not memory safe.
- pjmlp 10y agoNot only feasibility, but having a GC enabled programming language with AOT compilation to native code (until Java 10, you will need to buy such compilers as the FOSS aren't as good). After all the OP mentioned that only C and C++ were valid alternatives, hence why it is so important to implement such tools in other safer languages. If people don't do it, as usual there will be a myth that only those languages are able to produce such type of applications. Similar to the myth that C was the first systems programming language, when many of us were using something else before UNIX got widespread in the industry.
- MarkAntle 10y agoThank you for sharing this information.It was informative a s usual. Actually I looked up it on <a href="https://en.wikipedia.org">wiki</a> https://en.wikipedia.org">wiki</a>.
- MarkAntle 10y agoThank you for sharing this information.It was informative a s usual. Actually I looked up it on [url="https://en.wikipedia.org"]wikipedia[/url] https://en.wikipedia.org"]wikipedia[/url]
- MarkAntle 10y agoThank you for sharing this information.It was informative a s usual. Actually I looked up it on [url=https://en.wikipedia.org]wikipedia[/url] https://en.wikipedia.org]wikipedia[/url]
- dwhitena 10y agoThis is awesome! Great to see so much momentum on the Go data science front (see https://github.com/gopherds/resources/blob/master/tooling/README.md https://github.com/gopherds/resources/blob/master/tooling/RE... for more). Go provides a great integrity and deployment story for data scientists, and contributions like this are well worth the effort!
- stanislavb 10y agoYup, amazing work. I hope it gets traction. In fact, there's a lot happening in the "Machine Learning" front of Go https://go.libhunt.com/categories/507-machine-learning https://go.libhunt.com/categories/507-machine-learning
- perfmode 10y agoNumber one on my wish list for Go 2 is the ability to go from A to B: A = if xw, err = Mul(x,w); err != nil { log.Fatal(err) } if xwpb, err = Add(xw, b); err != nil { log.Fatal(err) } if prob, err = Sigmoid(xwpb); err != nil { log.Fatal(err) } B = probs := Sigmoid(Add(Mul(x,w), b))