4 ms·
Great read. I have 30 years combined experience in symbolic AI and neural networks and my current day job is managing a deep learning team. I could not agree mo
by mark_l_watson 8y ago
Great read. I have 30 years combined experience in symbolic AI and neural networks and my current day job is managing a deep learning team. I could not agree more with: “”I think it is far more likely that the two — deep learning and symbol-manipulation-will co-exist, with deep learning handling many aspects of perceptual classification, but symbol-manipulation playing a vital role in reasoning about abstract knowledge.””
My personal time projects are mostly combining symbolic AI and deep learning, but I am still trying to find a non-Python solution, including Haskell TensorFlow bindings, Armed Bear Common Lisp with DL4J, and exporting trained Keras models to a Racket environment - all plausible hacking environments but none feel ‘just right.’ If you are working on the same ideas please get in touch with me.
- peteradio 8y agoCan you hint the problem with a python based solution? What doesn't feel right about it.
- eli_gottlieb 8y agoPure Python is slow as a snail compared to both the underlying GPU code that has to run in deep learning applications, and the tuned, compiled code you would use for symbolic reasoning.
- peteradio 8y agoNaturally, I guess I thought there was something syntax related or otherwise based on the way he said "doesn't feel right"
- mark_l_watson 8y agoPython is great for short scripts and programs, but I don’t like it for large systems.
- amelius 8y agoYes, I found it is easy to run into memory or performance problems when code size becomes larger. Especially when the system is multi-threaded.
- ChrisFoster 8y agoI've just been trying out a few deep learning frameworks recently, and found the julia framework Flux to be enormously fun, flexible and generally just extremely hackable. If you like lisp you might also find it a pleasing language - even though it doesn't look it on the surface, there's a lot of lispy inspiration hiding underneath the skin.
- mark_l_watson 8y agoThanks Chris, I will look at Flux. EDIT: make that a double thanks, Flux looks great: concise and expressive.
- dreamcompiler 8y agoAmen. I'd love to find a way to access Tensorflow directly from a native compiled Common Lisp like CCL or SBCL. This would be ideal for combining symbolic AI with DL.
- mark_l_watson 8y agoI would like the same. I like TensorFlow with Keras much more than using DeepLearning4j, but DL4J is callable from Armed Bear Common Lisp. I have simple examples in my book that you can read free online: https://leanpub.com/lovinglisp https://leanpub.com/lovinglisp Unfortunately the example in the book is a little weak because I wrote the deep learning code in Java and just called my Java code from Common Lisp. I should modify the example to build models, train, and evaluate using the DL4J APIs directly from Lisp. EDIT: in the book example, I just call train and evaluate entry points in my Java code from Lisp so I can train new models from Lisp but can’t create new models with new architectures without modifying the Java code. I really should change that.
- agibsonccc 8y agoHi Mark, I want to correct one thing. My company is behind dl4j. We are also the second largest contributor to keras. We have near perfect model import (with keras contributors maintaining it). Dl4j also has TF import and a native graph runner: https://github.com/deeplearning4j/dl4j-examples/tree/master/tf-import-examples https://github.com/deeplearning4j/dl4j-examples/tree/master/... Java itself tends to be fairly verbose I know, but not only are there wrappers but for inference if you want to use python for basics and then us for inference. We enable several ways to do that. Folks tend to oversensationlize these "us vs them" framework "fights". Rather than do that, why not just use what's good for each use case? Python for training, java for deployment? Beyond that, you could also consider building your own wrapper maybe? In clojure, there's jutsu.ai for example: https://github.com/hswick/jutsu.ai https://github.com/hswick/jutsu.ai
- mark_l_watson 8y agoThanks for the info! Sorry if I sounded like I was knocking DL4J: I have used it in examples for two books I have written. I use Keras at work so that is why I prefer it - I am very used to Keras, and to a lesser degree TensorFlow. Just tonight I was experimenting with jutsu.ai, and your advice of training on Keras and running on DL4J is good advice. I hacked up a little code to import Keras weights into Racket, but I can only process dense layers, and not RNN or complex architectures.
- huahaiy 8y agoHere is an attempt in Clojure https://m.youtube.com/watch?v=phA4bMjKvCY https://m.youtube.com/watch?v=phA4bMjKvCY
- mark_l_watson 8y agoThanks I am watching this right now. EDIT: thank you so much, great talk! Even though the Juji system does not directly link TensorFlow models into Clojure, the architecture of using Kafka to manage low level black box model calls (they treat trained models as functions as I do in my Racket hacks) is a good idea. I also like the rule/template language they designed. Great work.
- mark_l_watson 8y agothis also looks good: https://github.com/hswick/jutsu.ai https://github.com/hswick/jutsu.ai