5 ms·

Let me chip in with some self-promotion. This book explains and executes every single line of code interactively, from low level operations to high-level netwo

Let me chip in with some self-promotion. This book explains and executes every single line of code interactively, from low level operations to high-level networks that do everything automatically. The code is built on the state of the art performance operations of oneDNN (Intel, CPU) and cuDNN (CUDA, GPU). Very concise readable and understandable by humans. https://aiprobook.com/deep-learning-for-programmers/ https://aiprobook.com/deep-learning-for-programmers/ Here's the open source library built throughout the book: https://github.com/uncomplicate/deep-diamond https://github.com/uncomplicate/deep-diamond Some chapters from the beginning of the book are available on my blog, as a tutorial series: https://dragan.rocks https://dragan.rocks

kyllo 6y ago
"Deep Learning in Clojure with Fewer Parentheses than Keras and Python" Love it! :D What better way to define a neural network in code than an S-expression?
- piokoch 6y ago
  I am not sure if I am that enthusiastic. The problem with Lisp is not a number of parenthesis but where they are and what is their role. In c-like languages parenthesis help parser compiler but they also help humans to read the code. In case of Lisp they are just for the sake of the parser. Let's look on the code: Python: model = Sequential() model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(28, 28, 1))) Clojure: (defonce net-bp (network (desc [128 1 28 28] :float :nchw) Which one is more readable? looking on the Clojure code I see 128 1 28 28 thrown on me, without digging in the documentation I have no idea what's happening.
  - susam 6y ago
    > Which one is more readable? Both are equally readable to me. Now, granted 128 1 28 28 can be difficult to understand without documentation but that is not due to Lisp's fully parenthesized prefix notation. The Clojure code would also look equally readable if it had used keyword arguments. Are you sure you are not confounding familiarity with readability? With Lisp, after a while, the parentheses become invisible to the programmer.
  - harperlee 6y ago
    But that's due to named parameters vs. positioning, not parenthesis; I can mirror your argument with: Python: model = Sequential() model.add(Conv2D(32, (3, 3),'relu', (28, 28, 1))) Clojure: (defonce net-bp (network (desc :input-shape [128 1 28 28] :type :float :something :nchw) It all depends on how desc is defined. Disclaimer: no idea what those functions mean, I only made syntactic changes.
godelski 6y ago
I'd actually love that material in C++/CUDA.
- dragandj 6y ago
  If only C++ supported interactive REPL and the rest of Clojure/Lisp goodies, that might be possible. However, the code is CLOSELY related to the actual CUDA/C++ api. It's a lot simpler, concise, and everything, but I explain everything so that you can use the relevant parts with cuDNN and DNNL APIs in any language that you're most proficient in.
  - dekhn 6y ago
    Does Cling C++ interpreter do what you want? https://github.com/root-project/cling https://github.com/root-project/cling
    - dragandj 6y ago
      Clojure does what I want. Perhaps Cling can do something, or not, but I guess that's up to people who prefer Cling to find out and utilize.
    - patrec 6y ago
      Have you used it? When I last tried Cling not that long ago, it wasn't even alpha quality software and given that it has been around for a while, my default assumption would be that this hasn't suddenly improved.
      - dekhn 6y ago
        it's used by ROOT at CERN, so it's unlikely it's "alpha quality software" since it's being used for production science at scale.
  - godelski 6y ago
    Thanks! That's helpful to know. I have no experience with Clojure/Lisp but a fair amount with C/C++ (minor in CUDA).
wokwokwok 6y ago
Concise isn’t always better. You’re throwing alway all the names of the arguments and using arbitrary words like “conv” to represent operations. This is typical bad clojure in my experience; write once, forget wtf the magic was, throw away and rewrite it again later. Clojure doesn’t have to be incomprehensible arcane magic that does everything in 10 lines. The more complex the code, the more important it is that what you do is clear and clearly documented. Don’t write a 1 line regex to solve a complicated problem; it’s the wrong tool for that job, no matter how smart your substring matches are. You don’t win a prize for making unmaintainable code. I similarly think the goal of being burning my concise in ML code is deeply misguided.
- dragandj 6y ago
  How is "conv" arbitrary? There is a function object that represents a convolutional layer in the network. It is bound to two symbols (because why not). You can either use "convolution" if you prefer full names, or "conv" if you prefer shorter. It doesn't represent the operation, but the layer. There are functions (with longer names) representing the convolution operation, which follow cuDNN and DNNL naming schemes. Regarding the magic, I believe you haven't read my writings related to this. Exactly the opposite - there is no magic other than usual Clojure-fu, which I explain in a layered way. But it's difficult to exactly reply to your critique, because you haven't given any example of an approach that would be good Clojure. Ok, give me an example of how you would do it in a comprehensible way (if what I provide is incomprehensible). You don't have to actually implement it. Show a non-working alternative. How would it look like?
- hansvm 6y ago
  Concision is a style choice to be used with care. Spending screen space on additional characters and descriptions detracts from the ability to fit more logic on the screen at once and grok the larger flow. Splashing symbolic alphabet soup into your IDE in the name of concision isn't usually a good idea, but naming something "conv" in the immediate local context of a convolutional layer doesn't seem so bad.
  - wokwokwok 6y ago
drzoltar 6y ago
Machine Learning in Clojure reminds me of Yann LeCun’s ML course from 2010, where we used an adorable language called Lush: http://lush.sourceforge.net/ http://lush.sourceforge.net/ which I suppose can best be described as Lisp and Python having a baby. It was immense fun to code neural networks from scratch in it. I hope Clojure can find a bigger place in the world of ML.
- YuriNiyazov 6y ago
  Another student of LeCun from NYU here. Can attest that lush is adorable. For example: For high performing parts of your code, a subset of lush would generate C code and compile them. I imagined that this is what it was like to write the first version of C++, the one that generated C code.
- lr1970 6y ago
  In those days there was a lovely LuaJIT based tensor manipulation language torch7 [1,2] developed by Leon Bottou. It later became basis for PyTorch. I still believe that Lua in general and LuaJIT in particular are much superior to Python for Deep Learning. [1] http://torch.ch/ http://torch.ch/ [2] https://github.com/torch/torch7 https://github.com/torch/torch7
runetech 6y ago
I really enjoy your work. I bought your book recently and appreciate your approach of building an understanding of the library based on "first principles". Really appreciate this performant and elegant option for working with deep-learning in Clojure - Thank you!
- dragandj 6y ago
  Thanks!

kyllo 6y ago

"Deep Learning in Clojure with Fewer Parentheses than Keras and Python" Love it! :D What better way to define a neural network in code than an S-expression?

piokoch 6y ago
I am not sure if I am that enthusiastic. The problem with Lisp is not a number of parenthesis but where they are and what is their role. In c-like languages parenthesis help parser compiler but they also help humans to read the code. In case of Lisp they are just for the sake of the parser. Let's look on the code: Python: model = Sequential() model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(28, 28, 1))) Clojure: (defonce net-bp (network (desc [128 1 28 28] :float :nchw) Which one is more readable? looking on the Clojure code I see 128 1 28 28 thrown on me, without digging in the documentation I have no idea what's happening.
- susam 6y ago
  > Which one is more readable? Both are equally readable to me. Now, granted 128 1 28 28 can be difficult to understand without documentation but that is not due to Lisp's fully parenthesized prefix notation. The Clojure code would also look equally readable if it had used keyword arguments. Are you sure you are not confounding familiarity with readability? With Lisp, after a while, the parentheses become invisible to the programmer.
- harperlee 6y ago
  But that's due to named parameters vs. positioning, not parenthesis; I can mirror your argument with: Python: model = Sequential() model.add(Conv2D(32, (3, 3),'relu', (28, 28, 1))) Clojure: (defonce net-bp (network (desc :input-shape [128 1 28 28] :type :float :something :nchw) It all depends on how desc is defined. Disclaimer: no idea what those functions mean, I only made syntactic changes.

godelski 6y ago

I'd actually love that material in C++/CUDA.

dragandj 6y ago
If only C++ supported interactive REPL and the rest of Clojure/Lisp goodies, that might be possible. However, the code is CLOSELY related to the actual CUDA/C++ api. It's a lot simpler, concise, and everything, but I explain everything so that you can use the relevant parts with cuDNN and DNNL APIs in any language that you're most proficient in.
- dekhn 6y ago
  Does Cling C++ interpreter do what you want? https://github.com/root-project/cling https://github.com/root-project/cling
  - dragandj 6y ago
    Clojure does what I want. Perhaps Cling can do something, or not, but I guess that's up to people who prefer Cling to find out and utilize.
  - patrec 6y ago
    Have you used it? When I last tried Cling not that long ago, it wasn't even alpha quality software and given that it has been around for a while, my default assumption would be that this hasn't suddenly improved.
    - dekhn 6y ago
      it's used by ROOT at CERN, so it's unlikely it's "alpha quality software" since it's being used for production science at scale.
- godelski 6y ago
  Thanks! That's helpful to know. I have no experience with Clojure/Lisp but a fair amount with C/C++ (minor in CUDA).

wokwokwok 6y ago

Concise isn’t always better. You’re throwing alway all the names of the arguments and using arbitrary words like “conv” to represent operations. This is typical bad clojure in my experience; write once, forget wtf the magic was, throw away and rewrite it again later. Clojure doesn’t have to be incomprehensible arcane magic that does everything in 10 lines. The more complex the code, the more important it is that what you do is clear and clearly documented. Don’t write a 1 line regex to solve a complicated problem; it’s the wrong tool for that job, no matter how smart your substring matches are. You don’t win a prize for making unmaintainable code. I similarly think the goal of being burning my concise in ML code is deeply misguided.

dragandj 6y ago
How is "conv" arbitrary? There is a function object that represents a convolutional layer in the network. It is bound to two symbols (because why not). You can either use "convolution" if you prefer full names, or "conv" if you prefer shorter. It doesn't represent the operation, but the layer. There are functions (with longer names) representing the convolution operation, which follow cuDNN and DNNL naming schemes. Regarding the magic, I believe you haven't read my writings related to this. Exactly the opposite - there is no magic other than usual Clojure-fu, which I explain in a layered way. But it's difficult to exactly reply to your critique, because you haven't given any example of an approach that would be good Clojure. Ok, give me an example of how you would do it in a comprehensible way (if what I provide is incomprehensible). You don't have to actually implement it. Show a non-working alternative. How would it look like?
hansvm 6y ago
Concision is a style choice to be used with care. Spending screen space on additional characters and descriptions detracts from the ability to fit more logic on the screen at once and grok the larger flow. Splashing symbolic alphabet soup into your IDE in the name of concision isn't usually a good idea, but naming something "conv" in the immediate local context of a convolutional layer doesn't seem so bad.
- wokwokwok 6y ago

drzoltar 6y ago

Machine Learning in Clojure reminds me of Yann LeCun’s ML course from 2010, where we used an adorable language called Lush: http://lush.sourceforge.net/ http://lush.sourceforge.net/ which I suppose can best be described as Lisp and Python having a baby. It was immense fun to code neural networks from scratch in it. I hope Clojure can find a bigger place in the world of ML.

YuriNiyazov 6y ago
Another student of LeCun from NYU here. Can attest that lush is adorable. For example: For high performing parts of your code, a subset of lush would generate C code and compile them. I imagined that this is what it was like to write the first version of C++, the one that generated C code.
lr1970 6y ago
In those days there was a lovely LuaJIT based tensor manipulation language torch7 [1,2] developed by Leon Bottou. It later became basis for PyTorch. I still believe that Lua in general and LuaJIT in particular are much superior to Python for Deep Learning. [1] http://torch.ch/ http://torch.ch/ [2] https://github.com/torch/torch7 https://github.com/torch/torch7

runetech 6y ago

I really enjoy your work. I bought your book recently and appreciate your approach of building an understanding of the library based on "first principles". Really appreciate this performant and elegant option for working with deep-learning in Clojure - Thank you!

dragandj 6y ago
Thanks!