6 ms·

I have spent a lot of time experimenting with the Haskell bindings for TensorFlow and a little time with HLearn. Sorry for a negative opinion, but as much as I

I have spent a lot of time experimenting with the Haskell bindings for TensorFlow and a little time with HLearn. Sorry for a negative opinion, but as much as I don’t particularly like Python, I would suggest to almost anyone that they stick with Python for deep learning for now. In a year or two, I might recommend TensorFlow implemented turtles all the way down in Swift, but let’s wait and see how that project progresses. I have only spent a few evenings playing with Clojure and mxnet and while I appreciate the efforts of the Clojure mxnet subproject team, I think you are still better off for now with Python, TensorFlow, and PyTorch. A little off topic: I had a deep learning example for Armed Bear Common Lisp (implemented in Java) and DeepLearning4j in the last edition of my Common Lisp book. In the latest edition of my book, I removed that example and the chapter that went with it and replaced it with two examples of Lisp code using REST services written in Python, SpaCy, and TensorFlow - I think that is more practical right now; the situation may change in the future. EDIT: I also added REST examples using Python, SpaCy, and TensorFlow to the second edition of my Haskell book.

lonelappde 7y ago
I tried to play with tensorflow in Haskell, but I found, as with all nontrivial libraries with external bindings, getting it to install on my system is a headache and a half, better incompatibilities, and massive compile times and memory demands (8GB+ system) and hard disk usage / IO. Unless stuff is published as whole images for docker or virtual box, it's just exhausting to work with non-pure Haskell (and sometimes even pure Haskell with many dependencies).
- lallysingh 7y ago
  Yeah I've given up on the standard cabal approach for nontrivial projects. Nix often works, but it's never free.
melling 7y ago
How quickly is Swift progressing for Deep Learning? In this podcast, Jeremy Howard (https://www.fast.ai/ https://www.fast.ai/ ) also sounded excited about Swift: https://lexfridman.com/jeremy-howard/ https://lexfridman.com/jeremy-howard/
- mark_l_watson 7y ago
  Thanks for the link for Lex’s interview. I watched it last week, a fascinating conversation. Jeremy is teaching a class with the Swift version of TensorFlow, which I experimented with but setup on my Mac sometimes worked, but a update broke it for me.
- deleted 7y ago
  [deleted]
- lostmsu 7y ago
  If you want static typing (mostly) on your TensorFlow, and are willing to explore a new language, I am making https://losttech.software/gradient.html https://losttech.software/gradient.html - basically .NET binding to the entirety of TensorFlow. If you want functional, it works with F#, but syntax is a bit worse due to the lack of DLR support. See samples: https://github.com/losttech/Gradient-Samples https://github.com/losttech/Gradient-Samples
syllogism 7y ago
You might find the approach I used to implement neural networks in spaCy interesting, if you haven't seen it: https://github.com/explosion/thinc/ https://github.com/explosion/thinc/ Basically it's all functional programming. Python is actually not a great language for this, I'm sure it would be better in a typed functional language.
- mark_l_watson 7y ago
  Thanks, Thinc looks good!
- johnfactorial 7y ago
  For what it's worth, Python is a strongly typed language[0] and is what the manual calls "multi-paradigm," i.e. you can write purely functionally in it[1] all you want, though it is a fair point to observe that the community, PEPs, Guido et. al. do not encourage it, and the language implementations are probably not optimized for it. [0]: https://stackoverflow.com/questions/11328920/is-python-strongly-typed https://stackoverflow.com/questions/11328920/is-python-stron... [1]: https://docs.python.org/3/howto/functional.html https://docs.python.org/3/howto/functional.html
  - syllogism 7y ago
    I mean...Come on, it's not like that's news to me :p. The mypy stuff in particular is somewhat promising, but it's a pretty awkward retrofit. A lot of the key APIs (e.g. numpy) were not designed around type declarations, so numpy will very often return either a float or an array depending on the input operations, keyword arguments, etc. This sort of thing is super common.
    - dllthomas 7y ago
      > numpy will very often return either a float or an array depending on the input operations, keyword arguments, etc That makes it harder to find good types, but it doesn't rule it out. Either you can tell what the shape of the output should be without calling the function in question or you can't. If you can't, you should be checking the shape of the output after the call, and the type of the output should be some kind of union so the type system encourages this. On the other hand, if you can look at the arguments and easily know how the output should look, there's a fair chance you can push this knowledge into the type system. I've done similar things in flow. As a recent example, the download function in the google storage API returns a promise of the file's contents, unless the argument contains a 'destination' key in which case it returns a promise of void. Describing that as the intersection of the two function types got me the checking I want. Amusingly, I don't think there's a way to write a function with this type without a cast through any, but that's not a problem for describing an untyped API. All of that said, I don't know how well this works out with numpy and mypy in particular.
abtinf 7y ago
You provided an opinion, but no reasoning or evidence. Could you help us by explaining how you came to your view?
- lonelappde 7y ago
  Cabal hell and FFI hell. Feel free to try it yourself if you don't believe others.
  - the_af 7y ago
    I don't think they said they didn't believe. They were just asking for actual arguments, which seems fair to me. As for Cabal hell, doesn't Stack [1] help with that? To be honest, I'm unhappy with dependency managers in most languages: Maven, the things Golang uses, or Python, or NPM (shudder!), etc. This really doesn't seem like a solved problem :( [1] https://docs.haskellstack.org/en/stable/README/ https://docs.haskellstack.org/en/stable/README/
    - lallysingh 7y ago
      Nix is a huge PITA to get working, but it's (afaict) the only thing that you can reliably get working.
  - deleted 7y ago
    [deleted]
  - zopa 7y ago
    Cabal hell is a solved problem these days. Solved three different competing ways --- nix, cabal new-build, Stack --- where ideally there'd just be one, but not the issue it was five years ago. That doesn't mean dependency management is always easy. It's a hard problem in any language. But you won't find yourself in an inconsistent state that requires blowing away your installation if you use one of those three approaches.
- mark_l_watson 7y ago
  I have been using Neural Networks since the 1980s, and I just finished a job at Capital One managing a deep learning team. I work mostly with GANs and RNN.
Scarbutt 7y ago
TensorFlow has a Java API now, so a thin Clojure wrapper shouldn't be much trouble.
- jjoonathan 7y ago
  Second-class-citizen APIs are always trouble. It's fair to expect the Clojure wrapper to not be much more trouble, though.
  - vemv 7y ago
    > Second-class-citizen APIs are always trouble. An unlikely absolute: nothing prevents the second-class citizen from gaining traction and eventually becoming first-class.
pj_mukh 7y ago
"turtles all the way down in Swift," This might be a little outdated but isn't everything (incl Python) a wrapper around C++ calls in TF?
- ddragon 7y ago
  Yes, the backend for Swift for Tensorflow will still be C++. The difference is that it will be the compiler (at compile-time) and not some library (at runtime) that will be mapping Swift native operations (instead of library calls) directly into the Tensorflow graph API. It's a much more sophisticated frontend, but there is no porting of the backend to Swift.
- tanilama 7y ago
  As a full-time Python DL developer, I would say the major pain point isn't about passing the ndarrays/tensors to BTS runtime. The major pain point of Python is the ... Python part of it, that is not related to ML itself. Like processing preprocessing data and sample it and feed to your downstream model stuff. This part is surprisingly effortful, because of Python's slow runtime and GIL, etc. TBH for a lot of stuff there can be a workaround, but it is just manual and brittle and can't by further away than the supposed Pythonistical experience you would assume. What Swift could be to the scene is a set of powerful and easy to work with preprocessing primitives that writes once, and runs both in training/production time. That would a revolutionary experience once it happened. I can totally see the market there, and won't mind be an early adopter to it. With that being said, Python isn't going anywhere any time soon. People bitch about it, but there is still no other option on the market that is more productive than Python. Python isn't perfect, as it isn't as bad as some posts here trying to convince the audience otherwise.
jackpirate 7y ago
As the author of HLearn, I 100% agree. I do all my actual ML work in python (which is most of my work these days). At this point, the Haskell ecosystem just doesn't have the engineering support of something like tensorflow, and so there's no competition.
- dmix 7y ago
  Community size always outranks the pros/cons of languages. Still they are never entirely always static and it always takes some bold people to push it towards something new and (hopefully) better. Although more often than not it stays the same.
lostmsu 7y ago
Disclaimer: self-promotion. You can use all TensorFlow APIs from F#, if you want functional language (but not pure functional style). See for example: https://github.com/losttech/Gradient-Samples/blob/master/FSharp/FashionMnistF/FashionMnist.fs https://github.com/losttech/Gradient-Samples/blob/master/FSh... It looks much better in C# though due to limited DLR support in F#.

lonelappde 7y ago

I tried to play with tensorflow in Haskell, but I found, as with all nontrivial libraries with external bindings, getting it to install on my system is a headache and a half, better incompatibilities, and massive compile times and memory demands (8GB+ system) and hard disk usage / IO. Unless stuff is published as whole images for docker or virtual box, it's just exhausting to work with non-pure Haskell (and sometimes even pure Haskell with many dependencies).

lallysingh 7y ago
Yeah I've given up on the standard cabal approach for nontrivial projects. Nix often works, but it's never free.

melling 7y ago

How quickly is Swift progressing for Deep Learning? In this podcast, Jeremy Howard (https://www.fast.ai/ https://www.fast.ai/ ) also sounded excited about Swift: https://lexfridman.com/jeremy-howard/ https://lexfridman.com/jeremy-howard/

mark_l_watson 7y ago
Thanks for the link for Lex’s interview. I watched it last week, a fascinating conversation. Jeremy is teaching a class with the Swift version of TensorFlow, which I experimented with but setup on my Mac sometimes worked, but a update broke it for me.
deleted 7y ago
[deleted]
lostmsu 7y ago
If you want static typing (mostly) on your TensorFlow, and are willing to explore a new language, I am making https://losttech.software/gradient.html https://losttech.software/gradient.html - basically .NET binding to the entirety of TensorFlow. If you want functional, it works with F#, but syntax is a bit worse due to the lack of DLR support. See samples: https://github.com/losttech/Gradient-Samples https://github.com/losttech/Gradient-Samples

syllogism 7y ago

You might find the approach I used to implement neural networks in spaCy interesting, if you haven't seen it: https://github.com/explosion/thinc/ https://github.com/explosion/thinc/ Basically it's all functional programming. Python is actually not a great language for this, I'm sure it would be better in a typed functional language.

mark_l_watson 7y ago
Thanks, Thinc looks good!
johnfactorial 7y ago
For what it's worth, Python is a strongly typed language[0] and is what the manual calls "multi-paradigm," i.e. you can write purely functionally in it[1] all you want, though it is a fair point to observe that the community, PEPs, Guido et. al. do not encourage it, and the language implementations are probably not optimized for it. [0]: https://stackoverflow.com/questions/11328920/is-python-strongly-typed https://stackoverflow.com/questions/11328920/is-python-stron... [1]: https://docs.python.org/3/howto/functional.html https://docs.python.org/3/howto/functional.html
- syllogism 7y ago
  I mean...Come on, it's not like that's news to me :p. The mypy stuff in particular is somewhat promising, but it's a pretty awkward retrofit. A lot of the key APIs (e.g. numpy) were not designed around type declarations, so numpy will very often return either a float or an array depending on the input operations, keyword arguments, etc. This sort of thing is super common.
  - dllthomas 7y ago
    > numpy will very often return either a float or an array depending on the input operations, keyword arguments, etc That makes it harder to find good types, but it doesn't rule it out. Either you can tell what the shape of the output should be without calling the function in question or you can't. If you can't, you should be checking the shape of the output after the call, and the type of the output should be some kind of union so the type system encourages this. On the other hand, if you can look at the arguments and easily know how the output should look, there's a fair chance you can push this knowledge into the type system. I've done similar things in flow. As a recent example, the download function in the google storage API returns a promise of the file's contents, unless the argument contains a 'destination' key in which case it returns a promise of void. Describing that as the intersection of the two function types got me the checking I want. Amusingly, I don't think there's a way to write a function with this type without a cast through any, but that's not a problem for describing an untyped API. All of that said, I don't know how well this works out with numpy and mypy in particular.

abtinf 7y ago

You provided an opinion, but no reasoning or evidence. Could you help us by explaining how you came to your view?

lonelappde 7y ago
Cabal hell and FFI hell. Feel free to try it yourself if you don't believe others.
- the_af 7y ago
  I don't think they said they didn't believe. They were just asking for actual arguments, which seems fair to me. As for Cabal hell, doesn't Stack [1] help with that? To be honest, I'm unhappy with dependency managers in most languages: Maven, the things Golang uses, or Python, or NPM (shudder!), etc. This really doesn't seem like a solved problem :( [1] https://docs.haskellstack.org/en/stable/README/ https://docs.haskellstack.org/en/stable/README/
  - lallysingh 7y ago
    Nix is a huge PITA to get working, but it's (afaict) the only thing that you can reliably get working.
- deleted 7y ago
  [deleted]
- zopa 7y ago
  Cabal hell is a solved problem these days. Solved three different competing ways --- nix, cabal new-build, Stack --- where ideally there'd just be one, but not the issue it was five years ago. That doesn't mean dependency management is always easy. It's a hard problem in any language. But you won't find yourself in an inconsistent state that requires blowing away your installation if you use one of those three approaches.
mark_l_watson 7y ago
I have been using Neural Networks since the 1980s, and I just finished a job at Capital One managing a deep learning team. I work mostly with GANs and RNN.

Scarbutt 7y ago

TensorFlow has a Java API now, so a thin Clojure wrapper shouldn't be much trouble.

jjoonathan 7y ago
Second-class-citizen APIs are always trouble. It's fair to expect the Clojure wrapper to not be much more trouble, though.
- vemv 7y ago
  > Second-class-citizen APIs are always trouble. An unlikely absolute: nothing prevents the second-class citizen from gaining traction and eventually becoming first-class.

pj_mukh 7y ago

"turtles all the way down in Swift," This might be a little outdated but isn't everything (incl Python) a wrapper around C++ calls in TF?

ddragon 7y ago
Yes, the backend for Swift for Tensorflow will still be C++. The difference is that it will be the compiler (at compile-time) and not some library (at runtime) that will be mapping Swift native operations (instead of library calls) directly into the Tensorflow graph API. It's a much more sophisticated frontend, but there is no porting of the backend to Swift.
tanilama 7y ago
As a full-time Python DL developer, I would say the major pain point isn't about passing the ndarrays/tensors to BTS runtime. The major pain point of Python is the ... Python part of it, that is not related to ML itself. Like processing preprocessing data and sample it and feed to your downstream model stuff. This part is surprisingly effortful, because of Python's slow runtime and GIL, etc. TBH for a lot of stuff there can be a workaround, but it is just manual and brittle and can't by further away than the supposed Pythonistical experience you would assume. What Swift could be to the scene is a set of powerful and easy to work with preprocessing primitives that writes once, and runs both in training/production time. That would a revolutionary experience once it happened. I can totally see the market there, and won't mind be an early adopter to it. With that being said, Python isn't going anywhere any time soon. People bitch about it, but there is still no other option on the market that is more productive than Python. Python isn't perfect, as it isn't as bad as some posts here trying to convince the audience otherwise.

jackpirate 7y ago

As the author of HLearn, I 100% agree. I do all my actual ML work in python (which is most of my work these days). At this point, the Haskell ecosystem just doesn't have the engineering support of something like tensorflow, and so there's no competition.

dmix 7y ago
Community size always outranks the pros/cons of languages. Still they are never entirely always static and it always takes some bold people to push it towards something new and (hopefully) better. Although more often than not it stays the same.

lostmsu 7y ago

Disclaimer: self-promotion. You can use all TensorFlow APIs from F#, if you want functional language (but not pure functional style). See for example: https://github.com/losttech/Gradient-Samples/blob/master/FSharp/FashionMnistF/FashionMnist.fs https://github.com/losttech/Gradient-Samples/blob/master/FSh... It looks much better in C# though due to limited DLR support in F#.