10 ms·

Deep learning may need a new programming language

117 points by bobjordan 8y ago

anigbrowl 8y ago
I'd much rather a deep learning system that could handle simplified natural language and file operations, and then grow a network for specific tasks. Surely the point of AI is to liberate humans from having to write everything in code, and especially from needing to learn another language (which he fairly observes few people want to do). I do everything flow-based these days. It's not the fastest way (and I also have the luxury of not having to please anyone but myself), but it allows me to only think about my domain problem instead of programming language issues.
- codetrotter 8y ago
  Could you tell a bit more about how you do FBP? I looked at it briefly in the past but haven’t tried it myself.
  - anigbrowl 8y ago
    Let me preface by saying I use it just for API bashing as a researcher, not for deployment of anything commercial or even public-facing. I'm totally unqualified to talk about a production environment. I came at it from audio synthesis, where modularity and interoperability are priorities and there is relative cooperation between manufacturers and developers on technical standards. I mention this because audio synthesis and the closely associated business of sequencing have a great deal in common with breadboard electronics and super-basic computing like adders, flip-flops, and so on. You can implement simple classic video games like Pong and Asteroids in a modular synthesizer and play them on an oscilloscope if you're that way inclined. If you find this interesting, I'd suggest Reaktor as the software platform of choice as it's affordable, ahs a large community, and excellent documentation. Flowstone started as audio software and is now aimed at the Robotics industry. It has one of the nicest UIs and allows you to write code directly into modules. I use KNIME for high level data processing because it has an extensive library of database connections/ format translators/ API hooks etc.
- mark_l_watson 8y ago
  Maybe not quite what you have in mind, but I have tried saving trained Keras models, loading them with Racket (a modern Scheme language), and implemented the required runtime. My idea was to encapsulate trained networks like ‘functions’ for use in other languages.
  - dunefox 8y ago
    And?
    - mark_l_watson 8y ago
      and... I think this is a neat idea and I thought other people might enjoy trying it. code (2 repos) is on github
risubramanian 8y ago
I wonder what he thinks about Julia. There are lots of projects to turn it into an "ML language".
- dunefox 8y ago
  I think it was explicitly designed as an "ML Language" from the ground up already. Do you mean packages like Flux and KNet?
sandGorgon 8y ago
This is the reason why swift for tensorflow initiative was started by Google - https://github.com/tensorflow/swift/blob/master/README.md https://github.com/tensorflow/swift/blob/master/README.md Facebook will obviously go its our own way. Given their investments in the JS ecosystem, im hoping they end up choosing Typescript for this.
- dtech 8y ago
  Is Facebook adopting Typescript or are they still full-in on Flow, their own competing typed Javscript alternative?
  - sandGorgon 8y ago
    https://news.ycombinator.com/item?id=18918038 https://news.ycombinator.com/item?id=18918038 Jest is a Facebook project for testing React. They moved from Flow to Typescript. https://github.com/facebook/jest/pull/7554 https://github.com/facebook/jest/pull/7554 Dont want this thread to go OT. But I do think Typescript will be an awesome counterpart to Swift-for-Tensorflow on the ML side.
henning 8y ago
Fortran is going to make a big comeback. I can feel it.
- enriquto 8y ago
  > Fortran is going to make a big comeback. I can feel it. It is indeed one of the few existing languages appropriate for generic experimentation in numeric programming. For example, computing the product of two matrices in C or Fortran is exactly as fast by writing three nested loops or by calling a library function. In python, julia, octave, etc, the difference is abysmal. This is a very sad state of affairs, that forces a mindset where you are allowed a limited toolset of fast operations and the rest are either slow or cumbersome. If you want to compute a variant of the matrix product using a slightly different formula, in Fortran it is trivial change, you just write the new formula inside the loop. But in python or julia you are stuck with either an unusably slow code or you have to write it in another language entirely. "Vectorized" operations are cool, elegant, and beautiful. But they should not be the only tool available. As differential geometers often resort to coordinates to express their tensor operations, so should programmers be able to.
  - cshenton 8y ago
    Have you used Julia recently? Vectorised operations are just as fast relative to hand written loops as c and FORTRAN, you just need to appropriately annotate @inbounds and @simd. Sure that’s more work, but removing safety checks should be explicit.
    - enriquto 8y ago
      Not recently, thanks. I will surely try! My main gripe with the julia interpreter was that it was ridiculously slow to startup (I was using it as a "calculator" from within a shell loop: each iteration spawned a julia to perform a simple matrix computation). Does this performance has improved recently? By the way, what do you mean by "removing safety checks should be explicit" ? This sounds like a problem that the language should be able to deal with itself without bothering the programer (e.g. if the bounds of the loop are variables of known value, it can be checked beforehand, so the bounds checks can be safely omitted).
pts_ 8y ago
Why did everyone dump C++? Oh wait, they didn't.
- pjmlp 8y ago
  Depends on the context. On GPGPU programming certainly not, in fact latest NVidia's hardware is designed explicitly for C++ workloads. On GUI frameworks, C++ no longer has the spotlight on OS SDKs that it once had.
  - pts_ 8y ago
    My point is all AI worth its salt (robots) is C++ based in production.
    - pjmlp 8y ago
      There I fully agree with you, specially with NVidia doing C++ in hardware.
      - eggy 8y ago
        NVidia is adopting ADA/Spark for autonomous vehicles work [1]. It's service-proven, compiles to C/C++ speeds, and is safe. I learned Turbo Pascal in college in the 80s after Basic, C and assembler in the late 70s, early 80s, and I am attracted to languages like Haskell, Julia, Lisp, and J/APL, yet after toying with Spark, I think it is probably a good fit to do safe, ML at C/C++ speeds. It would be easy to hook it into all the C++ of TensorFlow too. [1] https://blogs.nvidia.com/blog/2019/02/05/adacore-secure-autonomous-driving/ https://blogs.nvidia.com/blog/2019/02/05/adacore-secure-auto...
        pjmlp 8y ago
        Yeah that as well. I belong to the same fanboy club, which is kind of why I do like C++ and not so much about C. I just don't see many adopting it without legislation enforcement, which is why the focus is on autonomous vehicles, where Ada already has a good story.
known 8y ago
1. Features 2. Performance 3. Usability You can pick only two options;
- awestroke 8y ago
  Or use Rust to get all 3!
  - pjmlp 8y ago
    They still need a bit of work to improve the 3., specially against GC languages.
  - semi-extrinsic 8y ago
    I'm sorry, but Rust certainly doesn't score high on usability. Compare e.g. the "Guessing Game" intro from the official docs, just look at how much more complicated both the code and the tutorial is as compared to how you would do the same in Python, heck even in Fortran: https://doc.rust-lang.org/book/ch02-00-guessing-game-tutorial.html https://doc.rust-lang.org/book/ch02-00-guessing-game-tutoria...
    - adrianN 8y ago
      I don't think comparing 100 line programs is a good benchmark for usability. Most software is much larger and the design choices you have to make for a programming language should favor programs with at least a few thousand lines of code.
      - pjmlp 8y ago
        Then try to do a few thousand lines of code in Gtk-rs, including custom widgets, the issues with borrow checker and internal mutable struct data accessible to callbacks is what lead to the creation of Relm.
        adrianN 8y ago
        The GUI story in Rust is currently terrible, I agree. But writing GUIs in Python with wrappers to C(++) libraries is not exactly nice either ;)
melling 8y ago
Sounds like FB is also working on its own AI chips: https://www.bloomberg.com/news/articles/2019-02-18/facebook-s-ai-chief-researching-new-breed-of-semiconductor?srnd=technology-vp https://www.bloomberg.com/news/articles/2019-02-18/facebook-... https://www.ft.com/content/1c2aab18-3337-11e9-bd3a-8b2a211d90d5 https://www.ft.com/content/1c2aab18-3337-11e9-bd3a-8b2a211d9...
thecatspaw 8y ago
Im sorry, but this very much feels like a fluff piece. Yes we _may_ need a new programming language, or we may not. Half of this article is about hardware, while only a small part of it is about programming languages. > There are several projects at Google, Facebook, and other places to kind of design such a compiled language that can be efficient for deep learning, but it’s not clear at all that the community will follow, because people just want to use Python Are the libraries not implemented in native code? The brief mention that python gets does not really detail what is wrong with it, aside from not beeing compiled I guess. But that is no issue if the libraries are native, so I dont see a reason to move away from python, let alone create a new programming language for it
- dan-robertson 8y ago
  The libraries are native code in a similar sense to python being native code. The kernels are native and typically highly optimised but there is a possibility that better optimised kernels can be generated on the fly once the computational graph is known.
  - heavenlyblue 8y ago
    >> that better optimised kernels can be generated on the fly once the computational graph is known. Which is already done by python libraries.
- lugg 8y ago
  You don't have to check baby's diaper to know they went number 2. Python ML code is a bit of a joke. If you're any kind of semi professional developer who sees deep learning code for the first time and doesn't say "what the fuck would you do it like that for", I simply don't know what to tell you. Im not saying a new language will fix this problem but some new primitives and idiomatic options protecting people from themselves probably wouldn't hurt the industry. Likewise, something more ergonomic might actually improve onboarding new developers into the space.
  - mark_l_watson 8y ago
xvilka 8y ago
What about Julia[1]? Seems like a perfect fit[2]. [1] https://julialang.org/ https://julialang.org/ [2] https://juliacomputing.com/domains/ml-and-ai.html https://juliacomputing.com/domains/ml-and-ai.html
- supernes 8y ago
  Julia is probably the next best bet, according to the Swift for Tensorflow team - https://github.com/tensorflow/swift/blob/master/docs/WhySwiftForTensorFlow.md https://github.com/tensorflow/swift/blob/master/docs/WhySwif...
- xuejie 8y ago
  This is exactly what I thought as well. Julia is born for this use case.
  - aldanor 8y ago
    Except for the 1-indexing part.
    - bluescarni 8y ago
      Been wanting to try Julia out for a while, but the 1-based indexing is perennially putting me off. I can't see how I can avoid spending a large chunk of time (and endless frustration) unlearning years of 0-index muscle memory (nor I see why should I, for the sake of adapting to a single language that, at least initially, I will be just toying with). It was probably a good marketing move on their part, a wink towards the Fortran/Matlab crowd, but it certainly damages the language's appeal for potential converts from C, C++ and Python.
      - mcabbott 8y ago
        Then challenge yourself to write some code completely agnostic to the indexing, which will force you to learn some neat features. Doing pointer arithmetic with your bare hands (however much muscle memory they have) is very seldom necessary, computers are pretty good at that stuff.
pi-victor 8y ago
i always thought Haskell would be amazing for this task. but python is really popular and easy to grasp. you'll end up writing a new programming language and people would still use python in the end.
posnet 8y ago
I know Nim has some interesting working going on here, https://github.com/mratsim/Arraymancer https://github.com/mratsim/Arraymancer As well as that esolang that appeared a few months ago on the front page. https://github.com/mrakgr/The-Spiral-Language https://github.com/mrakgr/The-Spiral-Language
- deleted 8y ago
  [deleted]
amelius 8y ago
The main problem, imho, is that (for best performance) most libraries require the programmer to create a data flowgraph, and to think in terms of this graph. However, this is the perfect job for a compiler. In mainstrain compilers, dataflow analysis has traditionally been the task of the compiler, so it seems silly to break with this tradition. A new (compiled) language could bring us back on track.
- spinningslate 8y ago
  I'd probably argue the opposite. Dataflow is a natural way to think about ML activities. The problem is that no mainstream languages offer dataflow as a first class construct; at best it's a set of libraries. That creates some impedance mismatch between intent and implementation. That compilers already do dataflow under the covers would hopefully mean that implementing a first-class dataflow language wouldn't be too hard.
  - giornogiovanna 8y ago
    What would a language primitive for "dataflow" look like, and what would it offer over an implementation as a library?
    - spinningslate 8y ago
      There are a few things, nicely covered in "Concepts, Techniques and Models of Computer Programming" by Van Roy and Heridi [0]. Briefly: 1. First-class dataflow variables 2. Syntax for creating flows (c/f pipe in *nix shell) 3. Transparent parallelisation All of these things exist in libraries of various sorts, but can feel cumbersome compared to native syntax. [0] https://mitpress.mit.edu/books/concepts-techniques-and-models-computer-programming https://mitpress.mit.edu/books/concepts-techniques-and-model...
    - nikofeyn 8y ago
      a wire.
  - amelius 8y ago
return0 8y ago
More likely a markup language
qwerty456127 8y ago
> Deep learning may need a new programming language that’s more flexible and easier to work with than Python Whoever has imagination more rich than mine, how can a programming language can be easier than Python? What's difficult in it? The only thing I find inconvenient in Python is you can't simply put every function (including every member functions of a class - that's what I would love to do) in a separate file without having to import every one of them manually.
- 4233456 8y ago
  Read it again. He/She did not imply that Python is hard. I do believe that he/she meant that they need way higher levels of abstraction than Python. Let's say Python is current C, they want something X that's like Ruby in compare to current C.
  - qwerty456127 8y ago
    It's just hard for me to imagine a higher level of abstraction. That's why I invite whoever has imagination more rich than mine to suggest ideas.
    - mehh 8y ago
      Have a look at Prolog, that might expand your mind a little.
- tjpnz 8y ago
  >The only thing I find inconvenient in Python is you can't simply put every function (including every member functions of a class - that's what I would love to do) into separate file without having to import every one of them manually. That strikes me as an anti-pattern although you could mimic this behaviour by exposing each one at the package level. Performance is likely to be terrible if you're working on anything sufficiently large.
- klmr 8y ago
  Not generally easier but easier to express specific, complex concepts. Tensorflow does alright but it’s certainly not beyond improving.
widforss 8y ago
It would be hilarious if everybody suddenly shifted to a Prolog implementation with ML functionality ¯\(°_o)/¯
- synthc 8y ago
  That would be awesome
- rurban 8y ago
  More like Miranda. Strongly typed Prolog with a nice compiler
- XuMiao 8y ago
  I like the idea. There are two types of ml jobs. One for modeling and another for computation. The former deals with the communication with human. The latter deals with the communication with machines. I would prefer that a statistical logic programing language (maybe extended from prolog) for the modeling part. A strong compiler tool or service that compiles it into any machines, on cloud or edge. Most of the time, what human likes to achieve is small and declarative. But for any small modifications of the program, we have to think about the computational flow, deal with the type errors and tune the low level efficiency of CPU and GPU. It's a waste. I like the settings of SQL where modelers use SQL to mine data while engineers maintain the performance of the computational engines.
ovi256 8y ago
>Python ... the language forms the basis for Facebook’s PyTorch and Google’s TensorFlow frameworks The state of journalism today ... their Github repo web pages show that there's more C++ than Python in both repos.
- throwawaymath 8y ago
  To be fair, while that's true I think what's meant here is that Python is the language of choice for using Tensorflow and PyTorch. Probably no other language is used to interface with deep learning libraries and primitives as much as Python.
  - ovi256 8y ago
    >I think what's meant here is that Python is the language of choice for using Tensorflow and PyTorch So it's a good editor that's lacking too, besides the fact-checkers.
kensai 8y ago
I thought that language was Julia.
Scarbutt 8y ago
sql
nottorp 8y ago
This guy - or the article's author - hasn't heard of DSLs?
slack3r 8y ago
Yes. Deep Learning definitely needs a new programming language. Uber's Pyro is worth checking out. (https://eng.uber.com/pyro/ https://eng.uber.com/pyro/)
- wenc 8y ago
  Umm that's not a programming language as such, it's a probabilistic programming DSL built on top of Python/Pytorch. It's one level of abstraction up.
thosakwe 8y ago
I’m not the biggest Python fan, but I can definitely tolerate it, as long as I have a decent way to use the models from another language.
Kip9000 8y ago
There's already Nim (https://nim-lang.org/ https://nim-lang.org/), which is Python like syntax and statically typed, with easy parallelisations etc already. What's lacking is the adoption as it wasn't hyped up. There's rarely a need to create yet another language and wait till it becomes mature and fixed all the issues with the eco system etc. If at all what's required is a way of translating all the Python libs to Nim or some similar effort.
- dunefox 8y ago
  In my opinion, I'd rather see F# or Julia succeed.
tzhenghao 8y ago
We may need a programming language for DL, but I doubt it'll happen soon, if it even happens. Lindy effect working in favor of Python here, as many data scientists have prior "big data" experience in it, and typical software engineers from the "scripting/tooling" world. People have been calling for the phaseout of C/C++, but even today's most popular DL frameworks have backends written in C++ in lieu of Rust.

117 points by bobjordan 8y ago

I'd much rather a deep learning system that could handle simplified natural language and file operations, and then grow a network for specific tasks. Surely the point of AI is to liberate humans from having to write everything in code, and especially from needing to learn another language (which he fairly observes few people want to do). I do everything flow-based these days. It's not the fastest way (and I also have the luxury of not having to please anyone but myself), but it allows me to only think about my domain problem instead of programming language issues.

codetrotter 8y ago
Could you tell a bit more about how you do FBP? I looked at it briefly in the past but haven’t tried it myself.
- anigbrowl 8y ago
  Let me preface by saying I use it just for API bashing as a researcher, not for deployment of anything commercial or even public-facing. I'm totally unqualified to talk about a production environment. I came at it from audio synthesis, where modularity and interoperability are priorities and there is relative cooperation between manufacturers and developers on technical standards. I mention this because audio synthesis and the closely associated business of sequencing have a great deal in common with breadboard electronics and super-basic computing like adders, flip-flops, and so on. You can implement simple classic video games like Pong and Asteroids in a modular synthesizer and play them on an oscilloscope if you're that way inclined. If you find this interesting, I'd suggest Reaktor as the software platform of choice as it's affordable, ahs a large community, and excellent documentation. Flowstone started as audio software and is now aimed at the Robotics industry. It has one of the nicest UIs and allows you to write code directly into modules. I use KNIME for high level data processing because it has an extensive library of database connections/ format translators/ API hooks etc.
mark_l_watson 8y ago
Maybe not quite what you have in mind, but I have tried saving trained Keras models, loading them with Racket (a modern Scheme language), and implemented the required runtime. My idea was to encapsulate trained networks like ‘functions’ for use in other languages.
- dunefox 8y ago
  And?
  - mark_l_watson 8y ago
    and... I think this is a neat idea and I thought other people might enjoy trying it. code (2 repos) is on github

risubramanian 8y ago

I wonder what he thinks about Julia. There are lots of projects to turn it into an "ML language".

dunefox 8y ago
I think it was explicitly designed as an "ML Language" from the ground up already. Do you mean packages like Flux and KNet?

sandGorgon 8y ago

This is the reason why swift for tensorflow initiative was started by Google - https://github.com/tensorflow/swift/blob/master/README.md https://github.com/tensorflow/swift/blob/master/README.md Facebook will obviously go its our own way. Given their investments in the JS ecosystem, im hoping they end up choosing Typescript for this.

dtech 8y ago
Is Facebook adopting Typescript or are they still full-in on Flow, their own competing typed Javscript alternative?
- sandGorgon 8y ago
  https://news.ycombinator.com/item?id=18918038 https://news.ycombinator.com/item?id=18918038 Jest is a Facebook project for testing React. They moved from Flow to Typescript. https://github.com/facebook/jest/pull/7554 https://github.com/facebook/jest/pull/7554 Dont want this thread to go OT. But I do think Typescript will be an awesome counterpart to Swift-for-Tensorflow on the ML side.

henning 8y ago

Fortran is going to make a big comeback. I can feel it.

enriquto 8y ago
> Fortran is going to make a big comeback. I can feel it. It is indeed one of the few existing languages appropriate for generic experimentation in numeric programming. For example, computing the product of two matrices in C or Fortran is exactly as fast by writing three nested loops or by calling a library function. In python, julia, octave, etc, the difference is abysmal. This is a very sad state of affairs, that forces a mindset where you are allowed a limited toolset of fast operations and the rest are either slow or cumbersome. If you want to compute a variant of the matrix product using a slightly different formula, in Fortran it is trivial change, you just write the new formula inside the loop. But in python or julia you are stuck with either an unusably slow code or you have to write it in another language entirely. "Vectorized" operations are cool, elegant, and beautiful. But they should not be the only tool available. As differential geometers often resort to coordinates to express their tensor operations, so should programmers be able to.
- cshenton 8y ago
  Have you used Julia recently? Vectorised operations are just as fast relative to hand written loops as c and FORTRAN, you just need to appropriately annotate @inbounds and @simd. Sure that’s more work, but removing safety checks should be explicit.
  - enriquto 8y ago
    Not recently, thanks. I will surely try! My main gripe with the julia interpreter was that it was ridiculously slow to startup (I was using it as a "calculator" from within a shell loop: each iteration spawned a julia to perform a simple matrix computation). Does this performance has improved recently? By the way, what do you mean by "removing safety checks should be explicit" ? This sounds like a problem that the language should be able to deal with itself without bothering the programer (e.g. if the bounds of the loop are variables of known value, it can be checked beforehand, so the bounds checks can be safely omitted).

pts_ 8y ago

Why did everyone dump C++? Oh wait, they didn't.

pjmlp 8y ago
Depends on the context. On GPGPU programming certainly not, in fact latest NVidia's hardware is designed explicitly for C++ workloads. On GUI frameworks, C++ no longer has the spotlight on OS SDKs that it once had.
- pts_ 8y ago
  My point is all AI worth its salt (robots) is C++ based in production.
  - pjmlp 8y ago
    There I fully agree with you, specially with NVidia doing C++ in hardware.
    - eggy 8y ago
      NVidia is adopting ADA/Spark for autonomous vehicles work [1]. It's service-proven, compiles to C/C++ speeds, and is safe. I learned Turbo Pascal in college in the 80s after Basic, C and assembler in the late 70s, early 80s, and I am attracted to languages like Haskell, Julia, Lisp, and J/APL, yet after toying with Spark, I think it is probably a good fit to do safe, ML at C/C++ speeds. It would be easy to hook it into all the C++ of TensorFlow too. [1] https://blogs.nvidia.com/blog/2019/02/05/adacore-secure-autonomous-driving/ https://blogs.nvidia.com/blog/2019/02/05/adacore-secure-auto...
      - pjmlp 8y ago
        Yeah that as well. I belong to the same fanboy club, which is kind of why I do like C++ and not so much about C. I just don't see many adopting it without legislation enforcement, which is why the focus is on autonomous vehicles, where Ada already has a good story.

known 8y ago

1. Features 2. Performance 3. Usability You can pick only two options;

awestroke 8y ago
Or use Rust to get all 3!
- pjmlp 8y ago
  They still need a bit of work to improve the 3., specially against GC languages.
- semi-extrinsic 8y ago
  I'm sorry, but Rust certainly doesn't score high on usability. Compare e.g. the "Guessing Game" intro from the official docs, just look at how much more complicated both the code and the tutorial is as compared to how you would do the same in Python, heck even in Fortran: https://doc.rust-lang.org/book/ch02-00-guessing-game-tutorial.html https://doc.rust-lang.org/book/ch02-00-guessing-game-tutoria...
  - adrianN 8y ago
    I don't think comparing 100 line programs is a good benchmark for usability. Most software is much larger and the design choices you have to make for a programming language should favor programs with at least a few thousand lines of code.
    - pjmlp 8y ago
      Then try to do a few thousand lines of code in Gtk-rs, including custom widgets, the issues with borrow checker and internal mutable struct data accessible to callbacks is what lead to the creation of Relm.
      - adrianN 8y ago
        The GUI story in Rust is currently terrible, I agree. But writing GUIs in Python with wrappers to C(++) libraries is not exactly nice either ;)

melling 8y ago

Sounds like FB is also working on its own AI chips: https://www.bloomberg.com/news/articles/2019-02-18/facebook-s-ai-chief-researching-new-breed-of-semiconductor?srnd=technology-vp https://www.bloomberg.com/news/articles/2019-02-18/facebook-... https://www.ft.com/content/1c2aab18-3337-11e9-bd3a-8b2a211d90d5 https://www.ft.com/content/1c2aab18-3337-11e9-bd3a-8b2a211d9...

thecatspaw 8y ago

Im sorry, but this very much feels like a fluff piece. Yes we _may_ need a new programming language, or we may not. Half of this article is about hardware, while only a small part of it is about programming languages. > There are several projects at Google, Facebook, and other places to kind of design such a compiled language that can be efficient for deep learning, but it’s not clear at all that the community will follow, because people just want to use Python Are the libraries not implemented in native code? The brief mention that python gets does not really detail what is wrong with it, aside from not beeing compiled I guess. But that is no issue if the libraries are native, so I dont see a reason to move away from python, let alone create a new programming language for it

dan-robertson 8y ago
The libraries are native code in a similar sense to python being native code. The kernels are native and typically highly optimised but there is a possibility that better optimised kernels can be generated on the fly once the computational graph is known.
- heavenlyblue 8y ago
  >> that better optimised kernels can be generated on the fly once the computational graph is known. Which is already done by python libraries.
lugg 8y ago
You don't have to check baby's diaper to know they went number 2. Python ML code is a bit of a joke. If you're any kind of semi professional developer who sees deep learning code for the first time and doesn't say "what the fuck would you do it like that for", I simply don't know what to tell you. Im not saying a new language will fix this problem but some new primitives and idiomatic options protecting people from themselves probably wouldn't hurt the industry. Likewise, something more ergonomic might actually improve onboarding new developers into the space.
- mark_l_watson 8y ago

xvilka 8y ago

What about Julia[1]? Seems like a perfect fit[2]. [1] https://julialang.org/ https://julialang.org/ [2] https://juliacomputing.com/domains/ml-and-ai.html https://juliacomputing.com/domains/ml-and-ai.html

supernes 8y ago
Julia is probably the next best bet, according to the Swift for Tensorflow team - https://github.com/tensorflow/swift/blob/master/docs/WhySwiftForTensorFlow.md https://github.com/tensorflow/swift/blob/master/docs/WhySwif...
xuejie 8y ago
This is exactly what I thought as well. Julia is born for this use case.
- aldanor 8y ago
  Except for the 1-indexing part.
  - bluescarni 8y ago
    Been wanting to try Julia out for a while, but the 1-based indexing is perennially putting me off. I can't see how I can avoid spending a large chunk of time (and endless frustration) unlearning years of 0-index muscle memory (nor I see why should I, for the sake of adapting to a single language that, at least initially, I will be just toying with). It was probably a good marketing move on their part, a wink towards the Fortran/Matlab crowd, but it certainly damages the language's appeal for potential converts from C, C++ and Python.
    - mcabbott 8y ago
      Then challenge yourself to write some code completely agnostic to the indexing, which will force you to learn some neat features. Doing pointer arithmetic with your bare hands (however much muscle memory they have) is very seldom necessary, computers are pretty good at that stuff.

pi-victor 8y ago

i always thought Haskell would be amazing for this task. but python is really popular and easy to grasp. you'll end up writing a new programming language and people would still use python in the end.

posnet 8y ago

I know Nim has some interesting working going on here, https://github.com/mratsim/Arraymancer https://github.com/mratsim/Arraymancer As well as that esolang that appeared a few months ago on the front page. https://github.com/mrakgr/The-Spiral-Language https://github.com/mrakgr/The-Spiral-Language

deleted 8y ago
[deleted]

amelius 8y ago

The main problem, imho, is that (for best performance) most libraries require the programmer to create a data flowgraph, and to think in terms of this graph. However, this is the perfect job for a compiler. In mainstrain compilers, dataflow analysis has traditionally been the task of the compiler, so it seems silly to break with this tradition. A new (compiled) language could bring us back on track.

spinningslate 8y ago
I'd probably argue the opposite. Dataflow is a natural way to think about ML activities. The problem is that no mainstream languages offer dataflow as a first class construct; at best it's a set of libraries. That creates some impedance mismatch between intent and implementation. That compilers already do dataflow under the covers would hopefully mean that implementing a first-class dataflow language wouldn't be too hard.
- giornogiovanna 8y ago
  What would a language primitive for "dataflow" look like, and what would it offer over an implementation as a library?
  - spinningslate 8y ago
    There are a few things, nicely covered in "Concepts, Techniques and Models of Computer Programming" by Van Roy and Heridi [0]. Briefly: 1. First-class dataflow variables 2. Syntax for creating flows (c/f pipe in *nix shell) 3. Transparent parallelisation All of these things exist in libraries of various sorts, but can feel cumbersome compared to native syntax. [0] https://mitpress.mit.edu/books/concepts-techniques-and-models-computer-programming https://mitpress.mit.edu/books/concepts-techniques-and-model...
  - nikofeyn 8y ago
    a wire.
- amelius 8y ago

return0 8y ago

More likely a markup language

qwerty456127 8y ago

> Deep learning may need a new programming language that’s more flexible and easier to work with than Python Whoever has imagination more rich than mine, how can a programming language can be easier than Python? What's difficult in it? The only thing I find inconvenient in Python is you can't simply put every function (including every member functions of a class - that's what I would love to do) in a separate file without having to import every one of them manually.

4233456 8y ago
Read it again. He/She did not imply that Python is hard. I do believe that he/she meant that they need way higher levels of abstraction than Python. Let's say Python is current C, they want something X that's like Ruby in compare to current C.
- qwerty456127 8y ago
  It's just hard for me to imagine a higher level of abstraction. That's why I invite whoever has imagination more rich than mine to suggest ideas.
  - mehh 8y ago
    Have a look at Prolog, that might expand your mind a little.
tjpnz 8y ago
>The only thing I find inconvenient in Python is you can't simply put every function (including every member functions of a class - that's what I would love to do) into separate file without having to import every one of them manually. That strikes me as an anti-pattern although you could mimic this behaviour by exposing each one at the package level. Performance is likely to be terrible if you're working on anything sufficiently large.
klmr 8y ago
Not generally easier but easier to express specific, complex concepts. Tensorflow does alright but it’s certainly not beyond improving.

widforss 8y ago

It would be hilarious if everybody suddenly shifted to a Prolog implementation with ML functionality ¯\(°_o)/¯

synthc 8y ago
That would be awesome
rurban 8y ago
More like Miranda. Strongly typed Prolog with a nice compiler
XuMiao 8y ago
I like the idea. There are two types of ml jobs. One for modeling and another for computation. The former deals with the communication with human. The latter deals with the communication with machines. I would prefer that a statistical logic programing language (maybe extended from prolog) for the modeling part. A strong compiler tool or service that compiles it into any machines, on cloud or edge. Most of the time, what human likes to achieve is small and declarative. But for any small modifications of the program, we have to think about the computational flow, deal with the type errors and tune the low level efficiency of CPU and GPU. It's a waste. I like the settings of SQL where modelers use SQL to mine data while engineers maintain the performance of the computational engines.

ovi256 8y ago

>Python ... the language forms the basis for Facebook’s PyTorch and Google’s TensorFlow frameworks The state of journalism today ... their Github repo web pages show that there's more C++ than Python in both repos.

throwawaymath 8y ago
To be fair, while that's true I think what's meant here is that Python is the language of choice for using Tensorflow and PyTorch. Probably no other language is used to interface with deep learning libraries and primitives as much as Python.
- ovi256 8y ago
  >I think what's meant here is that Python is the language of choice for using Tensorflow and PyTorch So it's a good editor that's lacking too, besides the fact-checkers.

kensai 8y ago

I thought that language was Julia.

Scarbutt 8y ago

sql

nottorp 8y ago

This guy - or the article's author - hasn't heard of DSLs?

slack3r 8y ago

Yes. Deep Learning definitely needs a new programming language. Uber's Pyro is worth checking out. (https://eng.uber.com/pyro/ https://eng.uber.com/pyro/)

wenc 8y ago
Umm that's not a programming language as such, it's a probabilistic programming DSL built on top of Python/Pytorch. It's one level of abstraction up.

thosakwe 8y ago

I’m not the biggest Python fan, but I can definitely tolerate it, as long as I have a decent way to use the models from another language.

Kip9000 8y ago

There's already Nim (https://nim-lang.org/ https://nim-lang.org/), which is Python like syntax and statically typed, with easy parallelisations etc already. What's lacking is the adoption as it wasn't hyped up. There's rarely a need to create yet another language and wait till it becomes mature and fixed all the issues with the eco system etc. If at all what's required is a way of translating all the Python libs to Nim or some similar effort.

dunefox 8y ago
In my opinion, I'd rather see F# or Julia succeed.

tzhenghao 8y ago

We may need a programming language for DL, but I doubt it'll happen soon, if it even happens. Lindy effect working in favor of Python here, as many data scientists have prior "big data" experience in it, and typical software engineers from the "scripting/tooling" world. People have been calling for the phaseout of C/C++, but even today's most popular DL frameworks have backends written in C++ in lieu of Rust.