9 ms·

> I don't do a lot of artificial intelligence, natural-language processing or machine-learning research, but if I did, Haskell would be my first pick there too.

• nkozyra 6y ago
  > why aren't the "more expressive" (which in this post seems to be a synonym for "functional") languages used in these fields The real answer is probably inertia, but ... I think ultimately there isn't a ton of actual "programming" happening in ML, more implementation. People like Python. It's simple, it's clean, it's elegant and it's performant when it needs to be. I'm sure a lot of people who have thought "man why don't I just re-write this in [language]?" stopped short because they soon asked themselves "wait, why?" Most of your work is on either side of the algorithm (data - [ml] - analysis).
  • krebs_liebhaber 6y ago
    Functional languages are pretty good for the cleaning and analysis process as well. This blogpost in particular blew my mind: http://evelinag.com/blog/2015/11-18-f-tackles-james-bond/ http://evelinag.com/blog/2015/11-18-f-tackles-james-bond/
    • nkozyra 6y ago
      Oh I think functional languages would be great for a lot of this, my point is the middle bit ends up being such an inconsequential part of the process that people end up not bothering and just using whatever's out there.
• platinumrad 6y ago
  Elixir is a BEAM language and therefore not designed for computational throughput.
  • krebs_liebhaber 6y ago
    That's true - it's why I said you need a native library (something like numpy, but as a NIF) to back it up.
    • rhizome31 6y ago
      There's Matrex, which is based on CBLAS and has some integration with IElixir, Jupyter's kernel for Elixir. Also someone seems to be doing interesting work to integrate with GPU at https://github.com/sasagawa888/deeppipe2 https://github.com/sasagawa888/deeppipe2 Of course that's not to say that Elixir is ready to replace Python tomorrow. I just wanted to mention interesting projects in this area.
    • neutronicus 6y ago
      I think the main issue you face here is that there is already a big C/C++/Fortran ecosystem around distributed computation, one that happens to be very well impedance-matched to numerical linear algebra, and one with which experts in numerical linear algebra are intimately familiar (MPI). Why bother with Erlang as a thin wrapper over C++ when C++ has the necessary facilities already?
  • bogomipz 6y ago
    What is it specifically about BEAM languages that make them not good for computational throughput?
    • krebs_liebhaber 6y ago
      Immutability and thread-safety - if you want to get a value from one concurrent process to another, you have to copy it and send it to the other process. You can circumvent this by using a smart-pointer / reference to mutable data, which is manipulated by native code. One example would be a ResourceArc[0] in Rustler. [0] https://docs.rs/rustler/0.22.0-rc.0/rustler/resource/struct.ResourceArc.html https://docs.rs/rustler/0.22.0-rc.0/rustler/resource/struct....
• danieldk 6y ago
  why aren't the "more expressive" (which in this post seems to be a synonym for "functional") languages used in these fields Because very often the best solution does not win, which is why we are still using x86(_64), Unix, etc. Python is a good example of worse is better. A functional language could provide more elegant abstractions. But for most people the ML APIs that Python provides are good enough. By taking a pragmatic approach, Python was faster to capture some niches, including machine learning and data science. Now we are at a point that even if another language had better abstractions and had similar library support, it would face an uphill battle because of inertia. Why would someone invest in learning another language for machine learning when Python is good enough? To me it seems that only a language that has both a good model development story and a good deployment story stands a chance to compete with Python, since deployment (especially on mobile/embedded devices) is a weak point of the Python ML ecosystem.
  • rtkaratekid 6y ago
    Little tangent, but what do you consider better than Unix? I don’t have much experience with anything else so I’m curious.
    • pjmlp 6y ago
      Anything based on the workstation concepts explored at Xerox PARC, and further pursued at ETHZ, TI and Genera. Bitsavers has all the necessary papers and manuals, and computer museum as well. Here are some video sessions. "Yesterday's Computer of Tomorrow: The Xerox Alto │Smalltalk-76 Demo" https://www.youtube.com/watch?v=NqKyHEJe9_w https://www.youtube.com/watch?v=NqKyHEJe9_w "Alto System Project: Dan Ingalls demonstrates Smalltalk" https://www.youtube.com/watch?v=uknEhXyZgsg https://www.youtube.com/watch?v=uknEhXyZgsg "Eric Bier Demonstrates Cedar" https://www.youtube.com/watch?v=z_dt7NG38V4 https://www.youtube.com/watch?v=z_dt7NG38V4 "The Final Demonstration of the Xerox 'Star' computer" https://www.youtube.com/watch?v=_OwG_rQ_Hqw https://www.youtube.com/watch?v=_OwG_rQ_Hqw The end of the road from Plan 9, Inferno. http://www.vitanuova.com/inferno/index.html http://www.vitanuova.com/inferno/index.html http://doc.cat-v.org/inferno/4th_edition/ http://doc.cat-v.org/inferno/4th_edition/
• jlouis 6y ago
  The current way to get a lot of efficient computation on a modern computer is to engage the GPU of said machine. This means that any language hosted on the CPU is going to be at a disadvantage. In short: you are better off writing a compiler toward the GPU and use it as a compilation target. And a lot of things argues that language benefits from being a functional one, given how GPUs tend to work on data. If there were no prior work, and if we have the full GPU specs, I'd reckon you want languages which are strong in program translation: Haskell or OCaml comes to mind. Elixir---as well as its Erlang sibling---are really good at handling problems where there is concurrency involved. It is subtly different from parallel computation in that you often have many threads of execution all wanting to do different things at the same time. It is the same space e.g., Go occupies really well.
  • Firadeoclus 6y ago
    As someone who has been writing hand-optimised GPU code for a while now, I'd argue that this high level view (and thus an appropriate functional high-level language) can get you reasonably decent performance on a GPU for little effort for many highly parallel problems. And there are still plenty of cases where this will leave performance improvements of an order of magnitude on the table, because the compiler isn't smart enough to work out the optimal sequencing of operations. Often the computational core which benefits from running on a GPU is small and its complexity low enough that, if performance is a priority, you are better off writing it in an imperative low-level language as a library which can be accessed from higher level code.
    • Munksgaard 6y ago
      Do you have any examples of such cases? I'm asking because I'm part of the team that's working on the Futhark language[0], which is exactly what you're describing: a high-level array language in a functional style, targeting the GPU. We're always on the lookout for interesting applications or benchmarks. In particular, we're interested in knowing exactly where our shortcomings are, compared to hand-written GPU code, so that we may work on improving them. For the curious, we think we're doing reasonably well compared to benchmark suites such as Rodinia[1], though there are still cases where we cannot keep up. [0]: https://futhark-lang.org/ https://futhark-lang.org/ [1]: https://futhark-lang.org/performance.html https://futhark-lang.org/performance.html
• bregma 6y ago
  > why aren't the "more expressive" (which in this post seems to be a synonym for "functional") languages used in these fields? 90% of all programmers are cargo-culters whose knowledge of programming is gained from the answers they cut-and-paste from Stack Exchange or Quora. This is as true of "data scientisis" as it is of "web developers." The result is that there is a network effect making certain answers determinant of the language or monthly flavour of xxxScript that gets pasted most often, and reduces the domain in which further answers are examined. If you want it in simpler terms: the new dudes looked around and saw all the cool kids were using Python and only a few boffins (UK) or nerds (USA) were using anything else, so they knew who to crib the answers from if they also wanted to look valley cool.
  • deleted 6y ago
    [deleted]
  • mratsim 6y ago
    Seems to me like if you want to look cool nowadays you would use Rust not Python. Different communities have different needs. Data scientists, or physicists, or biologists don't care about memory safety, the programming language is a tool to run a computation for a limited time (not a server). The tool should not get in the way of fast iteration and rapid prototyping which many languages do in varying amount: C with memory management, Haskell with category theory, Rust with borrow checking, C++ with being C++, ... Basically you are criticizing data scientists for choosing to not invent a new thing every time and reusing an established ecosystem.
    • felixyz 6y ago
      So why aren't they using Clojure? I agree that people typically want to get from start to solving their problem as quickly as possible, and don't care about things that don't obviously contribute to that. But that doesn't mean they are perfect rational agents, or immune to cargo-culting, or that a little time invested in learning something else wouldn't pay off.
      • mratsim 6y ago
• neutronicus 6y ago
  I think it needs to be said that Haskell is just not actually all that ergonomic for mutably banging on buffers to solve linear algebra problems. I am a partial differential equations guy, so I have tried. In ML as in PDE, correctness ultimately springs from performance, be it processing more data or using a finer mesh. So at the end of the day impedance matching with imperative linear algebra is pretty much a must.
• mratsim 6y ago
  AI, NLP and ML are bottleneck first by memory accesses and second by CPU. Whatever language you end up using, the low-level implementation must allow you to precisely control memory layout. This is what is being done with C++ + Python but pure Elixir doesn't allow you to control memory layout and will kill your performance.
• nromiun 6y ago
  Two things. Effort and support. You can probably wrap BLAS in any language and make a framework out of that. With some more effort you can add support for GPUs and make your framework much faster. But why would you when most of that is already available? Some might really hate Python but learning Python just for the framework would probably be easier then making a ML framework for the language you like. And Python wasn't even dominant until Tensorflow. Torch used to be a good competetor. Many people make the mistake of promoting the language instead of the framework. And the next thing is support. When some big company releases a ML framework and starts using it for themselves people start using it for their own projects. It's mostly because people think that it will last longer then a month. There were lots of dead python ML projects before Tensorflow. So if you can get a big company to release a ML framework for your favourite language it will surely gain at least some market share. The language almost doesn't matter. I say almost because big companies are more likely to choose popular languages.
• ken 6y ago
  I've seen a definite shift in what "artificial intelligence" means. Norvig's PAIP is all about manipulating symbols. It's so fundamental that in any other language you'd have to Greenspun an interpreter first, just to solve the same problems in the same way. Today, AI seems to mostly mean matrices. If that's all you need, then you can do it in any language, especially if you're just wrapping old C or Fortran libraries. I haven't done any new-style AI but I get the feeling that most "research" is in finding new nails to pound with the same hammer. That's a legitimate area of research, but it also should be no surprise that the concept of "AI research" expanded so much in recent years because it changed from "learn to build your own lightsaber" to "pick up a hammer and hit something with it".
  • newen 6y ago
    There seems to be a misconception among non machine learning programmers that ML is all about multiplying matrices. In reality, there is a lot of architecture involved and having a programming languages that facilitates that would be enormously helpful. Yes, there is a lot of multiplying matrices. But you need to think of a matrix as like a <div> or something and multiplying matrices and placing two <div>s next to each other. Then, think of how to create a dynamic website from that. There is a lot of architecturing involved other than multiplying matrices.
  • thesz 6y ago
    Behind matrices, there is a lot of symbolic processing in the data cleanup part. Most of the time, Python suffices. Most of the time, people does not leave Python when it is not satisfactory. They stick with it because they have no clue, most of the time. For example, it is not quite natural for Python folks to even think about creating rewrite rules (as in Haskell) for programs to stay at higher level of expression. E.g., if you use dictionary with default value, why not lift all operations up (inline, so to say) and avoid calling these that are overloaded? It bit me many times, yet other users of Python just gave me blank stare when I tried to explain these things.
    • ianandrich 6y ago
      Ooh, good observation. I never really thought about how many problems I've had explaining why I'm lifting something or how it makes things better.
• softwarejosh 6y ago
  libraries, examples, classes, scaling architecture, gpu integration, ease of syntax. mostly the same reason new developers dont use functional languages for anything else either.
• DonaldPShimoda 6y ago
  > why aren't the "more expressive" (which in this post seems to be a synonym for "functional") Nobody else seems to have tackled this specific issue so I'd like to. Matt's use of "expressive" is not whimsical or full of personal ideology. He uses Felleisen's definition of "expressiveness" when comparing programming languages [0][1]. The work is a continuation of Landin's earlier efforts to formalize language comparison. From the text (where Felleisen is comparing two languages of "different size", which means that one provides more first-class "constructs" than the other): > Here "more expressive" means that the translation of a program with occurrences of one of the constructs c_i to the smaller language requires a global reorganization of the entire program. The paper goes on to provide a mathematical framework by which to make these comparisons. The languages Matt lists are more expressive than many non-functional languages for various reasons. Each of them has some functionality that, if you were to remove it, would require a "global reorganization of the entire program" to achieve the same goal. For example: if you have a Haskell program and remove automatic currying, you may now have to rewrite almost all of your functions and call sites to accommodate (perhaps by explicitly encoding delays, or by using continuations, etc.). Python does not feature automatic currying, so in this respect Haskell is more expressive than Python — there is a method in Haskell in which you can express your code which you cannot use in Python. Note that the relationship is not predicated entirely upon individual constructs: it is possible for two languages to be compared and to find that each has expressive capability that the other lacks. However, by and large the functional languages Matt lists have focused specifically on increases in expressive power compared to the more "traditional" languages that most programmers use daily. (Haskell did this explicitly by design, but the others Matt lists came to it by way of exploring alternate approaches to problems and arrived at increased expressiveness incidentally.) Later in the paper, Felleisen touches on Turing completeness (a common point brought up in these discussions): > If a programming language can represent all computable functions (on the integers), it contains a functionally equivalent counterpart to each program in a more expressive language. This raises the question as to what advantages there are to programming in the more expressive language when equivalent programs in the simpler language already exist. [...] > > [...W]e have come to the conclusion that programs in less expressive languages exhibit repeated occurrences of programming patterns, and that this pattern-oriented style is detrimental to the programming process. So this provides motivation for why one might want to investigate these "more expressive" languages. Being able to reduce code duplication is (almost) always a good thing! --- [0] https://doi.org/10.1016/0167-6423(91)90036-W https://doi.org/10.1016/0167-6423(91)90036-W [1] I know this to be the case because I've talked to Matt about this very article. :) The Felleisen paper is pretty popular in the PL communities Matt used to hang out in.