4 ms·

Accelerating scikit-learn is a smart move. At the algorithmic level for every ML use case there is probably x 10 non-ML data science projects. Also, it is good

Accelerating scikit-learn is a smart move. At the algorithmic level for every ML use case there is probably x 10 non-ML data science projects. Also, it is good to have a true community framework that does not depend on the success of the metaverse for funding ;-) The lock-in is an important consideration, but if the scikit-learn API is fully respected it would seem less relevant. It also suggests a pattern for how other hardware vendors could accelerate scikit-learn as a genuine contribution?

medo-bear 5y ago
i personally think that it would be a more interesting move in its foray into hardware acceleration if intel gives first class support to julia
- AstroDogCatcher 5y ago
  Intel are focused on data-parallel C++ for delivering high performance, rightly or wrongly. Julia is one of those "nice in theory" options which has failed to live up to the hype and at this point seems unlikely to unseat python for most use-cases; it just doesn't have a good enough UX when used as a general purpose language.
  - medo-bear 5y ago
    im not sure what you mean by UX in this context, but julias ecosystem for scientific computing (in a broad sense) has been growing tremendously. this is the area from which it wants to unseat python. general purpose programming is secondary. whether it can i don't know. but i definitely dont think its a settled question. python is my daily driver for machine learning work, but i definitely think julia can overtake its place eventually
  - eduardosalaz 5y ago
    Hi, I would love to hear more about your complaints regarding UX for general purpose programming
    - ampdepolymerase 5y ago
      The editor autocompletion and standard library documentation could use a lot of work. The introductory tutorials are overly focused on type theory and details and do not give a good overview of which generic data structures to use in production code. Julia's JIT is very different from other conventional mainstream languages and the process of selecting standard library generic data structures for optimal performance is very poorly documented. There is no Effective Julia style of guide. You either have to wade through infantile tutorials for those with minimal programming experience or several reference books worth of nitpicking on syntax. The actual methods themselves are not well documented and lack examples and usage guidelines. The language and ecosystem do not feel like a project backed by commercial funding, it feels like one of those functional languages out of academia research where the structure and design of the language are more important than actual developer experience. There are many new projects but most are not actively maintained and updated. The language itself feels massive, with syntactic sugar and weird types everywhere. Trying to understand the implementations of other people's Julia code is frustrating, similar to reading a library written in pure C++ templates. Compared to Go/Rust/Dart, Julia feels overly convoluted. Julia literature is structured in a way that seems to heavily encourage you to take regular classes and lectures to learn and pick up the language. It is hard to feel productive from the get-go.
    - AstroDogCatcher 5y ago
      Someone else commented with more detail, but personally I can't get past the package management and the dependency on using the REPL. Rust gets tooling and packages right.
      - disgruntledphd2 5y ago
        >the dependency on using the REPL This is a feature for a lot of Julia's core audience (data scientists like me, who grew up with R).
        medo-bear 5y ago
        exactly right. REPL is a key feature for computational scientists. in physics, for example, the value proposition of python was that it provided an open source alternative to matlab (also a REPL based environment) without sacrificing functionality. i believe that the really revolutionary thing with python was that it provided an extremely fertile ground for open source development of numerical methods that far exceeded what was offered on matlab in syntax that resembled pseudo code (much like matlab). julia's value proposition is all of that, plus a much more performant base language with arguably even better syntax
  - ogrisel 5y ago
    > Intel are focused on data-parallel C++ for delivering high performance, rightly or wrongly. They also invest efforts in making it possible to write high performance kernels in Python using an extension to the numba Python compiler: https://github.com/IntelPython/numba-dppy https://github.com/IntelPython/numba-dppy

medo-bear 5y ago

i personally think that it would be a more interesting move in its foray into hardware acceleration if intel gives first class support to julia

AstroDogCatcher 5y ago
Intel are focused on data-parallel C++ for delivering high performance, rightly or wrongly. Julia is one of those "nice in theory" options which has failed to live up to the hype and at this point seems unlikely to unseat python for most use-cases; it just doesn't have a good enough UX when used as a general purpose language.
- medo-bear 5y ago
  im not sure what you mean by UX in this context, but julias ecosystem for scientific computing (in a broad sense) has been growing tremendously. this is the area from which it wants to unseat python. general purpose programming is secondary. whether it can i don't know. but i definitely dont think its a settled question. python is my daily driver for machine learning work, but i definitely think julia can overtake its place eventually
- eduardosalaz 5y ago
  Hi, I would love to hear more about your complaints regarding UX for general purpose programming
  - ampdepolymerase 5y ago
    The editor autocompletion and standard library documentation could use a lot of work. The introductory tutorials are overly focused on type theory and details and do not give a good overview of which generic data structures to use in production code. Julia's JIT is very different from other conventional mainstream languages and the process of selecting standard library generic data structures for optimal performance is very poorly documented. There is no Effective Julia style of guide. You either have to wade through infantile tutorials for those with minimal programming experience or several reference books worth of nitpicking on syntax. The actual methods themselves are not well documented and lack examples and usage guidelines. The language and ecosystem do not feel like a project backed by commercial funding, it feels like one of those functional languages out of academia research where the structure and design of the language are more important than actual developer experience. There are many new projects but most are not actively maintained and updated. The language itself feels massive, with syntactic sugar and weird types everywhere. Trying to understand the implementations of other people's Julia code is frustrating, similar to reading a library written in pure C++ templates. Compared to Go/Rust/Dart, Julia feels overly convoluted. Julia literature is structured in a way that seems to heavily encourage you to take regular classes and lectures to learn and pick up the language. It is hard to feel productive from the get-go.
  - AstroDogCatcher 5y ago
    Someone else commented with more detail, but personally I can't get past the package management and the dependency on using the REPL. Rust gets tooling and packages right.
    - disgruntledphd2 5y ago
      >the dependency on using the REPL This is a feature for a lot of Julia's core audience (data scientists like me, who grew up with R).
      - medo-bear 5y ago
        exactly right. REPL is a key feature for computational scientists. in physics, for example, the value proposition of python was that it provided an open source alternative to matlab (also a REPL based environment) without sacrificing functionality. i believe that the really revolutionary thing with python was that it provided an extremely fertile ground for open source development of numerical methods that far exceeded what was offered on matlab in syntax that resembled pseudo code (much like matlab). julia's value proposition is all of that, plus a much more performant base language with arguably even better syntax
- ogrisel 5y ago
  > Intel are focused on data-parallel C++ for delivering high performance, rightly or wrongly. They also invest efforts in making it possible to write high performance kernels in Python using an extension to the numba Python compiler: https://github.com/IntelPython/numba-dppy https://github.com/IntelPython/numba-dppy