5 ms·

I get the sense that the Julia guys are doing very well on solving interesting, hard technical problems like AD. But maybe less well at “dull” problems like goo

by dash2 5y ago

I get the sense that the Julia guys are doing very well on solving interesting, hard technical problems like AD. But maybe less well at “dull” problems like good documentation and API design. That might bias them towards being a language for research rather than for production. Still very interested to see how things pan out.

mountainriver 5y ago
Yes and yes. Julia has a pretty bad developer experience. My biggest concern is that when you bring this up to the Julia crowd they just argue that it’s actually great and you just don’t get it. The Julia community feels a lot like the Scala community to me. Lots of proving why the theories are right, but in production it’s a mess
- pjmlp 5y ago
  Mostly because in regards to where they are coming from the pain points lie elsewhere.
isoprophlex 5y ago
Exactly. To me they have a rather big blind spot when it comes to reaching out to users with docs, and understanding their user needs, exactly because they focus so much on these hard challenges. You can see it in pt 2: "where is the Julia ML ecosystem currently inferior?". They identify some gap with using cuda optimized kernels, where Julia doesn't make use of built in cuda magic; a solution is proposed where users might manually override compiler behavior (if I understand correctly). However, no mention at all about the gap in dev experience. Some examples: - TERRIFIC efforts of various groups in porting ml models from research to easy-to-use torch code - the super ergonomic data wrangling tools in torch. Practically speaking, ml ecosystems need quality interop: do i get my data in and out of the ml framework in a sensible way - the ease of getting started building your first toy ML model in torch because the docs and the API are so great Getting the ball rolling, adoption-wise, means overcoming the cost of switching to something new. And I don't see that happening soon because python+torch is just so comfy.
oxff 5y ago
I took a quick look at it a while back and this seems correct. They are very hard to actually use, and the tooling seems to be lacking severely.
phonebucket 5y ago
> But maybe less well at “dull” problems like good documentation and API design. I'm inclined to agree with you here. To my mind, this means that there's ample scope for developers from many backgrounds to make positive contributions to the Julia ecosystem.
mistrial9 5y ago
I sat next to an active Julia dev from MIT for a semester, not long ago. He was a lot of what you would expect - accurate, on-time results, tending towards overwork, but on track. I like the Julia project. In this guy's defense, the environment is competitive by design. Some people do get grants and rewards for "dull" work but you know they always get those high marks for solving known domain problem, for real. "Docs lag behind" is fair to observe, but the forge that creates this correct, disciplined work makes the results, too. Other "soft" academically-rooted projects ultimately did produce docs, there is no reason to believe that Julia is different there. Personally, I think you do need to pay some living stipend to translators and core authors, within the context of their physical lives. The "libre" movement is real, and has to mature too.
- bmitc 5y ago
  > I think you do need to pay some living stipend to translators and core authors I agree. Institutions like MIT have the monetary resources to support things like this, but they just don't want to. Universities have still not figured out a great model to hire people from industry to help them with their problems. Some places, like the Broad Institute have.
BadInformatics 5y ago
I think it's more complicated than that. The projects that are getting the funding are usually the hard, technical ones, but that funding also supports better docs + more time for API design. This doesn't apply to bleeding edge stuff, but look back through the core SciML libraries and there's no shortage of effort directed towards "dull" stuff like docs + improving compile times. Likewise for the core language: a lot of recent work is bread and butter engineering like (again) improving compile times, filing rough edges off of APIs and (gradually) tackling the deployment story. Now, one area where this dull problem work isn't as noticeable is on the "core" deep learning libraries (Flux and Zygote). AFAICT those two haven't received any significant funding for a couple of years, and there is at most 1 full time, active contributor for both of them. Compare with JAX or even higher-level wrapper libraries like Flax, Haiku or PyTorch Lightning, which have 5-10+ full time core devs. Given this, is it surprising that progress on anything (including docs + interface design) is slow?
atty 5y ago
I feel similarly. I love playing around with Julia, and it’s nice for academic one offs, but I wouldn’t choose to use it for a serious deployment at my job. And the interesting part about ML is that industry is very close to academia relative to other fields. So that predisposes a lot of academia to using SKLearn/Pytorch/Jax etc., because that’s what industry is using, and it’s getting the most development.
credit_guy 5y ago
Documentation is a hard problem. Very hard. Julia actually does quite ok on that front. Here's the 1400 page Julia manual https://raw.githubusercontent.com/JuliaLang/docs.julialang.org/assets/julia-1.7.1.pdf https://raw.githubusercontent.com/JuliaLang/docs.julialang.o... What other language do you have in mind that has better documentation?
- sundarurfriend 5y ago
  I don't think they're talking about the language manual itself, which is certainly very good (though I have my quibbles with it), but the quality of documentation for packages/package ecosystems. Those vary a lot in quality: some good, some incomplete, some hard to access (another subthread here mentions cases where you have to know the implementation details of a package to know where to look up the docs), some poorly organized, some outdated, etc. It's not an insurmountable problem, just adds additional friction to the process and can make the experience more frustrating than it needs to be.

by dash2 5y ago

mountainriver 5y ago

Yes and yes. Julia has a pretty bad developer experience. My biggest concern is that when you bring this up to the Julia crowd they just argue that it’s actually great and you just don’t get it. The Julia community feels a lot like the Scala community to me. Lots of proving why the theories are right, but in production it’s a mess

pjmlp 5y ago
Mostly because in regards to where they are coming from the pain points lie elsewhere.

isoprophlex 5y ago

Exactly. To me they have a rather big blind spot when it comes to reaching out to users with docs, and understanding their user needs, exactly because they focus so much on these hard challenges. You can see it in pt 2: "where is the Julia ML ecosystem currently inferior?". They identify some gap with using cuda optimized kernels, where Julia doesn't make use of built in cuda magic; a solution is proposed where users might manually override compiler behavior (if I understand correctly). However, no mention at all about the gap in dev experience. Some examples: - TERRIFIC efforts of various groups in porting ml models from research to easy-to-use torch code - the super ergonomic data wrangling tools in torch. Practically speaking, ml ecosystems need quality interop: do i get my data in and out of the ml framework in a sensible way - the ease of getting started building your first toy ML model in torch because the docs and the API are so great Getting the ball rolling, adoption-wise, means overcoming the cost of switching to something new. And I don't see that happening soon because python+torch is just so comfy.

oxff 5y ago

I took a quick look at it a while back and this seems correct. They are very hard to actually use, and the tooling seems to be lacking severely.

phonebucket 5y ago

> But maybe less well at “dull” problems like good documentation and API design. I'm inclined to agree with you here. To my mind, this means that there's ample scope for developers from many backgrounds to make positive contributions to the Julia ecosystem.

mistrial9 5y ago

I sat next to an active Julia dev from MIT for a semester, not long ago. He was a lot of what you would expect - accurate, on-time results, tending towards overwork, but on track. I like the Julia project. In this guy's defense, the environment is competitive by design. Some people do get grants and rewards for "dull" work but you know they always get those high marks for solving known domain problem, for real. "Docs lag behind" is fair to observe, but the forge that creates this correct, disciplined work makes the results, too. Other "soft" academically-rooted projects ultimately did produce docs, there is no reason to believe that Julia is different there. Personally, I think you do need to pay some living stipend to translators and core authors, within the context of their physical lives. The "libre" movement is real, and has to mature too.

bmitc 5y ago
> I think you do need to pay some living stipend to translators and core authors I agree. Institutions like MIT have the monetary resources to support things like this, but they just don't want to. Universities have still not figured out a great model to hire people from industry to help them with their problems. Some places, like the Broad Institute have.

BadInformatics 5y ago

I think it's more complicated than that. The projects that are getting the funding are usually the hard, technical ones, but that funding also supports better docs + more time for API design. This doesn't apply to bleeding edge stuff, but look back through the core SciML libraries and there's no shortage of effort directed towards "dull" stuff like docs + improving compile times. Likewise for the core language: a lot of recent work is bread and butter engineering like (again) improving compile times, filing rough edges off of APIs and (gradually) tackling the deployment story. Now, one area where this dull problem work isn't as noticeable is on the "core" deep learning libraries (Flux and Zygote). AFAICT those two haven't received any significant funding for a couple of years, and there is at most 1 full time, active contributor for both of them. Compare with JAX or even higher-level wrapper libraries like Flax, Haiku or PyTorch Lightning, which have 5-10+ full time core devs. Given this, is it surprising that progress on anything (including docs + interface design) is slow?

atty 5y ago

I feel similarly. I love playing around with Julia, and it’s nice for academic one offs, but I wouldn’t choose to use it for a serious deployment at my job. And the interesting part about ML is that industry is very close to academia relative to other fields. So that predisposes a lot of academia to using SKLearn/Pytorch/Jax etc., because that’s what industry is using, and it’s getting the most development.

credit_guy 5y ago

Documentation is a hard problem. Very hard. Julia actually does quite ok on that front. Here's the 1400 page Julia manual https://raw.githubusercontent.com/JuliaLang/docs.julialang.org/assets/julia-1.7.1.pdf https://raw.githubusercontent.com/JuliaLang/docs.julialang.o... What other language do you have in mind that has better documentation?

sundarurfriend 5y ago
I don't think they're talking about the language manual itself, which is certainly very good (though I have my quibbles with it), but the quality of documentation for packages/package ecosystems. Those vary a lot in quality: some good, some incomplete, some hard to access (another subthread here mentions cases where you have to know the implementation details of a package to know where to look up the docs), some poorly organized, some outdated, etc. It's not an insurmountable problem, just adds additional friction to the process and can make the experience more frustrating than it needs to be.