10 ms·

Happy to address these points: - Startup performance/memory usage Yes, we are definitely very acutely aware of these. Julia is not currently optimized for f

Happy to address these points: - Startup performance/memory usage Yes, we are definitely very acutely aware of these. Julia is not currently optimized for frequently run short scripts. That's the price on pays for having to bring up the entire runtime system (initializing the compiler, RNG, external libraries etc). The good news is that there will be a solution to this soon, which is to statically compile your julia program. The area where this really comes up for most people using is package load times. We're very actively working on making that faster. - Syntax A little subjective, so not sure how much I can say here. I can say that I'm not a huge fan of our multi-line comment syntax. It's not entirely clear what a better syntax would be though (the original issue on this had some suggestions, but some of them were worse). - One-based indexing I think there has been plenty said on this topic, though interestingly this is one of the only times I've seen the argument made in a way that I actually agree with. That said, I do think there is an easy way to deal with this though. For packages that needs arrays of indices, it would be quite easy to define an `IndexArray` type that does the translation automatically. - String Formatting Yep, you're right, it's a mess. It'll have to be cleaned up. - Unsafe C Interface There's two projects (Clang.jl and Cxx.jl) which can help with this. The former automatically generates ccall definitions for you, the latter just parses the header and generates the call directly. - Slowing down in development I'm really not sure where that impression comes from. Perhaps it is that we're adding fewer features, but rather working on cleaning up existing features. Also, I personally at least have been doing a lot of work outside of base (particularly on the debugger). Not sure. Would love to know.

ihnorton 10y ago
> For packages that needs arrays of indices, it would be quite easy to define an `IndexArray` type that does the translation automatically. (just to add: there is currently an open pull request from a core array-focused developer adding such support to base)
- waldir 10y ago
  I believe this is the PR in question: https://github.com/JuliaLang/julia/pull/16260 https://github.com/JuliaLang/julia/pull/16260
vitaut 10y ago
Thanks for the detailed response. I hope that my post wasn't too harsh, the intent was mostly to attract attention to the current issues not to undermine the great work that you and others have been doing. I'm glad that many of the issues that I mentioned are being addressed. Maybe I'll give Julia another go in some time =). The question of syntax is subjective of course. From the set {C-like, Python, MATLAB} I'd definitely like to see more influence of the first two and less from the last one.
- bmh100 10y ago
  One-based indexing is also used in Fortran, which seems to be used in a great deal of numerical computing even today. Additionally, BLAS/LAPACK is an important linear algebra library written in Fortan. I am somewhat confused by your discussion of startup times. Since Julia is a "programming language for technical computing", what scenario are you imagining where startup times would be a significant concern?
  - jandrese 10y ago
    No less than Dykstra has weighed in on the numbering of arrays. http://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF http://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF Having worked with both, I'm inclined to prefer 0 based addressing in most cases. It's slightly less intuitive, but it generally leads to cleaner code.
    - Avshalom 10y ago
      No less than Dykstra spouted a half hearted list of some properties and then made a proclamation based entirely on aesthetics.
    - banku_brougham 10y ago
      no less than dykstra was convicted of grand theft auto. https://en.wikipedia.org/wiki/Lenny_Dykstra https://en.wikipedia.org/wiki/Lenny_Dykstra I think you mean Dijkstra, who did say that, because of an argument with mathematicians about indexing from 1.
rch 10y ago
I liked how the Template Numerical Toolkit implemented one-based indexing: // Construct matrix. Elements are not initialized. // To initialize all elements to zero use // Matrix<double> a(2, 2, 0.0). Matrix<double> a(2, 2); // Assign elements to first row using // Fortran-style one-based indexing. a(1,1) = 0.5; a(1,2) = 1.0; // Assign elements to second row using // C-style zero-based indexing. a[1][0] = 1.5; a[1][1] = 2.0; -- http://www.b-a-h.com/software/cpp/scppnt.html http://www.b-a-h.com/software/cpp/scppnt.html
- KenoFischer 10y ago
  This syntax is available in julia as well, but I'm not sure it's a great idea to encourage mixing the two indexing behaviors, even if they have different syntax. As I hinted in the original reply, I have seen very few cases where the choice of index offset actually makes a difference. For example, loops over indices generally use `eachindex` which doesn't care about your choice of index base.
  - rch 10y ago
    That's interesting - thanks. I don't think anyone is eager to mix indexing behaviors either.
    - KenoFischer 10y ago
      I just realized that when I said, "this syntax is available", people might have understood it to mean, "you can use this syntax to index arrays" (which is not true for the array type defined in base). What I meant was, "you can define an array type with this indexing behavior without changing the language (since you can override the behavior of both () and [] on a particular type)".
  - ScottPJones 10y ago
    Actually, for the sort of low-level bit twiddling code that I frequently do, 1-based indexing complicates the code greatly. I also need to pass structures that contain indices back and forth to C, so the 1-based indices cause extra overhead to constantly add or subtract the size of the element. I find that it's the largest cause of bugs in my code (which is why I really want Gallium working!)
nickpsecurity 10y ago
Might be as simple as creating a static and dynamic profile for Julia. I suggested this to developers of one language project before. Maybe Julia. A single keyword or declaration near top of the file tells it to compile for fast loading with no need for functions that change things at runtime. Or compiler option.
- KenoFischer 10y ago
  We do have various compiler flags, but the problem on startup is the initialization of external libraries and deserialization of the system image, primarily.
bluecalm 10y ago
The thing about 1 based indexing is that it's a kind of in your face "this is different" decision from the point of view of a programmers of most popular languages. To be honest I wouldn't want to start investing my time into a language where people who proposed 1 based indexing are making design decisions. It's not that I think they are incompetent but it's clear they care way more about some different world than about my programming world and are ready to make my life miserable stating the point. Now, I don't know if people from that different world (Fortran, Matlab, some other languages used in academia maybe) would feel the same way about 0 based indexing but it certainly sends the message to programmers outside those domains.
- chappi42 10y ago
  R and Mathematica are also 1 based. And R really is popular (http://www.tiobe.com/tiobe_index?page=index http://www.tiobe.com/tiobe_index?page=index - ok, popularity is droping right now, likely b/c of Julia ;-))
  - c3534l 10y ago
    Julia seems like it's meant to be friendly to people who know MATLAB, Python, and R (even Fortran). R and Python are really comfortable with working with data science and that's what Julia is geared toward. I have no experience with MATLAB, though. Julia's programming "world" may indeed be a different programming world than C programmers. I'm not defending Julia per se. I've been patiently observing from the sidelines to see how the language shapes up.
    - deleted 10y ago
      [deleted]
  - banku_brougham 10y ago
    R is ranked lower year-on-year, but if you click into the actual rating time series you can see that R is on a tear into new highs of popularity. For that matter so is MATLAB. Julia is still buried in the next 50 list. BTW i'm a julia proponent, I'm just starting into it but I like a lot about it.
SolarNet 10y ago
> - Slowing down in development I think part of this problem is the community, multiple people have reported having bad interactions with core language devs. Also the policies for inclusion of features, how to propose features, how decisions are made with respect to the code base, etc. all seem poorly documented.
- KenoFischer 10y ago
  > I think part of this problem is the community, multiple people have reported having bad interactions with core language devs. I can only think of a couple of instance where I've heard anything like this. The first and most prominent is Dan Luu. I'm very sad about losing him from the community. I've met him a couple of times in person, I still read his blog and I very much respect him. I don't think any of us have the full story of what happened there, but I sincerely hope that time may be able to smooth things over. The second was in an HN comment that I just went back to find, and discovered that is was you. I'm sorry if you had a bad experience with the community. I know my perspective is biased, but I've rarely interacted with a community that's as passionate, helpful and friendly as the julia community (LLVM is up there as well). As for documenting the process, I agree there could be some improvement there. We do have a contributors guide (CONTRIBUTING.md in the main repository). As for policies for feature inclusion, we've considered having more formal code owners for parts of the code base (right now there's mostly de facto code owners for various parts), but haven't found it necessary so far. I definitely expect some of this to be discussed at JuliaCon in June.
  - SolarNet 10y ago
    I remember plenty of cases I got simply from observing the Julia repo for a few months, I could list them if you like... But I've been told by Julia contributors before that discussing this on HN is not an appropriate place. So we can discuss it, where I start linking to public examples on HN, or you can ignore my opinion on a problematic community and we can not discuss it. As to the code standards, my problem isn't with the CONTRIBUTING.md; that seems fine. It's how people make long term suggestions about fixing lasting problems. As an example people who suggest refactoring large sections of Julia's expansive base library (1600 symbols? a problem that causes parts of the slow startup time and memory usage) into other libraries (which would improve batteries included, etc. features) are summarily ignored / shouted down / (predicated on my previous paragraph) banned. Important improvements to the language go in circles (often on JULEP tagged issues) especially any large improvement that would require multiple people and lots of dev work and seem to gain little traction unless a core dev just takes the time to just do it; rather than a timeline to reach consensus and then design a plan for implementing it. As smaller examples. There was a push to increase test coverage and then months later commits plummeted the code coverage numbers and no one seemed to care. Sections of important code are uncommented, undocumented, and (mostly) untested on purpose. There are modularity problems and software architecture limitations inherent in the language design that go unaddressed. Unpredictable un-typed exceptions, namespaces/modules being hard to use for non-trivial designs, "interfaces" that aren't really. And these are just the ones without serious work behind them I can remember off the top of my head (the debugger and threading I acknowledge being worked on seriously). Though things being worked on seriously suffer problems as I mentioned earlier. Issues like these cause many of the original poster's problems. Julia is a great language, but the community seems unable to take the steps it needs to to actually make a "Tier 1" general purpose language (e.g. like python, go, clojure etc.). Setup a serious set of policies regarding language proposals; setup a transparent community policy and actually follow it; acknowledge that certain development idiosyncrasies aren't going to fly anymore; take software architecture concerns seriously; remove the toxic core language dev (that to my count has cost you at least 4 serious contributors if not more). Julia is likely "slowing down" relative to the expected development (which many people have observed) likely (by my thinking) because it is failing to attract/retain the developers it should be as a promising language (in my opinion) because of it's poor development and community standards. But hey, these are just my suggestions, I don't have any skin in this game. I just wish Julia was as good as it promised to be, and I'm a bit bitter it isn't for human, rather than technological or financial, reasons. Edit: Sorry I expanded, it felt unfair to not be more complete about it.
ScottPJones 10y ago
Keno - take a look at what I did in https://github.com/ScottPJones/StringUtils.jl https://github.com/ScottPJones/StringUtils.jl (building on the backs of giants, such as Tom Breloff, Dahua Lin, and others, incorporating some code from Formatting.jl). It solves problems I've had both with string literals, formatting, and outputting funny Unicode characters (without having to use UTF-8 in my source code). I really like your Cxx.jl, that does seem to answer the issue raised here (are there any good features in pybind11 though that aren't already in Cxx.jl? that might be a good source for ideas on how to improve Cxx.jl if not)
vitaut 10y ago
Regarding the slowdown in development, perhaps it's a seasonal variation connected to GSoC. For example, notice a drop in postings since October in https://groups.google.com/forum/#!aboutgroup/julia-dev https://groups.google.com/forum/#!aboutgroup/julia-dev . This seems to be more or less consistent with the observation in http://www.davideaversa.it/2015/12/the-most-promising-languages-of-2016/ http://www.davideaversa.it/2015/12/the-most-promising-langua..., but that's just a guess.
- tavert 10y ago
  No summer of code students have had projects that involved work primarily on the core language since 2013. Not that many posts happen on julia-dev because most of the development communication happens on github.
- ihnorton 10y ago
  For better or worse, most development discussion happens on GitHub rather than julia-dev. I've been catching up on GitHub threads recently, and the progress towards 0.5 is really impressive -- both in the language itself, and in the tooling (especially the debugger). I think the nature of priorities has changed somewhat as the language matures, but I disagree that development has slowed. (disclosure: I'm a Julia committer, but have been a bit out of the loop for some months now) There is certainly a need for better interim communication on progress between releases, to let people know what is happening without reading every GitHub thread.
- ScottPJones 10y ago
  I do think it's seasonal, however not connected to GSoC. Many of the major contributors are (or were) students (Julia seems to be detrimental to people finishing their PhDs, except in the case of Jeff Bezanson, where in some sense he crowdsourced a lot of the research, as Julia was the topic of his thesis ;-) ) Many who aren't students are professors. Definite drop off when people are taking classes, exams, teaching, etc.

ihnorton 10y ago

> For packages that needs arrays of indices, it would be quite easy to define an `IndexArray` type that does the translation automatically. (just to add: there is currently an open pull request from a core array-focused developer adding such support to base)

waldir 10y ago
I believe this is the PR in question: https://github.com/JuliaLang/julia/pull/16260 https://github.com/JuliaLang/julia/pull/16260

vitaut 10y ago

Thanks for the detailed response. I hope that my post wasn't too harsh, the intent was mostly to attract attention to the current issues not to undermine the great work that you and others have been doing. I'm glad that many of the issues that I mentioned are being addressed. Maybe I'll give Julia another go in some time =). The question of syntax is subjective of course. From the set {C-like, Python, MATLAB} I'd definitely like to see more influence of the first two and less from the last one.

bmh100 10y ago
One-based indexing is also used in Fortran, which seems to be used in a great deal of numerical computing even today. Additionally, BLAS/LAPACK is an important linear algebra library written in Fortan. I am somewhat confused by your discussion of startup times. Since Julia is a "programming language for technical computing", what scenario are you imagining where startup times would be a significant concern?
- jandrese 10y ago
  No less than Dykstra has weighed in on the numbering of arrays. http://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF http://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF Having worked with both, I'm inclined to prefer 0 based addressing in most cases. It's slightly less intuitive, but it generally leads to cleaner code.
  - Avshalom 10y ago
    No less than Dykstra spouted a half hearted list of some properties and then made a proclamation based entirely on aesthetics.
  - banku_brougham 10y ago
    no less than dykstra was convicted of grand theft auto. https://en.wikipedia.org/wiki/Lenny_Dykstra https://en.wikipedia.org/wiki/Lenny_Dykstra I think you mean Dijkstra, who did say that, because of an argument with mathematicians about indexing from 1.

rch 10y ago

I liked how the Template Numerical Toolkit implemented one-based indexing: // Construct matrix. Elements are not initialized. // To initialize all elements to zero use // Matrix<double> a(2, 2, 0.0). Matrix<double> a(2, 2); // Assign elements to first row using // Fortran-style one-based indexing. a(1,1) = 0.5; a(1,2) = 1.0; // Assign elements to second row using // C-style zero-based indexing. a[1][0] = 1.5; a[1][1] = 2.0; -- http://www.b-a-h.com/software/cpp/scppnt.html http://www.b-a-h.com/software/cpp/scppnt.html

KenoFischer 10y ago
This syntax is available in julia as well, but I'm not sure it's a great idea to encourage mixing the two indexing behaviors, even if they have different syntax. As I hinted in the original reply, I have seen very few cases where the choice of index offset actually makes a difference. For example, loops over indices generally use `eachindex` which doesn't care about your choice of index base.
- rch 10y ago
  That's interesting - thanks. I don't think anyone is eager to mix indexing behaviors either.
  - KenoFischer 10y ago
    I just realized that when I said, "this syntax is available", people might have understood it to mean, "you can use this syntax to index arrays" (which is not true for the array type defined in base). What I meant was, "you can define an array type with this indexing behavior without changing the language (since you can override the behavior of both () and [] on a particular type)".
- ScottPJones 10y ago
  Actually, for the sort of low-level bit twiddling code that I frequently do, 1-based indexing complicates the code greatly. I also need to pass structures that contain indices back and forth to C, so the 1-based indices cause extra overhead to constantly add or subtract the size of the element. I find that it's the largest cause of bugs in my code (which is why I really want Gallium working!)

nickpsecurity 10y ago

Might be as simple as creating a static and dynamic profile for Julia. I suggested this to developers of one language project before. Maybe Julia. A single keyword or declaration near top of the file tells it to compile for fast loading with no need for functions that change things at runtime. Or compiler option.

KenoFischer 10y ago
We do have various compiler flags, but the problem on startup is the initialization of external libraries and deserialization of the system image, primarily.

bluecalm 10y ago

The thing about 1 based indexing is that it's a kind of in your face "this is different" decision from the point of view of a programmers of most popular languages. To be honest I wouldn't want to start investing my time into a language where people who proposed 1 based indexing are making design decisions. It's not that I think they are incompetent but it's clear they care way more about some different world than about my programming world and are ready to make my life miserable stating the point. Now, I don't know if people from that different world (Fortran, Matlab, some other languages used in academia maybe) would feel the same way about 0 based indexing but it certainly sends the message to programmers outside those domains.

chappi42 10y ago
R and Mathematica are also 1 based. And R really is popular (http://www.tiobe.com/tiobe_index?page=index http://www.tiobe.com/tiobe_index?page=index - ok, popularity is droping right now, likely b/c of Julia ;-))
- c3534l 10y ago
  Julia seems like it's meant to be friendly to people who know MATLAB, Python, and R (even Fortran). R and Python are really comfortable with working with data science and that's what Julia is geared toward. I have no experience with MATLAB, though. Julia's programming "world" may indeed be a different programming world than C programmers. I'm not defending Julia per se. I've been patiently observing from the sidelines to see how the language shapes up.
  - deleted 10y ago
    [deleted]
- banku_brougham 10y ago
  R is ranked lower year-on-year, but if you click into the actual rating time series you can see that R is on a tear into new highs of popularity. For that matter so is MATLAB. Julia is still buried in the next 50 list. BTW i'm a julia proponent, I'm just starting into it but I like a lot about it.

SolarNet 10y ago

> - Slowing down in development I think part of this problem is the community, multiple people have reported having bad interactions with core language devs. Also the policies for inclusion of features, how to propose features, how decisions are made with respect to the code base, etc. all seem poorly documented.

KenoFischer 10y ago
> I think part of this problem is the community, multiple people have reported having bad interactions with core language devs. I can only think of a couple of instance where I've heard anything like this. The first and most prominent is Dan Luu. I'm very sad about losing him from the community. I've met him a couple of times in person, I still read his blog and I very much respect him. I don't think any of us have the full story of what happened there, but I sincerely hope that time may be able to smooth things over. The second was in an HN comment that I just went back to find, and discovered that is was you. I'm sorry if you had a bad experience with the community. I know my perspective is biased, but I've rarely interacted with a community that's as passionate, helpful and friendly as the julia community (LLVM is up there as well). As for documenting the process, I agree there could be some improvement there. We do have a contributors guide (CONTRIBUTING.md in the main repository). As for policies for feature inclusion, we've considered having more formal code owners for parts of the code base (right now there's mostly de facto code owners for various parts), but haven't found it necessary so far. I definitely expect some of this to be discussed at JuliaCon in June.
- SolarNet 10y ago
  I remember plenty of cases I got simply from observing the Julia repo for a few months, I could list them if you like... But I've been told by Julia contributors before that discussing this on HN is not an appropriate place. So we can discuss it, where I start linking to public examples on HN, or you can ignore my opinion on a problematic community and we can not discuss it. As to the code standards, my problem isn't with the CONTRIBUTING.md; that seems fine. It's how people make long term suggestions about fixing lasting problems. As an example people who suggest refactoring large sections of Julia's expansive base library (1600 symbols? a problem that causes parts of the slow startup time and memory usage) into other libraries (which would improve batteries included, etc. features) are summarily ignored / shouted down / (predicated on my previous paragraph) banned. Important improvements to the language go in circles (often on JULEP tagged issues) especially any large improvement that would require multiple people and lots of dev work and seem to gain little traction unless a core dev just takes the time to just do it; rather than a timeline to reach consensus and then design a plan for implementing it. As smaller examples. There was a push to increase test coverage and then months later commits plummeted the code coverage numbers and no one seemed to care. Sections of important code are uncommented, undocumented, and (mostly) untested on purpose. There are modularity problems and software architecture limitations inherent in the language design that go unaddressed. Unpredictable un-typed exceptions, namespaces/modules being hard to use for non-trivial designs, "interfaces" that aren't really. And these are just the ones without serious work behind them I can remember off the top of my head (the debugger and threading I acknowledge being worked on seriously). Though things being worked on seriously suffer problems as I mentioned earlier. Issues like these cause many of the original poster's problems. Julia is a great language, but the community seems unable to take the steps it needs to to actually make a "Tier 1" general purpose language (e.g. like python, go, clojure etc.). Setup a serious set of policies regarding language proposals; setup a transparent community policy and actually follow it; acknowledge that certain development idiosyncrasies aren't going to fly anymore; take software architecture concerns seriously; remove the toxic core language dev (that to my count has cost you at least 4 serious contributors if not more). Julia is likely "slowing down" relative to the expected development (which many people have observed) likely (by my thinking) because it is failing to attract/retain the developers it should be as a promising language (in my opinion) because of it's poor development and community standards. But hey, these are just my suggestions, I don't have any skin in this game. I just wish Julia was as good as it promised to be, and I'm a bit bitter it isn't for human, rather than technological or financial, reasons. Edit: Sorry I expanded, it felt unfair to not be more complete about it.

ScottPJones 10y ago

Keno - take a look at what I did in https://github.com/ScottPJones/StringUtils.jl https://github.com/ScottPJones/StringUtils.jl (building on the backs of giants, such as Tom Breloff, Dahua Lin, and others, incorporating some code from Formatting.jl). It solves problems I've had both with string literals, formatting, and outputting funny Unicode characters (without having to use UTF-8 in my source code). I really like your Cxx.jl, that does seem to answer the issue raised here (are there any good features in pybind11 though that aren't already in Cxx.jl? that might be a good source for ideas on how to improve Cxx.jl if not)

vitaut 10y ago

Regarding the slowdown in development, perhaps it's a seasonal variation connected to GSoC. For example, notice a drop in postings since October in https://groups.google.com/forum/#!aboutgroup/julia-dev https://groups.google.com/forum/#!aboutgroup/julia-dev . This seems to be more or less consistent with the observation in http://www.davideaversa.it/2015/12/the-most-promising-languages-of-2016/ http://www.davideaversa.it/2015/12/the-most-promising-langua..., but that's just a guess.

tavert 10y ago
No summer of code students have had projects that involved work primarily on the core language since 2013. Not that many posts happen on julia-dev because most of the development communication happens on github.
ihnorton 10y ago
For better or worse, most development discussion happens on GitHub rather than julia-dev. I've been catching up on GitHub threads recently, and the progress towards 0.5 is really impressive -- both in the language itself, and in the tooling (especially the debugger). I think the nature of priorities has changed somewhat as the language matures, but I disagree that development has slowed. (disclosure: I'm a Julia committer, but have been a bit out of the loop for some months now) There is certainly a need for better interim communication on progress between releases, to let people know what is happening without reading every GitHub thread.
ScottPJones 10y ago
I do think it's seasonal, however not connected to GSoC. Many of the major contributors are (or were) students (Julia seems to be detrimental to people finishing their PhDs, except in the case of Jeff Bezanson, where in some sense he crowdsourced a lot of the research, as Julia was the topic of his thesis ;-) ) Many who aren't students are professors. Definite drop off when people are taking classes, exams, teaching, etc.