21 ms·

People are increasingly running out of reasons to not use Julia. It really is the future for data science and machine learning. No let me correct that. It reall

People are increasingly running out of reasons to not use Julia. It really is the future for data science and machine learning. No let me correct that. It really could replace anything R, Python, Perl, Ruby etc is used for today. Not right now as packages, larger communities etc need to be developed. But long term solutions the potential is obvious. A friendly, powerful and high performance dynamic language ought to have a very broad appeal.

packetlost 3y ago
Having 1-indexed arrays is unforgivable.
- huijzer 3y ago
  Okay. I’ll bite. Why?
  - xedrac 3y ago
    I used Julia for a project back in 2016, and had the same adverse reaction to 1-based indexing. It's the same reason I have a hard time with Lua. Why? I suppose it's all rather superficial, but it's one of those things that just grates on me and I'll avoid it if I can. I'm not sure there's a logical explanation, but I've found the sentiment to be rather common among developers.
    - sweezyjeezy 3y ago
      Same, but probably it's more of a familiarity bias rather than a logical one. I think there is a strong argument to index from 1 as is done in MATLAB, R etc. since it matches the way most people refer and think about lists in reality. A brave decision from the authors though, I imagine they would have got less flak matching the more popular 0-index languages.
      - sundarurfriend 3y ago
        > I think they would have got less flak matching the more popular 0-index languages. Possibly, though those of us already in the space would have found it a (slightly) less pleasant language then.
  - gcmrtc 3y ago
    This is the opinion of Dykstra: https://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF https://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF
    - Iwan-Zotow 3y ago
      sure, [0...N) ranges are composable, to be short
      - sundarurfriend 3y ago
        That is such a weird argument to me. I can count in the fingers of one hand the number of times in my life that I've had to compose ranges, it seems a strange thing to optimize for.
        Iwan-Zotow 3y ago
        Splitting and combining back ranges is very common activity since about Core 2 Duo, when multi-core getting to be a normal. I mean, computing mean - split range, get result per split and combine it back, and so on and so forth
      - npalli 3y ago
        You seem to have a weird fetish for composable ranges seeing your history of comments on exactly this topic (even this thread!). The number of times I have had to think about composability as you mention is close to zero. In fact, the off-by-one errors with C class of languages due to the open ended nature of the interval have bit me about 3-4 orders of magnitude more. There is an entire class of security bugs because of this. I find this obsession very strange, YMMV obviously.
- leephillips 3y ago
  Yes, that’s probably why Fortran was never used seriously.
- bobbylarrybobby 3y ago
  Not just that, but ranges are inclusive on both ends.
  - sundarurfriend 3y ago
    Ha, I was thinking of replying to your parent comment, "Half-open ranges are such an unpleasant thing as to be unforgivable to me. Different subjective preferences." I've always found 0-indexing mildly unpleasant too, even though C was the language I learnt programming with, and felt like I found home when I came across 1-indexed languages.
    - bobbylarrybobby 3y ago
      With half open ranges, the length of the range is the end minus the beginning, which is nice. It's also much more simpler to write an empty half inclusive range. For instance if left=right then left..right would be the empty range. Whereas in Julia I'd need... left:(right-1)? But is 1:0 the empty range or the right-to-left inclusive range [1,0]? Very confusing and hard to work with all around.
      - sundarurfriend 3y ago
        > With half open ranges, the length of the range is the end minus the beginning, which is nice. But I already know from real life that if there's maintanence in blocks 5 to 15, that's eleven blocks under maintanence. With half-open ranges, it once again introduces confusion and makes things unintuitive. We can each find countless examples where each style comes out better, for eg `1:N` most often expresses the intent better than `range(N+1)`. I find people who prefer half-open ranges to be weird and incomprehensible, you probably find my preferences the same, I just don't like the categorical statement often made in this regard that one is inherently and universally superior to the other.
        bobbylarrybobby 3y ago
        Yes, range(n+1) is bad. I hate python as well. You want something like Rust’s m..n and m..=n to have the choice. Anyway, if you have half open ranges, it's trivial to get a closed range (add one to the right endpoint), but going the other direction is not so simple.
- goatlover 3y ago
  This again. Fortran, Cobol, Smalltalk, R, Matlab, APL, Wofram, Lua. Its primary use is scientific computing. It's like the whitespace in Python debate that everyone gave up because the language got so popular that people stopped caring.
- upbeat_general 3y ago
  Agreed. I thought the same thing about Python's indenting originally (and still do). But the ecosystem was enough to overcome it for me eventually.
- pjmlp 3y ago
  Plenty of languages have it, so what, it is too much to ask for in skillets?
  - Iwan-Zotow 3y ago
    Julia ranges are not composable, that's it You cannot do easily [0...N) = [0...N/2) + [N/2...N)
    - webshit2 3y ago
      Idk typing +1 seems easy enough... 1:N == union(1:(N÷2), (N÷2+1):N). But really, these discussions are funny to me - each side pretending their convention is how God intended indexing to be done. You see it's naturally composable because N/2 shows up twice, which is really the perfect amount of times to show up, and as you recall I just defined composable in that way (not to mention it matches the fact that N/2 occurs twice in [0, N)!)
      - Iwan-Zotow 3y ago
        In python this is natural way to do ranges, and in Julia you have to remember this pesky +1 In Python (C, C++ with whole STL) you could split in the middle, or at any M [0...N)=[0...M)+[M...N) You could split it k times at any boundaries, still [0...N)=[0...M1)+...+[Mk...N) Another important thing is that number of elements to process is exactly the difference between last and first index of the range https://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF https://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF
        nequo 3y ago
        > number of elements to process is exactly the difference between last and first index of the range And in Julia, it’s length(a:b) which also works with non-unit step sizes like length(a:x:b) The intent of these expressions seems clearer than the intent of `b-a` and `ceiling((b-a)/x)` with your proposed approach in a zero-indexed language.
- Rarebox 3y ago
  It's jarring initially, but becomes natural very quickly. Writing loops like "for i in 1:length(arr) ... end" is pretty neat compared to C++ or even python. Plus in math sequences typically start at index 1.
  - threatofrain 3y ago
    > Plus in math sequences typically start at index 1. I'm not quite sure that this is the case enough to say "typically". In terms of undergraduate exposure to math, I think more people have taken Calculus (or Analysis) than Linear Algebra, and I think Calculus textbooks tend to index from 0 while Linear Algebra textbooks tend to index from 1.
  - leephillips 3y ago
    You should never write loops that way, at least in code you're going to share (assuming that you’re going to have some arr[i] in the loop body—if not, you would just do "for e in arr"). Assuming that arrays start at 1 is a source of occasional bugs in public packages. The existence of OffsetArrays means that arrays can start at any index (so for people who get nauseated by 1-based arrays, you can change it). Instead, write "for i in eachindex(arr)". In fact, Julia’s array syntax means you can loop over and manipulate arrays without using indexes much of the time, so don’t even need to know how they’re based.
  - inkyoto 3y ago
    I would go as far as to say that it is now largely an archaic idiom to write «for i in 1:length(arr) ... end», and there is no reason to write such a code to process a collection of elements (an array, a list etc) in its entirety. Yet, people keep on writing it and then spend countless man-hours chasing subtle bugs that blow out later in production. Most modern languages have a «foreach» operator or its functional equivalent as a collection item generator. «foreach elem in my_collection {}» also has a clearly defined semantics: «process everything». The code is cleaner, concise and a bit shorter and reduces the cognitive overload. And since many languages now support lambdas, the code can quickly and error free be converted into a staged data processing pipeline without the use of temporary variables that serve no purpose, e.g. «foreach elem in my_collection.filter (_ -> _.my_selection_criteria ("42"); ) { do_something (elem); }». Whether the collection is indexed from 0 or from 1 becomes irrelevant.
barrenko 3y ago
Still trying to wrap my head around what is Julia exactly. Sidenote - MIT's Introduction to Computational Thinking is a pretty decent course.
- MengerSponge 3y ago
  Matlab that compiles to LLVM. It's fast, pretty easy to write, and surprisingly expressive. It's also 1-indexed. I just think it's neat, but it's never solved a meaningful problem for me. My data gets filtered and well-structured upstream of my analysis, so python+pandas is perfect. Visualization tools aren't better in Julia than JS or Python. Really crunchy simulations tend to be written in C or C++.
- sundarurfriend 3y ago
  An attempt at an easy-to-use dynamic language which tries to compile things early as much as possible via aggressive type inference (helped by the type system's design and multiple dispatch), so that you don't leave performance on the table where it's possible to have it, while also having the flexibility of dynamism where you need it. That's my interpretation, I'd be curious what Julia folks who actually know what they're talking about think of it.
- bobbylarrybobby 3y ago
  Julia is a language that you write more or less like python, but in cases where python would chase a million pointers to figure out what the type of a variable is, get an instance variable, call a method, etc, Julia (if written correctly) does just in time compilation. The first time it has to call a function on a combination of types it hasn't seen before, it compiles the function for those specific types (into LLVM IR, and then native code) and stores the compiled code so that subsequent dispatches of that function with those types are instantaneous, calling directly into the compiled code. You do not need to add type annotations for this to happen; at runtime, Julia checks the types of variables (which may have even been determined when the function was compiled, eliding the runtime check, if it was used in a context where the types were statically known; see below) to determine which version of a function to call. This is a strength, as the only thing you have to do to get the performance of fully compiled code (modulo things like GC and bounds checking) is make sure your functions are “type stable”, i.e., their type information can be determined statically. (In fact, in Julia you get better performance by writing more, smaller functions because you'll have more regions within which the types are statically inferrable.) But it's also a weakness because the first time you start up a REPL and call a bunch of functions, each of those functions has to be compiled from scratch, which takes a long time (google “Julia time to first plot”). Julia has other niceties such as very flexible math (promotion between every pair of numeric types) and a lisp-like macro system with homoiconic code, which make writing numerical code and scientific algorithms highly ergonomic.
alwaysbeconsing 3y ago
I'm not a math or data scientist so personally when I touch Python or Ruby it's for a system scripting, small CLI tool use case. Would you say Julia fills that corner as well?
- sundarurfriend 3y ago
  I'm curious about this too. The latest 1.9 changes make it so that package precompilations are stored and reused, so the actual workflow will have to be to generate a package (which Julia has tools for), and have the script be just a tiny driver that calls into the package. With that workflow, what kind of latency can we expect? And how consistent would that be?
- currymj 3y ago
  you can do it but it's not great. there's no batteries-included standard argparse library and even with the new improvements, there is still some noticeable overhead of a few seconds for some scripts i've written.
- themodelplumber 3y ago
  Same. I went down this road with GPT and was kinda disappointed TBH, since IIRC it recommended the language for the purpose. I use ABS, Perl, PHP or Nim for most of that stuff but am always looking for new ones to try. (Preferably single binary these days since I have to migrate systems from time to time)
- xgdgsc 3y ago
  You can see some previous discussion on this at https://news.ycombinator.com/item?id=35477695 https://news.ycombinator.com/item?id=35477695 . I feel 1.9 would make it much more realistic. For CLI you can use https://comonicon.org/stable/ https://comonicon.org/stable/ .
cwp 3y ago
I think Julia really dropped the ball on the execution model. Just-ahead-of-time compilation ends up being the worst of both worlds - you can't compile a small fast binary for deployment, and you can't quickly run a script or REPL for development. It turns out that this really matters for adoption. And now Julia has competition from Mojo. Mojo makes some compromises for backward compatibility with the Python world, but it's really solving the problems that hurt AI most. And the folks behind Mojo have a lot of real-world experience migrating a community from one language to another. I think Julia will remain a niche language, confined to science and statistical computing outside of mainstream data science and machine learning.
- ivirshup 3y ago
  > it's really solving the problems that hurt AI most Isn't this a bit premature? Mojo doesn't tangibly exist for most people (we can't run it ourselves), and I am unaware of any ML/ AI applications built with Mojo.
  - garbagecoder 3y ago
    I have tried it. Iirc, the company developed it, or its compiler, specifically for ML/“””AI”””. If they successfully import Pythons libraries without a bunch of wrapping, its userbase will likely follow. It was faster than Python and easy to do paralleization like go. But it’s not quite fully baked yet.
  - cwp 3y ago
    Yeah, definitely early days. But they're saying all the right things: fast code off the bat, easy to vectorize and parallelize, efficient use of memory, easy to create abstractions that execute efficiently on various architectures. The demos are impressive and they have the track record to be credible. I've played with it and the stuff that's been implemented so far works well.
    - pjmlp 3y ago
      Swift for Tensorflow was such a success.
sundarurfriend 3y ago
You're not technically wrong, in that Julia has the potential to replace all of those languages. And if it could attract large communities, an amazing interoperating coherent package ecosystem could develop there that's much less of a pain to use than most current systems. But I don't think either of those are given. Projects with huge potential have unfortunately failed for "trivial" reasons like personpower, funding, approachability, etc. that have nothing to do with underlying technical merit. (edit: but, to be clear, I don't think at this point Julia can exactly "fail", that seems pretty implausible; it will definitely have a strong presence in the MATLAB-ish niche, in the sciences and in engineering. The uncertainty is about more general purpose, popular usage). I love Julia and would love to see it succeed, but at this point in time its future is still uncertain. (And it's also unclear what if any the effect of LLM code generators is going to be - are they going to reinforce the current popular languages since those are the ones they're good at? Or are they actually going to make less popular languages more approachable, as they grow to learn to generate code in them too?)
anonylizard 3y ago
This kind of thinking is very old school. Its 2023 now. 1.GPT-4 and co are really, really good at python, and will write 90% of boilerplate for you. That makes a python dev way more productive than the Julia one. There is way, way less data on Julia, so the ceiling for AI assisted Julia is way lower. 2.Data science is not just running fancy statistical functions. Its also API calls to like 10 services. You can setup a PDF->OCR->GPT processed text->Send to vector database with a few API calls in python. There's a python package for everything, and everything put into one script. 3.Did you know everyone using stable diffusion (millions), has to install pytorch and like 50 packages in python? Now 95% will just use auto-installers. But the remainder 5%, will be exposed to python, and forced to learn it if they want to use AI art professionally (scripting) or do further development in it. When you see hordes of nerds desperately learning python to generate waifu pictures on google collab notebooks, you know a language will be dominant for decades to come.
- jimsimmons 3y ago
  Proto AGI GPT4 can’t faithfully translate Python to Julia?!
  - themodelplumber 3y ago
    Or write a transpiler perhaps? Or is that too devilish...
- garbagecoder 3y ago
  GPT-4 seems best at C#. Scary good. Hmmm. I agree with 3, but it’s also an argument for Mojo, if it delivers on its promises.
mxkopy 3y ago
Julia:Python::RISC:CISC imo. The first example that comes to mind is that in Python, you'd write: `A = [ x for x in range(10) if not isprime(x) ]` In Julia, you'd instead write: `A = [ x for x in 0:10 if !isprime(x) ]` It feels like as if having an infix operator for not is redundant - why add a separate & distinct symbol when it means the exact same thing? There's also the experience of writing iterables/generators, where in Julia you have to subtype the very specific Base.iterate function, but in Python you can just append the magical `yield` to your function. Overall in Julia I feel like there is less 'magic', but with the added benefit of having a very clear picture of what's happening.
jimsimmons 3y ago
No classes, weak typing despite having multiple dispatch, 1-based indexing, no real DL libraries with comparable baseline to pytorch/weight portability There are no reasons to not use it if you like gesturing from a distance
- sgt101 3y ago
  No classes - thank the lord. This was a 30 year experiment that failed. Parametric polymorphism is the way and the light. >weak typing despite having multiple dispatch I'm struggling to understand, do you mean that it doesn't have type safety? Could you explain a bit? 1-based indexing - I don't care about... the "no DL libraries" and the community effects around having weight portability is significant I think. I need to spend more time with Transformers.jl.
  - Jtsummers 3y ago
    It isn't weakly typed, it has strong, dynamic typing. GP is either confusing weak typing with dynamic typing or making some other kind of error. To demonstrate, in contrast to say perl, try `1+"1"` in Julia. It'll happily error out for you instead of coercing that string into a number to perform arithmetic with.
    - sgt101 3y ago
      Thank you for some explanation. I thought more or less the same. Sometimes people think strong typing is type safety. Weak/strong typing is a very confusing concept now, I think dynamic / static is a much clearer division. Personally I have had a lot of value using the Julia type system for runtime inference in programs, I love it!
enriquto 3y ago
> It really could replace anything R, Python, Perl, Ruby etc is used for today. There is an (admittedly niche) use-case where Julia would be almost incapable to work, as compared to other interpreted languages like for example Lua or Perl. Making a "busybox" style executable that replaces all coreutils (cat, ls, head, tail, wc, sort, uniq, tr, cp, df, echo, printf, ...) with simple Julia implementations. The slow startup-time for each Julia instance would make most shell scripts unbearably sluggish. I agree that Python would be an equally bad choice for that, but at least it should be slightly faster. Regarding "serious" uses of Julia, especially in numerical mathematics, I find some basic things still lacking. For example a complete base package for sparse matrices, including kronecker products etc. Octave/Matlab have the "kron" function in the base language. In Julia, should I use things with dubious names like LuxurySparse or what?
- sundarurfriend 3y ago
  > Making a "busybox" style executable that replaces all coreutils I wonder if DaemonMode (https://github.com/dmolina/DaemonMode.jl https://github.com/dmolina/DaemonMode.jl) is the right approach for this, having each of those tools' functionality loaded in the server process, and calling out to them from a client when a tool is invoked. > a complete base package for sparse matrices, including kronecker products etc. SparseArrays is a standard library package for sparse matrices and arrays. kron is in the LinearAlgebra standard library. So you can just do using LinearAlgebra using SparseArrays and then do any kron product you want, whether sparse matrices with other sparse matrices, or dense ones, or any other array type.
  - enriquto 3y ago
    > and then do any kron product you want, whether sparse matrices with other sparse matrices Thanks! Will try it with the new release. Last time I tried, apparently it converted the sparse matrix to dense before calling kron, and (expectedly) it failed due to memory error. For example, try to compute the adjacency matrix of a grid graph of size 1000x1000. It is the kronecker product of two tridiagonal matrices of that size. It only has four million non-zero entries, which are instantaneous to compute, but if you want to store all the zeros you'll need a few terabites of RAM.
shakow 3y ago
> People are increasingly running out of reasons to not use Julia I'm sorry, but I can't really let you say that. I like Julia, it has many nice features, but it still IMHO exhibits a lot of reasons not to use it, all of which I stumbled upon while using it to develop non-trivial pieces of software in bioinformatics. - parallelism is very confusing, very easy to break, and not very efficient when it works; - cold-boot time is prohibitive; - the ecosystem is brittle, and very much underdocumented; - the dispatch mechanism is both a blessing and a curse; - the stdlib should include some way of encoding errors, otherwise everyone cook their own stuff in their corner (should I raise an exception? Return -1, or maybe null? Or a Union?); - not being able to redefine types in REPL is a PITA for REPL-driven development; - debugging cross-packages bugs with multi-dispatch in time-consuming; - the stdlib doc website is *excruciatingly* slow; - memory consumption is hard to control; - iterators are weird (i.e. eager and hardly composable), and either pipe should be a part of the language or they should be chainable, map(f, filter(g, sort(h, [...]map(t, xs)))) leads to unreadable & uneditable mess; - performances do not follow what was promised (*C-like order of magnitude*), unless maybe you're (a) a Julia god or (b) only using numeric computations. Now none of these flaws are intrinsic, and I very much hope that all these edges will be polished in the coming times. But as a ‶grunt on the field″, it is exasperating to see the core Julia community just telling you that you're holding it wrong and that all is best in the best of worlds.
contravariant 3y ago
Until there is a Julia package for everything it is not a replacement for python. I find it hard to make a case for Julia if performance is not really a consideration.

packetlost 3y ago

Having 1-indexed arrays is unforgivable.

huijzer 3y ago
Okay. I’ll bite. Why?
- xedrac 3y ago
  I used Julia for a project back in 2016, and had the same adverse reaction to 1-based indexing. It's the same reason I have a hard time with Lua. Why? I suppose it's all rather superficial, but it's one of those things that just grates on me and I'll avoid it if I can. I'm not sure there's a logical explanation, but I've found the sentiment to be rather common among developers.
  - sweezyjeezy 3y ago
    Same, but probably it's more of a familiarity bias rather than a logical one. I think there is a strong argument to index from 1 as is done in MATLAB, R etc. since it matches the way most people refer and think about lists in reality. A brave decision from the authors though, I imagine they would have got less flak matching the more popular 0-index languages.
    - sundarurfriend 3y ago
      > I think they would have got less flak matching the more popular 0-index languages. Possibly, though those of us already in the space would have found it a (slightly) less pleasant language then.
- gcmrtc 3y ago
  This is the opinion of Dykstra: https://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF https://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF
  - Iwan-Zotow 3y ago
    sure, [0...N) ranges are composable, to be short
    - sundarurfriend 3y ago
      That is such a weird argument to me. I can count in the fingers of one hand the number of times in my life that I've had to compose ranges, it seems a strange thing to optimize for.
      - Iwan-Zotow 3y ago
        Splitting and combining back ranges is very common activity since about Core 2 Duo, when multi-core getting to be a normal. I mean, computing mean - split range, get result per split and combine it back, and so on and so forth
    - npalli 3y ago
      You seem to have a weird fetish for composable ranges seeing your history of comments on exactly this topic (even this thread!). The number of times I have had to think about composability as you mention is close to zero. In fact, the off-by-one errors with C class of languages due to the open ended nature of the interval have bit me about 3-4 orders of magnitude more. There is an entire class of security bugs because of this. I find this obsession very strange, YMMV obviously.
leephillips 3y ago
Yes, that’s probably why Fortran was never used seriously.
bobbylarrybobby 3y ago
Not just that, but ranges are inclusive on both ends.
- sundarurfriend 3y ago
  Ha, I was thinking of replying to your parent comment, "Half-open ranges are such an unpleasant thing as to be unforgivable to me. Different subjective preferences." I've always found 0-indexing mildly unpleasant too, even though C was the language I learnt programming with, and felt like I found home when I came across 1-indexed languages.
  - bobbylarrybobby 3y ago
    With half open ranges, the length of the range is the end minus the beginning, which is nice. It's also much more simpler to write an empty half inclusive range. For instance if left=right then left..right would be the empty range. Whereas in Julia I'd need... left:(right-1)? But is 1:0 the empty range or the right-to-left inclusive range [1,0]? Very confusing and hard to work with all around.
    - sundarurfriend 3y ago
      > With half open ranges, the length of the range is the end minus the beginning, which is nice. But I already know from real life that if there's maintanence in blocks 5 to 15, that's eleven blocks under maintanence. With half-open ranges, it once again introduces confusion and makes things unintuitive. We can each find countless examples where each style comes out better, for eg `1:N` most often expresses the intent better than `range(N+1)`. I find people who prefer half-open ranges to be weird and incomprehensible, you probably find my preferences the same, I just don't like the categorical statement often made in this regard that one is inherently and universally superior to the other.
      - bobbylarrybobby 3y ago
        Yes, range(n+1) is bad. I hate python as well. You want something like Rust’s m..n and m..=n to have the choice. Anyway, if you have half open ranges, it's trivial to get a closed range (add one to the right endpoint), but going the other direction is not so simple.
goatlover 3y ago
This again. Fortran, Cobol, Smalltalk, R, Matlab, APL, Wofram, Lua. Its primary use is scientific computing. It's like the whitespace in Python debate that everyone gave up because the language got so popular that people stopped caring.
upbeat_general 3y ago
Agreed. I thought the same thing about Python's indenting originally (and still do). But the ecosystem was enough to overcome it for me eventually.
pjmlp 3y ago
Plenty of languages have it, so what, it is too much to ask for in skillets?
- Iwan-Zotow 3y ago
  Julia ranges are not composable, that's it You cannot do easily [0...N) = [0...N/2) + [N/2...N)
  - webshit2 3y ago
    Idk typing +1 seems easy enough... 1:N == union(1:(N÷2), (N÷2+1):N). But really, these discussions are funny to me - each side pretending their convention is how God intended indexing to be done. You see it's naturally composable because N/2 shows up twice, which is really the perfect amount of times to show up, and as you recall I just defined composable in that way (not to mention it matches the fact that N/2 occurs twice in [0, N)!)
    - Iwan-Zotow 3y ago
      In python this is natural way to do ranges, and in Julia you have to remember this pesky +1 In Python (C, C++ with whole STL) you could split in the middle, or at any M [0...N)=[0...M)+[M...N) You could split it k times at any boundaries, still [0...N)=[0...M1)+...+[Mk...N) Another important thing is that number of elements to process is exactly the difference between last and first index of the range https://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF https://www.cs.utexas.edu/users/EWD/ewd08xx/EWD831.PDF
      - nequo 3y ago
        > number of elements to process is exactly the difference between last and first index of the range And in Julia, it’s length(a:b) which also works with non-unit step sizes like length(a:x:b) The intent of these expressions seems clearer than the intent of `b-a` and `ceiling((b-a)/x)` with your proposed approach in a zero-indexed language.
Rarebox 3y ago
It's jarring initially, but becomes natural very quickly. Writing loops like "for i in 1:length(arr) ... end" is pretty neat compared to C++ or even python. Plus in math sequences typically start at index 1.
- threatofrain 3y ago
  > Plus in math sequences typically start at index 1. I'm not quite sure that this is the case enough to say "typically". In terms of undergraduate exposure to math, I think more people have taken Calculus (or Analysis) than Linear Algebra, and I think Calculus textbooks tend to index from 0 while Linear Algebra textbooks tend to index from 1.
- leephillips 3y ago
  You should never write loops that way, at least in code you're going to share (assuming that you’re going to have some arr[i] in the loop body—if not, you would just do "for e in arr"). Assuming that arrays start at 1 is a source of occasional bugs in public packages. The existence of OffsetArrays means that arrays can start at any index (so for people who get nauseated by 1-based arrays, you can change it). Instead, write "for i in eachindex(arr)". In fact, Julia’s array syntax means you can loop over and manipulate arrays without using indexes much of the time, so don’t even need to know how they’re based.
- inkyoto 3y ago
  I would go as far as to say that it is now largely an archaic idiom to write «for i in 1:length(arr) ... end», and there is no reason to write such a code to process a collection of elements (an array, a list etc) in its entirety. Yet, people keep on writing it and then spend countless man-hours chasing subtle bugs that blow out later in production. Most modern languages have a «foreach» operator or its functional equivalent as a collection item generator. «foreach elem in my_collection {}» also has a clearly defined semantics: «process everything». The code is cleaner, concise and a bit shorter and reduces the cognitive overload. And since many languages now support lambdas, the code can quickly and error free be converted into a staged data processing pipeline without the use of temporary variables that serve no purpose, e.g. «foreach elem in my_collection.filter (_ -> _.my_selection_criteria ("42"); ) { do_something (elem); }». Whether the collection is indexed from 0 or from 1 becomes irrelevant.

barrenko 3y ago

Still trying to wrap my head around what is Julia exactly. Sidenote - MIT's Introduction to Computational Thinking is a pretty decent course.

MengerSponge 3y ago
Matlab that compiles to LLVM. It's fast, pretty easy to write, and surprisingly expressive. It's also 1-indexed. I just think it's neat, but it's never solved a meaningful problem for me. My data gets filtered and well-structured upstream of my analysis, so python+pandas is perfect. Visualization tools aren't better in Julia than JS or Python. Really crunchy simulations tend to be written in C or C++.
sundarurfriend 3y ago
An attempt at an easy-to-use dynamic language which tries to compile things early as much as possible via aggressive type inference (helped by the type system's design and multiple dispatch), so that you don't leave performance on the table where it's possible to have it, while also having the flexibility of dynamism where you need it. That's my interpretation, I'd be curious what Julia folks who actually know what they're talking about think of it.
bobbylarrybobby 3y ago
Julia is a language that you write more or less like python, but in cases where python would chase a million pointers to figure out what the type of a variable is, get an instance variable, call a method, etc, Julia (if written correctly) does just in time compilation. The first time it has to call a function on a combination of types it hasn't seen before, it compiles the function for those specific types (into LLVM IR, and then native code) and stores the compiled code so that subsequent dispatches of that function with those types are instantaneous, calling directly into the compiled code. You do not need to add type annotations for this to happen; at runtime, Julia checks the types of variables (which may have even been determined when the function was compiled, eliding the runtime check, if it was used in a context where the types were statically known; see below) to determine which version of a function to call. This is a strength, as the only thing you have to do to get the performance of fully compiled code (modulo things like GC and bounds checking) is make sure your functions are “type stable”, i.e., their type information can be determined statically. (In fact, in Julia you get better performance by writing more, smaller functions because you'll have more regions within which the types are statically inferrable.) But it's also a weakness because the first time you start up a REPL and call a bunch of functions, each of those functions has to be compiled from scratch, which takes a long time (google “Julia time to first plot”). Julia has other niceties such as very flexible math (promotion between every pair of numeric types) and a lisp-like macro system with homoiconic code, which make writing numerical code and scientific algorithms highly ergonomic.

alwaysbeconsing 3y ago

I'm not a math or data scientist so personally when I touch Python or Ruby it's for a system scripting, small CLI tool use case. Would you say Julia fills that corner as well?

sundarurfriend 3y ago
I'm curious about this too. The latest 1.9 changes make it so that package precompilations are stored and reused, so the actual workflow will have to be to generate a package (which Julia has tools for), and have the script be just a tiny driver that calls into the package. With that workflow, what kind of latency can we expect? And how consistent would that be?
currymj 3y ago
you can do it but it's not great. there's no batteries-included standard argparse library and even with the new improvements, there is still some noticeable overhead of a few seconds for some scripts i've written.
themodelplumber 3y ago
Same. I went down this road with GPT and was kinda disappointed TBH, since IIRC it recommended the language for the purpose. I use ABS, Perl, PHP or Nim for most of that stuff but am always looking for new ones to try. (Preferably single binary these days since I have to migrate systems from time to time)
xgdgsc 3y ago
You can see some previous discussion on this at https://news.ycombinator.com/item?id=35477695 https://news.ycombinator.com/item?id=35477695 . I feel 1.9 would make it much more realistic. For CLI you can use https://comonicon.org/stable/ https://comonicon.org/stable/ .

cwp 3y ago

I think Julia really dropped the ball on the execution model. Just-ahead-of-time compilation ends up being the worst of both worlds - you can't compile a small fast binary for deployment, and you can't quickly run a script or REPL for development. It turns out that this really matters for adoption. And now Julia has competition from Mojo. Mojo makes some compromises for backward compatibility with the Python world, but it's really solving the problems that hurt AI most. And the folks behind Mojo have a lot of real-world experience migrating a community from one language to another. I think Julia will remain a niche language, confined to science and statistical computing outside of mainstream data science and machine learning.

ivirshup 3y ago
> it's really solving the problems that hurt AI most Isn't this a bit premature? Mojo doesn't tangibly exist for most people (we can't run it ourselves), and I am unaware of any ML/ AI applications built with Mojo.
- garbagecoder 3y ago
  I have tried it. Iirc, the company developed it, or its compiler, specifically for ML/“””AI”””. If they successfully import Pythons libraries without a bunch of wrapping, its userbase will likely follow. It was faster than Python and easy to do paralleization like go. But it’s not quite fully baked yet.
- cwp 3y ago
  Yeah, definitely early days. But they're saying all the right things: fast code off the bat, easy to vectorize and parallelize, efficient use of memory, easy to create abstractions that execute efficiently on various architectures. The demos are impressive and they have the track record to be credible. I've played with it and the stuff that's been implemented so far works well.
  - pjmlp 3y ago
    Swift for Tensorflow was such a success.

sundarurfriend 3y ago

You're not technically wrong, in that Julia has the potential to replace all of those languages. And if it could attract large communities, an amazing interoperating coherent package ecosystem could develop there that's much less of a pain to use than most current systems. But I don't think either of those are given. Projects with huge potential have unfortunately failed for "trivial" reasons like personpower, funding, approachability, etc. that have nothing to do with underlying technical merit. (edit: but, to be clear, I don't think at this point Julia can exactly "fail", that seems pretty implausible; it will definitely have a strong presence in the MATLAB-ish niche, in the sciences and in engineering. The uncertainty is about more general purpose, popular usage). I love Julia and would love to see it succeed, but at this point in time its future is still uncertain. (And it's also unclear what if any the effect of LLM code generators is going to be - are they going to reinforce the current popular languages since those are the ones they're good at? Or are they actually going to make less popular languages more approachable, as they grow to learn to generate code in them too?)

anonylizard 3y ago

This kind of thinking is very old school. Its 2023 now. 1.GPT-4 and co are really, really good at python, and will write 90% of boilerplate for you. That makes a python dev way more productive than the Julia one. There is way, way less data on Julia, so the ceiling for AI assisted Julia is way lower. 2.Data science is not just running fancy statistical functions. Its also API calls to like 10 services. You can setup a PDF->OCR->GPT processed text->Send to vector database with a few API calls in python. There's a python package for everything, and everything put into one script. 3.Did you know everyone using stable diffusion (millions), has to install pytorch and like 50 packages in python? Now 95% will just use auto-installers. But the remainder 5%, will be exposed to python, and forced to learn it if they want to use AI art professionally (scripting) or do further development in it. When you see hordes of nerds desperately learning python to generate waifu pictures on google collab notebooks, you know a language will be dominant for decades to come.

jimsimmons 3y ago
Proto AGI GPT4 can’t faithfully translate Python to Julia?!
- themodelplumber 3y ago
  Or write a transpiler perhaps? Or is that too devilish...
garbagecoder 3y ago
GPT-4 seems best at C#. Scary good. Hmmm. I agree with 3, but it’s also an argument for Mojo, if it delivers on its promises.

mxkopy 3y ago

Julia:Python::RISC:CISC imo. The first example that comes to mind is that in Python, you'd write: `A = [ x for x in range(10) if not isprime(x) ]` In Julia, you'd instead write: `A = [ x for x in 0:10 if !isprime(x) ]` It feels like as if having an infix operator for not is redundant - why add a separate & distinct symbol when it means the exact same thing? There's also the experience of writing iterables/generators, where in Julia you have to subtype the very specific Base.iterate function, but in Python you can just append the magical `yield` to your function. Overall in Julia I feel like there is less 'magic', but with the added benefit of having a very clear picture of what's happening.

jimsimmons 3y ago

No classes, weak typing despite having multiple dispatch, 1-based indexing, no real DL libraries with comparable baseline to pytorch/weight portability There are no reasons to not use it if you like gesturing from a distance

sgt101 3y ago
No classes - thank the lord. This was a 30 year experiment that failed. Parametric polymorphism is the way and the light. >weak typing despite having multiple dispatch I'm struggling to understand, do you mean that it doesn't have type safety? Could you explain a bit? 1-based indexing - I don't care about... the "no DL libraries" and the community effects around having weight portability is significant I think. I need to spend more time with Transformers.jl.
- Jtsummers 3y ago
  It isn't weakly typed, it has strong, dynamic typing. GP is either confusing weak typing with dynamic typing or making some other kind of error. To demonstrate, in contrast to say perl, try `1+"1"` in Julia. It'll happily error out for you instead of coercing that string into a number to perform arithmetic with.
  - sgt101 3y ago
    Thank you for some explanation. I thought more or less the same. Sometimes people think strong typing is type safety. Weak/strong typing is a very confusing concept now, I think dynamic / static is a much clearer division. Personally I have had a lot of value using the Julia type system for runtime inference in programs, I love it!

enriquto 3y ago

> It really could replace anything R, Python, Perl, Ruby etc is used for today. There is an (admittedly niche) use-case where Julia would be almost incapable to work, as compared to other interpreted languages like for example Lua or Perl. Making a "busybox" style executable that replaces all coreutils (cat, ls, head, tail, wc, sort, uniq, tr, cp, df, echo, printf, ...) with simple Julia implementations. The slow startup-time for each Julia instance would make most shell scripts unbearably sluggish. I agree that Python would be an equally bad choice for that, but at least it should be slightly faster. Regarding "serious" uses of Julia, especially in numerical mathematics, I find some basic things still lacking. For example a complete base package for sparse matrices, including kronecker products etc. Octave/Matlab have the "kron" function in the base language. In Julia, should I use things with dubious names like LuxurySparse or what?

sundarurfriend 3y ago
> Making a "busybox" style executable that replaces all coreutils I wonder if DaemonMode (https://github.com/dmolina/DaemonMode.jl https://github.com/dmolina/DaemonMode.jl) is the right approach for this, having each of those tools' functionality loaded in the server process, and calling out to them from a client when a tool is invoked. > a complete base package for sparse matrices, including kronecker products etc. SparseArrays is a standard library package for sparse matrices and arrays. kron is in the LinearAlgebra standard library. So you can just do using LinearAlgebra using SparseArrays and then do any kron product you want, whether sparse matrices with other sparse matrices, or dense ones, or any other array type.
- enriquto 3y ago
  > and then do any kron product you want, whether sparse matrices with other sparse matrices Thanks! Will try it with the new release. Last time I tried, apparently it converted the sparse matrix to dense before calling kron, and (expectedly) it failed due to memory error. For example, try to compute the adjacency matrix of a grid graph of size 1000x1000. It is the kronecker product of two tridiagonal matrices of that size. It only has four million non-zero entries, which are instantaneous to compute, but if you want to store all the zeros you'll need a few terabites of RAM.

shakow 3y ago

> People are increasingly running out of reasons to not use Julia I'm sorry, but I can't really let you say that. I like Julia, it has many nice features, but it still IMHO exhibits a lot of reasons not to use it, all of which I stumbled upon while using it to develop non-trivial pieces of software in bioinformatics. - parallelism is very confusing, very easy to break, and not very efficient when it works; - cold-boot time is prohibitive; - the ecosystem is brittle, and very much underdocumented; - the dispatch mechanism is both a blessing and a curse; - the stdlib should include some way of encoding errors, otherwise everyone cook their own stuff in their corner (should I raise an exception? Return -1, or maybe null? Or a Union?); - not being able to redefine types in REPL is a PITA for REPL-driven development; - debugging cross-packages bugs with multi-dispatch in time-consuming; - the stdlib doc website is *excruciatingly* slow; - memory consumption is hard to control; - iterators are weird (i.e. eager and hardly composable), and either pipe should be a part of the language or they should be chainable, map(f, filter(g, sort(h, [...]map(t, xs)))) leads to unreadable & uneditable mess; - performances do not follow what was promised (*C-like order of magnitude*), unless maybe you're (a) a Julia god or (b) only using numeric computations. Now none of these flaws are intrinsic, and I very much hope that all these edges will be polished in the coming times. But as a ‶grunt on the field″, it is exasperating to see the core Julia community just telling you that you're holding it wrong and that all is best in the best of worlds.

contravariant 3y ago

Until there is a Julia package for everything it is not a replacement for python. I find it hard to make a case for Julia if performance is not really a consideration.