16 ms·
Julia v1.3
- ashton314 7y agoI find Julia to be a beautiful little language. Applause to the team for this 1.3 release! In one of my CS classes we wrote a LISP using Julia. Our professor wrote the parser for us, so we just had to focus on the actual interpreter bit. The pattern matching/multi-dispatch mechanisms were really nice. We didn't get to play too much with the parallelism mechanisms, but I liked what I did see. I feel like Julia is a much richer language than, say, Python, and it would be a nice drop-in replacement for numerical computing tasks: it's faster, in some ways more ergonomic, and afaik can call out to Python libraries if needed. Has anyone here switched from Python to Julia for scientific/ML/AI purposes?
- dmos62 7y agoI played around with Julia a bit last year. One thing that stuck with me was the non-negligible start-up speed. If you want tight feedback loops you have to figure out a way to reuse a Julia instance (like Jupyter). On that train of thought, how interoperable is preexisting numpy/pandas code with Julia?
- short_sells_poo 7y agoI agree and this still holds it back from being the main language in a research environment where one wants to have quick turnaround time. On the other hand, it should work very well as a replacement for Numba. Numba already comes with a one-off jit cost, and it is much more limited in the sort of code it can speed up.
- socialdemocrat 7y agoYou can ;-) The secret sauce in Julia is to use Revise.jl package. Is makes your life so much better. It observes source code changes while you are at the REPL. No need to restart everything.
- ashton314 7y ago+1 for this package. Total game changer.
- jjoonathan 7y agoHow reliable is Revise.jl compared to ipython autoreload? My experience with ipython autoreload is... very poor, even though some people swear by it, so I have trust issues with this type of thing.
- montalbano 7y agoI use it extensively. In my experience it is very reliable. Onle once or twice in two years of using it has something strange happened with JIT compilation of functions. All I had to do to fix was restart the Julia REPL and load things in again.
- oxinabox 7y agoOne downside is Revise v2 actually itself has a fairly high startup cost. Best alternative is to pin it to Revise v1. I personsally don't mind the startup too much. But I do get where people are coming from
- enriquto 7y ago> One thing that stuck with me was the non-negligible start-up speed. Yes! this is the biggest roadblock for me. I find python startup time unbearably long. I do not want to imagine even using julia. And this is sad because the language is orders of magnitude better (for math, at least).
- timholy 7y agoIf you're used to launching everything from your OS's command prompt, this is a killer. But if you get used to launching a REPL and running things from there, it's much better, because you only pay the startup latency once.
- tfunnell 7y agoSome workflows require running scripts from the command line though
- JustFinishedBSG 7y agoJulia has 0-copy numpy array interop through PyCall.jl if that's what you're asking.
- dmos62 7y agoIt is. Thanks!
- ChrisRackauckas 7y agoAs for startup speed, note that in this round our research software engineer in the Julia Lab just brought the "first time to plot" to about 1/4 of what it was before. Julia 1.3 has a lot of these updates, and Julia 1.4 has a development sprint for bringing down the latency. I would be surprised if the common major latency issues still exist past next summer. Of course a JIT will always have some bumps compared to a pure interpreter, but Julia's compiler is definitely getting a lot smarter as to what to try and fully analyze (and what to cache)
- omaranto 7y ago> If you want tight feedback loops you have to figure out a way to reuse a Julia instance I think Julia is meant to be used like Lisp: - Step 1: Open the REPL. - Step 2: Don't close it until you are finished (both writing and running your program). My best guess is that people that complain about startup speed of Julia (and Emacs too for that matter) ignore step 2.
- tfunnell 7y agoFor myself, I write scripts that are part of automated analysis pipelines. So in my case (and probably others), using the REPL is mostly useful for testing code.
- The_rationalist 7y agoNot being 100% interoperable with python make Julia doomed. Library ecosystems have a human workforce value of billions of dollars, available for free. Any modern language that want to go mainstream must offer transparent and complete interoperability with an existing ecosystem. TypeScript -> js Kotlin -> jvm C++ -> C Elixir -> Erlang Hack -> php Crystal -> ruby As we can see there are still Big ecosystems lacking a new modern language™ The python ecosystem The C# (CLR) ecosystem It could be said that's mainly because those languages are already mostly "modern" There are also dying big ecosystem and there would be economic value to make their old libraries accessible. Fortran Pascal Perl Etc
- Tomte 7y ago> As we can see there are still Big ecosystems lacking a new modern language™ [...] The C# (CLR) ecosystem F#, obviously.
- coldtea 7y agoNot to mention C# is as modern as it gets...
- BubRoss 7y agoHow is C# modern? It is extremely verbose and rooted in Java style 'OO' with lots of memory hopping.
- SigmundA 7y agoEvery iteration of C# has made it less verbose and adds more modern concepts [1]. What do you mean by memory hopping, reference types? C# has had value types since the beginning unlike Java. 1. https://docs.microsoft.com/en-us/dotnet/csharp/whats-new/csharp-8 https://docs.microsoft.com/en-us/dotnet/csharp/whats-new/csh...
- coldtea 7y agoModern != less verbose
- bacr 7y agoIs there any chance the course material is available? I've been looking to better my Julia meta-programming. Most of the resources out there are geared towards other languages. It would be amazing to see some simple parser/interpreter implementations.
- montalbano 7y agoI switched to Julia from Python due to facing the classic '2-language problem' of scientists. We wanted an easy to use API and good speed. The main bottleneck was computing the Mittag-Leffler function. First we tried in Python, then we get a ~10x improvement by outsourcing that bit to Fortran. The Julia implementation of the Mittag-Leffler function is comparable to Fortran but writing the entire software package in Julia led to ~5x speedup due to avoiding language interop issues and taking advantage of other Julia-specific benefits.
- xenonite 7y agoSo, you obtained a 5x speedup of the whole package compared to the Python version with some Fortran extensions?
- montalbano 7y agoYes. And for anyone wondering about the state of the Julia package ecosystem compared to Python, someone had already written the main two packages we needed in Julia (for Mittag-Leffler computation and Inverse Laplace computation). At that time there was no Mittag-Leffler implemented in Python or SciPy that I could find. I could also add that we were computing integral equations with Mittag-Leffler function in the Kernels so in worst cases we needed O(n^2) calls to Mittag-Leffler function where n could be O(10^5).
- Rochus 7y agoIf you need another speedup consider LuaJIT which is used at CERN for heavy number crunching and data analysis, see http://cds.cern.ch/record/2157242?ln=de http://cds.cern.ch/record/2157242?ln=de. E.g. on slide 14: LuaJIT took 5.3 sec, C++/GCC 5.7 sec, PyPy 7.6 sec, Julia 35 sec and Python > 2 hours.
- improbable22 7y agoFrom the comments on that: > No one should use global scope of Julia for calculations: [...] which is said that takes 35 seconds. > If you wrap it in a function and not had a type instability, You'd get 4.704 seconds... Half-decently written Julia ought not to be much slower than C. But it does also allow you to write quick and dirty scripts with no care for speed, which is also useful. It would be nice if people wouldn't benchmark the latter. The manual has a pretty helpful section for avoiding this: https://docs.julialang.org/en/v1/manual/performance-tips/ https://docs.julialang.org/en/v1/manual/performance-tips/
- sgillen 7y ago>> Zero-dimensional arrays are now consistently preserved in the return values of mathematical functions that operate on the array(s) as a whole (and are not explicitly broadcasted across their elements). Previously, the functions +, -, *, /, conj, real and imag returned the unwrapped element when operating over zero-dimensional arrays. That's great news, it drives me crazy when languages or libraries get that wrong.
- The_rationalist 7y agoIn other words, mostly catching up features present in mainstream languages.
- gozzoo 7y agoThis is not a bad thing and it is quite natural considreing how young the language is. The same is true for golang and many other new cool languages. Being conservative when introuducing features in a new language is a very wise approach.
- The_rationalist 7y agoThis is natural indeed but it should be more frequently said, as I feel too many developers will go to the new shiny language and loose so many times because of what I call software poverty. (lack of libs, lack of features, lack of stackoverflow, lack of documentation, and even lack of stability) Indeed new languages have new shiny features but they do not outweigh the poverty cost. Especially since there are modern languages compatible with big ecosystems like typescript or kotlin, I don't really see how new languages that are not compatible can be competitive.
- gilbertmpanga12 7y agoSoftware poverty - Love the term.
- nnq 7y ago> modern languages compatible with big ecosystems like typescript or kotlin ...if you do anything towards ML or datascience, typescript and kotlin are the poor relatives. For most tasks 90% of the ecosystem is useless to you and you can spend more time wrangling the dependencies of a nasty nodejs package than quickly coding smth from scratch. That "richness" is also... "garbage".
- elcritch 7y agoIf the language fills a niche sufficiently better than existing ones. Julia doesn’t bring too many novel concepts (JIT, macros, multimethods, parallelism) but ties them together nicely for scientific computing. The best chance Julia has isn’t attracting average devs alone, but the library developers. Writing new python libraries for research purposes often requires writing C/C++ code. There’s already a number of Julia libraries focusing on interesting research topics in scientific, mathematics, and AI. Those lead to better and more useful libraries over time which can interact over time. Plus Julia Computing seems to be going towards building a successful commercial side of Julia. Personally I don’t find the software poverty thing much of an issue, outside a few core areas (database drivers, basic http, c interop, networking, text editors).
- short_sells_poo 7y agoI'm rooting for Julia. Currently we are using python and rust, but rust is not an easy language to do exploratory analysis in, so it's relegated to handling only the stable parts of our analytics libraries. It looks like Julia integrates quite seamlessly with python, so I'm hoping that we can start using it to easily speed up exploratory research code without having to spend a lot of time.
- playing_colours 7y agoI keep my eye on Julia, but still did not have a chance to dig into it. I would be interested to learn more about its place and capabilities: - I want to build an new (yet another) distributed data processing framework, say, Hadoop or Spark Next, generic or for a particular industry, and I do not want to use Java or C++. Can Julia be a feasible choice for a distributed computation like Spark, for distributed file system like HDFS, for resource allocator like Yarn? Can it be a good choice for a database engine, or its place is in layers above the engine - say a layer to support aggregation and computations? Or it's not a right choice comparing with Rust, C++, or Java for core systems, and it's better to stick with it just for computations on the top of those core systems?
- The_rationalist 7y agoWhat about kotlin? You would benefit from existing java/scala libraries.
- playing_colours 7y agoMy question was more about figuring out if Julia can be a good candidate, not like writing off Java or C++. Something when you build things for fun, for personal project, for prototype, or research project. If I am a Chief Engineer / CTO of a company, and need to choose a language for Yet Another data platform, I would definitely look first at Java / Kotlin / Scala for implementation - they are a proven pragmatic choice for such tasks. If there are memory and garbage collection concerns of the former, I would look at C++ or Rust (still risky for staffing). What I like in Julia is its expressiveness, and that it was built for writing maths and computations. So it's a great candidate for top level libraries above core components - it may be a reasonable choice for a business now. But Julia promises good performance as well. Therefore, if one does not have very strict requirements to avoid GC it may look like an interesting idea to stick with the same language for the core components. Here was my question above :)
- xiaodai 7y agoYes to distributed data processing. JuliaDB.jl has promise.
- falkaer 7y agoBy far the most interesting part of this release is the new multi-threading features, I recommend reading https://julialang.org/blog/2019/07/multithreading https://julialang.org/blog/2019/07/multithreading for an overview, and watching https://www.youtube.com/watch?v=YdiZa0Y3F3c https://www.youtube.com/watch?v=YdiZa0Y3F3c for a talk about some unique stuff that's being worked on with Julia's multi-threading (though the depth-first scheduling is not in 1.3 afaik).
- hokkos 7y agoDoes Julia plan to have a WASM backend ?
- ChrisRackauckas 7y agoYes, you can try it out: https://keno.github.io/julia-wasm/website/repl.htm https://keno.github.io/julia-wasm/website/repl.htm Here's the repo: https://github.com/Keno/julia-wasm https://github.com/Keno/julia-wasm It's still a work in progress to get the full package manager on there, but there's funding from Mozilla to get it done. I'm excited to see how that turns out.
- oconnor663 7y agoDoes the Rust Rayon library have the issue the speaker was describing, where parallel inner loops mostly get executed serially once an outer loop is parallelized? (I assume it does, from its Cilk heritage?) Are there workarounds for when that is a problem? Edit: Dumb mistake on my part. I was looking at the video linked in the top comment (https://www.youtube.com/watch?v=YdiZa0Y3F3c https://www.youtube.com/watch?v=YdiZa0Y3F3c) and then I came back to this thread and thought it was the topic.
- GolDDranks 7y agoWhat speaker you are talking about? The linked post is to Julia 1.3 release notes that doesn't mention Rayon at all?
- oconnor663 7y agoOops, dumb mistake on my part. I was referring to the video linked in the top comment: https://www.youtube.com/watch?v=YdiZa0Y3F3c https://www.youtube.com/watch?v=YdiZa0Y3F3c
- tempodox 7y agoI was still hoping that someday support for standalone binary executables would be added. Maybe next century.
- stilley2 7y agoDoes this not do what you want? https://github.com/JuliaLang/PackageCompiler.jl/blob/master/README.md https://github.com/JuliaLang/PackageCompiler.jl/blob/master/...
- eigenspace 7y agoNote that PackageCompiler.jl's author no longer wants to work on it and there are still many rough edges. However, there are several exciting successor projects making quick progress right now!
- cshenton 7y agoJulia feels like a scripting language but every function is effectively a template. I prototype lots of high perf stuff in it because it gives you control of memory layout and what gets evaluated at compile time, so it'll get you within spitting distance of an optimised c++ implementation but with 10x better iteration speed. Thoroughly recommend it to any Comp Sci / ML researchers. Only thing I'd wish for is better control of memory layout for mutable structs, but I know that's unlikely since that's one thing that the Julia folks want to keep abstracted from the user.
- igouy 7y ago??? Upcoming release: v1.3.0-rc5 (Nov 17, 2019) We're currently testing release candidates for Julia v1.3.0 https://julialang.org/downloads/ https://julialang.org/downloads/
- chrispeel 7y agoTry the Julia Github page [1] for the v1.3 binaries. I think julialang.org/downloads has not yet been updated. [1] https://github.com/JuliaLang/julia/releases https://github.com/JuliaLang/julia/releases
- oxinabox 7y agoHacker News is just super on the ball, it out-paced actually updating the website. Still when I built julia this morning, from the 1.3 banch, I can confirm the -RC5 label was gone.