8 ms·
Data Science in Julia for Hackers
- snicker7 6y agoThis book is being written entirely in Pluto notebooks!
- jjoonathan 6y agoI had heard of Pluto notebooks but didn't know that they were truly different. The elevator pitch seems to be Pluto: Jupyter without hidden state. I can't wait to try it out. I did my thesis work in Julia ~5 years ago but abandoned it afterwards precisely because state management was so painful. At that time, Julia took 30 seconds to start (compile the plotting library), so you basically had to use live sessions, and they had all the usual stateful foot-guns, which made for a very painful overall experience. Performance was still pretty rough last time I tried it (build caching on the libraries didn't happen on install and even post-install wasn't Just Working), but Pluto sounds like a strong enough mitigation to be worth a try!
- cbkeller 6y agoAdding Plots (or for that matter, Pluto!) to your sysimg with PackageCompiler.jl is a potentially significant QoL improvement on that front. Parallel auto-precompile in 1.6 is also awesome.
- jjoonathan 6y agoauto-precompile Ok, with Pluto it had my interest, but now it has my attention.
- adgjlsfhk1 6y agoYou should check out https://www.oxinabox.net/2021/02/13/Julia-1.6-what-has-changed-since-1.0.html https://www.oxinabox.net/2021/02/13/Julia-1.6-what-has-chang... if you've been away for a while. It's a pretty good list of what has been improved.
- adgjlsfhk1 6y agoFor the record, I'm currently seeing 2.8 seconds for `using Plots` and another 4 for the first plot (on 1.6).
- iagovar 6y agoReally liked the introduction, it resonates a lot with what I've been thinking for a while.
- 6gvONxR4sf7o 6y agoOff topic, but this is a few spots below "Hacker's Guide to Numerical Analysis" on the front page right now. It makes the naming seem pretty silly. Hacker News presents: X is all you need for Hackers considered harmful
- junippor 6y ago"For hackers" is a trick so you don't discourage people in HN. If you omit "for hackers", they'll say it has too much math and therefore no practical use. These are the same kind of people who think that the reason people use math is so they can feel smart about themselves, because practical people who want to do practical things don't need math - they have computers.
- ku-man 6y agoWell, "For Hackers" is much better than "For Engineers".
- dracodoc 6y agoThere really should be a better way to map values. There are numerous functions in R for this, I believe there should be similar functions in Julia. Is that because Julia have good performance so people tend to write loop to deal with this kind of task? And call this hacking? I tried paste the code but it was hard to read. You can search Histograms in this page https://datasciencejuliahackers.com/03_probability_intro.jl.html https://datasciencejuliahackers.com/03_probability_intro.jl....
- wenc 6y agoIf you're referring to the `for` loop replacing Spanish month names with English ones, that could have been done like this [1]: rainData.month = map(mth -> myEsEnDict[mth], rainData.month) [1] https://syl1.gitbook.io/julia-language-a-concise-tutorial/useful-packages/dataframes#edit-data https://syl1.gitbook.io/julia-language-a-concise-tutorial/us...
- ced 6y agoWhat do you mean, "map values"? There is [sin(x) for x in vec] map(sin, vec) sin.(vec) which are roughly equivalent? Is that what you're referring to?
- snicker7 6y agoDon't forget the "lazy" version: (sin(x) for x in vec)
- cbkeller 6y agoCan you give a code example in R of what you would like to do in Julia? There are quite a lot of different ways of mapping in Julia, including functions like `map`, `mapreduce`, `replace`, and various syntaxes for broadcasting and array comprehensions. For maximum SIMD performance there are also things like `vmap` and `vmapreduce` from LoopVectorization.jl.
- tfehring 6y agoUsing a loop and `if` statements as the book does is unidiomatic, the `replace` function (and the in-place equivalent `replace!`) work as you’d expect. https://docs.julialang.org/en/v1/base/collections/#Base.replace-Tuple%7BAny,Vararg%7BPair,N%7D%20where%20N%7D https://docs.julialang.org/en/v1/base/collections/#Base.repl... If you have a mapping dict defined I think the syntax is just `replace!.(rainData.month, monthMap...)`, not tested though.
- kasperset 6y agoI really want to love Julia language but the overhead of package loading annoys me. Perhaps this may be just me being lazy or spoiled after using R packages or Python libraries.
- chrisjharris 6y agoYou mean, the time it takes at run-time? I'm not quite sure what you mean - I think Julia's package management is generally better than Python (which is a mess) - it doesn't play well with Nix but I can't really blame it for that.
- kyllo 6y agoI think they're referring to the JIT compilation time. There are some ways to mitigate it, such as creating an image using PackageCompiler.jl, but it's definitely a noticeable issue IME. https://julialang.github.io/PackageCompiler.jl/dev/ https://julialang.github.io/PackageCompiler.jl/dev/
- alpaca128 6y agoOne of the things I took away from trying Julia was that unicode is incredibly dumb in some aspects. Unicode lacks support for parts of the latin alphabet in subscript and superscript formatting(similar for the greek alphabet)[0]. As Julia relies on Unicode for its neatly formatted variable names this is an annoying limitation that the developers can't even do much about. Unicode has over 100k symbols, including about 3500 emoji. But a continuous set of 26 small letters, either in lowercase or uppercase(or both) is too much bloat apparently. [0]: https://stackoverflow.com/questions/17908593/how-to-find-the-unicode-of-the-subscript-alphabet/17909597#17909597 https://stackoverflow.com/questions/17908593/how-to-find-the...
- jonniedie 6y agoJulia doesn’t rely on any Unicode. It just gives you the option to use it. If it’s not useful to you, feel free to not use it.
- deleted 6y ago[deleted]