43 ms·
PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)
- opus111 5y agoWhy do people keep claiming that Julia's ecosystem is limited? Julia can use all of Python's libraries. They all work great. No need to reimplement in Julia, just use PyCall and your favorite Python code.
- stabbles 5y agoWhat stage of Julia denial is this?
- jstx1 5y agoWell, it's agreeing that Julia's goals (promises? aspirations?) are worthwhile. Whether Julia itself actually delivers on those promises is a different question that isn't addressed in the original post.
- edgyquant 5y agoI like what Julia is doing but I just dislike the syntax. It seems to resemble ruby, whose syntax I also think is ugly, which to me resembles a modern form of basic.
- ted_dunning 5y agoHmmm... this seems to be an odd first impression. There is the use of @ (but to signal macros), but otherwise, the syntax is much closer to a cross between Python and matlab except nicer for doing math. I tried writing a few programs in Julia and got sucked in by how effective it is. The real surprise is that just a few weeks in instead of pulling up R to do a quick calculation my fingers decided they wanted Julia.
- dunefox 5y agoSimilar story for me. I did last years Advent of code in Julia and it was great.
- amelius 5y agoI know what you mean. I especially dislike the use of an "end" keyword everywhere without a corresponding "begin" keyword.
- dnautics 5y agoI think that's inherited from matlab.
- edgyquant 5y agoHuh okay. I’ve never used Matlab but one of the first languages I played with was a basic for simple game development
- edgyquant 5y agoYeah that’s exactly what I’m talking about
- freemint 5y agobegin is a Julia keyword though it. But yes end closes begin, for, function and while.
- cookieater 5y agoDoes it really matter though? I mean, whether it's a closing bracket or a whitespace demarcated control flow - does it really effect anything you do in the language? Julia solves a lot of problems, like real problems - quibbling over begin/end vs {/} or white space seems kinda silly. Meanwhile I have actually seen real world lost productivity due to white space in python. Curly brackets not so much. Never seen lost productivity over begin/end but I'm sure it's happened. It seems silly to me either way - it doesn't really effect anything.
- fault1 5y agoPersonally, I think when you write Julia in shorthand form, it looks nothing like ruby or basic. for example, check out parts of the stdlib: https://github.com/JuliaLang/julia/blob/master/base/operators.jl https://github.com/JuliaLang/julia/blob/master/base/operator... but in the end, julia really is a lisp: https://www.youtube.com/watch?v=dK3zRXhrFZY https://www.youtube.com/watch?v=dK3zRXhrFZY
- rg111 5y agoWith the slightest hint arising that Julia would be the future of ML and DL, I learned it. But, then what? I could not use it anywhere I worked. The ecosystem was lacking. Julia is good, but for what exactly? People involved with Julia are always big with words, but when will I see it in use somewhere?
- pjmlp 5y agoYou can apply to a job here, https://juliacomputing.com/case-studies/ https://juliacomputing.com/case-studies/
- dunefox 5y agoThe ecosystem is a superset of Pythons: https://github.com/JuliaPy/PyCall.jl https://github.com/JuliaPy/PyCall.jl
- cookieater 5y agoIt's probably more fair to claim that R/Python/Julia have the same ecosystem now if you are willing to deal with speed bumps and some clunky interfaces. Because there are packages going in both direction for all three.
- xbpx 5y agoJava has a massive ecosystem yet we continue to see rapid replacement of backends in JavaScript (Node now Deno) and Golang. Each of those language ecosystems rapidly became both large (arguably too large) and robust. Rust has been eating C++ lunch. Same rapid rise of ecosystem story. Instead of forcing Python to be a language it isn't it might be more efficient and ultimately the "right choice" to invest the time in Julia. Julia is great for numerical computing, it needs faster time to plot and more hands in the ecosystem. The former will be solved and the latter seems inevitable to me. Pitch in!
- zz865 5y agoWow I hadn't seen deno before, looks great. I dont like js/ts, but if you have to learn it for front end dev, it makes sense to use it at the back too. I'm not sure what advantage Julia has over Python. Yeah it has some typing and can be faster, but its too similar. Still single threaded.
- adgjlsfhk1 5y agothis is just false. Julia has partir based threading (like go).
- ur-whale 5y ago> I'm not sure what advantage Julia While I'm really not a fan of 1-based indexing, Julia's multiple dispatch is not something easy to match in Python. [EDIT]: one thing that's still not solved in Julia is code startup time. Many people will sell you some sort of workflow that works around the problem, but it's the same old tired arguments people would use to defend traditional compiled languages, and I'm not buying. I really wish they would find a way to truly solve this.
- Mikeb85 5y ago> [EDIT]: one thing that's still not solved in Julia is code startup time. > Many people will sell you some sort of workflow that works around the problem, but it's the same old tired arguments people would use to defend traditional compiled languages, and I'm not buying. I mean, Julia has a REPL so you can basically edit code as the process runs, which definitely makes startup time less of an issue. The fact Julia can produce such fast code is also pretty nice. Starting a new process every time you want to run a snippet of code isn't getting the best out of a dynamic language...
- adenozine 5y agoNeither language have proper tail call elimination, which, is absolutely insane to me. Yall really just write procedural code for everything?
- adgjlsfhk1 5y agoif you want tail calls in Julia, there is a 3 line macro that gives it to you.
- tbenst 5y agoCould you point to it? Thanks
- ted_dunning 5y agoHere is one: https://github.com/TakekazuKATO/TailRec.jl https://github.com/TakekazuKATO/TailRec.jl It works by inspecting the code and rewriting a function to turn tail calls into loops. The interesting bit is that it was very easy to write because of the strong macros in Julia.
- adenozine 5y agoThis is eye opening. I might take a look at this in depth and try Julia again.
- nick__m 5y agoThis repo should interest you https://github.com/TakekazuKATO/TailRec.jl https://github.com/TakekazuKATO/TailRec.jl
- doliveira 5y agoYou must be in quite the functional bubble (I envy you for that, though)
- adenozine 5y ago
- forrestthewoods 5y agoThe more I use Python the more I hate it. It’s genuinely a bad language, with a stellar ecosystem. Ironically, the most valuable parts of the ecosystem are often written in C (NumPy). It’d be interesting to see how much of the Python ecosystem is actually necessary to move PyTorch to a better language. I’m afraid we’re stuck with Python for the next 20 years. That makes me very, very sad.
- amelius 5y agoSomeone wanting to promote a new language could wrap Python as a growth hack.
- patagurbon 5y agoJulia has a package that does this fairly elegantly (not perfect of course, but you can wrap on top of this) https://github.com/JuliaPy/PyCall.jl https://github.com/JuliaPy/PyCall.jl.
- jshen 5y ago100% this. I wish it weren’t so.
- noitpmeder 5y agoWhat do you think the major drawbacks are? Speed would be the top of my list, but most projects to not need anything more than what python can currently pump out.
- forrestthewoods 5y agoBasically everything. If I had to pick two things I hate the most it would be dynamic typing and Python's deployment story. I wrote a somewhat tongue-in-cheek rant blog post. https://www.forrestthewoods.com/blog/things-i-like-about-python/ https://www.forrestthewoods.com/blog/things-i-like-about-pyt...
- Tsarbomb 5y agoAside from speed, one thing that really eats at me is that it makes any sort of functional programming overly verbose, unfun, and just not very idiomatic. Also the the vast majority of python programmers simply don't understand the best practices in their own ecosystem. I was recently writing code using Reactor/RxJava in Java 11 w/ Lombok. I don't think I've ever been so productive or lead a team as productive as when we were going ham on the functional/reactive Java. Now that I'm back in Python land, I am constantly frustrate on a daily basis with both the language and the runtime at every turn. Even with the asyncio we are working on, it feels like the absolute minimum viable product when compared to the java, node, or rust I have done. There are some fantastic python enhancements that bridge some of the gaps like PEP 484 (Type Hints) and PEP 517 (holy crap an actual build systems that are not dogcrap) but it feels like the python community does not care.
- geofft 5y ago> Julia says: > A language must compile to efficient code, and we will add restrictions to the language (type stability) to make sure this is possible. > A language must allow post facto extensibility (multiple dispatch), and we will organize the ecosystem around JIT compilation to make this possible. > The combination of these two features gives you a system that has dynamic language level flexibility (because you have extensibility) but static language level performance (because you have efficient code) Given those constraints, the first language that comes to mind is Java. Why is Java basically not a player in the scientific-computing game?
- adgjlsfhk1 5y agono data types, no operator overloading, no extendability of new functions to existing objects, no C/Fortran interop, mediocre performance. Need I go on?
- fantod 5y agoAlso no (official) REPL.
- AstroDogCatcher 5y agoThe Julia crowd really need to hang less value on the REPL - people writing serious production code don't use one, and scientists using Python have moved on to notebooks. At this point, having Julia be so focused on the REPL is a weird affectation.
- jakobnissen 5y agoI used notebooks when I did my PhD using Python, but have moved to the REPL using Julia on my PostDoc. The Julia REPL experience is so much nicer than Python's, it removes the need for notebooks completely for me.
- patagurbon 5y agoJulia has great notebook support, the Ju in Jupyter stands for Julia after all. There's also the fantastic Pluto.jl for reactive notebooks. And the VSCode extension isn't quite IDE level, but that's just a matter of resources.
- jszymborski 5y agoI've been thinking a lot about trying to use functional dialect of Python like Coconut[0] and Hy[1] w/ JAX so I can write functional DL code. Glad to see functorch[3] as PyTorch is the library I have the most experience with. [0] http://coconut-lang.org/ http://coconut-lang.org/ [1] https://docs.hylang.org/en/alpha/ https://docs.hylang.org/en/alpha/ [2] https://github.com/google/jax https://github.com/google/jax [3] https://github.com/pytorch/functorch https://github.com/pytorch/functorch
- 71a54xd 5y agoI was surprised when browsing PaperSpace.com (a gpu host for ML training) that Fast.AI is now considered a "legacy" software? I've built a few small classifiers / ML projects but not really enough to really branch out of an intermediate tutorial. With how quickly these frameworks change it's overwhelming to keep pace! Anyone have advice for solid frameworks that can reasonably leverage GPU's without too much heavy lifting?
- jstx1 5y agoKeras is integrated into TensorFlow and it's as solid and easy as it gets if you need a high level API for deep learning. If you need to write your own modules PyTorch is probably a better choice.
- nafizh 5y agoThey must be talking about fast ai version 1. Version 2 is used everywhere now and development is on-going as usual.
- FridgeSeal 5y agoThis seems so silly to me. It’s PyTorch-if they said “the next version of PyTorch will be in Julia, the ecosystem would shift accordingly. They’re practically saying “this language has every feature we need and want, most of them already existing, but we’re going to continue re-inventing them in this objectively less suitable language because we clearly wish to make life harder for ourselves”
- driscoll42 5y agoOr I read it as "We want to make life as easy for our userbase as possible, so we will put more work on ourselves to make our users lives easier" which is an attitude I very much appreciate.
- fault1 5y agoagreed, and this always been the driving philosophy of pytorch, and perhaps why it kind of won so much brainshare against tensorflow despite _long_ odds when torch was ported from lua. Soumith Chintala had a keynote talk in juliacon where he focused on these points; https://www.youtube.com/watch?v=6V6jk_OdH-w https://www.youtube.com/watch?v=6V6jk_OdH-w
- dnautics 5y agowere those odds really long? TensorFlow is a beastly mess; and there were some very serious breaking issues between minor version revisions, it was so bad that I used to work at a company where we joked "this company only exists because people can't install tensorflow". We also used to joke that in order to install TensorFlow, step 1: install the JVM.
- rg111 5y agoExactly. PyTorch is not only easy, but is a joy to work with. Among researchers, TensorFlow is rapidly losing ground to PyTorch, and, I think, will keep losing ground until it becomes a niche and only used by Googlers and some others. https://horace.io/pytorch-vs-tensorflow/ https://horace.io/pytorch-vs-tensorflow/
- pilooch 5y agoLoving pytorch but the reasoning remains strange to my ears: Python at all cost, not Julia, because of the ecosystem, well OK. But all bullet points are about things that are easily done right now with libtorch (pytorch underlying C++ core code), and the hassle is... Python. Well rational conclusion would be, just do everything in C++, and bind to Python. Make C++ first citizen here, since in all cases it'll be needed for performance, forever.
- anothernewdude 5y agoIf Julia had wanted to be taken seriously then it shouldn't have dropped the ball at the very first hurdle by having 1-based array indexing.
- dunefox 5y agoThis is such a non-issue and I'm tired of reading a version of this comment in any thread where Julia is mentioned.
- freemint 5y agoThis is a total non issue as indexing is an operation that is subject to multiple dispatch. For a humorous example see https://github.com/giordano/StarWarsArrays.jl https://github.com/giordano/StarWarsArrays.jl
- ChrisRackauckas 5y agoThe person who comes to a Julia discussion and says "so, how's 1-based indexing?" is the same person who walks up to people in the office and goes "so, how's the weather today?" every single day. It's the Smash Mouth Allstar of programming language conversations. The only things interesting about the conversation at this point are the ever more elaborate attempts at semi-comedically changing the topic towards something that's not so overly repeated.
- fault1 5y agoJulia (and Fortran) have a concept called offset arrays where you can basically start on any sort of index: https://github.com/JuliaArrays/OffsetArrays.jl https://github.com/JuliaArrays/OffsetArrays.jl IMHO, one of the biggest advantages of Julia _is_ arrays.
- nothrowaways 5y agoWhy not go? Go beats Julia in parts where python is not good at. Is it because fb vs Google?
- cookieater 5y agoTry using Go for any serious math project, then do the same using Julia. Report back as to how both approaches went :P. From someone who uses both languages for very different tasks regularly, I would never try to write Torch from scratch in Go. I can't envision a way for it not to be a serious maintenance or performance disaster. Maybe that's a lack of my own creativity, but I'd much sooner use C++ rather then write any large portion of it in Go. If only for template generics...
- dunefox 5y agoIt's because Go is a 90s (procedural) language in 2021 and I would maybe use it in a parallel universe where many other languages don't exist.
- ChrisRackauckas 5y ago>Go beats Julia in parts where python is not good at. I have not seen good results from differential equation solvers in Go.
- cookieater 5y agoI don't think you ever will unless Go >2.0 is a completely different language.
- ChrisRackauckas 5y agoYup exactly. I hate these super general statements like "better ecosystem". These blanket statements obviously make no sense since you can pull out 20,000 counter examples off the top of your head. Instead of "Go beats Julia in parts where python is not good at", name the one example where this might be true, maybe possibly, if there is one (is there is one?) to have a concrete discussion on. The internet and its vacuous statements are better for comedic value than discussion. We're at least on the same page.
- cookieater 5y agoI've used Julia for quite a few years now. It's biggest flaws in my opinion are basically cultural and not technological. It's been adopted mostly by serious domain experts rather then typical software engineers and more 'normal' people. I don't know say junior or senior scientists. This has lead to amazing results but also has it's own detriments. Some portions of the ecosystem are rock solid, especially the parts where JuliaComputing makes money from consulting(not all but some). Other parts are beds of sand/permanent research projects. The median experience is usually someone points you to a package and it doesn't really do what you hoped it would so you end up adapting it and rolling your own solution to a problem. Maybe you try to make a PR and it gets rejected because of "not invented here"/academia mindsets, either way you made a fix and your code works for you. What makes this barrier hard to overcome for adoption is: trust, and blind spots. People who aren't experts in a casual work area (maybe computer vision) realize they can't use a tool to do something `basic` and run away to easier ecosystems(R/Python). People who are experts in other areas, check credentials of packages see that an ivy league lead researcher made it and assumes it's great and usable for a general audience. So you'll get a lot of "there's a package for that" but when you go to use it you might find the package is barren for common and anticipatable use cases in industry (or even hobbies). This makes Julia best positioned as a research tool, or as a teaching tool. Unfortunately - where Julia actually shines is as a practical tool for accomplishing tasks very quickly and cleanly. So there's this uncomfortable mismatch between what Julia could be and what it's being used for today. (yes Julia can do both not arguing against it). The focus on getting headlines far outsurpasses stable useful stuff. Infact, very often after a paper gets published using Julia, a packages syntax will completely change - so no one really benefits except for the person who made the package. Interestingly, 1 person(with some help of course) fleshed out the majority of the ecosystems need for interchange format support(JSON), database connections, etc. It's not like that person is jobless spending all their days doing it - it was a manageable task for a single smart person to kick off and work hard to accomplish. Why? Because Julia is amazing for quickly developing world class software. That is also kind of its detriment right now. Because its so easy to create these amazing packages you'll find that a lot of packages have become deprecated or are undocumented. Some researcher just needed a 1 off really quickly to graduate, maybe the base language(or other parts of the ecosystem) changed many times since its release. Furthermore, if you try to revitalize one of these packages you'll sometimes find a rats nest of brilliance. The code is written very intelligently, but unpacking the design decisions to maintain world class performance can be prickly at best. One of Julia's strengths is it's easy/clean to write fast enough code. One of its downsides is, this attracts people who focus on shaving nanoseconds from a runtime (sometimes needlessly) at the expense of (sometimes) intense code complexity. Performance is important, but, stable and correct features/capabilities mean more to the average person. After-all, this is why people use, pay for, hire for: Matlab, Python and R in the first place - right? Most people don't want to have to figure out which ANOVA package they should use. Or find out in a bad way some weird bug in one of them and be forced to switch. Meanwhile in R: aov(...). Do I blame Torch for not using Julia? No. Should they consider using it? Yes, absolutely. Does Julia's cultural issue need attention before risking Python(or anything else) reinventing a flavor of Julia that's more widely used for stability reasons alone - in my opinion, yes (see numba, pyjion, etc). Still love the language, because technologically it's sound, but there are blemishes. I'd chalk it up to growing pains.
- streamofdigits 5y agoIt feels a risky proposition at this juncture to go short python. The arguments against/for have been rehashed a million times, the redeeming features of julia have been articulated very cogently... What has not been accounted for is that the huge community / network effect of the python ecosystem is very far from exhausting itself. If anything, it is just starting as the exponential growth has mostly been the last few years (tautology, he he) A major investment to eliminate python technical debt would make more sense if things were stagnant and the re-engineering would open up entirely new domains.
- adsharma 5y agoI'm surprised that no one brought up using a subset of python with an emphasis on static typing, efficiency and transpilation can give you both the ecosystem and the efficiency.