3 ms·
I partly agree with you, in that the author is mainly complaining about Flux and other packages they've used. But on the other hand, all the reasons the author
by blindseer 4y ago
I partly agree with you, in that the author is mainly complaining about Flux and other packages they've used.
But on the other hand, all the reasons the author has listed as annoyances I've experienced the exact same things and more. And I'm definitely not blaming the package maintainers for this. There's either 1) something wrong with the Julia language that results in code that is harder to maintain at scale, or 2) missing tooling to help package maintainers solve some of these problems.
In my personal opinion (and also almost universally the opinion of 20-25 of my colleagues some of whom are way smarter and more experienced than me), it is a little bit of both. Tooling like JET for linting is just taking off, and for years people have been writing and maintaining packages without linters. The ONLY way to ensure the Julia code you write is to write LOTS and LOTS of tests. In Python, if I use type hints and run mypy a whole class of errors is avoided. That's not the case in Julia. At work, we maintain a close to 10,000 line Julia code base and honestly if we'd known the tooling was going to be the way it is, we'd not have chosen Julia.
The fact that startup time is so painful + coupled with the fact that tests are basically the only way to guarantee that you are writing correct code, makes it EXTREMELY painful to work in a large team. Our internal CI takes hours to run for even the smallest changes. Even a small scale refactoring takes weeks of review from everyone involved and it is a time consuming frustrating and expensive process for the organization.
I've worked with similar size codebases in Python and C++ and have had no where near this kind of difficulty.
I can't find the link to the talk right now, but Bryan Cantrill talks about "values" of a programming language / ecosystem. And based on the way Julia is designed and implemented, I'd say the "values" are "run time performance", "scientific applications", "research suitability" (ease of creating environments, DataFrames.jl for data science etc). I think "correctness" as a value is FAR from the priority in this language, which incentivizes researchers to write code that gets the job done quick but fails to be long term maintainable or viable.
The worst part is I don't know how Julia will ever solve these issues. Leveraging LLVM to generate fast machine code at runtime is the very basis of the language, so you'll almost always (unless static compilation happens) pay a penalty at start up and first run. Using the type system for dispatch means that static analysis is just going to suck. How JET works is a mystery to me, so I'm not sure what the extent of the improvements there can be, but based on talking with others in the Julia community, I don't think it'll ever rival Python's or CPP's static linting capabilities.
The Julia devs are constantly working on improvements, so I expect things will get marginally better and maybe in 5 years or so it'll be a tolerable experience. But all in all, I'm not holding my breath.
- dklend122 4y agoPlease see my comment here: https://news.ycombinator.com/item?id=31269739 https://news.ycombinator.com/item?id=31269739 Based on the above, I expect more than marginal improvements in these areas. Would you agree?
- chalst 4y agoI guess the Cantrill talk you are referring to must be 'Software as a reflection of values' from 2018: https://www.youtube.com/watch?v=2wZ1pCpJUIM&t=128s https://www.youtube.com/watch?v=2wZ1pCpJUIM&t=128s