6 ms·
Don't ever try Scala. I work on a mid-sized CRUD application, and everything we need to recompile from scratch it takes 10 minutes. Typescript and Go are amazin
by fyfy18 6y ago
Don't ever try Scala. I work on a mid-sized CRUD application, and everything we need to recompile from scratch it takes 10 minutes. Typescript and Go are amazing in comparison in that department, but I miss the stronger typing of Scala.
- logicchains 6y ago>Don't ever try Scala. Don't ever try C++. I work on a medium-sized low-level C++ application, and every time we need to compile from scratch it takes over two hours on a single core. Even with distributing the compilation out across over a hundred cores the fastest it can get down to is around 20 minutes.
- Iwan-Zotow 6y agohow come hundred cores is only speedup of ~6?
- IshKebab 6y agoOn large C++ projects there are usually a ton of steps that can't be perfectly parallelized. Linking is usually single threaded, you might have dependencies that are way less than 100 CPP files. CMake is single threaded and very slow. Lots of projects have custom Python scripts for code generation or whatnot - all single threaded.
- gonzo41 6y agoYou can only say the alphabet one way, starting from A and ending in Z. Some tasks can't be parallelized. I expect a lot of those cores wait for critical paths to execute.
- fulafel 6y agoAmdahl's law.
- Syzygies 6y agoNot a law. Haskell can be much closer to linear, performance to cores. It starts banging its head on the last core; the little bit of other work the computer does seems to throw off Haskell's gait. (This is subjective first-hand experience.)
- logicchains 6y agoLinking in particular. At one stage, due to enabling LTO, our release build alone took over 20 minutes just to link. And linking cannot easily be parallelised (if at all? I'm not familiar with what the state of the art in linkers is).
- uluyol 6y agoI believe that ThinLTO does most of its work in parallel and provides most of the benefit of LTO.
- jedisct1 6y agoDon't ever try Rust.
- throwaway894345 6y agoI haven’t used Rust on any significantly sized project. Could you elaborate? Is it only slow to compile release builds?
- steveklabnik 6y agoIt is generally slower to compile than we would like. We have and will continue to spend a lot of engineering time improving it. It is often one of the largest feature requests from the community.
- erik_seaberg 6y agoAny compiler that's faster than a human brain is a win. More than that is gravy, and thank you for it.
- thurn 6y agoI started learning rust as a quarantine project, and my <5000 line project takes ~10 seconds to do an incremental development build and 1-2 seconds to typecheck
- lima 6y agoI work with a ~10k line project and it takes 60s to incrementally compile on a Ryen 3990X. Rust is great, but the compile times do need to improve :-)
- shurikdima 6y agoWhat do you mean by "incrementally compile"? an addition of a single line of code results in a 60s build? never experienced anything like that, only the first compile is slow for me.
- ben-schaaf 6y agoThe thing about C++ compilation time is that if you care you can spend the time to optimize it. Our app takes ~40 seconds to compile and run tests on a decent machine (16 cores).
- ainar-g 6y agoThis is interesting. What techniques are you using? Pimpl? Custom, non-STL containers?
- kcolford 6y agoI find headers with templates are always the problem. So external template instantiation and pre compiled headers can solve a lot of first time compilation slowness. Pimpl is more for reducing how much you have to change headers forcing a recompile during development. It's also really handy for keeping binary compatibility with published dynamic libraries so that you don't have to recompile dependants.
- logicchains 6y agoThat only works if you can clean up slow-compiling code faster than other people are writing more slow-compiling code.
- monocasa 6y agoOr... you set up your code standards to enforce the same constraints. Everyone pitches in and keeps it clean.
- jaco8 6y agoJust try Nim. It is fast.
- logicchains 6y agoNim also supports fancy compile-time metaprogramming and templates. What does it do to make itself compile faster than C++ and Scala?
- seertaak 6y agoI can't speak to Scala's compile time issues, but for C++ there are few factors contributing to long compile times. - "module" system based around textual inclusion of header file: essentially the compiler duplicates a lot of effort, reading the same files over and over for each translation unit, and you can't avoid the cost because of the interaction with text macros. That said, C++20 offers a solution through its new module system; it will be interesting to see how much of an improvement this will bring. - complicated parser requiring contextual information (i.e. information from typing and analysis phases) in order to distinguish between, say, a class object instantiation and function forward declaration (the notorious most vexing parse). Most other languages learned from the syntax mistakes of C++, but C++ had a clear mandate to not break existing C code, so it had no choice but to build what existed. - C++ template system revolves around a kind of syntactic substitution, where you have a template expression with some type variable, and at the template instantiation site (i.e. every call, not at the definition) you you essentially do template argument deduction yielding concrete type bindings for the type variables, and then your next step is the transform the AST with the template expressions into concrete types, functions, etc. This involves a lot of tree transformations which can be expensive. I think with other languages this is less of a problem in part because the type checking is done at the definition site, rather than at call site. Here again C++ concepts were introduced to address this issue, but as always in C++, it's not all milk and honey.
- ed25519FUUU 6y agoNim, the language everyone loves and nobody uses.
- lenkite 6y agoI guess the app uses lots of "header-only" libraries ?
- bad_op3rator 6y agoMb the ppl who work on that application should invest a little bit more time in learning how to use C++ properly. I know it sounds a bit silly but if you deal with C++ you should learn it first enough to use it in the right way, or just switch to another programming language, Go is a good candidate.
- gigatexal 6y agoCcache?
- dig1 6y agoIt's harsh to say this is due C++ fault, more likely due development culture and desire to be cool by using all the new cool features, without understanding benefits vs price. I was running medium-sized C++ project that included full GUI library - everything compiled under 2 minutes (including tests) on a single core. Formula known for years: keep it simple, minimal template (ab)use and know what you are putting in .h/.cpp files. Cramming everything into headers hoping compiler will inline stuff is a bed time story. Also, your build system will be grateful knowing that every compilation unit is independent as possible; it will take less time to calculate dependencies and will easily distribute compilation across cores, if requested.
- aphexairlines 6y agoHow many lines/files/classes split over how many targets for that 10 minute build? I just compiled around 240k lines of scala split over around 200 gradle subprojects with gradle 6.5rc1 and scalac 2.12.11. Limited to 10 gradle workers. Took 4 minutes without the gradle build cache enabled.
- melling 6y agoWhat’s the trick to your faster compile times?
- aphexairlines 6y agoIf you model your gradle (or bazel, or maven, or sbt) project as many smaller sub-projects, then the build tool can run a pool of compiler processes in parallel. In the case above, the repo I compiled was broken down into 200 gradle sub-projects. Bazel builds tend to be finer-grained still. Also gradle 6.5 fixes a scala compilation performance regression that was introduced in gradle 6: https://github.com/gradle/gradle/issues/12591 https://github.com/gradle/gradle/issues/12591
- dhash 6y agoSwitching off SBT to Bazel made writing scala so much more pleasant for this exact reason. Builds are 10x quicker, and the caching makes the initial build take 10 minutes, limited only by the speed of your internet connection!
- sivakon 6y agoDo you have any workable example? All the examples I tried haven’t worked on latest bazel version.
- mikelward 6y agoIs that due to Scala? Or SBT? https://www.lihaoyi.com/post/SowhatswrongwithSBT.html https://www.lihaoyi.com/post/SowhatswrongwithSBT.html https://engineering.linkedin.com/blog/2018/07/how-we-improved-build-time-by-400-percent https://engineering.linkedin.com/blog/2018/07/how-we-improve...