15 ms·

Crazy fast build times, or when 10 seconds starts to make you nervous (2012)

108 points by kiyanwang 6y ago

hardwaresofton 6y ago
Wayback machine if the site is still down for anyone: https://web.archive.org/web/20210324071034/http://dan.bodar.com/2012/02/28/crazy-fast-build-times-or-when-10-seconds-starts-to-make-you-nervous/ https://web.archive.org/web/20210324071034/http://dan.bodar....
gopalv 6y ago
As a summary, switched from scala → java, ant → make, removed GWT, Lucene, Hibernate, Velocity, Tomcat etc. This needs a (2012) on it, but it looks like that got hugged to death already [1]. [1] - https://web.archive.org/web/20210324055256/http://dan.bodar.com/2012/02/28/crazy-fast-build-times-or-when-10-seconds-starts-to-make-you-nervous/ https://web.archive.org/web/20210324055256/http://dan.bodar....
- jbverschoor 6y ago
  When I was still deep in the Java space, I always used hot code replace, otherwise it’s really unbearable. I also contributed a patch to hibernate to cache the compiled classes. That sometimes saved minutes per restart.
- eloisant 6y ago
  It's important to note that Scala compile time got better, both with improvements in the compiler and in the computers we're using (going from HDD to SSD was huge, and of course CPU improvements). Still slower than Java but I think it's well worth it for the productivity gain.
  - MrBuddyCasino 6y ago
    > Still slower than Java but I think it's well worth it for the productivity gain. I have never seen this play out it practise. The most productive engineers I know can touch type well and know their editor/IDE inside-out, have a solid grasp of the basics and preferring not too many abstraction layers, and a workflow optimised for fast dev cycles. Sure, build your startup with Scala or Haskell - I'll bet on the Ruby / Golang / Lisp competitors.
  - perennus 6y ago
    My project has 17k lines of Scala code (no whitespace). I don't have a SSD. It takes 10 minutes to perform SBT Assembly from a cold start, 40 seconds afterwards. And about 1-3 seconds to compile and run locally within IDEA.
lolc 6y ago
That was an interesting read for someone like me who just started splitting builds into frontend/backend to get build times down. That they would switch languages just to get faster builds sounds extreme to me. But in the end I already noticed how much better it feels when the CI-cycle is minutes and not tens of minutes. The suggestion to split off code with long-running tests into separate libraries is easy pickings. So I think I'll do that next time I wait for those tests to complete.
gumby 6y ago
Back when I compiled on a cray I had problems with make because files could compile within a second so make would want to recompile them. I use a lot of C++ templating so these days fast builds are just a fantasy no matter what I do. Unlike the author I rely on incremental builds for development but do a full build before pushing
- algo_trader 6y ago
  c++ templates are a real killer. Have u tried a cloud compiler? Did your company ever try a build server ?
  - gambiting 6y ago
    I work in video games, large AAA game with absolutely crazy amount of templating, the compilation time is about 10-15 minutes, and that's with Fastbuild and hundreds of workstations participating in the build process. Once it's built it's a bit quicker, about 60 seconds from pressing build to the game starting, the linking is killer, it takes soo long every time.
    - jcelerier 6y ago
      > the linking is killer, it takes soo long every time. did you try using lld for linking ? even on windows, it's much much much faster than link.exe
      - gambiting 6y ago
        Yes, we use Clang for everything, including lld
- pjmlp 6y ago
  No use of external templates or having common subset in a binary library and then use private inheritance?
  - 6y ago
david_allison 6y ago
QCon talk (2012): https://www.infoq.com/presentations/Crazy-Fast-Build-Times-or-When-10-Seconds-Starts-to-Make-You-Nervous/ https://www.infoq.com/presentations/Crazy-Fast-Build-Times-o...
bullen 6y ago
I use hot-reloading of tiny modules to get turnaround below 1 second on all my projects, server and client: My Java HTTP server hot-deploys a .jar containing new code with a simple classloader: https://github.com/tinspin/rupy https://github.com/tinspin/rupy My C+ 3D MMO engine uses .dll/.so hot-reloading so that the game code can change while the engine is running, it's not released yet: http://talk.binarytask.com/task?id=5959519327505901449 http://talk.binarytask.com/task?id=5959519327505901449
- Narann 6y ago
  Very nice work. About hot-loading: Where do you put the frontier between each modules? How big is each of them? Maybe you could share a typical module list? Keep the good work!
exikyut 6y ago
An incidental aside, but I actually wrote a comment yesterday on the benefit of using the fastest tooling you can find, along with a restart-from-scratch strategy (non-incremental, non-"reload these specific things while juggling ABI compatibility with X Y and Z already in memory") similar to clean rebuilding. It's a textbook wall of text, but IMO represents a bit of anecdata that 100% agrees with the idea that lowering build and processing times are absolutely worth it. https://news.ycombinator.com/item?id=26555366 https://news.ycombinator.com/item?id=26555366
presentation 6y ago
Right now I can't stand that my Create React App builds for a TypeScript project take several minutes to build on CI machines. Makes me want to ditch it entirely!
- christophilus 6y ago
  Do it. My Preact builds with esbuild take a hundred milliseconds or thereabouts. The slowest part of my build process is waiting for TwilwindCSS to minify.
oftenwrong 6y ago
I've started experimenting with the Bazel-like https://please.build/ https://please.build/ , which has actual support for incremental builds. I also advocate avoiding a lot of the typical dependencies in the Java ecosystem. There is a lot of bloated garbage that you'll miss less than you might think.
- EdwardDiego 6y ago
  What are the typical dependencies, in your mind?
  - oftenwrong 6y ago
    I should say lots of typical dependencies are fine, but some are common "problem children" or ways to shoot yourself in the foot. The Spring ecosystem comes to mind. With regard to the development cycle, Spring seems to turn things that normally would be compile-time errors into runtime errors. If I've wired up my components incorrectly, I'd prefer to find out sooner rather than later, so as not to waste my time.
    - EdwardDiego 6y ago
      Ah yep, thought you might have been referring to Spring, and totally agree on runtime vs. compile time. I've switched to Micronaut of late for that precise reason - all the injection happens at compile time, so far quicker to pick up. Downsides though are that Micronaut sometimes doesn't provide super-helpful error messages - or silently fails to do DI when `micronaut-inject-java` isn't in the dependencies.
josephg 6y ago
This is one of the things that really makes me hesitate with Rust. I have a pretty small database project which sometimes needs to slowly stream updates to a web browser. Pulling in actix and actix-web added over 1s to my debug mode build time on a fancy new zen3 computer. The web front end isn’t the core of this project - which makes the slowdown really sting. I’m tempted move all the web / actix stuff behind a compilation flag just to keep my coding flow while working on the rest of the project. I know compilation time is being worked on, but it’s still way slower than it could be. I’d love to see an incremental linker for rust some day that’s able to just replace the methods in the binary which were actually changed between builds.
- IshKebab 6y ago
  I agree. Hopefully when the Cranelift backend works it will get a lot faster, but even `cargo check` can be pretty slow, and I'm not sure if there is any way around it. One idea I had: can crates.io generate machine independent Rust IR for all of the crates and have cargo download it? Then at least you don't need to parse/check/compile the source code of dependencies for clean builds. Probably wouldn't work for any crates that use `build.rs` or conditional compilation anywhere though (which is probably most of them).
  - hobofan 6y ago
    Apart from the conditional compilation problems, you would also have to pre-compile for all different Rust versions. Given the size of a typical `target` directory the artifacts for just a single Rust version will also probably be ~10x as big as the current crate source.
    - IshKebab 6y ago
      Mmm good point. An ideal solution for CI would be something like sccache, but I think to get that to work reliably you need to consider it from the beginning (like Bazel did) and Rust just didn't. `build.rs` can do literally anything. Proc macros can do anything. Even a simple `include_str!()` can break distributed compilation. It would be great if Cargo had a "hermetic" mode where you couldn't use `build.rs` and proc macros couldn't do any IO, etc.
toolslive 6y ago
back in the HDD day, plenty of memory + a ram disk would really help.
- WildParser 6y ago
  I recently experimented with a ram disk. Practically it didn't change anything. OS-caching seems to already be clever enough and once the OS has figured out that some directories are important anything in there seemed to get done in RAM anyway. A RAM-disk will make this less black box and more deterministic regarding guaranteed access times, but in daily use the RAM-disk just didn't make a difference.
  - ghusbands 6y ago
    Which OS? And were you already on SSD? Those can both greatly influence the outcome.
    - WildParser 6y ago
      I'm on Windows. Other OS may differ. I experimented a lot with load times - and compared HDD, SSD, RAM-Disk. I wrote a small program that loads everything from a big code-repository into RAM. The first time HDD and SSD and RAM-Disk make a big difference, when reading files a second time the lag of HDD (50s?) almost disappeared completely. Caching kicked in. The RAM-Disk has less initial lag, but also it has to be filled first, so instead of moving everything to RAM-Disk just touching everything so the OS-Caching kicks in is just faster and more convenient.
EdwardDiego 6y ago
I definitely agree with "no Fitnesse", sorry, FitNesse, ugh, and likewise, no Selenium. Not so much for the slowness. In the case of FitNesse (ugh), it's the Wiki -> Java impedance mismatch, and in terms of Selenium, it's the brittleness. Move an element, watch Selenium tests generated by a QA's browser plugin break. I rather loathe Hibernate these days (JDBI is my favourite atm), and well, Tomcat feels rather legacy. But I guess that's the advantage of 9 years of advances. > you can use the maven structure and produce maven artifacts with your build but use a different tool (Make, sbt, buildr, ant etc) Omg, no, sbt is far, far, far, worse. Ditto ant. Not sure if buildr is still a thing? Looks like the last release was 2017. And as for make, how would you even integrate various Maven plugins into it?
- cies 6y ago
  > JDBI is my favourite atm Tried JOOQ? There's generator overhead on db schema changes, and it adds lots of classes to your classpath (but this could be inside a precompiled library). I guess it adds some build time, not sure how much though (for sure more than JDBI). But that sweet typesafety and auto-complete is worth something.
- karmakaze 6y ago
  Interesting choice of JDBI. I was working on an SQL-friendly ORM[0] also due to distaste with Hibernate/JPQL and chose JDBI, not because it was great in any way but it did what I needed and not much else. What influenced your choice and were there any close runner-ups? [0] https://github.com/karmakaze/safeql https://github.com/karmakaze/safeql
  - asadawadia 6y ago
    have you looked into ktorm?
    - karmakaze 6y ago
      I wasn't looking at Kotlin libraries, but this one seems to be very close to what I was going for. Specifically about being able to compose queries in a bottom-up fashion like the UnionAll example with subqueries. It's unfortunate that they chose to use .filter (.where would have been clearer) for DB executed conditions as well as post-fetch sequence filtering.
unnouinceput 6y ago
And at the other end of the spectrum you have 1M lines of code compiled in just 19 seconds, using latest Delphi compiler (https://blogs.embarcadero.com/delphi-10-4-2-compiler-speed-improvements https://blogs.embarcadero.com/delphi-10-4-2-compiler-speed-i...)
- WildParser 6y ago
  Great news... Compile time or a clean build? I use similar tooling (older version... :-)). And I'm starting to evaluate what's going on with those new versions.
  - unnouinceput 6y ago
    If you watch the video closely you'll see he's doing a full build for the entire components using a test project.
- PixelOfDeath 6y ago
  Pascal was an awesome learning language alone for its compile speed.
  - unnouinceput 6y ago
    It still is. Under Linux you have Free Pascal + Lazarus and can target, natively, everything under the moon (see target section for: https://en.wikipedia.org/wiki/Free_Pascal https://en.wikipedia.org/wiki/Free_Pascal)
WildParser 6y ago
Comparing compile times is done rarely. I sometimes tried to google some comparisons, but didn't find much. For a typical 1,5 MLOC project my clean build times are sitting around 40 seconds. I think that's pretty good, but I don't think I have a way to improve it much. Without writing a compiler from scratch I think 10 seconds are out of reach for me...
- algo_trader 6y ago
  Do u think your company will consider cloud compiler if this is x5 faster? There issues such as latency, delta synchronizing, and tool versioning. But its doable. If u build such a service (pun intended!), the issues are probably cost and security.
  - WildParser 6y ago
    Improvements in build-time are very valuable to me. But I'm not sure if a cloud would help me much. Synching can easily eat up 10s. Also my tools are very unique (including a parser that I wrote myself). To be faster than local a cloud would have to be massively overclocked - and I can already reach good results locally. For actual improvements I need a massive improvement in compiler technology (last time I looked there were just too many single-threaded bottlenecks in my build process). Nothing a cloud can solve for me. Security is one issue - but hackers will most likely only get confused when they try to understand what is going on - also costs I usually don't like...
erik_seaberg 6y ago
Did he measure how much worse the Java version is to work on? Scala infers a lot of boilerplate and wasted effort a lot faster than I can write or review it, because I'm made out of meat.
deleted 6y ago
[deleted]
gjvc 6y ago
rollcall of essential tools if you are in C/C++ land: no build should be without https://ccache.dev/ https://ccache.dev/, and after that, distcc https://distcc.github.io/ https://distcc.github.io/, icecream https://github.com/icecc/icecream https://github.com/icecc/icecream, and goma https://chromium.googlesource.com/infra/goma/ https://chromium.googlesource.com/infra/goma/ may be used for distributing the workload.
- rurban 6y ago
  Can I use that with GitHub actions? I have some crazy slow C/C++ build times with O2 and asan, which usually leads to g++ timing out. Not just 40min build times. Some huge autogenerated testcases to cross-test all iterator combinations against C. I'd really need -O3 for -finline, but -O2, -O3 and -Ofast all fail too build. Splitting it up didn't help. So far I went with #pragma optimize O1. https://github.com/rurban/ctl/actions/workflows/master.yml https://github.com/rurban/ctl/actions/workflows/master.yml C++ templates are the worst. The pure C variant builds in seconds. I pity all C++ folks
  - gjvc 6y ago
    my problem right now is that the OOM killer, is, err, killing compiler processes when asan is enabled. A build machine should have at least twice as much RAM in GB as it has number of cores for compiler processes.
herpderperator 6y ago
Is anyone else (who is not using a caching DNS server from their ISP or other publicly available one) having trouble resolving the domain? There is no A record -- in fact there is no domain at all as far as the root servers are concerned: http://www.dollardns.net/cgi-bin/dnscrawler/index.pl?server=a.root-servers.net&name=dan.bodar.com&type=A&class=IN&lr=12&submit=Send+Query#report http://www.dollardns.net/cgi-bin/dnscrawler/index.pl?server=... Snippet from the above link (to keep this comment relevant once things eventually get fixed): Querying a.root-servers.net (198.41.0.4)... delegated Querying e.gtld-servers.net (192.12.94.30)... name not found Unable to find: dan.bodar.com No Answer Records Looks like the domain record was recently updated to clientHold (about an hour after this article was posted): server ~ # whois bodar.com Domain Name: BODAR.COM Registry Domain ID: 4584805_DOMAIN_COM-VRSN Registrar WHOIS Server: whois.joker.com Registrar URL: http://www.joker.com Updated Date: 2021-03-24T07:06:55Z Creation Date: 1999-03-22T05:00:00Z Registry Expiry Date: 2022-03-22T04:00:00Z Registrar: CSL Computer Service Langenbach GmbH d/b/a joker.com Registrar IANA ID: 113 Registrar Abuse Contact Email: abuse@joker.com Registrar Abuse Contact Phone: +49.21186767447 Domain Status: clientHold https://icann.org/epp#clientHold Name Server: IVAN.NS.CLOUDFLARE.COM Name Server: JEAN.NS.CLOUDFLARE.COM DNSSEC: unsigned URL of the ICANN Whois Inaccuracy Complaint Form: https://www.icann.org/wicf/ clientHold apparently means[0]: > This status code tells your domain's registry to not activate your domain in the DNS and as a consequence, it will not resolve. It is an uncommon status that is usually enacted during legal disputes, non-payment, or when your domain is subject to deletion. Often, this status indicates an issue with your domain that needs resolution. If so, you should contact your registrar to resolve the issue. If your domain does not have any issues, but you need it to resolve, you must first contact your registrar and request that they remove this status code. It's quite surprising how many people on HN are talking about this article just because of however long the TTL was on the original DNS query and somehow almost everyone so far has got a cache hit. [0] https://icann.org/epp#clientHold https://icann.org/epp#clientHold
- sva_ 6y ago
  Presumably they got hugged to death and are currently trying to change to a different server or something. I can't access it either.
throwaway4good 6y ago
Apache Maven can be fast and easily sub 10 second if you just use in a straight-forward manner. From an arbitrary mid-sized project: About 400 classes split into 10 submodules. Clean build without tests: 3.7 seconds.
- bballer 6y ago
  Introduce spring and friends and prepare to see that number jump up at least 4-5x
jillesvangurp 6y ago
Interesting. I've always been aggressive on trying to keep my Java and lately Kotlin builds fast. Anything over a few minutes in CI becomes a drain on the team. Basically a productive team will have many open pull requests open at any point and lots of commits happening on all of them. That means builds start piling up. People start hopping between tasks (or procrastinating) while builds are happening. Cheap laptops become a drain on developer productivity. Etc. All of this is bad. Maintain the flow and keep things as fast as you can. It's worth investing time in. Some of the overhead is unavoidable unfortunately. E.g. the Kotlin compiler is a bit of a slouch despite some improvements recently. Many integration tests these days involve using docker or docker compose. Overall that's better than a lot of fakes and imperfect substitutes. But it sucks up time. A lot of Kotlin and Spring projects involve code generation. This adds to your build times. Breaking builds up into modules increases build times as well but tends to be needed. Be mindful of all this. A few performance tips not covered in the article that may also apply to other languages: - Run your tests concurrently and write your tests such that you can do so. Running thousands of tests sequentially is stupid. When using junit 5, you need to set junit.jupiter.execution.parallel.enabled=true in platform.properties (goes in your test resources). Use more threads (e.g. junit.jupiter.execution.parallel.config.dynamic.factor=4) than CPUs for this as your tests will likely be IO limited and not CPU limited. If you are not maxing out all your cores, throw more threads at it because you can go faster. If your tests don't pass when running in parallel, fix it. Yes, this is hard but it will make your tests better. - Don't do expensive cleanup and setup in between tests. This takes time and integration tests become more realistic if they don't operate in a vacuum (your production system is not a vacuum either). To enable this, randomize test data so that the same tests can run multiple times even if data already exists in your database. Docker will take care of cleaning up ephemeral data after your build. This also helps with running tests concurrently. - Distinguish between (proper) unit tests and scenario driven integration tests as the two ideal forms of a test. Anything in between is going to be slow and imperfect in terms of what it does. This means you can improve test coverage (of code, functionality, and edge cases) by making it a proper integration test or faster by making it a proper unit test (runs in milliseconds because there is no expensive setup). - With integration tests, add to your scenarios to make the most of your sunken cost (time to set up the scenario). Ensure they touch as much of your system as they can to do this. You are looking for e.g. feature interaction bugs, heisen-bugs related to concurrency, weird things that only happen in the real world. So make it as real as you can get away with. A unit test is not going to catch any of these things. That's why they are called integration tests. Make it as real as you can. - Fix flaky tests. This usually means understanding why they are flaky and addressing that. If that's technical debt in your production code, that's a good thing. Flaky tests tend to be slow and waste a lot of time. - Separate your unit and integration tests and make your builds fail fast. Compile + unit tests should be under a minute tops. So, if somebody messed up, you'll know in a minute after the commit is pushed to CI. - Eliminate sleep calls in tests. This is an anti pattern that indicates either flaky tests or naive strategies for dealing with testing asynchronous code (usually both). It's a mistake every time and it makes your tests slow. The solution is polling and ensuring that each test only takes as much time as it strictly needs to. - Run with more threads than your system can handle to flush out flaky tests. Interesting failures happen when your system is under load. Things time out, get blocked, deadlocked, etc. You want to learn about why this happens. Fix the tests until the tests pass reliably with way more threads than CPUs. Then back it down until you hit the optimum test performance. You'll have rock solid test that run as fast as they can. - Keep your build tools up to date and learn how to use them. Most good build tools work on performance issues all the time because it's important. I use gradle currently and the difference between now and even two years ago is quite substantial. Even good old maven got better over time. - Pay for faster CI machines. Every second counts. If your laptop builds faster than CI, fix it. There's no excuse for that. I once quadrupled our CI performance by simply switching from Travis CI to AWS code build with a proper instance type. 20 minutes to 5 minutes. Exact same build. And it removed the limits on concurrent builds as well. Massive performance boost and a rounding error on our IT cost. Most of this advice should work for any language. Life is too short for waiting for builds to happen.
JacobCarlborg 6y ago
If you want crazy fast build times, then D is your best bet [1]. D's compile times are faster than most other languages (I'm not counting Vox, it's too experimental). [1] https://github.com/nordlow/compiler-benchmark https://github.com/nordlow/compiler-benchmark
- brabel 6y ago
  Wow you're not joking... `dmd` is around 40x faster than `rustc` and 16x faster than `zig`! How the hell they've done it?
  - kristoff_it 6y ago
    Check Zig again once the self-hosted compiler is completed :)
    - brabel 6y ago
      Will that remove the dependency on LLVM?
habosa 6y ago
My personal theory of modern software development is that the #1 and #2 things you can to do improve software quality are: 1) Make sure devs can run a realistic version of the whole stack locally. 2) Drive incremental build times and effort on the whole stack to zero. With today's world of frameworks, dependencies, and microservices it's too hard to just "think through" a change. You need to try it, adjust, and try it again. Reducing friction on the iteration cycle makes this task pleasant and increases the likelihood of discovering the best solution. When this cycle is painful devs are likely to stop at the first working solution (or even partially working).
- gnrlst 6y ago
  I find 1) especially challenging with microservices. What technologies do you use/recommend to help with this?
  - Aeolun 6y ago
    Docker compose and very beefy dev machines.
  - rutthenut 6y ago
    This is where a Docker/K8s containerised setup can help. Not trivial to get to, but for deve/test environments it can be really beneficial. Can also mean the same clean starting point for every series of tests, if new containers are spun up each time through.
    - bosie 6y ago
      how do you combine this approach with some external services (i.e. databases aren't run in docker but AWS RDS or some part of the setup is using AWS lambdas)?
      - majewsky 6y ago
        Even if you have your prod database on RDS, you might be able to spin up a local instance for testing. My Postgres-using applications have a script that sets up and runs a local Postgres instance for the duration of the test. It takes about 2 seconds on first run, and less than noticeable time when the DB has already been initialized. https://github.com/sapcc/keppel/blob/master/testing/with-postgres-db.sh https://github.com/sapcc/keppel/blob/master/testing/with-pos...
derriz 6y ago
Kotlin compile times are killing me. In a current project, I have fewer than 20K lines and it's 50ish seconds for a clean re-compile on a very decent laptop (less than a year old, 32Gb RAM, quadcore i7, fast PCIe/NVMe SSD, etc.). Particularly as I'm a test driven type of developer - cycling between running unit tests and adding small fixes is painful. Even when you can trust the incremental compilation, 10 or 20 seconds between making small one-line fixes and the test re-starting means the temptation to check hackernews or whatever kicks in and any chance of achieving "flow" evaporates. I'm almost considering the suggestion (for Scala) in the article - and moving back to Java.
- henryjcee 6y ago
  it's a nightmare isn't it. Are you using the 1.4 compiler?
  - derriz 6y ago
    Yes it's a nightmare. I'm using 1.4.30. In response to your question, I just bumped it to 1.4.32 and haven't observed a significant improvement. Back in the old days, compiler metrics were often quoted using units of 100s of thousands or even millions of lines per second. I know the Kotlin compiler has to do a lot of work but I would have imagined hardware advances should balance this out somewhat. Instead I'm looking at compile speeds quoted in hundreds of lines per second.
- pbz 6y ago
  And I was complaining about a 4s rebuild time for a similar (number of lines) .NET 5 solution
- whateveracct 6y ago
  does it not have incremental compiles? at least Scala incremental compiles are fast. Haskell too!
- john4532452 6y ago
  Compile time is the only reason i moved to reactnative. It has some issues, but its well worth to have the fast build cycle.
giorgioz 6y ago
Somehow webpack/JS/React has way faster build times than Java/Android apps. That's the main reason we completely abandoned fully native apps in Java/Swift in favor of webapps with a tiny native wrapper app for the few only-native calls we need. And to move forward this trend, new bundlers are appearing in JS like esbuild/snowpack that are even faster than webpack. This is why the web will always win in the long run. While the Android team in working on their tooling and Apple on other tooling and Windows on other tooling on the web all tooling is shared. You don't need the OS/platform gatekeeper to come up with faster bundlers. You just need one company/opensource team to make a breakthrough tool. Finally this is the point why I believe TYPES ARE NOT WORTH IT. Types make the whole development slower. To me is better to be able to try and run the code faster in development and try it manually several times than to wait longer for type checks. Typing is overrated in my opinion.
- christophilus 6y ago
  With esbuild, typed code compiles just as fast as untyped code (esbuild doesn’t typecheck just strips the types) so I don’t buy your last argument. Esbuild gives the best of both worlds, imho.
  - Aeolun 6y ago
    The problem with esbuild is that it doesn’t do hot reload (yet?)
    - christophilus 6y ago
      I don’t like hotreload, so I don’t see it as a problem. I do a full rebuild and my browser auto refreshes and that’s good enough for me. I’ve had hot reload get me into an invalid state too often to find it useful.
      - metachris 6y ago
        For working on complex webapps that take a bunch of steps to arrive at a certain state, hot reloading can save a ton of time, since you don't have to redo the steps to get there. One of the bigger time-savers for web development, in my opinion.
AJRF 6y ago
I'm procrastinating on HN right now because my iOS build time is 2 and a half minutes long. About 30-50 seconds on an incremental build. This is true suffering. Oh wanted to update with Specs; 2019 MacBook Pro with 2.4Ghz Intel Core i9, 32GB RAM.
- rudedogg 6y ago
  Are you using CocoaPods, SwiftFormat, or SwiftLint? Running SwiftLint or SwiftFormat in a build phase killed my compile times (SwiftLint especially). It'll help a lot to run them manually or with a git pre-commit hook. ----- I know Swift had really bad compile times for awhile, but anymore I think it's pretty fast. I've moved all my WIP stuff into a single project (113 Swift files, ~5,500 LOC) for my convenience, and incremental builds are around 3 seconds on a 2015 MBPr. This article shows some flags you can use to find code that is slow for the compiler to deal with: https://www.avanderlee.com/optimization/analysing-build-performance-xcode/ https://www.avanderlee.com/optimization/analysing-build-perf...
airhead969 6y ago
When I used to use Go, I always wondered if there were stale cached object files because the tests and builds were so fast. I'm not complaining, because I want nearly instant builds or even continuously running background live builds.
RcouF1uZ4gsC 6y ago
The thing that worries me about this post is that they are making a lot of sacrifices in the name of build speed. They are switching languages and libraries and changing their code. Basically, they are creating some amount of tech debt in the name of build speed. Maybe because I started out with C++ on a relatively slow computer, build speeds have not been much of an issue for my productivity. Especially with code completion/etc running and underlining in red my errors as I typed. I guess I kind of treated build as an asynchronous process where I could spend time thinking about the next thing I wanted to do. That being said, definitely get an SSD and a beefy machine, as low build times are nice, but not worth changing your language, libraries, and code structure.
danielrpa 6y ago
It's funny how hardware is SO FAST today but yet we can't get the performance people used to get from... Borland Turbo Pascal in the 80s!
valyala 6y ago
Build times for a time series database written in Go [1]: git clone https://github.com/VictoriaMetrics/VictoriaMetrics cd VictoriaMetrics time go build ./app/victoria-metrics/ real 0m0.574s user 0m1.252s sys 0m0.534s [1] https://github.com/VictoriaMetrics/VictoriaMetrics https://github.com/VictoriaMetrics/VictoriaMetrics
nitzpetter 6y ago
Yes I think it is also possible but the truth remains that the deep web has made it easy for us to hire hackers anonymously without implicating ourselves. One of the places where you can find them will be Dark Basin forum where most of them are screened first before they allowed on that platform too. I was able to get a phone clone from one hacker grup BarrysanchezAThackermailDOTcom and the experience was strictly anonymous till date. another place you can find hackers will on cryptBB forum but I do not know much about that platform but some people I know have also hired from them. FYI always be careful when hiring a hacker and make sure not to reveal or pay with your personal details.

108 points by kiyanwang 6y ago

Wayback machine if the site is still down for anyone: https://web.archive.org/web/20210324071034/http://dan.bodar.com/2012/02/28/crazy-fast-build-times-or-when-10-seconds-starts-to-make-you-nervous/ https://web.archive.org/web/20210324071034/http://dan.bodar....

gopalv 6y ago

As a summary, switched from scala → java, ant → make, removed GWT, Lucene, Hibernate, Velocity, Tomcat etc. This needs a (2012) on it, but it looks like that got hugged to death already [1]. [1] - https://web.archive.org/web/20210324055256/http://dan.bodar.com/2012/02/28/crazy-fast-build-times-or-when-10-seconds-starts-to-make-you-nervous/ https://web.archive.org/web/20210324055256/http://dan.bodar....

jbverschoor 6y ago
When I was still deep in the Java space, I always used hot code replace, otherwise it’s really unbearable. I also contributed a patch to hibernate to cache the compiled classes. That sometimes saved minutes per restart.
eloisant 6y ago
It's important to note that Scala compile time got better, both with improvements in the compiler and in the computers we're using (going from HDD to SSD was huge, and of course CPU improvements). Still slower than Java but I think it's well worth it for the productivity gain.
- MrBuddyCasino 6y ago
  > Still slower than Java but I think it's well worth it for the productivity gain. I have never seen this play out it practise. The most productive engineers I know can touch type well and know their editor/IDE inside-out, have a solid grasp of the basics and preferring not too many abstraction layers, and a workflow optimised for fast dev cycles. Sure, build your startup with Scala or Haskell - I'll bet on the Ruby / Golang / Lisp competitors.
- perennus 6y ago
  My project has 17k lines of Scala code (no whitespace). I don't have a SSD. It takes 10 minutes to perform SBT Assembly from a cold start, 40 seconds afterwards. And about 1-3 seconds to compile and run locally within IDEA.

lolc 6y ago

That was an interesting read for someone like me who just started splitting builds into frontend/backend to get build times down. That they would switch languages just to get faster builds sounds extreme to me. But in the end I already noticed how much better it feels when the CI-cycle is minutes and not tens of minutes. The suggestion to split off code with long-running tests into separate libraries is easy pickings. So I think I'll do that next time I wait for those tests to complete.

gumby 6y ago

Back when I compiled on a cray I had problems with make because files could compile within a second so make would want to recompile them. I use a lot of C++ templating so these days fast builds are just a fantasy no matter what I do. Unlike the author I rely on incremental builds for development but do a full build before pushing

algo_trader 6y ago
c++ templates are a real killer. Have u tried a cloud compiler? Did your company ever try a build server ?
- gambiting 6y ago
  I work in video games, large AAA game with absolutely crazy amount of templating, the compilation time is about 10-15 minutes, and that's with Fastbuild and hundreds of workstations participating in the build process. Once it's built it's a bit quicker, about 60 seconds from pressing build to the game starting, the linking is killer, it takes soo long every time.
  - jcelerier 6y ago
    > the linking is killer, it takes soo long every time. did you try using lld for linking ? even on windows, it's much much much faster than link.exe
    - gambiting 6y ago
      Yes, we use Clang for everything, including lld
pjmlp 6y ago
No use of external templates or having common subset in a binary library and then use private inheritance?
- 6y ago

david_allison 6y ago

QCon talk (2012): https://www.infoq.com/presentations/Crazy-Fast-Build-Times-or-When-10-Seconds-Starts-to-Make-You-Nervous/ https://www.infoq.com/presentations/Crazy-Fast-Build-Times-o...

bullen 6y ago

I use hot-reloading of tiny modules to get turnaround below 1 second on all my projects, server and client: My Java HTTP server hot-deploys a .jar containing new code with a simple classloader: https://github.com/tinspin/rupy https://github.com/tinspin/rupy My C+ 3D MMO engine uses .dll/.so hot-reloading so that the game code can change while the engine is running, it's not released yet: http://talk.binarytask.com/task?id=5959519327505901449 http://talk.binarytask.com/task?id=5959519327505901449

Narann 6y ago
Very nice work. About hot-loading: Where do you put the frontier between each modules? How big is each of them? Maybe you could share a typical module list? Keep the good work!

exikyut 6y ago

An incidental aside, but I actually wrote a comment yesterday on the benefit of using the fastest tooling you can find, along with a restart-from-scratch strategy (non-incremental, non-"reload these specific things while juggling ABI compatibility with X Y and Z already in memory") similar to clean rebuilding. It's a textbook wall of text, but IMO represents a bit of anecdata that 100% agrees with the idea that lowering build and processing times are absolutely worth it. https://news.ycombinator.com/item?id=26555366 https://news.ycombinator.com/item?id=26555366

presentation 6y ago

Right now I can't stand that my Create React App builds for a TypeScript project take several minutes to build on CI machines. Makes me want to ditch it entirely!

christophilus 6y ago
Do it. My Preact builds with esbuild take a hundred milliseconds or thereabouts. The slowest part of my build process is waiting for TwilwindCSS to minify.

oftenwrong 6y ago

I've started experimenting with the Bazel-like https://please.build/ https://please.build/ , which has actual support for incremental builds. I also advocate avoiding a lot of the typical dependencies in the Java ecosystem. There is a lot of bloated garbage that you'll miss less than you might think.

EdwardDiego 6y ago
What are the typical dependencies, in your mind?
- oftenwrong 6y ago
  I should say lots of typical dependencies are fine, but some are common "problem children" or ways to shoot yourself in the foot. The Spring ecosystem comes to mind. With regard to the development cycle, Spring seems to turn things that normally would be compile-time errors into runtime errors. If I've wired up my components incorrectly, I'd prefer to find out sooner rather than later, so as not to waste my time.
  - EdwardDiego 6y ago
    Ah yep, thought you might have been referring to Spring, and totally agree on runtime vs. compile time. I've switched to Micronaut of late for that precise reason - all the injection happens at compile time, so far quicker to pick up. Downsides though are that Micronaut sometimes doesn't provide super-helpful error messages - or silently fails to do DI when `micronaut-inject-java` isn't in the dependencies.

josephg 6y ago

This is one of the things that really makes me hesitate with Rust. I have a pretty small database project which sometimes needs to slowly stream updates to a web browser. Pulling in actix and actix-web added over 1s to my debug mode build time on a fancy new zen3 computer. The web front end isn’t the core of this project - which makes the slowdown really sting. I’m tempted move all the web / actix stuff behind a compilation flag just to keep my coding flow while working on the rest of the project. I know compilation time is being worked on, but it’s still way slower than it could be. I’d love to see an incremental linker for rust some day that’s able to just replace the methods in the binary which were actually changed between builds.

IshKebab 6y ago
I agree. Hopefully when the Cranelift backend works it will get a lot faster, but even `cargo check` can be pretty slow, and I'm not sure if there is any way around it. One idea I had: can crates.io generate machine independent Rust IR for all of the crates and have cargo download it? Then at least you don't need to parse/check/compile the source code of dependencies for clean builds. Probably wouldn't work for any crates that use `build.rs` or conditional compilation anywhere though (which is probably most of them).
- hobofan 6y ago
  Apart from the conditional compilation problems, you would also have to pre-compile for all different Rust versions. Given the size of a typical `target` directory the artifacts for just a single Rust version will also probably be ~10x as big as the current crate source.
  - IshKebab 6y ago
    Mmm good point. An ideal solution for CI would be something like sccache, but I think to get that to work reliably you need to consider it from the beginning (like Bazel did) and Rust just didn't. `build.rs` can do literally anything. Proc macros can do anything. Even a simple `include_str!()` can break distributed compilation. It would be great if Cargo had a "hermetic" mode where you couldn't use `build.rs` and proc macros couldn't do any IO, etc.

toolslive 6y ago

back in the HDD day, plenty of memory + a ram disk would really help.

WildParser 6y ago
I recently experimented with a ram disk. Practically it didn't change anything. OS-caching seems to already be clever enough and once the OS has figured out that some directories are important anything in there seemed to get done in RAM anyway. A RAM-disk will make this less black box and more deterministic regarding guaranteed access times, but in daily use the RAM-disk just didn't make a difference.
- ghusbands 6y ago
  Which OS? And were you already on SSD? Those can both greatly influence the outcome.
  - WildParser 6y ago
    I'm on Windows. Other OS may differ. I experimented a lot with load times - and compared HDD, SSD, RAM-Disk. I wrote a small program that loads everything from a big code-repository into RAM. The first time HDD and SSD and RAM-Disk make a big difference, when reading files a second time the lag of HDD (50s?) almost disappeared completely. Caching kicked in. The RAM-Disk has less initial lag, but also it has to be filled first, so instead of moving everything to RAM-Disk just touching everything so the OS-Caching kicks in is just faster and more convenient.

EdwardDiego 6y ago

I definitely agree with "no Fitnesse", sorry, FitNesse, ugh, and likewise, no Selenium. Not so much for the slowness. In the case of FitNesse (ugh), it's the Wiki -> Java impedance mismatch, and in terms of Selenium, it's the brittleness. Move an element, watch Selenium tests generated by a QA's browser plugin break. I rather loathe Hibernate these days (JDBI is my favourite atm), and well, Tomcat feels rather legacy. But I guess that's the advantage of 9 years of advances. > you can use the maven structure and produce maven artifacts with your build but use a different tool (Make, sbt, buildr, ant etc) Omg, no, sbt is far, far, far, worse. Ditto ant. Not sure if buildr is still a thing? Looks like the last release was 2017. And as for make, how would you even integrate various Maven plugins into it?

cies 6y ago
> JDBI is my favourite atm Tried JOOQ? There's generator overhead on db schema changes, and it adds lots of classes to your classpath (but this could be inside a precompiled library). I guess it adds some build time, not sure how much though (for sure more than JDBI). But that sweet typesafety and auto-complete is worth something.
karmakaze 6y ago
Interesting choice of JDBI. I was working on an SQL-friendly ORM[0] also due to distaste with Hibernate/JPQL and chose JDBI, not because it was great in any way but it did what I needed and not much else. What influenced your choice and were there any close runner-ups? [0] https://github.com/karmakaze/safeql https://github.com/karmakaze/safeql
- asadawadia 6y ago
  have you looked into ktorm?
  - karmakaze 6y ago
    I wasn't looking at Kotlin libraries, but this one seems to be very close to what I was going for. Specifically about being able to compose queries in a bottom-up fashion like the UnionAll example with subqueries. It's unfortunate that they chose to use .filter (.where would have been clearer) for DB executed conditions as well as post-fetch sequence filtering.

unnouinceput 6y ago

And at the other end of the spectrum you have 1M lines of code compiled in just 19 seconds, using latest Delphi compiler (https://blogs.embarcadero.com/delphi-10-4-2-compiler-speed-improvements https://blogs.embarcadero.com/delphi-10-4-2-compiler-speed-i...)

WildParser 6y ago
Great news... Compile time or a clean build? I use similar tooling (older version... :-)). And I'm starting to evaluate what's going on with those new versions.
- unnouinceput 6y ago
  If you watch the video closely you'll see he's doing a full build for the entire components using a test project.
PixelOfDeath 6y ago
Pascal was an awesome learning language alone for its compile speed.
- unnouinceput 6y ago
  It still is. Under Linux you have Free Pascal + Lazarus and can target, natively, everything under the moon (see target section for: https://en.wikipedia.org/wiki/Free_Pascal https://en.wikipedia.org/wiki/Free_Pascal)

WildParser 6y ago

Comparing compile times is done rarely. I sometimes tried to google some comparisons, but didn't find much. For a typical 1,5 MLOC project my clean build times are sitting around 40 seconds. I think that's pretty good, but I don't think I have a way to improve it much. Without writing a compiler from scratch I think 10 seconds are out of reach for me...

algo_trader 6y ago
Do u think your company will consider cloud compiler if this is x5 faster? There issues such as latency, delta synchronizing, and tool versioning. But its doable. If u build such a service (pun intended!), the issues are probably cost and security.
- WildParser 6y ago
  Improvements in build-time are very valuable to me. But I'm not sure if a cloud would help me much. Synching can easily eat up 10s. Also my tools are very unique (including a parser that I wrote myself). To be faster than local a cloud would have to be massively overclocked - and I can already reach good results locally. For actual improvements I need a massive improvement in compiler technology (last time I looked there were just too many single-threaded bottlenecks in my build process). Nothing a cloud can solve for me. Security is one issue - but hackers will most likely only get confused when they try to understand what is going on - also costs I usually don't like...

erik_seaberg 6y ago

Did he measure how much worse the Java version is to work on? Scala infers a lot of boilerplate and wasted effort a lot faster than I can write or review it, because I'm made out of meat.

deleted 6y ago

[deleted]

gjvc 6y ago

rollcall of essential tools if you are in C/C++ land: no build should be without https://ccache.dev/ https://ccache.dev/, and after that, distcc https://distcc.github.io/ https://distcc.github.io/, icecream https://github.com/icecc/icecream https://github.com/icecc/icecream, and goma https://chromium.googlesource.com/infra/goma/ https://chromium.googlesource.com/infra/goma/ may be used for distributing the workload.

rurban 6y ago
Can I use that with GitHub actions? I have some crazy slow C/C++ build times with O2 and asan, which usually leads to g++ timing out. Not just 40min build times. Some huge autogenerated testcases to cross-test all iterator combinations against C. I'd really need -O3 for -finline, but -O2, -O3 and -Ofast all fail too build. Splitting it up didn't help. So far I went with #pragma optimize O1. https://github.com/rurban/ctl/actions/workflows/master.yml https://github.com/rurban/ctl/actions/workflows/master.yml C++ templates are the worst. The pure C variant builds in seconds. I pity all C++ folks
- gjvc 6y ago
  my problem right now is that the OOM killer, is, err, killing compiler processes when asan is enabled. A build machine should have at least twice as much RAM in GB as it has number of cores for compiler processes.

herpderperator 6y ago

Is anyone else (who is not using a caching DNS server from their ISP or other publicly available one) having trouble resolving the domain? There is no A record -- in fact there is no domain at all as far as the root servers are concerned: http://www.dollardns.net/cgi-bin/dnscrawler/index.pl?server=a.root-servers.net&name=dan.bodar.com&type=A&class=IN&lr=12&submit=Send+Query#report http://www.dollardns.net/cgi-bin/dnscrawler/index.pl?server=... Snippet from the above link (to keep this comment relevant once things eventually get fixed): Querying a.root-servers.net (198.41.0.4)... delegated Querying e.gtld-servers.net (192.12.94.30)... name not found Unable to find: dan.bodar.com No Answer Records Looks like the domain record was recently updated to clientHold (about an hour after this article was posted): server ~ # whois bodar.com Domain Name: BODAR.COM Registry Domain ID: 4584805_DOMAIN_COM-VRSN Registrar WHOIS Server: whois.joker.com Registrar URL: http://www.joker.com Updated Date: 2021-03-24T07:06:55Z Creation Date: 1999-03-22T05:00:00Z Registry Expiry Date: 2022-03-22T04:00:00Z Registrar: CSL Computer Service Langenbach GmbH d/b/a joker.com Registrar IANA ID: 113 Registrar Abuse Contact Email: abuse@joker.com Registrar Abuse Contact Phone: +49.21186767447 Domain Status: clientHold https://icann.org/epp#clientHold Name Server: IVAN.NS.CLOUDFLARE.COM Name Server: JEAN.NS.CLOUDFLARE.COM DNSSEC: unsigned URL of the ICANN Whois Inaccuracy Complaint Form: https://www.icann.org/wicf/ clientHold apparently means[0]: > This status code tells your domain's registry to not activate your domain in the DNS and as a consequence, it will not resolve. It is an uncommon status that is usually enacted during legal disputes, non-payment, or when your domain is subject to deletion. Often, this status indicates an issue with your domain that needs resolution. If so, you should contact your registrar to resolve the issue. If your domain does not have any issues, but you need it to resolve, you must first contact your registrar and request that they remove this status code. It's quite surprising how many people on HN are talking about this article just because of however long the TTL was on the original DNS query and somehow almost everyone so far has got a cache hit. [0] https://icann.org/epp#clientHold https://icann.org/epp#clientHold

sva_ 6y ago
Presumably they got hugged to death and are currently trying to change to a different server or something. I can't access it either.

throwaway4good 6y ago

Apache Maven can be fast and easily sub 10 second if you just use in a straight-forward manner. From an arbitrary mid-sized project: About 400 classes split into 10 submodules. Clean build without tests: 3.7 seconds.

bballer 6y ago
Introduce spring and friends and prepare to see that number jump up at least 4-5x

jillesvangurp 6y ago

Interesting. I've always been aggressive on trying to keep my Java and lately Kotlin builds fast. Anything over a few minutes in CI becomes a drain on the team. Basically a productive team will have many open pull requests open at any point and lots of commits happening on all of them. That means builds start piling up. People start hopping between tasks (or procrastinating) while builds are happening. Cheap laptops become a drain on developer productivity. Etc. All of this is bad. Maintain the flow and keep things as fast as you can. It's worth investing time in. Some of the overhead is unavoidable unfortunately. E.g. the Kotlin compiler is a bit of a slouch despite some improvements recently. Many integration tests these days involve using docker or docker compose. Overall that's better than a lot of fakes and imperfect substitutes. But it sucks up time. A lot of Kotlin and Spring projects involve code generation. This adds to your build times. Breaking builds up into modules increases build times as well but tends to be needed. Be mindful of all this. A few performance tips not covered in the article that may also apply to other languages: - Run your tests concurrently and write your tests such that you can do so. Running thousands of tests sequentially is stupid. When using junit 5, you need to set junit.jupiter.execution.parallel.enabled=true in platform.properties (goes in your test resources). Use more threads (e.g. junit.jupiter.execution.parallel.config.dynamic.factor=4) than CPUs for this as your tests will likely be IO limited and not CPU limited. If you are not maxing out all your cores, throw more threads at it because you can go faster. If your tests don't pass when running in parallel, fix it. Yes, this is hard but it will make your tests better. - Don't do expensive cleanup and setup in between tests. This takes time and integration tests become more realistic if they don't operate in a vacuum (your production system is not a vacuum either). To enable this, randomize test data so that the same tests can run multiple times even if data already exists in your database. Docker will take care of cleaning up ephemeral data after your build. This also helps with running tests concurrently. - Distinguish between (proper) unit tests and scenario driven integration tests as the two ideal forms of a test. Anything in between is going to be slow and imperfect in terms of what it does. This means you can improve test coverage (of code, functionality, and edge cases) by making it a proper integration test or faster by making it a proper unit test (runs in milliseconds because there is no expensive setup). - With integration tests, add to your scenarios to make the most of your sunken cost (time to set up the scenario). Ensure they touch as much of your system as they can to do this. You are looking for e.g. feature interaction bugs, heisen-bugs related to concurrency, weird things that only happen in the real world. So make it as real as you can get away with. A unit test is not going to catch any of these things. That's why they are called integration tests. Make it as real as you can. - Fix flaky tests. This usually means understanding why they are flaky and addressing that. If that's technical debt in your production code, that's a good thing. Flaky tests tend to be slow and waste a lot of time. - Separate your unit and integration tests and make your builds fail fast. Compile + unit tests should be under a minute tops. So, if somebody messed up, you'll know in a minute after the commit is pushed to CI. - Eliminate sleep calls in tests. This is an anti pattern that indicates either flaky tests or naive strategies for dealing with testing asynchronous code (usually both). It's a mistake every time and it makes your tests slow. The solution is polling and ensuring that each test only takes as much time as it strictly needs to. - Run with more threads than your system can handle to flush out flaky tests. Interesting failures happen when your system is under load. Things time out, get blocked, deadlocked, etc. You want to learn about why this happens. Fix the tests until the tests pass reliably with way more threads than CPUs. Then back it down until you hit the optimum test performance. You'll have rock solid test that run as fast as they can. - Keep your build tools up to date and learn how to use them. Most good build tools work on performance issues all the time because it's important. I use gradle currently and the difference between now and even two years ago is quite substantial. Even good old maven got better over time. - Pay for faster CI machines. Every second counts. If your laptop builds faster than CI, fix it. There's no excuse for that. I once quadrupled our CI performance by simply switching from Travis CI to AWS code build with a proper instance type. 20 minutes to 5 minutes. Exact same build. And it removed the limits on concurrent builds as well. Massive performance boost and a rounding error on our IT cost. Most of this advice should work for any language. Life is too short for waiting for builds to happen.

JacobCarlborg 6y ago

If you want crazy fast build times, then D is your best bet [1]. D's compile times are faster than most other languages (I'm not counting Vox, it's too experimental). [1] https://github.com/nordlow/compiler-benchmark https://github.com/nordlow/compiler-benchmark

brabel 6y ago
Wow you're not joking... `dmd` is around 40x faster than `rustc` and 16x faster than `zig`! How the hell they've done it?
- kristoff_it 6y ago
  Check Zig again once the self-hosted compiler is completed :)
  - brabel 6y ago
    Will that remove the dependency on LLVM?

habosa 6y ago

My personal theory of modern software development is that the #1 and #2 things you can to do improve software quality are: 1) Make sure devs can run a realistic version of the whole stack locally. 2) Drive incremental build times and effort on the whole stack to zero. With today's world of frameworks, dependencies, and microservices it's too hard to just "think through" a change. You need to try it, adjust, and try it again. Reducing friction on the iteration cycle makes this task pleasant and increases the likelihood of discovering the best solution. When this cycle is painful devs are likely to stop at the first working solution (or even partially working).

gnrlst 6y ago
I find 1) especially challenging with microservices. What technologies do you use/recommend to help with this?
- Aeolun 6y ago
  Docker compose and very beefy dev machines.
- rutthenut 6y ago
  This is where a Docker/K8s containerised setup can help. Not trivial to get to, but for deve/test environments it can be really beneficial. Can also mean the same clean starting point for every series of tests, if new containers are spun up each time through.
  - bosie 6y ago
    how do you combine this approach with some external services (i.e. databases aren't run in docker but AWS RDS or some part of the setup is using AWS lambdas)?
    - majewsky 6y ago
      Even if you have your prod database on RDS, you might be able to spin up a local instance for testing. My Postgres-using applications have a script that sets up and runs a local Postgres instance for the duration of the test. It takes about 2 seconds on first run, and less than noticeable time when the DB has already been initialized. https://github.com/sapcc/keppel/blob/master/testing/with-postgres-db.sh https://github.com/sapcc/keppel/blob/master/testing/with-pos...

derriz 6y ago

Kotlin compile times are killing me. In a current project, I have fewer than 20K lines and it's 50ish seconds for a clean re-compile on a very decent laptop (less than a year old, 32Gb RAM, quadcore i7, fast PCIe/NVMe SSD, etc.). Particularly as I'm a test driven type of developer - cycling between running unit tests and adding small fixes is painful. Even when you can trust the incremental compilation, 10 or 20 seconds between making small one-line fixes and the test re-starting means the temptation to check hackernews or whatever kicks in and any chance of achieving "flow" evaporates. I'm almost considering the suggestion (for Scala) in the article - and moving back to Java.

henryjcee 6y ago
it's a nightmare isn't it. Are you using the 1.4 compiler?
- derriz 6y ago
  Yes it's a nightmare. I'm using 1.4.30. In response to your question, I just bumped it to 1.4.32 and haven't observed a significant improvement. Back in the old days, compiler metrics were often quoted using units of 100s of thousands or even millions of lines per second. I know the Kotlin compiler has to do a lot of work but I would have imagined hardware advances should balance this out somewhat. Instead I'm looking at compile speeds quoted in hundreds of lines per second.
pbz 6y ago
And I was complaining about a 4s rebuild time for a similar (number of lines) .NET 5 solution
whateveracct 6y ago
does it not have incremental compiles? at least Scala incremental compiles are fast. Haskell too!
john4532452 6y ago
Compile time is the only reason i moved to reactnative. It has some issues, but its well worth to have the fast build cycle.

giorgioz 6y ago

Somehow webpack/JS/React has way faster build times than Java/Android apps. That's the main reason we completely abandoned fully native apps in Java/Swift in favor of webapps with a tiny native wrapper app for the few only-native calls we need. And to move forward this trend, new bundlers are appearing in JS like esbuild/snowpack that are even faster than webpack. This is why the web will always win in the long run. While the Android team in working on their tooling and Apple on other tooling and Windows on other tooling on the web all tooling is shared. You don't need the OS/platform gatekeeper to come up with faster bundlers. You just need one company/opensource team to make a breakthrough tool. Finally this is the point why I believe TYPES ARE NOT WORTH IT. Types make the whole development slower. To me is better to be able to try and run the code faster in development and try it manually several times than to wait longer for type checks. Typing is overrated in my opinion.

christophilus 6y ago
With esbuild, typed code compiles just as fast as untyped code (esbuild doesn’t typecheck just strips the types) so I don’t buy your last argument. Esbuild gives the best of both worlds, imho.
- Aeolun 6y ago
  The problem with esbuild is that it doesn’t do hot reload (yet?)
  - christophilus 6y ago
    I don’t like hotreload, so I don’t see it as a problem. I do a full rebuild and my browser auto refreshes and that’s good enough for me. I’ve had hot reload get me into an invalid state too often to find it useful.
    - metachris 6y ago
      For working on complex webapps that take a bunch of steps to arrive at a certain state, hot reloading can save a ton of time, since you don't have to redo the steps to get there. One of the bigger time-savers for web development, in my opinion.

AJRF 6y ago

I'm procrastinating on HN right now because my iOS build time is 2 and a half minutes long. About 30-50 seconds on an incremental build. This is true suffering. Oh wanted to update with Specs; 2019 MacBook Pro with 2.4Ghz Intel Core i9, 32GB RAM.

rudedogg 6y ago
Are you using CocoaPods, SwiftFormat, or SwiftLint? Running SwiftLint or SwiftFormat in a build phase killed my compile times (SwiftLint especially). It'll help a lot to run them manually or with a git pre-commit hook. ----- I know Swift had really bad compile times for awhile, but anymore I think it's pretty fast. I've moved all my WIP stuff into a single project (113 Swift files, ~5,500 LOC) for my convenience, and incremental builds are around 3 seconds on a 2015 MBPr. This article shows some flags you can use to find code that is slow for the compiler to deal with: https://www.avanderlee.com/optimization/analysing-build-performance-xcode/ https://www.avanderlee.com/optimization/analysing-build-perf...

airhead969 6y ago

When I used to use Go, I always wondered if there were stale cached object files because the tests and builds were so fast. I'm not complaining, because I want nearly instant builds or even continuously running background live builds.

RcouF1uZ4gsC 6y ago

The thing that worries me about this post is that they are making a lot of sacrifices in the name of build speed. They are switching languages and libraries and changing their code. Basically, they are creating some amount of tech debt in the name of build speed. Maybe because I started out with C++ on a relatively slow computer, build speeds have not been much of an issue for my productivity. Especially with code completion/etc running and underlining in red my errors as I typed. I guess I kind of treated build as an asynchronous process where I could spend time thinking about the next thing I wanted to do. That being said, definitely get an SSD and a beefy machine, as low build times are nice, but not worth changing your language, libraries, and code structure.

danielrpa 6y ago

It's funny how hardware is SO FAST today but yet we can't get the performance people used to get from... Borland Turbo Pascal in the 80s!

valyala 6y ago

Build times for a time series database written in Go [1]: git clone https://github.com/VictoriaMetrics/VictoriaMetrics cd VictoriaMetrics time go build ./app/victoria-metrics/ real 0m0.574s user 0m1.252s sys 0m0.534s [1] https://github.com/VictoriaMetrics/VictoriaMetrics https://github.com/VictoriaMetrics/VictoriaMetrics

nitzpetter 6y ago

Yes I think it is also possible but the truth remains that the deep web has made it easy for us to hire hackers anonymously without implicating ourselves. One of the places where you can find them will be Dark Basin forum where most of them are screened first before they allowed on that platform too. I was able to get a phone clone from one hacker grup BarrysanchezAThackermailDOTcom and the experience was strictly anonymous till date. another place you can find hackers will on cryptBB forum but I do not know much about that platform but some people I know have also hired from them. FYI always be careful when hiring a hacker and make sure not to reveal or pay with your personal details.