20 ms·

3 lines of code shouldn't take all day

302 points by devtailz 5y ago

Shubhi_29 5y ago
I took 4 days to find root folder and I'm still finding, I think coding is not for me
privacyonsec 5y ago
Now I understand why making games takes so much time
mysterydip 5y ago
The other day I was working on an embedded system and running out of space for the program (512KB). It took me two days to rewrite a few sections to use compressed data instead of the "easier to read but bulkier" original way. The end result to the user was the same, but used 10KB less space.
m0llusk 5y ago
That really depends on how critical the code is. The amount of attention that has gone into optimizing ObjC message send is astounding with many man hours of top programmer time in each and every single line.
dinvlad 5y ago
1000% this. THE secret to productivity is not some magic skills (though that helps) but just the right tools used correctly. That gets you ~80% there.
tomrod 5y ago
This is so important! Good tools, good processes, and even keeled egos can accomplish awesome things.
smcameron 5y ago
Heh. I remember a time back when I worked at Google, and we're using a proprietary internal language to construct some monitoring stuff. It took three of us a day to implement some simple thing that was a line or two of code, so opaque and intractable the language was. When we got it to do what we expected, we still weren't quite sure it was really correct. It was really kind of an existential moment where we collectively wondered, "wtf are we doing?"
- settrans 5y ago
  "No one has Borgmon readability."
dan353hehe 5y ago
Agree with this so much. One of the main things that I try to do is ensure that the turn around time between testing and writing code is as short as possible. As an example I inherited a codebase about 2 years ago that would take 15 minutes to run the test suite. It was painful. It was just long enough that it would waste an entire day just trying to implement a simple feature. The project was behind schedule and no one wanted to work on it because it just took too long to do anything. So I spent a few weeks running benchmarks on the tests and figuring out where the slow down was. I did a bunch of different things, reusing SQL tables, only truncating tables ones that were used, refactored how the tests worked, removed some code so that docker was not needed to do the testing, etc. I got those tests running in 3 seconds down from 15 minutes. And that changed the whole velocity of the project. Always sharpen your ax.
- switchbak 5y ago
  Absolutely. What I find astonishing is how undervalued test feedback time is. I don't know what drives that, but in most jobs I've had to fight to do the right thing. Just recently I've made some optimizations to get a functional test suite from 65 minutes down to 3.6. Parallelism is my favorite lever, as it scales so well, and core count is still going up!
  - HPsquared 5y ago
    Probably something to do with that XKCD about waiting for compiling, "The #1 programmer excuse for legitimately slacking off": https://xkcd.com/303/ https://xkcd.com/303/ Running tests is the same type of thing!
- faraaz98 5y ago
  How much time did removing docker shave off?
  - vbezhenar 5y ago
    Docker helped me to significantly increase unit test speed on one project. Each test was recreating database, run dozens of DDL scripts over and over (to ensure clean environment). I reimplemented in in a way that DDL scripts were run once, then container with database was commited to an image and that image was re-used for every test. Also multiple containers were run at once, so tests could be run simultaneously. Docker is a miracle technology sometimes.
sovietmudkipz 5y ago
I love this. I made an ask HN post about testing practices for games[0]. The answers I got are interesting and an opportunity to reflect. There are clearly some differences in programming cultural practices. I’m of the opinion that applying great testing practices to unity programming is very important. It’s been fun synthesizing ideas I’ve encountered into my hobby game dev career. I’m glad there are others out there doing the good work of generating content to popularize the benefits of good testing practice :) [0] https://news.ycombinator.com/item?id=29459292 https://news.ycombinator.com/item?id=29459292
aappleby 5y ago
This is why you hire old programmers, so that you have someone to yell "WTF" when they see you have no testbench. We had that stuff figured out in the PS1 days.
tayo42 5y ago
I see this is about game development, but in the web/server world idk how anyone ever thinks its acceptable to make development impossible on your local laptop/computer. Everyone time I do a code review I need to ensure they didn't break local development. It blows my mind that I need to do it, like they think its ok to build and deploy for every little test. When I first joined I spent so much time just getting it to work on my laptop. I really just don't understand the mindset. This is has been the case in multiple places. Who was this first person to break local development and decide it was ok. Please fire that person, black list them from getting jobs ever again. We have so many tools to make this easy now like docker and yet it still gets messed up.
- xmprt 5y ago
  I think this is why I love green field projects so much. More than being able to work on my own code and design from scratch, I appreciate the fact that no one has messed up how testable and runnable the code is from a developer machine.
- teeray 5y ago
  In some environments, I imposed a rule: “the tests must continue to pass, and I must be able to use the application on my local machine in an airplane on crappy wifi”
substructure 5y ago
AWS CloudFormation had one of the worst user stories related to iteration time a few years ago. The lack of feedback between the writing of the yaml file and validation of the structure/type/format was frustratingly slow. I would pay good money for better tooling in the Infrastructure as Code(IaC) space. Waiting for the resources to update, fail with cryptic error messages, then slowly rollback only to then fail the rollback. Now in an invalid state, manual resource creation was required before the rollback would succeed. The AWS cdk has improved this significantly. As a result the sun shines just a little bit brighter.
- suzzer99 5y ago
  Getting anything new working in CloudFormation is a great way to kill a day.
- ewuhic 5y ago
  What exactly you'd want to see from such tooling that it improves your experience with AWS?
kyruzic 5y ago
How I wish a single line code change only took a 15 second build. The java app I work on regularly takes 3 minutes for a single line change. Then you just need to pray that jrebel will work today and actually hot reload your change. Otherwise you have another 8 or so minutes of redploying weblogic. Of course the ideas suggested in this can help, but when you need to run an integration test which also takes minutes to run you realize that 3 lines of code isn't bad for a days work. One day soon I will never touch Java again.
- suzzer99 5y ago
  Going from Weblogic 10 minute reboots to Node 5 second reboots made development so much more productive and fun for me.
- icedchai 5y ago
  Your problem is WebLogic, not Java. Try a lightweight app framework (Dropwizard, SpringBoot...) that embeds the "app server" right into your app. Of course if you're using WebLogic, you probably don't have a choice.
senectus1 5y ago
For all the issues I have with Elon Musk, I really like his design philosophy. Step 1 Make it less dumb Step 2 Delete a part of the process If you're not adding step in 10% of the time, you're not deleting enough Step 3 Simplify or Optimize Step 4 Go Faster Step 5 Automate
- phone-account-1 5y ago
  Nitpick: he actually said "Make the requirements less dumb", which is quite different actually.
gitgud 5y ago
> “it takes too darn long to test these changes, is there a better way?” This is a question we should be asking ourselves every day. Agree, continuously re-evaluate your systems, they can always be better
thibran 5y ago
As someone writing Lisp code, this feels so archaic. Then I remind myself that Lisp is older and than those games and that the future just hasn't caught up yet.
joelbondurant 5y ago
Industry standard build systems require 500 remote servers because YAML is the favorite programming language of infrastructure teams.
yashap 5y ago
If I had to order the things that make me productive, starting with the most important: 1. It’s easy to have a fair amount of confidence in changes without even needing to run them. Strong static typing, good/clean abstractions, local reasoning - little to no mutability, code is mostly pure functions with side effects pushed to the edges. Being able to just write a bunch of code without needing to run it at all (and having it “just work” the first time, most of the time) is HUGE for productivity 2. Excellent automated test suite, with a nice pyramid (small number of E2E tests, solid number of integration tests, exhaustive unit tests). Also excellent monitoring and alerts, with automated canary releases and automated rollback. Basically make it so that it’s hard for mistakes to fully reach production. Being able to be a bit of a cowboy, safely, is huge for productivity too. I’ve worked on plenty of systems without this, and then I’m MUCH more careful, but when you really do have this, it makes everyone so much faster, especially once you learn to stop worrying and trust the guardrails (for simpler changes) 3. Test suite runs quickly locally, and is not flakey 4. Running the app locally is quick. Also ideally I can run just the one component I’m working on (like a front end, or service, or whatever), but have it fit into a full system running elsewhere 5. Full build/test/deploy process on CI is quick and reliable This article emphasizes 3 and 4, and they’re certainly very important, but I think 1 and 2 are even more important. With them in place, often I don’t even need to do 3 and 4 for more trivial changes - just “I’m pretty certain this works,” and then the tests all pass in CI, and I’m very certain. Compared to systems without 1 and 2, where I have to basically change a line, then run tests and/or the app because I’m not sure it’ll work, then do extensive manual testing to be sure because the automated test suite sucks. Muuuuuuch slower. 3 and 4 are great, and necessary for more complex changes, but 1 and 2 let you ship simple changes super quick, without even needing to run anything locally. That’s fastest of all, and can still be quite safe in the right environment.
- feffe 5y ago
  I agree with your list but really need 4 myself to be a happy camper. Often it's neglected once 2 is in place as most don't seem to find any value by it, or don't care to maintain it. I find that having the possibility to run the real thing (in some capacity) in a cozy development environment with debugger is very useful to quickly learn how parts in the system interact with each other. I have a hard time doing that with unit tests, perhaps because they are so boring.
meheleventyone 5y ago
Whilst I very much agree with the premise I do think this exposes a fallacy many programmers fall for. That passing unit tests means your code actually does what you think it does. Using unit tests this way is a crutch to avoid improving startup times, implementing hot reloading and other schemes to skip gameplay. You lose an awful lot by not testing your code for the job it’s actually expected to do in situ. In games this results in increasing the total iteration time as it’s usually now a different person who actually runs the code, find issues and submits them back.
- Ma8ee 5y ago
  You need both. If you don't have unit tests your feed back loop becomes way too slow, but then you of course have to test the "real" running of the program properly before shipping. And you need at least one person who didn't write the code to test it.
  - meheleventyone 5y ago
    Unit tests are useful for some things. I can’t unit test my way to a game feeling good. For that I need a fast feedback loop to the real program. There’s more of that in most applications than most people suspect. And routes to get there. As programmers we make a lot of little decisions along the way and those don’t get proper consideration unless you can see that as the feature evolves.
- rlayton2 5y ago
  Isn't the normal process, roughly, to: 1) Use unit tests to move quickly through to implementing The Feature 2) When The Feature is complete, run in a local environment to confirm it works 3) When that works, move The Feature to a development environment that more closely mimics Prod 4) Deploy to Prod Each one of those steps takes an order of magnitude longer than the previous one, so should be done an order of magnitude less often. However if you are finding inconsistencies between steps, then alter local/dev to ensure more consistent testing. (i.e. if it works on Dev, it should just work on Prod, however there are always little issues).
jeffrallen 5y ago
Neves Law says that "the harder a bug is to find, the smaller the fix will be". The worst Neves ratio I ever saw was about a week to two bits, where a plus (0x2b) had to be changed to a minus (0x2d).
- typon 5y ago
  Week/bit was common when I was working at an FPGA company. Yes FPGA software tools suck.
phendrenad2 5y ago
3 lines of code per day? Damn that's pretty good!
userbinator 5y ago
This is probably a controversial opinion, but I think that working in an environment where time-to-iterate is high is actually very beneficial for improving your skills. It's one of those things that may feel overly burdensome in the short term, but is better in the long term. I say this as someone who taught programming with beginners, and observed what many of them will do when given an IDE, which at the scale of the code they're writing, makes the time-to-iterate very short (press a key and you instantly see your changes). They end up getting into a "dopamine feedback loop" that causes them to continually make tiny changes and rebuild/run, and the code written as a result of this looks exactly like what you'd expect: it barely works, and is full of redundancies, "dead ends", and other evidence that its author was probably not thinking of anything more than the next line or two when writing it. In other words, reducing the iteration time reduces the motivation to get it right the first time, and the associated deep reasoning/"mental execution" skills which are required to do so. In order to develop those skills, one should strive to write as much code as possible before running it, and a high iteration time assists with that. Also, I'm tempted to continue the title with "...unless it's APL".
- jmnicolas 5y ago
  Your example is valid for beginners but once you're getting experience you don't code like that anymore. After 13+ years of C#, on simple projects I can code for a couple hours without compiling and get the pleasant surprise that my code work the first time I run it. And I'm certainly not a genius, I'm a 1X programmer.
  - rvense 5y ago
    Eh, ten years a professional, I frequently fall into it still, where you forget to think about the best solution and just.. mash. Sometimes it works to get a solution that can then cleaned up, other times I waste an hour before I snap out of it. I saw someone post "Stop thinking and look!" as universal advice once, but I need the inverse just as often.
  - cardanome 5y ago
    I still do. Of course dabbling into REPL-based solutions in my youth might have influenced that. For me it is crucial to always have have the code in some compile-able and testable state. I iteratively work towards the simplest, most naive solution that gets the job done. After that it is refactoring time and I form it into a proper solution. The reason I do so is because only after having coded the naive solution, I have an concrete and proven correct understanding of the task and can pick the appropriate abstractions. I always see colleagues with an less iterative style use premature-abstractions and over engineering on the basis of "we might need it" instead of knowing. I do invest some time into thinking about the right data structure though. (Not much about algorithms, as they mostly follow from the data structures anyway.) Also if I code a well understood problem, I might use a more top down approach. (Not a critique of your workflow, just wanted to share mine.)
KronisLV 5y ago
Recently decided on using TDD for some library code and aiming for >90% test coverage at my dayjob. Frankly, things do take longer to develop this way and i needed to put in more thought into how the interfaces should be structured, vs just doing what people sometimes do in other projects - just relying on concrete implementations or using frameworks that use reflection to do what they want. However, the experience itself feels like a positive one - even if the code takes longer to craft, it's much easier to work with, since now i don't need to worry about sloppily written if/else chains that depend on enums or runtime type checks, but can utilize different implementations as needed. Also, writing the actual tests allowed me to be sure about how everything would actually work, all the way up to discovering that Paths.get (Java) on Linux accepts almost anything as a valid path string, whereas Windows has actual validation rules, leading to many headaches in regards to the code coverage quality gates that i set up, since the coverage differs based on which platform you run on. But i guess that my point still stands: in certain circumstances slowing down might actually be a good thing, when you really want to know how your code will work under most circumstances and make it maintainable. Of course, on the other hand, compilation speeds and other factors in regards to the speed of iteration definitely shouldn't be overlooked either, and having everything else apart from writing tests and actually thinking about how everything will fit together be faster is a good thing! I'm not sure what i'd do if my test suite took 10 minutes to run instead of 10 seconds as it currently does. Probably drink lots of coffee. I guess it depends on slowing down for good reasons, vs just wasting time because of tooling or other sub optimal circumstances (e.g. what was described in the article, personally i've also seen a local API service be pretty chatty with a remote DB which was slow over a VPN, yet no one had a local DB with all of the migrations in place, and a bunch of other things like that). Offtopic: Anyone remember how fast Pascal compiled? Now that was a really nice stack to do some stuff in, it's a shame that it never got as popular as Java, or didn't have tooling like JetBrains has, it felt like a more ancient Go (which is also pretty good as far as ergonomics go).
- roeles 5y ago
  I have somewhat the same experience with my first real TDD-based project. Progress was slow, but the amount of debugging afterwards as almost zero. It feels slower, but more predictable. > Recently decided on using TDD for some library code and aiming for >90% test coverage at my dayjob. I thought the point of TDD was that you don't write code unless a test requires it? In my mind (and the little experience I have) that translates to 100% coverage, with exceptions of really annoying side-effects that are near-impossible to test reliably.
dinvlad 5y ago
Going from "manual testing" to automated testing is the biggest jump. Unfortunately, lots of folks (both developers and managers) fall into the trap of thinking they will "use up" their development time on automated tests, and not really thinking how much time they're already wasting just to do (incomprehensive and hand-wavy) manual tests each time. And while this strategy might work for the folks who actually wrote the code (and thus have tacit knowledge about it), the moment they leave and/or someone new joins the team, all that "speed advantage" is lost and it turns from minutes to hours to days instantly. Automated tests really are the only way to capture the business logic of any code for it to not to become "legacy code" before its time.
- cjfd 5y ago
  Very good points. In his book 'working effectively with legacy code' Michael Feathers actually defines 'legacy code' as code without tests. Largely because of reasons like you state.
  - vbezhenar 5y ago
    What do you think about code being too smart for current team? I encountered that use-case three times in my career. It was C++ program which was written by very bright person, used template magic, boost and stuff. Rest of the team were ordinary C developers. After that person left, they had to rewrite his program, because it took too much time to understand how it works and to fix or improve it. Second case was when someone wrote some helper program with Haskell which happened to be important. Same story, nobody knew Haskell, nobody wanted to invest into learning, so rewrote in node.js. Third story: someone used reactive API with Java (Spring Flux) and again same story: team don't understand it and slowly rewrote it with ordinary blocking API. Does that code counts as legacy?
    - hu3 5y ago
      This is where Go shines in my opinion. It's really hard to write smart Go code. Sure it is a bit verbose but dam is it easy to understand due to sheer bluntness and lack of magic.
pechay 5y ago
Aside from looking at changes in your dev methods, this is a good way to demonstrate to your management the value of having fast, up to date hardware on your desk.
onion2k 5y ago
In a future post, I will go over how web developers needs to start taking iteration time more seriously as the influx of new tools and frameworks starts to bloat up build times. The newer tools in web dev have crushed iteration times to a fraction of what they used to be. A combination of things like esbuild, fast refresh, yarn's offline cache, and a few other bits can get your iteration times on a site you're manually testing down to milliseconds to both build and update (literally). Decent devs have been writing tests for years; the test runners could be a bit faster but people are working on that as well. The React app I'm working on at the moment takes well under a second to build, under 10 seconds to run the test suite (could be a sign I need more coverage..), and milliseconds to update the code in the browser in dev mode. It's nice to work with. It's not even clever or special. A default Next.js app will do that. Create-React-app 5 launched a few days ago with more of the tooling too. In Vue things like Vite are based on the new fast tooling and it works well. Vue dev work is fast. SvelteKit even goes further by using Snowpack to remove the bundling step entirely. Snowpack claims a default refresh time of 50ms. I really hope the author's article about web dev is simply "Yeah, update your tools and you'll be fine."
- SrZorro 5y ago
  > SvelteKit even goes further by using Snowpack to remove the bundling step entirely. FIY SvelteKit is using Vite 2 instead of Snowpack Source: https://twitter.com/Rich_Harris/status/1367577006355976194 https://twitter.com/Rich_Harris/status/1367577006355976194
xwolfi 5y ago
This guy is young and probably not educated in modern dev. I refuse C++ jobs when I ask how they do unit test and they reply it's too hard in C++, and take Java jobs because there the first question THEY ask is HOW I do unit test, what I think of coverage metrics (hints: they don't matter much) and how I propose to enforce ALWAYS unit testing important code. In this blog post the guy took 3 job change to DISCOVER unit tests, it's insane :( It's not like they know but don't have time, it's that he's a "champion" just for proposing limited scope testing.
- est31 5y ago
  I think the main issue about automatted unit/integration testing is not the language, but the area you are writing code in. Yes, C++ not having good unit test support hurts. But it hugely depends on the problem domain and whether the outputs are machine verifiable, and the inputs are easy to simulate. It's hard to do unit tests in a large part of the area of gamedev. How do you ensure that a picture is rendered correctly? You can take a screenshot and compare it to a stored one. But then what happens if you do artistic changes that you want to do? You have to update the stored screenshots of the testsuite. Who will review that the changes all made sense? And more importantly, there might be slight differences in the output of GPUs, depending on driver versions, model, etc. So let's say you have a bug in your game where if you walk through a level in a specific direction, the game crashes. The error is easy to check for: just make sure that there is no crash. But how do you create a reproduction of the bug? You could record controller inputs and play them back. Then a different department changes something how quickly players move and increase their walking speed by 10%. Suddenly your player walks into a wall and the test is basically broken. Compare this to a CRUD app where you have well defined operations and their impact is well described. That being said, even in gamedev there are areas that are well testable. You can do a unit test of the low level networking layer by trying to make a server and a client, dropping some packets, then looking if the packets still arrived because the networking layer sent them again.
zuhayeer 5y ago
Another thing here is that often times people are so busy "developing" product that really trivial efficiency gains get overlooked. Things like simply removing unnecessary parts from the build at least when you're testing / in dev mode. Also compartmentalizing the build to the portion of the app you're working on, ensuring it's not building the entire app when you're only testing one feature (you'll be surprised how common this is even especially at large companies). It's a huge value add to the rest of your team to bite the bullet and take an hour or so to shave off some minutes from the time you have to wait for a build. That one hour saves you from many multiples more (hours / days / weeks) of fiddling your thumbs in the breakroom waiting for a build to finish. (better yet to have someone regularly scanning for these efficiency related things)
wojciii 5y ago
I do embedded work. I experienced this while working for a multinational. - A long build process and linker step that takes ages. 1-2 minutes for a small change. - Trying the change on a real device took perhaps 5 minutes. - A review process that can take days or even weeks sometimes. - There was a process that merged my change set which often failed and needs to be rerun several times - often 1-2 days were spent on trying to commit something where everyone was doing the same. - There are no unit test because some manger decided that they provide no business value. - There were no simulation tools for software be because some manger decided that they provide no business value. - Testing would be done overnight. With the right tools (unittest and pc simulation) I would be effective and could test my changes prior to testing on a real device. This was for me the bottleneck and not how long it took to build the project. I was testing code that was just C on a complex embedded device that I could have tested/debugged easily on a PC in 1/10 of the time. This was really a management problem where the management didn't understand what to do to make their employees efficient with the right tools.
- dezgeg 5y ago
  At least you are lucky enough to be able try changes yourself. I am in the "fun" situation where team of 50+ SW engineers don't have single piece of test equipment needed for operating the hardware, and everything has to be manually tested by some QA engineer, taking 1-2 hours minimum...
  - wojciii 5y ago
    It can always be worse. :) I feel your pain. This was from an old job. I work a different job now with different challenges.
- jeffrallen 5y ago
  Hello, fellow ex-Cisco employee.
kubi07 5y ago
My first full time job as a junior dev was to maintain a legacy GWT project which makes tons of $ every year.Setting up the development environment took 1 week. For some odd reason we couldn't build the project module by module. Build was taking at least 2 mins with the bare minimum module count. I quickly become depressed. Waiting the builds and half way into build an error pops-up and you start again. This was 4 months ago, i literally became a deppressed junior dev who hates his job in 1 month. I started applying for job after a month and quit that job after 2 months.
mro_name 5y ago
even less so should three letters take ages, shouldn't it? E = m c^2 Any 5-year old can write this.
roeles 5y ago
> This is the sweet spot amount of time where you become tempted to “do something else” while you wait. I think this is an important observation. The time where you "wait", you can allow your mind to drift off right after being immersed in the problem. In my experience, this time has the potential to give you great insights. I think fast feedback is great, but only if you know what you're trying to accomplish. Or trying to learn. Don't use it as a tool to throw stuff against the wall and see what sticks. Once you start to do that, spend some time away from the computer. Draw a picture, formulate some hypothesis based on your mental model. Make a map before you enter the jungle of trial-and-error.
max002 5y ago
Haha, so true. People are used to old ways. I know its not c++, but If you run docker on mac or Windows for web dev or any other project that has thousanda of deps you know how painful docker sync is and how slow/crash prone. But hey "docker is easy" :)
angarg12 5y ago
If what is slowing your iteration speed is testing, I'd say you are in a somewhat happy place. At least you can write test, speed up the build, or somehow improve the situation. What drains my soul is when you need to use legacy or poorly designed tools that slow you down and you can't do much about. Here are real examples of times I wasted entire mornings: tools that, if you make a mistake, leave the environment in an inconsistent state, and then you need to manually fix it (it isn't documented and changes on a case by case basis). If you miss anything or make another mistake, start from the beginning again.
wingerlang 5y ago
Regarding the "testbeds". I recently built this for an (iOS) application and it helps SO MUCH. Each module in the app has its own local target (the testbed) with a menu which lets you open the module for a given scenario. A scenario is a combination of local JSON for endpoints, device fakes (think fingerprint enabled/disabled) and module-specific configurations. The ability to near instantly get to a specific functionality with the same network requests makes everything super simple. I run the UI tests on these targets as well, and they are near perfectly non-flaky. The best part is when I receive a new bug report from QA, since they include the network logs I usually just need to create a new scenario, register the JSON and fix the reproduced issue.
globular-toast 5y ago
This is something I've worked to get right my entire career. Whenever I start a new project, I make sure the feedback loop is small. If it's a new technology for me, this can take a while to get to a process I'm happy with, but it's worth it. Way back when I was at uni I remember watching over people's shoulders at the painfully slow iteration process. I couldn't believe not a single one thought of doing anything about it. They looked more like factory workers, endlessly turning the same handle and waiting. If you spend your time doing this, you will have no time left for creativity.
solididiot 5y ago
Was working in a similar product and it was by far the worst dev experience I've ever had. It was actually worse than what the post describes. It was like having to go through a lot of stages to get to the area I'm working and half of them not working at all - so instead I was doing bug reports on other people's areas which was only slightly familiar with. (Yeah, our CI QA gateway was a joke. Our CI in general was a joke). It was hell. That much so that I jumped to the first half-decent role that came my way. Something is wrong with the way we build systems that integrate too many other systems. Perhaps if we designed them with a TDD approach all the way up things would be better...or worse. Dunno. Sometimes what we're trying to do is just way into the entropy zone.
nearmuse 5y ago
While it sounds painful because of the inefficient development & testing process, I think that if that day is spent on research instead (looking for a better solution or reading existing code) it is OK to spend 1 day on 3 lines of code.
deleted 5y ago
[deleted]
Sneha_1 5y ago
noob like me takes all day
rob_c 5y ago
By comparison I've worked with someone who pushed a '2 line fix' live which had fallout which lasted >2 weeks because their code was pushed without checking to production and had hard-coded their uid into the fix because it was a quick 'hack'... The main reason when cornered was 'running pip install is too difficult to run the test-suite' (tests themselves took <30sec to run all ~300). We eventually (1 week later) got permission to pull the release from our shared (pseudo write-only) storage area but as he was project lead he tagged a new minor version for his '2 line fix' without consulting anyone and our users were reluctant to move to a new bugfix release unless prodded with a red-hot poker. Can't say I miss working with _some_ programmers turned managers.
jarek83 5y ago
Sounds like EA has (had?) actually pretty good developers. The game itself has so many things gone wrong that from the players perspective it's very hard to find any good words for. Shame that the company hates their customers so then customers started to hate anyone involved in EA. In the forums of the game, devs of this game are considered to be the laziest and the least skilled across the industry. PS. Anyone did figure out whether DDA is officially in the code or is it just a matter of network quality difference between players and the server (the game stopped to be p2p in any mode)
ineedasername 5y ago
Testing changes here could mean progressing through several seasons of career mode in order to test out a change That's like the Groundhogs Day of programming. Living the same season over and over again, changing things until you get it right and can move on. Sure, ai guess that's kind of programming in general, but on this level it's most of your day. No developed cheat codes?
ChrisMarshallNY 5y ago
> 3 lines of code shouldn't take all day Sure they should. I often spend all day on negative numbers of lines (I always say that the best code I write, is the code I don't write). The complaint seems to center on CI/D services that provide inefficient build times. These increase the pain of "round-trip" implement-test-refactor. That's quite valid, as this is how we tend to work, these days (at least, that's how I work). I often take it a step further, and iterate the design as I progress[0]. But I am one that got their start in the waning days of "big iron," when compute time was the costliest and most time-consuming part of the whole process. You'd need to schedule for computer runs, which would often happen overnight. This meant that it was very important to have your code complete, and debugged, before submitting it for compilation. Argh. I miss it like I miss a case of food poisoning. It did teach me to do my homework, though. [0] https://littlegreenviper.com/miscellany/evolutionary-design-specification/ https://littlegreenviper.com/miscellany/evolutionary-design-...
- munificent 5y ago
  > Sure they should. I often spend all day on negative numbers of lines (I always say that the best code I write, is the code I don't write). You're missing the point of the article. It's not about valuing quantity of code. It's about valuing minimizing the time it takes to determine what you want the code to be. If it takes you a 30 minute compile loop to add a line of code, it's also going to take a 30 minute compile loop to refactor and eliminate a line of code. The latter is actually worse. When your iteration time is high, developers will deal with it and push through as necessary to get features implemented because they have to make the software do a certain thing. But they will absolutely not struggle through a shitting iteration cycle if the only result is cleaner refactored code. If you want nice codebases, you need a nice iteration cycle.
  - ChrisMarshallNY 5y ago
    > You're missing the point of the article. Actually, I didn't. Just sayin'.
    - 5y ago
atum47 5y ago
I once spent the better part of a day figuring out a off by one error, when I pushed the fix it changed two characters. Almost the whole day, imagine that.
flohofwoe 5y ago
Entirely relying on unit tests for development can never be the only solution, because tests are quite simply not the product, and they shouldn't be a substitute/workaround for fast iteration right on the product. Easier said than done of course, especially in such a complex environment as game development. The future for fast compiled-code iteration in game development will most likely be hot code reloading, so that code changes are compiled and implanted right into the running game instead of requiring a full linker run and then getting back to the right place in the game to be tested.
- commandlinefan 5y ago
  > Entirely relying on unit tests for development I don't think anybody is ever suggesting this - but I do see people suggest the opposite: that all testing should be end-to-end "black box" testing, and unit tests are a waste of time. If you actually want to ship something that works reliably, you have to do both unit testing and end-to-end (integration) testing. I've never seen anybody sacrifice integration testing. I have seen them sacrifice unit testing.
whalesalad 5y ago
I love building test harnesses. I think the trick is making sure that your system is built in such a way that 1. You can actually pull the core of it out to run in a different situation and 2. You keep enough of the real system such that you’re not getting lots of green passes which are going to completely fail in the real world.
cagenut 5y ago
The "sub-second response loop flow state" he mentions here is "the doherty threshold"[1] just applied to programing as a particular user experience. After working at a CDN that delivered a couple hundred kb objects in milliseconds around the world I thought "code is data, so why cant code be updated in milliseconds?" I tried starting a startup that could do this, treat code as data, and therefore achieve sub-second round-trip trial-and-error loops. Did not pan out, but to be honest we never really got to testing that thesis. I still think it could be amazing, just not sure how much of the lang/ide/build/test/deploy/validate cycle you could integrate and how much you'd have to build. IMHO this is the main thing that made PHP successful. The "edit a file -> alt-tab -> click refresh" test loop being faster than you could click. [1] - https://lawsofux.com/doherty-threshold/ https://lawsofux.com/doherty-threshold/
wwilim 5y ago
Coming across this article and reading it while waiting for a long build was very ironic
- MarkLowenstein 5y ago
  Haha, yes. If it weren't for the phenomenon written about in this article, I bet HN readership would plummet at least 20%.
Tarucho 5y ago
I worked in lots of projects like this. The most amazing part is most devs didn´t care. Never understood why. In one of those projects the code got so big and bloated that it took around 1 minute (sometimes more) to move from one line to the next while debugging. For me it was a torture, it was ok for most.
bitwize 5y ago
Makes me even sadder that something like GOAL didn't take off. In GOAL, you could make changes at the REPL, compile instantly, push the compiled changes out to the console and see them immediately in the running engine. There's a reason why old-school Naughty Dog games were so refined.
allo37 5y ago
IME reducing iteration time is one of the best ways to increase my productivity. I like Android Studio's approach to running unit tests: It will run them on the host PC instead of deploying them to the target, so you can avoid the often long deployment process.
webmaven 5y ago
The last time some one told me something like "changing three lines of code shouldn't take all day"[0], I responded with "Fuck you" because the way they managed to work 'faster' was to make and test their changes in production before committing them to version control. I also once spent a week pursuing a small-but-necessary authentication flow change across two dependencies (one of them owned by a different team) and three tests, only to be fired for talking to the other team members directly (I was a mere contractor) rather than playing 'telephone' and relaying everything through my management and their management, which would have made it take a month instead. [0] It was actually five lines of code.
PaulDavisThe1st 5y ago
In 1989, I was working at my first (very short lived) US job, for Bell Labs (not the Bell Labs, but related). I was working on their "office-scale" phone switches, the kind that allowed you to forward a call from one extension to another. There was a hard-coded limit of (I think) 16 forwards, and for some reason it was decided to increase this to (I think) 64. This involved changing a constant in a header file, a documentation string and one conditional in the code that for some reason didn't use the constant. Expected time for this work was: 1 week.
- e12e 5y ago
  Don't leave us hanging - how far over budget did it end up? ;)
  - PaulDavisThe1st 5y ago
    Under budget, but at the end of that week, having successfully pushed through this DRAMATIC CHANGE, I drove from Phila. to the Outer Banks (NC), crashed, rolled the car 3 times, totalled the car, broke my arm, could no longer get to work, decided to marry my first wife, and left to work for a company I could commute to with a broken arm. Small change, big ramifications :)
thrower123 5y ago
It's always fascinating how game development studios tend to consistently lag ten or fifteen years behind enterprise software development practices.
Jtsummers 5y ago
From a systems perspective, there is a balance to be achieved. You want faster iteration times in order to provide feedback, but if they're too fast then you neglect other things or end up swinging wildly. Consider a thermostat that turned on the heat or cooling at just .1 degree below or above its target, or turned it off as soon as it hit the reverse. Or if because the feedback comes in as a torrent of data you attend to that feedback instead of other elements of the system or your capabilities. As a system gets more complicated you have more factors to consider. In the case of programming, if you rely too much on that near-instant feedback, how much are you internalizing about your system, language, and environment? How often are you making the same errors but recovering quickly (so it's not slowing you down too much, or doesn't appear to be slowing you down too much) because of the feedback? How much faster could you ultimately be if your work were smoother and not so rough and jagged? Introducing a delay, here, gives you time to contemplate. Even if it's just taking an hour or two a day to sit back from the keyboard and ponder what you've done that day and what you will do the next. Create a plan instead of jump into action, even if the plan doesn't get executed perfectly or ends up being the wrong plan that bit of contemplation is when you learn. But too long a delay (especially a forced delay) causes other problems. The actually needed feedback (not just compiler errors and such, but your V&V issues discovered from testing and evaluation) getting delayed by a day or more can be too much (especially when working with a team, where other parts are potentially changing around your own changes). Too long a delay also promotes batching many unrelated changes together because you don't want to sit through the whole process again. Consider a system that takes a week or a month to get feedback from an external test team, you'd be tempted to throw many changes at them because of that week or month long delay (or more!) and not just one change. So strike a balance, find a point where your iteration cycle permits you time to think and not just act so you can really learn (both programming and the particular system you're trying to develop). Smooth out your development so that you're slowed down not by having to take corrective steps but having to ponder logical steps. "Is this the right data structure? Well, if I isolate it in my domain model then I can swap it out later and no one will be impacted." or "I'll take a walk around the building and think about what I actually need here."
literallyaduck 5y ago
You can pack in a lot more training, reddit, and mobile games into a day when you only write 3 loc.
makach 5y ago
Sometimes a negative amount of code lines took me whole days...
vbphprubyjsgo 5y ago
Uhhh... just improve your build process.
adolph 5y ago
Bill Atkinson, the author of Quickdraw and the main user interface designer, who was by far the most important Lisa implementor, thought that lines of code was a silly measure of software productivity. He thought his goal was to write as small and fast a program as possible, and that the lines of code metric only encouraged writing sloppy, bloated, broken code. https://www.folklore.org/StoryView.py?story=Negative_2000_Lines_Of_Code.txt https://www.folklore.org/StoryView.py?story=Negative_2000_Li...
nanoscopic 5y ago
Suppose you maintain a highly scaled service. Suppose it has a bug. Finding and fixing 3 lines of code that caused a problem that was costing the company money by the hour/minute could be well worth it and a great accomplishment in one day. The number of lines of code is irrelevant. The impact is what matters.

302 points by devtailz 5y ago

Shubhi_29 5y ago

I took 4 days to find root folder and I'm still finding, I think coding is not for me

privacyonsec 5y ago

Now I understand why making games takes so much time

mysterydip 5y ago

The other day I was working on an embedded system and running out of space for the program (512KB). It took me two days to rewrite a few sections to use compressed data instead of the "easier to read but bulkier" original way. The end result to the user was the same, but used 10KB less space.

m0llusk 5y ago

That really depends on how critical the code is. The amount of attention that has gone into optimizing ObjC message send is astounding with many man hours of top programmer time in each and every single line.

dinvlad 5y ago

1000% this. THE secret to productivity is not some magic skills (though that helps) but just the right tools used correctly. That gets you ~80% there.

tomrod 5y ago

This is so important! Good tools, good processes, and even keeled egos can accomplish awesome things.

smcameron 5y ago

Heh. I remember a time back when I worked at Google, and we're using a proprietary internal language to construct some monitoring stuff. It took three of us a day to implement some simple thing that was a line or two of code, so opaque and intractable the language was. When we got it to do what we expected, we still weren't quite sure it was really correct. It was really kind of an existential moment where we collectively wondered, "wtf are we doing?"

settrans 5y ago
"No one has Borgmon readability."

dan353hehe 5y ago

Agree with this so much. One of the main things that I try to do is ensure that the turn around time between testing and writing code is as short as possible. As an example I inherited a codebase about 2 years ago that would take 15 minutes to run the test suite. It was painful. It was just long enough that it would waste an entire day just trying to implement a simple feature. The project was behind schedule and no one wanted to work on it because it just took too long to do anything. So I spent a few weeks running benchmarks on the tests and figuring out where the slow down was. I did a bunch of different things, reusing SQL tables, only truncating tables ones that were used, refactored how the tests worked, removed some code so that docker was not needed to do the testing, etc. I got those tests running in 3 seconds down from 15 minutes. And that changed the whole velocity of the project. Always sharpen your ax.

switchbak 5y ago
Absolutely. What I find astonishing is how undervalued test feedback time is. I don't know what drives that, but in most jobs I've had to fight to do the right thing. Just recently I've made some optimizations to get a functional test suite from 65 minutes down to 3.6. Parallelism is my favorite lever, as it scales so well, and core count is still going up!
- HPsquared 5y ago
  Probably something to do with that XKCD about waiting for compiling, "The #1 programmer excuse for legitimately slacking off": https://xkcd.com/303/ https://xkcd.com/303/ Running tests is the same type of thing!
faraaz98 5y ago
How much time did removing docker shave off?
- vbezhenar 5y ago
  Docker helped me to significantly increase unit test speed on one project. Each test was recreating database, run dozens of DDL scripts over and over (to ensure clean environment). I reimplemented in in a way that DDL scripts were run once, then container with database was commited to an image and that image was re-used for every test. Also multiple containers were run at once, so tests could be run simultaneously. Docker is a miracle technology sometimes.

sovietmudkipz 5y ago

I love this. I made an ask HN post about testing practices for games[0]. The answers I got are interesting and an opportunity to reflect. There are clearly some differences in programming cultural practices. I’m of the opinion that applying great testing practices to unity programming is very important. It’s been fun synthesizing ideas I’ve encountered into my hobby game dev career. I’m glad there are others out there doing the good work of generating content to popularize the benefits of good testing practice :) [0] https://news.ycombinator.com/item?id=29459292 https://news.ycombinator.com/item?id=29459292

aappleby 5y ago

This is why you hire old programmers, so that you have someone to yell "WTF" when they see you have no testbench. We had that stuff figured out in the PS1 days.

tayo42 5y ago

I see this is about game development, but in the web/server world idk how anyone ever thinks its acceptable to make development impossible on your local laptop/computer. Everyone time I do a code review I need to ensure they didn't break local development. It blows my mind that I need to do it, like they think its ok to build and deploy for every little test. When I first joined I spent so much time just getting it to work on my laptop. I really just don't understand the mindset. This is has been the case in multiple places. Who was this first person to break local development and decide it was ok. Please fire that person, black list them from getting jobs ever again. We have so many tools to make this easy now like docker and yet it still gets messed up.

xmprt 5y ago
I think this is why I love green field projects so much. More than being able to work on my own code and design from scratch, I appreciate the fact that no one has messed up how testable and runnable the code is from a developer machine.
teeray 5y ago
In some environments, I imposed a rule: “the tests must continue to pass, and I must be able to use the application on my local machine in an airplane on crappy wifi”

substructure 5y ago

AWS CloudFormation had one of the worst user stories related to iteration time a few years ago. The lack of feedback between the writing of the yaml file and validation of the structure/type/format was frustratingly slow. I would pay good money for better tooling in the Infrastructure as Code(IaC) space. Waiting for the resources to update, fail with cryptic error messages, then slowly rollback only to then fail the rollback. Now in an invalid state, manual resource creation was required before the rollback would succeed. The AWS cdk has improved this significantly. As a result the sun shines just a little bit brighter.

suzzer99 5y ago
Getting anything new working in CloudFormation is a great way to kill a day.
ewuhic 5y ago
What exactly you'd want to see from such tooling that it improves your experience with AWS?

kyruzic 5y ago

How I wish a single line code change only took a 15 second build. The java app I work on regularly takes 3 minutes for a single line change. Then you just need to pray that jrebel will work today and actually hot reload your change. Otherwise you have another 8 or so minutes of redploying weblogic. Of course the ideas suggested in this can help, but when you need to run an integration test which also takes minutes to run you realize that 3 lines of code isn't bad for a days work. One day soon I will never touch Java again.

suzzer99 5y ago
Going from Weblogic 10 minute reboots to Node 5 second reboots made development so much more productive and fun for me.
icedchai 5y ago
Your problem is WebLogic, not Java. Try a lightweight app framework (Dropwizard, SpringBoot...) that embeds the "app server" right into your app. Of course if you're using WebLogic, you probably don't have a choice.

senectus1 5y ago

For all the issues I have with Elon Musk, I really like his design philosophy. Step 1 Make it less dumb Step 2 Delete a part of the process If you're not adding step in 10% of the time, you're not deleting enough Step 3 Simplify or Optimize Step 4 Go Faster Step 5 Automate

phone-account-1 5y ago
Nitpick: he actually said "Make the requirements less dumb", which is quite different actually.

gitgud 5y ago

> “it takes too darn long to test these changes, is there a better way?” This is a question we should be asking ourselves every day. Agree, continuously re-evaluate your systems, they can always be better

thibran 5y ago

As someone writing Lisp code, this feels so archaic. Then I remind myself that Lisp is older and than those games and that the future just hasn't caught up yet.

joelbondurant 5y ago

Industry standard build systems require 500 remote servers because YAML is the favorite programming language of infrastructure teams.

yashap 5y ago

If I had to order the things that make me productive, starting with the most important: 1. It’s easy to have a fair amount of confidence in changes without even needing to run them. Strong static typing, good/clean abstractions, local reasoning - little to no mutability, code is mostly pure functions with side effects pushed to the edges. Being able to just write a bunch of code without needing to run it at all (and having it “just work” the first time, most of the time) is HUGE for productivity 2. Excellent automated test suite, with a nice pyramid (small number of E2E tests, solid number of integration tests, exhaustive unit tests). Also excellent monitoring and alerts, with automated canary releases and automated rollback. Basically make it so that it’s hard for mistakes to fully reach production. Being able to be a bit of a cowboy, safely, is huge for productivity too. I’ve worked on plenty of systems without this, and then I’m MUCH more careful, but when you really do have this, it makes everyone so much faster, especially once you learn to stop worrying and trust the guardrails (for simpler changes) 3. Test suite runs quickly locally, and is not flakey 4. Running the app locally is quick. Also ideally I can run just the one component I’m working on (like a front end, or service, or whatever), but have it fit into a full system running elsewhere 5. Full build/test/deploy process on CI is quick and reliable This article emphasizes 3 and 4, and they’re certainly very important, but I think 1 and 2 are even more important. With them in place, often I don’t even need to do 3 and 4 for more trivial changes - just “I’m pretty certain this works,” and then the tests all pass in CI, and I’m very certain. Compared to systems without 1 and 2, where I have to basically change a line, then run tests and/or the app because I’m not sure it’ll work, then do extensive manual testing to be sure because the automated test suite sucks. Muuuuuuch slower. 3 and 4 are great, and necessary for more complex changes, but 1 and 2 let you ship simple changes super quick, without even needing to run anything locally. That’s fastest of all, and can still be quite safe in the right environment.

feffe 5y ago
I agree with your list but really need 4 myself to be a happy camper. Often it's neglected once 2 is in place as most don't seem to find any value by it, or don't care to maintain it. I find that having the possibility to run the real thing (in some capacity) in a cozy development environment with debugger is very useful to quickly learn how parts in the system interact with each other. I have a hard time doing that with unit tests, perhaps because they are so boring.

meheleventyone 5y ago

Whilst I very much agree with the premise I do think this exposes a fallacy many programmers fall for. That passing unit tests means your code actually does what you think it does. Using unit tests this way is a crutch to avoid improving startup times, implementing hot reloading and other schemes to skip gameplay. You lose an awful lot by not testing your code for the job it’s actually expected to do in situ. In games this results in increasing the total iteration time as it’s usually now a different person who actually runs the code, find issues and submits them back.

Ma8ee 5y ago
You need both. If you don't have unit tests your feed back loop becomes way too slow, but then you of course have to test the "real" running of the program properly before shipping. And you need at least one person who didn't write the code to test it.
- meheleventyone 5y ago
  Unit tests are useful for some things. I can’t unit test my way to a game feeling good. For that I need a fast feedback loop to the real program. There’s more of that in most applications than most people suspect. And routes to get there. As programmers we make a lot of little decisions along the way and those don’t get proper consideration unless you can see that as the feature evolves.
rlayton2 5y ago
Isn't the normal process, roughly, to: 1) Use unit tests to move quickly through to implementing The Feature 2) When The Feature is complete, run in a local environment to confirm it works 3) When that works, move The Feature to a development environment that more closely mimics Prod 4) Deploy to Prod Each one of those steps takes an order of magnitude longer than the previous one, so should be done an order of magnitude less often. However if you are finding inconsistencies between steps, then alter local/dev to ensure more consistent testing. (i.e. if it works on Dev, it should just work on Prod, however there are always little issues).

jeffrallen 5y ago

Neves Law says that "the harder a bug is to find, the smaller the fix will be". The worst Neves ratio I ever saw was about a week to two bits, where a plus (0x2b) had to be changed to a minus (0x2d).

typon 5y ago
Week/bit was common when I was working at an FPGA company. Yes FPGA software tools suck.

phendrenad2 5y ago

3 lines of code per day? Damn that's pretty good!

userbinator 5y ago

This is probably a controversial opinion, but I think that working in an environment where time-to-iterate is high is actually very beneficial for improving your skills. It's one of those things that may feel overly burdensome in the short term, but is better in the long term. I say this as someone who taught programming with beginners, and observed what many of them will do when given an IDE, which at the scale of the code they're writing, makes the time-to-iterate very short (press a key and you instantly see your changes). They end up getting into a "dopamine feedback loop" that causes them to continually make tiny changes and rebuild/run, and the code written as a result of this looks exactly like what you'd expect: it barely works, and is full of redundancies, "dead ends", and other evidence that its author was probably not thinking of anything more than the next line or two when writing it. In other words, reducing the iteration time reduces the motivation to get it right the first time, and the associated deep reasoning/"mental execution" skills which are required to do so. In order to develop those skills, one should strive to write as much code as possible before running it, and a high iteration time assists with that. Also, I'm tempted to continue the title with "...unless it's APL".

jmnicolas 5y ago
Your example is valid for beginners but once you're getting experience you don't code like that anymore. After 13+ years of C#, on simple projects I can code for a couple hours without compiling and get the pleasant surprise that my code work the first time I run it. And I'm certainly not a genius, I'm a 1X programmer.
- rvense 5y ago
  Eh, ten years a professional, I frequently fall into it still, where you forget to think about the best solution and just.. mash. Sometimes it works to get a solution that can then cleaned up, other times I waste an hour before I snap out of it. I saw someone post "Stop thinking and look!" as universal advice once, but I need the inverse just as often.
- cardanome 5y ago
  I still do. Of course dabbling into REPL-based solutions in my youth might have influenced that. For me it is crucial to always have have the code in some compile-able and testable state. I iteratively work towards the simplest, most naive solution that gets the job done. After that it is refactoring time and I form it into a proper solution. The reason I do so is because only after having coded the naive solution, I have an concrete and proven correct understanding of the task and can pick the appropriate abstractions. I always see colleagues with an less iterative style use premature-abstractions and over engineering on the basis of "we might need it" instead of knowing. I do invest some time into thinking about the right data structure though. (Not much about algorithms, as they mostly follow from the data structures anyway.) Also if I code a well understood problem, I might use a more top down approach. (Not a critique of your workflow, just wanted to share mine.)

KronisLV 5y ago

Recently decided on using TDD for some library code and aiming for >90% test coverage at my dayjob. Frankly, things do take longer to develop this way and i needed to put in more thought into how the interfaces should be structured, vs just doing what people sometimes do in other projects - just relying on concrete implementations or using frameworks that use reflection to do what they want. However, the experience itself feels like a positive one - even if the code takes longer to craft, it's much easier to work with, since now i don't need to worry about sloppily written if/else chains that depend on enums or runtime type checks, but can utilize different implementations as needed. Also, writing the actual tests allowed me to be sure about how everything would actually work, all the way up to discovering that Paths.get (Java) on Linux accepts almost anything as a valid path string, whereas Windows has actual validation rules, leading to many headaches in regards to the code coverage quality gates that i set up, since the coverage differs based on which platform you run on. But i guess that my point still stands: in certain circumstances slowing down might actually be a good thing, when you really want to know how your code will work under most circumstances and make it maintainable. Of course, on the other hand, compilation speeds and other factors in regards to the speed of iteration definitely shouldn't be overlooked either, and having everything else apart from writing tests and actually thinking about how everything will fit together be faster is a good thing! I'm not sure what i'd do if my test suite took 10 minutes to run instead of 10 seconds as it currently does. Probably drink lots of coffee. I guess it depends on slowing down for good reasons, vs just wasting time because of tooling or other sub optimal circumstances (e.g. what was described in the article, personally i've also seen a local API service be pretty chatty with a remote DB which was slow over a VPN, yet no one had a local DB with all of the migrations in place, and a bunch of other things like that). Offtopic: Anyone remember how fast Pascal compiled? Now that was a really nice stack to do some stuff in, it's a shame that it never got as popular as Java, or didn't have tooling like JetBrains has, it felt like a more ancient Go (which is also pretty good as far as ergonomics go).

roeles 5y ago
I have somewhat the same experience with my first real TDD-based project. Progress was slow, but the amount of debugging afterwards as almost zero. It feels slower, but more predictable. > Recently decided on using TDD for some library code and aiming for >90% test coverage at my dayjob. I thought the point of TDD was that you don't write code unless a test requires it? In my mind (and the little experience I have) that translates to 100% coverage, with exceptions of really annoying side-effects that are near-impossible to test reliably.

dinvlad 5y ago

Going from "manual testing" to automated testing is the biggest jump. Unfortunately, lots of folks (both developers and managers) fall into the trap of thinking they will "use up" their development time on automated tests, and not really thinking how much time they're already wasting just to do (incomprehensive and hand-wavy) manual tests each time. And while this strategy might work for the folks who actually wrote the code (and thus have tacit knowledge about it), the moment they leave and/or someone new joins the team, all that "speed advantage" is lost and it turns from minutes to hours to days instantly. Automated tests really are the only way to capture the business logic of any code for it to not to become "legacy code" before its time.

cjfd 5y ago
Very good points. In his book 'working effectively with legacy code' Michael Feathers actually defines 'legacy code' as code without tests. Largely because of reasons like you state.
- vbezhenar 5y ago
  What do you think about code being too smart for current team? I encountered that use-case three times in my career. It was C++ program which was written by very bright person, used template magic, boost and stuff. Rest of the team were ordinary C developers. After that person left, they had to rewrite his program, because it took too much time to understand how it works and to fix or improve it. Second case was when someone wrote some helper program with Haskell which happened to be important. Same story, nobody knew Haskell, nobody wanted to invest into learning, so rewrote in node.js. Third story: someone used reactive API with Java (Spring Flux) and again same story: team don't understand it and slowly rewrote it with ordinary blocking API. Does that code counts as legacy?
  - hu3 5y ago
    This is where Go shines in my opinion. It's really hard to write smart Go code. Sure it is a bit verbose but dam is it easy to understand due to sheer bluntness and lack of magic.

pechay 5y ago

Aside from looking at changes in your dev methods, this is a good way to demonstrate to your management the value of having fast, up to date hardware on your desk.

onion2k 5y ago

In a future post, I will go over how web developers needs to start taking iteration time more seriously as the influx of new tools and frameworks starts to bloat up build times. The newer tools in web dev have crushed iteration times to a fraction of what they used to be. A combination of things like esbuild, fast refresh, yarn's offline cache, and a few other bits can get your iteration times on a site you're manually testing down to milliseconds to both build and update (literally). Decent devs have been writing tests for years; the test runners could be a bit faster but people are working on that as well. The React app I'm working on at the moment takes well under a second to build, under 10 seconds to run the test suite (could be a sign I need more coverage..), and milliseconds to update the code in the browser in dev mode. It's nice to work with. It's not even clever or special. A default Next.js app will do that. Create-React-app 5 launched a few days ago with more of the tooling too. In Vue things like Vite are based on the new fast tooling and it works well. Vue dev work is fast. SvelteKit even goes further by using Snowpack to remove the bundling step entirely. Snowpack claims a default refresh time of 50ms. I really hope the author's article about web dev is simply "Yeah, update your tools and you'll be fine."

SrZorro 5y ago
> SvelteKit even goes further by using Snowpack to remove the bundling step entirely. FIY SvelteKit is using Vite 2 instead of Snowpack Source: https://twitter.com/Rich_Harris/status/1367577006355976194 https://twitter.com/Rich_Harris/status/1367577006355976194

xwolfi 5y ago

This guy is young and probably not educated in modern dev. I refuse C++ jobs when I ask how they do unit test and they reply it's too hard in C++, and take Java jobs because there the first question THEY ask is HOW I do unit test, what I think of coverage metrics (hints: they don't matter much) and how I propose to enforce ALWAYS unit testing important code. In this blog post the guy took 3 job change to DISCOVER unit tests, it's insane :( It's not like they know but don't have time, it's that he's a "champion" just for proposing limited scope testing.

est31 5y ago
I think the main issue about automatted unit/integration testing is not the language, but the area you are writing code in. Yes, C++ not having good unit test support hurts. But it hugely depends on the problem domain and whether the outputs are machine verifiable, and the inputs are easy to simulate. It's hard to do unit tests in a large part of the area of gamedev. How do you ensure that a picture is rendered correctly? You can take a screenshot and compare it to a stored one. But then what happens if you do artistic changes that you want to do? You have to update the stored screenshots of the testsuite. Who will review that the changes all made sense? And more importantly, there might be slight differences in the output of GPUs, depending on driver versions, model, etc. So let's say you have a bug in your game where if you walk through a level in a specific direction, the game crashes. The error is easy to check for: just make sure that there is no crash. But how do you create a reproduction of the bug? You could record controller inputs and play them back. Then a different department changes something how quickly players move and increase their walking speed by 10%. Suddenly your player walks into a wall and the test is basically broken. Compare this to a CRUD app where you have well defined operations and their impact is well described. That being said, even in gamedev there are areas that are well testable. You can do a unit test of the low level networking layer by trying to make a server and a client, dropping some packets, then looking if the packets still arrived because the networking layer sent them again.

zuhayeer 5y ago

Another thing here is that often times people are so busy "developing" product that really trivial efficiency gains get overlooked. Things like simply removing unnecessary parts from the build at least when you're testing / in dev mode. Also compartmentalizing the build to the portion of the app you're working on, ensuring it's not building the entire app when you're only testing one feature (you'll be surprised how common this is even especially at large companies). It's a huge value add to the rest of your team to bite the bullet and take an hour or so to shave off some minutes from the time you have to wait for a build. That one hour saves you from many multiples more (hours / days / weeks) of fiddling your thumbs in the breakroom waiting for a build to finish. (better yet to have someone regularly scanning for these efficiency related things)

wojciii 5y ago

I do embedded work. I experienced this while working for a multinational. - A long build process and linker step that takes ages. 1-2 minutes for a small change. - Trying the change on a real device took perhaps 5 minutes. - A review process that can take days or even weeks sometimes. - There was a process that merged my change set which often failed and needs to be rerun several times - often 1-2 days were spent on trying to commit something where everyone was doing the same. - There are no unit test because some manger decided that they provide no business value. - There were no simulation tools for software be because some manger decided that they provide no business value. - Testing would be done overnight. With the right tools (unittest and pc simulation) I would be effective and could test my changes prior to testing on a real device. This was for me the bottleneck and not how long it took to build the project. I was testing code that was just C on a complex embedded device that I could have tested/debugged easily on a PC in 1/10 of the time. This was really a management problem where the management didn't understand what to do to make their employees efficient with the right tools.

dezgeg 5y ago
At least you are lucky enough to be able try changes yourself. I am in the "fun" situation where team of 50+ SW engineers don't have single piece of test equipment needed for operating the hardware, and everything has to be manually tested by some QA engineer, taking 1-2 hours minimum...
- wojciii 5y ago
  It can always be worse. :) I feel your pain. This was from an old job. I work a different job now with different challenges.
jeffrallen 5y ago
Hello, fellow ex-Cisco employee.

kubi07 5y ago

My first full time job as a junior dev was to maintain a legacy GWT project which makes tons of $ every year.Setting up the development environment took 1 week. For some odd reason we couldn't build the project module by module. Build was taking at least 2 mins with the bare minimum module count. I quickly become depressed. Waiting the builds and half way into build an error pops-up and you start again. This was 4 months ago, i literally became a deppressed junior dev who hates his job in 1 month. I started applying for job after a month and quit that job after 2 months.

mro_name 5y ago

even less so should three letters take ages, shouldn't it? E = m c^2 Any 5-year old can write this.

roeles 5y ago

> This is the sweet spot amount of time where you become tempted to “do something else” while you wait. I think this is an important observation. The time where you "wait", you can allow your mind to drift off right after being immersed in the problem. In my experience, this time has the potential to give you great insights. I think fast feedback is great, but only if you know what you're trying to accomplish. Or trying to learn. Don't use it as a tool to throw stuff against the wall and see what sticks. Once you start to do that, spend some time away from the computer. Draw a picture, formulate some hypothesis based on your mental model. Make a map before you enter the jungle of trial-and-error.

max002 5y ago

Haha, so true. People are used to old ways. I know its not c++, but If you run docker on mac or Windows for web dev or any other project that has thousanda of deps you know how painful docker sync is and how slow/crash prone. But hey "docker is easy" :)

angarg12 5y ago

If what is slowing your iteration speed is testing, I'd say you are in a somewhat happy place. At least you can write test, speed up the build, or somehow improve the situation. What drains my soul is when you need to use legacy or poorly designed tools that slow you down and you can't do much about. Here are real examples of times I wasted entire mornings: tools that, if you make a mistake, leave the environment in an inconsistent state, and then you need to manually fix it (it isn't documented and changes on a case by case basis). If you miss anything or make another mistake, start from the beginning again.

wingerlang 5y ago

Regarding the "testbeds". I recently built this for an (iOS) application and it helps SO MUCH. Each module in the app has its own local target (the testbed) with a menu which lets you open the module for a given scenario. A scenario is a combination of local JSON for endpoints, device fakes (think fingerprint enabled/disabled) and module-specific configurations. The ability to near instantly get to a specific functionality with the same network requests makes everything super simple. I run the UI tests on these targets as well, and they are near perfectly non-flaky. The best part is when I receive a new bug report from QA, since they include the network logs I usually just need to create a new scenario, register the JSON and fix the reproduced issue.

globular-toast 5y ago

This is something I've worked to get right my entire career. Whenever I start a new project, I make sure the feedback loop is small. If it's a new technology for me, this can take a while to get to a process I'm happy with, but it's worth it. Way back when I was at uni I remember watching over people's shoulders at the painfully slow iteration process. I couldn't believe not a single one thought of doing anything about it. They looked more like factory workers, endlessly turning the same handle and waiting. If you spend your time doing this, you will have no time left for creativity.

solididiot 5y ago

Was working in a similar product and it was by far the worst dev experience I've ever had. It was actually worse than what the post describes. It was like having to go through a lot of stages to get to the area I'm working and half of them not working at all - so instead I was doing bug reports on other people's areas which was only slightly familiar with. (Yeah, our CI QA gateway was a joke. Our CI in general was a joke). It was hell. That much so that I jumped to the first half-decent role that came my way. Something is wrong with the way we build systems that integrate too many other systems. Perhaps if we designed them with a TDD approach all the way up things would be better...or worse. Dunno. Sometimes what we're trying to do is just way into the entropy zone.

nearmuse 5y ago

While it sounds painful because of the inefficient development & testing process, I think that if that day is spent on research instead (looking for a better solution or reading existing code) it is OK to spend 1 day on 3 lines of code.

deleted 5y ago

[deleted]

Sneha_1 5y ago

noob like me takes all day

rob_c 5y ago

By comparison I've worked with someone who pushed a '2 line fix' live which had fallout which lasted >2 weeks because their code was pushed without checking to production and had hard-coded their uid into the fix because it was a quick 'hack'... The main reason when cornered was 'running pip install is too difficult to run the test-suite' (tests themselves took <30sec to run all ~300). We eventually (1 week later) got permission to pull the release from our shared (pseudo write-only) storage area but as he was project lead he tagged a new minor version for his '2 line fix' without consulting anyone and our users were reluctant to move to a new bugfix release unless prodded with a red-hot poker. Can't say I miss working with _some_ programmers turned managers.

jarek83 5y ago

Sounds like EA has (had?) actually pretty good developers. The game itself has so many things gone wrong that from the players perspective it's very hard to find any good words for. Shame that the company hates their customers so then customers started to hate anyone involved in EA. In the forums of the game, devs of this game are considered to be the laziest and the least skilled across the industry. PS. Anyone did figure out whether DDA is officially in the code or is it just a matter of network quality difference between players and the server (the game stopped to be p2p in any mode)

ineedasername 5y ago

Testing changes here could mean progressing through several seasons of career mode in order to test out a change That's like the Groundhogs Day of programming. Living the same season over and over again, changing things until you get it right and can move on. Sure, ai guess that's kind of programming in general, but on this level it's most of your day. No developed cheat codes?

ChrisMarshallNY 5y ago

> 3 lines of code shouldn't take all day Sure they should. I often spend all day on negative numbers of lines (I always say that the best code I write, is the code I don't write). The complaint seems to center on CI/D services that provide inefficient build times. These increase the pain of "round-trip" implement-test-refactor. That's quite valid, as this is how we tend to work, these days (at least, that's how I work). I often take it a step further, and iterate the design as I progress[0]. But I am one that got their start in the waning days of "big iron," when compute time was the costliest and most time-consuming part of the whole process. You'd need to schedule for computer runs, which would often happen overnight. This meant that it was very important to have your code complete, and debugged, before submitting it for compilation. Argh. I miss it like I miss a case of food poisoning. It did teach me to do my homework, though. [0] https://littlegreenviper.com/miscellany/evolutionary-design-specification/ https://littlegreenviper.com/miscellany/evolutionary-design-...

munificent 5y ago
> Sure they should. I often spend all day on negative numbers of lines (I always say that the best code I write, is the code I don't write). You're missing the point of the article. It's not about valuing quantity of code. It's about valuing minimizing the time it takes to determine what you want the code to be. If it takes you a 30 minute compile loop to add a line of code, it's also going to take a 30 minute compile loop to refactor and eliminate a line of code. The latter is actually worse. When your iteration time is high, developers will deal with it and push through as necessary to get features implemented because they have to make the software do a certain thing. But they will absolutely not struggle through a shitting iteration cycle if the only result is cleaner refactored code. If you want nice codebases, you need a nice iteration cycle.
- ChrisMarshallNY 5y ago
  > You're missing the point of the article. Actually, I didn't. Just sayin'.
  - 5y ago

atum47 5y ago

I once spent the better part of a day figuring out a off by one error, when I pushed the fix it changed two characters. Almost the whole day, imagine that.

flohofwoe 5y ago

Entirely relying on unit tests for development can never be the only solution, because tests are quite simply not the product, and they shouldn't be a substitute/workaround for fast iteration right on the product. Easier said than done of course, especially in such a complex environment as game development. The future for fast compiled-code iteration in game development will most likely be hot code reloading, so that code changes are compiled and implanted right into the running game instead of requiring a full linker run and then getting back to the right place in the game to be tested.

commandlinefan 5y ago
> Entirely relying on unit tests for development I don't think anybody is ever suggesting this - but I do see people suggest the opposite: that all testing should be end-to-end "black box" testing, and unit tests are a waste of time. If you actually want to ship something that works reliably, you have to do both unit testing and end-to-end (integration) testing. I've never seen anybody sacrifice integration testing. I have seen them sacrifice unit testing.

whalesalad 5y ago

I love building test harnesses. I think the trick is making sure that your system is built in such a way that 1. You can actually pull the core of it out to run in a different situation and 2. You keep enough of the real system such that you’re not getting lots of green passes which are going to completely fail in the real world.

cagenut 5y ago

The "sub-second response loop flow state" he mentions here is "the doherty threshold"[1] just applied to programing as a particular user experience. After working at a CDN that delivered a couple hundred kb objects in milliseconds around the world I thought "code is data, so why cant code be updated in milliseconds?" I tried starting a startup that could do this, treat code as data, and therefore achieve sub-second round-trip trial-and-error loops. Did not pan out, but to be honest we never really got to testing that thesis. I still think it could be amazing, just not sure how much of the lang/ide/build/test/deploy/validate cycle you could integrate and how much you'd have to build. IMHO this is the main thing that made PHP successful. The "edit a file -> alt-tab -> click refresh" test loop being faster than you could click. [1] - https://lawsofux.com/doherty-threshold/ https://lawsofux.com/doherty-threshold/

wwilim 5y ago

Coming across this article and reading it while waiting for a long build was very ironic

MarkLowenstein 5y ago
Haha, yes. If it weren't for the phenomenon written about in this article, I bet HN readership would plummet at least 20%.

Tarucho 5y ago

I worked in lots of projects like this. The most amazing part is most devs didn´t care. Never understood why. In one of those projects the code got so big and bloated that it took around 1 minute (sometimes more) to move from one line to the next while debugging. For me it was a torture, it was ok for most.

bitwize 5y ago

Makes me even sadder that something like GOAL didn't take off. In GOAL, you could make changes at the REPL, compile instantly, push the compiled changes out to the console and see them immediately in the running engine. There's a reason why old-school Naughty Dog games were so refined.

allo37 5y ago

IME reducing iteration time is one of the best ways to increase my productivity. I like Android Studio's approach to running unit tests: It will run them on the host PC instead of deploying them to the target, so you can avoid the often long deployment process.

webmaven 5y ago

The last time some one told me something like "changing three lines of code shouldn't take all day"[0], I responded with "Fuck you" because the way they managed to work 'faster' was to make and test their changes in production before committing them to version control. I also once spent a week pursuing a small-but-necessary authentication flow change across two dependencies (one of them owned by a different team) and three tests, only to be fired for talking to the other team members directly (I was a mere contractor) rather than playing 'telephone' and relaying everything through my management and their management, which would have made it take a month instead. [0] It was actually five lines of code.

PaulDavisThe1st 5y ago

In 1989, I was working at my first (very short lived) US job, for Bell Labs (not the Bell Labs, but related). I was working on their "office-scale" phone switches, the kind that allowed you to forward a call from one extension to another. There was a hard-coded limit of (I think) 16 forwards, and for some reason it was decided to increase this to (I think) 64. This involved changing a constant in a header file, a documentation string and one conditional in the code that for some reason didn't use the constant. Expected time for this work was: 1 week.

e12e 5y ago
Don't leave us hanging - how far over budget did it end up? ;)
- PaulDavisThe1st 5y ago
  Under budget, but at the end of that week, having successfully pushed through this DRAMATIC CHANGE, I drove from Phila. to the Outer Banks (NC), crashed, rolled the car 3 times, totalled the car, broke my arm, could no longer get to work, decided to marry my first wife, and left to work for a company I could commute to with a broken arm. Small change, big ramifications :)

thrower123 5y ago

It's always fascinating how game development studios tend to consistently lag ten or fifteen years behind enterprise software development practices.

Jtsummers 5y ago

From a systems perspective, there is a balance to be achieved. You want faster iteration times in order to provide feedback, but if they're too fast then you neglect other things or end up swinging wildly. Consider a thermostat that turned on the heat or cooling at just .1 degree below or above its target, or turned it off as soon as it hit the reverse. Or if because the feedback comes in as a torrent of data you attend to that feedback instead of other elements of the system or your capabilities. As a system gets more complicated you have more factors to consider. In the case of programming, if you rely too much on that near-instant feedback, how much are you internalizing about your system, language, and environment? How often are you making the same errors but recovering quickly (so it's not slowing you down too much, or doesn't appear to be slowing you down too much) because of the feedback? How much faster could you ultimately be if your work were smoother and not so rough and jagged? Introducing a delay, here, gives you time to contemplate. Even if it's just taking an hour or two a day to sit back from the keyboard and ponder what you've done that day and what you will do the next. Create a plan instead of jump into action, even if the plan doesn't get executed perfectly or ends up being the wrong plan that bit of contemplation is when you learn. But too long a delay (especially a forced delay) causes other problems. The actually needed feedback (not just compiler errors and such, but your V&V issues discovered from testing and evaluation) getting delayed by a day or more can be too much (especially when working with a team, where other parts are potentially changing around your own changes). Too long a delay also promotes batching many unrelated changes together because you don't want to sit through the whole process again. Consider a system that takes a week or a month to get feedback from an external test team, you'd be tempted to throw many changes at them because of that week or month long delay (or more!) and not just one change. So strike a balance, find a point where your iteration cycle permits you time to think and not just act so you can really learn (both programming and the particular system you're trying to develop). Smooth out your development so that you're slowed down not by having to take corrective steps but having to ponder logical steps. "Is this the right data structure? Well, if I isolate it in my domain model then I can swap it out later and no one will be impacted." or "I'll take a walk around the building and think about what I actually need here."

literallyaduck 5y ago

You can pack in a lot more training, reddit, and mobile games into a day when you only write 3 loc.

makach 5y ago

Sometimes a negative amount of code lines took me whole days...

vbphprubyjsgo 5y ago

Uhhh... just improve your build process.

adolph 5y ago

Bill Atkinson, the author of Quickdraw and the main user interface designer, who was by far the most important Lisa implementor, thought that lines of code was a silly measure of software productivity. He thought his goal was to write as small and fast a program as possible, and that the lines of code metric only encouraged writing sloppy, bloated, broken code. https://www.folklore.org/StoryView.py?story=Negative_2000_Lines_Of_Code.txt https://www.folklore.org/StoryView.py?story=Negative_2000_Li...

nanoscopic 5y ago

Suppose you maintain a highly scaled service. Suppose it has a bug. Finding and fixing 3 lines of code that caused a problem that was costing the company money by the hour/minute could be well worth it and a great accomplishment in one day. The number of lines of code is irrelevant. The impact is what matters.