7 ms·
Solid analysis. A word of warning from personal experience: I am part of a medium-sized software company (2k employees). A few years ago, we wanted to improve
by LASR 3y ago
Solid analysis.
A word of warning from personal experience:
I am part of a medium-sized software company (2k employees). A few years ago, we wanted to improve dev productivity. Instead of going with new laptops, we decided to explore offloading the dev stack over to AWS boxes.
This turned out to be a multi-year project with a whole team of devs (~4) working on it full-time.
In hindsight, the tradeoff wasn't worth it. It's still way too difficult to remap a fully-local dev experience with one that's running in the cloud.
So yeah, upgrade your laptops instead.
- jiggawatts 3y agohttps://xkcd.com/1205/ https://xkcd.com/1205/
- mdbauman 3y agoThis xkcd seems relevant also: https://xkcd.com/303/ https://xkcd.com/303/ One thing that jumps out at me is the assumption that compile time implies wasted time. The linked Martin Fowler article provides justification for this, saying that longer feedback loops provide an opportunity to get distracted or leave a flow state while ex. checking email or getting coffee. The thing is, you don't have to go work on a completely unrelated task. The code is still in front of you and you can still be thinking about it, realizing there's yet another corner case you need to write a test for. Maybe you're not getting instant gratification, but surely a 2-minute compile time doesn't imply 2 whole minutes of wasted time.
- chiefalchemist 3y agoSpot on. The mind often needs time and space to breathe, especially after it's been focused and bearing down on something. We're humans, not machines. Creativity (i.e., problem solving) needs to be nurtured. It can't be force fed. More time working doesn't translate to being more effective and more productive. If that were the case then why are a disproportionate percentage of my "Oh shit! I know what to do to solve that..." in the shower, on my morning run, etc.?
- majikandy 3y agoI love those moments. Your brain has worked on it in the background like a ‘bombe’ machine cracking the day’s enigma code. And suddenly “ding… the days code is in!”
- chiefalchemist 3y agoYou might like the book "Your Brain on Work" by Dr David Rock. In fact, I'm due for a re- read. https://davidrock.net/about/ https://davidrock.net/about/
- seadan83 3y agoI agree to some extent. Though, I don't think it has to be a trade-off though. After a sub-5 second compile time, I go over to get a coffee to ponder the results of the compile rather than imagine what those results might be. Taking time to think is not mutually exclusive to a highly responsive dev process.
- perrygeo 3y agoYes! Pauses allow you to reflect on your expectations of what you're actually compiling. As you sit in anticipation, you reflect on how your recent changes will manifest and how you might QA test it. You design new edge cases to add to the test suite. You sketch alternatives in your notebook. You realize oh compilation will surely fail on x because I forgot to add y to module z. You realize your logs, metrics, tests and error handling might need to be tweaked to unearth answers to the questions that you just now formulated. This reflection time is perhaps the most productive time a programmer will spend in their day. Calling it "wasted" reflects a poor understanding of the software development process.
- norir 3y agoI get what you are saying but I still think fast compilation is essential to a pleasant dev experience. Regardless of how fast the compiler is, there will always be time when we are just sitting there thinking, not typing. But when I am implementing, I want to verify that my changes work as quickly as possible and there is really no upside to waiting around for two minutes.
- newaccount74 3y agoIf you can figure out something useful to do during a two minute window, I envy you. I really struggle with task switching, and two minutes is the danger zone. Just enough time to get distracted, by something else; too little time to start meaningful work on anything else... Hour long compiles are okay, I plan them, and have something else to do while they are building. 30 second compiles are annoying, but don't affect my productivity much (except when doing minor tweaks to UI or copywriting). 2-10 minute compiles are the worst.
- behnamoh 3y agoI disagree though. If a task is boring and repetitive, I just won't ever do it. So the comparison for people like me is: "spend X time to automate this task vs not do the task at all". Whereas the xkcd is like (n = frequency that you do the task): "Spend X time to automate this task that takes Y×n time normally and get it down to Z×n time, vs spend Y×n time to do the task"
- faeriechangling 3y agoEven where a task being automated is likely to be done, automation can mean it’s done more reliably or quickly. Automation is generally needed to advance to greater and greater levels of complexity, time-efficient or not. Also not all minutes have equal value, spending a few hours to save 5 minutes in an emergency with automation can be well worth it.
- WaxProlix 3y agoI suspect things like GitHub's Codespaces offering will be more and more popular as time goes on for this kind of thing. Did you guys try out some of the AWS Cloud9 or other 'canned' dev env offerings?
- hmottestad 3y agoMy experience with GitHub Codespaces is mostly limited to when I forgot my laptop and had to work from my iPad. It was a horrible experience, mostly because Codespaces didn’t support touch or Safari very well and I also couldn’t use IntelliJ which I’m more familiar with. Can’t really say anything for performance, but I don’t think it’ll beat my laptop unless maven can magically take decent advantage of 32 cores (which I unfortunately know it can’t).
- Kwpolska 3y agoAWS Cloud9 is a web IDE that can run on any EC2 box. The web IDE is a custom Amazon thing and is quite mediocre.
- rsanek 3y agoThis might have to do with scale. At my employer (~7k employees) we started down this path a few years ago as well, and while it has taken longer for remote to be better than local, it now definitively is and has unlocked all kinds of other stuff that wasn't possible with the local-only version. One example is working across multiple branches by switching machines instead of files on local has meant way lower latency when switching between tasks.
- vghaisas 3y agoHaha, as I read more words of your comment, I got more sure that we worked at the same place. Totally agree, remote devboxes are really great these days! However, I also feel like our setup was well suited to remote-first dev anyway (eg. syncing of auto-generated files being a pain for local dev).
- kaishiro 3y agoOne thing I've never understood (and admittedly have not thoroughly researched) is how a remote workspace jives with front-end development. My local tooling is all terminal-based, but after ssh'ing into the remote box to conduct some "local" development, how do I see those changes in a browser? Is the local just exposed on an ip:port?
- avbor 3y agoYou can expose the browser port via ssh, with a command line flag like `-L 8080:127.0.0.1:8080`. So you can still preview locally
- dmos62 3y agoMy dev box died (that I used for remote work), and instead of buying something new immediately, I moved my setup to a Hetzner cloud vps. Took around 2 days. Stuff like setting up termux on my tablet and the cli environment on the vps was 90 percent of that. The plus side was that I then spent the remaining summer working outside in the terrace and in the park. Was awesome. I was able to do it because practically all of my tools are command line based (vim, etc).
- sweetjuly 3y agoHow much does this cost you? I've been dealing with a huge workstation-server thing for years in order to get this flexibility and while the performance/cost is amazing, reliability and maintenance has been a pain. I've been thinking about buying some cloud compute but an equivalent workstation ends up being crazy expensive (>$100/mo).
- dmos62 3y ago6 eur/month. It's the most basic offering. I don't really need power for what I'm doing. Running tests locally took a bit longer than I had been used to, but just a bit. I'm pretty used to underpowered environments though. I find that underpowering is a good way to enforce a sort of hygiene in terms of what you do and how.
- dinvlad 3y agoThere’s a crazy good deal for a dedicated server with 14-core/20-thread i5-13500 CPU and 64GB RAM, for just around 40 EUR/mo: https://www.hetzner.com/dedicated-rootserver/matrix-ex https://www.hetzner.com/dedicated-rootserver/matrix-ex This is honestly a bit overkill for a dev workstation (unless you compile Rust!), but since it’s a dedicated server it can also host any number of fully isolated services for homelab or saas. There’s nothing else like it in the wild, afaik.
- randomgiy3142 3y agoI’d be careful with Hetzner. I was doing nothing malicious and signed up. I had to submit a passport which was valid US. It got my account cancelled. I asked why and they said they couldn’t say for security reasons. They seem like an awesome service, I don’t want knock them I just simply asked if I could resubmit or something the mediate and they said no. I don’t blame them just be careful. I’m guessing my passport and face might have trigged some validation issues? I dunno.
- xvector 3y agoThis is just a manpower thing. At large tech companies like Google, Meta, etc the dev environment is entirely in the cloud for the vast majority of SWEs. This is a much nicer dev experience than anything local.
- makeitdouble 3y agoIt of course steongly depends on what your stack is, my current job provides a full remote dev server for our backend and it's the best experience I've seen in a long time. In particular having a common DB is suprinsingly uneventful (nobody's dropping tables here and there) while helping a lot. We have interns coming in and fully ready within an hour or two of setup. Same way changing local machines is a breeze with very little downtime.
- __jem 3y agoIsn't the point of a dev environment precisely that the intern can drop tables? Idk, I've never had a shared database not turn to mush over a long enough period, and think investing the effort to build data scripts to rebuild dev dbs from scratch has always been the right call.
- makeitdouble 3y agoDropping tables to see what happens or resetting DBs every hour is fine with a small dataset, but it becomes impractical when you work on a monolith that talks to a set of DB with a hundred+ tables in total and takes 5 hours to restore. As you point out rebuilding small test datasets instead of just filtering the prod DB is an option, but those also need maintenance, and take a hell of time to make sure all the relevant cases are covered. Basically, trying to flee from the bulk and complexity tends to bring a different set of hurdles and missing parts that have to be paid in time, maintenance and bugs only discovered in prod. PS: the test DB is still reset everyday. Eorse thing happening is we need to do something else for a few hours until it's restored.
- arthens 3y ago> We have interns coming in and fully ready within an hour or two of setup. Same way changing local machines is a breeze with very little downtime. This sounds like the result of a company investing in tooling, rather than something specific to a remote dev env. Our local dev env takes 3 commands and less than 3 hours to go from a new laptop to a fully working dev env.
- tuananh 3y agoThanks for the insight. It maybe depends on each team too. While my team (platform & infra) much prefer remote devbox, the development teams are not. It could be specific to my org because we have way too many restrictions on the local dev machine (eg: no linux on laptop but it's ok on server and my team much prefer linux over crippled Windows laptop).
- eysi 3y agoMy team has been developing against a fully remote environment (K8s cluster) for some years now and it makes for a really powerful DevEx. Code sits on our laptops but live syncs to the remote services without requiring a Docker build or K8s deploy. It really does feel like local. In particular it lets us do away with the commit-push-pray cycle because we can run integ tests and beyond as we code as opposed to waiting for CI. We use Garden, (https://docs.garden.io https://docs.garden.io) for this. (And yes I am afilliated :)). But whether you use Garden or not, leveraging the power of the cloud for “inner loop” dev can be pretty amazing with right tooling. I wrote a bit more about our experience here: https://thenewstack.io/one-year-of-remote-kubernetes-development-lessons-learned/ https://thenewstack.io/one-year-of-remote-kubernetes-develop...
- jayd16 3y agoKind of interesting to think that CI is significantly slower in practice and both systems need to be maintained. Is it just the overhead of pushing through git or are there other reasons as well?
- mewpmewp2 3y agoYou would need a very perfect and flexible CI system in place that wouldn't need to rebuild anything it doesn't need and only run the tests you want or only recently failed tests etc. Many CI systems would spin up a new box instead of using persistent so likely have to rebuild if no cache, etc. So basically I would say most of the overhead is in not having a persistent box with knowledge of last build or ability to choose what to run in there, which pretty much just equals to local capabilities.
- jeffbee 3y agoMy company piles so much ill-considered Linux antivirus and other crap in cloud developer boxes that even on a huge instance type, the builds are ten or more times slower than a laptop, and hundreds of times slower than a real dev box with a Threadripper or similar. It's just a pure waste of money and everyone's time. It turns out that hooking every system call with vendor crapware is bad for a unix-style toolchain that execs a million subprocesses.
- rc_kas 3y agomy company did this. fuck i hate it so much. if anyone wants to hire me away from this remote desktop hellscape, please do.
- vb6sp6 3y agoI've been working this way for years, really nice. What is main complaint?
- Aeolun 3y agoSlowness, latency, lack of control. The usual suspects? There’s moments where you try to do a thing that normal on a local PC and it’s impossible on remote. That cognitive dissonance is the worst.
- rc_kas 3y agoYep. And shortcut keys and other smaller behaviors like that get weird sometimes.
- vladvasiliu 3y agoIf I understand correctly, they're not talking about remote desktops. Rather, the editor is local and responds normally, while the heavy lifting of compilation is done remotely. I've dabbled in this myself, and it's nice enough.
- tracker1 3y agoSecond on this. Not being able to run a solution entirely local introduces massive friction in terms of being able to reason with said solution. When you need to have 200+ parts running to do anything, it can be hard to work in a single piece that touches a couple others. With servers that have upwards of 128+ cores and 256+ threads, my opinion is swinging back in favor of monoliths for most software.
- deleted 3y ago[deleted]