4 ms·

Unpopular opinion with a lot of my peers, but "fragile local dev environment" is usually a sign of an unwieldy, tightly coupled codebase with a lot of parts, sc

by gedy 5y ago

Unpopular opinion with a lot of my peers, but "fragile local dev environment" is usually a sign of an unwieldy, tightly coupled codebase with a lot of parts, scripts, etc. Trying to hide that complexity with docker or codespaces, etc is just a bandaid in many cases vs dealing with the root issues.

jshen 5y ago
This has not been my experience. First, I work on lots of code bases and this is where a lot of the brittleness comes from. They don’t use the same versions of the same things. Second, C dependencies are a giant pain in the ass when you add the first problem above.
- cat199 5y ago
  Meanwhile, on BSD UNIX derivatives, I can 'make world' since the dawn of time...
  - jshen 5y ago
    Ah yes, then there is the third big complexity. Some devs are on Mac, some windows, some Linux, and a few rare people on BSD.
  - remram 5y ago
    You still have to grab the dependencies before running make, some of which you'll grab from apt/yum/chocolatey/macports, some you'll need to compile manually and put in a platform-dependent location, ... Then you can run `make world`. Of course, BSD make is not completely compatible with GNU make...
    - umanwizard 5y ago
      OpenBSD and FreeBSD at least can compile themselves from /usr/src on a default install, fresh out of the box with zero additional packages.
      - remram 5y ago
        "It can compile on a single platform from curated sources" is a very easy requirement to meet.
jtsiskin 5y ago
For an org the size of GitHub, it’s not unreasonable that a local dev environment is difficult to accomplish
annowiki 5y ago
In my experience, even with simple flask projects, I've found just the difference between Ubuntu and Arch to be irritating: 1. Ubuntu seems to have trouble with psycopg2 while Arch does not, so I often use psycopg2-bin just so Ubuntu developers can easily install the requirements (learned the hard way). 2. Some older (supported) editions of Ubuntu create extra lines in `pip freeze`, including `pkg-resource=0.0.0` which throws an error when installing on nearly any machine (including cloud versions of Ubuntu in Github Actions. I have an application that uses docker-compose and has Elasticsearch as an image. In order to run it, I have to expand the vm_max_map (or something, I forget). But there are different ways of doing this in Windows vs Linux. Differences between environments are legion. Unless a company wants to just send standardized laptops to every developer, it seems way better to just spin up a cloud vm that is already set. Plus, there's easier reproducibility in a cloud vm vs a physical machine (unless you want to reach for something like nixos, but hell, what company is equipped for that yet?
- l-p 5y ago
  > difference between Ubuntu and Arch The difference between Debian and Ubuntu is enough to be a nuisance. The Dell XPS laptop my employer provided cannot boot Debian and I'm stuck with Ubuntu. Same packages, maybe a version or two are more recent right? Nope. Aside from the egregious transparent snap installs when using apt (which can't uninstall snaps, thanks Canonical), there's a lot of minute details that amount to quite a time loss (that and having a laptop with no USB/RJ45/HDMI ports). Simple scripts that work on one OS and not the other, font scaling and rendering making my standard 1080p screen unreadable, absurd default configuration that makes WiFi unusable, etc. Glad I don't have to manage the same oddities for the actual work I do. Everything is simple once it's all Dockerized. But that's not true either. I had a different problem setting up the repository with every developer and they were all using the same OS. No repo in sources.list (how?), no docker group created when re/installing the docker package (why?), all kind of weird stuff happening. So you tell me there's a way to have all devs working and testing on the same environment? That's great! But it is _not_ worth handing over everything to Microsoft or any other third party. I'll never trust GitHub, GitLab, or any other forge with business-critial stuff. Host your own code, commit your dependencies, have your build script work locally. I've deployed through Git{Hub,Lab} outages and the leftpad fiasco, and the cost was insignificant. Host a mirror there for cheap if you want, that's what I do for my public projects (but after the copilot debacle how can I trust GitHub with anything?). And what if you're a remote team? You can't rely on the internet. My ISP was kind enough to remind me of this when I got my very first outage with them one hour before starting at my current company. As valuable a product GitHub is and Coderspace may be, it is a step closer to the Minitel 2.0 where everything is centralized and controlled by a single entity. It's not the internet, it's MSN at best.
michaelpb 5y ago
I'm totally with you here. Like, ideally you should be able to clone, install packages, and then do ``npm run start'' or ``python manage.py runserver'' or whatever, and the result should be a mostly functioning web app running locally. If it needs certain attached services, they should be 1) as standard/boring as possible (eg SQL, so you can have mocked versions of them for tests), 2) and added sparingly in only well-motivated situations (eg do you really need to have that caching DB, before you even have profiled real traffic?) IMO investing time to keep applications "normal" like this pays off in the long run. It keeps development closer to "greenfield", which causes a general multiplying effect. Tutorials are slightly more likely to work, 3rd party packages will be quicker to integrate, etc etc. (Note that this is separate from dev/prod parity, which is also good.)
decebalus1 5y ago
It depends on how exactly the tools are used. If you're using docker for reproducible builds with locked-in/platform specific versions of dependencies that need to work seamlessly across platforms then that's pretty good. If you NEED docker/codespaces/etc.. because in order to do basic development and run tests you need N servers running with some specific state, duct taped with non-idempotent bash scripts then this type of stuff is just piling on tech debt.
adrianmonk 5y ago
I think a lot of times the reason local dev environments are fragile isn't because they're local, it's because they are an afterthought and not a lot of effort goes into making them good and stable: (1) On a team of developers, there usually isn't someone who is designated to be responsible for designing and maintaining a good dev environment. (2) Many devs don't have the expertise to do it. Build tools, operating systems, dependencies, and configuration management are a related but different skillset than coding. (3) Many devs don't want to work on it because they enjoy coding more. Just because you enjoy one type of work doesn't mean you enjoy the related work. (This also applies to writing documentation.) (4) Fixing / improving the dev environment is not rewarded. You need to get X feature done by Y date, and fixing the dev environment may slow you down. It provides a long-term benefit, but not one that is easily measurable, plus that benefit is off the radar when evaluating the contributions of someone whose primary role is coding. With this cloud environment, they've separated those responsibilities out and given them to a specialized team dedicated to it. IMHO, some of the success is probably due to the organizational structure, not just the technical differences.

by gedy 5y ago

jshen 5y ago

This has not been my experience. First, I work on lots of code bases and this is where a lot of the brittleness comes from. They don’t use the same versions of the same things. Second, C dependencies are a giant pain in the ass when you add the first problem above.

cat199 5y ago
Meanwhile, on BSD UNIX derivatives, I can 'make world' since the dawn of time...
- jshen 5y ago
  Ah yes, then there is the third big complexity. Some devs are on Mac, some windows, some Linux, and a few rare people on BSD.
- remram 5y ago
  You still have to grab the dependencies before running make, some of which you'll grab from apt/yum/chocolatey/macports, some you'll need to compile manually and put in a platform-dependent location, ... Then you can run `make world`. Of course, BSD make is not completely compatible with GNU make...
  - umanwizard 5y ago
    OpenBSD and FreeBSD at least can compile themselves from /usr/src on a default install, fresh out of the box with zero additional packages.
    - remram 5y ago
      "It can compile on a single platform from curated sources" is a very easy requirement to meet.

jtsiskin 5y ago

For an org the size of GitHub, it’s not unreasonable that a local dev environment is difficult to accomplish

annowiki 5y ago

In my experience, even with simple flask projects, I've found just the difference between Ubuntu and Arch to be irritating: 1. Ubuntu seems to have trouble with psycopg2 while Arch does not, so I often use psycopg2-bin just so Ubuntu developers can easily install the requirements (learned the hard way). 2. Some older (supported) editions of Ubuntu create extra lines in `pip freeze`, including `pkg-resource=0.0.0` which throws an error when installing on nearly any machine (including cloud versions of Ubuntu in Github Actions. I have an application that uses docker-compose and has Elasticsearch as an image. In order to run it, I have to expand the vm_max_map (or something, I forget). But there are different ways of doing this in Windows vs Linux. Differences between environments are legion. Unless a company wants to just send standardized laptops to every developer, it seems way better to just spin up a cloud vm that is already set. Plus, there's easier reproducibility in a cloud vm vs a physical machine (unless you want to reach for something like nixos, but hell, what company is equipped for that yet?

l-p 5y ago
> difference between Ubuntu and Arch The difference between Debian and Ubuntu is enough to be a nuisance. The Dell XPS laptop my employer provided cannot boot Debian and I'm stuck with Ubuntu. Same packages, maybe a version or two are more recent right? Nope. Aside from the egregious transparent snap installs when using apt (which can't uninstall snaps, thanks Canonical), there's a lot of minute details that amount to quite a time loss (that and having a laptop with no USB/RJ45/HDMI ports). Simple scripts that work on one OS and not the other, font scaling and rendering making my standard 1080p screen unreadable, absurd default configuration that makes WiFi unusable, etc. Glad I don't have to manage the same oddities for the actual work I do. Everything is simple once it's all Dockerized. But that's not true either. I had a different problem setting up the repository with every developer and they were all using the same OS. No repo in sources.list (how?), no docker group created when re/installing the docker package (why?), all kind of weird stuff happening. So you tell me there's a way to have all devs working and testing on the same environment? That's great! But it is _not_ worth handing over everything to Microsoft or any other third party. I'll never trust GitHub, GitLab, or any other forge with business-critial stuff. Host your own code, commit your dependencies, have your build script work locally. I've deployed through Git{Hub,Lab} outages and the leftpad fiasco, and the cost was insignificant. Host a mirror there for cheap if you want, that's what I do for my public projects (but after the copilot debacle how can I trust GitHub with anything?). And what if you're a remote team? You can't rely on the internet. My ISP was kind enough to remind me of this when I got my very first outage with them one hour before starting at my current company. As valuable a product GitHub is and Coderspace may be, it is a step closer to the Minitel 2.0 where everything is centralized and controlled by a single entity. It's not the internet, it's MSN at best.

michaelpb 5y ago

I'm totally with you here. Like, ideally you should be able to clone, install packages, and then do ``npm run start'' or ``python manage.py runserver'' or whatever, and the result should be a mostly functioning web app running locally. If it needs certain attached services, they should be 1) as standard/boring as possible (eg SQL, so you can have mocked versions of them for tests), 2) and added sparingly in only well-motivated situations (eg do you really need to have that caching DB, before you even have profiled real traffic?) IMO investing time to keep applications "normal" like this pays off in the long run. It keeps development closer to "greenfield", which causes a general multiplying effect. Tutorials are slightly more likely to work, 3rd party packages will be quicker to integrate, etc etc. (Note that this is separate from dev/prod parity, which is also good.)

decebalus1 5y ago

It depends on how exactly the tools are used. If you're using docker for reproducible builds with locked-in/platform specific versions of dependencies that need to work seamlessly across platforms then that's pretty good. If you NEED docker/codespaces/etc.. because in order to do basic development and run tests you need N servers running with some specific state, duct taped with non-idempotent bash scripts then this type of stuff is just piling on tech debt.

adrianmonk 5y ago

I think a lot of times the reason local dev environments are fragile isn't because they're local, it's because they are an afterthought and not a lot of effort goes into making them good and stable: (1) On a team of developers, there usually isn't someone who is designated to be responsible for designing and maintaining a good dev environment. (2) Many devs don't have the expertise to do it. Build tools, operating systems, dependencies, and configuration management are a related but different skillset than coding. (3) Many devs don't want to work on it because they enjoy coding more. Just because you enjoy one type of work doesn't mean you enjoy the related work. (This also applies to writing documentation.) (4) Fixing / improving the dev environment is not rewarded. You need to get X feature done by Y date, and fixing the dev environment may slow you down. It provides a long-term benefit, but not one that is easily measurable, plus that benefit is off the radar when evaluating the contributions of someone whose primary role is coding. With this cloud environment, they've separated those responsibilities out and given them to a specialized team dedicated to it. IMHO, some of the success is probably due to the organizational structure, not just the technical differences.