8 ms·

Dev onboarding, then and now

42 points by fahhem 3y ago

pasc1878 3y ago
The first issue about the machine being macOS or Windows but the code assuming Linux has never been a problem in my experience. Now containers or Nix do make it easier. If the system is deployed on Linux then you run Linux in a VM. (Which also deals with some of the other issues) I have done that since the early 2000 or even before we remotely logged in to a machine running the correct OS (X Windows is useful) albeit I was on site so a fast connection to the server. Now whether it is good design to make assumptions that only Linux matters is another question. I am old enough that I had to use multiple Unix systems SunOS Solaris NeXTStep and Linux. And good practice was to compile C++ with two different compilers as the error messages and edge cases were not very edge.
- xorcist 3y ago
  The hoops people jump through in order to avoid using an operating system that just works in that setting never ceases to surprise me. It was something that started when Macs were popularized among developers and never really stopped.
  - stavros 3y ago
    What "operating system that just works" can I use to run today my application that I developed eight years ago?
  - pasc1878 3y ago
    Depends on what you mean. If you mean just use Linux then not possible. You need MS Excel for any financial work - the alternatives are just not complete enough, Mail also (well used to) needs the OS. And I suspect most people prefer the GUI of macOS and Windows which do just work. If you mean why isn't the app written to just work then that is often ignorance of what calls and things are Linux specific - a non GUIO business app should be easily portable;le between macOS and Linux and SunOS etc you don't need low level API calls for that.
rightbyte 3y ago
I don't see how replacing the "it works on my machine" with "it works in this VM" is much better. It is the same problem. You just added the overhead of running another whole computer in your computer to solve it. Also the author states he is locked into using VS Code, which seems like a risky dependency to have.
- stavros 3y ago
  > I don't see how replacing the "it works on my machine" with "it works in this VM" is much better. It is the same problem. I can ship the VM to a thousand people. I can't do the same with my machine.
  - rightbyte 3y ago
    You can ship a list of your dependencies too. In my experience, VM centric deployment processes lead to extremely brittle programs. But I mean, ye, if you got a brittle mess, you better use VMs for deployment. I am not against using that method, as long as I gotta admit I've made a mess.
    - lnenad 3y ago
      What are you talking about? A list of dependencies is less brittle than having a fully fledged env that is ready to go? In what world? Even if you use hard locked versions of deps there is a whole multiverse of problems you can encounter when compared to having everything ready to go that is identical to your expectations.
      - nerdponx 3y ago
        Because unless your entire environment set up procedure is reproducible from top to bottom, something can break along the way. Whereas if you are already shipping a VM (or a container image, which on Linux is much lighter than a VM) you are really only relying on the VM runtime, everything else will work automatically if the VM runtime is working. It's only one thing that can break, only one thing you need to figure out how to fix in order to get the system working, instead of 1000 unknown and unknowable things. And if you're using some kind of hosted solution, you're literally paying for a working VM runtime.
tiziano88 3y ago
This is a solved problem, and the solution is Nix, not Docker
- bomewish 3y ago
  Was looking for this one.
- charcircuit 3y ago
  Dev onboarding / dev containers are not a solved problem and Nix alone is insufficient for solving them. Nix also doesn't always work since you are going to be using versions of libraries from nixpkgs instead of the the OS you are targeting. Developers would test against different libraries than end users are using. With the a container or VM you can reproduce exactly the environment the software will run in.
  - sisve 3y ago
    I feel devbox[1] is really good here. And solves the painpoints your addressing. You do not really need to get into the details of nix but get the benefits. But it do not seems to have gotten the love I feel it deserves from the nix community. 1.https://www.jetpack.io/devbox/docs/quickstart/ https://www.jetpack.io/devbox/docs/quickstart/
mgaunard 3y ago
You should never run a command you don't understand. The whole premise of this article is shaky.
- fahhem 3y ago
  Which command do I not understand? Devpod.sh is open source, and while I'm not a master of Go personally I can tell it's basically running docker commands for me. Should I not run docker itself without reading the source code too?
  - mgaunard 3y ago
    You said dev onboarding was about blindly running a command someone else wrote.
    - fahhem 3y ago
      Yes, it is, but I don't think there's a productive world in which new engineers don't onboard by running commands blindly based on a doc somewhere. I just want a world where those commands reliably work with very few host dependencies. With devpod, I think we're close (only 1 dep that's statically built and docker engine)
      - mgaunard 3y ago
        Again, you shouldn't run commands you don't understand, onboarding or not. Understand what the command does and why you're told to run it before you run it.
        fahhem 3y ago
        What would you recommend onboarding to a new project entail? Studying thousands of lines of code before doing anything? Should they study containerd or just the docker CLI?
tamarlikesdata 3y ago
Has anyone faced any challenges integrating these tools with specific IDEs or version control systems? How did you address issues like plugin compatibility or build automation within these environments?
- fahhem 3y ago
  I avoid IDE specific workflows, that's why I'm using devpod. As a result, I can 'ssh' into the container from anywhere, or docker exec, and build automation just gets a 'docker exec' shoved in front of every command
k__ 3y ago
Containers: Nobody gets the installation right. We give up, let's ship everything! Nix/Guix: Why? We literally solved this issue 20 years ago.
- x86x87 3y ago
  My favorite quote is: containers and k8s are perfect for shipping poorly architected software that was built to yolo the dependencies and the os configuration it runs on. In the real world you're going to need to understand what your dependencies are (if only for understanding and closing security holes when they are discovered) and you will be in a world of pain when the yolo approach catches up with you. I'm not saying to not use containers btw. I think containers are great and can save time when use properly. They are just not the panacea people make them out to be.
- ForkMeOnTinder 3y ago
  With nix, you can't precisely specify versions of all your tooling. You have to choose a channel, and you get whatever that channel gives you. You can't upgrade one tool without upgrading everything else at the same time to whatever version the channel provides. For many dev workflows, that's not enough control. See https://github.com/NixOS/nixpkgs/issues/93327 https://github.com/NixOS/nixpkgs/issues/93327 (which grew from https://github.com/NixOS/nixpkgs/issues/9682 https://github.com/NixOS/nixpkgs/issues/9682)
  - earthling8118 3y ago
    That's not strictly true, particularly with flakes. You can easily pin multiple versions of nixpkgs as a flake input. Also it is possible to override the version of a specific derivation
bitwize 3y ago
Some painpoints wrt DevPod: In order to get devpod ssh to work, DevPod injects itself into your host machine (where all the devcontainers run). When you ssh into your devcontainer, devpod's ssh implementation acts as a proxy into the container, allowing you to get an in-container shell without having to run sshd inside the container. But people are already injecting VS Code into remote machines, so maybe this doesn't matter to you. DevPod's ssh implementation does not handle newlines properly; full-screen displays can get garbled on many terminals. But most DevPod users are using the terminal inside VS Code, which seems to work fine.
- fahhem 3y ago
  Looking at my ssh config, devpod injected a Host and that's it there's nothing extra in the container. Instead, there's a ProxyCommand that runs devpod, which probably ends up in a docker exec somewhere. That's why you're having trouble with newline and larger terminals, I used to as well because the terminal escape codes for resizing weren't getting through to the shell inside the container. In the past, I had to resize the terminal once manually after execing in, but I think either docker or bash or something in the chain got an update in the past few years that fixed it to automatically work
  - bitwize 3y ago
    Not in the container, in the host. If containers are running on your local PC, you'll be fine, but if you're logging in remotely DevPod will inject itself into your remote host. Do a 'ps -aux | less' while ssh'd in to your container sometime and be ready for a surprise.
    - fahhem 3y ago
      Ah, I never considered running it remotely. I'm not onboard the 'dev in the cloud' train, mainly because of round trip latency
      - bitwize 3y ago
        There's no round trip latency when you use VS Code's remote editing capability.
maxbrydak 3y ago
Tbh I've found nix and flakes as remarkable in terms of solving that issue.
miked85 3y ago
Kind of a long way to say that the author uses Docker for development environments.
- fahhem 3y ago
  Haha, yeah, but I've been using them for years and it's only now getting easier than writing really long scripts over and over again
bx376 3y ago
https://coder.com/ https://coder.com/
anotherhue 3y ago
I have found that a nix devshell (with direnv) to be the most miraculous solution to weird such as these. https://github.com/numtide/devshell https://github.com/numtide/devshell

42 points by fahhem 3y ago

The first issue about the machine being macOS or Windows but the code assuming Linux has never been a problem in my experience. Now containers or Nix do make it easier. If the system is deployed on Linux then you run Linux in a VM. (Which also deals with some of the other issues) I have done that since the early 2000 or even before we remotely logged in to a machine running the correct OS (X Windows is useful) albeit I was on site so a fast connection to the server. Now whether it is good design to make assumptions that only Linux matters is another question. I am old enough that I had to use multiple Unix systems SunOS Solaris NeXTStep and Linux. And good practice was to compile C++ with two different compilers as the error messages and edge cases were not very edge.

xorcist 3y ago
The hoops people jump through in order to avoid using an operating system that just works in that setting never ceases to surprise me. It was something that started when Macs were popularized among developers and never really stopped.
- stavros 3y ago
  What "operating system that just works" can I use to run today my application that I developed eight years ago?
- pasc1878 3y ago
  Depends on what you mean. If you mean just use Linux then not possible. You need MS Excel for any financial work - the alternatives are just not complete enough, Mail also (well used to) needs the OS. And I suspect most people prefer the GUI of macOS and Windows which do just work. If you mean why isn't the app written to just work then that is often ignorance of what calls and things are Linux specific - a non GUIO business app should be easily portable;le between macOS and Linux and SunOS etc you don't need low level API calls for that.

rightbyte 3y ago

I don't see how replacing the "it works on my machine" with "it works in this VM" is much better. It is the same problem. You just added the overhead of running another whole computer in your computer to solve it. Also the author states he is locked into using VS Code, which seems like a risky dependency to have.

stavros 3y ago
> I don't see how replacing the "it works on my machine" with "it works in this VM" is much better. It is the same problem. I can ship the VM to a thousand people. I can't do the same with my machine.
- rightbyte 3y ago
  You can ship a list of your dependencies too. In my experience, VM centric deployment processes lead to extremely brittle programs. But I mean, ye, if you got a brittle mess, you better use VMs for deployment. I am not against using that method, as long as I gotta admit I've made a mess.
  - lnenad 3y ago
    What are you talking about? A list of dependencies is less brittle than having a fully fledged env that is ready to go? In what world? Even if you use hard locked versions of deps there is a whole multiverse of problems you can encounter when compared to having everything ready to go that is identical to your expectations.
    - nerdponx 3y ago
      Because unless your entire environment set up procedure is reproducible from top to bottom, something can break along the way. Whereas if you are already shipping a VM (or a container image, which on Linux is much lighter than a VM) you are really only relying on the VM runtime, everything else will work automatically if the VM runtime is working. It's only one thing that can break, only one thing you need to figure out how to fix in order to get the system working, instead of 1000 unknown and unknowable things. And if you're using some kind of hosted solution, you're literally paying for a working VM runtime.

tiziano88 3y ago

This is a solved problem, and the solution is Nix, not Docker

bomewish 3y ago
Was looking for this one.
charcircuit 3y ago
Dev onboarding / dev containers are not a solved problem and Nix alone is insufficient for solving them. Nix also doesn't always work since you are going to be using versions of libraries from nixpkgs instead of the the OS you are targeting. Developers would test against different libraries than end users are using. With the a container or VM you can reproduce exactly the environment the software will run in.
- sisve 3y ago
  I feel devbox[1] is really good here. And solves the painpoints your addressing. You do not really need to get into the details of nix but get the benefits. But it do not seems to have gotten the love I feel it deserves from the nix community. 1.https://www.jetpack.io/devbox/docs/quickstart/ https://www.jetpack.io/devbox/docs/quickstart/

mgaunard 3y ago

You should never run a command you don't understand. The whole premise of this article is shaky.

fahhem 3y ago
Which command do I not understand? Devpod.sh is open source, and while I'm not a master of Go personally I can tell it's basically running docker commands for me. Should I not run docker itself without reading the source code too?
- mgaunard 3y ago
  You said dev onboarding was about blindly running a command someone else wrote.
  - fahhem 3y ago
    Yes, it is, but I don't think there's a productive world in which new engineers don't onboard by running commands blindly based on a doc somewhere. I just want a world where those commands reliably work with very few host dependencies. With devpod, I think we're close (only 1 dep that's statically built and docker engine)
    - mgaunard 3y ago
      Again, you shouldn't run commands you don't understand, onboarding or not. Understand what the command does and why you're told to run it before you run it.
      - fahhem 3y ago
        What would you recommend onboarding to a new project entail? Studying thousands of lines of code before doing anything? Should they study containerd or just the docker CLI?

tamarlikesdata 3y ago

Has anyone faced any challenges integrating these tools with specific IDEs or version control systems? How did you address issues like plugin compatibility or build automation within these environments?

fahhem 3y ago
I avoid IDE specific workflows, that's why I'm using devpod. As a result, I can 'ssh' into the container from anywhere, or docker exec, and build automation just gets a 'docker exec' shoved in front of every command

k__ 3y ago

Containers: Nobody gets the installation right. We give up, let's ship everything! Nix/Guix: Why? We literally solved this issue 20 years ago.

x86x87 3y ago
My favorite quote is: containers and k8s are perfect for shipping poorly architected software that was built to yolo the dependencies and the os configuration it runs on. In the real world you're going to need to understand what your dependencies are (if only for understanding and closing security holes when they are discovered) and you will be in a world of pain when the yolo approach catches up with you. I'm not saying to not use containers btw. I think containers are great and can save time when use properly. They are just not the panacea people make them out to be.
ForkMeOnTinder 3y ago
With nix, you can't precisely specify versions of all your tooling. You have to choose a channel, and you get whatever that channel gives you. You can't upgrade one tool without upgrading everything else at the same time to whatever version the channel provides. For many dev workflows, that's not enough control. See https://github.com/NixOS/nixpkgs/issues/93327 https://github.com/NixOS/nixpkgs/issues/93327 (which grew from https://github.com/NixOS/nixpkgs/issues/9682 https://github.com/NixOS/nixpkgs/issues/9682)
- earthling8118 3y ago
  That's not strictly true, particularly with flakes. You can easily pin multiple versions of nixpkgs as a flake input. Also it is possible to override the version of a specific derivation

bitwize 3y ago

Some painpoints wrt DevPod: In order to get devpod ssh to work, DevPod injects itself into your host machine (where all the devcontainers run). When you ssh into your devcontainer, devpod's ssh implementation acts as a proxy into the container, allowing you to get an in-container shell without having to run sshd inside the container. But people are already injecting VS Code into remote machines, so maybe this doesn't matter to you. DevPod's ssh implementation does not handle newlines properly; full-screen displays can get garbled on many terminals. But most DevPod users are using the terminal inside VS Code, which seems to work fine.

fahhem 3y ago
Looking at my ssh config, devpod injected a Host and that's it there's nothing extra in the container. Instead, there's a ProxyCommand that runs devpod, which probably ends up in a docker exec somewhere. That's why you're having trouble with newline and larger terminals, I used to as well because the terminal escape codes for resizing weren't getting through to the shell inside the container. In the past, I had to resize the terminal once manually after execing in, but I think either docker or bash or something in the chain got an update in the past few years that fixed it to automatically work
- bitwize 3y ago
  Not in the container, in the host. If containers are running on your local PC, you'll be fine, but if you're logging in remotely DevPod will inject itself into your remote host. Do a 'ps -aux | less' while ssh'd in to your container sometime and be ready for a surprise.
  - fahhem 3y ago
    Ah, I never considered running it remotely. I'm not onboard the 'dev in the cloud' train, mainly because of round trip latency
    - bitwize 3y ago
      There's no round trip latency when you use VS Code's remote editing capability.

maxbrydak 3y ago

Tbh I've found nix and flakes as remarkable in terms of solving that issue.

miked85 3y ago

Kind of a long way to say that the author uses Docker for development environments.

fahhem 3y ago
Haha, yeah, but I've been using them for years and it's only now getting easier than writing really long scripts over and over again

bx376 3y ago

https://coder.com/ https://coder.com/

anotherhue 3y ago

I have found that a nix devshell (with direnv) to be the most miraculous solution to weird such as these. https://github.com/numtide/devshell https://github.com/numtide/devshell