11 ms·

Programming Inside a Container

173 points by nkurz 6y ago

0xcoffee 6y ago
I program this way also. My preferred method is simply to download the VSCode Remote Extension Pack: https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack https://marketplace.visualstudio.com/items?itemName=ms-vscod... Then in a project, simply `ctrl+shift+p` -> Add container configuration files. Then `ctrl+shift+p` -> Rebuild Container Couldn't be easier. The only downsides are lack of GUI, often in python it's nice to just do e.g. plot.show() rather then export to a file and view that file. But for most of my non-visual programming work, a docker is amazing. Once it works for me, I can guarantee it will work for all other developers, and more importantly, my dev environment is really similar to my prod environment.
- fluffything 6y ago
  You can run a, e.g., jupyter notebook inside the container, and connect to it with your web-browser.
  - screye 6y ago
    Yep, that's how I develop. Remote machine + port forwarding + jupyter notebooks / python files. The productionizing happens on my local VScode, which is a lot more "decked out".
    - chrisweekly 6y ago
      Any chance you have devnotes / gists / blog post to share more dets abt your setup?
- mbauman 6y ago
  I've found https://github.com/novnc/noVNC https://github.com/novnc/noVNC useful when I want to run something with a GUI in a container. Spin it up, forward the port, connect via browser.
- lioeters 6y ago
  I've started doing this recently, using VS Code's Remote SSH to develop projects in a virtual machine (or actually remote). It's so comfortable now, with the advantages of containers I can spin up and destroy, ensuring consistent environments. For lack of GUI, I first tried VNC and X Window client/server, but a bit awkward. As a sibling comment mentioned, I find it simpler to put together and serve a "web frontend" from the container.
  - alfiedotwtf 6y ago
    > ensuring consistent environments This. I really don’t understand why all the hate with working in containers. If you can bring up and tear down quickly, it’s almost free like branching in git. It’s consistency by design!
- deleted 6y ago
  [deleted]
- zarkov99 6y ago
  I do think this is the future of development and all IDEs will need to support this use case very well. BTW, non reason to not have a GUI just start docker with: -v /tmp:/tmp -e DISPLAY=$DISPLAY And you should be able to launch X applications inside docker just fine. I run graphical emacs inside docker everyday.
- madduci 6y ago
  You can use xvfb And use your Xauth in the container (assuming your host is Linux)
cblackthornekc 6y ago
We do this very thing at my work. When you have a million and one guidelines around what can and can't be installed on a device. Especially on one that has access to the internet, it is easy to say here is this container that has no access to the internet. That seems to make infosec happy. Also, makes local testing difficult if you need access to a web based API. The other issue you do have is the issue of configuring them for each application, but luckily most teams have had at least one or two people take up the role of maintaining the images and the startup scripts.
- GordonS 6y ago
  On one of my Windows work laptops, the restrictions placed on devs are ridiculous, they won't even grant admin rights. Instead they have some bug-ridden software installed that let's you elevate specific apps. You need to request access to those apps, and your request goes through a weeks or even months long process before you get a response. If granted, you need to enter your username and password for each elevation, every single time. This machine has hyper-v installed, so I created a Windows 10 VM and worked in that fullscreen, permanently. Gods but I hate working for this mega corp!
mailslot 6y ago
I’ve worked with a few developers that have been adamant about developing inside of containers. What I’ve noticed: Terrible performance. One of my engineers recently thought that 150ms was terrific for a HTTP request. Break out of the container and it was <10ms. YMMV. Fragile everything: Because one expects a “pristine” environment, often any slight change causes the entire stack to fall apart. This doesn’t happen at the start, but creeps in over time, until you can’t even update base images. I’ve seen it a lot. It ends up only adding an additional layer of complication. Etc. There are definitely reasons to do this... But when a pedantic developer that needs everything to be “just right” does it, it often becomes a disaster, leading to shortcuts and a lack of adaptability. There’s also the developer that has no idea WTF is going on. They use a standard Rails/PHP/NodeJS/etc container and don’t understand how it works. Sometimes, they don’t even know that their system can run their stack natively. I’ve been on teams that have said “Let’s just use Docker because X doesn’t know how to install Y.” Docker is fantastic for many things, but let’s stop throwing it at everything.
- paloaltokid 6y ago
  This feels like very much a YMMV situation. I think my own personal thinking is mostly the same as yours. But for the OP it may be perfect. On the blog he indicates that he's a CS professor. I could imagine that in a research environment maybe he gets better mileage out of this than someone coding in a for-money work environment.
  - throwaway_pdp09 6y ago
    Dan Lemire is a prof but also a very hands-dirty type. He measures stuff down to CPU IPC levels. Check other stuff on his blog, he's not an ivory tower type[0] [0] Not that there's anything wrong with that at all, it's just a different kind of person with different strengths, but DL is not one.
    - paloaltokid 6y ago
      Ah, that makes sense. It sounds like he balances being academic with trying to work through things in a real-world way.
latortuga 6y ago
It's handy to be able to do this with things you already know you want to be able to do inside your container. The downside for me though, has been having to throw away all the careful tweaks I have made to my development environment over the years. The muscle memory of my long list of .bash_aliases. My custom git config with shortcuts. My scripts in ~/bin. Hell even installed software that doesn't fit in with the purpose of the project, think things like `jq`. Also worth noting that the "permissions issues" he mentions are handled automatically by Docker for Mac. I had the exact problem of files created inside the container being owned by root on Linux, and all my mac-using colleagues just stared at me like I had horns on my head. "It works on my machine" still exists, even with Docker.
- heavenlyblue 6y ago
  You could just remap the uids in Docker, even on Linux.
  - sp332 6y ago
    I thought Docker only recently supported user namespaces.
- casept 6y ago
  This is why I prefer nix over docker for dev environments. I can just start a nix-shell and have all of my regular tooling available in addition to the environment-specific packages.
- TechBro8615 6y ago
  These are all things that your teammates do not have on their machine, so IMO it's good that docker forces you to put all the tools you use into one place.
kitotik 6y ago
This is not a good experience. I’ve tried it a few times because it sounds great on paper: declarative, portable, consistent dev environment! Who doesn’t want that? In practice it’s slow, laggy, and a maintenance time sink. I haven’t tried it yet, but something like NixOS seems far better suited to this problem than containers.
- takeda 6y ago
  > I haven’t tried it yet, but something like NixOS seems far better suited to this problem than containers. You don't need to go whole NixOS to get those benefits. Installing Nix on your OS can also provide it. Although experience is better on NixOS.
z3t4 6y ago
You could probably do fine with just chroot. Docker is overkill for most use cases, but one advantage is that it works on Windows and Mac too. Using the same uid for your user on both the host and the container will likely fix the weird user issue. One thing I miss when using Docker is the pipe command in unix like shells.
- capdeck 6y ago
  And if you need a step up from chroot, systemd-nspawn is a good next step. Just as light, but supports many more isolated features including all kinds of networking. No need for go runtime as with docker, the same convenience of use via machinectl command.
l31g 6y ago
I have done this from time to time when debugging applications that run inside Docker. I'd bash into the container as root and then download and install my development tools and check out the repo where the code lives. Once the code is functionally correct, then I can worry about performance optimizations given Docker...
deleted 6y ago
[deleted]
silviogutierrez 6y ago
It's a great idea. But on non-native Docker, like OSX, it's simply too slow. Even when it's acceptable speeds, I find my computer running at 100% CPU usage with the fan out of control. Nix is a similar idea and makes the environmental identical for all developers. Makes on-boarding trivial. And speeds are "native" on all platform. As a nice side effect: it normalizes all tools like sed, grep and find. With that said, as a consumer of a Nix setup, it's about as easy if not an easier experience than using Docker[1]. But properly learning to create your own Nix packages, etc, involves a very, very steep learning curve. Far higher than learning the basics of a Dockerfile. [1] Catalina issues notwithstanding, though there is a 4 command solution now: https://github.com/NixOS/nix/issues/2925 https://github.com/NixOS/nix/issues/2925
- zapita 6y ago
  Anecdotically, I develop inside a container using Docker for Mac, and don't have any performance issues. I think it depends on what exactlty you're developing, in particular filesystem access patterns.
  - silviogutierrez 6y ago
    Definitely comes down to file access, above all.
- zokier 6y ago
  But you don't really get "identical" without running VM; with your setup the software still runs on Darwin which is pretty different from Linux
  - CraftThatBlock 6y ago
    Docker on macOS runs inside a Linux VM, so yes it's functionally identical. The performance issues arise from FS mostly due to macOS (see post at top of front-page earlier)
- bdcravens 6y ago
  You can mitigate MacOS Docker issues using docker-sync, and if it's an application with a lot of file churn while it runs, it may make sense to add to paths to your .dockerignore
gorgoiler 6y ago
LXC (not LXD) is an utter delight compared to the competition. If you have IPv6 it works without any hacks and it’s like having a whole datacenter all controllable at the command line. Anything that inserts itself into iptables feels like a no no. That’s meant I’ve not really put much effort into LXD or Docker beyond discovering they are kind of heavy. The latter had poor IPv6 support the last time I tried to use it (9 months ago.). It’s there, but it felt like a second class citizen. LXD just feels like Ubuntu to LXC’s Debian, so I also didn’t play with it beyond the initial few hours. LXC itself is a joy. I run Alpine, Debian, and Ubuntu depending on my needs. Everything is disposable, with actual data either in github or a filer. I don’t even bother changing the hostname on the VPSs I provision your use it. Boot, install a firewall, create containers, and forget the original host OS even exists. I’d really recommend getting to grips with the low lever (non-LXD) LXC stuff, especially when you are one “vagrant up” away from having LXC tools on your non-Linux OS!
- dahfizz 6y ago
  I totally agree. People have it stuck in their heads that container == cgroup, and they don't realize how much cruft e.g. docker layers on top. Anything that requires you to do networking on your CPU absolutely kills performance.
- inetknght 6y ago
  > Anything that inserts itself into iptables feels like a no no. I have a setup which works great with VirtualBox. I have pfSense installed in a guest VM and the host machine routes through it -- without that guest VM running the host machine can't connect anywhere. It's really handy to have a consistent firewall interface despite every host OS having a different idea of what a firewall should do or look like. Docker works with the VirtualBox setup too. I tried to do the same thing with libvirt for four weekends and eventually gave up. I couldn't get libvirt to play nice with iptables at all. LXC is for containers, of course, instead of VMs. But since Docker works with the VM-guest-as-a-router setup, perhaps I'll really give LXC a try too.
- Arkanosis 6y ago
  LXC has been a life changer for me. Making stuff compatible with multiple distros, making sure the dependency list is complete and experimenting something very quickly without taking the risk to compromise my main system are solved problems, now. Just install the distro with the most recent kernel on the host (you do already if you use arch) and spawn your Debians, Ubuntus, CentOSes in seconds…
Marazan 6y ago
After coming to a shuddering halt trying to get a build env setup for a trivial python script (poetry couldn't install the requirements because pip.... SSL... Blah blah blah) I just used our image used for the drone step to run and test it. It was such a relief
- 0xcoffee 6y ago
  Python was also what prompted me to start docker. Different packages being installed on Windows vs Linux. Different packages being installed pip vs (some other package manager) Simple user mistakes in the requirements file (not strict version). Going to docker fixed #1 and #2, and the constant rebuilding of the environment meant we quickly identified issues in requirement files. Working in docker is the equivalent of fail-fast programming imo, its just applied to the environment.
  - TechBro8615 6y ago
    > Python was also what prompted me to start docker. Indeed. Nothing will drive me to containers on a new project faster than trying to get Python working on my mac. Funnily enough, Docker can trace its roots back to frustrations with Python packaging (dotcloud was originally an easy way to deploy Python apps).
dirtydroog 6y ago
I am the opposite. I develop on Linux via VMWare Player running on a Windows host. I don't want Linux as the main OS on my personal desktop and I can keep work related stuff on its own VM. I've never noticed any non-bare metal speed issues. As for distro versioning... surely you only get the user-land package versions, and the kernel is still the version of your main OS?
- milesvp 6y ago
  When I left my previous job, I was starting to think it might be a good idea to start using docker containers for dev. We were using chef kitchen to maintain virtual machines, but we had just enough production systems, that depending on what you were working on, you might need 3 virtual machines running at once. I didn't care on my desktop with 16GB of ram, but we had a number of devs using a laptop with only 8GB, and 3 virtual machines was just enough to cause headaches. It was the first use case for docker that I thought might make sense. The second was the headache we were likely to start running into on our CI server when dealing with different versions of Node/NPM for different codebases, a drift that is inevitable as older codebases get less support.
3fe9a03ccd14ca5 6y ago
> The idea of a container approach is to always start from a pristine state. So you define the configuration that your database server needs to have, and you launch it, in this precise state each time. This makes your infrastructure predictable. I’ve tried this before, but there’s still a lot of overhead maintained the pristine state. For example troubleshooting why a python package won’t run, you end up I installing and upgrading a lot of other packages. You’re not sure if that helped or it was something else — now what? You’ll spend time wondering if you want to carry your changes over or deal with the drift.
perlgeek 6y ago
Regarding the fiddly permissions between host and container, at least on Linux podman seems to be the solution. It is command line compatible to docker (and uses the same image format), but instead of launching containers through a demon, it launches container directly as the calling user (needs user namespaces, Linux 3.8+). Now you can share volumes between host and container, and don't run into those pesky permission problems that come from different UIDs on the inside and outside. Very useful in CI contexts, for development environment and so on.
erik_seaberg 6y ago
> The image will only contain the compiler and build tools. There is no reason to change any of these tools. Smacks of "you're holding it wrong" thinking about consumers. Professionals should sharpen the saw by continually customizing their environment.
- zarkov99 6y ago
  Sure, and now they can keep track of those customization in a reproducible way, by keeping their docker file under source control. What is the problem?
  - erik_seaberg 6y ago
    I probably edit ~/.bash_profile every couple of days, but reimaging my machine is a big deal. Code can live in a git remote, but what happens to my homedir and notes and ad hoc analyses if I recreate the container that often?
    - zarkov99 6y ago
      Nothing. None of those things would be kept in the container. The only thing you would keep in your container would be the tools and libraries you need to build your project. You mount your home file system inside the container and just use it to build/debug. Everything else you keep outside.
anotherevan 6y ago
I gotta admit, I was expecting a picture of a guy in a shipping container with a laptop.
darkteflon 6y ago
> These glitches come from the strange way in which Docker deals with permissions and security. Contrary that what you mean read, it is not a simple matter of setting user and group identifiers: it may be sufficient on some systems but not on systems supporting Security-Enhanced Linux which require additional care. Does anyone have any general advice on the best way to deal with Docker file permissions issues on recent Ubuntu LTS distros? As a new Docker user this has proven surprisingly intractable. As mentioned in the quoted text, simply setting UID / GUID to the same doesn’t seem to do the trick most of the time. I see podman has been mentioned in this thread but is there a native, no-dependency, no-new-tool way to handle this? I feel like I must be missing something simple. Grateful to hear anyone’s experience.
- zarkov99 6y ago
  I do not use SEL, but on regular Ubuntu the below works perfectly with respect to permissions: #!/bin/bash docker run \ --rm \ --net host \ -i \ -v /etc/passwd:/etc/passwd \ -v /etc/group:/etc/group \ -v /usr/local/share/fonts:/usr/local/share/fonts \ -v /usr/share/fonts:/usr/share/fonts \ -v /run/user/:/run/user/ \ -v /tmp:/tmp \ -v /home:/home \ -u $UID \ -e DISPLAY=$DISPLAY \ -t container \ /bin/bash
Narigo 6y ago
I like the sandboxing effect of using Docker. Having node_modules (or any kind of dependencies) not being able to access all files on the host is a big relief - especially since there were attacks on popular dependencies already, stealing wallets or similar.
overgard 6y ago
I think the challenge with this is most our workstations are either macs or windows, in which case your containers are running in a VM anyway (so, sluggish). You only really get native performance if youre running it in linux, which is fine but.. the year of the linux desktop has not quite arrived yet.
- jfkebwjsbx 6y ago
  You need better VM software then. A properly configured VM has near native performance nowadays.
wicket 6y ago
And now we've gone full circle. One of the proposed use cases for containers when Solaris Containers came about (and probably also FreeBSD Jails before it) was that you could easily give a software developer a full OS instance to play about and break stuff or to do whatever they desire, instead of restricting them to the confinement of their home directory.
benoror 6y ago
I recently came across dip, which implements a similar approach: https://github.com/bibendi/dip https://github.com/bibendi/dip
madduci 6y ago
If you use Visual Studio Code as your daily driver editor, you can use the remote environments with SSH or Docker. I made a couple of Docker images to work with the official VS Code plugin to develop for C++, Rust, Go, Java, PHP and Python https://github.com/madduci/remote-environments https://github.com/madduci/remote-environments
- panpanna 6y ago
  I just wish vscode also supported LXC containers...
  - madduci 6y ago
    Indeed, but I've found out that only VSCode can handle remote environments, VSCodium (the stripped version of Code) doesn't work
peterwwillis 6y ago
The script and Dockerfile I use are pretty long, but they set up everything needed to map in the most common stuff like ssh and aws creds, ptrace, sudo support, uid/gid runtime mapping, etc. Once you start developing your shared repos from a container, you realize that it's much easier to automate running things from it than to develop inside of it, so it's actually easier to get people to build CI/CD pipelines now where before they'd wait for someone else to do it for them. And the only problem there is there's no easy + good + free self-hosted CI/CD software out there. Yes, you probably use X just fine, but X probably doesn't scale, or isn't enterprise compatible, etc. The biggest barrier to automation is both a business problem and a technical problem.
p2t2p 6y ago
Developing a Java app in devcontainer on Linux using VSCode. I absolutely love it. I need to use an SDK that does ugly things to your local maven installation so I put everything in container, VSCode start it automatically and embedded terminal is opened inside that container, the only difference I notices that VSCode is not Idea.
zubairq 6y ago
If you want to program inside a container you could use something like yazz pilot which is designed to do exactly that https://github.com/zubairq/pilot https://github.com/zubairq/pilot
bJGVygG7MQVF8c 6y ago
Developing inside a container isn't worth the friction to me. I know my environment well, and I'm invested in continuing to maintain that knowledge.

173 points by nkurz 6y ago

I program this way also. My preferred method is simply to download the VSCode Remote Extension Pack: https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack https://marketplace.visualstudio.com/items?itemName=ms-vscod... Then in a project, simply `ctrl+shift+p` -> Add container configuration files. Then `ctrl+shift+p` -> Rebuild Container Couldn't be easier. The only downsides are lack of GUI, often in python it's nice to just do e.g. plot.show() rather then export to a file and view that file. But for most of my non-visual programming work, a docker is amazing. Once it works for me, I can guarantee it will work for all other developers, and more importantly, my dev environment is really similar to my prod environment.

fluffything 6y ago
You can run a, e.g., jupyter notebook inside the container, and connect to it with your web-browser.
- screye 6y ago
  Yep, that's how I develop. Remote machine + port forwarding + jupyter notebooks / python files. The productionizing happens on my local VScode, which is a lot more "decked out".
  - chrisweekly 6y ago
    Any chance you have devnotes / gists / blog post to share more dets abt your setup?
mbauman 6y ago
I've found https://github.com/novnc/noVNC https://github.com/novnc/noVNC useful when I want to run something with a GUI in a container. Spin it up, forward the port, connect via browser.
lioeters 6y ago
I've started doing this recently, using VS Code's Remote SSH to develop projects in a virtual machine (or actually remote). It's so comfortable now, with the advantages of containers I can spin up and destroy, ensuring consistent environments. For lack of GUI, I first tried VNC and X Window client/server, but a bit awkward. As a sibling comment mentioned, I find it simpler to put together and serve a "web frontend" from the container.
- alfiedotwtf 6y ago
  > ensuring consistent environments This. I really don’t understand why all the hate with working in containers. If you can bring up and tear down quickly, it’s almost free like branching in git. It’s consistency by design!
deleted 6y ago
[deleted]
zarkov99 6y ago
I do think this is the future of development and all IDEs will need to support this use case very well. BTW, non reason to not have a GUI just start docker with: -v /tmp:/tmp -e DISPLAY=$DISPLAY And you should be able to launch X applications inside docker just fine. I run graphical emacs inside docker everyday.
madduci 6y ago
You can use xvfb And use your Xauth in the container (assuming your host is Linux)

cblackthornekc 6y ago

We do this very thing at my work. When you have a million and one guidelines around what can and can't be installed on a device. Especially on one that has access to the internet, it is easy to say here is this container that has no access to the internet. That seems to make infosec happy. Also, makes local testing difficult if you need access to a web based API. The other issue you do have is the issue of configuring them for each application, but luckily most teams have had at least one or two people take up the role of maintaining the images and the startup scripts.

GordonS 6y ago
On one of my Windows work laptops, the restrictions placed on devs are ridiculous, they won't even grant admin rights. Instead they have some bug-ridden software installed that let's you elevate specific apps. You need to request access to those apps, and your request goes through a weeks or even months long process before you get a response. If granted, you need to enter your username and password for each elevation, every single time. This machine has hyper-v installed, so I created a Windows 10 VM and worked in that fullscreen, permanently. Gods but I hate working for this mega corp!

mailslot 6y ago

I’ve worked with a few developers that have been adamant about developing inside of containers. What I’ve noticed: Terrible performance. One of my engineers recently thought that 150ms was terrific for a HTTP request. Break out of the container and it was <10ms. YMMV. Fragile everything: Because one expects a “pristine” environment, often any slight change causes the entire stack to fall apart. This doesn’t happen at the start, but creeps in over time, until you can’t even update base images. I’ve seen it a lot. It ends up only adding an additional layer of complication. Etc. There are definitely reasons to do this... But when a pedantic developer that needs everything to be “just right” does it, it often becomes a disaster, leading to shortcuts and a lack of adaptability. There’s also the developer that has no idea WTF is going on. They use a standard Rails/PHP/NodeJS/etc container and don’t understand how it works. Sometimes, they don’t even know that their system can run their stack natively. I’ve been on teams that have said “Let’s just use Docker because X doesn’t know how to install Y.” Docker is fantastic for many things, but let’s stop throwing it at everything.

paloaltokid 6y ago
This feels like very much a YMMV situation. I think my own personal thinking is mostly the same as yours. But for the OP it may be perfect. On the blog he indicates that he's a CS professor. I could imagine that in a research environment maybe he gets better mileage out of this than someone coding in a for-money work environment.
- throwaway_pdp09 6y ago
  Dan Lemire is a prof but also a very hands-dirty type. He measures stuff down to CPU IPC levels. Check other stuff on his blog, he's not an ivory tower type[0] [0] Not that there's anything wrong with that at all, it's just a different kind of person with different strengths, but DL is not one.
  - paloaltokid 6y ago
    Ah, that makes sense. It sounds like he balances being academic with trying to work through things in a real-world way.

latortuga 6y ago

It's handy to be able to do this with things you already know you want to be able to do inside your container. The downside for me though, has been having to throw away all the careful tweaks I have made to my development environment over the years. The muscle memory of my long list of .bash_aliases. My custom git config with shortcuts. My scripts in ~/bin. Hell even installed software that doesn't fit in with the purpose of the project, think things like `jq`. Also worth noting that the "permissions issues" he mentions are handled automatically by Docker for Mac. I had the exact problem of files created inside the container being owned by root on Linux, and all my mac-using colleagues just stared at me like I had horns on my head. "It works on my machine" still exists, even with Docker.

heavenlyblue 6y ago
You could just remap the uids in Docker, even on Linux.
- sp332 6y ago
  I thought Docker only recently supported user namespaces.
casept 6y ago
This is why I prefer nix over docker for dev environments. I can just start a nix-shell and have all of my regular tooling available in addition to the environment-specific packages.
TechBro8615 6y ago
These are all things that your teammates do not have on their machine, so IMO it's good that docker forces you to put all the tools you use into one place.

kitotik 6y ago

This is not a good experience. I’ve tried it a few times because it sounds great on paper: declarative, portable, consistent dev environment! Who doesn’t want that? In practice it’s slow, laggy, and a maintenance time sink. I haven’t tried it yet, but something like NixOS seems far better suited to this problem than containers.

takeda 6y ago
> I haven’t tried it yet, but something like NixOS seems far better suited to this problem than containers. You don't need to go whole NixOS to get those benefits. Installing Nix on your OS can also provide it. Although experience is better on NixOS.

z3t4 6y ago

You could probably do fine with just chroot. Docker is overkill for most use cases, but one advantage is that it works on Windows and Mac too. Using the same uid for your user on both the host and the container will likely fix the weird user issue. One thing I miss when using Docker is the pipe command in unix like shells.

capdeck 6y ago
And if you need a step up from chroot, systemd-nspawn is a good next step. Just as light, but supports many more isolated features including all kinds of networking. No need for go runtime as with docker, the same convenience of use via machinectl command.

l31g 6y ago

I have done this from time to time when debugging applications that run inside Docker. I'd bash into the container as root and then download and install my development tools and check out the repo where the code lives. Once the code is functionally correct, then I can worry about performance optimizations given Docker...

deleted 6y ago

[deleted]

silviogutierrez 6y ago

It's a great idea. But on non-native Docker, like OSX, it's simply too slow. Even when it's acceptable speeds, I find my computer running at 100% CPU usage with the fan out of control. Nix is a similar idea and makes the environmental identical for all developers. Makes on-boarding trivial. And speeds are "native" on all platform. As a nice side effect: it normalizes all tools like sed, grep and find. With that said, as a consumer of a Nix setup, it's about as easy if not an easier experience than using Docker[1]. But properly learning to create your own Nix packages, etc, involves a very, very steep learning curve. Far higher than learning the basics of a Dockerfile. [1] Catalina issues notwithstanding, though there is a 4 command solution now: https://github.com/NixOS/nix/issues/2925 https://github.com/NixOS/nix/issues/2925

zapita 6y ago
Anecdotically, I develop inside a container using Docker for Mac, and don't have any performance issues. I think it depends on what exactlty you're developing, in particular filesystem access patterns.
- silviogutierrez 6y ago
  Definitely comes down to file access, above all.
zokier 6y ago
But you don't really get "identical" without running VM; with your setup the software still runs on Darwin which is pretty different from Linux
- CraftThatBlock 6y ago
  Docker on macOS runs inside a Linux VM, so yes it's functionally identical. The performance issues arise from FS mostly due to macOS (see post at top of front-page earlier)
bdcravens 6y ago
You can mitigate MacOS Docker issues using docker-sync, and if it's an application with a lot of file churn while it runs, it may make sense to add to paths to your .dockerignore

gorgoiler 6y ago

LXC (not LXD) is an utter delight compared to the competition. If you have IPv6 it works without any hacks and it’s like having a whole datacenter all controllable at the command line. Anything that inserts itself into iptables feels like a no no. That’s meant I’ve not really put much effort into LXD or Docker beyond discovering they are kind of heavy. The latter had poor IPv6 support the last time I tried to use it (9 months ago.). It’s there, but it felt like a second class citizen. LXD just feels like Ubuntu to LXC’s Debian, so I also didn’t play with it beyond the initial few hours. LXC itself is a joy. I run Alpine, Debian, and Ubuntu depending on my needs. Everything is disposable, with actual data either in github or a filer. I don’t even bother changing the hostname on the VPSs I provision your use it. Boot, install a firewall, create containers, and forget the original host OS even exists. I’d really recommend getting to grips with the low lever (non-LXD) LXC stuff, especially when you are one “vagrant up” away from having LXC tools on your non-Linux OS!

dahfizz 6y ago
I totally agree. People have it stuck in their heads that container == cgroup, and they don't realize how much cruft e.g. docker layers on top. Anything that requires you to do networking on your CPU absolutely kills performance.
inetknght 6y ago
> Anything that inserts itself into iptables feels like a no no. I have a setup which works great with VirtualBox. I have pfSense installed in a guest VM and the host machine routes through it -- without that guest VM running the host machine can't connect anywhere. It's really handy to have a consistent firewall interface despite every host OS having a different idea of what a firewall should do or look like. Docker works with the VirtualBox setup too. I tried to do the same thing with libvirt for four weekends and eventually gave up. I couldn't get libvirt to play nice with iptables at all. LXC is for containers, of course, instead of VMs. But since Docker works with the VM-guest-as-a-router setup, perhaps I'll really give LXC a try too.
Arkanosis 6y ago
LXC has been a life changer for me. Making stuff compatible with multiple distros, making sure the dependency list is complete and experimenting something very quickly without taking the risk to compromise my main system are solved problems, now. Just install the distro with the most recent kernel on the host (you do already if you use arch) and spawn your Debians, Ubuntus, CentOSes in seconds…

Marazan 6y ago

After coming to a shuddering halt trying to get a build env setup for a trivial python script (poetry couldn't install the requirements because pip.... SSL... Blah blah blah) I just used our image used for the drone step to run and test it. It was such a relief

0xcoffee 6y ago
Python was also what prompted me to start docker. Different packages being installed on Windows vs Linux. Different packages being installed pip vs (some other package manager) Simple user mistakes in the requirements file (not strict version). Going to docker fixed #1 and #2, and the constant rebuilding of the environment meant we quickly identified issues in requirement files. Working in docker is the equivalent of fail-fast programming imo, its just applied to the environment.
- TechBro8615 6y ago
  > Python was also what prompted me to start docker. Indeed. Nothing will drive me to containers on a new project faster than trying to get Python working on my mac. Funnily enough, Docker can trace its roots back to frustrations with Python packaging (dotcloud was originally an easy way to deploy Python apps).

dirtydroog 6y ago

I am the opposite. I develop on Linux via VMWare Player running on a Windows host. I don't want Linux as the main OS on my personal desktop and I can keep work related stuff on its own VM. I've never noticed any non-bare metal speed issues. As for distro versioning... surely you only get the user-land package versions, and the kernel is still the version of your main OS?

milesvp 6y ago
When I left my previous job, I was starting to think it might be a good idea to start using docker containers for dev. We were using chef kitchen to maintain virtual machines, but we had just enough production systems, that depending on what you were working on, you might need 3 virtual machines running at once. I didn't care on my desktop with 16GB of ram, but we had a number of devs using a laptop with only 8GB, and 3 virtual machines was just enough to cause headaches. It was the first use case for docker that I thought might make sense. The second was the headache we were likely to start running into on our CI server when dealing with different versions of Node/NPM for different codebases, a drift that is inevitable as older codebases get less support.

3fe9a03ccd14ca5 6y ago

> The idea of a container approach is to always start from a pristine state. So you define the configuration that your database server needs to have, and you launch it, in this precise state each time. This makes your infrastructure predictable. I’ve tried this before, but there’s still a lot of overhead maintained the pristine state. For example troubleshooting why a python package won’t run, you end up I installing and upgrading a lot of other packages. You’re not sure if that helped or it was something else — now what? You’ll spend time wondering if you want to carry your changes over or deal with the drift.

perlgeek 6y ago

Regarding the fiddly permissions between host and container, at least on Linux podman seems to be the solution. It is command line compatible to docker (and uses the same image format), but instead of launching containers through a demon, it launches container directly as the calling user (needs user namespaces, Linux 3.8+). Now you can share volumes between host and container, and don't run into those pesky permission problems that come from different UIDs on the inside and outside. Very useful in CI contexts, for development environment and so on.

erik_seaberg 6y ago

> The image will only contain the compiler and build tools. There is no reason to change any of these tools. Smacks of "you're holding it wrong" thinking about consumers. Professionals should sharpen the saw by continually customizing their environment.

zarkov99 6y ago
Sure, and now they can keep track of those customization in a reproducible way, by keeping their docker file under source control. What is the problem?
- erik_seaberg 6y ago
  I probably edit ~/.bash_profile every couple of days, but reimaging my machine is a big deal. Code can live in a git remote, but what happens to my homedir and notes and ad hoc analyses if I recreate the container that often?
  - zarkov99 6y ago
    Nothing. None of those things would be kept in the container. The only thing you would keep in your container would be the tools and libraries you need to build your project. You mount your home file system inside the container and just use it to build/debug. Everything else you keep outside.

anotherevan 6y ago

I gotta admit, I was expecting a picture of a guy in a shipping container with a laptop.

darkteflon 6y ago

> These glitches come from the strange way in which Docker deals with permissions and security. Contrary that what you mean read, it is not a simple matter of setting user and group identifiers: it may be sufficient on some systems but not on systems supporting Security-Enhanced Linux which require additional care. Does anyone have any general advice on the best way to deal with Docker file permissions issues on recent Ubuntu LTS distros? As a new Docker user this has proven surprisingly intractable. As mentioned in the quoted text, simply setting UID / GUID to the same doesn’t seem to do the trick most of the time. I see podman has been mentioned in this thread but is there a native, no-dependency, no-new-tool way to handle this? I feel like I must be missing something simple. Grateful to hear anyone’s experience.

zarkov99 6y ago
I do not use SEL, but on regular Ubuntu the below works perfectly with respect to permissions: #!/bin/bash docker run \ --rm \ --net host \ -i \ -v /etc/passwd:/etc/passwd \ -v /etc/group:/etc/group \ -v /usr/local/share/fonts:/usr/local/share/fonts \ -v /usr/share/fonts:/usr/share/fonts \ -v /run/user/:/run/user/ \ -v /tmp:/tmp \ -v /home:/home \ -u $UID \ -e DISPLAY=$DISPLAY \ -t container \ /bin/bash

Narigo 6y ago

I like the sandboxing effect of using Docker. Having node_modules (or any kind of dependencies) not being able to access all files on the host is a big relief - especially since there were attacks on popular dependencies already, stealing wallets or similar.

overgard 6y ago

I think the challenge with this is most our workstations are either macs or windows, in which case your containers are running in a VM anyway (so, sluggish). You only really get native performance if youre running it in linux, which is fine but.. the year of the linux desktop has not quite arrived yet.

jfkebwjsbx 6y ago
You need better VM software then. A properly configured VM has near native performance nowadays.

wicket 6y ago

And now we've gone full circle. One of the proposed use cases for containers when Solaris Containers came about (and probably also FreeBSD Jails before it) was that you could easily give a software developer a full OS instance to play about and break stuff or to do whatever they desire, instead of restricting them to the confinement of their home directory.

benoror 6y ago

I recently came across dip, which implements a similar approach: https://github.com/bibendi/dip https://github.com/bibendi/dip

madduci 6y ago

If you use Visual Studio Code as your daily driver editor, you can use the remote environments with SSH or Docker. I made a couple of Docker images to work with the official VS Code plugin to develop for C++, Rust, Go, Java, PHP and Python https://github.com/madduci/remote-environments https://github.com/madduci/remote-environments

panpanna 6y ago
I just wish vscode also supported LXC containers...
- madduci 6y ago
  Indeed, but I've found out that only VSCode can handle remote environments, VSCodium (the stripped version of Code) doesn't work

peterwwillis 6y ago

The script and Dockerfile I use are pretty long, but they set up everything needed to map in the most common stuff like ssh and aws creds, ptrace, sudo support, uid/gid runtime mapping, etc. Once you start developing your shared repos from a container, you realize that it's much easier to automate running things from it than to develop inside of it, so it's actually easier to get people to build CI/CD pipelines now where before they'd wait for someone else to do it for them. And the only problem there is there's no easy + good + free self-hosted CI/CD software out there. Yes, you probably use X just fine, but X probably doesn't scale, or isn't enterprise compatible, etc. The biggest barrier to automation is both a business problem and a technical problem.

p2t2p 6y ago

Developing a Java app in devcontainer on Linux using VSCode. I absolutely love it. I need to use an SDK that does ugly things to your local maven installation so I put everything in container, VSCode start it automatically and embedded terminal is opened inside that container, the only difference I notices that VSCode is not Idea.

zubairq 6y ago

If you want to program inside a container you could use something like yazz pilot which is designed to do exactly that https://github.com/zubairq/pilot https://github.com/zubairq/pilot

bJGVygG7MQVF8c 6y ago

Developing inside a container isn't worth the friction to me. I know my environment well, and I'm invested in continuing to maintain that knowledge.