8 ms·
Programmer: "I don't know whats wrong, it works on my machine" Manager: "Fine, then we will ship your machine" And thus docker was born.
by somat 3y ago
Programmer: "I don't know whats wrong, it works on my machine"
Manager: "Fine, then we will ship your machine"
And thus docker was born.
- salawat 3y agoI have never been able to realize the alleged ergonomic gains of containers. Ever. It always adds more friction to actually getting something initially stood up, prototyped, and deployed. I'm guessing it may be one of these things where it only starts to make sense after something has matured enough to warrant being replicated en-masse in a data-center environment. Then again, I tend to live in a world where I'm testing the ever-loving crap out of everything; and all that instrumentation has to go somewhere!
- turtlebits 3y agoBeing able to run self contained postgres with a single command is an easy ergonomic win. You don't necessarily need to be an engineer or write code for containers to be useful.
- ilyt 3y agoHow you inform your backup system where to get backups ? How you set pg_hba and other configs? Few other "how?" and you're doing what you'd be doing on VM anyway
- deleted 3y ago[deleted]
- rad_gruchalski 3y ago> How you inform your backup system where to get backups ? How you set pg_hba and other configs? Simple answer: with configuration. In Docker Compose or Kubernetes. Less often in Mesos. Maybe I want to run it on a fleet of VMs, maybe on bare metal. > and you're doing what you'd be doing on VM anyway Right. But with containers I can have different apps using different dependency versions. Some things use nginx, some use some other web server. Some things run with node 12, some with node 16, some things use MySQL, some other things need Postgres. This is so easy with containers.
- saurik 3y agoSo my issue is that now I have PostgreSQL running... inside a container? So I need to figure out where the data for it is stored and figure out how to ensure it is being backed up. And like, that's just minimum: I generally care deeply about the filesystem that PostgreSQL is running on and I will want to ensure the transaction log is on a different disk than the data files... and I now have to configure all of this stuff multiple times. At some point I am going to have to edit the configuration file for PostgreSQL... is it inside of the container? Did I have to manually map it from the outside of the container? The way you access PostgreSQL locally--maybe if you find yourself ready to add a copy of pgbouncer, but also just to run the admin tools--is via unix domain sockets. Are those correctly mapped outside of the container, and where did I put them? I honestly don't get it for something like PostgreSQL. I even use containers, but I can only see downsides for this particular use case. You know how easy it is to run PostgreSQL in some reasonable default confirmation? It is effectively 0 commands as you just install it and your distribution generally already has it running.
- rad_gruchalski 3y ago> So my issue is that now I have PostgreSQL running... inside a container? So I need to figure out where the data for it is stored and figure out how to ensure it is being backed up. And how is that different from running directly on the OS?
- serf 3y agothe OS will have an understood abstraction behind the filesystem structure, whereas container-systems often create an entirely new abstraction that they view as the best way. this makes sense when you're trying to be deployable universally, but it increases the amount of onboarding that someone needs to receive before they're proficient with the container system; onboarding they may have not actually needed to get the software working and well understood, simply 'docker overhead'. from personal experience : i'm a long time old linux person, the insistence on going 'all in' on Docker (or whatever) just to run a python script that has two or three common shared dependencies gains me nothing but the hassle of now having to maintain and understand a container system. if you're shipping truly fragile software that is dependent on version 1.29382828 rather than version 1.29382827 then I understand the benefits gained, but just to containerize something very simple in order to follow industry trends is obnoxious, increasingly common, and seemingly has soured a lot of people on a good idea. p.s. : I can also understand the idea of containerizing very simple things as parts of a larger mechanism; I just don't get it with the promise that it'll reduce end-user complexity, it isn't that simple.
- lelanthran 3y ago> Being able to run self contained postgres with a single command is an easy ergonomic win. You don't necessarily need to be an engineer or write code for containers to be useful. Postgres is a poor example; it's far far easier to do `apt-get install postgresql` than to run it from a container. The latter needs a container set up with correct ports, plus it needs to be pointed at the correct data directory before it is as usable as the former.
- imtringued 3y agoIs this some kind of joke? Those things are absolutely trivial and once you wrote them in your bash script or compose file you can forget about these things.
- j1elo 3y agoFor me, it solved A LOT of cases where users said "your software install fails", "the software doesn't run", or similar claims. They tend to be complicated to troubleshoot: people try setting up new software in an already customized development environments, and all sorts of mistakes are done. It was refreshing being able to say: this works perfectly from a base Ubuntu 20.04 container, launching this and that commands to install and run the software. Anything that diverges from there, you're on uncharted territory and/or is a problem in your system, not a defect in the software.
- ilyt 3y agoWell there is one unarguable one, although with consequences. Instead of keeping some old machine with old dependencies because project is on life support and doing update to latest version of language/tools is unviable, you can just keep a docker container for it while machine it is running on is up to date. It doesn't solve the problem of application, but it is no longer ops problem that app is old. You can also do that partially, like keep old PHP version running with just fcgi socket exposd in a container, while rest of the app lives on "normal" VM (or other container, if that's what you want).
- MilStdJunkie 3y agoThat's what got me on the Docker kick. Old unsupported programs that are tightly coupled to a kernel version, a Windows thing, some combination of Java / JDK shenanigans, an old SoundBlaster DLL, "The Python Carnival", or, really, what-have-you. Honestly, every big ERP demo ever should be done via container, because - and I swear I am not making this up - I have sat in months-long meetings with dozens and dozens of staff, both mine and theirs, trying to get Big ERP X up and running on-prem under whatever weird-ass policies the customer has on all their environments. Granted, a lot of THAT benefits from the fact that the container actually has to be a documented flavor of good solid *NIX, unlike the "Windows Image X" which is usually just a big "whatever" of old iron, management fad spoor, and seven different kinds of antivirus software. Hell, last time I did this, IT couldn't even find me any kind of description of what the "official" windows image actually was. So no fault of Windows there, it's just that, as the big bus everyone rides in, it gets all the goop from everyone wrenching on it.
- deleted 3y ago[deleted]
- pxc 3y ago> seven different kinds of antivirus software The last time I counted up all the different persistent, security-related agents running on my corporate Windows box, it was actually 14.
- _8j50 3y agoTo me, containers are like OOP. Nice ideas on paper but a pain the ass in practice. but again, like OOP, a lot of people do great things with them but I don't think they can't do the same things with VMs. My theory is eventually things will come full circle and there will entire app ecosystems running in a unikernel executable which is running on physical hardware and the hardware itself is segmented. I can imagine companies like netflix and cloudflare having racks of servers with 1TB ram with no disk, booting a unikernel from the network for maximum throughput and latency and then everyone in tech follows along.
- imtringued 3y agoI think you don't understand the problem. What people want is the ability to do all these things with a single command that they put inside a bash script.
- golergka 3y agoHave you tried setting up a python project with native dependencies on a different version of OS than original developer?
- auggierose 3y agoIf you cannot even set up your dependencies, a container might look like a solution, but I can assure you, it isn't.
- golergka 3y agoIt is the best solution. In my particular situation, at last. Another solution would find a senior developer who can manage his dependencies properly, but I can't afford him. However, I can pay a junior ML developer wage and just throw what he develops in a docker image instead of wasting time and money on doing things the hard way for a prototype. As engineers, we're supposed to be rational people, but unfortunately, we tend to forget that time and money are often happen to be the most important constraints we have to work around.
- twelve40 3y agoNo, i haven't. (not OP but) The last decade or more 90% of people i collaborated with were on the same operating system. Which moves the docker development setup solidly into "nice to have" category for me, especially when working on small teams <10.
- golergka 3y agoThings easily break even between minor releases of the same system.
- Karunamon 3y agoI guess we live in entirely different worlds, because, for me, containers have been a godsend. Standing up the average application is a matter of reading a compose file and modifying it according to instructions, or using it to construct a proper manifest for something like k8s for full production. For toy/evaluation use, it's hard to beat "tweak compose file, docker-compose up". You now have a golden source of truth for how the entire application and all of its moving parts were set up. Going back to deploying applications on bare metal feels positively medieval by comparison. From the admin side, configuration drift is effectively not a thing anymore. That's huge.
- wruza 3y agoIt feels like admins just offloaded the job of deploying apps back to developers. And as a developer I woudn’t even mind, if not for the f-ing slowdown of, well, the pace of development. All external teams that I worked with and who were using containers for development were like “oh you need that trivial endpoint or a bugfix? No problem, we’ll do it in few hours and it will autodeploy tomorrow”. And I didn’t ask why, because I know. As if we weren’t snail-paced money burners already. Provided all that, I don’t get why “devops” is even a high-paying job at most places except few really web scale ones. They are basically former sysadmins who reject anything except plugging colored squares into square sockets.
- ChoHag 3y ago[dead]
- rad_gruchalski 3y agoI'm on the other end of the spectrum. I don't want to deal with written instructions. A shell program will do but a container is the best. Give me an environment with all dependencies, software, and some decent configuration method. A config file over a volume will do, env vars are better. Dockerfile tells me everything I need to know about the environment the app requires. I can run that container in CI, in tests (with stuff like testcontainers), on my laptop, in production, on Kubernetes, in Mesos, Lambda, whatever. Instrumentation usually ends up in Prometheus and Jaeger.
- Danjoe4 3y agoYou answered your own question. When you join a large project and there's a perfect dev environment available in 3 minutes with `make init`, it makes sense. If you're the solo dev, it doesn't make sense to spend 5 hours figuring out the docker build system.
- earthling8118 3y agoIn my experience it never ends up being this simple. At least not with a combo of make and docker.
- ehnto 3y agoI have yet to be handed a containerized dev env that "just works". One company came close, they had someone dedicated to maintaining the dev tooling including the containers. It still didn't "just work" but they handled the troubleshooting, and the fix made it into the repo for the container so it wasn't just an unwritten adhoc fix I needed to remember. Close enough. I totally get the concept, I wish it worked so seamlessly that the promise was realized. But since I usually work with tooling I am familiar with, there is no point for me personally. I can stand up a local env faster than I can troubleshoot a broken container. Since I am not interested in the infrastructure and just want to get to work, that is what I do.
- computerfriend 3y agoEvery time I see this, I peek at the init target and it's always doing something weird like writing to my SSH config or something. I have learned to never trust installers written by colleagues.
- flatline 3y agoInstead of composing libraries into an executable you can use containers to compose services into an application [0]. As noted in the sibling comment, this still can have a high utility for just a single workstation/service. Also as you noted, scaling to a datacenter is already partly automated. The entire environment is captured in code and reproducible/portable. But you need to be familiar with the tools and ecosystem to make use of them. [0] We don’t have concise terms for the types of distributed, complex systems which have become standard.
- muxator 3y ago> Instead of composing libraries into an executable you can use containers to compose services into an application This is the most concise and exact definition I've heard lately. My only problem with the consequences of this approach is that the amount of overhead for performing the same operations is astonishing. Messages pass on a network instead of stsying in RAM, CPU context switches every other ms to handle IO for what could have been a simple function call, etc. It's amazing when it's needed, but seeing this approach becoming the standard for a big part of the industry almost turns it into an environmental problem. How much energy is wasted juggling bits around?
- lmz 3y agoIt doesn't have to be a library replacement. It could be that your app uses a DB server and a message queue for async tasks. Containers allow you to easily run those servers locally for dev if you don't care about performance and maybe in prod you'll have a dedicated server for them, or use some DB as a service offering.
- muxator 3y agoYeah, agreed: in that case spinning a container is evidently an operational advantage. However, I was specifically answering to the different case highlighted by the parent comment: how a potentially cohesive application is cut along some of its internal APIs, and some functionality is allocated to different processes living in different containers. It is an extreme point in the continuum "single thread" -> "multi thread" -> "multiple processes" -> "fully distributed". In that continuum scalability increases, while efficiency progressively decreases. Cornering oneself to a specific point in the design space is problematic, and for some cases has direct implications on how many resources are wasted. A very didactic experience is, for example, running a simple local application under a microarchitecture profiler, such as Intel Vtune. It is not uncommon seeing that even straightforward C/C++ programs use a core resources less than 10%. What I am reflecting about is that the choice of fragmenting that program among tens of systems (maybe in a scripting langiage) should be conscious, and done after encountering performance or scalability bottlenecks. How much of the resulting total system workload would be useful work? The quantity of potentially wasted resources is astonishing if you think about it.
- lmm 3y agoThey make sense if you're using Python, because Python dependency management is awful. For anything else they're not really worth it IME.
- coderenegade 3y agoNot at all. They make cross compiling fairly trivial, and give teams a way of managing cross compiled dependencies. This was vital a few years ago for things like torch and even opencv, where deploying on anything other than x86 meant you were probably on your own, and you almost certainly wanted control over the compiler flags. That's just one example, though. It also lets you standardize the dev environment (work directly out of the build container directly, or use your host OS and run unit tests in a local build), and it allows for easy, standardized testing within the same environment.
- lmm 3y ago> Not at all. They make cross compiling fairly trivial, and give teams a way of managing cross compiled dependencies. How so? A container is the same architecture as the host, how does that help with cross compiling?
- coderenegade 3y agoYou combine qemu with docker. You can then run arm containers and compile your code locally for deployment to an arm board.
- lmm 3y agoQemu is the part that's making that possible though. What's docker adding?
- imtringued 3y agoThe ability to kickstart the whole process with a few commands, possibly just a single bash script.
- time0ut 3y agoI can’t live without it. The power of docker compose for the development process is unparalleled. Being able to declare my entire local environment in a single file in a consistent way, create and destroy it with a simple command, and share it with my co-workers eliminates so many issues. Then being able to package my application in way that is repeatable and predictable is great. Then that same artifact is run across multiple environments spanning data centers around the world and on other developers’ machines via their compose files if needed. We test the crap out of all of this as well. Docker helps there to generate ephemeral environments to run the bulk of the integration tests against every time we push to git. Its just so damn powerful compared to what we had before…
- stavros 3y agoAgreed, and the fact that I can launch and develop my web app from seven years ago, no matter how much my environment has changed, is a lifesaver. There's a lot of code I've written that's not deployable any more because the versions of the dependencies it was using don't exist on my OS any more, and are too old to install.
- deleted 3y ago[deleted]
- friendzis 3y agoAs long as you have saved the container image and contents of all mapped volumes somewhere. Which is not much different from snapshotting a VM.
- imtringued 3y agoSnapshotting a VM is a pain in the ass. It takes forever and the tooling sucks. The concept of snapshotting is also completely wrong. Snapshots are tied to a VM with the specific VM being front and center, you know, the thing that is disposable with containers. Also, from my misadventures with Ubuntu VMs I always ran into the problem of them breaking over time.
- 3y ago
- marcosdumay 3y agoIt solves the "we can't possibly package the software for all versions of libc out there" and the "there are all those programs that must run at the same time, it's hard to correctly install them all" problems. People push it to solve the "this program needs executable A version 1, that program needs executable A version 2, both programs must run at the same time" one, that should never be a big issue anyway. But then, we have none of those problems at work and people still believe it's a panacea. I don't know what people believe they gain by it.
- 8organicbits 3y agoDependency management is one thing Docker handles, and I don't really think it does a great job of that. To me containers solve the challenge of getting a development environment that closely matches production. It standardizes how deployments are performed, how testing environments are set up. Before this we had complex, immutable ansible scripts, READMEs, install scripts that had to be babysat, snowflake servers that no-one knew how they were configured. It's also an extremely widely used technology, so if you use it in your stack you can expect many people you hire to know how to use it from day one versus everyone learning a bespoke set of custom scripts.
- jjjfdjunnmko23 3y agoThis was already trivially solvable by establishing a new sysroot, which in all fairness is 90% of the core benefit of containers in my view. The rest of the container features are bloat for most of my use cases.
- nerdponx 3y agoIt also solves the "we have limited developer-hours available and just want to deploy some stuff" problem. Docker is amazing for teams with limited resources. If you can deploy one image, you can deploy any image.
- marcosdumay 3y agoThat's my point. I hear a lot of what you say, but I don't understand it. In general, without some kind of qualification, deploying an image is way worse than a flat binary, or a tar from some interpreted language.
- hinkley 3y agoContainers aren't about you. They're about the person you're trying to hand things to. I got stuck creating VMs for testers for a distributed workflow (several services, several tools), and keeping those VMs up to date and working was a PITA, but far less painful than dealing with them filing bug reports based on thinking they were doing X when they were doing Y. I ended up creating a workflow that felt like layers, so when Docker came along I didn't need a salespitch. I could just do what I'd done with a script instead of a runbook. Where do I sign up?
- paulddraper 3y agoExactly. Containers are packaging for executables.
- transcriptase 3y agoAnother decade or two and perhaps the Unix community will finally reinvent the .exe or .app
- coderenegade 3y agoIt's more than just creating a distributable, though. It's also a dev fixed environment and a cross-compiler when paired with qemu, and a test suite all rolled into one. It's hugely powerful for how simple it is.
- paulddraper 3y agoThat's just.......wrong. MacOS makes static linking quite difficult. And Windows is infamous for "DLL Hell."
- pjmlp 3y agoWindows 9X/2000/NT was long time ago, nowadays that it more a GNU/Linux problem than Windows one.
- nerdponx 3y ago
- imtringued 3y agoI have an app that needs a media server, a webrtc STUN/TURN server, a PostgreSQL database, a JVM based web server, a worker that may or may not run on the same host and a way to run liquibase for migrations and certbot for letsencrypt certificates. The whole app can be started with a single command and it works on most Linux distributions. I can't imagine wasting the time of users or newbie developers demanding that they install all these things separately and with no easy way to clean it up if they want to undo everything.
- andrewedstrom 3y agoHonestly, a pretty reasonable solution to that problem. It's cool that we have the technology to make that work.
- treeman79 3y agoOwner hired an extremely “senior” developer. Was told to let him do his thing. After he spent 3 months building a web app, I asked him how he wanted to deploy it. Perfectly straight face he said we would take his developer machine to a local data center and plug it in. We could then buy him a new developer machine. It went downhill from there. I ended up writing the application from scratch and deploying it that same evening. Owner hired a lot Of strange people.
- bandrami 3y agoAn idea meant to lighten the load on sysadmins now means I have seven different OS versions to worry about
- dunham 3y ago> "I don't know whats wrong, it works on my machine" I had one of these years ago where QA had an issue that I couldn't reproduce. I walked over to his desk, watched the repro and realized that he was someone who clicked to open a dropdown and then clicked again to select, while I would hold the mouse button down and then let up to select.
- voyagerfan5761 3y agoI do not even want to know what sort of weird-ass GUI code or behavior made those two operations dissimilar.
- dunham 3y agoYeah, it's been way too long to remember (maybe a decade ago). Best guess is the click event was propagating up the dom and triggering something. These days, it looks like chrome generates a click even if you don't let up on the mouse.
- ParetoOptimal 3y ago> he was someone who clicked to open a dropdown and then clicked again to select, while I would hold the mouse button down and then let up to select. I didn't know the latter was possible, but I also don't know which of these I do.
- RF_Savage 3y agoFriend found some developers vacation photos on an industrial controller. Turns out they did ship a 1:1 image of his machine.
- marcus_holmes 3y agoWe used to do literally this back in the day. Dev would get the thing working on their machine configured for a customer. We'd take their machine and put it in the server room, and use it as the server for that customer. Dev would get a new machine. Yes, I know it's stupid. But if it's stupid and it works, it isn't stupid. DLL Hell was real. Spending days trying to get the exact combination of runtimes and DLL's that made the thing spring into life wasn't fun, especially with a customer waiting and management breathing down our necks. This became the easiest option. We started speccing dev machines with half an eye on "this might end up in the server room".
- salawat 3y ago...what? nm -D is not hard. Debugging missing symbols isn't even that hard. Biggest damn challenge I see is we've completely dropped the ball at educating people about how linkers, loaders, and dynamic library symbols work. ...and so people understand something here... I've transplanted/frankensteined userspaces that were completely hosed/disjoint into working before. In fact, every now and again I do it just to remain in practice. It's actually gotten to the point I don't even worry about dependency hell anymore. I just find the right version of the library/source and expand my archives just in case I need it down the road.
- rahoulb 3y agoThat's basically what Smalltalk was back in the 20th Century. The OS, the development environment and the application (both code and live objects) where one and the same thing. To ship an "app" you would export the image and the user would load it into their Smalltalk VM.