23 ms·

The sad state of sysadmin in the age of containers

• zenlot 11y ago
  "Stack is the new term for "I have no idea what I'm actually using"." - this made my day!
  • skratlo 11y ago
    Same for "framework" which is: I have no idea what I'm doing
    • highmastdon 11y ago
      Same goes for 'abstraction', it hides the essence of what is happening. Therefore every abstraction is evil.
      • mercurial 11y ago
        Hopefully this is sarcasm. Code without abstraction can also very efficiently hide what is happening by having a disastrous signal-to-noise ratio, combined with all the potential for errors you get when repeating the same pattern many times.
        • Lawtonfogle 11y ago
          Code is abstraction. With no abstraction you have to build your systems with lots of nand gates and a clock.
      • willstepp 11y ago
        Abstractions are necessary to write software, unless you speak binary code, so calling them evil is a bit hyperbolic. My point is at some level you have to trust the abstractions of a system or else nothing would get done. That doesn't mean you shouldn't have a conceptual understanding of the lower levels, but they aren't evil!
      • VLM 11y ago
        Its possible to use abstractions that aren't inversions or leaky. Practically no one does a good job of it, so you are correct in practice and experience, although in theory it is possible and sometimes people do pull it off successfully. http://en.wikipedia.org/wiki/Abstraction_inversion http://en.wikipedia.org/wiki/Abstraction_inversion http://en.wikipedia.org/wiki/Leaky_abstraction http://en.wikipedia.org/wiki/Leaky_abstraction
      • TallGuyShort 11y ago
        I think you should strive to understand what is happening under the abstraction, but abstraction is a useful tool. It's a bit like calling something a "crutch". Sounds bad, but what if you have a broken leg? Crutches allow you to get over the problem and make progress. I have to write software that runs on any hardware from any number of vendors and multiple operating systems. Not using abstractions would destroy my effectiveness.
      • rjbwork 11y ago
        How are people missing the sarcasm of this comment?
    • pmontra 11y ago
      Sometimes yes, but sometimes you started writing CGIs in C, then Perl, than you wrote your microframework, then you decided to use a standard one. This has been my evolution and even if I don't understand everything inside the frameworks I'm using now I have a general idea. And furthermore, what can we do about it? Writing code from scratch or maintaining or own frameworks is more or less the way to losing customers, unless you are a Facebook and you call engineer and push a React.
      • moron4hire 11y ago
        If more people went through that process, the frameworks we have might be fewer and of better quality.
        • jessaustin 11y ago
          Judging by the sheer number of frameworks we have, everyone who went through that process actually ended up writing one, or five.
      • collyw 11y ago
        That sounds like where I am coming from. Looking at Django questions on Stack Overflow, a lot of people don't.
      • sanderjd 11y ago
        Thank you for this. I hate the "frameworks are for people who don't know what they're doing!" meme. Sometimes they're just for people who have thought about the trade-offs and decided a popular framework has many advantages.
    • manicdee 11y ago
      Programming languages are for lazy people who don't like flipping switches to code aseembly by hand.
    • bandrami 11y ago
      "Framework" at least pretty reliably means some form of code generation is going on.
• deleted 11y ago
  [deleted]
• blfr 11y ago
  This bothers me as well. Even tasks as simple as adding a repository are now being "improved" with a curl | sudo bash style setup[1]. However, installing from source with make was (and remains) a mess. It may work if you're dedicated to maintaining one application and (part of) its stack. But even then it usually leads to out of date software and tracking versions by hand. Many people have this weird aversion to doing basic sysadmin stuff with Linux. What makes it weird is that it's really simple. Often easier than figuring out another deploy system. (The neckbeard in me blames the popularity of OSX on dev machines.) [1] https://nodesource.com/blog/nodejs-v012-iojs-and-the-nodesource-linux-repositories https://nodesource.com/blog/nodejs-v012-iojs-and-the-nodesou...
  • deleted 11y ago
    [deleted]
  • hurin 11y ago
    > Many people have this weird aversion to doing basic sysadmin stuff with Linux. What makes it weird is that it's really simple. Often easier than figuring out another deploy system. While I agree with the articles main points - the GNU build system is far from simple. Basically an arcane syntax limited to unix-based systems and 5 or 6 100+ page manuals to cover. It doesn't excuse it - but I think it's easy to see why people turn to curl | sudo bash as the author puts it.
    • pjc50 11y ago
      Maintaining autoconf/automake stuff is a pain. Using it is usually as simple as "configure;make;make install". It doesn't do dependency management though, which is an externalised cost. But that's what rpm/deb do. I see the attraction of containers and disk image based management. It's much less time consuming. But it's very much the opposite of ISO9001-style input traceability.
      • sltkr 11y ago
        > Using it is usually as simple as "configure;make;make install". "Usually" indeed. Because if it breaks, you do need to know the implementation details to figure out what's wrong.
• pkrumins 11y ago
  Give me the command line and I'll build anything!
• quonn 11y ago
  There is some truth in this, yes. On the other hand, maven (mentioned by the author) clearly was very successful at abstracting from the tools we use. I can remember how much time I wasted with build scripts and dependency management and all that before. (And I still do on some other platforms.) The problems only arise if the abstraction is not working well enough. This might indeed be true for containers - there is maybe too much complexity in there that currently can't be properly encapsulated.
• NhanH 11y ago
  So, asking the obvious question: what's the solution to that?
  • prottmann 11y ago
    The obvious question is: what's the real problem with that? A container is a container, as long as docker itself has not bug, the container can only harm the containers content. Most problems exists in the custom created software in the container (e.g. web-services with bugs, backdoors, ....), this will be a problem for Docker, VMs, Real-Servers, whatever too. The real problem is the interoperability of different container, if you link the whole data, without any audit, to another container, you can have a problem, but this problem is not docker specific.
    • tribaal 11y ago
      You seem to ignore or downplay all the other ways a container can cause problems, including but not limited to: - Being a backdoor to the rest of your network (sniffing network traffic, or more simply reverse ssh-tunneling to an outside server) - All the various "fun" botnet-related activities (spam being the king here) - Actively serving malware to the rest of your network. EDIT: Formatting, and well, others answered your question less specifically but more eloquently.
    • Nursie 11y ago
      >> A container is a container, as long as docker itself has not bug, the container can only harm the containers content. Presumably a container has network access of some sort? Malicious code could start probing and attacking anything exposed that way. >> this will be a problem for Docker, VMs, Real-Servers, whatever too. The implication is that you wouldn't get into this situation with a 'Real-Server' so easily, because you wouldn't just download an image and run it, without having an update/patch strategy or having much more idea of what's going on inside it.
      • prottmann 11y ago
        But you assume that a container HAS full network access. A firewall must be configured, but a firewall must be configured for a VM too. My point is, that their is not so a huge difference for production systems.
• confiscate 11y ago
  haha you're my new hero YOU ONLY LIVE ONCE MAN! trust the (maven) system
• omnibrain 11y ago
  If you read german I can recommend you https://plus.google.com/+KristianK%C3%B6hntopp/posts/gPpHx5Trec6 https://plus.google.com/+KristianK%C3%B6hntopp/posts/gPpHx5T... and https://plus.google.com/+KristianK%C3%B6hntopp/posts/54v3MNX8ud7 https://plus.google.com/+KristianK%C3%B6hntopp/posts/54v3MNX...
• DomreiRoam 11y ago
  "Maven, ivy and sbt are the go-to tools for having your system download unsigned binary data from the internet and run it on your computer." You should setup a maven repository (Nexus, Artifactory) for your organisation if you want to have more control on binaries. Seems that artifactory can host docker files: https://www.jfrog.com/confluence/display/RTF/Docker+Repositories https://www.jfrog.com/confluence/display/RTF/Docker+Reposito...
  • loopbit 11y ago
    Kind of what I was going to say... The article seems to blame the tools, but there are more secure ways of using these same tools.
  • rickette 11y ago
    Right, do folks really belief Maven, Ivy, Gradle, Sbt are tools you use in production? These are developer tools for use on workstations and CI servers. If you want to promote your stuff to other environments like production use your own private repository (Nexus, etc).
    • caw 11y ago
      They may be if you have a team without much sysadmin experience. The way you develop could be the way you deploy to production. These are the same teams that have overprivileged accounts for the database or sudo-enabled users running applications or chmod 777 all over the place. Even things like Chef cookbooks have this going on. If you want to build from source because it's not in your repository, then you're necessarily going to need to drag in sbt or gradle. (see https://github.com/hw-cookbooks/kafka/blob/develop/recipes/default.rb https://github.com/hw-cookbooks/kafka/blob/develop/recipes/d... as an example). Sure you could figure out the mirrors and download the correct binary from the website. You could also use this recipe to compile everything and then package it up to host yourself. (Both of these actions require writing custom recipes). Not everyone has time to do this, and this magical recipe you found online works great on the development server! Just add it to the production server and now we've just used sbt in production on a software team.
• lmm 11y ago
  make is the least-auditable build tool imaginable. You don't have to obfuscate a Makefile, they come pre-obfuscated; you could put the "own me" commands right there in "plain" Make. Not to mention that it's often easier to tell whether a Java .class file is doing anything nefarious than whether a .c file is. How many sysadmins read the entire source of everything they install anyway? Maven, on the contrary, is the biggest single source of signed packages around. Every package in maven central has a GPG signature - the exact same gold standard that Debian follows. The problems Debian faces with packaging Hadoop are largely of their own making; Debian was happy to integrate Perl/CPAN into apt, but somehow refuses to do the same with any other language. > Instead of writing clean, modular architecture, everything these days morphs into a huge mess of interlocked dependencies. Last I checked, the Hadoop classpath was already over 100 jars. I bet it is now 150 That's exactly what clean modular architecture means. Small jars that do one thing well. They're all signed. Bigtop is indeed terrible for security, but its target audience is people who want a one-stop build solution - not the kind of people who want to build everything themselves and carefully audit it. If you are someone who cares about security, the hadoop jars are right there with pgp signatures in the maven central repository, and the source is there if you want to build it.
  • geocar 11y ago
    Makefiles don't really enter into it and getting software signed by the developer isn't that valuable or useful. The value of debian is not that they package (or repackage) everything into deb files but that they resolve versioning and dependancy conflicts, slip security fixes into old versions of libraries (when newer version break API/ABI), and make it possible to integrate completely disparate software into a system. They also have a great track record at it. Maven does not do any of these things; Maven does nothing to protect the system administrator from a stupid developer, it just makes it easier for their code to breed and fester. You must understand that the sysadmin has an enormous responsibility that is difficult for programmers to fully appreciate: You don't feel responsible for your bugs, you don't feel responsible for mistakes made by the developer of a library you use, and you certainly don't feel responsible for the behaviour of some other program on the same machine as your software, after all: Your program is sufficiently modular and scalable and even if it isn't, programming is hard, and every software has bugs. But the sysadmin does feel responsible. He is responsible for the decisions you make, so if you seem to be making decisions that help him (like making it easy for you to get your software into debian) then he finds it easier to trust you. If you make him play whackamole with dependencies, and require a server (or a container) all to yourself, and don't document how to deal with your logfiles (or even where they show up), how or when you will communicate with remote hosts, how much bandwidth you'll use, and so on: That's what Maven is. It's a surprise box that encourages shotgun debugging and using ausearch features to do upgrades. Maven is a programmer-decision that causes a lot of sysadmins grief a few months to a few years after deployment, so it shouldn't surprise you to find that the seasoned sysadmin is hostile to it.
• deleted 11y ago
  [deleted]
  • peteretep 11y ago
    > As far as I know, it's also still standard practice in > most companies to either read the source code of open- > source stuff before deploying it to production (binary > or build) or get a support contract from someone else > who has I'm afraid I have no better, more cogent response for this than 'lol'.
    • empressplay 11y ago
      This is an interesting read: http://www.bearingpoint.com/en-no/download/Open_Source_Governance_In_Highly_Regulated_Companies.pdf http://www.bearingpoint.com/en-no/download/Open_Source_Gover...
      • ckozlowski 11y ago
        Thanks for the share. =) I'll queue this to read later, but just reading the executive statement, it seems to jive with a lot of the discussion here. The issue isn't that code review /shouldn't/ be taking place, or even that there aren't directives stating that it should be done. It's that it isn't being done, and that's a problem.
    • empressplay 11y ago
      At the risk of my karma I'll have to maintain that for companies who are subject to regulation (publicly-traded companies, banks, etc.) what I said is still standard. Unless you have any specific instances to the contrary you're willing to offer?
      • ckozlowski 11y ago
        Coming from my background (DoD, don't laugh.), code review of anything other than in-house developed applications never occurs. In any instance open source is used, it is mandated that it come with a support contract (per DISA STIG) which provides the support and accountability the organization is looking for. So, with regards to your assertion, the second clause (support contract)? Definitely. the first (code review), never. That's the view from my side of the fence, anyways.
• martinp 11y ago
  Previous discussion: https://news.ycombinator.com/item?id=9190955 https://news.ycombinator.com/item?id=9190955
• morgante 11y ago
  Except that Docker explicitly allows and encourages signing of core infrastructure containers.
  • TheDong 11y ago
    Not even close. Docker now has some terribly attempts at signing images on their registry iirc (docker inc signs them for the docker client). There is no option for me, as a user, to build and sign my own image with my own pgp key afaik. My organization might already have a chain of trust, and docker is asking me to ignore that and just trust their signatures (which also only work on dockerhub as of docker 1.5... don't know about 1.6 because you can't use docker for at least a month after a release else security holes galore). Docker did nothing to encourage signing containers. At 1.0 they had no capability to do any signature, verification, whatsoever. It's being added as an afterthought, and poorly. If you look at the AppContainer specs, signatures (pgp based) were built in from the very beginning, it lets me create my own chain of trust (including incorporating other's keys), sign my own images, trust someone elses signature, does not trust the transport or storage medium, and has integration with the clients. If you want to convince me docker cares, you're going to have to give me examples of where they didn't fuck up... Tell me how I can use docker's tools to sign my own images, optionally trust my friend Alice, and securely download images that she uploaded to her own registry or dockerhub but signed with her gpg key without me having to trust docker inc. To my knowledge, all docker has right now is doing a 'tarsum' of images which assumes the registry is trusted and, even given that, can be downgraded for backwards compatibility reasons fairly trivially.
    • amouat 11y ago
      Docker didn't fuck up when they hired the square guys. http://blog.docker.com/2015/03/secured-at-docker-diogo-monica-and-nathan-mccauley/ http://blog.docker.com/2015/03/secured-at-docker-diogo-monic... I agree that security and provenance is a real issue in Docker. It is however being worked on, and it will be solved. Presumably we will end up with some sort of app-store like framework with proper signatures and verification. Docker can't do everything at once. Give them a chance. The new version of the registry is a major step forward in this regard. In the meantime, what you can do is take redhat's advice. Rather than using a registry to get your images, operate a download site which stores archives of docker images that you can import with `docker load`. You can then also store signatures and check them yourself.
• __mp 11y ago
  As an ex sysadmin I really like the container infrastructure. Manage the whole configuration on the main machine with puppet and deploy the blackbox applications (everything ruby and java related) with docker/rocket.
  • blumkvist 11y ago
    Maybe you can send a copy of your BRMS to your competitors while you are it.
  • mrweasel 11y ago
    It's nice to have the option. Containers are awesome for many things/projects, but sometime you just want to run the damn application on a server of your choice, without any container stuff. I can't remember what the application was, but I've seen an application where the only installation instructions where for Docker. That's just plain silly. My concern with containers is that the wrong people will use it. There is a ton of software out there with just barely runs and make all kinds of assumption about it's environment. I fear that rather than design better, more correct software, these people/companies will start packing up their development environments as containers (more or else) and just ship those. Of cause that's no reason to discourage the use of containers, we just need to be critical of what is inside them.
• deleted 11y ago
  [deleted]
• bshimmin 11y ago
  So much truth in this. We've been doing some work with Elastic Beanstalk lately, and - while it certainly does one or two things that are extremely clever and useful - in the end it just feels like this bizarre mix of complete magic and incredibly convoluted arcana. Everything feels very out of our control and locks us into an ecosystem that considerably limits our choices and flexibility (unless we invest the time in becoming experts in EB, which really isn't particularly something we have the time for). And, as the author of this post says, the security ramifications, while orthogonal, are also deeply troubling.
• rsanders 11y ago
  I was doing sysadmin the "right way" a long, long time ago, and I don't see much difference. Maybe the author regularly does full audits of the source code of every package he downloads, and of course disassembles every executable and library in the underlying OS, but most of us don't. There's no wisdom or security to be gained from the act of running "make", much less "make install".
  • rlpb 11y ago
    There's a huge difference. > Maybe the author regularly does full audits of the source code of every package he downloads, and of course disassembles every executable and library in the underlying OS, but most of us don't. What matters is that the source code is auditable. It only takes one person to investigate something suspicious, raise a flag and get it fixed. This is certainly still true for Debian - not being able to build from source is considered a release blocking bug.
    • kibibu 11y ago
      Assuming you: - trust your compiler and linker - trust your tar extractor / package manager / whatever - trust your editor - trust your http library (or whatever you used to download/distribute the code)
      • Nursie 11y ago
        >> - trust your http library (or whatever you used to download/distribute the code) This can be overcome with signing. We're all well aware of how deep this rabbit-hole goes, however that doesn't mean that it's a good idea to throw all trust away.
      • rlpb 11y ago
        It comes down to trusting two key things. Trusting your initial distribution download (which contains the package signing keys), and trusting the toolchain (in a Reflections on Trusting Trust way). But the deeper you go, the harder it is for malicious code to reside there. In theory it's possible, but in practice I'd like someone to show me some code somebody could have written into the toolchain a decade ago, without hindsight, which could still exist today. Whichever way, it's clearly far tougher for a malicious actor to compromise a system by injecting something into a distribution ecosystem than it is to inject a signed-by-unknown-reputation binary-only package into the Maven ecosystem.
• pjmlp 11y ago
  > »Docker is the new 'curl | sudo bash'«. Fully agree with this. Maybe I am just only another grey beard grumpy developer, but the new generations that grew up with GNU/Linux instead of UNIX, bash the security of other OSes and then go running such commands all the time.
  • antocv 11y ago
    Sad really. docker and any user in the group docker, or lets say, any user capable of sending commands to the docker daemon running as root - is root on that system. docker -v /:/f -w /f yourimage /bin/bash -c "echo root:and:so:on > /f/etc/shadow"
• antocv 11y ago
  So much truth spoken in the linked text. Thanks. It has to be said. Damn the containers and windowization of Linux.
  • madez 11y ago
    To isolate closed-source programs containers are very helpful. I don't like to run steam within my normal debian system.
• upofadown 11y ago
  Every once in a while someone figures out that we could entirely solve the dependency problem by packaging all the dependencies with the application. Everyone gets excited. After a while everyone gets unexcited when the problems associated with this approach become obvious. Docker is merely a more extreme example of the "package everything with the application" idea...
  • parasubvert 11y ago
    I'll bite. What's obviously the problem about it? Or, "if it's good enough for Google...." Vendoring dependencies and static linking is quite popular in executables, not just docker. Dynamic linking and shared libraries seem to be becoming a relic, deservedly. BTW, the extreme example of "package everything with the app" is the unikernel movement.
• cgb_ 11y ago
  This rant is about containers, prebuilt VMs, and the incredible mess they cause because their concept lacks notions of "trust" and "upgrades". Prebuild VMs? Sure, I wouldn't touch them except for evaluating a project, and for commercial software you may not have a choice. But docker containers at least usually provide a dockerfile that describes exactly how a binary image is built. You just clone the source repo, audit the few lines of build commands and then build your own private registry. It's nearly no more trust than following the instructions of README or INSTALL. Just because fools are pulling down pre-build images and running in their datacentre, doesn't mean that's what way you should do it. And the problem with 'old-school' sysadmins is they are often far too quick to reject new practices, citing tired excuses based on misunderstandings of the technologies. Ever tried to security update a container? Yeah I have. It's easy if you have already built your 'stack' to scale horizontally (which means you have at least 2 or more of everything in a HA or LB config). You rebuild against a fully patched base-OS container, spin-up, send some test load to it & validate, then bring into service. Repeat for rest of nodes at that tier. If you are trying to be an old-school sysadmin that expects to console or SSH in and run 'yum upgrade' or 'apt-get upgrade' your containers then you are doing containers wrong...
  • vacri 11y ago
    then you are doing containers wrong... The old-school sysadmins I know scoff at Docker's idea of 'containers'. Linux containers were already a thing, and don't need an entire copy of an OS ported around with them. To them, containers are a way of enveloping a process to limit it, not a way of distributing packaged software. They may or may not be doing 'docker' right, but they certainly know what 'linux containers' are.
    • cgb_ 11y ago
      Well I should have qualified it with 'docker containers'. But yeah, those that have been around long enough in Linux container land have all dealt with vserver, openvz, lxc, etc, and all of those carried around this 'entire copy' of an OS, per container (ignoring vserver's vhashify). Docker helps you to spin up N containers running all sorts of applications based on the single master image. Docker, whether your view is good or bad, brings something more than just another container implementation to the table...
• mercurial 11y ago
  I love a good rant as much as the next guy, but unfortunately, rants are rarely actionable. > Maven, ivy and sbt are the go-to tools for having your system download unsigned binary data from the internet and run it on your computer. The root of the problem is that out of the total number of libraries available in language X, only a small subset is packaged in Debian/RHEL. This may be more egregious with large, Java enterprisy software, but you could easily end up with the same problem in Ruby or Python. You cannot reasonably expect developers to package and maintain all their dependencies properly. The least worse solution would be to: - still use maven to manage dependencies - create a Debian/RHEL package incorporating the dependencies (effectively vendoring them in the package) Unfortunately, it is not that simple, because you need to make sure that your vendored-in-the-package dependencies are somewhere where they will not conflict with another package with the same idea and the same dependencies (or better, the same idea and a different version of the same dependencies). Which means you need to keep them out of /usr/share/java and make sure the classpath points at the right location. However, it seems that developer tend to avoid this kind of rigmarole and instead go for the "install dependencies as a local user" for certain classes of application (eg, webapps) because packaging is not fun.
  • rlpb 11y ago
    > You cannot reasonably expect developers to package and maintain all their dependencies properly. I think that this is a good point, but it all comes down to quality control. You wouldn't accept a new dependency into your project if it is buggy or has a bad API. So why is bad packaging, a hacked-up build system or inability to build from an auditable source considered acceptable in many communities today?
    • mercurial 11y ago
      I don't think that's acceptable, but ranting and ignoring the underlying issue doesn't help.
  • mike_hearn 11y ago
    Maven packages aren't actually unsigned either. They're downloaded over SSL and to get into Maven Central you need to sign with a GPG key. The problem is that you normally cannot find a path to the developers through the web of trust, of course, but that's not Maven's fault. That's the fault of the web of trust (more accurately called "handful of strings of trust"). Debian/Red Hat code signing doesn't prove very much either. All it proves is that the package came from Debian or Red Hat. But did they modify the software along the way, doing some kind of MITM attack on the upstream developers? Quite possibly! At least with Maven you don't have that problem.
• duggan 11y ago
  The contract between operations and dev (as concepts, not as people) is in need of renewal. To my mind, that was what "devops" was supposed to be, but it's been a bit of a dogpile in the years since the term gained popularity. Systems are opaque to most developers, and many developers wish to make their software opaque to the system on which it runs. This is a failure on behalf of our entire profession, not any one group. Infrastructure software is in a bit of a renaissance period, but it's very early days. Packaging software is a total mystery to most developers. I don't even need to back that up with examples, most of us can recall the last time we can across a well packaged piece of software with joy due to sheer rarity. I'd be very surprised to find the average age of a Debian maintainer was trending anything but upwards, and steeply. Containers are being misused, but that's because the alternatives we've been building for ourselves have not kept up with the strong user experience narrative of web and mobile software. We need to do better.
  • nvarsj 11y ago
    It would be nice to have a well known 'devops manifesto'. I google'd it and came across this: https://sites.google.com/a/jezhumble.net/devops-manifesto/ https://sites.google.com/a/jezhumble.net/devops-manifesto/. Which I think is actually pretty decent - the emphasis on cross functional product teams, for instance. In my mind, that is largely what devops is about - team ownership of the entire product, which includes infrastructure. Instead of having a silo'd 'ops' team writing ansible scripts and doing deployment, this should be part of the team (which could mean having an opsy guy on the team). Anyways, as it pertains to containers, I think containers are more a practice than a principle. It tends to happen naturally when you want reproducible builds and continuous delivery. It's not really about making systems opaque to software, imo, but rather making your product artifacts reproducible (if you rely on running ./configure; make at deploy time, you never know what you'll end up with since dependencies are dynamically determined).
    • kungfudevops 11y ago
      IMO here's the best "devops manifesto" out there: https://github.com/chef/devops-kungfu https://github.com/chef/devops-kungfu https://www.youtube.com/watch?v=_DEToXsgrPc https://www.youtube.com/watch?v=_DEToXsgrPc
• freedombeer 11y ago
  +100 upvotes
• dschiptsov 11y ago
  What is really ironic is that none of this "tools" solves any fundamental problem of so-called version hell and none of this containers are fundamentally different from ./configure --prefix=/xxxx && make && make -s install with or without following chroot /yyyy The big "innovation" of having so-called "virtual env" (they call it "reproducible [development] environment) for each "hello world" (a whole python/ruby/java/etc installation with all packages and its dependencies in your [home] project directory) solves no real problem, only pushes it to the next guy (what they call devops). Some idiots even advocating to have a whole snapshot of an OS attached to your "hello world", and even to make it what they call "purely functional" or even "monadic" (why not, of someone pays for that). Unfortunately, there is no way to ignore complexity of versions and package dependencies or easily pushing it to "devops". Creating a zillion of "container images" with just your "reproducible development environment" or a whole "OS snapshot" just multiplies entities without a necessity. Programmer must be aware of which version of what API implemented with what version of package or library he using and explicitly assert and maintain these requirements, like all the very few sane software projects (git, nginx, redis, postgress) do. btw, the GNU autotools (which gives us ./configure) is somewhat evolved real-world solution - you have to explicitly check each version of each API both at the compile (build) time and refuse to build in case of unsatisfied dependencies and at the install time (and package manager must refuse to install if case of mismatch). This is the only way back to sanity, however "painful" it is.
  • TeMPOraL 11y ago
    > Programmer must be aware of which version of what API implemented with what version of package or library he using and explicitly assert and maintain these requirements, like all the very few sane software projects (git, nginx, redis, postgress) do. Except that when you're doing anything that looks like an actual end-user application (as opposed to infrastructure), you end up using dozens of libraries which themselves have dependencies, so suddenly you're supposed to "explicitly assert and maintain" hundreds of different library versions, none of which is in any way relevant to the application you're building. I myself see Docker containers as the only reasonable way for giving a service application to people to deploy on their machines, because even the programming runtime I need is 5 years out of date on Debian/Ubuntu, and installing that stuff manually is a) pain, and b) different on every operating system.
• mahouse 11y ago
  Gentoo and BSD ports exist for a reason.
• Fastidious 11y ago
  Fully agree with everything written there, with the exception of "apps." I believe it is not a Microsoft term, but an Apple term. Is it not?
• failathon 11y ago
  Is this the sad rabbit hole reality of attempting to abstract every last component?
• oskarth 11y ago
  At my last job we used a micro-service architecture on AWS (EC2 and RDS). Using Ansible playbooks for various types of servers and roles for each service, we created a new server instance every deploy. All servers were running FreeBSD and using daemontools to control services. For testing, hotfixes, and manual checking of logs, it was easy to complement with manual ssh. Save old and new instance in case something goes wrong. Ansible is just a thin layer on top of shell scripts, and reasonably straightforward to understand and parameterize. Worked wonderful in most cases (possible exception of build server because of shared libraries and a complex workflow with git pull/trigger, but I don't think that was the fault of the overall architecture). That said, I agree that sbt is an abomination and doesn't lend itself to a sane and secure workflow, unfortunately. http://martinfowler.com/bliki/PhoenixServer.html http://martinfowler.com/bliki/PhoenixServer.html http://www.ansible.com/ http://www.ansible.com/
  • ploxiln 11y ago
    I also learned from a place with good practices, which used daemontools to run services and a custom deploy system in bash and python which actually did the right kinds of things. (and I was fully-manually admin-ing my own linux systems for years before that) As an early employee at a new place, I'm now using ansible and docker because nowadays people want to use that stuff, and it is a lot faster to get started with than writing a new proper deploy system from scratch. But I build all the docker images we use and version them with the date. I also don't use ansible roles from ansible-galaxy, and I don't even organize tasks into roles, just into task-include files. Our ansible tasks use bash helper scripts wherever necessary to do things the right way, because the built-in modules are often too granular / not connected enough to fully check state. I also replaced the docker plugin for ansible, to manage state a bit better. So overall it's not too bad. I guess my point is that, having done it all from-scratch first, using some of the modern automation stuff isn't too bad. But you have to know what not to use. People new to "devops", using all the fancy stuff now available, who didn't have the introduction I did... it's not surprising they end up in a mess and don't even recognize it.
    • NathanKP 11y ago
      This is the key. The OP's major complaint is with prebuilt containers from potentially untrustworthy sources, but he passes this off as a fundamental problem with containers themselves. The reality is that you can (and probably should) build your own container rather than using a public one from docker hub. You know exactly what is in it, and can trust it completely.
• jasonsync 11y ago
  Sys admins got relegated to tech support so web devs could add sys admin to their work flow.
• msane 11y ago
  Containers and VMs are part of the solution to this problem, not a cause. Try managing the same dependencies across N platforms rather than one container!
• thebouv 11y ago
  I've often thought about just offering my services as a sys-admin to the multitude of small startups that popup locally. Many, many developers -- almost no sys-admining skills amongst them. Just fire up a server on AWS and away you go.
• jasonwocky 11y ago
  Can someone tell me what realistic security problems this mode of operation introduces that can't be mitigated/avoided with sensible network and backup configurations?
  • darkstar999 11y ago
    Precompiled binaries from random sources is a major security concern.
• lighthawk 11y ago
  > Update: it was pointed out that this started way before Docker Yes, like in the 90's, at least, when people started using Java. Even prior to Maven there were jars, and we didn't really know what was in them. And prior to that, I didn't understand how every piece of software or hardware worked. I was a big proponent of Gentoo when it came out because of building everything from source, but the fact is: I don't have time to look through and understand every line of code. Even compilers can and have injected malicious behavior in the past. Firmware cannot even be trusted. Some level of trust and reliance on others needs to be there. While it is true that there will always be people that betray that trust, without the trust, we would be hermits living alone off the land- which may not be so bad, but that's another story.
• gibsonje 11y ago
  I thought this post was overly cynical and full of generalizations. I don't really understand what point is trying to be made here. "Everybody" "Nobody" "Nobody" "None of" "everything got Windows-ized" every sentence is a broad generalization on top of cynicism so it's hard to find any value in the point trying to be made.
• fab13n 11y ago
  It seems rather easy and effective, for NSA-like agencies, to hide crude exploits in complex projects. An unintended effect of Snowden's whistle blowing is that it has become easier, because it has let them know that they don't need plausible deniability requirement anymore. Until Snowden, they were very cautious not to be caught, because, you know, what might happen if public opinion knew what a bunch of crooks they were? Now, they know that public opinion doesn't really care, and that if they're caught, they can mostly shrug it off, with politicians' complicity. So, shoving a rather crude and detectable exploit in a messy product has become practically doable. If I were in charge of subsidies distribution for some 3-letters agency, I'd pour more money on Docker, Maven etc. than on TLS.
• tux 11y ago
  Nicely said, I thought I'm the only one who noticed this ^_^ This is one of the reason why I tried Docker/Vagrant images few times and said no thanks :) I would rather spend my time and install everything on separate server my self then have unknown set of packages or security holes. As few articles on HN shown this containers not secure at all.
• exelius 11y ago
  System administration is as important as ever. Docker and other containers just simplify system administration across many different machines. The standard Unix user land tools are excellent and very flexible, but they are fucking god awful at configuration management. Docker solves the problem of "how do I make sure I have the same versions and configurations of everything on all 500 of my compute nodes without having to lock them down completely?" This question is meaningless if your base system image sucks, so you still need a proper sysadmin to build your Docker images. Many places are rolling this type of sysadmin work up into DevOps. This scares graybeard sysadmins, because they see DevOps automating them out of a job. What they fail to see is that DevOps is a step up for them: it's an explicit admission that system administration is as important as software development, and needs to be integrated into the software development process and managed through whatever management processes and tools the core dev team uses. The ultimate driver behind this is a shift in the way technology organizations are managed. A few years ago, you would have functional silos: development, operations, product, etc. that would all contribute to one or more products. Employees reported up through the functional lead, and incentives were doled out based on cost effectiveness. This didn't work well. So what started happening is that engineering executives began building product-focused silos instead. A development manager is no longer in charge of just software developers, but also QA, scalability and deployment. If the operations folks fuck up the deployment, the development manager gets chewed out about it. So the dev manager is going to bring as much of that under her control as she can. Docker/Maven/etc. are the abstraction layer between the teams that manage the infrastructure (physical servers, VMWare pools, storage, network, etc) and the teams that manage the applications. This is no excuse for bad sysadmin practices; you still need good sysadmins in the DevOps role. But here's the kicker: DevOps often pays more than system administration! And if you're a SME in a very specific thing (say, Cassandra administration) you can be in a support role across a number of different teams, making sure their DevOps folks deploy Cassandra in a sane way. (Yes, I realize all of this is centered on huge companies with massive engineering organizations. Small organizations have always required sysadmins to wear multiple hats, so none of this is new.)
  • erikb 11y ago
    Sorry, I don't see a response to the key point here. If Docker doesn't sign its containers and doesn't check signatures before applying one to a running system, then it's simply not secure. It may be one little feature of all the things Docker does for you. But that's what's lacking according to the blog post's author.
• fromtheoutside 11y ago
  And before "curl | sh" it was download from freshmeat and run "tar xfz; cd; make install". It's not really better. Fact is we are running huge and complicated frameworks with lots of dependencies. These technologies are new and evolve fast. Distros don't have enough volunteers to decouple this mess and thus fail to provide stable packages. There is a good chance nobody wants an old version of hadoop anyways. Containers are a whole other problem. Always bothered me that no one cares about building these images themselves. The documentation is there, you can build your own docker/vagrant/... containers and vms. It's just nobody seems to care anymore? Sometimes I don't even know where these images come from, distro, community, ...?
• vbezhenar 11y ago
  Looks like description of the projects with bad build system, not like problem with e.g. maven. Maven downloads binaries from the HTTPS server. You can always get those libraries and rebuild them from sources into your internal repository.
• tobz 11y ago
  An interesting point that I didn't see the author bring up is the concept of how Docker images can be built in a layered fashion, and the potential for a false sense of security. For example, you start with some sort of base image -- say phusion/baseimage-docker[1] -- and proceed to layer your application on top of it. You "trust" Phusion. They do Phusion Passenger, it's a real piece of software you heard of, and it's not some random person on the internet. At some point, there's a bug, a problem, a security flaw, and you're waiting on them to fix it... nothing, nothing. Maybe they get hacked and their base image is now infected. I haven't bothered to look, but I'm guessing it would be a trivial amount of work to start the process of culling the most popular base images used by public Dockerflles, looking for the biggest trojan horse. It seems like the whole model is ripe for pushing an understanding of what is actually running on a machine -- soup to nuts -- to the way side, and establishing a non-existent trust on the building blocks you're using, lulling people into a false sense of security about their containers. A lot of people already believe that they're already doing something much more secure by running containers, and arguably, they are... except for all of the places where malicious software can be added in, and the potential container breakout techniques. [1] https://github.com/phusion/baseimage-docker https://github.com/phusion/baseimage-docker
  • andrewvc 11y ago
    FWIW you can easily recreate a base image by just copy/pasting the Dockerfile for that image at the top of your own. I did this for the Jruby images we base our stack on. I've been doing both dev and ops work for nearly a decade. I feel for what the guy is saying, but these aren't tech problems, they're process problems. Relying on apt packages for everything makes using more recent features ridiculously hard and slows up the works in pushing features out. I'll trade a little security to be more nimble. I say that because as someone who's worn the hats of operations, development, and co-founder, I realize that you can't have it all. There simply isn't enough time and bandwidth in most companies.
    • tobz 11y ago
      Sure, and that's all reasonable stuff. I mostly posted this because while encouraging people to use wildly insecure installation processes like 'curl ... | sudo bash' is terrible, it's easily recognized as being terrible. To me, the Docker ethos is, perhaps, deceptively bad in terms of security. Deceptive enough that it it can lull people into a false sense of security, etc etc. I mean, we'll see if it happens. My fears might be entirely unfounded, or phusion/baseimage-docker might get trojaned. Who knows. :P
• patsplat 11y ago
  Is it a coincidence that all the technologies the OP complains about are Java (Hadoop, Apache Bigtop, Maven, ivy, sbt, HBaseGiraphFlumeCrunchPigHiveMahoutSolrSparkElasticsearch)?
  • mordocai 11y ago
    I imagine they are just involved with the java ecosystem so the examples they know involve java.
  • derefr 11y ago
    I don't think it's a coincidence. The Java ecosystem is intentionally isolated from the Unix ecosystem, because one of Java's goals was portability in an age when Windows, Mac, and Linux were all very different operating systems with very little in common. Java has its own Java-y build infrastructure, which relies much less on the concept of "trusting the source", and much more on the simple fact that the JVM is a sandbox that can be tuned to whatever security requirements the sysadmin desires. Running Java apps (especially Docker-ized Java apps) is less like installing a Unix package (even if it's masquerading as doing so), and more like starting an instance of some untrusted VM image on your (software-defined-)network. It can use some of your computer's resources, but it has no permissions to touch any of your data or services unless you grant them to it. It really is like an app, or a web page.
  • MichaelGG 11y ago
    Node/npm should get a mention. On a simple static website I've seen, a couple of grunt tasks end up pulling in over 14,000 files. And a hugely nested directory structure. Part of it is the ... interesting ... idea that individual functions should come in their own module. Some npm modules are literally 6 lines of code. But they get packaged up just like everything else. There's no concept of having a stdlib or something. (Apparently node/v8/minifiers aren't smart enough to do a good job if you use a stdlib.)
• cyberpanther 11y ago
  Because Docker is good at containing crap, it is used to cover a multitude of sins. Before using Docker, please simplify your install and upgrade processes.
• markbnj 11y ago
  I think the author is mixing up a few different topics. If you're going to blame container frameworks for people sharing software in insecure ways you might as well blame the fact that executables are portable between compatible systems. Might as well blame the fact that there's a network while you're at it. We run docker throughout our infrastructure, but it is a deployment and dependency management technology, not a vector for infection. We run only our own images, which are all built from source or validated binaries. So what do insecure or unreliable practices have to do with containers, specifically?
  • xofer 11y ago
    The article is not an attack of Docker, but the way it's being used by many.
    • markbnj 11y ago
      >> This rant is about containers, prebuilt VMs, and the incredible mess they cause because their concept lacks notions of "trust" and "upgrades". Oh, ok.
• parasubvert 11y ago
  If you want to have guaranteed runtime linkage built from trusted source, you might want to give BOSH (http://bosh.io http://bosh.io) a look for config/release management - it insists/prefers on compiling all dependencies from source, from trusted links, with signature checks. For example with Hadoop, here is the build script: https://github.com/cf-platform-eng/hadoop-boshrelease/tree/master/packages/hadoop https://github.com/cf-platform-eng/hadoop-boshrelease/tree/m... Learning curve is a bit steep but it's another approach to this immutable infrastructure trend that's built for large production, enables rolling canary upgrades, etc.
• michaelochurch 11y ago
  I don't have a strong opinion either way about Docker, but I understand the OP's gripes. Stack is the new term for "I have no idea what I'm actually using". This was great. It leaves me to wonder what "full-stack" means. For one thing, we have a culture of trust inversion. I wrote about it in a blog post about a month ago: https://michaelochurch.wordpress.com/2015/03/25/never-invent-here-the-even-worse-sibling-of-not-invented-here/ https://michaelochurch.wordpress.com/2015/03/25/never-invent... . The "startup" brand (and it is a brand) has won and most companies trust in-house programmers less than they trust off-the-shelf solutions. This tends to be a self-fulfilling prophecy. Because few corporations will budget the time to do something well (make it fast, make it secure, make it maintainable) it only makes sense to use third-party software heavily and use one's own people to handle the glue code, integration, and icky custom work. (That, of course, leads to talent loss, and soon enough, when it comes to build vs. buy your only option is to buy, because your build-capable people are gone.) At some point, however, you end up with a large amount of nearly-organic legacy complexity in your system that no one really understands. Although it's not limited to one language or culture, this is one of my main beefs with Java culture. It has thoroughly given up on reading code. Don't get me wrong: reading code (at least, typical code, not best-of-class code) is difficult, unpleasant, and slow and, because of this, you invariably have to trust a lot of code without manually auditing it. But I like having the idea that I can. The cultures of C, OCaml, Haskell, and to a degree Python, all still have this. People still read source code of the infrastructure that they rely upon. But the Java culture is one that has given up on the concept of reading code (except with an IDE that, one hopes, does enough of your thinking for you to get you to the right spot for the bug you are fighting) and understanding anything in its entirety is generally not done.
• ExpiredLink 11y ago
  > Maven, ivy and sbt are the go-to tools for having your system download unsigned binary data from the internet and run it on your computer. Not Maven.
• skywhopper 11y ago
  I agree that many of these convenient setups are embarrassingly sloppy, but it's the sysadmin's responsibility to insist on production deployments being far more rigorous. No one can tell you how to build hadoop? Well, figure it out. Random Docker containers being downloaded? Use a local Docker repo with vetted containers and Dockerfiles only. I don't even allow vendor installers to run on my production systems. My employer buys some software that is distributed as binary installers. So I've written a script that will run that installer in a VM, and repackage the resulting files into something I'm comfortable working with to deploy to production. If a sysadmin is unable to insist on good deployment practices, it's a failure of the company or organization or of his own communication skills. If a sysadmin allows sloppy developer-created deployments and doesn't make constant noise about it, then they aren't doing their job properly.
  • Nursie 11y ago
    >> No one can tell you how to build hadoop? Well, figure it out. I get the impression that several people working on debian couldn't work this one out!
    • azinman2 11y ago
      Surely at minimum Hadoop developers could tell you!
      • marcosdumay 11y ago
        Have you even been in a project where the developers didn't know how to build it? It's a strange situation, with huge environments being passed from one computer to another, and treasured with more care than the code itself.
        • StillBored 11y ago
          This happened to me about a decade ago. A very smart sysadmin in the company created an acronis image for machine deployments. They very carefully documented everything they changed, and how to recreate it. Then someone else created an image from one of the imaged machines without documenting what they changed. This happened a couple dozen or so times until the image pretty much was a mess of hand installed binaries, configuration hacks, etc. It literally took another person 6 months to untwist what was actually on the machine by md5suming the crap out of everything guessing at versions until they found a match, and documenting it. That sounds like the state of a lot of docker images.
• api 11y ago
  70s system software is really showing its age. Containers are just a hack to make its complexity sometimes easier to manage.
  • quotemstr 11y ago
    Emperor Joseph II: My dear young man, don't take it too hard. Your work is ingenious. It's quality work. And there are simply too many notes, that's all. Just cut a few and it will be perfect. Mozart: Which few did you have in mind, Majesty?
• eranation 11y ago
  As a Java / Hadoop / Spark / Scala fan, all I can say is, it's a little embarrassing, not sure how the Java ecosystem around hadoop became so sloppy (I witness it first hand on a daily basis). I wish more people who are concerned with security / ease of build would turn into contributing to maven, sbt, ivy and the hadoop project. Instead of hating the Java ecosystem, why not join it and make it better? Hadoop is ubiquitous, maven (and ivy / sbt) are the de facto dependency management and built tools for that ecosystem, and if it's broken, (or alienating people who are used to just have make / rpm / deb for anything) then those people should join and try to make it better. Whether you like java / maven / ivy / sbt or not, good chances you'll end up forced to work with Hadoop (Java) or Spark (Scala), both of which use maven / sbt for dependency and build. I say, It's all open source, if it's broken, and you know where it's broken, I think the Hadoop / Java community will be happy to get suggestions / pull requests to improve it.
  • fs111 11y ago
    In theory you are right, unfortunately at least the hadoop "community" is difficult to work with. Hundreds of JIRAs with patches in limbo state for month/years b/c nobody of the paid developers at cloudera/horton bothers to take a look. Also the political things happening behind the scenes are way more complex than you might think. It is frustrating...
  • deleted 11y ago
    [deleted]
  • the8472 11y ago
    I for one like java/scala but am quite wary of maven&co. All I need is a build tool. I can download a bunch of jar dependencies myself if needed. In the "old days" we just put them into the CVS repository or in a tarball to download along with the source and an ant script to build the whole thing. Granted, this did lead to outdated libraries at times and thus potentially was a security threat in itself. But at least it didn't roll the concerns of obtaining dependencies and building into a single tool. Often it's nice to just get the dependencies and build the project your own way, e.g. in your IDE of choice. Or vice versa, obtain the depdencies on your own (e.g. plugging in a different version than recommended) and then use the automated build. I think those things should be provided by separate, simple tools.
• devmonster 11y ago
  The problem is you old sysadmins are so passé. Software has replaced you, and you need to get over it. Developers are finally liberated to move at full speed without hearing "NO"
  • lafar6502 11y ago
    there certainly are sysadmins that build their authority and power only on having exclusive access to root account.
• smutticus 11y ago
  On the one hand it's nice that developers are using more libraries and writing less from scratch. On the other hand dependencies are out of control. The one thing that really bothers me is code that depends on a particular IDE. That shit drives me up the wall.
• JoergR 11y ago
  Back in the day we would install packages by making holes into a cardboard card with our own teeth! And look what these kids do today! Get off my lawn!!
• moonbug 11y ago
  Today I learned that the "curl | sudo" idiom is actually a thing people really do. Truly, everything is awful.
• devy 11y ago
  The 'curl | sudo bash' mention reminds me of OS X Homebrew. The one-liner installation script is still published in the home page front and center (though it's running from a trusted source Github). http://brew.sh/ http://brew.sh/ One might argue that that easy of oneliner installer script is exactly what make Homebrew gains popularity. And dev machine is different from production environment in terms installation packages. Still, I agree that proper container management in local network and perhaps new security features from upstream container vendors would help the situation.
  • squar1sm 11y ago
    I agree, I think accessibility made it popular. Security and ease of use are usually opposing forces. The article has some interesting discussion points. I don't understand the absolute fear of | bash installers. It's open source, read the script. That's the argument people make for `./configure; make; make install` programs. I think it's because it's new, or it's too easy. But the article does have a point about trusted containers. But security isn't a download or a product anyway. Security isn't even guaranteed.
• taude 11y ago
  I don't think any experienced organization is going to just download containers off the internet to use on their servers. Which is why there is self-hosted Registry applications that corps and big companies buy to host their own, where they build their images to support their applications and that are vetted through traditional corp policy
• zupa-hu 11y ago
  I'm a bit puzzled. Let's say I decide to not download the binary but build it from source. Unless I actually read the source, I'm trusting the community to have read it, which consists of other people thinking I have read it. In my view, this is true for the OS itself. So unless I read everything, I'm fucked. And I don't. Thus I'm fucked. Do I miss something?
  • Gigablah 11y ago
    This is also pretty much what happened with OpenSSL. Which is why I'm amused by the holier-than-thou attitudes in here.
• reeboo 11y ago
  This is the price of devops.
• myth17 11y ago
  For people who are interested in learning more about the problem. This is a really great paper : https://www.informatik.tu-darmstadt.de/fileadmin/user_upload/Group_TRUST/PubsPDF/BNPSS11.pdf https://www.informatik.tu-darmstadt.de/fileadmin/user_upload...
• dysinger 11y ago
  This 1 page poorly titled wrong rant is the #2 story on this site? "Ever tried to security update a container?" lol. you are doing it wrong. "Essentially, the Docker approach boils down to downloading an unsigned binary, running it, and hoping it doesn't contain any backdoor into your companies network." nope https://blog.docker.com/2014/10/docker-1-3-signed-images-process-injection-security-options-mac-shared-directories/ https://blog.docker.com/2014/10/docker-1-3-signed-images-pro... "»Docker is the new 'curl | sudo bash'«" no it's not. most intelligent companies are building their own images from scratch. People that care about what's in their stack take the time to understand what's in there & how to build things.
  • chrissnell 11y ago
    I think you're wrong. I think most users are not installing trusted builds from their OS vendors. Piping curl to bash is incredibly common--many popular software packagers are doing it [1]. About a year and a half ago, I was playing around with Docker and made a build of memcached for my local environment and uploaded it to the registry [2] and then forgot all about it. Fast-forward to me writing this post and checking on it: 12 people have downloaded this! Who? I have no idea. It doesn't even have a proper description, but people tried it out and presumably ran it. It wasn't a malicious build but it certainly could have been. I'm sure that it would have hundreds of downloads if I had taken the time to make a legit-sounding description with b.s. promises of some special optimization or security hardening. The state of software packaging in 2015 is truly dreadful. We spent most of the 2000's improving packaging technology to the point where we had safe, reliable tools that were easy for most folks to use. Here in the 2010's, software authors have rejected these toolsets in favor of bespoke, "kustom" installation tools and hacks. I just don't get it. Have people not heard of fpm [3]? [1] http://output.chrissnell.com/post/69023793377/stop-piping-curl-1-to-sh-1 http://output.chrissnell.com/post/69023793377/stop-piping-cu... [2] https://registry.hub.docker.com/u/chrissnell/memcached/ https://registry.hub.docker.com/u/chrissnell/memcached/ [3] https://github.com/jordansissel/fpm https://github.com/jordansissel/fpm
    • ossreality 11y ago
      Shit, it's going to blow your mind to know that people download EXEs from the internet and run them every fucking day. this thread is a joke.
• swills 11y ago
  I completely agree with this except I think of it more as a problem of release engineering rather than system administration. The trouble is, the sysadmin's job is to deploy things. The developers job is to write code. Often release engineering isn't thought of at all or if it is it's given to the last qualified or least suspecting folks without any requirement from operations. Developers aren't taught about release engineering or deployment in school at all. In fact, it seems to me most university curricula do everything possible to hide all that from students. Compounding that is the developers desire to get new code out conflicting with the sysadmins requirement to keep things stable in the face of limited QA automation. This leads to the common conflict between dev and ops. This is to me a large part of what has led to the DevOps movement. This gives the developers information about the deployment and perhaps even access to it or a version of it and/or a voice in deciding how things are deployed. Hopefully we can standardize things widely enough that universities can teach this without fear of focusing on useless technologies that will be discarded in 3-5 years.
• SrslyJosh 11y ago
  It's not just system administration... Minecraft is a meta-game about downloading unsigned JARs from the 'net and running them with your own user account.
• octref 11y ago
  I think it's because people are getting worse at explaining things and writing docs. As a student many times I wanted to learn how things work, but most tutorials/docs just ask you to type in a few magical lines without much explanation. Maybe the authors think their audience won't understand anyway, but I think it's the authors' ineptitude if they can't explain what their programs do in an accessible way. I really hope there could be more projects like i3[1] and flask[2]. 1: http://i3wm.org/docs/userguide.html http://i3wm.org/docs/userguide.html 2: http://flask.pocoo.org/docs/0.10/ http://flask.pocoo.org/docs/0.10/
• patsplat 11y ago
  Regarding curl PACKAGE | sudo bash... Just what do you think happens when you run `yum update` or Windows Update? If you don't trust DNS or the network then you have serious challenges which frankly aren't even solved by air gapping file transfers.
• bayesianhorse 11y ago
  Incidentally I feel like my admin "skills" have never improved faster than since I started working with docker. Docker lets me iterate on system configuration faster than ever, and that means learning the details and quirks of certain software faster. Then again, I usually don't use prebuilt VMs and containers, but have to prepare them for people who don't want to pay for good sysadmins..
• sorin-panca 11y ago
  This "working out of the box" phylosophy is the direct result of devs using Windows and OSX platforms for creating those programs. They now think "Linux and *BSD should be as easy to use as Mac.". Indeed, the majority of devs is mediocre amateur sysadmins. They know next to nothing beyond their preffered language.
• vorg 11y ago
  > And then hope the gradle build doesn't throw a 200 line useless backtrace This is more the fault of the language Gradle chose for its build configuration language. Most build scripts are between 20 to 50 lines long, but reading through those Groovy stack traces eliminates its supposedly write-one-read-many-times benefits. Hopefully Gradleware will fix this problem for Gradle 3. They've already enabled Gradle to be configured on the fly by Java code, and could by working towards allowing any dynamic language to be a build language through an API. Alternatively, they've just employed one of the ex-Groovy developers recently made jobless by Pivotal pulling funding from Groovy and Grails last month -- they might get him to write a better lightweight DSL from scratch that parses the existing syntax, but isn't weighed down by all of the present cruft.
• jheriko 11y ago
  A lot of big projects are terrible to build. Once upon a time minimising dependencies was considered good practice. Now I get a pasting if I write clean code without reusing someone else's library... even if the suggestions don't solve my problem directly or at all and comes complete with a sloppy 'no one click build/deploy' configuration... the kind I was embarrassed to produce on my stand alone projects in my teenage bedroom days. Shame on developers everywhere for tolerating this mess. I (am lucky enough to enjoy the freedom of choice that I) would leave a job if not allowed to start fixing such a situation from day one. That being said, good sysadmins and developers should work out these problems properly instead of shortcutting through someone else's half arsed effort via Google.
• halfcat 11y ago
  As a Windows/VMWare/Exchange/Cisco admin, this is all completely foreign to me. Does this indirectly make a case for paying a vendor real money to manage their product properly? All vendors we work with provide installers that handle installing any dependencies. Updates are a few clicks. Occasionally we run into a vendor who provides installation/upgrade instructions, that involve manually copying files and hand editing config files. We replace those vendors. Error-prone people should not be doing manual file copying/editing or dependency checking, tasks that computers are dozens or orders of magnitude more competent at. This is B2B stuff where businesses should manage their product or risk getting sued out of business. The current environment seems to be that using free or opensource products is "free, with purchase of a team of consultants". Why not just pay the money to a vendor to provide, and support, a real product? It seems backwards to call this a sad state of sysadmin. This is like Boeing providing its leftover parts and a 9000 page manual on 747 assembly, and people complaining about the "sad state of mechanics". That's backwards. Buy a 747 from Boeing if that's what you need.
• bmoresbest55 11y ago
  So I like this article and want to learn more about sys admin/dev ops/whatever but where do I go. Is docker bad? What is a good starting point? What are best practices?