24 ms·

This is one weird comment section. There are people attacking the author for a statement made about CoreOS, and for some hate towards Kubernetes. The key poin

by mvanga 6y ago

This is one weird comment section. There are people attacking the author for a statement made about CoreOS, and for some hate towards Kubernetes. The key point of the article is not really being addressed here: vested interests from large companies are able to introduce huge complexity into simple, well-designed projects. While the complexity may be good for some end that said vested interest has in mind, they are also in a position to absorb the cost of that increased complexity. In the meantime, the simpler version of the software is long gone, with the added complexity placing a large burden on the solo developer or small team. It's almost like there's no good representation in the open-source world for the solo developer or small team. Funding and adoption (in the extreme, some might say hijacking) of open-source projects from large corporations dictates the direction of all major software components today. Along with the roses come some very real thorns. Just my 2c.

microcolonel 6y ago
A major problem that leads to dysfunctional bloat is the inherent ego traps of software developed by teams of well-adjusted people. Even if it's for the good of the quality of a product, it is not often easy telling your team mate that they wrote an unnecessary component that should just be deleted. I have done it lots in my short career (and in terms of engineering outcomes, it's been spectacular), but it's easier for me I think, since I'm more disagreeable than the average person.
pwdisswordfish2 6y ago
"... of all major software components today." Thankfully, we still have the "minor" components to ourselves. Popularity has its downsides.
jchw 6y ago
The problem, as I pointed out elsewhere, is that the post makes very few constructive comments and presents a lot of opinions and assertions without justification or reasoning, making it extremely hard to actually make meaningful comments about. As someone else pointed out, if it were an HN comment, it would be flagged. I don’t agree that this is a statement about complexity, so much as it is a rant about the author being inconvenienced by technology they personally deem pointless and unnecessary. If the CoreOS comments aren’t a sufficient red flag, the HTTP/2 ones really ought to be. Also, some etcd users debate their assertions about newer versions being more complex internally. On the other hand, its difficult to argue with overly cynical type attitudes.
mistercow 6y ago
It doesn't seem to me like the comment section is weird at all. This is kind of a perfect object lesson on pitfalls of writing a persuasive blogpost. Basically, if you want to argue for a somewhat controversial conclusion, you shouldn't back it up with even more controversial and unsupported (or weakly supported) premises. Particularly if "even more controversial" means "in disagreement with the vast majority of your industry."
dilandau 6y ago
I think a large problem is open source projects actively marketing themselves, and seeking large user bases. This leads to a wasteland of zombie frameworks, and developers selling out (so to speak) in exchange for the reputation. It is important to note, too, that etcd is permissively licensed and anyone can run the old version in whatever way they please. The final, and biggest, takeaway? Hackernews apparently has drank the koolaid and will reach for any convenient justification to write off an opinion they disagree with. Stockholm syndrome all through the comments. Conservatism and pragmatism apparently are no longer current in the cloud economy.
kelnos 6y ago
> There are people attacking the author for a statement made about CoreOS, and for some hate towards Kubernetes. That's not really surprising, and maybe should be a lesson to the author. If you want to make a particular point, you shouldn't make inflammatory/controversial statements about other, only-tangentially-related things. It distracts from and dilutes your point. For my part, I know one of the CoreOS founders, and while I have no opinion on CoreOS itself, it's really off-putting to see some random armchair quarterback on the internet shitting on the work of someone you know. While I more or less agree with the actual real point of the article, the unnecessary attacks really divided my attention. The article starts off with a cheap potshot at CoreOS, followed by a screed about Kubernetes, and then finally, more than half-way through, we get an admission that the author has "digressed", and we get to the actual point. Certainly the point required some background about Kubernetes, but I can understand why people focus on the negative CoreOS and Kubernetes aspects of the post, since they consist of more than half of the content.
- growlist 6y ago
  Eh, idk. It makes for entertaining reading for a devops-sceptic.
  - edude03 6y ago
    Off-topic, but I've never seen anyone describe themselves as a "devops-sceptic" - could you elaborate more on what that looks like for you?
    - growlist 6y ago
      You're just asking so that you can attack my response, I suggest.
      - edude03 6y ago
        While I know there are people who argue in bad faith on the internet, I am genuinely curious and am not looking for a fight. I would have reached out to you privately but you don't have an email or contact listed on your profile. I'm mostly curious about which part of devops you're skeptical about, if it's tooling, or process, or the mentalities or something else.
rodgerd 6y ago
> The key point of the article is not really being addressed here: Perhaps if the author wanted their key point addressed they could have tried focusing on it, rather than on an ill-considered spray across a group of largely unrelated bugbears.
c3534l 6y ago
There was some generic big-company hate thrown in there, but the author really didn't say anything half as coherent in the OP as you did.
dayjobpork 6y ago
When you fine some cool tool you can: 1. Use as is. 2. Make behind the scenes changes that don't alter functionality. 3. Make changes that add functionality. 4. Make changes that remove existing functionality. 4 is the problem. It's like joining a car club then keeping pushing to have petrol cars banned. It is really shitty to find a cool tool that you can use, then try to force remove features that existing users use.
shadowgovt 6y ago
Why can't solo developers or small teams represent solo developers or small teams? If an etcd that has a thin, inflexible API is of value, a small team should be able to fork and manage it (the notion it's smaller and simpler ought to imply it's a lot cheaper to maintain than a multi-API etcd, right?). If the economics of the situation is "large teams have to maintain tools for small teams and individual users," that doesn't actually scale well, does it?
withinboredom 6y ago
They haven't gotten to beanstalkd yet...
jnxx 6y ago
> introduce huge complexity into simple, well-designed projects why does cmake come to my mind?
bfb09 6y ago
It's a shot across the bow at the Kool-Aid which everyone here has imbibed to some extent. Of course they're going to shoot the messenger. When I clicked this thread I said out loud to my team, "oh, this should be good," and I hadn't even fathomed precisely how predictable the discussion would be. It's so far removed from any consideration that it lands something like explaining to an American that they're a subject of the Queen. It makes no sense to objectively review the thinking when you're inside, and buy, the thinking. Bear in mind the livelihood of nearly everyone participating in this conversation is impacted by the situation being lamented. It's telling that CoreOS employees felt compelled to set the record straight and casually "float ideas". It's further curious to note the difference in HN's reaction to an extremely salty and offensive post from, say, Linus Torvalds, versus this one. People are really bad at objectively setting aside their ideology and tone is a kind excuse to get out of having to genuinely think about whether the angry individual might have a point, and when it hits close to home, well. All bets are off. You're 100% correct on your overarching point but the less kind interpretation is "the incentives of modern software engineering value complexity to artificially strengthen a software engineering job economy". The good news is that the more Big Tech screws the pooch with user data management and software engineering, the stronger an argument I have among very receptive ears in government circles to start thinking of Big Web/FAANG as hostile to responsible computing innovation, because they never really bought what Mikey sold them, and the troupe that followed Mikey to Washington didn't hang around to explain the downsides or long-term management needs of the current landscape.
- mekoka 6y ago
  You should also consider that the contrarian nature of HN would set it up to exactly the opposite reaction had the post expressed the reverse of its current positions. The big irony is that it's not inconceivable that it would involve some of the same actors voicing different opinions. > "the incentives of modern software engineering value complexity to artificially strengthen a software engineering job economy" Not sure I would agree. Hanlon's razor comes to mind. To me, the referred complexity is reminiscent of a number of other complex patterns that somehow found their ways in software engineering, out of possibly good intentions. Regardless, an idea needs to be sold and when the marketing is well executed the community will ingest the kool-aid. Which it then tries to digest for a few years. Then begins a game of chicken that consists in not being the first to admit that you're in pain, out of fear of being branded an infidel, or out of sunk cost bias. Eventually a few courageous and influential souls finally spit it out in disgust and scream "wtf is this shit?!?" Then the rest awakes from the common stupor and a holy war ensues. I still remember SOAP. I still remember with what verve I was sold Scrum. As for Kubernetes, only time will tell.
simonjgreen 6y ago
Couldn't agree with the sentiment of this comment more.
dynamite-ready 6y ago
Agreed. I saw it as a general lament against over-engineering. I don't think the point got lost in the super specific example... You could just as easily level similar rants against the likes of React and it's wider ecosystem, Tensorflow, Typescript (many will disagree), Docker... I'm sure others have their own bugbears. Much of this is subjective, of course. But to me, it feels like software development is trending towards unecessarily complicated development processes and architectures. And the only beneficiaries appear to be the large technology companies. I suppose in exchange, you're getting a guarantee of maintenance. But is that really worth the additional complexity associated with the common use of these tools?
- discordance 6y ago
  I attribute this companies that can afford to keep way too many high skilled employees on the books while needing to keep them occupied
  - Polylactic_acid 6y ago
    It seems to me the problem is all of our software is built by and for megacorps. In the context of what they need it for its the best tool for the job but it was never made for smaller orgs where 90% of the bells and whistles are not needed and end up as needless complexity.
- Polylactic_acid 6y ago
  I'm not sure how react fits in here. I have found it to be very simple to understand enough to be productive and upgrading to new versions has required next to no changes to our large codebase. Perhaps you mean SPA react apps which I agree are a bit more work than needed for a solo dev. Docker seems to be optimal for medium size teams. Configuring a server the old way is easier than docker for 1 server but less so for when you have 10 or CI/review apps.
  - earthboundkid 6y ago
    React fits in because it's in the process of jumping the shark because FB needs it to. React is already hugely bloated for no discernible reason compared to Preact, but they're just going to keep bloating it with Suspense. Cf. https://crank.js.org/blog/introducing-crank https://crank.js.org/blog/introducing-crank
majormajor 6y ago
When I've worked on small teams we avoided complexity by hardcoding locations and using GUI management tools and all these other things that still exist - and honestly, are better today than they were in the past, when you'd also need to get yourself a rack in a datacenter to run stuff. I think there's a big swath of people adopting fancy tools to try to solve problems they don't have and won't have for years, but I don't see that having destroyed the ability to do things the easy way.
tyingq 6y ago
Meh, having your opener call out a specific project as bullshit has that effect. Pulling the snark would have resulted in more people getting the main point.
- bfb09 6y ago
  Perhaps the people who treat CoreOS being called bullshit as dispositive of the entire thesis, and an inclusion worthy of disregarding the author and thinking of him in any way other than making an argument, aren't the intended audience. Pull the snark out and there will remain any element of this post that kicks off a violent HN centithread that conveniently avoids the core issue. You know it, I know it.
  - tyingq 6y ago
    Snark directly related to the main point is fine. The CoreOS comment is unrelated. In fact the author liked etcd in the time period when it was directly associated with the "bullshit" project. And even mentions "but that doesn't really matter". Bashing K8S is at least related to the main point.
- mvanga 6y ago
  Definitely true. The author could have chosen differently and it would have probably led to more constructive comments. However, I'll also point out that focusing on the snark and ignoring the rest of the article is also a choice made on the part of the commenters. Given a choice, I personally prefer to take the strongest and most charitable interpretation of someones ideas before choosing to criticize them. At least the discussion is interesting that way...
  - saagarjha 6y ago
    That things I write for the web usually go through a "lazy comment" filter where I pull out references that would lead to lazy comments. Things that often get stripped are one reference to a specific example when I'm talking about something general–because invariably the comments will focus on "how I don't get the software" or "how I singled it out" or "how what I wrote isn't even true if you look at this way so my point is now invalid". Either I mention no examples, and have the commenters come up with their own, or I have a few so that I don't get attacked by a specific group of people.
fwsgonzo 6y ago
I think everyone here remembers that time they wrote a simple HTTP server, that just works, and would continue to work today. If anything, I hope that we can recognize that there has been advancements in computing. Back in the day you would be open to attacks left and right. It was the wild west back in the day. Today, you can take that tiny webserver and put it in a VM, and voila, now it doesn't matter if anyone breaks in. As long as you didn't add the SSH key to github or anything like that. We can have complex programs, as long as every-man doesn't have to know the details behind it, or implement it or risk losing access he previously had. I think new protocols that are extremely hard to implement falls under that, and I don't think Qemu or Linux does, as nobody loses anything when Linux adds new complex features. We have to make sure, at least, that we continue to support HTTP/1. I think we already lost a lot when we moved away from IRC.
- Animats 6y ago
  FCGI controlling Go programs, maybe front-ended by a load balancer and back-ended by MySQL/MariaDB/Postgres, can get an awful lot of work done cheaply. FCGI is an automatic "orchestration system" - it starts up more instances as needed, restarts them if they crash, shuts them down when they go idle, and restarts them once in a while in case they leak some resource. It's like the parts of Kubernetes you really need. Go is an OK language. Not a great language, but an OK language. It comes with almost everything you need for server-side applications. Also, the "goroutine" model means you don't have to fool with threads vs async. If a goroutine blocks on a lock, no problem; the CPU gets redispatched. If you need a million goroutines, that works, too. None of that "oh, no, the async thread made a call that blocked and screwed up the whole system". MySQL/MariaDB/Postgres have been around for decades, and they work pretty well. You can have replication if you need it. (NoSQL seems to have come and gone.) Get the stuff at the bottom right, instead of piling more cruft on top. It's cheaper.
- bfb09 6y ago
  What if 'advancements' were anything but? How would we know whether we're advancing a nascent engineering discipline in the best way possible? Is public cloud, an optimization for a certain calculus of engineering resources, the optimal form of computing delivery? There's an underlying presumption about computing progress that has weaseled its way into your thinking, and the devil of it is that there are vanishingly few people willing to question it (can hardly blame them, given this thread).
cryptica 6y ago
>> It's almost like there's no good representation in the open-source world for the solo developer or small team. As a solo open source developer who maintains a popular OSS project, this is 100% the case. Sometimes it feels like big corporations are actively working against us. It's almost impossible to get any coverage at all on any kind of media. No matter how much organic growth your project has. My project has been growing steadily and linearly for almost a decade. Now it has almost 6K stars on GitHub so you'd think that this by itself would draw some attention? Not so. It's almost impossible to find it on Google search results. Most users find the project through direct word of mouth from other users of the project. We've never received a single consulting contract from any corporation.
- IggleSniggle 6y ago
  so what's the project?
  - jauco 6y ago
    I think it’s socketcluster. However, i’m not quite sure what it does (it obviously does something really well, but I’m probably not precisely the target audience) It might help adoption and re-share value if your readme starts with some problem and shows how your tool solves it. The cadence/temporal people did this really well: https://www.temporal.io/ https://www.temporal.io/
- mzz80 6y ago
  You’re going to be bitter and resentful for a long time if you believe that the fact that you haven’t received a single consulting contract is due to big corporations working against you. It sounds like you’ve built a useful piece of software. However, you don’t get handouts because you’ve built something. Those who are successful actively promote themselves and their products. There’s a reason companies aren’t comprised of just software engineers.
013a 6y ago
Then the author should have focused the article. If their goal was to address that, then don't take a left-turn into hating on Kubernetes for no reason, before even reaching the point trying to be made. > Funding and adoption (in the extreme, some might say hijacking) of open-source projects from large corporations dictates the direction of all major software components today This is because large corporations are the biggest users of these pieces of software. Its literally that simple; money only plays a smart part of it. Etcd's biggest user is Kubernetes. I would estimate that 2nd place isn't even close. I would estimate that Etcd wouldn't even exist today if not for Kubernetes. The direction of open source projects is usually some approximation of a democracy; its users contribute the code they need. If they need gRPC, it gets added. gRPC wasn't added for no reason, or because politics; its a sister CNCF project, and would increase the performance of the API surface, which greatly benefits all of its users. Lets do a little experiment: Go start an etcd clone. Just go do it! The code is still there! It would be four clicks in the Github UI. Strip gRPC, add the HTTP API, and maintain it. I'd bet a thousand dollars the author won't do it, for two reasons. First, the author hasn't actually used etcd. I know this because, as I said, startlingly few people outside of Kube use etcd directly, and the author isn't providing any substantive technical reasons why the gRPC change was bad; just philosophical ones. And Second, because maintaining large open source software projects is soul crushingingly, destructively, enormously hard. Its way too easy to armchair-quarterback decisions like these based on your own personal flavor of ethics, then cherry-pick the evidence you want to say "corporations are evil! they destroyed etcd by getting rid of HTTP!" Meanwhile, people out there do develop it, and use it, and service billions of requests, and deliver value to their users, and sometimes, turn off their computer and cry because despite all of the hard work and positive results people with no skin in the game still pipe up to say what they're doing is horrible. Did the author join in on the discussion when these changes were made? Did they voice their concerns? Or did they just read about it after-the-fact and say "jeeze, that piece of software I played around with two years ago really took a turn, I'm going to write a post about how evil corporations are."
- pessimizer 6y ago
  > maintaining large open source software projects is soul crushingingly, destructively, enormously hard. Are you making a case in this package that complex open source projects are impossible unless completely controlled by very large, highly opinionated companies? That can be proven wrong by endless examples. > Did the author join in on the discussion when these changes were made? Did they voice their concerns? Or did they just read about it after-the-fact and say "jeeze, that piece of software I played around with two years ago really took a turn, I'm going to write a post about how evil corporations are." You don't know this to be true, so why make this accusation? Why not address the argument as it is instead of inventing ad hominems?
Florin_Andrei 6y ago
But that's just the "circle of life" no matter what. Things start very simple, and then inevitably get ambitious and try to add new gizmos. Sooner or later you end up with a monster. Over and over again.
- solumos 6y ago
  This is true - but I think the interesting (and understated) part of this argument is Big Tech's influence on the process. Say I want to fork etcd or build a greenfield alternative - I now have to compete with (or support) the Hivemind's preferences in direction/progress. Who's to say their way is the "best" way? I think this tension is much greater than it was 10 years ago.
- wahern 6y ago
  Historically FOSS has been far more resistant to this cycle; not all FOSS projects, but many. It's much easier for FOSS projects to just say no, and they can spend years or even decades refining things. In the commercial world you can never say no. Not only that, but you have to constantly add features. Now that corporate interests dominate in many corners of the FOSS sphere, this benefit of FOSS is diminishing. Which is a shame, because it's a big reason why FOSS was so successful, and why so many old commercial projects collapsed under their own weight, even though they always had more features than their FOSS counterparts and thus theoretically should have only grown their user base.
alpb 6y ago
The author is not aware that etcd is still offering the JSON API he mentioned. https://news.ycombinator.com/item?id=23835651 https://news.ycombinator.com/item?id=23835651 so his main argument doesn't hold in the first place. The rest of the post is "Google's technologies like protobuf/grpc influence people too much, everything should be a json api".
- pavlov 6y ago
  He's aware of the old API, but believes it will be killed soon. From the article: "The v2 API lives on for now, but upstream threatens to remove it in every new version and there will surely come a time when it'll be removed entirely."
loriverkutya 6y ago
He is talking in absolutes (Kubernetes being the worst piece of software, etc) and trying to convince reader that "megacorporations" are the faceless evil enemy and they are making everything worse for everybody. Either you are with him or against him (servant of the evil megacorps' interest), there is no middle ground. This person is not looking for a conversation, but looking for a fight. He seems to be angry and cannot direct his anger to "make good art" (Neil Gaiman - Make Good Art), instead trying to insult people and pushing them to a place where their emotions takes control and can be dragged into the mud.
- sriku 6y ago
  > direct his anger to "make good art" Here is one from me with the solo-dev/small-team in mind - https://github.com/imaginea/inai https://github.com/imaginea/inai . Good or not I don't know. It came out of some reading I was doing and I'm having fun with it due to the "deploy first and then dev" mindset it encourages.
- raxxorrax 6y ago
  Art or not, there are quite a few contribution on the same page he hosted this post on. Do you work with Kubernetes in a professional setting?
- gcbw3 6y ago
  > "megacorporations" are the faceless evil enemy There are some irony on the text, but to understand this is very naive. The critic is at malicious/dumb people who latch to technologies without understanding them. For argument sake, see how many people are defending coreOS etc, and how they claim to have decided to like it based on their superb expertise, yet they will likely fail to explain to you chroot or anything else containers are based on. Or how they will proclaim that LXC is pure garbage and Docker is better (i will not explain the joke, let them downvote :) Bottom line: we are being sold bad ideas from the past, in new clothing, by the incompetents/malicious people, which megacorp is made of.
  - BrainlingPdx 6y ago
    So your response to him pointing out that the author talks in absolutes, and tries to shit on any tech he doesn't like...is to talk in absolutes and shit on tech you don't like, with little jabs and dumb in jokes? Very convincing.
jacques_chester 6y ago
> It's almost like there's no good representation in the open-source world for the solo developer or small team. Availability heuristic: large companies get more press. Partly because they're large and therefore more likely to attract readers for stories about them, partly because their scope is vast and so journalists know about them, and partly because they intentionally seek press for their work. The scratch-an-itch period of mass-market OSS is long gone. If indeed it ever truly existed.
acje 6y ago
I love k8s, think it’s the best thing that happened to distributed computing since .. sliced bread? Anyway, it’s good, but I don’t disagree with the author I just have a different perspective. I see k8s as a vehicle for learning about distributed computing not just for me but the industry as a whole. Same thing about Linux, it’s just a stepping stone for node management and has served us well in figuring out a bunch of problems. K8s removes a lot of duplicated effort in the learning process for the industry, just close your eyes and think back to the past decades of ESB wars.. that was a waste! “Under the radar” comes a new generation of better systems and components that learns from these learning vehicles of computing and will change how we deliver services fundamentally, but without k8s and Linux we would never have gotten there. Open source RISC-V co-evolved with seL4, WASI, webassembly and NATS comes to mind, but I’m sure the future will take its own path.
- halbritt 6y ago
  I also love k8s and agree with the author. K8s gives us declarative infrastructure with eventual reconciliation. That's massive, and there's nothing else around that really does that. I have a hard time taking anyone sesriously that makes the case that k8s is bad that hasn't fully grokked the above.
  - jashmatthews 6y ago
    Personally I see this as coming from K8s having effectively two parallel uses. One is for making it easier to manage large, complex infrastructure. The other is as a quasi-OS for containerized software. Instead of using a new technology to move infrastructure complexity away from developers we now run development environments on k8s, piling even more complexity and layers of abstraction on.
  - ridv 6y ago
    Disclaimer: I'm was part of the Apache Aurora PMC before the project went into the attic. Not sure what you mean by declarative infrastructure but eventual reconciliation has been implemented in at least one other project (now retired) called Apache Aurora[1] which runs on Apache Mesos[2]. Twitter ran (and probably still runs) a combo of these to huge scale and great success so I don't think it's fair to say that there's nothing else around that does that :). K8s is great, but we should be careful not to rewrite history. [1] http://aurora.apache.org/ http://aurora.apache.org/ [2] http://mesos.apache.org/ http://mesos.apache.org/
vinay_ys 6y ago
These open source projects you talk about were developed by profit seeking enterprises for solving complex distributed systems problems at scale. Because they chose to make them open-source (not out of altruistic motives but out of the same profit seeking perfectly good business motives), these solo developers (who are also profit seeking) can benefit from those projects for free and can afford to build to serve lot more users than they would be able to otherwise. I don't get the entitled attitude of the author. If he wants a 'simple' system by his standards, he can build one easily enough with the modern tools like golang, node.js and rust. Btw, the web development tools were never simpler than today to operate a safe and secure production web application to serve any number of users. I'm saying this with about two decades of experience doing the same.
empath75 6y ago
The simple version of etcd still exists. Nobody deleted the code. Just use that.
santoshalper 6y ago
I don't get it, isn't that the point of open source and free software, why doesn't the author, or a group of like minded people simply fork the old software and release a SimpleEtcD? Of course, you could argue that the people who made it more complicated in the first place could/should have forked instead of taking over, but that ship obviously sailed a long time ago.
lemmsjid 6y ago
The problem, I think, is that the article presents all of its views as almost self evidently true. If you distill it down, the complaint is that etcd added gRPC. I think it was a good move for an infrastructure piece like etcd to add gRPC. Now I can just grab a generated client in a language of my choice. There's certainly valid critiques of gRPC / protocol buffers but I've found things like gRPC and Thrift to reduce complexity in applications that use them, because there's a thousand interpretations of REST out there, not to mention the crazy things people do with 'simple' HTTP, such as long polling, which introduce implicitly complex semantics into something that is on the surface quite simple.
- mikro2nd 6y ago
  > the article presents all of its views as almost self evidently true Not so. The article, imho, presents all of its views as unabashedly opinionated views of the author. The complaint that simple, lucid and well-fit functional designs are getting hijacked by large-corp vested interests has been adequately taken up elsewhere in this thread.
- chillfox 6y ago
  "Now I can just grab a generated client in a language of my choice." Except when you can't because no client is available in your favorite language. Then it is a lot harder than a rest api would have been.
- wrsh07 6y ago
  I agree with you. I like grpc. But I also think that the original author has a point: for many solo developers it's an extra dependency that they have to learn, and they get virtually no benefit from using grpc because they aren't performance-bound. They are simplicity-bound.
  - lemmsjid 6y ago
    Yep! That's the nuanced argument I'd look for, as opposed to 'gRPC bad, they added it because they're from google!' Simplicity wise my favorite scenario is where if I'm using Python I can pip install an interface (or maven, or NPM, etc. etc) which is the versioned, supported interface for a component. REST + Swagger can help a lot there. So can gRPC. But I can also get how your favorite scenario might be to explore a REST API and build your own client.
SquishyPanda23 6y ago
> vested interests from large companies are able to introduce huge complexity into simple, well-designed projects. In this case, the vested interest are interested in making it as easy as possible for real world customers to reliably deploy their applications to the Cloud. That's not an interest that favors unneeded complexity. The complexity comes from the need for a container orchestration system that satisfies the needs of large organizations. If you could satisfy those needs more simply, the incentives are to simplify. Undoubtedly there is space to build a simpler tool with a subset of Kubernetes' features. And sure, you can pick at specific flaws in the way it's evolved. But I'm not so convinced by the claim that the whole thing is a mess because tech employees like replicating the stack they are used to.
js8 6y ago
> It's almost like there's no good representation in the open-source world for the solo developer or small team. IMHO, there is a simple solution to this - release the software under the GPL license. Then no corporation will want to touch it.
- GoblinSlayer 6y ago
  Only if it's proper GPL3. Google and Microsoft touch linux everywhere: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=49decddd39e5f6132ccd7d9fdc3d7c470b0061bb https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
- linuxftw 6y ago
  AGPL would be more apropos in this day and age of "SaaS all the things."
  - amiga-workbench 6y ago
    https://opensource.google/docs/using/agpl-policy/ https://opensource.google/docs/using/agpl-policy/ They're absolutely allergic to it.
- Jeff_Brown 6y ago
  If your project is sufficiently popular, that's safe; otherwise maybe not. As someone else mentioned, when a giant corp absorbs a free project, you can reasonably expect them to maintain it. If you GPL the project, that won't happen. But if it's a popular project, then if it's GPL it will attract more maintenance from people who can't free-ride on the corp's efforts. (Note that the relevant measure of "popular" is not number of users, but number of people enthusiastic enough to actually contribute.)
cbsmith 6y ago
A lot of the complexity is just intrinsic to working on distributed systems, and if you're doing that kind of work, then these products really can help reduce overall complexity by giving you an admittedly complex solution that is at least robust and pretty well tested; that tends to be better than constructing your own. I think where a lot of the pain comes from, and the article hints at this, is there are a LOT of problems that do not require a distributed solution, and while you can totally solve those problems with a distributed system, it's not really useful to do so in a way that dramatically increases complexity. This is a known issue, and there are simpler models for these simpler cases. I think the problem is all the hype around k8s has driven a lot of square pegs into round holes.
- StillBored 6y ago
  Yes distributed systems are complex, but IMHO at least a large part of that complexity is caused by people who think they understand the problem, and go off and create some "great" new technology. Which then gets a lot of press, and other people building systems on it, only to discover that the edge case being ignored as inconsequential starts to create lots of failure modes. Sometimes these critical problems are "human factor" ones like with kubernetes configuration, or they are just a lack of proper engineering hygiene. Then you get another layer of crap on top. Repeat that a few times and what you actually have is a giant mess that frequently only works through the sheer effort of a large operations team. To put this another way, if you read the Jepsen reports overwhelmingly what comes across is a lot of hubris. Its overwhelmingly old stodgy technology these new systems are meant to replace (posgres for example) that are satisfying the promises being proclaimed on high by $NEW_TECHNOLOGY.
  - cbsmith 6y ago
    I don't look at the Jepsen reports as being a representation of hubris. Most of what Jepsen finds may have been preventable problems, but they are also fixable problems, which is why I don't see hubris. There's just a reality that we don't really build these systems on top of a distributed operating system. They're starting with an platform abstraction that is already terrible imperfect for the job. There is a ton of pressure to get something out quickly and make it accessible to a broad set of developers rather than to do it correctly (MongoDB is perhaps the most classic example of this). I was building big data systems in the early days, and the whole mentality was that you were building tools that were riddled with flaws, limitations, and outright bugs, but in the right context would make capabilities accessible to a broad set of developers that were otherwise completely off the table. The truth is, if developers were already experts at distributed systems, they wouldn't need or want most of these tools; a LOT of the value is in accessibility and having something that is "good enough" rather than correct. The products reflect that more than they reflect hubris. What I do see is a combination of marketing and customer ignorance that does believe in silver bullets and belief that these tools don't have to be understood to be effective. It's an unfortunate byproduct of focusing on that accessibility over correctness.
hyperpape 6y ago
The author may or may not have the correct conclusion about over-engineering coming from big-co alumni, but if they can't get their one example correct, why should we discuss their post? Are their insults that novel?
zokier 6y ago
> In the meantime, the simpler version of the software is long gone The thing is that with free software the old software is almost never really long gone. It's still out there, free for your perusal. That is the beauty of free software, it's always additive process. But it does also put responsibility on the user to pick the patches they want, either themselves or by proxy. Ultimately you are responsible for the free code you are running, not some vague upstream dominated by fnaag et al.
- swagonomixxx 6y ago
  While this is technically true, using old, unsupported versions of software (such as etcd) in production is a recipe for disaster, especially when you start running into problems with it. I think, in the case of etcd, it would have been better for the k8s folks to fork the project and add whichever features they felt were needed to be able to use it for k8s. Instead, it seems like they swarmed the original project and added a ton of features that made it a behemoth of complexity rather than the simple tool it once was.
  - pdonis 6y ago
    > I think, in the case of etcd, it would have been better for the k8s folks to fork the project This would only make a difference if someone was willing to maintain the non-k8s fork. From what I understand, nobody was willing to take that on, so the k8s version became the only version by default.
  - zokier 6y ago
    It's not unsupported if you support it yourself or hire/contract someone to support it for you

by mvanga 6y ago

microcolonel 6y ago

A major problem that leads to dysfunctional bloat is the inherent ego traps of software developed by teams of well-adjusted people. Even if it's for the good of the quality of a product, it is not often easy telling your team mate that they wrote an unnecessary component that should just be deleted. I have done it lots in my short career (and in terms of engineering outcomes, it's been spectacular), but it's easier for me I think, since I'm more disagreeable than the average person.

pwdisswordfish2 6y ago

"... of all major software components today." Thankfully, we still have the "minor" components to ourselves. Popularity has its downsides.

jchw 6y ago

The problem, as I pointed out elsewhere, is that the post makes very few constructive comments and presents a lot of opinions and assertions without justification or reasoning, making it extremely hard to actually make meaningful comments about. As someone else pointed out, if it were an HN comment, it would be flagged. I don’t agree that this is a statement about complexity, so much as it is a rant about the author being inconvenienced by technology they personally deem pointless and unnecessary. If the CoreOS comments aren’t a sufficient red flag, the HTTP/2 ones really ought to be. Also, some etcd users debate their assertions about newer versions being more complex internally. On the other hand, its difficult to argue with overly cynical type attitudes.

mistercow 6y ago

It doesn't seem to me like the comment section is weird at all. This is kind of a perfect object lesson on pitfalls of writing a persuasive blogpost. Basically, if you want to argue for a somewhat controversial conclusion, you shouldn't back it up with even more controversial and unsupported (or weakly supported) premises. Particularly if "even more controversial" means "in disagreement with the vast majority of your industry."

dilandau 6y ago

I think a large problem is open source projects actively marketing themselves, and seeking large user bases. This leads to a wasteland of zombie frameworks, and developers selling out (so to speak) in exchange for the reputation. It is important to note, too, that etcd is permissively licensed and anyone can run the old version in whatever way they please. The final, and biggest, takeaway? Hackernews apparently has drank the koolaid and will reach for any convenient justification to write off an opinion they disagree with. Stockholm syndrome all through the comments. Conservatism and pragmatism apparently are no longer current in the cloud economy.

kelnos 6y ago

> There are people attacking the author for a statement made about CoreOS, and for some hate towards Kubernetes. That's not really surprising, and maybe should be a lesson to the author. If you want to make a particular point, you shouldn't make inflammatory/controversial statements about other, only-tangentially-related things. It distracts from and dilutes your point. For my part, I know one of the CoreOS founders, and while I have no opinion on CoreOS itself, it's really off-putting to see some random armchair quarterback on the internet shitting on the work of someone you know. While I more or less agree with the actual real point of the article, the unnecessary attacks really divided my attention. The article starts off with a cheap potshot at CoreOS, followed by a screed about Kubernetes, and then finally, more than half-way through, we get an admission that the author has "digressed", and we get to the actual point. Certainly the point required some background about Kubernetes, but I can understand why people focus on the negative CoreOS and Kubernetes aspects of the post, since they consist of more than half of the content.

growlist 6y ago
Eh, idk. It makes for entertaining reading for a devops-sceptic.
- edude03 6y ago
  Off-topic, but I've never seen anyone describe themselves as a "devops-sceptic" - could you elaborate more on what that looks like for you?
  - growlist 6y ago
    You're just asking so that you can attack my response, I suggest.
    - edude03 6y ago
      While I know there are people who argue in bad faith on the internet, I am genuinely curious and am not looking for a fight. I would have reached out to you privately but you don't have an email or contact listed on your profile. I'm mostly curious about which part of devops you're skeptical about, if it's tooling, or process, or the mentalities or something else.

rodgerd 6y ago

> The key point of the article is not really being addressed here: Perhaps if the author wanted their key point addressed they could have tried focusing on it, rather than on an ill-considered spray across a group of largely unrelated bugbears.

c3534l 6y ago

There was some generic big-company hate thrown in there, but the author really didn't say anything half as coherent in the OP as you did.

dayjobpork 6y ago

When you fine some cool tool you can: 1. Use as is. 2. Make behind the scenes changes that don't alter functionality. 3. Make changes that add functionality. 4. Make changes that remove existing functionality. 4 is the problem. It's like joining a car club then keeping pushing to have petrol cars banned. It is really shitty to find a cool tool that you can use, then try to force remove features that existing users use.

shadowgovt 6y ago

Why can't solo developers or small teams represent solo developers or small teams? If an etcd that has a thin, inflexible API is of value, a small team should be able to fork and manage it (the notion it's smaller and simpler ought to imply it's a lot cheaper to maintain than a multi-API etcd, right?). If the economics of the situation is "large teams have to maintain tools for small teams and individual users," that doesn't actually scale well, does it?

withinboredom 6y ago

They haven't gotten to beanstalkd yet...

jnxx 6y ago

> introduce huge complexity into simple, well-designed projects why does cmake come to my mind?

bfb09 6y ago

It's a shot across the bow at the Kool-Aid which everyone here has imbibed to some extent. Of course they're going to shoot the messenger. When I clicked this thread I said out loud to my team, "oh, this should be good," and I hadn't even fathomed precisely how predictable the discussion would be. It's so far removed from any consideration that it lands something like explaining to an American that they're a subject of the Queen. It makes no sense to objectively review the thinking when you're inside, and buy, the thinking. Bear in mind the livelihood of nearly everyone participating in this conversation is impacted by the situation being lamented. It's telling that CoreOS employees felt compelled to set the record straight and casually "float ideas". It's further curious to note the difference in HN's reaction to an extremely salty and offensive post from, say, Linus Torvalds, versus this one. People are really bad at objectively setting aside their ideology and tone is a kind excuse to get out of having to genuinely think about whether the angry individual might have a point, and when it hits close to home, well. All bets are off. You're 100% correct on your overarching point but the less kind interpretation is "the incentives of modern software engineering value complexity to artificially strengthen a software engineering job economy". The good news is that the more Big Tech screws the pooch with user data management and software engineering, the stronger an argument I have among very receptive ears in government circles to start thinking of Big Web/FAANG as hostile to responsible computing innovation, because they never really bought what Mikey sold them, and the troupe that followed Mikey to Washington didn't hang around to explain the downsides or long-term management needs of the current landscape.

mekoka 6y ago
You should also consider that the contrarian nature of HN would set it up to exactly the opposite reaction had the post expressed the reverse of its current positions. The big irony is that it's not inconceivable that it would involve some of the same actors voicing different opinions. > "the incentives of modern software engineering value complexity to artificially strengthen a software engineering job economy" Not sure I would agree. Hanlon's razor comes to mind. To me, the referred complexity is reminiscent of a number of other complex patterns that somehow found their ways in software engineering, out of possibly good intentions. Regardless, an idea needs to be sold and when the marketing is well executed the community will ingest the kool-aid. Which it then tries to digest for a few years. Then begins a game of chicken that consists in not being the first to admit that you're in pain, out of fear of being branded an infidel, or out of sunk cost bias. Eventually a few courageous and influential souls finally spit it out in disgust and scream "wtf is this shit?!?" Then the rest awakes from the common stupor and a holy war ensues. I still remember SOAP. I still remember with what verve I was sold Scrum. As for Kubernetes, only time will tell.

simonjgreen 6y ago

Couldn't agree with the sentiment of this comment more.

dynamite-ready 6y ago

Agreed. I saw it as a general lament against over-engineering. I don't think the point got lost in the super specific example... You could just as easily level similar rants against the likes of React and it's wider ecosystem, Tensorflow, Typescript (many will disagree), Docker... I'm sure others have their own bugbears. Much of this is subjective, of course. But to me, it feels like software development is trending towards unecessarily complicated development processes and architectures. And the only beneficiaries appear to be the large technology companies. I suppose in exchange, you're getting a guarantee of maintenance. But is that really worth the additional complexity associated with the common use of these tools?

discordance 6y ago
I attribute this companies that can afford to keep way too many high skilled employees on the books while needing to keep them occupied
- Polylactic_acid 6y ago
  It seems to me the problem is all of our software is built by and for megacorps. In the context of what they need it for its the best tool for the job but it was never made for smaller orgs where 90% of the bells and whistles are not needed and end up as needless complexity.
Polylactic_acid 6y ago
I'm not sure how react fits in here. I have found it to be very simple to understand enough to be productive and upgrading to new versions has required next to no changes to our large codebase. Perhaps you mean SPA react apps which I agree are a bit more work than needed for a solo dev. Docker seems to be optimal for medium size teams. Configuring a server the old way is easier than docker for 1 server but less so for when you have 10 or CI/review apps.
- earthboundkid 6y ago
  React fits in because it's in the process of jumping the shark because FB needs it to. React is already hugely bloated for no discernible reason compared to Preact, but they're just going to keep bloating it with Suspense. Cf. https://crank.js.org/blog/introducing-crank https://crank.js.org/blog/introducing-crank

majormajor 6y ago

When I've worked on small teams we avoided complexity by hardcoding locations and using GUI management tools and all these other things that still exist - and honestly, are better today than they were in the past, when you'd also need to get yourself a rack in a datacenter to run stuff. I think there's a big swath of people adopting fancy tools to try to solve problems they don't have and won't have for years, but I don't see that having destroyed the ability to do things the easy way.

tyingq 6y ago

Meh, having your opener call out a specific project as bullshit has that effect. Pulling the snark would have resulted in more people getting the main point.

bfb09 6y ago
Perhaps the people who treat CoreOS being called bullshit as dispositive of the entire thesis, and an inclusion worthy of disregarding the author and thinking of him in any way other than making an argument, aren't the intended audience. Pull the snark out and there will remain any element of this post that kicks off a violent HN centithread that conveniently avoids the core issue. You know it, I know it.
- tyingq 6y ago
  Snark directly related to the main point is fine. The CoreOS comment is unrelated. In fact the author liked etcd in the time period when it was directly associated with the "bullshit" project. And even mentions "but that doesn't really matter". Bashing K8S is at least related to the main point.
mvanga 6y ago
Definitely true. The author could have chosen differently and it would have probably led to more constructive comments. However, I'll also point out that focusing on the snark and ignoring the rest of the article is also a choice made on the part of the commenters. Given a choice, I personally prefer to take the strongest and most charitable interpretation of someones ideas before choosing to criticize them. At least the discussion is interesting that way...
- saagarjha 6y ago
  That things I write for the web usually go through a "lazy comment" filter where I pull out references that would lead to lazy comments. Things that often get stripped are one reference to a specific example when I'm talking about something general–because invariably the comments will focus on "how I don't get the software" or "how I singled it out" or "how what I wrote isn't even true if you look at this way so my point is now invalid". Either I mention no examples, and have the commenters come up with their own, or I have a few so that I don't get attacked by a specific group of people.

fwsgonzo 6y ago

I think everyone here remembers that time they wrote a simple HTTP server, that just works, and would continue to work today. If anything, I hope that we can recognize that there has been advancements in computing. Back in the day you would be open to attacks left and right. It was the wild west back in the day. Today, you can take that tiny webserver and put it in a VM, and voila, now it doesn't matter if anyone breaks in. As long as you didn't add the SSH key to github or anything like that. We can have complex programs, as long as every-man doesn't have to know the details behind it, or implement it or risk losing access he previously had. I think new protocols that are extremely hard to implement falls under that, and I don't think Qemu or Linux does, as nobody loses anything when Linux adds new complex features. We have to make sure, at least, that we continue to support HTTP/1. I think we already lost a lot when we moved away from IRC.

Animats 6y ago
FCGI controlling Go programs, maybe front-ended by a load balancer and back-ended by MySQL/MariaDB/Postgres, can get an awful lot of work done cheaply. FCGI is an automatic "orchestration system" - it starts up more instances as needed, restarts them if they crash, shuts them down when they go idle, and restarts them once in a while in case they leak some resource. It's like the parts of Kubernetes you really need. Go is an OK language. Not a great language, but an OK language. It comes with almost everything you need for server-side applications. Also, the "goroutine" model means you don't have to fool with threads vs async. If a goroutine blocks on a lock, no problem; the CPU gets redispatched. If you need a million goroutines, that works, too. None of that "oh, no, the async thread made a call that blocked and screwed up the whole system". MySQL/MariaDB/Postgres have been around for decades, and they work pretty well. You can have replication if you need it. (NoSQL seems to have come and gone.) Get the stuff at the bottom right, instead of piling more cruft on top. It's cheaper.
bfb09 6y ago
What if 'advancements' were anything but? How would we know whether we're advancing a nascent engineering discipline in the best way possible? Is public cloud, an optimization for a certain calculus of engineering resources, the optimal form of computing delivery? There's an underlying presumption about computing progress that has weaseled its way into your thinking, and the devil of it is that there are vanishingly few people willing to question it (can hardly blame them, given this thread).

cryptica 6y ago

>> It's almost like there's no good representation in the open-source world for the solo developer or small team. As a solo open source developer who maintains a popular OSS project, this is 100% the case. Sometimes it feels like big corporations are actively working against us. It's almost impossible to get any coverage at all on any kind of media. No matter how much organic growth your project has. My project has been growing steadily and linearly for almost a decade. Now it has almost 6K stars on GitHub so you'd think that this by itself would draw some attention? Not so. It's almost impossible to find it on Google search results. Most users find the project through direct word of mouth from other users of the project. We've never received a single consulting contract from any corporation.

IggleSniggle 6y ago
so what's the project?
- jauco 6y ago
  I think it’s socketcluster. However, i’m not quite sure what it does (it obviously does something really well, but I’m probably not precisely the target audience) It might help adoption and re-share value if your readme starts with some problem and shows how your tool solves it. The cadence/temporal people did this really well: https://www.temporal.io/ https://www.temporal.io/
mzz80 6y ago
You’re going to be bitter and resentful for a long time if you believe that the fact that you haven’t received a single consulting contract is due to big corporations working against you. It sounds like you’ve built a useful piece of software. However, you don’t get handouts because you’ve built something. Those who are successful actively promote themselves and their products. There’s a reason companies aren’t comprised of just software engineers.

013a 6y ago

Then the author should have focused the article. If their goal was to address that, then don't take a left-turn into hating on Kubernetes for no reason, before even reaching the point trying to be made. > Funding and adoption (in the extreme, some might say hijacking) of open-source projects from large corporations dictates the direction of all major software components today This is because large corporations are the biggest users of these pieces of software. Its literally that simple; money only plays a smart part of it. Etcd's biggest user is Kubernetes. I would estimate that 2nd place isn't even close. I would estimate that Etcd wouldn't even exist today if not for Kubernetes. The direction of open source projects is usually some approximation of a democracy; its users contribute the code they need. If they need gRPC, it gets added. gRPC wasn't added for no reason, or because politics; its a sister CNCF project, and would increase the performance of the API surface, which greatly benefits all of its users. Lets do a little experiment: Go start an etcd clone. Just go do it! The code is still there! It would be four clicks in the Github UI. Strip gRPC, add the HTTP API, and maintain it. I'd bet a thousand dollars the author won't do it, for two reasons. First, the author hasn't actually used etcd. I know this because, as I said, startlingly few people outside of Kube use etcd directly, and the author isn't providing any substantive technical reasons why the gRPC change was bad; just philosophical ones. And Second, because maintaining large open source software projects is soul crushingingly, destructively, enormously hard. Its way too easy to armchair-quarterback decisions like these based on your own personal flavor of ethics, then cherry-pick the evidence you want to say "corporations are evil! they destroyed etcd by getting rid of HTTP!" Meanwhile, people out there do develop it, and use it, and service billions of requests, and deliver value to their users, and sometimes, turn off their computer and cry because despite all of the hard work and positive results people with no skin in the game still pipe up to say what they're doing is horrible. Did the author join in on the discussion when these changes were made? Did they voice their concerns? Or did they just read about it after-the-fact and say "jeeze, that piece of software I played around with two years ago really took a turn, I'm going to write a post about how evil corporations are."

pessimizer 6y ago
> maintaining large open source software projects is soul crushingingly, destructively, enormously hard. Are you making a case in this package that complex open source projects are impossible unless completely controlled by very large, highly opinionated companies? That can be proven wrong by endless examples. > Did the author join in on the discussion when these changes were made? Did they voice their concerns? Or did they just read about it after-the-fact and say "jeeze, that piece of software I played around with two years ago really took a turn, I'm going to write a post about how evil corporations are." You don't know this to be true, so why make this accusation? Why not address the argument as it is instead of inventing ad hominems?

Florin_Andrei 6y ago

But that's just the "circle of life" no matter what. Things start very simple, and then inevitably get ambitious and try to add new gizmos. Sooner or later you end up with a monster. Over and over again.

solumos 6y ago
This is true - but I think the interesting (and understated) part of this argument is Big Tech's influence on the process. Say I want to fork etcd or build a greenfield alternative - I now have to compete with (or support) the Hivemind's preferences in direction/progress. Who's to say their way is the "best" way? I think this tension is much greater than it was 10 years ago.
wahern 6y ago
Historically FOSS has been far more resistant to this cycle; not all FOSS projects, but many. It's much easier for FOSS projects to just say no, and they can spend years or even decades refining things. In the commercial world you can never say no. Not only that, but you have to constantly add features. Now that corporate interests dominate in many corners of the FOSS sphere, this benefit of FOSS is diminishing. Which is a shame, because it's a big reason why FOSS was so successful, and why so many old commercial projects collapsed under their own weight, even though they always had more features than their FOSS counterparts and thus theoretically should have only grown their user base.

alpb 6y ago

The author is not aware that etcd is still offering the JSON API he mentioned. https://news.ycombinator.com/item?id=23835651 https://news.ycombinator.com/item?id=23835651 so his main argument doesn't hold in the first place. The rest of the post is "Google's technologies like protobuf/grpc influence people too much, everything should be a json api".

pavlov 6y ago
He's aware of the old API, but believes it will be killed soon. From the article: "The v2 API lives on for now, but upstream threatens to remove it in every new version and there will surely come a time when it'll be removed entirely."

loriverkutya 6y ago

He is talking in absolutes (Kubernetes being the worst piece of software, etc) and trying to convince reader that "megacorporations" are the faceless evil enemy and they are making everything worse for everybody. Either you are with him or against him (servant of the evil megacorps' interest), there is no middle ground. This person is not looking for a conversation, but looking for a fight. He seems to be angry and cannot direct his anger to "make good art" (Neil Gaiman - Make Good Art), instead trying to insult people and pushing them to a place where their emotions takes control and can be dragged into the mud.

sriku 6y ago
> direct his anger to "make good art" Here is one from me with the solo-dev/small-team in mind - https://github.com/imaginea/inai https://github.com/imaginea/inai . Good or not I don't know. It came out of some reading I was doing and I'm having fun with it due to the "deploy first and then dev" mindset it encourages.
raxxorrax 6y ago
Art or not, there are quite a few contribution on the same page he hosted this post on. Do you work with Kubernetes in a professional setting?
gcbw3 6y ago
> "megacorporations" are the faceless evil enemy There are some irony on the text, but to understand this is very naive. The critic is at malicious/dumb people who latch to technologies without understanding them. For argument sake, see how many people are defending coreOS etc, and how they claim to have decided to like it based on their superb expertise, yet they will likely fail to explain to you chroot or anything else containers are based on. Or how they will proclaim that LXC is pure garbage and Docker is better (i will not explain the joke, let them downvote :) Bottom line: we are being sold bad ideas from the past, in new clothing, by the incompetents/malicious people, which megacorp is made of.
- BrainlingPdx 6y ago
  So your response to him pointing out that the author talks in absolutes, and tries to shit on any tech he doesn't like...is to talk in absolutes and shit on tech you don't like, with little jabs and dumb in jokes? Very convincing.

jacques_chester 6y ago

> It's almost like there's no good representation in the open-source world for the solo developer or small team. Availability heuristic: large companies get more press. Partly because they're large and therefore more likely to attract readers for stories about them, partly because their scope is vast and so journalists know about them, and partly because they intentionally seek press for their work. The scratch-an-itch period of mass-market OSS is long gone. If indeed it ever truly existed.

acje 6y ago

I love k8s, think it’s the best thing that happened to distributed computing since .. sliced bread? Anyway, it’s good, but I don’t disagree with the author I just have a different perspective. I see k8s as a vehicle for learning about distributed computing not just for me but the industry as a whole. Same thing about Linux, it’s just a stepping stone for node management and has served us well in figuring out a bunch of problems. K8s removes a lot of duplicated effort in the learning process for the industry, just close your eyes and think back to the past decades of ESB wars.. that was a waste! “Under the radar” comes a new generation of better systems and components that learns from these learning vehicles of computing and will change how we deliver services fundamentally, but without k8s and Linux we would never have gotten there. Open source RISC-V co-evolved with seL4, WASI, webassembly and NATS comes to mind, but I’m sure the future will take its own path.

halbritt 6y ago
I also love k8s and agree with the author. K8s gives us declarative infrastructure with eventual reconciliation. That's massive, and there's nothing else around that really does that. I have a hard time taking anyone sesriously that makes the case that k8s is bad that hasn't fully grokked the above.
- jashmatthews 6y ago
  Personally I see this as coming from K8s having effectively two parallel uses. One is for making it easier to manage large, complex infrastructure. The other is as a quasi-OS for containerized software. Instead of using a new technology to move infrastructure complexity away from developers we now run development environments on k8s, piling even more complexity and layers of abstraction on.
- ridv 6y ago
  Disclaimer: I'm was part of the Apache Aurora PMC before the project went into the attic. Not sure what you mean by declarative infrastructure but eventual reconciliation has been implemented in at least one other project (now retired) called Apache Aurora[1] which runs on Apache Mesos[2]. Twitter ran (and probably still runs) a combo of these to huge scale and great success so I don't think it's fair to say that there's nothing else around that does that :). K8s is great, but we should be careful not to rewrite history. [1] http://aurora.apache.org/ http://aurora.apache.org/ [2] http://mesos.apache.org/ http://mesos.apache.org/

vinay_ys 6y ago

These open source projects you talk about were developed by profit seeking enterprises for solving complex distributed systems problems at scale. Because they chose to make them open-source (not out of altruistic motives but out of the same profit seeking perfectly good business motives), these solo developers (who are also profit seeking) can benefit from those projects for free and can afford to build to serve lot more users than they would be able to otherwise. I don't get the entitled attitude of the author. If he wants a 'simple' system by his standards, he can build one easily enough with the modern tools like golang, node.js and rust. Btw, the web development tools were never simpler than today to operate a safe and secure production web application to serve any number of users. I'm saying this with about two decades of experience doing the same.

empath75 6y ago

The simple version of etcd still exists. Nobody deleted the code. Just use that.

santoshalper 6y ago

I don't get it, isn't that the point of open source and free software, why doesn't the author, or a group of like minded people simply fork the old software and release a SimpleEtcD? Of course, you could argue that the people who made it more complicated in the first place could/should have forked instead of taking over, but that ship obviously sailed a long time ago.

lemmsjid 6y ago

The problem, I think, is that the article presents all of its views as almost self evidently true. If you distill it down, the complaint is that etcd added gRPC. I think it was a good move for an infrastructure piece like etcd to add gRPC. Now I can just grab a generated client in a language of my choice. There's certainly valid critiques of gRPC / protocol buffers but I've found things like gRPC and Thrift to reduce complexity in applications that use them, because there's a thousand interpretations of REST out there, not to mention the crazy things people do with 'simple' HTTP, such as long polling, which introduce implicitly complex semantics into something that is on the surface quite simple.

mikro2nd 6y ago
> the article presents all of its views as almost self evidently true Not so. The article, imho, presents all of its views as unabashedly opinionated views of the author. The complaint that simple, lucid and well-fit functional designs are getting hijacked by large-corp vested interests has been adequately taken up elsewhere in this thread.
chillfox 6y ago
"Now I can just grab a generated client in a language of my choice." Except when you can't because no client is available in your favorite language. Then it is a lot harder than a rest api would have been.
wrsh07 6y ago
I agree with you. I like grpc. But I also think that the original author has a point: for many solo developers it's an extra dependency that they have to learn, and they get virtually no benefit from using grpc because they aren't performance-bound. They are simplicity-bound.
- lemmsjid 6y ago
  Yep! That's the nuanced argument I'd look for, as opposed to 'gRPC bad, they added it because they're from google!' Simplicity wise my favorite scenario is where if I'm using Python I can pip install an interface (or maven, or NPM, etc. etc) which is the versioned, supported interface for a component. REST + Swagger can help a lot there. So can gRPC. But I can also get how your favorite scenario might be to explore a REST API and build your own client.

SquishyPanda23 6y ago

> vested interests from large companies are able to introduce huge complexity into simple, well-designed projects. In this case, the vested interest are interested in making it as easy as possible for real world customers to reliably deploy their applications to the Cloud. That's not an interest that favors unneeded complexity. The complexity comes from the need for a container orchestration system that satisfies the needs of large organizations. If you could satisfy those needs more simply, the incentives are to simplify. Undoubtedly there is space to build a simpler tool with a subset of Kubernetes' features. And sure, you can pick at specific flaws in the way it's evolved. But I'm not so convinced by the claim that the whole thing is a mess because tech employees like replicating the stack they are used to.

js8 6y ago

> It's almost like there's no good representation in the open-source world for the solo developer or small team. IMHO, there is a simple solution to this - release the software under the GPL license. Then no corporation will want to touch it.

GoblinSlayer 6y ago
Only if it's proper GPL3. Google and Microsoft touch linux everywhere: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=49decddd39e5f6132ccd7d9fdc3d7c470b0061bb https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
linuxftw 6y ago
AGPL would be more apropos in this day and age of "SaaS all the things."
- amiga-workbench 6y ago
  https://opensource.google/docs/using/agpl-policy/ https://opensource.google/docs/using/agpl-policy/ They're absolutely allergic to it.
Jeff_Brown 6y ago
If your project is sufficiently popular, that's safe; otherwise maybe not. As someone else mentioned, when a giant corp absorbs a free project, you can reasonably expect them to maintain it. If you GPL the project, that won't happen. But if it's a popular project, then if it's GPL it will attract more maintenance from people who can't free-ride on the corp's efforts. (Note that the relevant measure of "popular" is not number of users, but number of people enthusiastic enough to actually contribute.)

cbsmith 6y ago

A lot of the complexity is just intrinsic to working on distributed systems, and if you're doing that kind of work, then these products really can help reduce overall complexity by giving you an admittedly complex solution that is at least robust and pretty well tested; that tends to be better than constructing your own. I think where a lot of the pain comes from, and the article hints at this, is there are a LOT of problems that do not require a distributed solution, and while you can totally solve those problems with a distributed system, it's not really useful to do so in a way that dramatically increases complexity. This is a known issue, and there are simpler models for these simpler cases. I think the problem is all the hype around k8s has driven a lot of square pegs into round holes.

StillBored 6y ago
Yes distributed systems are complex, but IMHO at least a large part of that complexity is caused by people who think they understand the problem, and go off and create some "great" new technology. Which then gets a lot of press, and other people building systems on it, only to discover that the edge case being ignored as inconsequential starts to create lots of failure modes. Sometimes these critical problems are "human factor" ones like with kubernetes configuration, or they are just a lack of proper engineering hygiene. Then you get another layer of crap on top. Repeat that a few times and what you actually have is a giant mess that frequently only works through the sheer effort of a large operations team. To put this another way, if you read the Jepsen reports overwhelmingly what comes across is a lot of hubris. Its overwhelmingly old stodgy technology these new systems are meant to replace (posgres for example) that are satisfying the promises being proclaimed on high by $NEW_TECHNOLOGY.
- cbsmith 6y ago
  I don't look at the Jepsen reports as being a representation of hubris. Most of what Jepsen finds may have been preventable problems, but they are also fixable problems, which is why I don't see hubris. There's just a reality that we don't really build these systems on top of a distributed operating system. They're starting with an platform abstraction that is already terrible imperfect for the job. There is a ton of pressure to get something out quickly and make it accessible to a broad set of developers rather than to do it correctly (MongoDB is perhaps the most classic example of this). I was building big data systems in the early days, and the whole mentality was that you were building tools that were riddled with flaws, limitations, and outright bugs, but in the right context would make capabilities accessible to a broad set of developers that were otherwise completely off the table. The truth is, if developers were already experts at distributed systems, they wouldn't need or want most of these tools; a LOT of the value is in accessibility and having something that is "good enough" rather than correct. The products reflect that more than they reflect hubris. What I do see is a combination of marketing and customer ignorance that does believe in silver bullets and belief that these tools don't have to be understood to be effective. It's an unfortunate byproduct of focusing on that accessibility over correctness.

hyperpape 6y ago

The author may or may not have the correct conclusion about over-engineering coming from big-co alumni, but if they can't get their one example correct, why should we discuss their post? Are their insults that novel?

zokier 6y ago

> In the meantime, the simpler version of the software is long gone The thing is that with free software the old software is almost never really long gone. It's still out there, free for your perusal. That is the beauty of free software, it's always additive process. But it does also put responsibility on the user to pick the patches they want, either themselves or by proxy. Ultimately you are responsible for the free code you are running, not some vague upstream dominated by fnaag et al.

swagonomixxx 6y ago
While this is technically true, using old, unsupported versions of software (such as etcd) in production is a recipe for disaster, especially when you start running into problems with it. I think, in the case of etcd, it would have been better for the k8s folks to fork the project and add whichever features they felt were needed to be able to use it for k8s. Instead, it seems like they swarmed the original project and added a ton of features that made it a behemoth of complexity rather than the simple tool it once was.
- pdonis 6y ago
  > I think, in the case of etcd, it would have been better for the k8s folks to fork the project This would only make a difference if someone was willing to maintain the non-k8s fork. From what I understand, nobody was willing to take that on, so the k8s version became the only version by default.
- zokier 6y ago
  It's not unsupported if you support it yourself or hire/contract someone to support it for you