10 ms·

It's important to note that they are holding off on deprecating dependents to see if they can swap in replacements. For example, programs that depend on Terrafo

• wg0 3y ago
  Nomad was overly ambitious in running everything under the sun. Didn't catch on. Setting it up would also require Consul. Docker swarm is simpler and superior solution to Nomad and is built right into docker engine itself.
  • speedgoose 3y ago
    I would rather sink in an ocean of kubernetes YAML documents than having to deal with Docker swarm issues ever again. Thankfully Docker swarm is kinda dead so it’s unlikely to be relevant again. Kubernetes won.
    • michens 3y ago
      Can you elaborate on its issues? I've never used it on production myself, but it looks like a simplified k8n, which is something I feel a lot of people desire
      • da768 3y ago
        Last used it years ago, networking would just die randomly, requiring container restarts when using multiple nodes
        • eropple 3y ago
          I have heard (though haven't tried) that V2 of Compose/Swarm is significantly better.
        • drdaeman 3y ago
          Anything can and will break. Symptomatically similar thing happened to me with K8s with (IIRC) Calico. And I’ve had issues with Swarm too, although of a different kind (it quietly refused to run a data channel for me over an IPv6-only network, while control channel worked just fine). It was all quite a while (3-4 years) ago, so I don’t remember much details, except that Swarm was then not exactly capable of working in IPv6-only environments. It was significantly easier to navigate Docker’s codebase than Kubernetes’. (Ultimately, my solution was that I understood that I didn’t need any kind of orchestration there, just HA/failover. But that’s another story.)
      • raphinou 3y ago
        I've been very happy with until now, though i must admit i'm using one node swarms. But i'd rather use that (1) than sticking together a custom solution using docker compose as i see it mentioned sometimes. (1) https://dockerswarm.rocks/traefik/ https://dockerswarm.rocks/traefik/
      • speedgoose 3y ago
        I haven’t used it for many years but in short it just didn’t work reliably. I mostly encountered network iptables issues from memory.
      • nunez 3y ago
        Infinite issues with overlay networking once you scale past 30 nodes in a swarm. Weave aside, network drivers for anything not CNI are basically deprecated/dead.
    • talent_deprived 3y ago
      I like what kube achieves, I don't like the power and memory hungry requirements of it, it seems ill optimized. A cluster with no pods uses 5%-15% CPU, that's a lot of electricity being used for nothing. It's really odd since Google claims to have green datacenters.
      • philipov 3y ago
        Does Google use carbon swaps to conceal their emissions? Maybe that's how they reconcile the difference. A quick search turned up this: https://www.greencentury.com/google-parent-alphabet-pledges-greater-transparency-on-use-of-carbon-offsets-after-green-century-shareholder-proposal-2/ https://www.greencentury.com/google-parent-alphabet-pledges-... Looks like a possible case of greenwashing to me.
        • dilyevsky 3y ago
          Google doesn’t use kubernetes to run anything production. They use Borg
        • tedivm 3y ago
          I got into an argument with someone at google who said they don't use carbon swaps and ended up pulling up a shareholder report that said they did. Google is real big on the greenwashing.
      • nixgeek 3y ago
        Google doesn’t run on Kubernetee.
      • teraflop 3y ago
        I'm curious, are you seeing that on cloud instances, or what? My experience with bare-metal Kubernetes hasn't been that bad. On my home cluster with very underpowered J4105 CPUs, I get about 5% CPU usage on the master node at idle, and 2% on workers.
        • dilyevsky 3y ago
    • tedivm 3y ago
      I've actually been using docker swarm for an odroid cluster that runs a lot of services in my house. I really like it- it's extremely low overhead and the network mesh works way better than I was expecting considering I put no effort into configuring it. That said this is a three node cluster running a handful of containers, with services accessed by my wife and I. So not exactly a large scale production experience.
      • SteveNuts 3y ago
        As someone whose wife will immediately call me if the important services in our household (Plex, pihole, home assistant), I think those definitely count as production services!
    • wg0 3y ago
      Kubernetes won because of its composability. The API is the killer feature. The notion of objects, labels, pretty much an object oriented, declarative interface. You'd have to do a lots of custom morphing if you want that kind of interface over docker swarm. For smaller setups, simplicity of docker swarm might be useful but I am curious to know from people who are running clusters of 25 to 50 nodes.
  • jen20 3y ago
    > Docker swarm is simpler and superior solution to Nomad This is the voice of absolute delusion. Swarm is a mess. Nomad actually scales.
  • chromatin 3y ago
    Recent nomad versions replace lots of consul functionality. Consul is definitely not needed at small scale, even for service discovery.
  • kawsper 3y ago
    With Nomad 1.3 and newer, you can run a Nomad cluster without Consul, there's support for built-in service discovery: https://www.hashicorp.com/blog/nomad-service-discovery https://www.hashicorp.com/blog/nomad-service-discovery
  • mosselman 3y ago
    I am a happy swarm user for a long time now. It is pretty simple. The only complicated part was the reverse proxy, but traefik has been a solid solution. You will spend quite some time figuring the labels out, but once you have them for one service, it works for many. A new solution that might be even better is Kamal by 37 Signals. It runs all their production stuff, so it must be capable. Can’t wait for a version 2 where, I’d suspect, by then there will be more community contributions for some more diverse setups merged in.
  • busterarm 3y ago
    Having worked with Nomad, K8s and Docker Swarm at reasonable scale, Docker Swarm is the only one of those three that I would never use again under pain of death.
• erulabs 3y ago
  Nomad looked like a good product until I used it. I was so looking forward to a simpler take on orchestration, having spent years in k8s-land and generally being a fan of Hashicorp in the past. My tldr now is nomad should be considered harmful.
  • nanmu42 3y ago
    Recent version of Nomad has introduced many QoL features like secrets and service discovery, though logging collection remains unsupported.
    • mdaniel 3y ago
      > though logging collection remains unsupported. Do you happen to have a link to the issue (I guess or docs, if it's their official stance)? I'd enjoy reading how they ended up in that circumstance
      • kawsper 3y ago
        Logcollection doesn't come out of the box, and you're kinda on your own to set it up, this article (although a bit dated) shows you what you must go through: https://atodorov.me/2021/07/09/logging-on-nomad-and-log-aggregation-with-loki/ https://atodorov.me/2021/07/09/logging-on-nomad-and-log-aggr... It also doesn't help that Nomad handles everything from Docker containers to managing regular applications.
      • jen20 3y ago
        Also worth noting that nomad’s closest competitor (Kubernetes) does not do this either. It’s a huge oversight in both ecosystems.
        • mdaniel 3y ago
          We may be talking about different things, but GP's comment said "unsupported" whereas in kubernetes it's just not part of the control plane offered by kubernetes itself, in exactly the same way that cloud load balancer or even cloud auto-scaling isn't provided by kubernetes itself Having read the sibling's comment, I think I better understand the situation since Nomad isn't a container orchestrator and thus there isn't one homogeneous place from which _to_ collect the logs, but in any sane k8s setup that's not true since both dockerd and containerd have mechanisms to influence that behavior and k8s is perfectly able, and supports, scheduling per-Node utility plumbing stuff like that without drama
• nanmu42 3y ago
  I like Nomad a lot. It is even more lightweight than k3s and has served my low-budget projects very well. A little sad to see things are going this way.
• mathverse 3y ago
  Wait is Nomad really closed source?
  • cipherboy 3y ago
    Nomad, along with the rest of Hashicorp's flagship products, transitioned to the BUSL-1.1: https://github.com/hashicorp/nomad/blob/main/LICENSE https://github.com/hashicorp/nomad/blob/main/LICENSE
    • c7DJTLrn 3y ago
      Am I misunderstanding something? I thought BUSL means the source is still open, but not free as in freedom.
      • toenail 3y ago
        > Am I misunderstanding something? Yes, source being available is not enough to be called open source.
        • rwmj 3y ago
          It's worse in some ways because if you read the source and then work on another open source project in the same field you could end up in legal trouble.
      • cipherboy 3y ago
        Perhaps a more apt term would be freeware. The BUSL is not OSI approved and places restrictions around usage, in an effort to sell some product or service off of a project and prohibit competition. It is not free as in beer either, if you hit such usage. The source is available (hence source available being another terminology) but even Hashicorp has dropped "open source" from nearly every relevant page in understanding that their products no longer have an open source core (see instead "Vault Community" or "Terraform Community"). The general consensus is that no, the BUSL is not OSS as OSI has not blessed it as such: https://opensource.org/licenses/ https://opensource.org/licenses/
• jacurtis 3y ago
  Honestly, all Hashi products are generally really good. I do think they do suffer from moving too fast syndrome. They pushed to go public and most or all of their recent shortcomings can be traced back to trying to push stock prices. Early Hashicorp was incredible. They were open source stewards and looked like an up-and-coming Redhat or Canonical. Their products were ground-breaking and truly huge value adds to the open-source eco system. But they got extremely popular (mostly thanks to Terraform, which took off and drew more attention to their other products). But since going public it has been clear that they are trying to get money and enterprise customers at any cost. Terraform itself has felt like it was in maintenance-mode since hitting version 1. Terraform providers frequently break. For production I have found you need to pin providers down to the patch level, because i've had multiple problems just in small patch-level updates over the recent years. They famously started rejecting any open-source contributions that didn't provide a business value for Hashicorp. Since TF hitting v1, almost all the attention seems to be towards Terraform Cloud and Terraform Enterprise. At Hashicon, it feels like every talk is just propaganda pushing these products. It is all they care about anymore. Nomad was a product that Hashicorp was very excited about for a while and then they seemed to just abandon on the side of the road in their quest for enterprise dominance (probably after learning that most Enterprises are all-in on K8s and Nomad is more beneficial to fast-moving start-ups). Vault was an incredible tool, especially in the Open-Source space. But in the past few years they have really split the open-source and licensed versions of Vault significantly which made the open-source version feel more like a burden to Hashicorp. The last time I talked to Hashicorp about Vault (we were deeply considering a switch to Vault at work last year), they treated the open-source self-hosted solution as a "trial" of the real Vault, and that is what it felt like. Almost everything we ran into during setup, they would respond back with "oh that's fine in the enterprise version". Overall, they have peeled back on all but the absolute minimum efforts required to support their open source versions of products and have been an entirely enterprise focused company for a while. And I can't blame them, sure you need to make money. But I can't help but point to organizations like RedHat and Canonical as examples of what Hashicorp could have been. At this point I feel like the parent that watches their kids, talking about how they missed out on reaching their potential, thanks mostly to what seems like greed or over-ambition. "I'm not mad, I'm just disappointed" comes to mind when I think of Hashi. I have high-hopes for OpenTofu to fill the Terraform void. I've moved past Vault and am using one of the big hyperscalers' secrets management tools (which I enjoy a lot less, but is cheaper and less complicated). I use kubernetes instead of Nomad, which again is fine, it has become the standard anyway. So i'll be just fine... but I'm dissapointed at you Hashicorp. That's all.
• cipherboy 3y ago
  If anyone does start an open Vault fork, I'd be interested in contributing as I get time. Edit to add: https://github.com/hashicorp/vault/graphs/contributors?from=2021-09-20&to=2023-09-22&type=c https://github.com/hashicorp/vault/graphs/contributors?from=...
  • tedivm 3y ago
    I wish I had the time or energy for it- especially with someone like you willing to contribute. I really hope someone takes up this task.