3 ms·

I have to ask, why bother with Kubernetes and all the associated config and pain? Why not just start a new spot instance? I can’t see any reason for Kubernetes

I have to ask, why bother with Kubernetes and all the associated config and pain? Why not just start a new spot instance? I can’t see any reason for Kubernetes in this architecture even though it’s the title of the post. Also personally I wouldn’t use rabbitmq … it’s pretty heavyweight… there’s lots of lightweight queues out there. Overall this architecture looks like it could be simplified. Also, the post doesn’t mention if the video encoding uses GPU hardware acceleration. Makes a big difference especially if using spot instances …. ffmpeg in CPU is extremely computationally expensive. Presumably all input videos need reencoding to convert them to HLS.

malux85 3y ago
This is what I was wondering, in the article it looks like kubernetes is just used to launch the node containers - why is the database and rabbitmq outside of kubernetes? This architecture looks like it’s been cobbled together by a junior
- robertlagrant 3y ago
  Kubernetes loves stateless services. Zero wrong with moving RabbitMQ or a database outside of it.
  - malux85 3y ago
    Except kubernetes has a whole storage provisioning system that gives you redundancy and automatic failover, if you’re going to the trouble of running kubernetes why not just run your whole infra on it? I run https://atomictessellator.com https://atomictessellator.com solo, using kubernetes, and my database, Minio object store, application servers, quantum workers, everything is all on kubernetes, it’s self healing and much simpler to run all the infrastructure the same. Recently I had a node failure while I was sleeping and the whole system healed itself while I slept, the monitoring system didn’t even alarm me because the small blip of increased latency while the pods rebalanced wasn’t above the alert threshold so it didn’t even wake me up. What happens in the article infra when the rabbitmq or database nodes fail? The whole system goes offline, which seems very silly setup when you have kubernetes sitting right there, who’s primary function is to handle all of this.
    - robertlagrant 3y ago
      What happens when your storage detaches from your k8s cluster? Your services start 503ing, hopefully, because you didn't design your system thinking that k8s == 100% uptime.
      - malux85 3y ago
        Anybody can invent random problems ad nauseam - that doesn’t prove anything. I’m not claiming that it’s totally bullet proof, I never said that - I’m saying that if you had a kubernetes cluster anyway why not benefit from its abilities? Especially when the alternative is single node, single points of failure, which is clearly inferior. The "what if the storage detaches" argument could easily apply to the single node VMs too, in which case the outcome would be a total system failure. We are discussing the contrast between the articles architecture and running everything on K8s ... and I'm saying that running everything on K8s is clearly better
        jrockway 3y ago
        Yeah, what happens when someone in AWS clicks "delete database" accidentally? It's the same thing that happens when K8s blows up in some weird way. You restore from your backup. (Fun fact: deleting the instance deletes the backups!)
        robertlagrant 3y ago
        Again, I was replying to: > What happens in the article infra when the rabbitmq or database nodes fail?
        robertlagrant 3y ago
        > Anybody can invent random problems ad nauseam I agree; I was replying to this invented problem: > What happens in the article infra when the rabbitmq or database nodes fail? It makes sense if you read the reply as a reply.
    - codenesium 3y ago
- baq 3y ago
  There some of us who still perform four extra steps before putting any DB in k8s and we have good reasons.
- sass_muffin 3y ago
  It is actually the opposite, it is currently considered a good practice to run stateful workloads outside of kubernetes and stateless workloads inside of kubernetes.
  - KronisLV 3y ago
    > It is actually the opposite, it is currently considered a good practice to run stateful workloads outside of kubernetes and stateless workloads inside of kubernetes. Is that still true? I wouldn't call the parent comment charitable enough, because there definitely can be some reasons for running stateful workloads even outside of containers altogether (familiarity included), but at the same time it feels like a lot of effort has been invested into making that a non-issue. For example, how many database Operators are now available for Kubernetes: https://operatorhub.io/?category=Database&capabilityLevel=%5B%22Seamless+Upgrades%22%2C%22Full+Lifecycle%22%2C%22Deep+Insights%22%2C%22Auto+Pilot%22%5D https://operatorhub.io/?category=Database&capabilityLevel=%5... Honestly, as long as you have storage and config setup correctly, it's not like you even need an Operator, that's for more advanced setups. I've been running databases in containers (even without Kubernetes) for years, haven't had that many issues at small/medium scale.
mihaitodor 3y ago
I believe loads of auxiliary microservices have been omitted for brevity. Of course, those also don’t require Kubernetes, but maybe they have some standardised deployment system which keeps things manageable. Don’t forget about Observability and whatnot.
deleted 3y ago
[deleted]
alexandreolive 3y ago
Hello, I'm the writer of the article. We are using Kubernetes for our whole architecture, consisting of around 40 microservices and cron jobs. I just wanted in this article to give an example of asynchronous architecture using Kubernetes and RabbitMQ. We are using RabbitMQ because it's my company target solution. There might better so lighter solution that would fit us but having just one for every solution is easier to maintain. Great comment about GPU hardware acceleration for encoding, I'm going to look this up.
- andrewstuart 3y ago
  So Kubernetes is only in this architecture because other systems use it and its required by the parent company but not needed. That's pretty important context.
  - alexandreolive 3y ago
    That's not what I said; sorry if that was not clear. The parent company requires RabbitMQ, we are using Kubernetes because managing 40 microservices without it would be hell. In the article, I only showed 1 user-facing API, but it's actually multiple services, I just did not want to complicate it too much.
    - deleted 3y ago
      [deleted]
pyrophane 3y ago
Why do you say RabbitMQ is heavyweight? What queues do you consider more lightweight and what would be your go-to in a situation like this?
- KronisLV 3y ago
  They might be thinking of something like ZeroMQ, which is pretty well liked: https://zeromq.org/ https://zeromq.org/ That said, I wouldn't call RabbitMQ that heavyweight myself, at least when compared to something like Apache Kafka.

malux85 3y ago

This is what I was wondering, in the article it looks like kubernetes is just used to launch the node containers - why is the database and rabbitmq outside of kubernetes? This architecture looks like it’s been cobbled together by a junior

robertlagrant 3y ago
Kubernetes loves stateless services. Zero wrong with moving RabbitMQ or a database outside of it.
- malux85 3y ago
  Except kubernetes has a whole storage provisioning system that gives you redundancy and automatic failover, if you’re going to the trouble of running kubernetes why not just run your whole infra on it? I run https://atomictessellator.com https://atomictessellator.com solo, using kubernetes, and my database, Minio object store, application servers, quantum workers, everything is all on kubernetes, it’s self healing and much simpler to run all the infrastructure the same. Recently I had a node failure while I was sleeping and the whole system healed itself while I slept, the monitoring system didn’t even alarm me because the small blip of increased latency while the pods rebalanced wasn’t above the alert threshold so it didn’t even wake me up. What happens in the article infra when the rabbitmq or database nodes fail? The whole system goes offline, which seems very silly setup when you have kubernetes sitting right there, who’s primary function is to handle all of this.
  - robertlagrant 3y ago
    What happens when your storage detaches from your k8s cluster? Your services start 503ing, hopefully, because you didn't design your system thinking that k8s == 100% uptime.
    - malux85 3y ago
      Anybody can invent random problems ad nauseam - that doesn’t prove anything. I’m not claiming that it’s totally bullet proof, I never said that - I’m saying that if you had a kubernetes cluster anyway why not benefit from its abilities? Especially when the alternative is single node, single points of failure, which is clearly inferior. The "what if the storage detaches" argument could easily apply to the single node VMs too, in which case the outcome would be a total system failure. We are discussing the contrast between the articles architecture and running everything on K8s ... and I'm saying that running everything on K8s is clearly better
      - jrockway 3y ago
        Yeah, what happens when someone in AWS clicks "delete database" accidentally? It's the same thing that happens when K8s blows up in some weird way. You restore from your backup. (Fun fact: deleting the instance deletes the backups!)
        robertlagrant 3y ago
        Again, I was replying to: > What happens in the article infra when the rabbitmq or database nodes fail?
      - robertlagrant 3y ago
        > Anybody can invent random problems ad nauseam I agree; I was replying to this invented problem: > What happens in the article infra when the rabbitmq or database nodes fail? It makes sense if you read the reply as a reply.
  - codenesium 3y ago
baq 3y ago
There some of us who still perform four extra steps before putting any DB in k8s and we have good reasons.
sass_muffin 3y ago
It is actually the opposite, it is currently considered a good practice to run stateful workloads outside of kubernetes and stateless workloads inside of kubernetes.
- KronisLV 3y ago
  > It is actually the opposite, it is currently considered a good practice to run stateful workloads outside of kubernetes and stateless workloads inside of kubernetes. Is that still true? I wouldn't call the parent comment charitable enough, because there definitely can be some reasons for running stateful workloads even outside of containers altogether (familiarity included), but at the same time it feels like a lot of effort has been invested into making that a non-issue. For example, how many database Operators are now available for Kubernetes: https://operatorhub.io/?category=Database&capabilityLevel=%5B%22Seamless+Upgrades%22%2C%22Full+Lifecycle%22%2C%22Deep+Insights%22%2C%22Auto+Pilot%22%5D https://operatorhub.io/?category=Database&capabilityLevel=%5... Honestly, as long as you have storage and config setup correctly, it's not like you even need an Operator, that's for more advanced setups. I've been running databases in containers (even without Kubernetes) for years, haven't had that many issues at small/medium scale.

mihaitodor 3y ago

I believe loads of auxiliary microservices have been omitted for brevity. Of course, those also don’t require Kubernetes, but maybe they have some standardised deployment system which keeps things manageable. Don’t forget about Observability and whatnot.

deleted 3y ago

[deleted]

alexandreolive 3y ago

Hello, I'm the writer of the article. We are using Kubernetes for our whole architecture, consisting of around 40 microservices and cron jobs. I just wanted in this article to give an example of asynchronous architecture using Kubernetes and RabbitMQ. We are using RabbitMQ because it's my company target solution. There might better so lighter solution that would fit us but having just one for every solution is easier to maintain. Great comment about GPU hardware acceleration for encoding, I'm going to look this up.

andrewstuart 3y ago
So Kubernetes is only in this architecture because other systems use it and its required by the parent company but not needed. That's pretty important context.
- alexandreolive 3y ago
  That's not what I said; sorry if that was not clear. The parent company requires RabbitMQ, we are using Kubernetes because managing 40 microservices without it would be hell. In the article, I only showed 1 user-facing API, but it's actually multiple services, I just did not want to complicate it too much.
  - deleted 3y ago
    [deleted]

pyrophane 3y ago

Why do you say RabbitMQ is heavyweight? What queues do you consider more lightweight and what would be your go-to in a situation like this?

KronisLV 3y ago
They might be thinking of something like ZeroMQ, which is pretty well liked: https://zeromq.org/ https://zeromq.org/ That said, I wouldn't call RabbitMQ that heavyweight myself, at least when compared to something like Apache Kafka.