5 ms·

Dropping by to express healthy interest in this project. Rancher have a pretty good track record so far: - the Rancher platform itself (https://rancher.com/

Dropping by to express healthy interest in this project. Rancher have a pretty good track record so far: - the Rancher platform itself (https://rancher.com/) is a really powerful and user friendly way to manage container clusters of all sorts, giving you a self-hosted dashboard for both your cloud and on prem clusters, for a variety of Kubernetes distributions; you can even manage the available drivers and also create deployments graphically - the K3s distribution (https://k3s.io/) is in my eyes one of the best ways to run Kubernetes yourself, both in development environments and production ones. I benchmarked K3s alongside Docker Swarm as a part of my Master's thesis and it was surprising to see that its overhead was actually very close to that of Docker Swarm (a more lightweight orchestrator that's included with Docker and uses the Docker Compose format), only exceeding it by a few hundred MB with similar deployments being active, making K3s passable for small nodes As for this particular project, it's very positive to see that it supports all of the big OSes, even though the 0.6.0 version tag would still advise caution for a while, even though it can definitely be considered as a replacement for Docker Desktop. Admittedly, it's also nice to see that Docker and the ecosystem around it is still supported and is alive and kicking, since for many projects out there it's a perfectly serviceable stack with tools that a lot of people will be familiar with, as opposed to having to migrate to podman, while it's still becoming more and more stable, yet isn't quite there yet. Now, that may be a controversial take, and Docker Inc also have their fair share of challenges, about which there was a very nice writeup here: https://www.infoworld.com/article/3632142/how-docker-broke-in-half.html https://www.infoworld.com/article/3632142/how-docker-broke-i...

rightbyte 5y ago
> Admittedly, it's also nice to see that Docker and the ecosystem around it is still supported and is alive and kicking When the controlling organization is starting to go down the user hostile route (e.g. payed update opt-outs), it is more or less sad to see that the Docker is alive and kicking. Developers should ran away as fast as possible.
- elzbardico 5y ago
  I see it as a legitimate way to pay programmers. You still have the core functionality, but the kind of features that the enterprise likes is paid and supports the development of open source. Why should everyone work for free to enrich the Kleiner Perkins of the world?
hardwaresofton 5y ago
They've also got Longhorn, a distributed container-attached storage solution that's very simple to understand and easy to deploy. Performance is another thing but that's the same with all of the general networked storage solutions (Ceph included). Rancher's got a well deserved good impression in my mind, though early on I avoided them somewhat. [0]: https://github.com/longhorn/longhorn https://github.com/longhorn/longhorn
- BiteCode_dev 5y ago
  Wasn't it the code name for Vista ?
  - stib 5y ago
    Yes, but a different Longhorn. The early 00s Windows codenames all had a PNW skiing theme, and that Longhorn is a well-known bar at the base of Whistler mountain.
- remram 5y ago
  More than performance, I am worried about security. They don't seem to have considered it at all when building it: https://github.com/longhorn/longhorn/issues/1805 https://github.com/longhorn/longhorn/issues/1805 Probably not a good recommendation until this hole is plugged.
TobTobXX 5y ago
Also there's k3d (k3s in Docker). It lets you run all kinds of k3s clusters on the desktop with just a simple command. Great way to test your k3s deployments.
supermatt 5y ago
Looking over k3s and it really pushes the IoT/embedded aspect. Any comments on what makes it different to a server-oriented k8s solution? Why wouldnt I run it on servers?
- mfer 5y ago
  k3s, by default, uses sqlite for storage instead of etcd. This is one of the ways you can get a performance improvement. If you are going to run a cluster you're likely going to want to have your database be an HA cluster. While k3s can support other databases, like PostgreSQL, it doesn't have handling for clusters in the config. This is a limitation I see for k3s in clusters, right now. I would love to see cluster handling in k3s. etcd can be a scaling bottleneck and replacing it with PostgreSQL could help it scale much higher. Disclaimer, I work on Rancher Desktop which uses k3s
  - zaphirplane 5y ago
    Assuming k8s still uses watch on etcd keys how is that implemented for SQLite ? I suppose polling is possible way if it’s low tps desktop focused
    - h3h3 5y ago
      yes looks to be polling https://github.com/k3s-io/kine/blob/master/pkg/server/watch.go https://github.com/k3s-io/kine/blob/master/pkg/server/watch....
  - skrtskrt 5y ago
    As someone using k3s on a single node right now, could you clarify/expand on "it doesn't have handling for clusters in the config"?
    - pixiemagic 5y ago
      I think they are talking about using a clustered RDMBS as storage for k3s, rather than running multiple nodes as part of your Kubernetes cluster.
mindrunner 5y ago
any chance we could look at your thesis when it is done? :)
- KronisLV 5y ago
  I finished it about a year ago, but due to reasons outside of my control (administrative decisions), i had to write it in my native language, Latvian, so it's probably a tad useless to a wider audience. In case anyone desires to mess around with PDFs and machine translation: - research praxis: https://files.kronis.dev/s/AJRCs84D7WngLzD - development praxis: https://files.kronis.dev/s/Xz6mtbAamoA7Pe8 - the full text: https://files.kronis.dev/s/ioiW96dpnD5YcLk Here's a tl;dr summary of what i did during it: essentially i set out to improve the way applications are run within the infrastructure of the company that currently employs me. To achieve that, i researched both the ways to utilize systemd services for everything as well as improve configuration management with Ansible, later introduce containers into the mix and compare their orchestrators (Docker Swarm and K3s in this case), as well as how to manage them, in this case, with Portainer. Then, after finishing that stuff up for the company, i proceeded with some further research of my own, into whether it's feasible to develop bespoke server configuration management tools and to integrate them with container management technologies, as well as do further benchmarks on how well the orchestrators actually handle containers running under load, since this doesn't often get tested. There are probably a few reasons for those choices: - at work, it was really disappointing to see environments where servers are started manually - similarly, in the current day and age it's not acceptable to keep domain knowledge about how to start services and where the configuration is to yourself - manual configuration management simply increases the risks of human error greatly, especially due to turnover - in contrast, containers are a surprisingly usable way to introduce "infrastructure as code" - that said, their orchestrators still need a lot of work and it's probably a good idea to compare them - i was also curious to see whether it would be hard to create my own automation tools like Ansible, but executing remote Bash scripts through SSH - lastly, i was a participant in the development on https://apturicovid.lv/ which was a Latvian COVID contact tracing solution; i was curious about the choice of using Ruby, so i wanted to create my own mock system to see how it'd perform under load in a real world scenario, with tools like K6s https://k6.io/ More importantly, i wanted to see how much load a GPS based approach (as opposed to Bluetooth) with PostGIS would generate and whether Ruby would still be okay, as described on my blog https://blog.kronis.dev/articles/covid-19-contact-tracing-with-gps-versus-bluetooth-and-the-unexplored-potential-for-the-apps https://blog.kronis.dev/articles/covid-19-contact-tracing-wi... And, without further ado, here's my findings: - if you're not using Ansible or another alternative for managing the configuration of your environments, you should definitely look into it - if you can't or don't want to run containers, at least consider having systemd services with environment config files on your servers - Docker Swarm and the lighter distributions of Kubernetes, like K3s are largely comparable, even though Kubernetes takes a little bit more resources - tools like Portainer can make managing either or both of them a breeze, since it's one of the few solutions that support both Docker, Docker Swarm and Kubernetes clusters - despite that, you'll still probably want to configure your applications (e.g. HTTP thread pool sizes, DB pool sizes) and introduce server monitoring or APM into the mix - as for Python, it's pretty good for developing all sorts of solutions, even for remote server management: with Paramiko (for SSH), jsonpickle (serialization), Typer (CLI apps) and other libraries - for my tool, i used JSON as an application data format, so one command could pipe its JSON output (e.g. returned tokens from cluster leader server for follower servers) as an input for another; there are few cases where this can work, but when it does, it is pretty nice - that said, any such tool that you write will probably just be a novelty and in 90% of the cases you should just look at established solutions out there - load testing is actually pretty feasible with tools like K6s, but only as long as you just want to test Web APIs - anything more complex like that might be better tested with something like a server farm running instances of Selenium Here's a few Git repos of the projects: - A very simple COVID infection tracking system which generates heatmaps to attempt to preserve anonimity: https://git.kronis.dev/rtu1/kvps5_masters_degree_covid_1984 - The K3s benchmarks for it: https://git.kronis.dev/rtu1/kvps5_masters_degree_covid_1984_load_test - The Python configuration management tool: https://git.kronis.dev/rtu1/kvps5_masters_degree_astolfo_cloud_servant Oh, and here's a test environment for the mock system that shows the generated heatmaps, you'll need to press the "Start" button to preview the data, though you can change the visualization parameters at runtime: https://covid1984.kronis.dev/ https://covid1984.kronis.dev/

rightbyte 5y ago

> Admittedly, it's also nice to see that Docker and the ecosystem around it is still supported and is alive and kicking When the controlling organization is starting to go down the user hostile route (e.g. payed update opt-outs), it is more or less sad to see that the Docker is alive and kicking. Developers should ran away as fast as possible.

elzbardico 5y ago
I see it as a legitimate way to pay programmers. You still have the core functionality, but the kind of features that the enterprise likes is paid and supports the development of open source. Why should everyone work for free to enrich the Kleiner Perkins of the world?

hardwaresofton 5y ago

They've also got Longhorn, a distributed container-attached storage solution that's very simple to understand and easy to deploy. Performance is another thing but that's the same with all of the general networked storage solutions (Ceph included). Rancher's got a well deserved good impression in my mind, though early on I avoided them somewhat. [0]: https://github.com/longhorn/longhorn https://github.com/longhorn/longhorn

BiteCode_dev 5y ago
Wasn't it the code name for Vista ?
- stib 5y ago
  Yes, but a different Longhorn. The early 00s Windows codenames all had a PNW skiing theme, and that Longhorn is a well-known bar at the base of Whistler mountain.
remram 5y ago
More than performance, I am worried about security. They don't seem to have considered it at all when building it: https://github.com/longhorn/longhorn/issues/1805 https://github.com/longhorn/longhorn/issues/1805 Probably not a good recommendation until this hole is plugged.

TobTobXX 5y ago

Also there's k3d (k3s in Docker). It lets you run all kinds of k3s clusters on the desktop with just a simple command. Great way to test your k3s deployments.

supermatt 5y ago

Looking over k3s and it really pushes the IoT/embedded aspect. Any comments on what makes it different to a server-oriented k8s solution? Why wouldnt I run it on servers?

mfer 5y ago
k3s, by default, uses sqlite for storage instead of etcd. This is one of the ways you can get a performance improvement. If you are going to run a cluster you're likely going to want to have your database be an HA cluster. While k3s can support other databases, like PostgreSQL, it doesn't have handling for clusters in the config. This is a limitation I see for k3s in clusters, right now. I would love to see cluster handling in k3s. etcd can be a scaling bottleneck and replacing it with PostgreSQL could help it scale much higher. Disclaimer, I work on Rancher Desktop which uses k3s
- zaphirplane 5y ago
  Assuming k8s still uses watch on etcd keys how is that implemented for SQLite ? I suppose polling is possible way if it’s low tps desktop focused
  - h3h3 5y ago
    yes looks to be polling https://github.com/k3s-io/kine/blob/master/pkg/server/watch.go https://github.com/k3s-io/kine/blob/master/pkg/server/watch....
- skrtskrt 5y ago
  As someone using k3s on a single node right now, could you clarify/expand on "it doesn't have handling for clusters in the config"?
  - pixiemagic 5y ago
    I think they are talking about using a clustered RDMBS as storage for k3s, rather than running multiple nodes as part of your Kubernetes cluster.

mindrunner 5y ago

any chance we could look at your thesis when it is done? :)

KronisLV 5y ago
I finished it about a year ago, but due to reasons outside of my control (administrative decisions), i had to write it in my native language, Latvian, so it's probably a tad useless to a wider audience. In case anyone desires to mess around with PDFs and machine translation: - research praxis: https://files.kronis.dev/s/AJRCs84D7WngLzD - development praxis: https://files.kronis.dev/s/Xz6mtbAamoA7Pe8 - the full text: https://files.kronis.dev/s/ioiW96dpnD5YcLk Here's a tl;dr summary of what i did during it: essentially i set out to improve the way applications are run within the infrastructure of the company that currently employs me. To achieve that, i researched both the ways to utilize systemd services for everything as well as improve configuration management with Ansible, later introduce containers into the mix and compare their orchestrators (Docker Swarm and K3s in this case), as well as how to manage them, in this case, with Portainer. Then, after finishing that stuff up for the company, i proceeded with some further research of my own, into whether it's feasible to develop bespoke server configuration management tools and to integrate them with container management technologies, as well as do further benchmarks on how well the orchestrators actually handle containers running under load, since this doesn't often get tested. There are probably a few reasons for those choices: - at work, it was really disappointing to see environments where servers are started manually - similarly, in the current day and age it's not acceptable to keep domain knowledge about how to start services and where the configuration is to yourself - manual configuration management simply increases the risks of human error greatly, especially due to turnover - in contrast, containers are a surprisingly usable way to introduce "infrastructure as code" - that said, their orchestrators still need a lot of work and it's probably a good idea to compare them - i was also curious to see whether it would be hard to create my own automation tools like Ansible, but executing remote Bash scripts through SSH - lastly, i was a participant in the development on https://apturicovid.lv/ which was a Latvian COVID contact tracing solution; i was curious about the choice of using Ruby, so i wanted to create my own mock system to see how it'd perform under load in a real world scenario, with tools like K6s https://k6.io/ More importantly, i wanted to see how much load a GPS based approach (as opposed to Bluetooth) with PostGIS would generate and whether Ruby would still be okay, as described on my blog https://blog.kronis.dev/articles/covid-19-contact-tracing-with-gps-versus-bluetooth-and-the-unexplored-potential-for-the-apps https://blog.kronis.dev/articles/covid-19-contact-tracing-wi... And, without further ado, here's my findings: - if you're not using Ansible or another alternative for managing the configuration of your environments, you should definitely look into it - if you can't or don't want to run containers, at least consider having systemd services with environment config files on your servers - Docker Swarm and the lighter distributions of Kubernetes, like K3s are largely comparable, even though Kubernetes takes a little bit more resources - tools like Portainer can make managing either or both of them a breeze, since it's one of the few solutions that support both Docker, Docker Swarm and Kubernetes clusters - despite that, you'll still probably want to configure your applications (e.g. HTTP thread pool sizes, DB pool sizes) and introduce server monitoring or APM into the mix - as for Python, it's pretty good for developing all sorts of solutions, even for remote server management: with Paramiko (for SSH), jsonpickle (serialization), Typer (CLI apps) and other libraries - for my tool, i used JSON as an application data format, so one command could pipe its JSON output (e.g. returned tokens from cluster leader server for follower servers) as an input for another; there are few cases where this can work, but when it does, it is pretty nice - that said, any such tool that you write will probably just be a novelty and in 90% of the cases you should just look at established solutions out there - load testing is actually pretty feasible with tools like K6s, but only as long as you just want to test Web APIs - anything more complex like that might be better tested with something like a server farm running instances of Selenium Here's a few Git repos of the projects: - A very simple COVID infection tracking system which generates heatmaps to attempt to preserve anonimity: https://git.kronis.dev/rtu1/kvps5_masters_degree_covid_1984 - The K3s benchmarks for it: https://git.kronis.dev/rtu1/kvps5_masters_degree_covid_1984_load_test - The Python configuration management tool: https://git.kronis.dev/rtu1/kvps5_masters_degree_astolfo_cloud_servant Oh, and here's a test environment for the mock system that shows the generated heatmaps, you'll need to press the "Start" button to preview the data, though you can change the visualization parameters at runtime: https://covid1984.kronis.dev/ https://covid1984.kronis.dev/