3 ms·
> Functionally, MicroShift repackages OpenShift core components* into a single binary that weighs in at a relatively tiny 160MB executable (without any compress
by latchkey 5y ago
> Functionally, MicroShift repackages OpenShift core components* into a single binary that weighs in at a relatively tiny 160MB executable (without any compression/optimization).
Sorry, 160meg isn't 'relatively tiny'.
I have many thousands of machines running in multiple datacenters and even getting a ~4mb binary distributed onto them without saturating the network (100mbit) and slowing everything else down, is a bit of a challenge.
Edit: It was just over 4mb using .gz and I recently changed to use xz, which decreased the size by about 1mb and I was excited about that given the scale that I operate at.
- dralley 5y agoRelative to Kubernetes (well, specifically OpenShift, which is Kubernetes packaged with a lot of commonly-needed extra functionality / tools) A "hello world" written in Go is 2mb to begin with so ~4mb is a bit unrealistic for any substantial piece of software written in Go. Although if that colors your opinion of Go itself, you're certainly allowed to have that opinion :)
- latchkey 5y agoYour logic doesn't work because adding functionality doesn't necessary translate to a huge increase in binary size. The complex app (15k lines of code) that is distributed to these machines, written in Go, is just over 3mb compressed with xz. Plus this is still many orders of magnitude off with trying to get 160mb off to mobile devices per the image in the article. It is a non-starter.
- deleted 5y ago[deleted]
- dralley 5y agoI'm going to take a guess that even a "mini" distribution of Kubernetes is more than 15k lines of code though. k3s is quite a bit smaller than this but it's still only described as "<50 megabytes"
- latchkey 5y agoI'm not sure the point of your argument. Somehow we should just accept 160meg downloads to a phone, because why?
- dralley 5y agoI don't understand the point of your argument, because nobody (including the announcement above) is talking about phones. From the post: > Imagine one of these small devices located in a vehicle to run AI algorithms for autonomous driving, or being able to monitor oil and gas plants in remote locations to perform predictive maintenance, or running workloads in a satellite as you would do in your own laptop. Now contrast this to centralized, highly controlled data centers where power and network conditions are usually very stable thanks to high available infrastructure — this is one of the key differences that define edge environments. > Field-deployed devices are often Single Board Computers (SBCs) chosen based on performance-to-energy/cost ratio, usually with lower-end memory and CPU options. These devices are centrally imaged by the manufacturer or the end user’s central IT before getting shipped to remote sites such as roof-top cabinets housing 5G antennas, manufacturing plants, etc. > At the remote site, a technician will screw the device to the wall, plug the power and the network cables and the work is done. Provisioning of these devices is “plug & go” with no console, keyboard or qualified personnel. In addition, these systems lack out-of-band management controllers so the provisioning model totally differs from those that we use with regular full-size servers. I don't read this and think "phones". This sounds like it's targeted at embedded industrial / telecom devices. (At least based on the examples they chose, I'm sure you could use it for other things). The word "mobile" doesn't actually appear anywhere on the page so I'm not sure where you got "mobile devices" from.
- jrockway 5y agoI recently wrote a single-process k8s distribution (for tests). Compiled into a linux/amd64 binary with no special settings it's 183M with go1.18beta1. Kubernetes is a lot of code, plain and simple.
- hrez 5y ago
- The_Colonel 5y ago> The complex app (15k lines of code) I would argue that 15K lines of code is far from a complex app as is normally understood. It's a rather small app, especially in Go which isn't exactly the most expressive language out there.
- dark-star 5y agoThey reported the 160mb as uncompressed (and probably unstripped?) size. If you compress that it will be aroudn 50mb, still more than your 3mb xz example but at least you're now comparing apples to apples
- capableweb 5y ago> I have many thousands of machines running in multiple datacenters and even getting a ~4mb binary distributed onto them without saturating the network (100mbit) and slowing everything else down, is a bit of a challenge. Maybe I suggest CAS (Content-addressable storage) or something similar for distributing it instead? I've had good success using torrents for distributing large binaries to a large fleet of servers (that were also in close proximity to each other physically, in clusters with some more distance between them) relatively easily.
- latchkey 5y agoThanks for the suggestion, but as weird as this sounds, we also don't have central servers to use at the data centers for this. The machines don't all need to be running the same version of the binary at the same time, so I took a simpler approach, which is that each machine checks for updates on a random schedule over a configureable amount of time. This distributes the load evenly and everything becomes eventually consistent. After about an hour everything is updated without issues. I use CloudFlare workers to cache at the edge. On a push to master, the binary is built in Github CI and a release is made after all the tests pass. There is a simple json file where I can define release channels for a specific version for a specific CIDR (also on a release ci/cd as well, so I can validate the json with tests). I can upgrade/downgrade and test on subsets of machines. The machines on their random schedule hit the CF worker which checks the cached json file and either returns a 304 or the binary to install depending on the parameters passed in on the query string (current version, ip address, etc.). Binary is downloaded, installed and then it quits and systemd restarts the new version. Works great.
- aspenmayer 5y agoI’d seriously look into BitTorrent for this use case, as it sounds ideal. You can even configure your client to run a script after torrents are completed, so you could script the update migration with minimal code. You can also setup upload/download limits easily. I think Resilio Sync might also be a good option; I think it may even use BitTorrent internally. (It was formerly known as BitTorrent Sync, not sure why they changed the name.)
- hosteur 5y agoSounds like a use case for BitTorrent.
- latchkey 5y agoSeparate vlans
- marktangotango 5y ago> without saturating the network (100mbit) Datacenter to datacenter and only 100Mb? Clearly more to the story here.... :)
- latchkey 5y agoWithin the data center.
- hericium 5y ago> I have many thousands of machines running in multiple datacenters and even getting a ~4mb binary distributed onto them without saturating the network (100mbit) and slowing everything else down, is a bit of a challenge. You may find murder[1] / Herd[2] / Horde[3] -type tool of some use. [1] https://github.com/lg/murder https://github.com/lg/murder [2] https://github.com/russss/Herd https://github.com/russss/Herd [3] https://github.com/naterh/Horde https://github.com/naterh/Horde
- freedomben 5y ago"relatively" is the operative word there. Compared to regular/full openshift, it is tiny. I would imagine they chose the word "relative" because in absolute terms nobody would call 160mb tiny.
- eyegor 5y agoIf you're really that sensitive to size, may want to try 7z. I can usually get a few % smaller archive sizes than xz with faster decompression to boot. Of course then you might need to install a 7z lib which could be an issue.
- latchkey 5y agoThanks, I just tested that idea out: Input binary size: 12996608 Folder contains a few more bytes for the service file and installer script. 4476005 (gz) 3421456 (xz) 3447940 (7Z) tar c -C ./build $(BINARY) | gzip -9 - > $(PKG_NAME_GZ) tar c -C ./build $(BINARY) | xz -z -9e - > $(PKG_NAME_XZ) tar c -C ./build $(BINARY) | 7z a -si $(PKG_NAME_7Z) I played around with the compression options on xz. If you have some suggestions on improving 7z, I'm all ears. Decompression time isn't an issue here. Installing 7z on the hosts isn't great, but could be done.
- eyegor 5y agoYou could try the compression options for 7z, not sure if they would help in your case since I don't know the defaults off hand. Here's an example from the man page. > ... | 7z a -si -m0=lzma -mx=9 -mfb=64 -md=32m https://linux.die.net/man/1/7za https://linux.die.net/man/1/7za
- latchkey 5y agoThanks again. Using those exact options. 3421498 (7z) 3420832 (xz) Wild that xz is not the same every time.
- orf 5y ago160mb is indeed relatively tiny compared to a full k8s distribution. The "i have issues distributing a tiny 4mb binary to my severs" sized hole that you've dug yourself doesn't change that.