3 ms·

I recently wrote a single-process k8s distribution (for tests). Compiled into a linux/amd64 binary with no special settings it's 183M with go1.18beta1. Kubern

I recently wrote a single-process k8s distribution (for tests). Compiled into a linux/amd64 binary with no special settings it's 183M with go1.18beta1. Kubernetes is a lot of code, plain and simple.

hrez 5y ago
I found go compiler isn't very efficient in not linking in the dead code. I ran into it by switching from aws-sdk-go to aws-sdk-go-v2 and the binary jumped from 27Mb to 66Mb [1] Granted fully featured app will likely use all of the module's code so it's not a factor. [1] https://github.com/aws/aws-sdk-go-v2/discussions/1532 https://github.com/aws/aws-sdk-go-v2/discussions/1532
- jrockway 5y ago
  Yeah, I have a feeling that these giant autogenerated clients are tough to optimize. Too many layers of indirection; even the compiler is confused and just crosses its fingers and hopes it doesn't have to investigate too much.
smarterclayton 5y ago
If you use the reflection package in Go, the dead code elimination optimization step does not run if you use reflect.Method to look up a method because you can use reflect to reach those methods and the linker preserves the possibility. Unfortunately, almost every large client library has public methods AND references someone who calls the reflection method somewhere, so in many cases you get the full tree of dependencies. It’s unfortunate because any one dependency can bring in reflect and trigger the behavior. It hurts large generated method libraries the most since you usually only use a small subset. See https://go.dev/src/cmd/link/internal/ld/deadcode.go https://go.dev/src/cmd/link/internal/ld/deadcode.go line 318
- jrockway 5y ago
  That makes sense. Reflection is the enemy of anyone trying to produce a compact binary (or webpack bundle). A quick grep of the k8s codebase and there are plenty of situations that kill the dead code elimination: staging/src/k8s.io/apimachinery/pkg/runtime/scheme.go: if m := reflect.ValueOf(obj).MethodByName("DeepCopyInto"); m.IsValid() && m.Type().NumIn() == 1 && m.Type().NumOut() == 0 && m.Type().In(0) == reflect.TypeOf(obj) { That's quite the line of code too.

I recently wrote a single-process k8s distribution (for tests). Compiled into a linux/amd64 binary with no special settings it's 183M with go1.18beta1. Kubernetes is a lot of code, plain and simple.

hrez 5y ago

I found go compiler isn't very efficient in not linking in the dead code. I ran into it by switching from aws-sdk-go to aws-sdk-go-v2 and the binary jumped from 27Mb to 66Mb [1] Granted fully featured app will likely use all of the module's code so it's not a factor. [1] https://github.com/aws/aws-sdk-go-v2/discussions/1532 https://github.com/aws/aws-sdk-go-v2/discussions/1532

jrockway 5y ago
Yeah, I have a feeling that these giant autogenerated clients are tough to optimize. Too many layers of indirection; even the compiler is confused and just crosses its fingers and hopes it doesn't have to investigate too much.

smarterclayton 5y ago

If you use the reflection package in Go, the dead code elimination optimization step does not run if you use reflect.Method to look up a method because you can use reflect to reach those methods and the linker preserves the possibility. Unfortunately, almost every large client library has public methods AND references someone who calls the reflection method somewhere, so in many cases you get the full tree of dependencies. It’s unfortunate because any one dependency can bring in reflect and trigger the behavior. It hurts large generated method libraries the most since you usually only use a small subset. See https://go.dev/src/cmd/link/internal/ld/deadcode.go https://go.dev/src/cmd/link/internal/ld/deadcode.go line 318

jrockway 5y ago
That makes sense. Reflection is the enemy of anyone trying to produce a compact binary (or webpack bundle). A quick grep of the k8s codebase and there are plenty of situations that kill the dead code elimination: staging/src/k8s.io/apimachinery/pkg/runtime/scheme.go: if m := reflect.ValueOf(obj).MethodByName("DeepCopyInto"); m.IsValid() && m.Type().NumIn() == 1 && m.Type().NumOut() == 0 && m.Type().In(0) == reflect.TypeOf(obj) { That's quite the line of code too.