7 ms·

I'm not sure why people are so worried about the size of the executable file here. If the runtime.pclntab table is never[1] used then it won't be paged into me

by tytso 7y ago

I'm not sure why people are so worried about the size of the executable file here. If the runtime.pclntab table is never[1] used then it won't be paged into memory, and disk space is mostly free these days. [1] Well, hardly _ever_! (Sorry not sorry for the obligatory Gilbert and Sullivan reference.) If you're using the Go executable on a system without virtual memory support, yeah, that's going to suck, but it appears the Go runtime is horribly bloated and not really suited for super-tiny 16-bit processors in the micro-embedded space. But for something like Cockroachdb, why worry about the file size?

caiobegotti 7y ago
Bandwidth [to transfer big binaries around] is not free however.
- pushpop 7y ago
  True, but you’d need a lot of transfers before it starts to add up but if you run into that edge case then thankfully you don’t actually need the file to be executable during transit so you probably should compress the file for transit and decompress it at the other end (assuming your CI/CD pipeline allows for that) or compile it on the destiny nodes (since Go’s compiler is fast). Admittedly neither are perfect solutions but software architecting is always about making smart compromises.
F-0X 7y ago
> disk space is mostly free these days. This is the only "argument" ever presented, and I don't think it is any good. I care about file sizes. I want to get the most out of my hardware. Not needing to buy another drive is always going to be cheaper for me and every other user.
- trevor-e 7y ago
  Then don't use Go, you aren't their target audience in that case. And I don't mean this in a harsh way, just that Google is clearly opinionated in how they are building Go.
- everdev 7y ago
  > Not needing to buy another drive is always going to be cheaper for me and every other user. 128GB+ drives are standard on mid-range laptops. Even at 64GB are you really going to fill up disk space because of Go executables? CockroachDB (a large software project) is only 123MB. I doubt most people even have 100 pieces of non-default software on their laptop or that executables are going to fill up storage and break anyone's bank these days. If you're short on HD space, I'm typically targeting photos and videos, not software.
  - mimi89999 7y ago
    Well. Grow your entire system by 60%. Even with 128GB it will be non negligeable.
rb808 7y ago
I used to think that but now with containers its annoying to have to wait for a big binary image to get copied to the node and loaded up.
- koffiezet 7y ago
  In my experience, the bare Go container images are the smallest of them all, averaging out at 35mb here. The nodejs stuff clocks in at 500mb, of which only 130 is the shared base node image, the rest is "application" and dependencies...
siscia 7y ago
While it's true that disk space is virtually free, that is not true for bandwidth.
temac 7y ago
Also if you have multiple instances, I guess it is better to not allocate N versions of the same thing in anonymous memory.
- inlined 7y ago
  Since Go is statically typed, the runtime data should be constant. Couldn’t a copy on write cache mean that the logical RAM redundancy doesn’t actually affect real memory?
  - temac 7y ago
    If you have to decompress it at startup, you will typically do it to anonymous memory. You can attempt to be fancy at user-level by trying some silly tricks like putting them to shared memory, although I don't know the API of common ones enough to know if it is even possible in reality because of all the details to handle (ref count of the users with auto destruction when the last one closes, and that atomic with the creation of just one when none exists, etc.) Ideally, to get all the optims, you would want some compression support at FS level, or even a specialized mapper of data coming from executable files in the kernel (or in cooperation with the kernel), but this will bring added complexity. (Thinking more about it a solution involving a microkernel would be really cool, but I digress...)
    - inlined 7y ago
      I guess this would specifically be a benefit of fork/exec. Though would it need to be decompressed after 1.2? That was my assumption is that it trades speed for memory on the first launch, and memory in subsequent launches would be virtual only
reilly3000 7y ago
If you’re using something like GCP Cloud Run to execute containers on demand, cold start time (which affects both new invocations and scaling events) is directly impacted by container size. As you said, not as much of a concern for a database, but extremely relevant for an HTTP server.
deleted 7y ago
[deleted]

by tytso 7y ago

caiobegotti 7y ago

Bandwidth [to transfer big binaries around] is not free however.

pushpop 7y ago
True, but you’d need a lot of transfers before it starts to add up but if you run into that edge case then thankfully you don’t actually need the file to be executable during transit so you probably should compress the file for transit and decompress it at the other end (assuming your CI/CD pipeline allows for that) or compile it on the destiny nodes (since Go’s compiler is fast). Admittedly neither are perfect solutions but software architecting is always about making smart compromises.

F-0X 7y ago

> disk space is mostly free these days. This is the only "argument" ever presented, and I don't think it is any good. I care about file sizes. I want to get the most out of my hardware. Not needing to buy another drive is always going to be cheaper for me and every other user.

trevor-e 7y ago
Then don't use Go, you aren't their target audience in that case. And I don't mean this in a harsh way, just that Google is clearly opinionated in how they are building Go.
everdev 7y ago
> Not needing to buy another drive is always going to be cheaper for me and every other user. 128GB+ drives are standard on mid-range laptops. Even at 64GB are you really going to fill up disk space because of Go executables? CockroachDB (a large software project) is only 123MB. I doubt most people even have 100 pieces of non-default software on their laptop or that executables are going to fill up storage and break anyone's bank these days. If you're short on HD space, I'm typically targeting photos and videos, not software.
- mimi89999 7y ago
  Well. Grow your entire system by 60%. Even with 128GB it will be non negligeable.

rb808 7y ago

I used to think that but now with containers its annoying to have to wait for a big binary image to get copied to the node and loaded up.

koffiezet 7y ago
In my experience, the bare Go container images are the smallest of them all, averaging out at 35mb here. The nodejs stuff clocks in at 500mb, of which only 130 is the shared base node image, the rest is "application" and dependencies...

siscia 7y ago

While it's true that disk space is virtually free, that is not true for bandwidth.

temac 7y ago

Also if you have multiple instances, I guess it is better to not allocate N versions of the same thing in anonymous memory.

inlined 7y ago
Since Go is statically typed, the runtime data should be constant. Couldn’t a copy on write cache mean that the logical RAM redundancy doesn’t actually affect real memory?
- temac 7y ago
  If you have to decompress it at startup, you will typically do it to anonymous memory. You can attempt to be fancy at user-level by trying some silly tricks like putting them to shared memory, although I don't know the API of common ones enough to know if it is even possible in reality because of all the details to handle (ref count of the users with auto destruction when the last one closes, and that atomic with the creation of just one when none exists, etc.) Ideally, to get all the optims, you would want some compression support at FS level, or even a specialized mapper of data coming from executable files in the kernel (or in cooperation with the kernel), but this will bring added complexity. (Thinking more about it a solution involving a microkernel would be really cool, but I digress...)
  - inlined 7y ago
    I guess this would specifically be a benefit of fork/exec. Though would it need to be decompressed after 1.2? That was my assumption is that it trades speed for memory on the first launch, and memory in subsequent launches would be virtual only

reilly3000 7y ago

If you’re using something like GCP Cloud Run to execute containers on demand, cold start time (which affects both new invocations and scaling events) is directly impacted by container size. As you said, not as much of a concern for a database, but extremely relevant for an HTTP server.

deleted 7y ago

[deleted]