7 ms·

> In other words, the Go team decided to make executable files larger to save up on initialization time. I mean... Im genuinely curious if this is a "we have e

> In other words, the Go team decided to make executable files larger to save up on initialization time. I mean... Im genuinely curious if this is a "we have extra engineering resources and can explore/complain about this" or "we have a client who is running cockroachdb and can't handle a 172mb binary install for a database server". Is there really someone out there who installs Cockroach (a global distributed auto-sharded database) and thinks twice about 172mb of disk space? Sure, it'd be nice to have smaller binaries but outside of some embedded applications Go's binaries sizes are well within the nothing-burger range for most compute systems.

gtirloni 6y ago
It affects the download and instantiation time for containers.
- henvic 6y ago
  One word: trade-offs.
- mikepurvis 6y ago
  I worry about that for 10gb omnibus containers, not so much for <500mb.
- parhamn 6y ago
  The article says they don't really care about initialization time though, which is right. Remember: cockroachdb is always synchronizing data across the cluster, that 175mb of ingress to start up a DB node, probably pales in comparison to data synchronization/relocations that happen on a cluster. Which is why worrying about ingress/egress costs over binary size is nonsense here too. The bandwidth you need to run a distributed database cluster could download 172mb binary in milliseconds. If your node initiation time for DB failovers needs anything faster, you're doing something wrong. There are stakeholders to this problem, Cockroach probably isn't one.
  - spockz 6y ago
    For production, yes. But it also effects startup and download time on developer machines. Want to have multiple versions installed? Now it takes more space. Takes longer to download on 4g while on the road or on crappy corporate/conference WiFi, etc . In the end this all ends up because it is for all go binaries. I’ve come appreciate attention for leanness because in the end it does add up.
    - ClumsyPilot 6y ago
      Here is a question: imagine you could double performance of cockroa hDB by making the executable 2000MB - every db admin would make that choice
      - knz42 6y ago
        This is effectively what is happening btw; the crdb binary went from 80MB to 200MB in the same time it took to make it twice as fast. The % growth in size is not a problem on its own; it's more the % size attributed to the program vs. the % size attributed to unclear purposes, that's a problem.
        aflag 6y ago
        What's the use case where a 200MB binary size is a problem?
        knz42 6y ago
        The use case where said binary is shipped to GCE instances hundreds/thousands times per day, for stress testing and unit testing of cockroachdb.
        aflag 6y ago
        If most of the bytes are indeed cruft, can't you just send binary diffs around? I imagine they should change that much between compilations? It seems to be an issue for db developers but not for users.
- Thaxll 6y ago
  172MB for a Docker image that contains a DB is pretty small, Go Docker image are among the smallest because Go binary can run with minimal deps.
- sagichmal 6y ago
  This isn't an interesting or convincing argument because it revolves around a non-satisfiable metric. No matter the size of an artifact, this line of reasoning can always be used to claim it's too big and needs to be smaller.
Touche 6y ago
Go binary size makes it not an option for wasm.
- gen220 6y ago
  There are go variants for this. See tinygo [1], which targeted embedded originally (iirc) but now also targets wasm. So you’re definitely correct that core Go is not an option, but options exist within the “greater metropolitan area” that’s built up around downtown. These are among the benefits of having a relatively simple language and a 1.0 compatibility commitment, I think. [1]: https://tinygo.org/ https://tinygo.org/ their FAQ is quite good.
  - oefrha 6y ago
    Tinygo is severely limited, e.g. it doesn't support encoding/json. Want to deal with any JSON API, or anything that indirectly uses JSON serialization? Forget about it. A current side project of mine uses golang's wasm. Not a big codebase, but the wasm is 2.7MB brotli'ed. Certainly huge to me (I'm sure it's almost in the lean and mean camp compared to the average website today, though).
    - gray_-_wolf 6y ago
      encoding/json sucks anyway, so using something 3rd party would likely be better option anyway
      - wlll 6y ago
        How so?
        gen220 6y ago
        I think it’s well demonstrated to be slow (ns/op) and wasteful (allocs/op) in benchmarks, especially in bigger payloads. The HTTP library is similar. Most 3rd party libs make some trade offs that are pretty reasonable to achieve superior perf, but it’s worth understanding them to make sure it’s best for your use case.
blinkingled 6y ago
So tomorrow if the Go team doubled your binary size by adding "darkest bytes" area with garbage in it your response would be to question the users's inability to be ok with it and not the Go team's choice and reasoning?
- whateveracct 6y ago
  Feels like the Go team could just...expose a flag? Which they are loathe to do
  - majjgepolja 6y ago
    Flags are juvenile. When I was young I used to play with flags. Now I set environment variables.
- parhamn 6y ago
  If you think those are the alternatives, I don't think the rest of this discussion will go well.
dvfjsdhgfv 6y ago
> Is there really someone out there who installs Cockroach (a global distributed auto-sharded database) and thinks twice about 172mb of disk space? I think it is exactly this mindset that caused the current situation (that the 2/3 of the compiled binaries are useless to the users).
- dbt00 6y ago
  > (that the 2/3 of the compiled binaries are useless to the users). That the bytes are not visible in the symbol table is inarguable, that they are useless is a highly contentious statement and very probably wrong.
  - dvfjsdhgfv 6y ago
    It's possible, but it is the "who cares" mindset I object to. File sizes are important. Memory usage is important. CPU cycles are important. Many devs with powerful machines don't give a heck and in the end everybody has to pay, in different ways.
- parhamn 6y ago
  The alternative isn't avoiding fixing it. The alternative is a boring bug report on github with actual stakeholders discussing the best strategies and trade offs. This is only on HN's homepage because of the langauge flame wars. It's a garbage post.
  - dvfjsdhgfv 6y ago
    I agree 100% and I sincerely hope the author did that already. Just complaining about it to vent off one's frustration will accomplish nothing.
deleted 6y ago
[deleted]

gtirloni 6y ago

It affects the download and instantiation time for containers.

henvic 6y ago
One word: trade-offs.
mikepurvis 6y ago
I worry about that for 10gb omnibus containers, not so much for <500mb.
parhamn 6y ago
The article says they don't really care about initialization time though, which is right. Remember: cockroachdb is always synchronizing data across the cluster, that 175mb of ingress to start up a DB node, probably pales in comparison to data synchronization/relocations that happen on a cluster. Which is why worrying about ingress/egress costs over binary size is nonsense here too. The bandwidth you need to run a distributed database cluster could download 172mb binary in milliseconds. If your node initiation time for DB failovers needs anything faster, you're doing something wrong. There are stakeholders to this problem, Cockroach probably isn't one.
- spockz 6y ago
  For production, yes. But it also effects startup and download time on developer machines. Want to have multiple versions installed? Now it takes more space. Takes longer to download on 4g while on the road or on crappy corporate/conference WiFi, etc . In the end this all ends up because it is for all go binaries. I’ve come appreciate attention for leanness because in the end it does add up.
  - ClumsyPilot 6y ago
    Here is a question: imagine you could double performance of cockroa hDB by making the executable 2000MB - every db admin would make that choice
    - knz42 6y ago
      This is effectively what is happening btw; the crdb binary went from 80MB to 200MB in the same time it took to make it twice as fast. The % growth in size is not a problem on its own; it's more the % size attributed to the program vs. the % size attributed to unclear purposes, that's a problem.
      - aflag 6y ago
        What's the use case where a 200MB binary size is a problem?
        knz42 6y ago
        The use case where said binary is shipped to GCE instances hundreds/thousands times per day, for stress testing and unit testing of cockroachdb.
        aflag 6y ago
        If most of the bytes are indeed cruft, can't you just send binary diffs around? I imagine they should change that much between compilations? It seems to be an issue for db developers but not for users.
Thaxll 6y ago
172MB for a Docker image that contains a DB is pretty small, Go Docker image are among the smallest because Go binary can run with minimal deps.
sagichmal 6y ago
This isn't an interesting or convincing argument because it revolves around a non-satisfiable metric. No matter the size of an artifact, this line of reasoning can always be used to claim it's too big and needs to be smaller.

Touche 6y ago

Go binary size makes it not an option for wasm.

gen220 6y ago
There are go variants for this. See tinygo [1], which targeted embedded originally (iirc) but now also targets wasm. So you’re definitely correct that core Go is not an option, but options exist within the “greater metropolitan area” that’s built up around downtown. These are among the benefits of having a relatively simple language and a 1.0 compatibility commitment, I think. [1]: https://tinygo.org/ https://tinygo.org/ their FAQ is quite good.
- oefrha 6y ago
  Tinygo is severely limited, e.g. it doesn't support encoding/json. Want to deal with any JSON API, or anything that indirectly uses JSON serialization? Forget about it. A current side project of mine uses golang's wasm. Not a big codebase, but the wasm is 2.7MB brotli'ed. Certainly huge to me (I'm sure it's almost in the lean and mean camp compared to the average website today, though).
  - gray_-_wolf 6y ago
    encoding/json sucks anyway, so using something 3rd party would likely be better option anyway
    - wlll 6y ago
      How so?
      - gen220 6y ago
        I think it’s well demonstrated to be slow (ns/op) and wasteful (allocs/op) in benchmarks, especially in bigger payloads. The HTTP library is similar. Most 3rd party libs make some trade offs that are pretty reasonable to achieve superior perf, but it’s worth understanding them to make sure it’s best for your use case.

blinkingled 6y ago

So tomorrow if the Go team doubled your binary size by adding "darkest bytes" area with garbage in it your response would be to question the users's inability to be ok with it and not the Go team's choice and reasoning?

whateveracct 6y ago
Feels like the Go team could just...expose a flag? Which they are loathe to do
- majjgepolja 6y ago
  Flags are juvenile. When I was young I used to play with flags. Now I set environment variables.
parhamn 6y ago
If you think those are the alternatives, I don't think the rest of this discussion will go well.

dvfjsdhgfv 6y ago

> Is there really someone out there who installs Cockroach (a global distributed auto-sharded database) and thinks twice about 172mb of disk space? I think it is exactly this mindset that caused the current situation (that the 2/3 of the compiled binaries are useless to the users).

dbt00 6y ago
> (that the 2/3 of the compiled binaries are useless to the users). That the bytes are not visible in the symbol table is inarguable, that they are useless is a highly contentious statement and very probably wrong.
- dvfjsdhgfv 6y ago
  It's possible, but it is the "who cares" mindset I object to. File sizes are important. Memory usage is important. CPU cycles are important. Many devs with powerful machines don't give a heck and in the end everybody has to pay, in different ways.
parhamn 6y ago
The alternative isn't avoiding fixing it. The alternative is a boring bug report on github with actual stakeholders discussing the best strategies and trade offs. This is only on HN's homepage because of the langauge flame wars. It's a garbage post.
- dvfjsdhgfv 6y ago
  I agree 100% and I sincerely hope the author did that already. Just complaining about it to vent off one's frustration will accomplish nothing.

deleted 6y ago

[deleted]