5 ms·

> Tiny output binaries are also very quick to transfer and spin up in serverless as well - often reducing costs further. Your point is specifically about cloud

> Tiny output binaries are also very quick to transfer and spin up in serverless as well - often reducing costs further. Your point is specifically about cloud so maybe I shouldn't use go on Edge? In fact I'm struggling with the binary size right now (and have only just started with go after decades of C - because reasons beyond my control). with go 1.17 the resulting executable (running on ARM) is 11MB (stripped ldflags -w -s). I can run upx --brute and reduce it to 3MB. But that is still heavy when another lua based application (a complete NodeRed clone - with massive complexity) and 100x as many lines of code (before importing external packages) is still just short of half a MB. (apples & oranges yes ...) Maybe I shouldn't use go on an edge devices with such poor hardware? I heard about tiny-go but looking at their website could only find instructions on how to x-compile for specific boards but not for specific HW (like ARM-5 softfloat). Though I totally love the free stuff I get out of the box with go while also it feeling very familiar coming from C. The only thing I dislike is that it's not C :P

dagmx 5y ago
Is your Lua code standalone? i.e is the interpreter part of the application size?
- edflsafoiewq 5y ago
  Even if it isn't, LuaJIT is half a meg, and regular Lua is about half that.
  - tgtweak 5y ago
    If you're thinking about an edge where you have less than 3MB of storage space, and likely very limited in-memory space, you'd probably still be better to go with C. These kinds of environments are becoming very small % of use case over time. Usually such resource-constrained devices & systems have their own build system or crutch libraries built in to help reduce binary sizes to this scale. It is against golang's design principle to make a language that requires external frameworks/libraries to run. As it is, you can build your golang app, send it over an ssh tunnel and have it running on the remote system without much thought at all.
marcus_cemes 5y ago
If you're coming from C, I understand that 11MB seems like a lot to you. I've done quite a bit of serverless Node.js, a relatively slim docker image is always a few hundred MB and can easily stretch to a GB. However, I don't consider using C a viable alternative for that purpose. What's cool about Cloudflare workers, for example, is that they provide the V8 runtime, you just provide the code, which like with Lua will be a very small footprint. I'm curious, as a C developer, would you say that you prefer Go over Rust? I've used both and in my opinion Rust feels closer to C/C++ in terms of control and flexibility that the language gives you. To me (from a web service point of view, not systems programming), Go is just a faster (and better designed) pre-compiled version of JS/Node, it may look like C with it's simplicity (you end up having to write a lot of repetitive code), but still packs a garbage collector and (albeit very fast!) runtime under the hood.
- feffe 5y ago
  I've written a lot of C code and to me Go is like a high level C with some sharp corners removed and a GC. Maybe I don't see the limitations in Go as a problem because of this. Have done quite a bit of C++ as well. I prefer simpler languages that I can keep in my head and like C and Go for that reason. Do I prefer it to rust? I've not bothered to learn rust yet. I think I would like it better than C++ but it's a bit large for my taste. I like to see the language level as an abstraction level that I want to be able to keep entirely in my head. I fear that rust (just as C++) may have too many features for that.
- usrbinbash 5y ago
  > I'm curious, as a C developer, would you say that you prefer Go over Rust? Not the guy you asked, but coming from a similar background: Yes, I absolutely would. Reason: Rust is an amazing language: Amazingly safe, amazingly fast, amazingly complicated. And the last isn't an advantage but a huge showstopper. What do I get from Rust over Go? A bit more performance. Okay, but I also get that from C, so the real question is, what do I get from Rust over C? Memory safety guaranteed by the compiler. Cool. But in exchange, I have to learn a language easily as complex as C++, but completely different from C. No thank you. Bottom Line: For me, the advantages of Rust over C do not justify the added complexity, not even close.
blub 5y ago
Why is the size a concern, start-up or limited disk space? If the latter, this seems quite resource constrained and Go might not be a good fit. If you know C, you could try a minimal C++: C + vector,string,unique_ptr. And depending on your tastes, classes and shared_ptr.
- pjmlp 5y ago
  In addition to that, look at the compiler flags how to enable bounds checking even in release builds.
ghusbands 5y ago
It's almost never worth using binary packers - every saving in on-disk size is lost on extra memory usage and extra cpu usage on starting up; it also interferes with memory management features (a binary typically takes up no writeable swappable memory - a packed one does).
- tgtweak 5y ago
  You can make notable compression gains by using a simple 7z self-extracting PE file on your already-static golang binary. In some cases this can yield another 10-50% savings in binary size. For environments where fresh startup time are important, then I would skip that as it does add some unnecessary overhead before your app comes to life. If you need something smaller than that, you're looking at writing c and building for your target arch.
Cthulhu_ 5y ago
> But that is still heavy when another lua based application [...] and 100x as many lines of code (before importing external packages) is still just short of half a MB. (apples & oranges yes ...) Apples and oranges indeed; a Go binary is a fully self-contained, external-runtime-less application. That said, I can picture (in my head, don't expect me to build it) an edge / lambda environment that only takes your codebase and will run / feel like a script moreso than a complete binary. I'm willing to put money that if you add the libraries + runtime to the lua application, you'd end up with more than 11MB. Anyway, I'm trying to keep neutral; if your binary size is that important then staying with C may be the way to go. It's a compromise between binary size, language features, 'safety' (e.g. memory management is easier in Go thanks to auto zeroed variables and garbage collection), concurrency (not as much an issue in lambdas), etc.
- tgtweak 5y ago
  compiled LUA interpreter and runtime is pretty slim - but we're glossing over the reality that LUA is a drastically less functional language than golang - much in the way LUA is less functional than C. I think to each their own - it is possible to have a lua app compile down, including the interpreter, to sub-megabyte size, but a 10k LOC lua app could probably be a 200 LOC golang app with some libraries pulled in. I shudder thinking about the development cost of building and maintaining a lua codebase but if you had 100 million client devices running your code (say a wifi lightbulb) then it might make sense to save on client device and put that money into development of something very bespoke to optimize it.
Abishek_Muthian 5y ago
To me Go feels like a natural progression of C, And could see C often when I write it. That said my experience with Go on ARM with low memory and CPU power hasn't been as same as my cloud services. I had to resort to ugly GC hacks and even preemptively killing the application on memory threshold. But it's just my experience, I wouldn't be surprised if there are highly optimized Go apps running successfully on low-mem devices on edge. I don't think Go can replace C for embedded programming, Perhaps newer systems programming language like Rust, Zig is the way to Go. I used to have Netdata on my edge devices, It would trip on its own Go plugin usage(mem) while the plugins written on NodeJS, Python ran fine; Likely because of unabated use of Goroutines. Go's strength is its Network stack, It's been years since I've touched Apache/ngnix as deploying Go binary and setting up systemd rule is all that's required. For non-network application on edge, Especially on low memory devices I think Go is not ready yet.
stjohnswarts 5y ago
Comparing size of executables is basically useless. You have to compare memory footprint and execution speed to be honest in a comparison between applications, and the applications shoudl serve roughly the same complexity of the problem being solved.

dagmx 5y ago

Is your Lua code standalone? i.e is the interpreter part of the application size?

edflsafoiewq 5y ago
Even if it isn't, LuaJIT is half a meg, and regular Lua is about half that.
- tgtweak 5y ago
  If you're thinking about an edge where you have less than 3MB of storage space, and likely very limited in-memory space, you'd probably still be better to go with C. These kinds of environments are becoming very small % of use case over time. Usually such resource-constrained devices & systems have their own build system or crutch libraries built in to help reduce binary sizes to this scale. It is against golang's design principle to make a language that requires external frameworks/libraries to run. As it is, you can build your golang app, send it over an ssh tunnel and have it running on the remote system without much thought at all.

marcus_cemes 5y ago

If you're coming from C, I understand that 11MB seems like a lot to you. I've done quite a bit of serverless Node.js, a relatively slim docker image is always a few hundred MB and can easily stretch to a GB. However, I don't consider using C a viable alternative for that purpose. What's cool about Cloudflare workers, for example, is that they provide the V8 runtime, you just provide the code, which like with Lua will be a very small footprint. I'm curious, as a C developer, would you say that you prefer Go over Rust? I've used both and in my opinion Rust feels closer to C/C++ in terms of control and flexibility that the language gives you. To me (from a web service point of view, not systems programming), Go is just a faster (and better designed) pre-compiled version of JS/Node, it may look like C with it's simplicity (you end up having to write a lot of repetitive code), but still packs a garbage collector and (albeit very fast!) runtime under the hood.

feffe 5y ago
I've written a lot of C code and to me Go is like a high level C with some sharp corners removed and a GC. Maybe I don't see the limitations in Go as a problem because of this. Have done quite a bit of C++ as well. I prefer simpler languages that I can keep in my head and like C and Go for that reason. Do I prefer it to rust? I've not bothered to learn rust yet. I think I would like it better than C++ but it's a bit large for my taste. I like to see the language level as an abstraction level that I want to be able to keep entirely in my head. I fear that rust (just as C++) may have too many features for that.
usrbinbash 5y ago
> I'm curious, as a C developer, would you say that you prefer Go over Rust? Not the guy you asked, but coming from a similar background: Yes, I absolutely would. Reason: Rust is an amazing language: Amazingly safe, amazingly fast, amazingly complicated. And the last isn't an advantage but a huge showstopper. What do I get from Rust over Go? A bit more performance. Okay, but I also get that from C, so the real question is, what do I get from Rust over C? Memory safety guaranteed by the compiler. Cool. But in exchange, I have to learn a language easily as complex as C++, but completely different from C. No thank you. Bottom Line: For me, the advantages of Rust over C do not justify the added complexity, not even close.

blub 5y ago

Why is the size a concern, start-up or limited disk space? If the latter, this seems quite resource constrained and Go might not be a good fit. If you know C, you could try a minimal C++: C + vector,string,unique_ptr. And depending on your tastes, classes and shared_ptr.

pjmlp 5y ago
In addition to that, look at the compiler flags how to enable bounds checking even in release builds.

ghusbands 5y ago

It's almost never worth using binary packers - every saving in on-disk size is lost on extra memory usage and extra cpu usage on starting up; it also interferes with memory management features (a binary typically takes up no writeable swappable memory - a packed one does).

tgtweak 5y ago
You can make notable compression gains by using a simple 7z self-extracting PE file on your already-static golang binary. In some cases this can yield another 10-50% savings in binary size. For environments where fresh startup time are important, then I would skip that as it does add some unnecessary overhead before your app comes to life. If you need something smaller than that, you're looking at writing c and building for your target arch.

Cthulhu_ 5y ago

> But that is still heavy when another lua based application [...] and 100x as many lines of code (before importing external packages) is still just short of half a MB. (apples & oranges yes ...) Apples and oranges indeed; a Go binary is a fully self-contained, external-runtime-less application. That said, I can picture (in my head, don't expect me to build it) an edge / lambda environment that only takes your codebase and will run / feel like a script moreso than a complete binary. I'm willing to put money that if you add the libraries + runtime to the lua application, you'd end up with more than 11MB. Anyway, I'm trying to keep neutral; if your binary size is that important then staying with C may be the way to go. It's a compromise between binary size, language features, 'safety' (e.g. memory management is easier in Go thanks to auto zeroed variables and garbage collection), concurrency (not as much an issue in lambdas), etc.

tgtweak 5y ago
compiled LUA interpreter and runtime is pretty slim - but we're glossing over the reality that LUA is a drastically less functional language than golang - much in the way LUA is less functional than C. I think to each their own - it is possible to have a lua app compile down, including the interpreter, to sub-megabyte size, but a 10k LOC lua app could probably be a 200 LOC golang app with some libraries pulled in. I shudder thinking about the development cost of building and maintaining a lua codebase but if you had 100 million client devices running your code (say a wifi lightbulb) then it might make sense to save on client device and put that money into development of something very bespoke to optimize it.

Abishek_Muthian 5y ago

To me Go feels like a natural progression of C, And could see C often when I write it. That said my experience with Go on ARM with low memory and CPU power hasn't been as same as my cloud services. I had to resort to ugly GC hacks and even preemptively killing the application on memory threshold. But it's just my experience, I wouldn't be surprised if there are highly optimized Go apps running successfully on low-mem devices on edge. I don't think Go can replace C for embedded programming, Perhaps newer systems programming language like Rust, Zig is the way to Go. I used to have Netdata on my edge devices, It would trip on its own Go plugin usage(mem) while the plugins written on NodeJS, Python ran fine; Likely because of unabated use of Goroutines. Go's strength is its Network stack, It's been years since I've touched Apache/ngnix as deploying Go binary and setting up systemd rule is all that's required. For non-network application on edge, Especially on low memory devices I think Go is not ready yet.

stjohnswarts 5y ago

Comparing size of executables is basically useless. You have to compare memory footprint and execution speed to be honest in a comparison between applications, and the applications shoudl serve roughly the same complexity of the problem being solved.