4 ms·

Go simply can't really do well unless it has globs of memory or properly set kernel parameters (that affect the whole global machine). To get it to run reliably

Go simply can't really do well unless it has globs of memory or properly set kernel parameters (that affect the whole global machine). To get it to run reliably in low-memory conditions also requires careful tuning of GOGC + GOMEMLIMIT + MADV_DONTNEED. Go is geared more towards servers with tons of available memory, not IoT devices. Further, it's networking stack does really well on low-latency connections (ie, a datacenter) and starts to do poorly if you throw a in link with lots of data loss (this includes clients across the internet on crappy wifi). I love that they basically reimplemented libc to get more performance, but they also threw away decades of edge cases that it doesn't (yet) handle very well. I'm confident they'll get there, eventually, especially when people do things like this and report issues.

denysvitali 3y ago
That's why you have TinyGo: https://tinygo.org/ https://tinygo.org/
- withinboredom 3y ago
  This looks like a really interesting project. Thanks for sharing it! I'm going to have to try this out with a couple of projects. Just a cursory experiment reveals that it's GC seems to be much better than Go's GC and I'm very curious as to why that is.
  - dolmen 3y ago
    https://tinygo.org/docs/reference/lang-support/#garbage-collection https://tinygo.org/docs/reference/lang-support/#garbage-coll...
    - withinboredom 3y ago
      Interesting that it says it is slower. It may actually be slower, but tends to not suffer as much degradation as the Go's GC. In other words, on my test setup using Go, I get ~1k rps once I hit memory limits, but with tinygo I get 7k rps once I hit memory limits. So it might be "slower" but it looks like it may be more efficient as well.
- rob74 3y ago
  Also good for WASM...
- sapiogram 3y ago
  TinyGo is not Go, it's more accurate to see it as a dialect rather than a separate Go implementation imo. Many packages (even the stdlib) don't compile under it, and as I understand it, certain language features involving runtime reflection are likely to never be supported.
  - pjmlp 3y ago
    Just like what many people call C is not ISO C, and similarly many ISO C libraries don't compile with "C" compilers for the same environments as TinyGo.
    - hamburglar 3y ago
      The differences between TinyGo and Go are vast compared to “C vs ANSI C” though.
      - pjmlp 3y ago
        Depends on which embedded toolchain we are talking about.
oefrha 3y ago
The targets of this project are RPis (with 512MB+ RAM) or similar. That’s tons of RAM compared to “IoT devices” you have in mind, and not an issue at all for Go.
- withinboredom 3y ago
  2Gi of ram is nearly the bare minimum to run a single Go server without suffering (huge) performance penalties -- we're talking a 50-99% degradation of performance while GC sucks up all the CPU power to gain more memory.
  - bradfitz 3y ago
    That's not true in general. Sounds like you have a really inefficient server that allocates too much.
    - withinboredom 3y ago
      It's not a matter of allocation since there isn't a way to manually free memory in Go. The GC basically doesn't do anything if there isn't memory pressure. Once your applications starts feeling some memory pressure, the GC will start spending CPU to free up memory. This is a non-disputable fact of how Go works, in general.
      - logicchains 3y ago
        >It's not a matter of allocation It's absolutely a matter of allocation, since the more you allocate, the more work the GC will have to do. If you write clean, memory-friendly code (e.g. reusing buffers, favouring stack allocation), then you'll generate a lot less garbage and the GC will spend less time collecting it.
      - 4ad 3y ago
        > It's not a matter of allocation > Once your applications starts feeling some memory pressure I rest my case.
      - 3y ago
tedunangst 3y ago
What networking stack?
1vuio0pswjnm7 3y ago
This project always seemed to be swimming against the current by using Go but for the longest time no one would state the obvious. Finally. Now I know I'm not krazy.
4ad 3y ago
The whole point of an appliance is that it runs a single application. Go does not have any sort of networking stack. This comment is 100% nonsense.
Thaxll 3y ago
The Go processes I'm currently working on do thousand of requests per seconds doing tls and DB calls / serialization with less than 128MB of memory. Go does not need me much memory, it's in the same ballpark as a C++ server, using way way less memory than Java / C#.
psanford 3y ago
I'm running gokrazy on a raspberry pi zero 2 w with 512MB of ram. I'm using it as a record-on-motion security camera[0]. The actual recording happens with ffmpeg, but the Go server does motion detection, hls streaming, and cloud upload of video chunks. It also does this all in-memory, the sdcard in the pi is mounted read-only. It works great! No memory pressure/GC issues at all. Its really not that hard to write go that can perform well in moderately constrained environments. [0]: https://github.com/psanford/rom-cam https://github.com/psanford/rom-cam
dsff3f3f3f 3y ago
I've seen fairly naive Go servers that do 10,000+ RPS that issue database requests on every request that consistently use less than 50MB of RAM without issue. Several of them barely ever break 30MB of RAM.

denysvitali 3y ago

That's why you have TinyGo: https://tinygo.org/ https://tinygo.org/

withinboredom 3y ago
This looks like a really interesting project. Thanks for sharing it! I'm going to have to try this out with a couple of projects. Just a cursory experiment reveals that it's GC seems to be much better than Go's GC and I'm very curious as to why that is.
- dolmen 3y ago
  https://tinygo.org/docs/reference/lang-support/#garbage-collection https://tinygo.org/docs/reference/lang-support/#garbage-coll...
  - withinboredom 3y ago
    Interesting that it says it is slower. It may actually be slower, but tends to not suffer as much degradation as the Go's GC. In other words, on my test setup using Go, I get ~1k rps once I hit memory limits, but with tinygo I get 7k rps once I hit memory limits. So it might be "slower" but it looks like it may be more efficient as well.
rob74 3y ago
Also good for WASM...
sapiogram 3y ago
TinyGo is not Go, it's more accurate to see it as a dialect rather than a separate Go implementation imo. Many packages (even the stdlib) don't compile under it, and as I understand it, certain language features involving runtime reflection are likely to never be supported.
- pjmlp 3y ago
  Just like what many people call C is not ISO C, and similarly many ISO C libraries don't compile with "C" compilers for the same environments as TinyGo.
  - hamburglar 3y ago
    The differences between TinyGo and Go are vast compared to “C vs ANSI C” though.
    - pjmlp 3y ago
      Depends on which embedded toolchain we are talking about.

oefrha 3y ago

The targets of this project are RPis (with 512MB+ RAM) or similar. That’s tons of RAM compared to “IoT devices” you have in mind, and not an issue at all for Go.

withinboredom 3y ago
2Gi of ram is nearly the bare minimum to run a single Go server without suffering (huge) performance penalties -- we're talking a 50-99% degradation of performance while GC sucks up all the CPU power to gain more memory.
- bradfitz 3y ago
  That's not true in general. Sounds like you have a really inefficient server that allocates too much.
  - withinboredom 3y ago
    It's not a matter of allocation since there isn't a way to manually free memory in Go. The GC basically doesn't do anything if there isn't memory pressure. Once your applications starts feeling some memory pressure, the GC will start spending CPU to free up memory. This is a non-disputable fact of how Go works, in general.
    - logicchains 3y ago
      >It's not a matter of allocation It's absolutely a matter of allocation, since the more you allocate, the more work the GC will have to do. If you write clean, memory-friendly code (e.g. reusing buffers, favouring stack allocation), then you'll generate a lot less garbage and the GC will spend less time collecting it.
    - 4ad 3y ago
      > It's not a matter of allocation > Once your applications starts feeling some memory pressure I rest my case.
    - 3y ago

tedunangst 3y ago

What networking stack?

1vuio0pswjnm7 3y ago

This project always seemed to be swimming against the current by using Go but for the longest time no one would state the obvious. Finally. Now I know I'm not krazy.

4ad 3y ago

The whole point of an appliance is that it runs a single application. Go does not have any sort of networking stack. This comment is 100% nonsense.

Thaxll 3y ago

The Go processes I'm currently working on do thousand of requests per seconds doing tls and DB calls / serialization with less than 128MB of memory. Go does not need me much memory, it's in the same ballpark as a C++ server, using way way less memory than Java / C#.

psanford 3y ago

I'm running gokrazy on a raspberry pi zero 2 w with 512MB of ram. I'm using it as a record-on-motion security camera[0]. The actual recording happens with ffmpeg, but the Go server does motion detection, hls streaming, and cloud upload of video chunks. It also does this all in-memory, the sdcard in the pi is mounted read-only. It works great! No memory pressure/GC issues at all. Its really not that hard to write go that can perform well in moderately constrained environments. [0]: https://github.com/psanford/rom-cam https://github.com/psanford/rom-cam

dsff3f3f3f 3y ago

I've seen fairly naive Go servers that do 10,000+ RPS that issue database requests on every request that consistently use less than 50MB of RAM without issue. Several of them barely ever break 30MB of RAM.