5 ms·

The targets of this project are RPis (with 512MB+ RAM) or similar. That’s tons of RAM compared to “IoT devices” you have in mind, and not an issue at all for Go

by oefrha 3y ago

The targets of this project are RPis (with 512MB+ RAM) or similar. That’s tons of RAM compared to “IoT devices” you have in mind, and not an issue at all for Go.

withinboredom 3y ago
2Gi of ram is nearly the bare minimum to run a single Go server without suffering (huge) performance penalties -- we're talking a 50-99% degradation of performance while GC sucks up all the CPU power to gain more memory.
- bradfitz 3y ago
  That's not true in general. Sounds like you have a really inefficient server that allocates too much.
  - withinboredom 3y ago
    It's not a matter of allocation since there isn't a way to manually free memory in Go. The GC basically doesn't do anything if there isn't memory pressure. Once your applications starts feeling some memory pressure, the GC will start spending CPU to free up memory. This is a non-disputable fact of how Go works, in general.
    - logicchains 3y ago
      >It's not a matter of allocation It's absolutely a matter of allocation, since the more you allocate, the more work the GC will have to do. If you write clean, memory-friendly code (e.g. reusing buffers, favouring stack allocation), then you'll generate a lot less garbage and the GC will spend less time collecting it.
    - 4ad 3y ago
      > It's not a matter of allocation > Once your applications starts feeling some memory pressure I rest my case.
    - Yasuraka 3y ago
      >The GC basically doesn't do anything if there isn't memory pressure It will run every 2 minutes, if it hasn't run otherwise https://github.com/golang/go/blob/9b4b3e5acca2dabe107fa2c3ed963097d78a4562/src/runtime/proc.go#L5930 https://github.com/golang/go/blob/9b4b3e5acca2dabe107fa2c3ed...
    - d-z-m 3y ago
      I think bradfitz knows how Go works.
- oefrha 3y ago
  Without talking about your use case, this is such an absurd claim that doesn’t deserve a response. The rest of us can easily run a dozen or more typical light load HTTP services within 512MB.
  - withinboredom 3y ago
    I think the keywords there are "light load", you probably won't notice any issues as long as your requests are spaced out.
    - oefrha 3y ago
      Light load as in serves a couple rps persistently without problem (complete with SQLite database operations), and I’m not talking about highly optimized code. Enough for most home or small business intranet use cases. You’re not going to serve google.com in 50MB, obviously. If your code has to allocate ~100 megs for a request without an obvious reason that requires that much allocation, that’s on your code, not the language.
    - BillinghamJ 3y ago
      I don't know how you'd define "light load", but I have a few services that average about 100rps. Only 1 active container most of the time, they're allocated 128MiB RAM, with a p99 utilization of 55%. They don't have any GC config or special optimization - just operating with regular JSON POST requests, making MongoDB queries Go just does incredibly well in these kinds of conditions. The only reason we allocate 128MiB is because that's the minimum Google Cloud Run allows - we'd probably go for 64MiB on most systems otherwise
    - jerf 3y ago
      No, request rate doesn't really matter. Increased request rate will increase the rate of the GC running, but it doesn't intrinsically increase the amount of RAM used. (More simultaneous requests can, of course, but that isn't necessarily implied by an increased request rate.) Look, let me blunt, since you have insistently posted several times: You don't know what you're talking about. The code is open source, the algorithms publicly available, and it simply is not the case that Go is programmed to simply run up to gigabytes of memory usage before going "hurr durr I guess I should free some stuff". Like many other posters here, I also in possession of numerous 24/7 Go servers that run all sorts of things through them without needing gigabytes of RAM. I can tell you what happened to you with high probability. You wrote a memory leak, and mistook the result for Go's error rather than yours. Hit your leaking code with the built-in memory profiler and figure out what you're leaking and fix it. It's OK, it happens to all of us, I've written many such leaks in many different languages, with many different allocation schemes. RPis are abundantly capable of running Go. They can run a lot of Go. They're way, way above minimum spec. Of course Go won't let you magically hold a decompressed video stream in memory any more than anything else does or work other magic, but it's a heck of a lot more efficient than Python, and can run an awful lot of server in the resources a web browser takes just to display a blank window.
- _lvbh 3y ago
  Skill issue. A 512 MB server is able to run gitea and dendrite along with a bunch of other of my own services just fine with acceptable performance.
- devjam 3y ago
  I’m bemused by these sorts of disparaging comments that come from people who seem to have no real world experience running Go applications at scale. It’s entirely possible to run a server that will handle tens of thousands of req/sec on hardware with comparatively small amounts (by todays standards) of RAM e.g. 512MB
  - withinboredom 3y ago
    I don't see how you think my comments are "disparaging," this is just code doing what it is programmed to do. > It’s entirely possible to run a server that will handle tens of thousands of req/sec Indeed, you can do this. I'm talking about what happens once the Go starts getting memory pressure and the GC starts dominating the CPU. This is what it is designed to do. If your code can prevent that from happening, you are correct then, as it won't be an issue.
    - veqq 3y ago
      > Go starts getting memory pressure and the GC starts dominating the CPU That does not happen. You must be thinking of Java 15 years ago.
      - pjmlp 3y ago
        Even Java 17 years ago is ironic, considering the hardware requirements of J2ME, Java Embedded, Blue Ray players, Ricoh and Xerox copiers, Cisco phones, and so forth.
      - jrockway 3y ago
        Prior to GOMEMLIMIT being a thing, people would get themselves into this situation occasionally. Their steady state working set size is something like 2.1GB, but they want to fit into a 4GB container limit, so GOGC=100 led to OOMs occasionally. They would bring GOGC down to 10 or something (why not 90!?), and then they are running GC every time they allocated 10% or 200MB, and then their CPU profiles would show they were burning a lot on GC, because it's pretty easy for an RPC server to generate 200MB of garbage. GOMEMLIMIT definitely improves things, taking the very blunt instrument GOGC out of people's hands, and does what they actually want instead, which is to prioritize GC only when you're about to run out of memory. People dislike tunables like this, and rightfully so, but a similar problem exists somewhat insidiously in non-garbage collected languages. Calling malloc() and free() is not a zero-cost operation, and you can certainly waste CPU by calling free() when nothing actually needs the memory you are freeing. GC lets cases like "this is a CLI app, the system has 128GB of RAM, the working set size is 2GB, and it's over in 1 second" basically never do any memory freeing. You allocate as much as you need, GC never runs, and the app exits. That kind of optimization is actually pretty neat. But, for steady state servers that run for an unbounded period of time, it is inevitable that you are going to spend time trying to reuse memory that you no longer need, because the system only has a finite amount of memory. Then the compromises start to balance throughput with efficiency. gc + GOMEMLIMIT is a very nice compromise that is excellent for many applications, but no heuristic is perfect for every possible computer program.
- tgv 3y ago
  That's bollocks. I've got a 4GB server which runs 5 production services, and (maximally) 20 test services, and only one of those ever goes over 40MB, and then drops below that again. The production services handle about 65k requests per day. Not much, but it shows that you can easily run go applications in less than 2GB.
- sapiogram 3y ago
  It's not an exact comparison, but at work we have tons of Go apps running in Docker containers with a 50MB memory limit. For lighter workloads, it works just fine.

by oefrha 3y ago

The targets of this project are RPis (with 512MB+ RAM) or similar. That’s tons of RAM compared to “IoT devices” you have in mind, and not an issue at all for Go.

withinboredom 3y ago

2Gi of ram is nearly the bare minimum to run a single Go server without suffering (huge) performance penalties -- we're talking a 50-99% degradation of performance while GC sucks up all the CPU power to gain more memory.

bradfitz 3y ago
That's not true in general. Sounds like you have a really inefficient server that allocates too much.
- withinboredom 3y ago
  It's not a matter of allocation since there isn't a way to manually free memory in Go. The GC basically doesn't do anything if there isn't memory pressure. Once your applications starts feeling some memory pressure, the GC will start spending CPU to free up memory. This is a non-disputable fact of how Go works, in general.
  - logicchains 3y ago
    >It's not a matter of allocation It's absolutely a matter of allocation, since the more you allocate, the more work the GC will have to do. If you write clean, memory-friendly code (e.g. reusing buffers, favouring stack allocation), then you'll generate a lot less garbage and the GC will spend less time collecting it.
  - 4ad 3y ago
    > It's not a matter of allocation > Once your applications starts feeling some memory pressure I rest my case.
  - Yasuraka 3y ago
    >The GC basically doesn't do anything if there isn't memory pressure It will run every 2 minutes, if it hasn't run otherwise https://github.com/golang/go/blob/9b4b3e5acca2dabe107fa2c3ed963097d78a4562/src/runtime/proc.go#L5930 https://github.com/golang/go/blob/9b4b3e5acca2dabe107fa2c3ed...
  - d-z-m 3y ago
    I think bradfitz knows how Go works.
oefrha 3y ago
Without talking about your use case, this is such an absurd claim that doesn’t deserve a response. The rest of us can easily run a dozen or more typical light load HTTP services within 512MB.
- withinboredom 3y ago
  I think the keywords there are "light load", you probably won't notice any issues as long as your requests are spaced out.
  - oefrha 3y ago
    Light load as in serves a couple rps persistently without problem (complete with SQLite database operations), and I’m not talking about highly optimized code. Enough for most home or small business intranet use cases. You’re not going to serve google.com in 50MB, obviously. If your code has to allocate ~100 megs for a request without an obvious reason that requires that much allocation, that’s on your code, not the language.
  - BillinghamJ 3y ago
    I don't know how you'd define "light load", but I have a few services that average about 100rps. Only 1 active container most of the time, they're allocated 128MiB RAM, with a p99 utilization of 55%. They don't have any GC config or special optimization - just operating with regular JSON POST requests, making MongoDB queries Go just does incredibly well in these kinds of conditions. The only reason we allocate 128MiB is because that's the minimum Google Cloud Run allows - we'd probably go for 64MiB on most systems otherwise
  - jerf 3y ago
    No, request rate doesn't really matter. Increased request rate will increase the rate of the GC running, but it doesn't intrinsically increase the amount of RAM used. (More simultaneous requests can, of course, but that isn't necessarily implied by an increased request rate.) Look, let me blunt, since you have insistently posted several times: You don't know what you're talking about. The code is open source, the algorithms publicly available, and it simply is not the case that Go is programmed to simply run up to gigabytes of memory usage before going "hurr durr I guess I should free some stuff". Like many other posters here, I also in possession of numerous 24/7 Go servers that run all sorts of things through them without needing gigabytes of RAM. I can tell you what happened to you with high probability. You wrote a memory leak, and mistook the result for Go's error rather than yours. Hit your leaking code with the built-in memory profiler and figure out what you're leaking and fix it. It's OK, it happens to all of us, I've written many such leaks in many different languages, with many different allocation schemes. RPis are abundantly capable of running Go. They can run a lot of Go. They're way, way above minimum spec. Of course Go won't let you magically hold a decompressed video stream in memory any more than anything else does or work other magic, but it's a heck of a lot more efficient than Python, and can run an awful lot of server in the resources a web browser takes just to display a blank window.
_lvbh 3y ago
Skill issue. A 512 MB server is able to run gitea and dendrite along with a bunch of other of my own services just fine with acceptable performance.
devjam 3y ago
I’m bemused by these sorts of disparaging comments that come from people who seem to have no real world experience running Go applications at scale. It’s entirely possible to run a server that will handle tens of thousands of req/sec on hardware with comparatively small amounts (by todays standards) of RAM e.g. 512MB
- withinboredom 3y ago
  I don't see how you think my comments are "disparaging," this is just code doing what it is programmed to do. > It’s entirely possible to run a server that will handle tens of thousands of req/sec Indeed, you can do this. I'm talking about what happens once the Go starts getting memory pressure and the GC starts dominating the CPU. This is what it is designed to do. If your code can prevent that from happening, you are correct then, as it won't be an issue.
  - veqq 3y ago
    > Go starts getting memory pressure and the GC starts dominating the CPU That does not happen. You must be thinking of Java 15 years ago.
    - pjmlp 3y ago
      Even Java 17 years ago is ironic, considering the hardware requirements of J2ME, Java Embedded, Blue Ray players, Ricoh and Xerox copiers, Cisco phones, and so forth.
    - jrockway 3y ago
      Prior to GOMEMLIMIT being a thing, people would get themselves into this situation occasionally. Their steady state working set size is something like 2.1GB, but they want to fit into a 4GB container limit, so GOGC=100 led to OOMs occasionally. They would bring GOGC down to 10 or something (why not 90!?), and then they are running GC every time they allocated 10% or 200MB, and then their CPU profiles would show they were burning a lot on GC, because it's pretty easy for an RPC server to generate 200MB of garbage. GOMEMLIMIT definitely improves things, taking the very blunt instrument GOGC out of people's hands, and does what they actually want instead, which is to prioritize GC only when you're about to run out of memory. People dislike tunables like this, and rightfully so, but a similar problem exists somewhat insidiously in non-garbage collected languages. Calling malloc() and free() is not a zero-cost operation, and you can certainly waste CPU by calling free() when nothing actually needs the memory you are freeing. GC lets cases like "this is a CLI app, the system has 128GB of RAM, the working set size is 2GB, and it's over in 1 second" basically never do any memory freeing. You allocate as much as you need, GC never runs, and the app exits. That kind of optimization is actually pretty neat. But, for steady state servers that run for an unbounded period of time, it is inevitable that you are going to spend time trying to reuse memory that you no longer need, because the system only has a finite amount of memory. Then the compromises start to balance throughput with efficiency. gc + GOMEMLIMIT is a very nice compromise that is excellent for many applications, but no heuristic is perfect for every possible computer program.
tgv 3y ago
That's bollocks. I've got a 4GB server which runs 5 production services, and (maximally) 20 test services, and only one of those ever goes over 40MB, and then drops below that again. The production services handle about 65k requests per day. Not much, but it shows that you can easily run go applications in less than 2GB.
sapiogram 3y ago
It's not an exact comparison, but at work we have tons of Go apps running in Docker containers with a 50MB memory limit. For lighter workloads, it works just fine.