7 ms·

Ah, memory management. Here’s my basic understanding: - Go initially allocates 2KB stack per routine. When it exceeds it it copies all of it into 2x the space.

Ah, memory management. Here’s my basic understanding: - Go initially allocates 2KB stack per routine. When it exceeds it it copies all of it into 2x the space. - This was happening once or twice per request. They didn’t explain exactly why all that stack memory was being used (maybe someone can chime in), but contributing factors were a 30 function deep call stack and a minor code change that tipped it into the next stack growth tier. - Also this doesn’t get freed up until garbage collection runs. - They worked around it by implementing a kind of go routine pool that keeps assigning work to the same (stack-expanded) routines, staying ahead of the garbage collector. My takeaways: 1. Fantastic analysis and job well done. 2. Pooling does not seem to be how things “should” work in Go. It’s more of a hack around undesirable allocator / garbage collector behavior. 3. I’m really interested in reference counted languages like Swift on the server for these reasons. I know ARC means more predictable latency when it comes to garbage collector behavior (which is only indirectly the problem here). Now I’m really curious how Swift allocates the stack and whether it would avoid this “morestack” growth penalty that Go has.

e12e 8y ago
Does this mean go doesn't do (enough) inlining of functions?
- shereadsthenews 8y ago
  Everyone agrees this is a problem. Today Go only inlines a function under certain very narrow circumstances.
  - GordonS 8y ago
    In C# you can apply an `AggressiveInlining` to remove certain limiting restrictions, making it more likely that a method will be inlined - does Go have something similar?
  - fwip 8y ago
    This has actually improved in the latest release, version 1.12. I'm sure it's still behind other languages, though.
    - tmaly 8y ago
      I can only imagine where Go will be with compiler optimizations in 3-5 years. Look at gcc, it has tons, but it took years to get there. Go has the added advantage of being able to learn off this knowledge
      - masklinn 8y ago
        > I can only imagine where Go will be with compiler optimizations in 3-5 years. Nowhere, optimisations cost CPU time and the Go developers focus on compilation time. They could already have implemented plenty of well-known and well-understood optimisation passes in the 10 years since the initial release, that they didn't do so and restricted themselves to the sort of optimisation passes you'd find in a bytecode interpreter was by design. In theory gccgo is where you'd get an optimising go compiler, but apparently even with gcc 8.1's improvements it has issues which lead to the end result being slower than the mainline compiler. I'm sure they'd be glad to get more people involved to resolve this. > Look at gcc, it has tons, but it took years to get there. > Go has the added advantage of being able to learn off this knowledge Go could have used LLVM, they specifically refused to do so on grounds of compilation time being too slow, preferring to not optimise instead.
        pjmlp 8y ago
        Yet D, Delphi, Eiffel, .NET Native, Free Pascal, OCaml among others, manage to offer rich languages, deliver performance while having fast compilation times.
        masklinn 8y ago
        I'm pretty sure OCaml doesn't use an optimising compiler, I don't think Delphi or FreePascal do either, not sure about the others. Just as Go does, these compilers will perform some optimisations but generally fairly local and simple. They "deliver performance" in the same way Go does. According to its documentation, dmd doesn't even do function inlining until you specifically opt into that with `-inline`.
jadbox 8y ago
It would be nice if the initial stack per routine size was configurable. Ultimately, it would be interesting if the Go runtime could profile the typical stack usage for new routines and dynamically adjust the pre-allocated stack size configuration. It's a little crude and might waste some memory, but it would help.
- firethief 8y ago
  Isn't lacking such nicities part of what keeps Go simple enough that forking the runtime is a relatively lightweight approach? The other extreme is a toolchain clever enough it requires carnal knowledge of its workings to coax reliable performance out of it
pjmlp 8y ago
Swift memory management lost to Go, C# and OCaml in the benchmark of the user space implementation of network drivers, presented at last year's CCC.
- abalone 8y ago
  Yes I've seen that[1] but I think it's premature to extrapolate general conclusions about server engineering from a low-level network driver. Chris Lattner has noted that Swift will take longer to optimize for cases like low-level systems programming because unlike most languages it was initially designed for generality.[2] What's interesting about Swift on the server is that the super lightweight runtime and lack of a tracing garbage collector hold the potential of much more consistent latency with low memory overhead. And as for reference counting overhead, application-level code is much more likely to be able to take advantage of Swift's preferred functional-esque "value type semantics", which don't suffer from that. [1] https://news.ycombinator.com/item?id=18788069 https://news.ycombinator.com/item?id=18788069 [2] https://developers.slashdot.org/story/17/01/23/085232/slashdots-interview-with-swift-creator-chris-lattner https://developers.slashdot.org/story/17/01/23/085232/slashd...
  - firethief 8y ago
    OCaml is hardly a language specialized for low level network drivers
    - abalone 8y ago
      Actually OCaml has low-level memory manipulation features that help like cstruct.[1] Swift isn't currently optimized for the direct manipulation of external device memory. That's solvable, but moreover it's not really something you can extrapolate from to draw conclusions about server applications. [1] https://dsheets.github.io/codoc/cstruct.1.5.0/_build/lib/cstruct/ https://dsheets.github.io/codoc/cstruct.1.5.0/_build/lib/cst...
  - pjmlp 8y ago
    Value type semantics exist in plenty of GC enabled languages, like those from CCC benchmarks, hardly a Swift advantage.
Someone 8y ago
Swift doesn’t have threads in the same sense that go has them. The language doesn’t have special constricts for creating threads; you use library calls to do that. You typically either use Grand Central Dispatch, or directly make OS calls to create threads. In either case, you get way larger stack sizes, 512kB for secondary threads by default (https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/Multithreading/CreatingThreads/CreatingThreads.html https://developer.apple.com/library/archive/documentation/Co...) Also, AFAIK, those stacks never grow. If you try to grow a thread’s stack too much, it’s end of game (for that thread, I think, but possibly even for the application) Go’s smaller stacks allow for way more goroutines to exist in parallel, but as this article shows, if go has to grow a stack, things slow down.
- saagarjha 8y ago
  Trying to grow a stack too much will cause a segmentation fault, which is recoverable but usually not something you would want to do.
  - masklinn 8y ago
    You can't grow a C stack, it's pre-allocated and fixed size. Trying to use too much of a stack - beyond what's initially allocated — is why you're going to segfault (in the best case), as you'll be traipsing on unallocated addresses and hopefully unmapped pages.
    - saagarjha 8y ago
      I'm using "grow" in the sense of moving the stack pointer down. But yes, you're correct, it's trying to access that memory that will cause a segmentation fault. And to address your first point, I'm sure there's hijinks you can perform with mmap to add a couple more pages if you so desire.
      - masklinn 8y ago
        > I'm sure there's hijinks you can perform with mmap to add a couple more pages if you so desire. I don't think you can do that, but I think you'd just create a bigger stack when spawning your process / thread: the stack allocation is virtual (at least on unices, possibly not on Windows?) so on a 64b system it's unlikely to be very expensive, physical pages will get allocated on-demand as stack use increases. This could probably be tested / benched using pthread: pthread_attr_setstacksize lets you define the (virtual) stack size before spawning the process.
roca 8y ago
Threadsafe refcounting is just terrible for throughput. Not only do the atomic ops hurt you, but logically read-only operations like data structure traversals require modifying refcounts, and therefore become read-write operations, with associated cache line ping-ponging between cores. The Swift people are smart and maybe they have a plan to mitigate this but I don't know what it might be.
- roca 8y ago
  So using Go and Java on the server makes sense when the developer ergonomics of GC outweigh the performance costs (and the developer ergonomics disadvantages of having to analyze GC performance). Using Rust makes sense when performance matters more. Swift seems somewhere in between; maybe there's a sweet spot there but, but it wouldn't be surprising if not.
  - iainmerrick 8y ago
    Refcounting makes sense when latency and responsiveness are more important than throughput. That’s true of both native UIs and most kinds of servers, which between them cover a good fraction of code being written today, maybe the majority of code. I’m not yet convinced by Swift the language, but I think it’s positioned in just the right place.
  - mrath 8y ago
    For me the benefit of Rust like language is the added benefit provided by the type system. Even when performance is not the critical factor. There some level of unit tests that are not required in Rust like language which would be required in Java, even more so in Golang.
  - abalone 8y ago
    > Swift seems somewhere in between I would say Swift is easier to learn/use than Rust while aiming to deliver some of the same performance characteristics. At least that's the sweet spot it's shooting for on the server. On throughput, two things: - Reference counting only applies object references. It does not apply to value types, and Swift is very strong on the "value semantics" approach to coding. It's like a sensible incorporation of some functional programming concepts without getting dogmatic about it. I'm not sure exactly how this will work out on the server but it seems very promising from a performance and thread safety standpoint. - Even when we consider throughput burdened by ARC, latency is probably more important of a metric than throughput for a whole lot of services. Throughput is sort of the knee-jerk metric which I feel is a holdover from desktop benchmarking days. For services at scale, p99 latency is where you get the real headaches. And that's one area where ARC is especially good.
tmaly 8y ago
I have used goroutine pooling. It has it uses when you want to limit resource usage.

e12e 8y ago

Does this mean go doesn't do (enough) inlining of functions?

shereadsthenews 8y ago
Everyone agrees this is a problem. Today Go only inlines a function under certain very narrow circumstances.
- GordonS 8y ago
  In C# you can apply an `AggressiveInlining` to remove certain limiting restrictions, making it more likely that a method will be inlined - does Go have something similar?
- fwip 8y ago
  This has actually improved in the latest release, version 1.12. I'm sure it's still behind other languages, though.
  - tmaly 8y ago
    I can only imagine where Go will be with compiler optimizations in 3-5 years. Look at gcc, it has tons, but it took years to get there. Go has the added advantage of being able to learn off this knowledge
    - masklinn 8y ago
      > I can only imagine where Go will be with compiler optimizations in 3-5 years. Nowhere, optimisations cost CPU time and the Go developers focus on compilation time. They could already have implemented plenty of well-known and well-understood optimisation passes in the 10 years since the initial release, that they didn't do so and restricted themselves to the sort of optimisation passes you'd find in a bytecode interpreter was by design. In theory gccgo is where you'd get an optimising go compiler, but apparently even with gcc 8.1's improvements it has issues which lead to the end result being slower than the mainline compiler. I'm sure they'd be glad to get more people involved to resolve this. > Look at gcc, it has tons, but it took years to get there. > Go has the added advantage of being able to learn off this knowledge Go could have used LLVM, they specifically refused to do so on grounds of compilation time being too slow, preferring to not optimise instead.
      - pjmlp 8y ago
        Yet D, Delphi, Eiffel, .NET Native, Free Pascal, OCaml among others, manage to offer rich languages, deliver performance while having fast compilation times.
        masklinn 8y ago
        I'm pretty sure OCaml doesn't use an optimising compiler, I don't think Delphi or FreePascal do either, not sure about the others. Just as Go does, these compilers will perform some optimisations but generally fairly local and simple. They "deliver performance" in the same way Go does. According to its documentation, dmd doesn't even do function inlining until you specifically opt into that with `-inline`.

jadbox 8y ago

It would be nice if the initial stack per routine size was configurable. Ultimately, it would be interesting if the Go runtime could profile the typical stack usage for new routines and dynamically adjust the pre-allocated stack size configuration. It's a little crude and might waste some memory, but it would help.

firethief 8y ago
Isn't lacking such nicities part of what keeps Go simple enough that forking the runtime is a relatively lightweight approach? The other extreme is a toolchain clever enough it requires carnal knowledge of its workings to coax reliable performance out of it

pjmlp 8y ago

Swift memory management lost to Go, C# and OCaml in the benchmark of the user space implementation of network drivers, presented at last year's CCC.

abalone 8y ago
Yes I've seen that[1] but I think it's premature to extrapolate general conclusions about server engineering from a low-level network driver. Chris Lattner has noted that Swift will take longer to optimize for cases like low-level systems programming because unlike most languages it was initially designed for generality.[2] What's interesting about Swift on the server is that the super lightweight runtime and lack of a tracing garbage collector hold the potential of much more consistent latency with low memory overhead. And as for reference counting overhead, application-level code is much more likely to be able to take advantage of Swift's preferred functional-esque "value type semantics", which don't suffer from that. [1] https://news.ycombinator.com/item?id=18788069 https://news.ycombinator.com/item?id=18788069 [2] https://developers.slashdot.org/story/17/01/23/085232/slashdots-interview-with-swift-creator-chris-lattner https://developers.slashdot.org/story/17/01/23/085232/slashd...
- firethief 8y ago
  OCaml is hardly a language specialized for low level network drivers
  - abalone 8y ago
    Actually OCaml has low-level memory manipulation features that help like cstruct.[1] Swift isn't currently optimized for the direct manipulation of external device memory. That's solvable, but moreover it's not really something you can extrapolate from to draw conclusions about server applications. [1] https://dsheets.github.io/codoc/cstruct.1.5.0/_build/lib/cstruct/ https://dsheets.github.io/codoc/cstruct.1.5.0/_build/lib/cst...
- pjmlp 8y ago
  Value type semantics exist in plenty of GC enabled languages, like those from CCC benchmarks, hardly a Swift advantage.

Someone 8y ago

Swift doesn’t have threads in the same sense that go has them. The language doesn’t have special constricts for creating threads; you use library calls to do that. You typically either use Grand Central Dispatch, or directly make OS calls to create threads. In either case, you get way larger stack sizes, 512kB for secondary threads by default (https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/Multithreading/CreatingThreads/CreatingThreads.html https://developer.apple.com/library/archive/documentation/Co...) Also, AFAIK, those stacks never grow. If you try to grow a thread’s stack too much, it’s end of game (for that thread, I think, but possibly even for the application) Go’s smaller stacks allow for way more goroutines to exist in parallel, but as this article shows, if go has to grow a stack, things slow down.

saagarjha 8y ago
Trying to grow a stack too much will cause a segmentation fault, which is recoverable but usually not something you would want to do.
- masklinn 8y ago
  You can't grow a C stack, it's pre-allocated and fixed size. Trying to use too much of a stack - beyond what's initially allocated — is why you're going to segfault (in the best case), as you'll be traipsing on unallocated addresses and hopefully unmapped pages.
  - saagarjha 8y ago
    I'm using "grow" in the sense of moving the stack pointer down. But yes, you're correct, it's trying to access that memory that will cause a segmentation fault. And to address your first point, I'm sure there's hijinks you can perform with mmap to add a couple more pages if you so desire.
    - masklinn 8y ago
      > I'm sure there's hijinks you can perform with mmap to add a couple more pages if you so desire. I don't think you can do that, but I think you'd just create a bigger stack when spawning your process / thread: the stack allocation is virtual (at least on unices, possibly not on Windows?) so on a 64b system it's unlikely to be very expensive, physical pages will get allocated on-demand as stack use increases. This could probably be tested / benched using pthread: pthread_attr_setstacksize lets you define the (virtual) stack size before spawning the process.

roca 8y ago

Threadsafe refcounting is just terrible for throughput. Not only do the atomic ops hurt you, but logically read-only operations like data structure traversals require modifying refcounts, and therefore become read-write operations, with associated cache line ping-ponging between cores. The Swift people are smart and maybe they have a plan to mitigate this but I don't know what it might be.

roca 8y ago
So using Go and Java on the server makes sense when the developer ergonomics of GC outweigh the performance costs (and the developer ergonomics disadvantages of having to analyze GC performance). Using Rust makes sense when performance matters more. Swift seems somewhere in between; maybe there's a sweet spot there but, but it wouldn't be surprising if not.
- iainmerrick 8y ago
  Refcounting makes sense when latency and responsiveness are more important than throughput. That’s true of both native UIs and most kinds of servers, which between them cover a good fraction of code being written today, maybe the majority of code. I’m not yet convinced by Swift the language, but I think it’s positioned in just the right place.
- mrath 8y ago
  For me the benefit of Rust like language is the added benefit provided by the type system. Even when performance is not the critical factor. There some level of unit tests that are not required in Rust like language which would be required in Java, even more so in Golang.
- abalone 8y ago
  > Swift seems somewhere in between I would say Swift is easier to learn/use than Rust while aiming to deliver some of the same performance characteristics. At least that's the sweet spot it's shooting for on the server. On throughput, two things: - Reference counting only applies object references. It does not apply to value types, and Swift is very strong on the "value semantics" approach to coding. It's like a sensible incorporation of some functional programming concepts without getting dogmatic about it. I'm not sure exactly how this will work out on the server but it seems very promising from a performance and thread safety standpoint. - Even when we consider throughput burdened by ARC, latency is probably more important of a metric than throughput for a whole lot of services. Throughput is sort of the knee-jerk metric which I feel is a holdover from desktop benchmarking days. For services at scale, p99 latency is where you get the real headaches. And that's one area where ARC is especially good.

tmaly 8y ago

I have used goroutine pooling. It has it uses when you want to limit resource usage.