3 ms·
One issue is that the server uses Google Protocol Buffers. The following definition, for instance: message Foo { required string a = 1; required int32 b
by logicchains 12y ago
One issue is that the server uses Google Protocol Buffers. The following definition, for instance:
message Foo {
required string a = 1;
required int32 b = 2;
optional int64 c = 3;
}
Generates a struct like:
type Foo struct {
a *string
b *int32
c *int64
}
This is not particularly friendly to the garbage collector compared to a POD struct with no pointers.
- whyever 12y agoHow can you have a POD struct without pointers if one of your fields is a string?
- logicchains 12y agoCouldn't it be done with a fixed-size array of runes or bytes? It's definitely possible in C/C++.
- z92 12y agoNot without wasting a lot of space depending on the difference between average size of string vs maximum size of string.
- a8da6b0c91d 12y agoYou can use a string class like the one in Folly that decays from a fixed array to heap allocated space if the string gets too long.
- logicchains 12y agoWe generally know the maximum string sizes for each property, and they're pretty small, so this wouldn't have been a problem.
- masklinn 12y agoI believe it's also possible to use variable-size arrays in C or C++ (and end up with a variable-size struct). Go does allow fixed size arrays in structs, and they're inline in the struct, so `struct { foo [8]uint }` is 32 bytes, whereas `struct { foo []uint }` is 12 bytes and `struct { foo string }` is 8.
- stingraycharles 12y agoProtocol buffers allows you to reuse the memory of your objects. You can simply reuse the object where you parse your message into. You pool these objects, and you just saved yourself a huge amount of heap allocations. https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.message_lite#MessageLite.Clear.details https://developers.google.com/protocol-buffers/docs/referenc...
- logicchains 12y agoThis would work if we were only using them for messaging, but the protobuffer objects are also used as the datastructure in which the information they contain is stored. This was done in the name of simplicity, to avoid creating a separate internal datastructure for each kind of message. The flow of data is currently: various external sources -> merged into protocol buffer struct -> later sent to client To avoid keeping a heap of protocol buffer structs in the heap, this would need to change to: various external sources -> merged into some internal datastructure -> later converted into protocol buffer struct and sent to client.
- scottlamb 12y ago> This would work if we were only using them for messaging, but the protobuffer objects are also used as the datastructure in which the information they contain is stored. This was done in the name of simplicity, to avoid creating a separate internal datastructure for each kind of message. Did you consider keeping strings (serialized messages) in your cache rather than message objects? I do this in C++ code simply for memory efficiency. Here it would also allow you to avoid these extra pointers.
- dsymonds 12y agoFYI, if you don't need required/optional, non-zero defaults and extensions, try `syntax = "proto3"`. It generates much better Go code.