3 ms·

Read() being a blocking call, wouldn't some hack like that work ? _, err := conn.Read([]byte{}) if err != io.EOF { return err } // we

by rakoo 10y ago

Read() being a blocking call, wouldn't some hack like that work ? _, err := conn.Read([]byte{}) if err != io.EOF { return err } // we know there is something to read buf := pool.Get().([]byte) n, err := conn.Read(buf) // process n, err and buf as needed // if there is more to read, you may need to loop over conn.Read // after some inactivity timeout buf = buf[:0] pool.Put(buf)

eridius 10y ago
Depends on how Read() is implemented. Unless the documentation explicitly says otherwise (and it doesn't look like it does), it would be perfectly reasonable for the method to say "this buffer has a size 0, so I can just return immediately without even touching the fd".
zzzcpan 10y ago
No, it checks for zero-length buffers and returns immediately. You would have to read at least a byte and copy it into a new buffer. Won't matter much though, as the memory usage of a single goroutine is quite significant and doubling it by having a buffer preallocated is not something to care about.
- masklinn 10y ago
  > Won't matter much though, as the memory usage of a single goroutine is quite significant 2K as of 1.4 (was 4K before 1.2, 8K in 1.2 and 1.3). But yeah it's a good point, the intrinsic memory overhead of goroutines means even if buffer pooling worked the memory use of the system would still mostly follow the "naive" estimate. You may want to message the author to remind them of that, they may not have thought of that concern.
Matthias247 10y ago
Have never tried it with Go and don't know that the contract is there, but the approach in general is valid. Afaik you can use it with Windows IOCP to avoid the occupied buffer problem: Start an async read with a zero byte buffer and when you get the completion notificaton issue a non-blocking read with the then allocated target buffer.

by rakoo 10y ago

eridius 10y ago

Depends on how Read() is implemented. Unless the documentation explicitly says otherwise (and it doesn't look like it does), it would be perfectly reasonable for the method to say "this buffer has a size 0, so I can just return immediately without even touching the fd".

zzzcpan 10y ago

No, it checks for zero-length buffers and returns immediately. You would have to read at least a byte and copy it into a new buffer. Won't matter much though, as the memory usage of a single goroutine is quite significant and doubling it by having a buffer preallocated is not something to care about.

masklinn 10y ago
> Won't matter much though, as the memory usage of a single goroutine is quite significant 2K as of 1.4 (was 4K before 1.2, 8K in 1.2 and 1.3). But yeah it's a good point, the intrinsic memory overhead of goroutines means even if buffer pooling worked the memory use of the system would still mostly follow the "naive" estimate. You may want to message the author to remind them of that, they may not have thought of that concern.

Matthias247 10y ago

Have never tried it with Go and don't know that the contract is there, but the approach in general is valid. Afaik you can use it with Windows IOCP to avoid the occupied buffer problem: Start an async read with a zero byte buffer and when you get the completion notificaton issue a non-blocking read with the then allocated target buffer.