4 ms·
Yes, but this is just proof of concept. For any given case, you can optimize your approach to your needs. E.g. single goroutine ReadCloser: type ioContextR
by bheadmaster 2y ago
Yes, but this is just proof of concept. For any given case, you can optimize your approach to your needs. E.g. single goroutine ReadCloser:
type ioContextReadCloser struct {
io.ReadCloser
ctx context.Context
ch chan *readReq
}
type readReq struct {
p []byte
n *int
err *error
m sync.Mutex
}
func NewIoContextReadCloser(ctx context.Context, rc io.ReadCloser) *ioContextReadCloser {
rcc := &ioContextReadCloser{
ReadCloser: rc,
ctx: ctx,
ch: make(chan *readReq),
}
go rcc.readLoop()
return rcc
}
func (rcc *ioContextReadCloser) readLoop() {
for {
select {
case <-rcc.ctx.Done():
return
case req := <-rcc.ch:
*req.n, *req.err = rcc.ReadCloser.Read(req.p)
if *req.err != nil {
req.m.Unlock()
return
}
req.m.Unlock()
}
}
}
func (rcc *ioContextReadCloser) Read(p []byte) (n int, err error) {
req := &readReq{p: p, n: &n, err: &err}
req.m.Lock() // use plain mutex as signalling for efficiency
select {
case <-rcc.ctx.Done():
return 0, rcc.ctx.Err()
case rcc.ch <- req:
}
req.m.Lock() // wait for readLoop to unlock
return n, err
}
Again, this is not to say this is the right way, only that it is possible and does not require any shenanigans that e.g. Python needs when dealing with when mixing sync & async, or even different async libraries.
- orf 2y agoA mutex in a hot Read (or any IO) path isn’t efficient.
- bheadmaster 2y agoWhat would you suggest as an alternative?
- orf 2y ago[flagged]
- bheadmaster 2y agoThank you for your helpful input.
- orf 2y agoNo worries, this help also generalises to any time you want to add locking/synchronisation or other overheads to a hot code path. It’s best not to acquire a mutex and launch a goroutine to read 3 bytes of data at a time. Also, hot tip: you can like… benchmark this. It’s not illegal.
- bheadmaster 2y ago> It’s best not to acquire a mutex and launch a goroutine to read 3 bytes of data at a time. io.Copy uses 32KB buffers. Other parts of standard library do too. If you're using Read() to read 3 bytes of data at a time, mutex is the least of your worries. Since you seem to be ignoring sarcasm of my previous comment - just saying "don't do that" without suggesting an alternative way in the particular code context you're referring to, isn't useful at all. It's just annoying.
- orf 2y agoIt may well use 32KB buffers, or any size, but that doesn’t translate to “reading 32KB at a time”. If you’re aborting specifically an io.Copy, then there are better ways to do that: abort in the write path rather than the read path. It’s not my job to provide you with alternative code. That’s your job.
- bheadmaster 2y ago> It’s not my job to provide you with alternative code. That’s your job. It is not your job to tell me "that is wrong", yet you do it because it's easy. Suggesting an alternative (not necessarily providing the code) is less easy, so you don't wanna do it. That's fine. I just want you to be aware that the former without the latter is pretty much useless.
- kiitos 2y agoI think you're missing the forest for the trees, here. The io.Reader/Writer interfaces, and their implementations, are meant to provide a streaming model for reading and writing bytes, which is as efficient as reasonably possible, within the constraints of the core language. If your goal is to make an io.Reader that respects a context.Context cancelation, then you can just do type ContextReader struct { ctx context.Context r io.Reader } func NewContextReader(ctx context.Context, r io.Reader) *ContextReader { return &ContextReader{ ctx: ctx, r: r, } } func (cr *ContextReader) Read(p []byte) (int, error) { if err := cr.ctx.Err(); err != nil { return 0, err } return cr.r.Read(p) } No goroutines or mutexes or whatever else required. Extending to a ReadCloser is a simple exercise left to the, er, reader.