4 ms·
Absolutely. I think the app may have interesting challenges, like whatever some real-time stream processing could be introduced to avoid loading entire stuff in
by imhoguy 8y ago
Absolutely. I think the app may have interesting challenges, like whatever some real-time stream processing could be introduced to avoid loading entire stuff into memory or even to skip disk IO.
But I assume the OP did some effort/gain calculation which made him choose the existing path.
- yawboakye 8y agoOP here. You're right. The processed outputs and the recorded tracks are expected to be available <= 2s after the stream ends. That forces our hand to not be able to use memory conservatively. For example, one way to avoid massive memory usage will be to record concurrently (this has small memory footprint since we flush to disk after the write buffer hits 512KB so for about 50 concurrent recordings we need ~25MB), but process and upload sequentially. The recorded audio tracks are big, and have to be loaded to memory since the other tools we use for processing can't work with file on disk yet. The <= 2s requirement is impossible to meet when things are done sequentially.
- imhoguy 8y agoCheck memory-mapped files, may possibly work for you https://godoc.org/golang.org/x/exp/mmap https://godoc.org/golang.org/x/exp/mmap