16 ms·
The title is a bit misleading, the performance was memory/GC related and the specific in-memory cache that didn't work well were .Net data structures that didn'
by mamp 10y ago
The title is a bit misleading, the performance was memory/GC related and the specific in-memory cache that didn't work well were .Net data structures that didn't solve memory problems. He didn't try Redis or Memcache but ended up writing files to disk.
- barrkel 10y agoSo he's essentially using the OS's disk cache as his cache, and tracking the disk files substituting for manual memory allocation. In this situation on .NET, I wrote a resource manager that handed out handles to large byte arrays (in particular, ones that were too big to be collected outside of gen2, in the large object heap, which kicked in at 80kb at the time IIRC). The handles implemented IDisposable so taking care of handing back the byte array was no more or less tedious than any other resource you need to manage explicitly. The resource manager kept a hold of the arrays internally using a weak pointer so they could still be collected when gen2 collections actually happened, but allocating the buffers themselves would never cause gen2 collections in a steady state. To turn that into a cache, you'd need another layer with keys, an eviction policy and an invalidation mechanism. I think it ought still be better than round-tripping to disk. I wrote a different version of the resource manager that used P/Invoke helpers and unsafe code to allocate from unmanaged memory directly, but it didn't perform any better - it didn't relieve any pressure on the GC, which was 2% of CPU usage at full load in any case.
- dom0 10y ago> So he's essentially using the OS's disk cache as his cache, and tracking the disk files substituting for manual memory allocation. And that's often not the worst idea, because, when done correctly, this stuff never hits the disk when enough RAM is around. Plus, life cycle management is done by the OS, not by you. It also tends to work better than "not at all" on memory pressure or if there isn't a lot of memory in the first place.
- Filligree 10y agoIf you write a file to disk, then wait fifteen minutes before deleting it, chances are the OS will have flushed it to disk sometime in the meantime. It won't have to re-read it if there's still free memory, but it's extra load on disk io. If you write a lot of them, then even if you have the memory, you may overrun the size limit of the write buffer and cause application stalls. Writing to a ramdisk (e.g. tmpfs) is always an option, though.
- dom0 10y agoEvery OS has flags for this. Windows' CreateFile has a _TEMPORARY flag and most unixes have something like O_TMPFILE.
- barrkel 10y agoIt'll be flushed to disk within seconds. I agree in a memory-rich environment it's far from the worst idea. However now you need to manage the files. You've just pushed the problem somewhere else.
- dom0 10y agoEvery OS has flags for this. Windows' CreateFile has a _TEMPORARY flag and most unixes have something like O_TMPFILE. These avoids flushing them to disk, even on a background writer timeout (Linux/Windows). Further, on many *nix systems /tmp and the like are often in-memory FSes.