4 ms·
In the game and simulation industry, texture resolution is a problem. If you run the math on how big textures need to be to match pixel resolution 1:1 in 3D spa
by shawn 8y ago
In the game and simulation industry, texture resolution is a problem. If you run the math on how big textures need to be to match pixel resolution 1:1 in 3D space, it quickly becomes infeasable for large scenes.
One solution is to stream the mip levels, and to create virtualized mipmaps. This basically lets you create a 128k by 128k texture, which works fine because only small blocks are loaded. It’s literally the same thing as virtual memory, but for texture memory.
One downside is that when you shut down the game / simulation, all of that data no longer exists. It was virtual. It either streamed from the sever or you packed it to disk somewhere. Either way you have to set it up again the next time the program starts.
I don’t know whether the present topic would let you bypass this limitation, but if there was some way to start a program with a huge amount of memory allocations preloaded, that would be very attractive. And if you can reboot without losing that, then suddenly you can start crafting persistent worlds that load instantly with ~infinite resolution.
This is the dream that voxel tech was supposed to make into reality, and it will only become more of a concern over time as mixed reality tech matures.
- sterlind 8y agothe feature you're looking for is called opening a memory mapped file. I've seen tremendous performance improvements, but it's a really underused feature for some reason. Memory map Performance is incredible.
- wtallis 8y agoDoes Windows have any analog to madvise()? It seems like that's probably necessary to making things work well when you're mapping huge data sets, unless you're sure they reside on really fast storage.
- PeCaN 8y agoWindows 8 and later has PrefetchVirtualMemory[1]. Before that your best bet is to get a little hacky and try reading the file asynchronously to get it in the cache (Windows caches files pretty aggressively, so this more or less works). [1] https://msdn.microsoft.com/en-us/library/windows/desktop/hh780543(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/hh7...
- senozhatsky 8y agoSeems that VirtualAlloc() [1] is a little bit closer to madvise(), but still is pretty far. madvise() is quite powerful. For instance, take a look at MADV_FREE. [1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa366887(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/aa3... -ss
- PeCaN 8y agoI figured the use-case here was MADV_WILLNEED, which is what PrefetchVirtualMemory is for. VirtualAlloc flags would be for things like reserving virtual pages without committing them. There's no single API on Windows that does the myriad of things that madvise does.
- ddorian43 8y agoYep, see lmdb read performance. Or many search engines doing mmap.
- bluejekyll 8y ago> it's a really underused feature for some reason My suspicion is that it’s mainly because memory mapping files, while safe for reading, have a whole bunch of edge cases where the file can accidentally be corrupted.
- mmt 8y ago> I don’t know whether the present topic would let you bypass this limitation I can't imagine that it would, since it's just swapping. That's why I asked the question. How to take advantage of the nonvolatile aspect of that memory as you describe is a much broader, more complicated topic, involving bootloaders and kernels.