5 ms·
You might look into using memory mapped IO for reading input and writing your output files. This can save some memory allocations and file read and write times
by jclarkcom 1y ago
You might look into using memory mapped IO for reading input and writing your output files. This can save some memory allocations and file read and write times. I did this with a project where I got more than 10x speed up. For many cases file IO is going to be your bottleneck.
- Sesse__ 1y agommap-based I/O still needs to go through the kernel, including memory allocation (in the page cache) and all. If you've got 10x speedup from mmap, it is usually because your explicit I/O was very inefficient; there are situations where mmap is useful, but it's rarely a high-performance strategy, as it's really hard for it to guess what your intended I/O patterns are just from the page faults it's seeing.
- jclarkcom 1y agoWindows uses memory mapped IO for loading all executable processes because it allows you to start executing a process after loading a few pages even if the exe is megabytes. You can use the same to reduce latency for starting to assemble data before the rest of the file loads, the rest can be loaded using more efficienct asynchronous mechanisms. Using for output also means your process doesnt waits on flushes that is also async. And in memory constrained environments the OS doesn’t have to write your data to swap, it can just reload it from the meeting mapped file.
- Sesse__ 1y agoLinux also uses mmap for running executables. But explicit I/O does not mean you have to start off by a gigabyte-long read().
- jclarkcom 1y agoMore detailed explanation from ChatGPT. As quick estimate you could achieve a >2x speed up using memory mapped files for a typical assembler workload. https://chatgpt.com/share/68b5e0db-a6d0-8005-9101-d326d2af0a04 https://chatgpt.com/share/68b5e0db-a6d0-8005-9101-d326d2af0a...
- Sesse__ 1y agoWhy would anyone be interested in arguing against a confused AI?
- jclarkcom 1y agoI was trying to provide a more detailed explanation without typing a lot. I studied this problem a lot as PE at vmware.
- Sesse__ 1y agohttps://distantprovince.by/posts/its-rude-to-show-ai-output-to-people/ https://distantprovince.by/posts/its-rude-to-show-ai-output-... In any case, if you really believe mmap is great for an assembler, then sure, go ahead. But it's not.
- jclarkcom 1y agoI implemented an assembler as part of VMware thinapp and this was big performance boost for me but maybe you have a different experience from your efforts?