3 ms·
Love the video! Being totally unfamiliar with shaders: can they be used to improve performance of things like file operations on multiple files? This week I was
by memco 5y ago
Love the video! Being totally unfamiliar with shaders: can they be used to improve performance of things like file operations on multiple files? This week I was working on hashing files in parallel in Python and although it's faster now than the single threaded version, the project would hugely benefit from being able to work faster since some of the work being done takes thirty minutes - several hours.
- raphlinus 5y agoPotentially, but it depends on the task. You need thousands of threads to get real performance boost, and hashing is an inherently sequential task, not really GPU-optimized (hash functions in the Merkle-Damgård family like SHA; you can do Merkle trees too). In fact, I'm not sure why anyone would want to hash on a GPU, unless, say, they wanted to hash lots of short strings as some kind of science project or something.
- memco 5y agoThe “short” could be a problem depending on how short we’re talking. I’m working on thousands of files that can sometimes be only a few kilobytes or less. I wouldn’t expect to use it to speed the process of a single file, but if it could be used to hash a thousand or multiple thousands of files at a time it could offer one or more orders of magnitude of improvement for some use caes.
- raphlinus 5y agoYes, for thousands of files of a few kilobytes each, GPU might be a good speedup. On a discrete card, a potential bottleneck is copying the files to device memory, but this might be a case where high end integrated graphics might perform well, as there's no need for staging buffers and the copy. But it's hard to know the real performance without trying it and measuring.