6 ms·
Seared into my soul is the experience porting a linux pipe-based application to Windows, thinking it's all posix and given it's all in memory the performance wi
by zh3 1y ago
Seared into my soul is the experience porting a linux pipe-based application to Windows, thinking it's all posix and given it's all in memory the performance will be more or less the same. The performance was hideous, even after we found that having pipes waiting for a connection more or less ground windows to a halt.
Some years later this got revisited due to needing to use the same thing under C# on Win10 and while it was better it was still a major embarrassment how big the performance gap was.
- andrewmcwatters 1y agoDid you find that you needed interprocess communication to replace the gap?
- spacechild1 1y agopipes are a form of interprocess communication :) I guess you meant shared memory?
- andrewmcwatters 1y agoYes. Yeah, you're right. Sockets could also be used, but I guess when I think of IPC, I generally think of shared memory.
- asveikau 1y agoSome years back Windows added AF_UNIX sockets, I wonder how those would perform relative to Win32 pipes. My guess is better.
- manwe150 1y agoSeems to reportedly be slightly faster in a few cases, but nothing particularly dramatic https://www.yanxurui.cc/posts/server/2023-11-28-benchmark-tcp-uds-namedpipe/ https://www.yanxurui.cc/posts/server/2023-11-28-benchmark-tc...
- asveikau 1y agoAre we reading the same tables? It seems to be about 3x faster than named pipes, and marginally faster than local TCP. It's worth noting that in Win32, an unnamed pipe is just a named pipe with the name discarded. So this "3x faster" is, I think, the exact comparison we're interested in.
- SoftTalker 1y agoWell POSIX only defines behavior, not performance. Every platform and OS will have its own performance idiosyncracies.
- klysm 1y agoHow on earth would POSIX define performance of something like pipes?
- SoftTalker 1y agoI was addressing "it's all posix and given it's all in memory the performance will be more or less the same." Not claiming that POSIX should or could attempt to address performance.
- pjmlp 1y agoBy using Big O notation, or deadlines like on RTOS APIs, as two possible examples on how to express performance on a standard.
- variadix 1y agoSome standards do define performance requirements, e.g. operations on data structures, in BigO notation.
- dataflow 1y ago> The performance was hideous, even after we found that having pipes waiting for a connection more or less ground windows to a halt. When you say the performance was hideous, are you referring to I/O after the pipe is already connected/open, or before? The former would he surprising, but the latter not - opening and closing a ton of pipes is not something you'd expect an OS to be optimized for - and it would be somewhat surprising if your use case requires the latter.
- zh3 1y agoLiterally just having spare listening sockets, ready for incoming connections (and obv. not busy-waiting on them). Just reducing to the number actually in-use was the biggest speed-up - it was like Windows was busy-waiting internally for new connections (it wasn't a huge number either, something like 8 or 12).
- dataflow 1y agoBy "spare listening sockets" do you mean having threads on the server calling ConnectNamedPipe? A bit confused by your terminology since these aren't called listening sockets. (You're not referring to socket() or AF_UNIX, right?) And yeah, that seems more or less what I expected. The implementation is probably optimized for repeated I/O on established connections, not repeated unestablished ones. Which would be similar to filesystem I/O on Windows in that way - it's optimized for I/O on open files (especially larger ones), not for repeatedly opening and closing files (especially small ones). It makes me wonder what kinds of use cases require repeated connections on named pipes. If the performance is comparable to Linux's after the connection, then I think that's important to note - since that's what matters to a lot of applications.
- zh3 1y agoYes, it was indeed using ConnectNamedPipe - just had a look at the code (which I can't share) to refresh my memory. The main problem was traced to setup delays in WaitForSingleObject()/WaitForMultipleObjects(); we fixed it as above (once all sessions were connected there were no spares left, so no problems), actual throughput was noted as quite inferior to linux but more than enough for our application so we left it there.
- hk1337 1y agoI remember years ago, we had an opposite experience. Not necessarily with pipes. We were running on Linux with a php app that would communicate with a soap api on .net and found that a .net implementation had better response time.
- vardump 1y agoLast I checked, on Windows local TCP outperforms pipes by a large margin.