4 ms·

He probably doesn't have 64 threads, but he may have much more than 64 files or sockets. After all, the response was to the statement that "WaitForMultipleObjec

by ambrop7 13y ago

He probably doesn't have 64 threads, but he may have much more than 64 files or sockets. After all, the response was to the statement that "WaitForMultipleObjects as a kind of super-select to wait on anything". It clearly isn't, with this arbitrary limitation. The problem with Windows is that there isn't any such super-select, unlike on Linux (see epoll, signalfd, timerfd...). You should take a look at my SO answer which lists six different ways to use sockets in Windows: http://stackoverflow.com/questions/11830839/when-using-iocp-should-i-set-wsaoverlappeds-hevent-to-null-or-to-a-valid-handl/11831450#11831450 http://stackoverflow.com/questions/11830839/when-using-iocp-...

acqq 13y ago
Whatever he has more, waitfor is not the optimal way on Windows for that so it's still a complaint that round holes on Win32 are poor fit for his cubes. First choosing tools then complaining that they don't fit to some goal is typical for beginners.
- huhtenberg 13y ago
  Do tell me, oh, enlightened one, what I should be using if I have 10K idle network connections and I can't really use IOCP, because I need keep things reasonably portable. That's not even considering that IOCP is a great example of premature optimization at cost of clarity and simplicity. It tries to solve a problem that I don't have and it doesn't offer a simpler solution that would just do. It is not a "hole" for my square peg, it's a freaking snowflake. And expecting a modern OS to have an uncomplicated O(1) API for monitoring sockets is clearly too much to ask.
  - ambrop7 13y ago
    It's possible to abstract IOCP and non-blocking sockets into a single interface with reasonable efficiency. For example: - To send data, user calls Socket::send(const char *data, size_t length). This starts sending the given data; the buffer must remain available while data is being sent. IOCP implementation will initiate I/O using WSASend() or similar. - When send operation is complete, the socket implementation calls a Done() callback of the user, and from that point on, the user is allowed to call Socket::send() again. With proper implementation, send() can directly be called from the callback. This interface is very easy to implement on Linux. But on Windows there's a complication, because it's non-trivial to just stop an IOCP I/O operation that is in progress, in case you decide you don't need the socket any more and want to red rid of it NOW. If you just forget about it, there could be a crash when you release your buffer but Windows is still using it. A simple solution is to CancelIo() the socket and do blocking GetQueuedCompletionStatus()s until you get an event inidicating the completion of the pending I/O operation - taking care to queue up any unrelated results you may have gotten, for later processing. However, a more efficient solution is to use reference counting or similar with your buffers, so they only get released when Windows is done with them. I didn't say it's easy, but it's possible.
    - huhtenberg 13y ago
      Yes, I know the abstraction is possible. In fact, I ended up implementing a nearly complete BSD socket API emulation on top of IOCP. Just don't lose the sight of the context, which is Windows does a lot of things right and my point being is that it doesn't.
  - wfunction 13y ago
    > The problem with Windows is that there isn't any such super-select, unlike on Linux (see epoll, signalfd, timerfd...) > I can't really use IOCP, because I need keep things reasonably portable LOL! You outright dismiss the solution because it isn't "portable". What does "portable" mean here? Portable in a particular platform?! Portable between what exactly? IOCP has been there since Windows NT 3.51 -- it's "portable" between the Windows-es. Your epoll, signalfd, timerfd, etc. are Linux functions They're "portable" between the Linux-es. i.e. The "portable" solutions you like are just as non-portable as the "non-portable" solutions you outright dismiss, making Microsoft's Windows designers look like idiots. You're never going to find a solution while considering the Linux platform to be the "portable platform" and Windows to be the weird kid on the block. Sadly, you're not the only person who thinks like this.
    - huhtenberg 13y ago
      A scalable variation of BSD socket API is supported by every major OS with a sole exception of Windows. It's an abstraction that is both simpler and more universal than IOCP model, not to mention that it also predates it by a decade. So "LOL!" yourself, bud.
      - wfunction 13y ago
        > A scalable variation of BSD socket API is supported by every major OS with a sole exception of Windows It's funny, you mention "epoll, signalfd, timerfd" as an example of a portable API, and then when I point out they're Linux APIs, you suddenly start talking about "a scalable variation" of the BSD sockets API. As for your "scalable variation of BSD sockets API", why don't you tell us the name so we know what you're referring to? Are you talking about Kqueue for example, or something else? > an abstraction that is both simpler I/O completion ports are just read-write queues... you ReadFile(), and when it's done, an item goes into the queue. (You can insert your own tasks into the queue, too, if you want.) Now just pop off the next item in the queue and off goes your thread. What part of that was so complex compared to your portable API (which for some reason you didn't feel was appropriate to mention the name of)? > and more universal Can't reply to it without knowing what "scalable variation" you're referring to, and why you didn't actually mention it as an example of portability when you were actually talking about the subject. Is portability actually a concern for you or no? (If it is, why did you bring up non-portable examples?!)

by ambrop7 13y ago

acqq 13y ago

Whatever he has more, waitfor is not the optimal way on Windows for that so it's still a complaint that round holes on Win32 are poor fit for his cubes. First choosing tools then complaining that they don't fit to some goal is typical for beginners.

huhtenberg 13y ago
Do tell me, oh, enlightened one, what I should be using if I have 10K idle network connections and I can't really use IOCP, because I need keep things reasonably portable. That's not even considering that IOCP is a great example of premature optimization at cost of clarity and simplicity. It tries to solve a problem that I don't have and it doesn't offer a simpler solution that would just do. It is not a "hole" for my square peg, it's a freaking snowflake. And expecting a modern OS to have an uncomplicated O(1) API for monitoring sockets is clearly too much to ask.
- ambrop7 13y ago
  It's possible to abstract IOCP and non-blocking sockets into a single interface with reasonable efficiency. For example: - To send data, user calls Socket::send(const char *data, size_t length). This starts sending the given data; the buffer must remain available while data is being sent. IOCP implementation will initiate I/O using WSASend() or similar. - When send operation is complete, the socket implementation calls a Done() callback of the user, and from that point on, the user is allowed to call Socket::send() again. With proper implementation, send() can directly be called from the callback. This interface is very easy to implement on Linux. But on Windows there's a complication, because it's non-trivial to just stop an IOCP I/O operation that is in progress, in case you decide you don't need the socket any more and want to red rid of it NOW. If you just forget about it, there could be a crash when you release your buffer but Windows is still using it. A simple solution is to CancelIo() the socket and do blocking GetQueuedCompletionStatus()s until you get an event inidicating the completion of the pending I/O operation - taking care to queue up any unrelated results you may have gotten, for later processing. However, a more efficient solution is to use reference counting or similar with your buffers, so they only get released when Windows is done with them. I didn't say it's easy, but it's possible.
  - huhtenberg 13y ago
    Yes, I know the abstraction is possible. In fact, I ended up implementing a nearly complete BSD socket API emulation on top of IOCP. Just don't lose the sight of the context, which is Windows does a lot of things right and my point being is that it doesn't.
- wfunction 13y ago
  > The problem with Windows is that there isn't any such super-select, unlike on Linux (see epoll, signalfd, timerfd...) > I can't really use IOCP, because I need keep things reasonably portable LOL! You outright dismiss the solution because it isn't "portable". What does "portable" mean here? Portable in a particular platform?! Portable between what exactly? IOCP has been there since Windows NT 3.51 -- it's "portable" between the Windows-es. Your epoll, signalfd, timerfd, etc. are Linux functions They're "portable" between the Linux-es. i.e. The "portable" solutions you like are just as non-portable as the "non-portable" solutions you outright dismiss, making Microsoft's Windows designers look like idiots. You're never going to find a solution while considering the Linux platform to be the "portable platform" and Windows to be the weird kid on the block. Sadly, you're not the only person who thinks like this.
  - huhtenberg 13y ago
    A scalable variation of BSD socket API is supported by every major OS with a sole exception of Windows. It's an abstraction that is both simpler and more universal than IOCP model, not to mention that it also predates it by a decade. So "LOL!" yourself, bud.
    - wfunction 13y ago
      > A scalable variation of BSD socket API is supported by every major OS with a sole exception of Windows It's funny, you mention "epoll, signalfd, timerfd" as an example of a portable API, and then when I point out they're Linux APIs, you suddenly start talking about "a scalable variation" of the BSD sockets API. As for your "scalable variation of BSD sockets API", why don't you tell us the name so we know what you're referring to? Are you talking about Kqueue for example, or something else? > an abstraction that is both simpler I/O completion ports are just read-write queues... you ReadFile(), and when it's done, an item goes into the queue. (You can insert your own tasks into the queue, too, if you want.) Now just pop off the next item in the queue and off goes your thread. What part of that was so complex compared to your portable API (which for some reason you didn't feel was appropriate to mention the name of)? > and more universal Can't reply to it without knowing what "scalable variation" you're referring to, and why you didn't actually mention it as an example of portability when you were actually talking about the subject. Is portability actually a concern for you or no? (If it is, why did you bring up non-portable examples?!)