3 ms·

Does it? You could implement a classic select-based reader and writer in a single goroutine, to handle many sockets at once.

by orisho 6y ago

jeffbee 6y ago
Please describe how you would actually do that in Go.
- lossolo 6y ago
  Using epoll directly (via syscalls) like here https://github.com/tidwall/evio https://github.com/tidwall/evio
  - refenestrator 6y ago
    My information is outdated, but last I knew, Go always parked a goroutine when making a syscall. Seems like a lot of overhead to read bytes that you know are there from an epolled fd.
    - coder543 6y ago
      Not exactly. It depends on the syscall. Syscalls that are known to be slow are treated the way you're describing, but syscalls that are often fast are treated optimistically in a way that allows the work queue to be stolen and spawned onto a new thread only if the syscall doesn't return quickly. I don't know which category epoll fits into, but it seems like it should be treated optimistically, since epoll is used for non-blocking I/O. https://utcc.utoronto.ca/~cks/space/blog/programming/GoSchedulerAndSyscalls https://utcc.utoronto.ca/~cks/space/blog/programming/GoSched...
      - refenestrator 6y ago
        read(2) has to fit into the "might block" bucket, though, right? It would take a very smart runtime to figure out you just called epoll recently and established it won't block.
    - WGH_ 6y ago
      You could put epoll fd in non-blocking mode, wrap it with os.NewFile, use SyscallConn() to get syscall.RawConn object, and then use its Read method. Its Read method is special: you can return "not ready", and it will use the Go runtime poller to wait until it's readable, in this particular case effectively putting epoll in a epoll. In epoll case, using RawConn.Read would look like this: err := rawConn.Read(func(fd uintptr) bool { nevents, err = syscall.EpollWait(int(fd), events[:], 0) if nevents == 0 { return false // try again } return true }) Note that using RawConn.Read here is only necessary because epoll needs epoll_wait(2) instead of typical read(2). For ordinary file descriptors, like pipes, etc., setting them to non-blocking mode, wrapping them with os.NewFile, and using its ordinary Read/Write methods is sufficient.

Does it? You could implement a classic select-based reader and writer in a single goroutine, to handle many sockets at once.

by orisho 6y ago

Does it? You could implement a classic select-based reader and writer in a single goroutine, to handle many sockets at once.

jeffbee 6y ago

Please describe how you would actually do that in Go.

lossolo 6y ago
Using epoll directly (via syscalls) like here https://github.com/tidwall/evio https://github.com/tidwall/evio
- refenestrator 6y ago
  My information is outdated, but last I knew, Go always parked a goroutine when making a syscall. Seems like a lot of overhead to read bytes that you know are there from an epolled fd.
  - coder543 6y ago
    Not exactly. It depends on the syscall. Syscalls that are known to be slow are treated the way you're describing, but syscalls that are often fast are treated optimistically in a way that allows the work queue to be stolen and spawned onto a new thread only if the syscall doesn't return quickly. I don't know which category epoll fits into, but it seems like it should be treated optimistically, since epoll is used for non-blocking I/O. https://utcc.utoronto.ca/~cks/space/blog/programming/GoSchedulerAndSyscalls https://utcc.utoronto.ca/~cks/space/blog/programming/GoSched...
    - refenestrator 6y ago
      read(2) has to fit into the "might block" bucket, though, right? It would take a very smart runtime to figure out you just called epoll recently and established it won't block.
  - WGH_ 6y ago
    You could put epoll fd in non-blocking mode, wrap it with os.NewFile, use SyscallConn() to get syscall.RawConn object, and then use its Read method. Its Read method is special: you can return "not ready", and it will use the Go runtime poller to wait until it's readable, in this particular case effectively putting epoll in a epoll. In epoll case, using RawConn.Read would look like this: err := rawConn.Read(func(fd uintptr) bool { nevents, err = syscall.EpollWait(int(fd), events[:], 0) if nevents == 0 { return false // try again } return true }) Note that using RawConn.Read here is only necessary because epoll needs epoll_wait(2) instead of typical read(2). For ordinary file descriptors, like pipes, etc., setting them to non-blocking mode, wrapping them with os.NewFile, and using its ordinary Read/Write methods is sufficient.