4 ms·

> You have to read from it, and deal with the result. They try to keep it simple. As with all I/O: it's blocking your current goroutine, so if you want it non-

> You have to read from it, and deal with the result. They try to keep it simple. As with all I/O: it's blocking your current goroutine, so if you want it non-blocking you spawn a new goroutine to do this in, pass it a channel, the spawned routine proxies the result to it and your main routine does select on that. The semantics for reading sockets are different from reading channels (compare the types; one is multi valued, the other single valued---EDIT: no they're not, exactly, see [1]), so it kind of makes sense you can't share the same construct. I'm not a Go apologist by any means, I think it has a tremendous amount of flaws. But I/O, (a)synchronicity and concurrency are not among them. They are simple and consistent, and (most importantly) orthogonal building blocks which you can combine at your leisure to create more complex constructs. (Besides, how do you interrupt a read, anyway? Or any syscall? You can't do that in C, either...?) [1] edit: fair is fair, go channel reads are (of course) also multi valued. however, the types still don't exactly match, since you can't pass arguments (buffer) to a channel read. you'd need some extra hoops to jump through, as other commenters suggest, which starts becoming magic and breaking down the simplicity and orthogonality of the constructs.

lobster_johnson 10y ago
Sorry for being unclear. You don't interrupt the read as such. You interrupt the select. The usual trick on POSIX is to use a self-pipe, since select()/epoll() only work on file descriptors and not, say, pthread primitives. Windows is slightly better here: WaitForMultipleObjects supports waiting on any primitive that supports blocking, so you wait on the socket + on an event object, and signaling the event object will unblock the wait. Windows also has completion ports, and the APIs are generally more extensive (not a fan of Windows, but they got this part very right). I do understand why Go is the way it is. But it's important to recognize that it's a compromise in design that inherently reduces your flexibility — such as in this particular case. I don't think there's anything technically preventing Go from supporting non-blocking I/O, though. Channels and FDs/sockets are both conceptually multi-valued; what did you mean by that? One major difference is that reads can fail with an error, and channel reads can't. Edit: Also, I didn't mean that select{} on a file descriptor ought to actually return data. It would be more appropriate for it to yield a true/false value to signal that the FD has data, so you'd do "case <-fd: fd.Read(bufSize)".
- nothrabannosir 10y ago
  Agree with everything. Just to reply to this: > Channels and FDs/sockets are both conceptually multi-valued; what did you mean by that? I was trying to preempt pedantry, but ended up being the pedant myself :P "You can simulate multi valued responses in a channel by passing a struct."
- mentat 10y ago
  You can select on the read and an time.After channel which "interrupts" the select for a timeout.
  - lobster_johnson 10y ago
    You can't select on a read, though. select{} only works on channels. You can run the read in a separate goroutine, but that doesn't stop the read from happening.
    - tptacek 10y ago
      You can't use the language "select", but you can call the select(2) system call.
      - lobster_johnson 10y ago
        Indeed, someone already pointed this out: https://news.ycombinator.com/item?id=13332847 https://news.ycombinator.com/item?id=13332847.
- echlebek 10y ago
  How is your "case <-fd: fd.Read(bufSize)" not a race condition?
  - atombender 10y ago
    Race condition? It's no different from select() followed by recv() in C. But I'm talking about semantics here, not suggesting that this would necessarily be how you'd write it. The main idea was allowing select{} to work on non-channel sources of notifications, such as file descriptors.
alexhutcheson 10y ago
> I'm not a Go apologist by any means, I think it has a tremendous amount of flaws. Hopefully not to derail this too much, but what do you see as the major flaws? I'm asking out of curiosity - not trying to start a flame war.
heavenlyhash 10y ago
There's a problem lurking behind this "simple" model: the goroutine you park on that blocking IO operation can't be collected reliably itself. I care about collecting goroutines, and would advocate that everyone should care about this. There's been a litany [1] of blogs about this recently, if anyone doesn't consider this self-evident. It's just systematically sane. Letting goroutines spin off into the distance with no supervision is extremely likely to represent either a resource leak, a missing error handle path, or an outright logic bug. In a dream world, we might have Erlang-style supervisor trees so this can all be done with less boilerplate. It's almost even possible to do this as a library in go! ... except for this blocking IO problem, which throws a wrench in things: any supervisor of a goroutine that may be IO-blocked may itself be indefinitely stuck, and so on up-tree, making the concept unworkable. --- [1] Some recent articles hammering the point of "collect your goroutines": - https://dave.cheney.net/2016/12/22/never-start-a-goroutine-without-knowing-how-it-will-stop https://dave.cheney.net/2016/12/22/never-start-a-goroutine-w... - https://rakyll.org/leakingctx/ https://rakyll.org/leakingctx/
- tptacek 10y ago
  It's particularly a problem with timeouts and file descriptors.
- lobster_johnson 10y ago
  That's also a great point. I also wish for supervisor trees and killable processes.
  - heavenlyhash 10y ago
    This is an attempt to implement supervisor trees in go as a library: https://github.com/polydawn/go-sup https://github.com/polydawn/go-sup (Hej; author.) It takes the Wait-for-children behavior path by default, and that seems much safer to me. The downside is, as we're describing: if you do get stuck in IO somewhere, you're a pretty unhappy camper. This library makes the choice of (by default; it's a handler func you can override) logging warnings to stderr if a task doesn't exit within 2 seconds of being told to quit... but this still feels like something of a hack; more of a debug measure than a concrete way to improve system stability. Better ideas would be highly welcome!

lobster_johnson 10y ago

Sorry for being unclear. You don't interrupt the read as such. You interrupt the select. The usual trick on POSIX is to use a self-pipe, since select()/epoll() only work on file descriptors and not, say, pthread primitives. Windows is slightly better here: WaitForMultipleObjects supports waiting on any primitive that supports blocking, so you wait on the socket + on an event object, and signaling the event object will unblock the wait. Windows also has completion ports, and the APIs are generally more extensive (not a fan of Windows, but they got this part very right). I do understand why Go is the way it is. But it's important to recognize that it's a compromise in design that inherently reduces your flexibility — such as in this particular case. I don't think there's anything technically preventing Go from supporting non-blocking I/O, though. Channels and FDs/sockets are both conceptually multi-valued; what did you mean by that? One major difference is that reads can fail with an error, and channel reads can't. Edit: Also, I didn't mean that select{} on a file descriptor ought to actually return data. It would be more appropriate for it to yield a true/false value to signal that the FD has data, so you'd do "case <-fd: fd.Read(bufSize)".

nothrabannosir 10y ago
Agree with everything. Just to reply to this: > Channels and FDs/sockets are both conceptually multi-valued; what did you mean by that? I was trying to preempt pedantry, but ended up being the pedant myself :P "You can simulate multi valued responses in a channel by passing a struct."
mentat 10y ago
You can select on the read and an time.After channel which "interrupts" the select for a timeout.
- lobster_johnson 10y ago
  You can't select on a read, though. select{} only works on channels. You can run the read in a separate goroutine, but that doesn't stop the read from happening.
  - tptacek 10y ago
    You can't use the language "select", but you can call the select(2) system call.
    - lobster_johnson 10y ago
      Indeed, someone already pointed this out: https://news.ycombinator.com/item?id=13332847 https://news.ycombinator.com/item?id=13332847.
echlebek 10y ago
How is your "case <-fd: fd.Read(bufSize)" not a race condition?
- atombender 10y ago
  Race condition? It's no different from select() followed by recv() in C. But I'm talking about semantics here, not suggesting that this would necessarily be how you'd write it. The main idea was allowing select{} to work on non-channel sources of notifications, such as file descriptors.

alexhutcheson 10y ago

> I'm not a Go apologist by any means, I think it has a tremendous amount of flaws. Hopefully not to derail this too much, but what do you see as the major flaws? I'm asking out of curiosity - not trying to start a flame war.

heavenlyhash 10y ago

There's a problem lurking behind this "simple" model: the goroutine you park on that blocking IO operation can't be collected reliably itself. I care about collecting goroutines, and would advocate that everyone should care about this. There's been a litany [1] of blogs about this recently, if anyone doesn't consider this self-evident. It's just systematically sane. Letting goroutines spin off into the distance with no supervision is extremely likely to represent either a resource leak, a missing error handle path, or an outright logic bug. In a dream world, we might have Erlang-style supervisor trees so this can all be done with less boilerplate. It's almost even possible to do this as a library in go! ... except for this blocking IO problem, which throws a wrench in things: any supervisor of a goroutine that may be IO-blocked may itself be indefinitely stuck, and so on up-tree, making the concept unworkable. --- [1] Some recent articles hammering the point of "collect your goroutines": - https://dave.cheney.net/2016/12/22/never-start-a-goroutine-without-knowing-how-it-will-stop https://dave.cheney.net/2016/12/22/never-start-a-goroutine-w... - https://rakyll.org/leakingctx/ https://rakyll.org/leakingctx/

tptacek 10y ago
It's particularly a problem with timeouts and file descriptors.
lobster_johnson 10y ago
That's also a great point. I also wish for supervisor trees and killable processes.
- heavenlyhash 10y ago
  This is an attempt to implement supervisor trees in go as a library: https://github.com/polydawn/go-sup https://github.com/polydawn/go-sup (Hej; author.) It takes the Wait-for-children behavior path by default, and that seems much safer to me. The downside is, as we're describing: if you do get stuck in IO somewhere, you're a pretty unhappy camper. This library makes the choice of (by default; it's a handler func you can override) logging warnings to stderr if a task doesn't exit within 2 seconds of being told to quit... but this still feels like something of a hack; more of a debug measure than a concrete way to improve system stability. Better ideas would be highly welcome!