4 ms·

My very first choice, which is typically missed by "modern" [0] languages and approaches, would be to put the stuff in a buffer, and have the client read it out

My very first choice, which is typically missed by "modern" [0] languages and approaches, would be to put the stuff in a buffer, and have the client read it out when it is ready. Callbacks are usually inferior: • The client must create lots of small functions that need to conform to some strange interface that needs some context parameter type (void * in C or complicated type hackery in other languages). • Each of those callback functions is harder and more boilerplatey to implement because it's completely broken out of the client's control flow - there is no context around. All this applies to inheritances / "interface" mechanisms in some languages just as well. I don't know why we still haven't abandonded this crap, it adds nothing but new words and types to achieve the same things. What do we gain from using simple buffers? • Better decoupling of client vs library/framework/whatever implementation: temporal decoupling. Client can decide when is the right time to take some action. • Client can setup the necessary boilerplate in a single function (stack frame) once, and then process all messages, of all types, instead of having redundant boilerplate for each callback. • No inconvenient context type (void * or whatever) or other conformance to any interface needed. What can the library do when the buffer is full? • Simple - it should back out and return to user code. The user needs to process the existing messages in the buffer first. The user should then call into the library and have it reattempt what it did last. When do simple buffers not work? • Only when the library needs some immediate reaction to the "event". In other words, if it is inconvenient to implement the library in an event-driven fashion and is better implemented in a completely synchronous fashion and needs feedback from the user. For example, the library might request some memory from the user that must be satisfied immediately because the implementation can't back out of the current function. I think libraries that requires the user to make callbacks should be a rare exception, and not the norm. [0] When I read "modern" and I am in a cynical mood, I tend to think "ignorant of the old, simple, and proven ways".

travisjungroth 5y ago
I don't find your first two points convincing. The structure of the data put in the buffer is equivalent to the interface of the function. The buffer does encourage you to use data instead of classes. I want more guidance to my users about the contract that they're expected to fulfill and I don't think buffers help with that problem. I do appreciate the temporal aspect you're talking about. I'd be all about if I was Erlang, but I don't think it's worth it for my purpose in Python.
- deleted 5y ago
  [deleted]
ptx 5y ago
An example of this would be a pull parser for XML, right? A callback-based API can be built on top of the pull-based one if desired, but not the other way around, as far as I can tell, without using a separate thread. So the pull-based approach (i.e. putting stuff in a buffer) is more flexible. But the issues you point out around callbacks and passing context parameters apply mostly to C. In languages that support closures it's easy to give callbacks whatever context they need.
- jstimpfle 5y ago
  You could make a callback that only pushes to a buffer for later processing - to achieve the same behaviour that I suggested. So in a way, callbacks are more general (for an event that was pushed to a buffer, you can't have it be immediately processed by the other party, unless you add major threading and scheduling hackery). Nevertheless I recommend to never make a callback based API unless there is a clear need for callbacks. If there isn't a need, plain buffers (Maybe I should better say "queues") and decoupled processing are the right thing. Callbacks encourage synchronous processing and where they are used to only push to a buffer they are only overhead.

travisjungroth 5y ago

I don't find your first two points convincing. The structure of the data put in the buffer is equivalent to the interface of the function. The buffer does encourage you to use data instead of classes. I want more guidance to my users about the contract that they're expected to fulfill and I don't think buffers help with that problem. I do appreciate the temporal aspect you're talking about. I'd be all about if I was Erlang, but I don't think it's worth it for my purpose in Python.

deleted 5y ago
[deleted]

ptx 5y ago

An example of this would be a pull parser for XML, right? A callback-based API can be built on top of the pull-based one if desired, but not the other way around, as far as I can tell, without using a separate thread. So the pull-based approach (i.e. putting stuff in a buffer) is more flexible. But the issues you point out around callbacks and passing context parameters apply mostly to C. In languages that support closures it's easy to give callbacks whatever context they need.

jstimpfle 5y ago
You could make a callback that only pushes to a buffer for later processing - to achieve the same behaviour that I suggested. So in a way, callbacks are more general (for an event that was pushed to a buffer, you can't have it be immediately processed by the other party, unless you add major threading and scheduling hackery). Nevertheless I recommend to never make a callback based API unless there is a clear need for callbacks. If there isn't a need, plain buffers (Maybe I should better say "queues") and decoupled processing are the right thing. Callbacks encourage synchronous processing and where they are used to only push to a buffer they are only overhead.