5 ms·
I don't understand why you would want an entire new execution scheduling mechanism just to print results in a specific order. The post dismisses the idea of inc
by tptacek 2y ago
I don't understand why you would want an entire new execution scheduling mechanism just to print results in a specific order. The post dismisses the idea of including an index (a "sequence number" if you really want to be snooty about the idea of counting your requests), but "asynchronously waitable objects" is much, much more mechanism than that.
The irony of all of this is that the core thing the post wants to do, of posting results in order to reduce confusion for users, is also probably (I don't know the complete use case, but in general) wrong! Print when you get the results back! Don't block results behind a slow early request! It's a command line program, what's the virtue of queueing fast responses? The only point of the program is to print the output.
- alexjurkiewicz 2y ago> The only point of the program is to print the output. A consistent structure helps humans parse the output. I have host(1)'s output order in my mental memory.
- tptacek 2y agoAnother way to think about this: if the only point of the program is to print the output, why not just serialize, or pipeline only one request past the current one? Or you could just use a sync.WaitGroup and not print anything until all the results are in, which is what the proposed design does anyways in the case where an early request is the slowest to resolve.
- klodolph 2y agoWhy not serialize / just print one? Parallel is faster. Why not wait? Because it’s nice to see results as they come in. Yes, there are pathological possibilities here, if an early request is slow. But this complaint sounds like a complaint about the author’s goals, which are stated clearly enough. “The author has the wrong desires or goals”, your comment seems to say.
- tptacek 2y agoIs it faster? The design proposed deliberately withholds fast results in order to ensure they're printed in sequence. Anybody who has ever run "traceroute" knows what this UX is like. If I wasn't going full TUI, I would just print the results as they come in, on stderr (with an index number). A sync.WaitGroup is as fast as the proposed behavior, for what it's worth, and doesn't have the ordering complexity.
- skywhopper 2y agoDude, just accept the premise of the article. He gives the requirements: Execute in parallel, print in order as quickly as possible. That’s the premise. The point of the article is in how to achieve that, not whether you should. “Print as soon as possible and ignore ordering” is not a challenge nor an interesting topic to write an article about.
- kaashif 2y agoYeah, this is funny. It's like presenting someone with a thought experiment and them saying "yeah but that would never happen".
- strken 2y agoFrom the article: > (The specific context is that I've got a little Go program to do IP to name DNS lookups (it's in Go for reasons), and on the one hand it would be handy to do several DNS lookups in parallel because sometimes they take a while, but on the other hand I want to print the results in command line order because otherwise it gets confusing.) If I was using this program, I too would appreciate ordered results and an early return where possible without breaking the ordering. Edit: and other similar programs might use a limited pool of workers, in which case the user would want to see ordered results as they came in and they would be expected to complete roughly in the same order they were started in.
- tptacek 2y agoTo be clear: you can still easily get this result without anything nearly as complex as promises. But as someone who has written this program many times in many languages, I'm not so sure you're right that this behavior is desirable. People go out of their way to install CLI network tools that don't do this.
- strken 2y agoI agree that it seems a bit pointless unless you're running a huge batch and that I don't think reading from channels in a for loop should be referred to as promises, but I've written CLI tools where running batches of jobs in parallel and printing the results in order is desirable. I don't think it's a generally undesirable thing. Edit: not quite parallel: I mean with a limited pool of workers, or a partial ordering, or something else that means there's a reason to return results early.
- da_chicken 2y agoI think it's safe to assume that it's a toy problem. A simple example. Picking apart the example is a poor counterargument. It might be difficult to imagine a specific trivial example, but expressing a convincing example concretely can itself be a major diversion. Unless you're saying you can guarantee that your alternative is always better in all conceivable cases? Similarly, the existence of others wanting different behavior is also a poor counterargument. The fact that someone wants a Bag doesn't mean that someone else is wrong for wanting an Ordered Set.
- hnlmorg 2y agoYou can print the results when you get them back but also print them in order too. You’ll need a little bit of ANSI escape magic to do it but it’s not a complicated problem to solve. An example of this done well is docker-compose. Personally, I’d have written this code using a sequence number as well. Albeit I’d have attached the sequence number to a context. To me, this sound like the kind of problem that contexts do well.
- tptacek 2y agoThe ironic thing about the docker-compose UX, which I agree is probably the optimum here, is that it doesn't benefit from promises.
- hnlmorg 2y agoI’m not going to defend the authors desire for promises because it’s a pattern I’ve never liked in any language. But that’s just my personal preferences.
- teddyh 2y agoThis depends on your definition of “print”. What if your stdout is not a TTY? Also, if your stdout is a TTY, but your terminal is not ANSI X3.64? You might be able to look up the corresponding escape codes in terminfo based on $TERM (or simply use tput(1)), but the terminal might not even be able to perform those operations. What do you do then?
- hnlmorg 2y ago> This depends on your definition of “print”. What if your stdout is not a TTY? If it’s not a tty then you don’t print the ansi escape sequences. Just like how every other well behaving command line utility works. > Also, if your stdout is a TTY, but your terminal is not ANSI X3.64? You might be able to look up the corresponding escape codes in terminfo based on $TERM (or simply use tput(1)), but the terminal might not even be able to perform those operations. What do you do then? This feels unnecessarily argumentative but I’ll humour you. Don’t use $TERM as your only method of fingerprinting a terminal. $TERM is as useless at determining terminal capabilities as the User-Agent string is for determining a web browser’s capabilities. That all said, the actual ANSI sequences needed to perform the UX I suggested has been standardised since the days of vt100. So even nearly every hardware terminal will support it. And if you do happen to be running an actual teletype, well you’re in a museum so your use case is very different. In terms of terminal emulators, most strive for xterm compatibility. And even those that don’t, will still support vt100 sequences. If they didn’t, then even things like readline (and thus Bash) wouldn’t work in their usual expected way. Sure, the “correct” way to use ansi escape sequences is to query terminfo. But in reality you can get away with hardcoding vt100 sequences for most things. Plus you’ll reduce overhead for your utilities for something that is guaranteed to be supported anyway. In fact perversely, you might actually improve portability because terminfo itself isn’t universal (plenty of non-POSIX systems support ansi sequences but don’t ship terminfo).
- OJFord 2y agoWould a single channel + sequence numbers actually be better, does quantity of channels matter? (I'm genuinely asking, I don't use Go much.)
- deleted 2y ago[deleted]
- grahamj 2y agoAgree RE using sequence/index numbers. I would set up a reply array, send indices along with the requests and store each result in the corresponding index. Keep track of the lowest received index and print when the next one comes in. Loop the same for the next index if it's already completed. Or write it in JS where this would be trivial and compile to executable with Deno.
- halifaxbeard 2y agoI had to solve a similar problem in Go once, partially to scratch an itch and partially to speed things up. The S3 AWS SDK has support for doing chunked/parallel downloads if you can supply an io.WriterAt [1] implementation. type WriterAt interface { WriteAt(p []byte, off int64) (n int, err error) } The AWS SDK comes with an implementation that has a Bytes() method to return the contents, so you can download faster but you can't stream the results. I created a WriteAtReader that implements io.WriterAt and io.Reader. You can read up to the first byte that hasn't been written to the internal buffers. If there's nothing new to read but there's still pending writes, calls to Read will block and immediately return after the next WriteTo call. As bytes are read through Read, the WriterAtReader frees the memory associated with them. It provides the benefits/immediate results you get with a streaming implementation and the performance of parallel download streams (at the cost of increased memory usage at the outset). The first "chunk" can be a little slow, but because the second chunk is almost ready to be read by the time the first is done? zoom zoom. [1] https://pkg.go.dev/io#WriterAt https://pkg.go.dev/io#WriterAt