5 ms·

Praise, 2 suggestions and a question. Color coding OOM (order-of-magnitude) is smart, but the colors chosen seem quite hard to distinguish. Consider picking di

Praise, 2 suggestions and a question. Color coding OOM (order-of-magnitude) is smart, but the colors chosen seem quite hard to distinguish. Consider picking different default colors; also consider adding a redundant representation of scale, like an integer representing OOM scale. It is strange that a thoughtful design around OOM would also choose keep so many digits. If the goal is to summarize, then please throw those extra digits away. (e.g. 10.3992s is specifying the time down to the millisecond, but those extra ms are not meaningful at the second time scale.) What is this about the runtime polling tasks? Does that happen in Tokio? Why? I am only familiar with 2 async runtimes (node, vert.x) and I was under the impression that neither of these "poll their threads" in any meaningful way. Threads (or process or fiber or whatever you want to call it in this context) never initiate action on their own. They are ALWAYS waiting for something to happen to them, and that includes timeouts, which would be triggered by passing a thread a clock tick. The runtime's job is to centrally manage resources that must persist between thread behavior, so at most it is going to be polling external resources, and not it's own threads. Or maybe I don't understand what polling means here? I would interpret it as meaning "keep checking a well-known place for changes, and then do something if a change is found". But since async threads can't initiate any change on their own, this is nonsensical.

carllerche 5y ago
Also, thanks for the thought re: digit precision. I am tracking it here: https://github.com/tokio-rs/console/issues/224 https://github.com/tokio-rs/console/issues/224
aliceryhl 5y ago
Generally the way async Rust works is that you have a Future trait with a poll method, and if you call it the future will attempt to make progress if it can — e.g. if the task is a timer it will check the time and complete the task if so and otherwise return Pending. However, async Rust includes an additional concept: Wakers. When your runtime (Tokio) calls poll on a future, it gives the future a waker object, and when the future is ready to continue work, something needs to call wake on the waker. Once this happens, Tokio will poll the task again soon, and Tokio wont poll tasks that have not been woken. For example, for timers, Tokio includes a timer wheel (a sort of priority queue) with all the registered timers, and the timer wheel calls wake on the appropriate waker whenever a timer expires. Similarly with a message passing channel, the sender calls wake on the receiver's waker when a message is sent.
carllerche 5y ago
I'll answer the polling question. The Tokio runtime (and async rust in general) works a bit differently than other async runtimes like node. With node, callbacks are provided and executed when an OS event is received. With Tokio, there are no callbacks. Instead, async logic is organized in terms of tasks (kind of like async green threads). When the task is blocked on external events, it goes into a waiting state. When external OS events are received, the task is scheduled by the runtime and eventually the runtime "polls" it. Because the poll happens in response to an OS event, most times, the poll results in the task making progress. Sometimes there are false positive polls. This page goes into a bit more depth and shows an example of how one would implement a (very simple) runtime/executor: https://tokio.rs/tokio/tutorial/async https://tokio.rs/tokio/tutorial/async
- Hercuros 5y ago
  I sometimes wonder what is the fundamental distinction between a callback-based API and this wakeup-based task API. I guess the main difference is that in a callback you generally provide the result as an argument whereas with a wakeup-based API you just wake the task and it has to look for the result in some stored state somewhere. But ultimately both of them take the "rest of the computation"/continuation and store it somewhere (i.e. on some sleeping task/callback list) to be awakened/invoked later.
  - tijsvd 5y ago
    The callback is really hard to implement without allocating memory for each wakeup. The poll mechanism can simply leave the task in place. I suspect the poll thing is also easier to generate.
  - carllerche 5y ago
    It is fairly subtle and mostly an implementation detail. In Rust, the concept of "polling tasks" was very exposed before the async/await keywords were introduced, so the lingo kind of stuck. There is an argument that we should move away from that lingo now that it is mostly hidden as an implementation detail, but we haven't yet.
  - 5y ago
mycoliza 5y ago
(primary author of the console here) you're right that there are too many digits of precision right now...the reason for that is that it's actually _not_ a thoughtful design at all, though I appreciate you saying that it is; I just picked an arbitrary number when I was writing the format string and didn't really think about it. We probably don't want to display that much precision --- for smaller units, we probably don't want any fractional digits, for larger units like seconds, we probably want two digits of precision maximum. Regarding the color scheme, glad you like the idea. Because it's a terminal application, the choice of the colors was constrained a bit by the ANSI 256 color palette (https://www.ditig.com/256-colors-cheat-sheet); https://www.ditig.com/256-colors-cheat-sheet); I wanted it to be obviously a gradient, so I just picked colors that were immediately adjacent to each other in the ANSI palette. It might be better to pick colors that are one step apart from each other, instead, so they're more distinguishable visually...but there's kind of a balancing act between distinguishability and having a clear gradient. We'll keep playing with it!
- chc4 5y ago
  Quite a lot of terminal emulators support 24bit TrueColor escape codes, so it might be worth using that along with a color space designed for visual intensity corrolation (there's a lot of study in this for e.g. heatgraphs for maps)
  - mycoliza 5y ago
    Yeah, currently, the console knows how to detect TrueColor, but in this case, I just used the ANSI 256 palette rather than picking a better one when TrueColor is available...we should probably fix that! Side note, it turns out that detecting what color palettes a terminal supports 24-bit colors is surprisingly fraught. There are a couple env variables that may be set...but not every terminal emulator will set them. And then you can use `tput`...but the terminal may not have correct data in the tput database. So that was fun to learn about!
    - namibj 5y ago
      Consider going for one of the Colorcet[0] maps if you detect you can display them. They are really useful to have a neutral view of the subject. I suggest log-scale before applying the colormap for order-of-magnitude visualization. There's a Rust crate for these (I forgot the name). [0]: https://colorcet.holoviz.org/ https://colorcet.holoviz.org/

carllerche 5y ago

Also, thanks for the thought re: digit precision. I am tracking it here: https://github.com/tokio-rs/console/issues/224 https://github.com/tokio-rs/console/issues/224

aliceryhl 5y ago

Generally the way async Rust works is that you have a Future trait with a poll method, and if you call it the future will attempt to make progress if it can — e.g. if the task is a timer it will check the time and complete the task if so and otherwise return Pending. However, async Rust includes an additional concept: Wakers. When your runtime (Tokio) calls poll on a future, it gives the future a waker object, and when the future is ready to continue work, something needs to call wake on the waker. Once this happens, Tokio will poll the task again soon, and Tokio wont poll tasks that have not been woken. For example, for timers, Tokio includes a timer wheel (a sort of priority queue) with all the registered timers, and the timer wheel calls wake on the appropriate waker whenever a timer expires. Similarly with a message passing channel, the sender calls wake on the receiver's waker when a message is sent.

carllerche 5y ago

I'll answer the polling question. The Tokio runtime (and async rust in general) works a bit differently than other async runtimes like node. With node, callbacks are provided and executed when an OS event is received. With Tokio, there are no callbacks. Instead, async logic is organized in terms of tasks (kind of like async green threads). When the task is blocked on external events, it goes into a waiting state. When external OS events are received, the task is scheduled by the runtime and eventually the runtime "polls" it. Because the poll happens in response to an OS event, most times, the poll results in the task making progress. Sometimes there are false positive polls. This page goes into a bit more depth and shows an example of how one would implement a (very simple) runtime/executor: https://tokio.rs/tokio/tutorial/async https://tokio.rs/tokio/tutorial/async

Hercuros 5y ago
I sometimes wonder what is the fundamental distinction between a callback-based API and this wakeup-based task API. I guess the main difference is that in a callback you generally provide the result as an argument whereas with a wakeup-based API you just wake the task and it has to look for the result in some stored state somewhere. But ultimately both of them take the "rest of the computation"/continuation and store it somewhere (i.e. on some sleeping task/callback list) to be awakened/invoked later.
- tijsvd 5y ago
  The callback is really hard to implement without allocating memory for each wakeup. The poll mechanism can simply leave the task in place. I suspect the poll thing is also easier to generate.
- carllerche 5y ago
  It is fairly subtle and mostly an implementation detail. In Rust, the concept of "polling tasks" was very exposed before the async/await keywords were introduced, so the lingo kind of stuck. There is an argument that we should move away from that lingo now that it is mostly hidden as an implementation detail, but we haven't yet.
- 5y ago

mycoliza 5y ago

(primary author of the console here) you're right that there are too many digits of precision right now...the reason for that is that it's actually _not_ a thoughtful design at all, though I appreciate you saying that it is; I just picked an arbitrary number when I was writing the format string and didn't really think about it. We probably don't want to display that much precision --- for smaller units, we probably don't want any fractional digits, for larger units like seconds, we probably want two digits of precision maximum. Regarding the color scheme, glad you like the idea. Because it's a terminal application, the choice of the colors was constrained a bit by the ANSI 256 color palette (https://www.ditig.com/256-colors-cheat-sheet); https://www.ditig.com/256-colors-cheat-sheet); I wanted it to be obviously a gradient, so I just picked colors that were immediately adjacent to each other in the ANSI palette. It might be better to pick colors that are one step apart from each other, instead, so they're more distinguishable visually...but there's kind of a balancing act between distinguishability and having a clear gradient. We'll keep playing with it!

chc4 5y ago
Quite a lot of terminal emulators support 24bit TrueColor escape codes, so it might be worth using that along with a color space designed for visual intensity corrolation (there's a lot of study in this for e.g. heatgraphs for maps)
- mycoliza 5y ago
  Yeah, currently, the console knows how to detect TrueColor, but in this case, I just used the ANSI 256 palette rather than picking a better one when TrueColor is available...we should probably fix that! Side note, it turns out that detecting what color palettes a terminal supports 24-bit colors is surprisingly fraught. There are a couple env variables that may be set...but not every terminal emulator will set them. And then you can use `tput`...but the terminal may not have correct data in the tput database. So that was fun to learn about!
  - namibj 5y ago
    Consider going for one of the Colorcet[0] maps if you detect you can display them. They are really useful to have a neutral view of the subject. I suggest log-scale before applying the colormap for order-of-magnitude visualization. There's a Rust crate for these (I forgot the name). [0]: https://colorcet.holoviz.org/ https://colorcet.holoviz.org/