7 ms·
Show HN: Visualizing disk IO activity using log-scale banded graphs
- JepZ 9y agoWhile I find the graphs not very intuitive to read, they hold great information once you understand how to read them :-)
- ryandrake 9y agoRecommend testing the site on mobile. I stared confused at a black screen for a while before realizing that it had a massive fixed width and I had to pan right to see anything.
- apankrat 9y agoYep, noted. We are a desktop software shop, so mobile browser support hasn't been a top priority.
- azinman2 9y agoBut that doesn’t mean people who want desktop software aren’t browsing HN on their phone...
- adrianN 9y agoFor me it shows the number 2 and nothing else, even if I view source.
- dsl 9y agoSame here. Looks like the server broke.
- apankrat 9y agoThings are OK on the server side. Ah, scratch that. I see the issue!... Check now?
- sirsuki 9y agoI'm confused. What does "2" mean in the context of a web page? If the server broke wouldn't it cause a 500 HTTP response? This just seems odd: 'if error write "2"'. Maybe it is some kinda of secret society code: "I'm sorry Dave, I can not do that... TWO!!!!"
- gbrown_ 9y agoThe images really need to be larger or at the least be available in a higher resolution.
- Gys 9y ago(For Windows)
- apankrat 9y agoAs I said in another comment, the interesting part that I wanted to share is how the data is graphed, and not that there's an executable.
- nemo1618 9y agoReally neat! Any plans for Linux support? I would love to do some guided optimizations using this tool.
- apankrat 9y agoZero plans, sorry. This is an internal tool that just happens to produce good looking graphs, so we released a build to let people play with it if they want to. It's not a beta of a product, more of a tech demo... if even that.
- TheWiseOne 9y agoIs this going to be part of Bvckup 2 by any chance?
- core-questions 9y agoA better target would be a Grafana plugin.
- vatys 9y agoYes, I’d love to be able to do this kind of y-axis scaling in Grafana. I was just messing with a network I/O bandwidth graph in Grafana this morning and thinking about log scales. It hadn’t occurred to me to band it by binary/1024 orders of magnitude, but I like the result here.
- 9y ago
- apankrat 9y agoAuthor here. We had a need for a tool to graph IO activity at arbitrary sampling rates, so there was finally a solid excuse to do some datavis work :) The thing I wanted to show is not the program itself, but the idea of factoring values and displaying factorization to cover several magnitude levels in one go. I suspect that this has been done before, but I couldn't find anything similar when I was researching this beforehand.
- LukeShu 9y agoI've never seen something similar before either, and it is a very useful visualization. I'm thinking about how to do something similar in my monitoring. How would you extend it to allow zooming on the X-axis? With a continuous log scale, or a simple linear scale, it's easy to know how to compress the data horizontally: take the min/max/avg of the datapoints that go in to one horizontal pixel. I'm not sure how to apply that to your discrete log scale; it seems to work on the assumption that points along the X-axis are discrete. Edit: I said Y-axis, I meant X-axis. I'm embarrassed.
- apankrat 9y ago> How would you extend it to allow zooming on the Y-axis? Well, the whole point is to NOT need to do any Y-scaling. What would be the case for zooming in?
- LukeShu 9y agoI'm sorry, I meant X-axis. I'm embarrassed.
- apankrat 9y agoHa. Good question, actually. The simplest option would obviously be to just show a conventional candlestick when zoomed out. There might be an option to do something interesting with rendering the stick when min/max have the same higher-order factors... but it might also end up looking like a royal mess.
- hs86 9y agoThis just reminds me that I miss something like iStat Menus [0] on Windows. Process Explorer [1] can show some small graphs as tray icons but there is not much detail visible without opening the main application's window. XMeters [2] does a little more but its configuration is very inflexible and clicking on any item just opens the plain Windows Taskmanager instead of showing additional details and graphs like iStat Menus. [0] https://bjango.com/mac/istatmenus/ https://bjango.com/mac/istatmenus/ [1] https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer https://docs.microsoft.com/en-us/sysinternals/downloads/proc... [2] https://entropy6.com/xmeters/ https://entropy6.com/xmeters/
- java-man 9y agoYour work is very pleasant to look at, Alex!
- apankrat 9y agoThanks, glad you like it!
- cbcoutinho 9y agoThis is a really cool visualization, and a nice implementation of an idea. Thanks for sharing it
- yoodenvranx 9y agoTo the people who use log-scales for visualization of discrete events (e.g. 0, 1, 2, 3, 4, ...): How do you deal with a value of 0? Do you use the same distance between "0" and 1 as between 1 and 10? On a different note: In a lot of cases I prefer using something like sqrt(data) or data^(0.x) because a) it smoothly supports the case of 0 and b) it squashes the data less than log. Regarding b): In a lot of cases I think that log() squashes the information too much and you easily lose a sense of scale. A scaling funtion like sqrt(data) still squashes data with high values but it maintains the feeling of "Oh, this value is much higher than that value" better than log(data).
- aidenn0 9y agoThe intercept of a log-scaled axis should always be positive, and a value of 0 would not appear on the chart. Many times it's non-sensical for a value of 0 (e.g. pressure, frequency), but wikipedia[1] has an example of tracking pandemic cases (which tends to be logistic, so includes an exponential growth phase) and you see that values below 0 do not appear at all on the graph. In this case, the slope of each dataset is the important thing, and very low values are probably too noisy to give you information anyways 1: https://en.wikipedia.org/wiki/File:Influenza-2009-cases-logarithmic.png https://en.wikipedia.org/wiki/File:Influenza-2009-cases-loga...
- stenl 9y agoThe typical approach is to plot log(x+1) instead, which deals with the zeros as long as you know what you’re looking at. It’s nice to add some jitter too, e.g. a uniform random float number in the -0.25 to 0.25 range.
- productionx 9y agoI like it, the size is way to small though. Resource Monitor gives this information as well btw:edit, reread your post for the arbitrary file size thing-makes more sense now. Couple requests: option to scroll the timeline, option to export the timeline, ability to view more than one drive at a time.
- apankrat 9y agoYou can't get "Read X bytes since last reboot" from the standard performance counters, so, no, Resource Monitor doesn't have this data. The best it can do is the IO rate at one-second resolution.
- edmundhuber 9y agoThis is a great idea, and I have one suggestion: It can be kinda hard to see if two neighboring entities are the same in a given band (kbs, mbs, gbs), because you're looking for the absence of a difference of a pixel or two. Maybe if they are the same and neighboring, join the two columns rather than have them spaced? That would visually instantly tell you that those two samples are the same.
- apankrat 9y agoInteresting idea, thanks. I'll give it a try.
- Groxx 9y agoPage isn't loading for me (I just get a "2", no other content), but archive.org appears to have it cached: https://web.archive.org/web/20180411180009/https://bvckup2.com/wip/10042018 https://web.archive.org/web/20180411180009/https://bvckup2.c...
- apankrat 9y agoIt was a hiccup on the server side. Sorry about that. Should be back to normal now.
- sgc 9y agoUnreadable for me on a 15 inch 1920x1080 monitor. I can't see the MB/KB/b labels. It's tiny and should be resizable. I don't know what the lines indicate, so I can only see real time data, I have to guess what past values represent.
- imhoguy 9y agoClever idea! I would see this nicely applicable to system tray icons for monitoring tools. 16 pixels - 4 log bands with 0.25 precision.
- TwistedWave 9y agoI don't understand. In most of your samples, there is no bar in the first two bands. Does that mean that your samples are a multiple of 1 MB? If your read/write rate is not constant, such as in the non-optimized file copy and you display the rate mod 1024 in the first band, I expect to see seemingly random bars that would not show any meaningful information. What am I missing?
- apankrat 9y ago> Does that mean that your samples are a multiple of 1 MB? No my samples, but the buffer sizes in read/write requests are multiples of 1 MB. Consider when a disk driver would update its read/write counters - it happens when a request is completed, so even if a request takes longer than a sampling interval to complete, the byte counter will still go up in buffer size, atomically.
- deleted 9y ago[deleted]
- boromi 9y agogreat tool however the font size is really small and hard too read
- jamesbrink 9y agoHere is a quick, far less useful visualization of file activity using Gource :) This was run on OSx, linux users can use strace to watch for open files, much better performance. # Using lsof in a loop (no sudo required) (while :; do lsof |gsed -r "s#(\w+)\s+.\s+(.+)#$(gdate +"%s")\|\1\|M\|\2#;t;d";done) | \ gource --realtime --filename-time 2 --highlight-users --1280x800 --log-format custom - # Using Dtrace (better option, could replace with opensnoop possibly but still requires dtrace) sudo dtrace -n 'syscall::open:entry { printf("%u|%s|%s",walltimestamp,execname,copyinstr(arg0)); }' | \ gsed -r 's#.+ ([0-9]{10}).+\|(.+)\|(.+).+#\1\|\2\|M\|\3#;t;d' |\ gource --realtime --filename-time 2 --highlight-users --1280x800 --log-format custom -