4 ms·
I use the pv “Pipe Viewer” tool to do the same. You can either put it in the middle of a pipe, or pass it a PID using -p http://www.ivarch.com/programs/pv.shtml
by lathiat 3y ago
I use the pv “Pipe Viewer” tool to do the same. You can either put it in the middle of a pipe, or pass it a PID using -p
http://www.ivarch.com/programs/pv.shtml http://www.ivarch.com/programs/pv.shtml
It works by reading /proc/PID/fdinfo/*
- mavhc 3y agoI used PV's rate limit when trying to do to a multi terabyte zfs send, slow it down and speed it back up as required
- geraldhh 3y agowhy would one want to slow it down?
- LeFantome 3y agoTo stop from saturating a slow shared connection for an extended period? This kind of transfer can interfere with what other users are trying to accomplish over the same link.
- geraldhh 3y agoah, sure thing! you want some kind of managed buffer (aqm/sqm/qos) in your bottleneck router. openwrt or any linux can do it as the needed parts (fq_codel or cake qdisc) are in upstream for some years now and wrappers like firewall distros or libre-qos make it available via gui. there is also some proprietary soho gear that allegedly works, big-iron is still on red/pie w/o fair queueing afaik
- yabones 3y agoQoS is sort of a last resort, since it works by basically breaking TCP intermittently (dropping ACK segments). Any time you can throttle further up the stack is ideal.
- geraldhh 3y agotl;dr SQM is needed in most bottleneck routers as endpoints cannot be relied upon to know or respect resource constraints. yes and no. "throttling up the stack" is crude but effective in a narrow set of circumstances (just takes an additional transfer from another device to render it ineffective). ack thinning (dropping ack segments) is supported in cake but it's not a mechanism of throttling but for very asymmetric lines such as docsis where surplus ack's can saturate the upstream. dropping of regular packets is the only way to reliably communicate "slow down" between tcp-endpoints (yes ecn exists) but this is not the main selling point here. every queue has to drop packets at some point, codel does it in a smart way. fair queueing dissects diverse packet streams into flows (same set of src/dst ip/port) and separates them in order to minimize interference. this is what you want basically everywhere as ip-networks are by definition "best effort" and no amount of userspace whishmakeing (dscp,l4s) is going to fix it.
- dtaht 3y agothank you gerald for getting this all exactly right.
- katdork 3y agoso that you can stop the activity during work / life hours and let it run while you're asleep, perhaps? so if your internet isn't great you don't lose all of your capacity to use the internet
- geraldhh 3y agosure. see my other comment regarding SQM/QOS. this can be fixed
- mavhc 3y agoin my case because the drives it was writing too did not like going fast (SMR) and would fail, replugging them in caused zfs to fix them, but I'd rather it slows down until I'm physically there so it's not got so much to resilver that it fails yet again
- geraldhh 3y agoyou have my sympathies
- pimlottc 3y agoI've used pv before to down-throttle a massive wipe of old database entries in production. Doing it full-speed would have killed the site for active users.
- geraldhh 3y agofair use :) thou it suggests to me that you might want to investigate your i/o scheduling arrangements
- vntok 3y agoOn a VMware virtualized infra, when you migrate a virtual machine from one (ssd, very fast) datastore to another (mecha, much slower) datastore over a very fast link, sometimes you get a corrupted VM on the target.
- geraldhh 3y agouh that does not sound like a feature ;)
- foreigner 3y agoI use pv all the time and didn't know you could use it to watch an existing process. Thanks for the tip!