3 ms·

While the article is factually correct, the tone strikes me as being disingenuous. The problem seems to be that the servers were running on gp2 disks, which off

While the article is factually correct, the tone strikes me as being disingenuous. The problem seems to be that the servers were running on gp2 disks, which offer a performance baseline with free short term bursts based on credits collected. The author has just realised that for consistent throughput, they would have to choose provisioned throughput and pay accordingly. This isn’t some conspiracy by AWS, though. It’s all in the documentation and isn’t even hard to find. If you want X ops per second baseline with occasional bursts you pick option A, or if you want consistent Y ops per second you provision and pay for Y. Read the manual - not having read the docs or explored the console is not an excuse to say that a service provider is being shady or rent seeking.

sudhirj 9y ago
Even the title is misleading - Docker operations are not slowing down, the author has chosen to use cheap disks which offer temporary burst performance, and is being surprised when the temporary burst credit runs out.
znep 9y ago
Exactly. I'd go further and say the burst capacity is actually a super useful and powerful feature that is very hard to get on your own hardware. Definitely can catch you unaware if you aren't on top of it and can get expensive for some needs, but as you say not hidden at all. I have a hacky shell script I sometimes use for moderately sized environments that don't have better monitoring setup, it reports the minimum percent of burst balance remaining over the past day, requires the aws cli and parallel: #!/bin/bash while getopts "p:" opt; do case $opt in p) export AWS_PROFILE=$OPTARG ;; \?) echo "Invalid option: -$OPTARG" >&2 ;; esac done if [ ! "$AWS_PROFILE" ] then echo "-p <aws profile> or AWS_PROFILE env var required" exit 1 fi start=$(date -v-1d +%Y-%m-%dT%H:%M:%S) end=$(date +%Y-%m-%dT%H:%M:%S) aws ec2 describe-volumes --output text --query 'join(`"\n"`, Volumes[*].VolumeId)' | parallel -j 20 "echo -ne {}\\\t && echo \$(aws cloudwatch get-metric-statistics --start-time $start --end-time $end --period 86400 --namespace AWS/EBS --statistics Minimum --metric-name BurstBalance --dimensions Name=VolumeId,Value={} --output text --query 'Datapoints[*].Minimum')" | sort -n -k 2
- kangman 9y ago
  thanks for this script, pretty helpful!
falcolas 9y ago
The addition of the burst mechanism to EBS volumes is relatively new. If you have been running on AWS for any reasonable amount of time, the sudden degradation of your disks will leave you scratching your head. Dedicated IOPS wasn't originally pitched as the alternative to burst disks - it was simply that normal ELBs were not predictable based on peer load, and dedicated IOPS ELBs were.

sudhirj 9y ago

Even the title is misleading - Docker operations are not slowing down, the author has chosen to use cheap disks which offer temporary burst performance, and is being surprised when the temporary burst credit runs out.

znep 9y ago

Exactly. I'd go further and say the burst capacity is actually a super useful and powerful feature that is very hard to get on your own hardware. Definitely can catch you unaware if you aren't on top of it and can get expensive for some needs, but as you say not hidden at all. I have a hacky shell script I sometimes use for moderately sized environments that don't have better monitoring setup, it reports the minimum percent of burst balance remaining over the past day, requires the aws cli and parallel: #!/bin/bash while getopts "p:" opt; do case $opt in p) export AWS_PROFILE=$OPTARG ;; \?) echo "Invalid option: -$OPTARG" >&2 ;; esac done if [ ! "$AWS_PROFILE" ] then echo "-p <aws profile> or AWS_PROFILE env var required" exit 1 fi start=$(date -v-1d +%Y-%m-%dT%H:%M:%S) end=$(date +%Y-%m-%dT%H:%M:%S) aws ec2 describe-volumes --output text --query 'join(`"\n"`, Volumes[*].VolumeId)' | parallel -j 20 "echo -ne {}\\\t && echo \$(aws cloudwatch get-metric-statistics --start-time $start --end-time $end --period 86400 --namespace AWS/EBS --statistics Minimum --metric-name BurstBalance --dimensions Name=VolumeId,Value={} --output text --query 'Datapoints[*].Minimum')" | sort -n -k 2

kangman 9y ago
thanks for this script, pretty helpful!

falcolas 9y ago

The addition of the burst mechanism to EBS volumes is relatively new. If you have been running on AWS for any reasonable amount of time, the sudden degradation of your disks will leave you scratching your head. Dedicated IOPS wasn't originally pitched as the alternative to burst disks - it was simply that normal ELBs were not predictable based on peer load, and dedicated IOPS ELBs were.