4 ms·

No way t2's are a good choice for your average webservice. You don't get the full CPU with t2's. With a t2.medium if you're using over 20% of the cpu (40% of 1

by ryeguy 9y ago

No way t2's are a good choice for your average webservice. You don't get the full CPU with t2's. With a t2.medium if you're using over 20% of the cpu (40% of 1 vcpu or 20% of both), you're burning CPU credits. So unless you have a webapp that uses 8gb of memory yet stays somewhere under 20% cpu utilization (maybe with some peaks), you'll eventually get throttled. t2's are for really bursty workloads, which only makes sense for a webservice if you aren't handling much traffic in general and you have maybe 2 instances total.

eropple 9y ago
Most web apps are IO bound, not CPU bound, and a throttled T2 has IO to spare versus its CPU usage.
- Johnny555 9y ago
  t2's are great until something happens and you really need the full CPU, then you get throttled and suddenly your service goes down because it can't keep up with the load. If you use t2's for anything important, keep an eye on the CPU credit balance.
  - cjsuk 9y ago
    Just applying selinux policy on CentOS 7 will kill the CPU credits on a micro instance. Running updates is a risky businesss.
  - blasdel 9y ago
    I work on T2, and boy do I have a fix for you!: https://aws.amazon.com/blogs/aws/new-t2-unlimited-going-beyond-the-burst-with-high-performance/ https://aws.amazon.com/blogs/aws/new-t2-unlimited-going-beyo...
- ryeguy 9y ago
  But if you get throttled in the first place that means you're going to have some kind of performance degradation since your server was using more than the baseline level. Web apps being IO bound isn't relevant here, because the only requirement for issues to arise is for your server to have consistent 20%+ cpu usage.
  - jrochkind1 9y ago
    Well, they're relevant in that a heavily IO bound app is probably unlikely to use much CPU -- it's too busy waiting on IO to use 90% of CPU, and maybe does stay mostly under 20%. Obviously this depends on a lot of details beyond "IO-bound", but it is not implausible, and I think does accurately describe many rails apps.
eknkc 9y ago
Just for a data point; We have been serving around 10 million web requests / day per t2.medium. Not lightweight static files or small json responses, actual dynamic pages with database load. Our servers mostly sit around 20% though. Might not be that much compared to larger web services but we have a localised audience so nighttime cpu credits get collected and used during peak times. So it fits our workload. They are great value when the credit system work in your setting. Have a grafana dashboard constantly monitoring credits and have alerts in place though. But haven’t had a sudden issue that we needed to manually remedy.
- Xorlev 9y ago
  > Have a grafana dashboard constantly monitoring credits and have alerts in place though. This, otherwise your infrastructure will come grinding to a halt at the worst time. It's a potential DoS vector in that way too, if you're relying on charging up credits overnight to handle traffic during the day.
  - scoates 9y ago
    Can help avoid that with the even-newer t2.unlimited [1] instances, it seems. [1] https://aws.amazon.com/blogs/aws/new-t2-unlimited-going-beyond-the-burst-with-high-performance/ https://aws.amazon.com/blogs/aws/new-t2-unlimited-going-beyo...
- jlouis 9y ago
  Counter data point, my erlang nodes on t2 often reports lockups of more than 400ms. This is not acceptable when queries are handled in 10ms on a typical day. Erlang had built in monitoring when the underlying system scheduler makes a error
wdewind 9y ago
> So unless you have a webapp that uses 8gb of memory yet stays somewhere under 20% cpu utilization (maybe with some peaks), you'll eventually get throttled. FWIW this is, unfortunately, a pretty common description of many low traffic Rails apps.
dstillman 9y ago
t2 instances work great for many web services with daily load fluctuations if you view them as basically a billing mechanism. AWS originally offered a reserved-instance model where you didn't pay for instances that weren't running, but now that you pay for RI instance-hours regardless of usage, scaling down doesn't save you any money, so you either buy RIs to cover your peak usage (and waste money off-peak) or pay much higher on-demand prices some of the time. With t2's (which were introduced around the same time as the RI change), you can just run the right number of instances to keep your CPU Credit Balance above water (stockpiling credits off-peak) and buy RIs for them all, making them extremely cheap. And you can still auto-scale on the credit balance to avoid throttling in case of unexpected load.
- blasdel 9y ago
  I work on T2, and we just released a change that makes your CPUCreditBalance recover immediately instead of through the old complex 24h expiration model. There is now much less need to stockpile or manage your CPU credits: https://forums.aws.amazon.com/ann.jspa?annID=5196 https://forums.aws.amazon.com/ann.jspa?annID=5196 With T2 Standard auto-scaling based on the CPUCreditBalance can be a really bad idea, because the rate at which an account can launch T2s with an initial credit balance is limited. If your application has a bug that causes a health check to fail, the ASG can quickly burn through your launch credits by cycling the instances, and then it's possible for your health checks to keep failing because the later instances start with a CPUCreditBalance of zero. We just released T2 Unlimited in part to solve that. In this new version all instances start with a zero balance, but none are ever throttled and you can do a sustained burst immediately at launch: https://aws.amazon.com/blogs/aws/new-t2-unlimited-going-beyond-the-burst-with-high-performance/ https://aws.amazon.com/blogs/aws/new-t2-unlimited-going-beyo...
  - dkersten 9y ago
    I'm a bit confused by the wording on the T2 unlimited post: > T2 Unlimited instances have the ability to borrow an entire day’s worth of future credits, allowing them to perform additional bursting. What does this mean? Specifically, what is meant by "borrow"? Do they have to be paid back? Does the next day then have fewer credits?

by ryeguy 9y ago

eropple 9y ago

Most web apps are IO bound, not CPU bound, and a throttled T2 has IO to spare versus its CPU usage.

Johnny555 9y ago
t2's are great until something happens and you really need the full CPU, then you get throttled and suddenly your service goes down because it can't keep up with the load. If you use t2's for anything important, keep an eye on the CPU credit balance.
- cjsuk 9y ago
  Just applying selinux policy on CentOS 7 will kill the CPU credits on a micro instance. Running updates is a risky businesss.
- blasdel 9y ago
  I work on T2, and boy do I have a fix for you!: https://aws.amazon.com/blogs/aws/new-t2-unlimited-going-beyond-the-burst-with-high-performance/ https://aws.amazon.com/blogs/aws/new-t2-unlimited-going-beyo...
ryeguy 9y ago
But if you get throttled in the first place that means you're going to have some kind of performance degradation since your server was using more than the baseline level. Web apps being IO bound isn't relevant here, because the only requirement for issues to arise is for your server to have consistent 20%+ cpu usage.
- jrochkind1 9y ago
  Well, they're relevant in that a heavily IO bound app is probably unlikely to use much CPU -- it's too busy waiting on IO to use 90% of CPU, and maybe does stay mostly under 20%. Obviously this depends on a lot of details beyond "IO-bound", but it is not implausible, and I think does accurately describe many rails apps.

eknkc 9y ago

Just for a data point; We have been serving around 10 million web requests / day per t2.medium. Not lightweight static files or small json responses, actual dynamic pages with database load. Our servers mostly sit around 20% though. Might not be that much compared to larger web services but we have a localised audience so nighttime cpu credits get collected and used during peak times. So it fits our workload. They are great value when the credit system work in your setting. Have a grafana dashboard constantly monitoring credits and have alerts in place though. But haven’t had a sudden issue that we needed to manually remedy.

Xorlev 9y ago
> Have a grafana dashboard constantly monitoring credits and have alerts in place though. This, otherwise your infrastructure will come grinding to a halt at the worst time. It's a potential DoS vector in that way too, if you're relying on charging up credits overnight to handle traffic during the day.
- scoates 9y ago
  Can help avoid that with the even-newer t2.unlimited [1] instances, it seems. [1] https://aws.amazon.com/blogs/aws/new-t2-unlimited-going-beyond-the-burst-with-high-performance/ https://aws.amazon.com/blogs/aws/new-t2-unlimited-going-beyo...
jlouis 9y ago
Counter data point, my erlang nodes on t2 often reports lockups of more than 400ms. This is not acceptable when queries are handled in 10ms on a typical day. Erlang had built in monitoring when the underlying system scheduler makes a error

wdewind 9y ago

> So unless you have a webapp that uses 8gb of memory yet stays somewhere under 20% cpu utilization (maybe with some peaks), you'll eventually get throttled. FWIW this is, unfortunately, a pretty common description of many low traffic Rails apps.

dstillman 9y ago

t2 instances work great for many web services with daily load fluctuations if you view them as basically a billing mechanism. AWS originally offered a reserved-instance model where you didn't pay for instances that weren't running, but now that you pay for RI instance-hours regardless of usage, scaling down doesn't save you any money, so you either buy RIs to cover your peak usage (and waste money off-peak) or pay much higher on-demand prices some of the time. With t2's (which were introduced around the same time as the RI change), you can just run the right number of instances to keep your CPU Credit Balance above water (stockpiling credits off-peak) and buy RIs for them all, making them extremely cheap. And you can still auto-scale on the credit balance to avoid throttling in case of unexpected load.

blasdel 9y ago
I work on T2, and we just released a change that makes your CPUCreditBalance recover immediately instead of through the old complex 24h expiration model. There is now much less need to stockpile or manage your CPU credits: https://forums.aws.amazon.com/ann.jspa?annID=5196 https://forums.aws.amazon.com/ann.jspa?annID=5196 With T2 Standard auto-scaling based on the CPUCreditBalance can be a really bad idea, because the rate at which an account can launch T2s with an initial credit balance is limited. If your application has a bug that causes a health check to fail, the ASG can quickly burn through your launch credits by cycling the instances, and then it's possible for your health checks to keep failing because the later instances start with a CPUCreditBalance of zero. We just released T2 Unlimited in part to solve that. In this new version all instances start with a zero balance, but none are ever throttled and you can do a sustained burst immediately at launch: https://aws.amazon.com/blogs/aws/new-t2-unlimited-going-beyond-the-burst-with-high-performance/ https://aws.amazon.com/blogs/aws/new-t2-unlimited-going-beyo...
- dkersten 9y ago
  I'm a bit confused by the wording on the T2 unlimited post: > T2 Unlimited instances have the ability to borrow an entire day’s worth of future credits, allowing them to perform additional bursting. What does this mean? Specifically, what is meant by "borrow"? Do they have to be paid back? Does the next day then have fewer credits?