5 ms·

Not trying to be mean but, why wait? If you're stuck on AWS it's not impossible (though difficult and annoying certainly) to get a decently-performant Postgres

Not trying to be mean but, why wait? If you're stuck on AWS it's not impossible (though difficult and annoying certainly) to get a decently-performant Postgres instance going. The experience setting it up will lessen the vendor-lock that Amazon has on you (convenience always has a price).

dilap 12y ago
Outside observer: Man, I figure setting up a "decently performing" Postgres instance on AWS would be such a common thing to do that it would either be known to be impossible, or have cookie-cutter instructions, if not a script. How is it that it's "difficult and annoying" still? Not trolling, genuinely curious :)
- mbell 12y ago
  It isn't "difficult and annoying" anymore, it used to be before SSD backed EBS and/or provisioned IOPS because you had to RAID0 together a dozen or so magnetic EBS volumes to get decent disk performance and then deal with the annoyance of sorting out a way to take consistent snapshots of the RAID array for backups. Now you can just toss a single 1TB SSD backed EBS volume on an instance and get ~3k iops, or use provisioned iops to get almost any performance level you need.
  - joevandyk 12y ago
    That performs decently now? I got scared off using EBS a few years ago and use only the ephemeral storage + failovers.
    - rpedela 12y ago
      The SSD EBS claims a default performance of 3 IOPS/GB with a burst of up to 3000 IOPS with a 99% consistency. One of my larger instances uses SSD EBS and so far I am happy with it. In particular, I was very impressed how much faster updating Ubuntu was when I first booted the instance. I am planning to move my Postgres RDS instance to SSD EBS next week. I think (I hope) it will fix a particular stall my application experiences sometimes.
    - bkeroack 12y ago
      Regardless of whether they're backed by SSD, all EBS volumes on an instance sit behind a 1 Gbps pipe (except for the more exotic and expensive instance types). That's part of the reason why Amazon talks about IOPs instead of raw disk bandwidth. Go ahead and run: $ sudo du -hs /* ..on a vanilla m3.* instance and run iotop in a different session. You'll see bandwidth numbers that 2002 would be embarrassed about.
      - correctdarecord 12y ago
        Bandwidth available for EBS volumes varies. For instance, with EBS-optimized volumes it can be 500Mbps, 1Gbps, 2Gbps, or 10Gbps, depending on the instance type, as shown in this chart from an Amazon presentation: http://cl.ly/image/440b142t0T1x/Screen%20Shot%202014-12-18%20at%2013.16.23.png http://cl.ly/image/440b142t0T1x/Screen%20Shot%202014-12-18%2... The "NA" of the 8xlarge instance types is because there's no ebs-optimization as an optional feature on that instance type; you automatically get access to 10Gbps. Here's a video of the whole talk: https://www.youtube.com/watch?v=3OH4-Hx3tlE https://www.youtube.com/watch?v=3OH4-Hx3tlE Here's the slides: http://www.slideshare.net/AmazonWebServices/stg302-28617072 http://www.slideshare.net/AmazonWebServices/stg302-28617072
- gtaylor 12y ago
  It's not just the setup. For smaller companies with tiny ops teams (or a developer that also serves as an ops guy), RDS can save a bunch of time in maintenance/security/upgrade land. You are going to get decent to good performance for most general cases, and you can easily create failovers and scale up/down without much of any Postgres knowledge. Especially when your product is in "get traction and survive" mode, it's nice to be able to worry more about your app and less about things like running a Postgres server or cluster. Yes, you'll wait a bit longer for new versions to come out, but that may or may not be a big deal. It isn't for us. We have the in-house knowledge to run our own cluster, but RDS does such a good job that we don't need to.
  - dahdum 12y ago
    Same here, have the skills to run it but not the motivation - rather spend all the time on product dev. Our one exception was PLV8 support, and RDS added that a couple months ago.
gfodor 12y ago
The answer is simple, as with any AWS service, a whole slew of concerns (but not all of them) become Not My Problem. I have enough problems.
- bkeroack 12y ago
  ...in exchange for a number of different problems including (in no particular order): severe vendor lock-in, mediocre to acceptable performance, and relatively high cost per unit of performance.
  - gfodor 12y ago
    Yup, and for startups this tradeoff is usually a no-brainer.
  - personZ 12y ago
    I don't personally get the RDS value proposition, but how does it lead to vendor lock-in, much less severe vendor lock-in? I don't believe there is really any lock-in whatsoever -- backup your database and move it wherever you want, whether you're running the pgsql, Mysql, or SQL Server variants of RDS.
    - bkeroack 12y ago
      While of course you're correct, it's a bit like saying "you can just pack up your code and run it somewhere else". If the entire architecture and workflow are based around proprietary AWS tools (Elastic Beanstalk, Elastic load balancers, RDS, Cloud Formation, autoscaling, etc) even if your code is technically portable, you're still suffering from vendor lock-in. Arguably worse than the bad old days of the Microsoft monopoly.
      - Xorlev 12y ago
        RDS is just a database though. All the standard tools work with it. Unless you start dipping into more exotic services, it's pretty hard to become locked in. EC2+ELB+RDS are fairly independent. Even autoscaling doesn't lock you in. That said, CloudFormation, SQS, Kinesis, DynamoDB, etc. are all very sticky services.

dilap 12y ago

Outside observer: Man, I figure setting up a "decently performing" Postgres instance on AWS would be such a common thing to do that it would either be known to be impossible, or have cookie-cutter instructions, if not a script. How is it that it's "difficult and annoying" still? Not trolling, genuinely curious :)

mbell 12y ago
It isn't "difficult and annoying" anymore, it used to be before SSD backed EBS and/or provisioned IOPS because you had to RAID0 together a dozen or so magnetic EBS volumes to get decent disk performance and then deal with the annoyance of sorting out a way to take consistent snapshots of the RAID array for backups. Now you can just toss a single 1TB SSD backed EBS volume on an instance and get ~3k iops, or use provisioned iops to get almost any performance level you need.
- joevandyk 12y ago
  That performs decently now? I got scared off using EBS a few years ago and use only the ephemeral storage + failovers.
  - rpedela 12y ago
    The SSD EBS claims a default performance of 3 IOPS/GB with a burst of up to 3000 IOPS with a 99% consistency. One of my larger instances uses SSD EBS and so far I am happy with it. In particular, I was very impressed how much faster updating Ubuntu was when I first booted the instance. I am planning to move my Postgres RDS instance to SSD EBS next week. I think (I hope) it will fix a particular stall my application experiences sometimes.
  - bkeroack 12y ago
    Regardless of whether they're backed by SSD, all EBS volumes on an instance sit behind a 1 Gbps pipe (except for the more exotic and expensive instance types). That's part of the reason why Amazon talks about IOPs instead of raw disk bandwidth. Go ahead and run: $ sudo du -hs /* ..on a vanilla m3.* instance and run iotop in a different session. You'll see bandwidth numbers that 2002 would be embarrassed about.
    - correctdarecord 12y ago
      Bandwidth available for EBS volumes varies. For instance, with EBS-optimized volumes it can be 500Mbps, 1Gbps, 2Gbps, or 10Gbps, depending on the instance type, as shown in this chart from an Amazon presentation: http://cl.ly/image/440b142t0T1x/Screen%20Shot%202014-12-18%20at%2013.16.23.png http://cl.ly/image/440b142t0T1x/Screen%20Shot%202014-12-18%2... The "NA" of the 8xlarge instance types is because there's no ebs-optimization as an optional feature on that instance type; you automatically get access to 10Gbps. Here's a video of the whole talk: https://www.youtube.com/watch?v=3OH4-Hx3tlE https://www.youtube.com/watch?v=3OH4-Hx3tlE Here's the slides: http://www.slideshare.net/AmazonWebServices/stg302-28617072 http://www.slideshare.net/AmazonWebServices/stg302-28617072
gtaylor 12y ago
It's not just the setup. For smaller companies with tiny ops teams (or a developer that also serves as an ops guy), RDS can save a bunch of time in maintenance/security/upgrade land. You are going to get decent to good performance for most general cases, and you can easily create failovers and scale up/down without much of any Postgres knowledge. Especially when your product is in "get traction and survive" mode, it's nice to be able to worry more about your app and less about things like running a Postgres server or cluster. Yes, you'll wait a bit longer for new versions to come out, but that may or may not be a big deal. It isn't for us. We have the in-house knowledge to run our own cluster, but RDS does such a good job that we don't need to.
- dahdum 12y ago
  Same here, have the skills to run it but not the motivation - rather spend all the time on product dev. Our one exception was PLV8 support, and RDS added that a couple months ago.

gfodor 12y ago

The answer is simple, as with any AWS service, a whole slew of concerns (but not all of them) become Not My Problem. I have enough problems.

bkeroack 12y ago
...in exchange for a number of different problems including (in no particular order): severe vendor lock-in, mediocre to acceptable performance, and relatively high cost per unit of performance.
- gfodor 12y ago
  Yup, and for startups this tradeoff is usually a no-brainer.
- personZ 12y ago
  I don't personally get the RDS value proposition, but how does it lead to vendor lock-in, much less severe vendor lock-in? I don't believe there is really any lock-in whatsoever -- backup your database and move it wherever you want, whether you're running the pgsql, Mysql, or SQL Server variants of RDS.
  - bkeroack 12y ago
    While of course you're correct, it's a bit like saying "you can just pack up your code and run it somewhere else". If the entire architecture and workflow are based around proprietary AWS tools (Elastic Beanstalk, Elastic load balancers, RDS, Cloud Formation, autoscaling, etc) even if your code is technically portable, you're still suffering from vendor lock-in. Arguably worse than the bad old days of the Microsoft monopoly.
    - Xorlev 12y ago
      RDS is just a database though. All the standard tools work with it. Unless you start dipping into more exotic services, it's pretty hard to become locked in. EC2+ELB+RDS are fairly independent. Even autoscaling doesn't lock you in. That said, CloudFormation, SQS, Kinesis, DynamoDB, etc. are all very sticky services.