7 ms·

A New Approach to Amazon EC2 Networking

125 points by jeffbarr 16y ago

jordw 16y ago
I work at Amazon (not on AWS). I must say, the frequency that new features are rolled out impresses even me. Congrats on shipping, guys.
- staunch 16y ago
  Completely agree. The AWS team is one of the very few examples of rapid iteration and improvement from a big company. I'm as interested in the AWS team as I am in any startup that exists today. I'd love to read about the tech challenges/team make up, etc. Is there any good coverage of this?
  - g123g 16y ago
    Yes, would definitely be interesting to find out how they maintain the quality and release so often. Would be a good data point to see if they use any of the agile processes and any tweaks they have done to make it work for them.
    - tybris 16y ago
      It's a bit old, but still relevant: http://www.fastcompany.com/magazine/85/bezos_2.html http://www.fastcompany.com/magazine/85/bezos_2.html If Bezos's personality is decidedly noncorporate, so are some of his ideas about how to run a large organization. One of Bezos's more memorable behind-the-scenes moments came during an off-site retreat, says Risher. "People were saying that groups needed to communicate more. Jeff got up and said, 'No, communication is terrible!' " The pronouncement shocked his managers. But Bezos pursued his idea of a decentralized, disentangled company where small groups can innovate and test their visions independently of everyone else. He came up with the notion of the "two-pizza team": If you can't feed a team with two pizzas, it's too large. That limits a task force to five to seven people, depending on their appetites.
- d_r 16y ago
  Seconded. I first tried Rackspace Cloud, who would send me frequent marketing-style e-mails with stock photos of intelligent-looking office employees, ask me to participate in raffles, and other nonsense that I would quickly filter. I much prefer seeing new feature announcements from AWS in my inbox! (And on HN.)
- kbatten 16y ago
  I use AWS both for work and for personal (love the free tier micro to play around with on my own time) and I am always impressed with how polished new features feel.
thematt 16y ago
At the rate they're innovating and growing, I wouldn't be surprised to see the Amazon Web Services operation grow to be larger than the rest of Amazon's business.
- asnyder 16y ago
  Jeff Bezos spoke about this in some depth at startup school in 2008. He even highlighted a journalist's quote suggesting that Amazon was sneaking web services through the back door and it would soon be larger than Amazon itself. I don't know the numbers offhand, but I wouldn't be surprised if they've already surpassed the storefront side of their business.
  - staunch 16y ago
    I believe estimates are that AWS revenues are somewhere in the lower hundreds of millions. Very far from rivaling their tens of billions retail business.
    - spitfire 16y ago
      What about the profits? Shipping, stocking and moving around physical books has got to be more expensive than marking up racks of computers and bandwidth.
      - staunch 16y ago
        They're probably following the same strategy they used in retail: undercut everyone and go for long-term market share domination. I'd bet they're still significantly in the red every year. Eventually they'll get big enough to start skimming a few points of profit off the top. Margins will always be low because that's a huge part of their strategy.
        fierarul 16y ago
        I keep hearing this but when you compare the same machine you get on Slicehost or linode with the Amazon prices, Amazon ends up being the most expensive. I use Amazon because of it's convenience and brand name, but I've always seen it as having a decent premium.
dcreemer 16y ago
any clues if broadcast / multicast will be allowed on my private subnets? The article makes no claims one way or another...
- wmf 16y ago
  That means it doesn't work.
- spahl 16y ago
  It's not supported: http://aws.amazon.com/vpc/faqs/#R4 http://aws.amazon.com/vpc/faqs/#R4
cemetric 16y ago
I just came to tell I love Amazon EC2, it's a treat to use.
rmoriz 16y ago
What about IPv6?
mleonhard 16y ago
I was hoping that this would finally be a way to have an ELB inside a firewall, but alas VPC doesn't support ELBs yet. AWS Elastic Beanstalk, Elastic Load Balancing, Amazon Elastic MapReduce, Amazon Relational Database Service (Amazon RDS) are not available for use in a VPC at this time. http://aws.amazon.com/vpc/#legal http://aws.amazon.com/vpc/#legal
- SriniK 16y ago
  Good point. They seem to target the last leg(enterprise) of cloud shift with this. Pretty awesome to see how they are churning features.
tzs 16y ago
Has anyone done credit card handling (input, submission to payment gateway, and storing for subscription billing and on-file orders) on EC2? A while back I recall Amazon saying that this was possible. We're looking into the possibility of moving to the cloud, and on first look our PCI guy saw some problems. We've just started experimenting so could easily have overlooked something, but these were the stumbling blocks we saw. It looks like these new features address 2 of these 3: • PCI requires limitations be based on outbound traffic from the cardholder environment. Amazon only allowed inbound filtering. Now they have outbound filtering, so this may be no longer problematic. • PCI requires internal machines to be placed on internal private networks using NAT. Amazon did not support NAT. Now they do, so this block may be gone. • PCI requires that all traffic be monitored with an IDS in the cardholder data environment. It doesn't appear possible to do a central monitoring machine with IDS in EC2.
- tybris 16y ago
  http://aws.amazon.com/security/pci-dss-level-1-compliance-faqs/ http://aws.amazon.com/security/pci-dss-level-1-compliance-fa...
  - tzs 16y ago
    Yeah, we've seen that. That's what I was thinking of when I said Amazon said it was possible. However, when it comes to actually doing that things seem a bit less clear. For instance, outbound filtering is a requirement, but Amazon just added that, so how did one satisfy that requirement before today?
    - smanek 16y ago
      would just running iptables on the instance suffice?
      - tzs 16y ago
        That could be what they had in mind. Our PCI guy tends to frown on that, though, because the purpose of restricting outbound access is so that if the machine does get compromised the bad guy can't ship off credit card information. If the outbound restriction is implemented on the machine, the bad guy might simply be able to turn them off. It's possible that we've got an overly strict PCI guy. Anyway, this particular issue appears dead now, as these new EC2 features add outbound filtering.
deleted 16y ago
[deleted]
lecha 16y ago
Any word on performance properties of various network topologies. What topology would provide absolute maximum network performance between instances?

125 points by jeffbarr 16y ago

jordw 16y ago

I work at Amazon (not on AWS). I must say, the frequency that new features are rolled out impresses even me. Congrats on shipping, guys.

staunch 16y ago
Completely agree. The AWS team is one of the very few examples of rapid iteration and improvement from a big company. I'm as interested in the AWS team as I am in any startup that exists today. I'd love to read about the tech challenges/team make up, etc. Is there any good coverage of this?
- g123g 16y ago
  Yes, would definitely be interesting to find out how they maintain the quality and release so often. Would be a good data point to see if they use any of the agile processes and any tweaks they have done to make it work for them.
  - tybris 16y ago
    It's a bit old, but still relevant: http://www.fastcompany.com/magazine/85/bezos_2.html http://www.fastcompany.com/magazine/85/bezos_2.html If Bezos's personality is decidedly noncorporate, so are some of his ideas about how to run a large organization. One of Bezos's more memorable behind-the-scenes moments came during an off-site retreat, says Risher. "People were saying that groups needed to communicate more. Jeff got up and said, 'No, communication is terrible!' " The pronouncement shocked his managers. But Bezos pursued his idea of a decentralized, disentangled company where small groups can innovate and test their visions independently of everyone else. He came up with the notion of the "two-pizza team": If you can't feed a team with two pizzas, it's too large. That limits a task force to five to seven people, depending on their appetites.
d_r 16y ago
Seconded. I first tried Rackspace Cloud, who would send me frequent marketing-style e-mails with stock photos of intelligent-looking office employees, ask me to participate in raffles, and other nonsense that I would quickly filter. I much prefer seeing new feature announcements from AWS in my inbox! (And on HN.)
kbatten 16y ago
I use AWS both for work and for personal (love the free tier micro to play around with on my own time) and I am always impressed with how polished new features feel.

thematt 16y ago

At the rate they're innovating and growing, I wouldn't be surprised to see the Amazon Web Services operation grow to be larger than the rest of Amazon's business.

asnyder 16y ago
Jeff Bezos spoke about this in some depth at startup school in 2008. He even highlighted a journalist's quote suggesting that Amazon was sneaking web services through the back door and it would soon be larger than Amazon itself. I don't know the numbers offhand, but I wouldn't be surprised if they've already surpassed the storefront side of their business.
- staunch 16y ago
  I believe estimates are that AWS revenues are somewhere in the lower hundreds of millions. Very far from rivaling their tens of billions retail business.
  - spitfire 16y ago
    What about the profits? Shipping, stocking and moving around physical books has got to be more expensive than marking up racks of computers and bandwidth.
    - staunch 16y ago
      They're probably following the same strategy they used in retail: undercut everyone and go for long-term market share domination. I'd bet they're still significantly in the red every year. Eventually they'll get big enough to start skimming a few points of profit off the top. Margins will always be low because that's a huge part of their strategy.
      - fierarul 16y ago
        I keep hearing this but when you compare the same machine you get on Slicehost or linode with the Amazon prices, Amazon ends up being the most expensive. I use Amazon because of it's convenience and brand name, but I've always seen it as having a decent premium.

dcreemer 16y ago

any clues if broadcast / multicast will be allowed on my private subnets? The article makes no claims one way or another...

wmf 16y ago
That means it doesn't work.
spahl 16y ago
It's not supported: http://aws.amazon.com/vpc/faqs/#R4 http://aws.amazon.com/vpc/faqs/#R4

cemetric 16y ago

I just came to tell I love Amazon EC2, it's a treat to use.

rmoriz 16y ago

What about IPv6?

mleonhard 16y ago

I was hoping that this would finally be a way to have an ELB inside a firewall, but alas VPC doesn't support ELBs yet. AWS Elastic Beanstalk, Elastic Load Balancing, Amazon Elastic MapReduce, Amazon Relational Database Service (Amazon RDS) are not available for use in a VPC at this time. http://aws.amazon.com/vpc/#legal http://aws.amazon.com/vpc/#legal

SriniK 16y ago
Good point. They seem to target the last leg(enterprise) of cloud shift with this. Pretty awesome to see how they are churning features.

tzs 16y ago

Has anyone done credit card handling (input, submission to payment gateway, and storing for subscription billing and on-file orders) on EC2? A while back I recall Amazon saying that this was possible. We're looking into the possibility of moving to the cloud, and on first look our PCI guy saw some problems. We've just started experimenting so could easily have overlooked something, but these were the stumbling blocks we saw. It looks like these new features address 2 of these 3: • PCI requires limitations be based on outbound traffic from the cardholder environment. Amazon only allowed inbound filtering. Now they have outbound filtering, so this may be no longer problematic. • PCI requires internal machines to be placed on internal private networks using NAT. Amazon did not support NAT. Now they do, so this block may be gone. • PCI requires that all traffic be monitored with an IDS in the cardholder data environment. It doesn't appear possible to do a central monitoring machine with IDS in EC2.

tybris 16y ago
http://aws.amazon.com/security/pci-dss-level-1-compliance-faqs/ http://aws.amazon.com/security/pci-dss-level-1-compliance-fa...
- tzs 16y ago
  Yeah, we've seen that. That's what I was thinking of when I said Amazon said it was possible. However, when it comes to actually doing that things seem a bit less clear. For instance, outbound filtering is a requirement, but Amazon just added that, so how did one satisfy that requirement before today?
  - smanek 16y ago
    would just running iptables on the instance suffice?
    - tzs 16y ago
      That could be what they had in mind. Our PCI guy tends to frown on that, though, because the purpose of restricting outbound access is so that if the machine does get compromised the bad guy can't ship off credit card information. If the outbound restriction is implemented on the machine, the bad guy might simply be able to turn them off. It's possible that we've got an overly strict PCI guy. Anyway, this particular issue appears dead now, as these new EC2 features add outbound filtering.

deleted 16y ago

[deleted]

lecha 16y ago

Any word on performance properties of various network topologies. What topology would provide absolute maximum network performance between instances?