12 ms·

Root Domain Website Hosting for Amazon S3

254 points by jeffbarr 14y ago

toomuchtodo 14y ago
Great work guys! Now can we work on functionality to prepay for AWS services? ;) https://forums.aws.amazon.com/thread.jspa?threadID=51931 https://forums.aws.amazon.com/thread.jspa?threadID=51931
- deleted 14y ago
  [deleted]
- 1qaz2wsx3edc 14y ago
  Considering this is off topic and 2 year old request, I'd say drop it. Also I see little reason for this functionality, what is a use case that cannot be worked around?
  - EwanToo 14y ago
    Pretty much all the posts on that thread are reasons for the functionality: * Transaction fees that people have to pay on their credit card * Budgeting purposes * Expense handling if you're buying the company services on a personal card (this happens a lot more than you'd expect) * Company shared accounts where the finance/purchasing department wants to do the old invoice / payment dance - finance department gets invoice for $5000, pays it in the background, developers get on with their work. * Applying a cap on your spending * Lack of credit cards in many parts of the world I'm sure there's dozens of reasons why you'd want to prepay, and it's clearly something Amazon want to support, judging by their staff's comments on the page.
  - toomuchtodo 14y ago
    You miss the point; AWS staff say its something they're working on. For two years. Either say you're not doing it or commit to doing it. Don't drag it out. Its a somewhat pathetic way to respond to customers.
- nikcub 14y ago
  Prepay and keeping user balances puts you in an entirely new compliance bracket that is much more complicated than charging a credit card and shipping a product. You get regulated almost like a bank. I'm not surprised that they aren't bothering with all that yet
  - JoshTriplett 14y ago
    Depends on how you treat it. If you treat it like buying a gift card, and either don't allow refunds or only refund to the same payment mechanism the user originally used, then you shouldn't trigger any of the "financial instrument" regulations. That just leaves you with any issues raised by your credit card processor, about whether you've "delivered" the product the user purchased.
  - toomuchtodo 14y ago
    Really? Because Twilio seems to handle it just fine. Charge card when balance falls below certain balance, and you can't get the funds back without a really good reason. You know, just like Amazon gift card balances applied to your consumer Amazon account.
mmastrac 14y ago
This is great news. Back when we were building DotSpots (now defunct, sadly), our website was built entirely in GWT. We weren't just serving a static site - we had the full functionality being run in GWT-compiled code. This meant that the majority of the site itself could be hosted from a static domain, and only the APIs behind the scenes needed to be running on our EC2 fleet. At least that was the idea. We managed to get the majority of the static images and JS running off CloudFront back then, but were always stuck serving html from our EC2 boxes (a combination of what was available from AWS at the time, and the previous one-day minimum timeout of CloudFront). We put a lot of work into optimizing them so that they'd be super lightweight and could be served quickly. It was pretty fast considering that we weren't able to put all of the static-like files into S3/CloudFront. Now that you can host the whole static bits of your domain through S3/CloudFront, I'd love to give this another shot with a future project. With strongly-named resources (ie: resource name == MD5) being served with one-year expiration times from Amazon's infrastructure, you could build a blazing fast dynamic site without having to spin up a single EC2 box. Exciting! (My original comment was on the dupe here... http://news.ycombinator.com/item?id=4976475 http://news.ycombinator.com/item?id=4976475)
- twistedpair 14y ago
  One of the many reasons I <3 GWT. Why run all the logic on the server if it can live in the client? Currently working on the same with www.runpartner.com from S3/EC2/CF.
jvoorhis 14y ago
This is great news. Now I'm looking forward to root domain support for CloudFront.
nodesocket 14y ago
Instead of using www.mydomain.com as the primary and 301 redirecting requests from mydomain.com, is it possible to do this the other way around? I want mydomain.com as the primary and 301 redirect www.mydomain.com.
- surrealize 14y ago
  Yeah, that's what this example does: http://docs.amazonwebservices.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html http://docs.amazonwebservices.com/AmazonS3/latest/dev/websit... "In this step, you will configure both buckets for website hosting. First, you will configure example.com as a website and then you'll configure www.example.com to redirect all requests to the example.com bucket."
- ceejayoz 14y ago
  I just set it up with non-www as the primary, works like a charm.
mdrussell 14y ago
So now all your root domains can suffer extended downtime on a monthly basis too? :)
- zwily 14y ago
  What are you talking about? The last big S3 outage was like 5 years ago.
- davidandgoliath 14y ago
  Troll elsewhere ;)
donretag 14y ago
Now only if Google App Engine would support the same. The amount of features that AWS release is astonishing given their size. Great work.
arikrak 14y ago
I have sites on Amazon S3 and OpenShift, but my Domain Name registar (1&1) didn't support redirection, so I pointed it to Cloudflare and created a rule there to redirect from the root to 'www'. This looks simpler though.
- roozbeh18 14y ago
  back when google apps for business was free, they had an option to redirect your naked domain to www under domains. then all you had to do was to point your A record to google dns and it would forward it to www.
yosun 14y ago
i guess amazon no longer needs the naturally free publicity from having 90% of CDN content URLs being of the format xxx.s3.amazonaws.com ... you can now have CDN URLs that look like xxx.yourdomain.com :O ... AMAZING!!! well, let's hope adding domain hosting to amazon doesn't add to further downtime.
- yosun 14y ago
  sorry for sarc over-simplifying the new offering. :(
  - ceejayoz 14y ago
    You're not over-simplifying, you're just wrong. S3 and CloudFront have supported custom subdomains for years.
- Inufu 14y ago
  You could always have that, but now you can use just "yourdomain.com" instead of a subdomain.
gregr 14y ago
Interesting. Root level cnames go against the RFC spec, yet it seems this solution is technically on par. Looking at the linked site, users mark the root as an A record and then select a new option in Route53 called 'Alias'. Since many DNS providers enforce actual A records, sounds like route53 will often be required for this feature (not a show stopper for most, just checking). FTA: "In the Amazon Route 53 Management Console, create two records for your domain. Create an A (alias) record in the domain's DNS hosted zone, mark it as an Alias, then choose the value that corresponds to your root domain name. Create a CNAME record and set the value to the S3 website endpoint for the first bucket."
- isb 14y ago
  Route 53 alias feature is RFC compliant - we will return an A record for an apex alias. Aliases can be used at non-apex too to avoid following CNAME chains. So, for e.g., the configuration in docs can be improved upon by having www.example be an alias to example.com instead of a CNAME.
saurik 14y ago
Why do people host static websites on S3 at all? It really isn't designed for that: it is an object store. Yes: it has a URL structure accessible in a way that makes it look like static hosting, and Amazon caved pretty early to people wanting to use it that way by adding features to make it more reasonable, but it doesn't fix the underlying problem. Specifically, it is both allowed to--and often does--return 50x errors to requests. The documentation for the S3 API states you should immediately retry; that's fine for an API where I can code that logic into my client library, but is simply an unacceptable solution on the web. Maybe there are one or two exceptions to this, but I have simply never seen a web browser retry these requests: the result is you just get broken images, broken stylesheets, or even entire broken pages. Back when Twitter used to serve user avatar pictures directly from S3 the issue was downright endemic (as you'd often load a page with 30 small images to request from S3, so every few pages you'd come across a dud). Sure, it only happens to some small percentage of requests, but for a popular website that can be a lot of people (and even for an unpopular one, every user counts), and it is orders of magnitude higher of a random error rate than I've experience with my own hosting on EC2; it is also irritating because it is random: when my own hosting fails, it fails in its entirely: I don't have some tiny fraction of requests for users all over the world failing. Regardless, I actually have an administrative need to stop hosting a specific x.com to www.x.com redirect on some non-AWS hosting I have (the DNS is hosted by Route53, etc., but I was left with a dinky HTTP server in Kentucky somewhere handling the 301), and I figured "well, if it doesn't have to actually request through to an underlying storage system, maybe I won't run into problems; I mean, how hard is it to take a URL and just immediately return a 301?", but after just a few minutes of playing with it I managed to get a test request that was supposed to return a 301 returning a 500 error instead. :( HTTP/1.1 500 Internal Server Error x-amz-request-id: 1A631406498520D6 x-amz-id-2: hXQ1YXyu0gxaiGITKvcB+P8+tgPsP3UITX/Or4emyjZtaL16ULAyHFx2ROT4QPXY Content-Type: text/html; charset=utf-8 Content-Length: 354 Date: Fri, 28 Dec 2012 07:19:24 GMT Connection: close Server: AmazonS3 This wasn't just a single-time problem either: I've setup a loop requesting random files (based on the timestamp of the test run and a sequence number from the test) off this pure-redirect bucket that I've left running for a few minutes, and some of the S3 nodes I'm talking to (72.21.194.13 being a great example) are just downright unreliable, often returning 500 errors in small clumps (that one node is giving me a 2% failure rate!!). S3 is simply not an appropriate mechanism for site hosting, and it is a shame that Amazon is encouraging people to misuse it in this fashion. (edit: Great, and now someone downvoted me: would you like more evidence that this is a problem?)
aneth4 14y ago
You can also use a service like dnsimple that offers more generalized aliases to any domain. As I understand it, you can only alias to AWS services with Route 53, but that may have changed. I've been using Dnsimple for quite a while to host root domains on heroku.
- donavanm 14y ago
  Sad face indeed. I wish route 53 had generic "alias" support like other providers. IIRC Route 53 can ALIAS to 1) your own hosted zone 2) an ELB instance 3) an S3 website bucket.
akulbe 14y ago
Given Amazon's recent outages (and the fact that they happen a lot more frequently)... hearing about this new service offering doesn't instill a lot of confidence.
- ceejayoz 14y ago
  Essentially all outages in the last few years have been EBS related. S3 doesn't have an EBS dependency, and has been pretty rock-solid for half a decade now.
magoon 14y ago
I host a web site for my small community on S3 because it's nearly free to do so and, for such a small site with no server-side programming, it performs better and is more fault tolerant than a cheap VPS.
weakwire 14y ago
Time for heroku to implement root domain redirection!
asc76 14y ago
AWS just keeps on getting better and better. Although, this particular release isn't that exciting and press-worthy.
api 14y ago
Combined with the ability to set headers to permit AJAX requests to another site, it would be possible to host a webapp on S3 and have the "live" portion be a JSON API only.
bradgessler 14y ago
Check out shart at https://github.com/polleverywhere/shart https://github.com/polleverywhere/shart if you're looking for an easy way to deploy static websites, like Middleman, to S3.
philip1209 14y ago
You can do similar hosting on the Rackspace/Akamai CDN: http://www.rackspace.com/blog/running-jekyll-on-rackspace-cloud-files/ http://www.rackspace.com/blog/running-jekyll-on-rackspace-cl... That's how the Obama fundraising website was hosted
- WestCoastJustin 14y ago
  As far as I know, Rackspace was not involved with the Obama fundraising website (other than this blog post which talks about Jekyll). Obama fundraising website was Jekyll/Akamai/AWS[EC2,S3,RDS] [1]. You can also watch a video from most of the Obama ops team about the stack on AWS [2]. [1] http://net.tutsplus.com/articles/interviews/chatting-with-obama-for-americas-director-of-frontend-development-daniel-ryan/ http://net.tutsplus.com/articles/interviews/chatting-with-ob... [2] https://www.youtube.com/watch?v=X1tJAT7ioEg https://www.youtube.com/watch?v=X1tJAT7ioEg
  - miles932 14y ago
    I can confirm that.
mleonhard 14y ago
If you want hosted root domain redirects without switching to Route 53, you can use https://www.RootRedirect.com/ https://www.RootRedirect.com/ . I run it on multiple EC2 instances and EIPs. Notable customers include Pizza Hut Australia and Airport Nürnberg. My 301 responses are cached on the client so subsequent redirects are instantaneous.
saurik 14y ago
These 301 redirects seem to be billed as "WebsiteGetObject,Requests-Tier2", which I believe is $0.01 per 10,000 (not including bandwidth, and with no decrease in price for large scale). I mean, its nice that its hosted in the cloud and all, but billing at the same rate as a much more complex S3 GET request (or "GetObject,Requests-Tier2") seems much too expensive (although I can imagine that if you have only a small number of requests; if you have any other server on which you can host the 301 redirect, you can do so with a marginal cost near 0, even at a load of many millions of redirects per day.

254 points by jeffbarr 14y ago

Great work guys! Now can we work on functionality to prepay for AWS services? ;) https://forums.aws.amazon.com/thread.jspa?threadID=51931 https://forums.aws.amazon.com/thread.jspa?threadID=51931

deleted 14y ago
[deleted]
1qaz2wsx3edc 14y ago
Considering this is off topic and 2 year old request, I'd say drop it. Also I see little reason for this functionality, what is a use case that cannot be worked around?
- EwanToo 14y ago
  Pretty much all the posts on that thread are reasons for the functionality: * Transaction fees that people have to pay on their credit card * Budgeting purposes * Expense handling if you're buying the company services on a personal card (this happens a lot more than you'd expect) * Company shared accounts where the finance/purchasing department wants to do the old invoice / payment dance - finance department gets invoice for $5000, pays it in the background, developers get on with their work. * Applying a cap on your spending * Lack of credit cards in many parts of the world I'm sure there's dozens of reasons why you'd want to prepay, and it's clearly something Amazon want to support, judging by their staff's comments on the page.
- toomuchtodo 14y ago
  You miss the point; AWS staff say its something they're working on. For two years. Either say you're not doing it or commit to doing it. Don't drag it out. Its a somewhat pathetic way to respond to customers.
nikcub 14y ago
Prepay and keeping user balances puts you in an entirely new compliance bracket that is much more complicated than charging a credit card and shipping a product. You get regulated almost like a bank. I'm not surprised that they aren't bothering with all that yet
- JoshTriplett 14y ago
  Depends on how you treat it. If you treat it like buying a gift card, and either don't allow refunds or only refund to the same payment mechanism the user originally used, then you shouldn't trigger any of the "financial instrument" regulations. That just leaves you with any issues raised by your credit card processor, about whether you've "delivered" the product the user purchased.
- toomuchtodo 14y ago
  Really? Because Twilio seems to handle it just fine. Charge card when balance falls below certain balance, and you can't get the funds back without a really good reason. You know, just like Amazon gift card balances applied to your consumer Amazon account.

mmastrac 14y ago

This is great news. Back when we were building DotSpots (now defunct, sadly), our website was built entirely in GWT. We weren't just serving a static site - we had the full functionality being run in GWT-compiled code. This meant that the majority of the site itself could be hosted from a static domain, and only the APIs behind the scenes needed to be running on our EC2 fleet. At least that was the idea. We managed to get the majority of the static images and JS running off CloudFront back then, but were always stuck serving html from our EC2 boxes (a combination of what was available from AWS at the time, and the previous one-day minimum timeout of CloudFront). We put a lot of work into optimizing them so that they'd be super lightweight and could be served quickly. It was pretty fast considering that we weren't able to put all of the static-like files into S3/CloudFront. Now that you can host the whole static bits of your domain through S3/CloudFront, I'd love to give this another shot with a future project. With strongly-named resources (ie: resource name == MD5) being served with one-year expiration times from Amazon's infrastructure, you could build a blazing fast dynamic site without having to spin up a single EC2 box. Exciting! (My original comment was on the dupe here... http://news.ycombinator.com/item?id=4976475 http://news.ycombinator.com/item?id=4976475)

twistedpair 14y ago
One of the many reasons I <3 GWT. Why run all the logic on the server if it can live in the client? Currently working on the same with www.runpartner.com from S3/EC2/CF.

jvoorhis 14y ago

This is great news. Now I'm looking forward to root domain support for CloudFront.

nodesocket 14y ago

Instead of using www.mydomain.com as the primary and 301 redirecting requests from mydomain.com, is it possible to do this the other way around? I want mydomain.com as the primary and 301 redirect www.mydomain.com.

surrealize 14y ago
Yeah, that's what this example does: http://docs.amazonwebservices.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html http://docs.amazonwebservices.com/AmazonS3/latest/dev/websit... "In this step, you will configure both buckets for website hosting. First, you will configure example.com as a website and then you'll configure www.example.com to redirect all requests to the example.com bucket."
ceejayoz 14y ago
I just set it up with non-www as the primary, works like a charm.

mdrussell 14y ago

So now all your root domains can suffer extended downtime on a monthly basis too? :)

zwily 14y ago
What are you talking about? The last big S3 outage was like 5 years ago.
davidandgoliath 14y ago
Troll elsewhere ;)

donretag 14y ago

Now only if Google App Engine would support the same. The amount of features that AWS release is astonishing given their size. Great work.

arikrak 14y ago

I have sites on Amazon S3 and OpenShift, but my Domain Name registar (1&1) didn't support redirection, so I pointed it to Cloudflare and created a rule there to redirect from the root to 'www'. This looks simpler though.

roozbeh18 14y ago
back when google apps for business was free, they had an option to redirect your naked domain to www under domains. then all you had to do was to point your A record to google dns and it would forward it to www.

yosun 14y ago

i guess amazon no longer needs the naturally free publicity from having 90% of CDN content URLs being of the format xxx.s3.amazonaws.com ... you can now have CDN URLs that look like xxx.yourdomain.com :O ... AMAZING!!! well, let's hope adding domain hosting to amazon doesn't add to further downtime.

yosun 14y ago
sorry for sarc over-simplifying the new offering. :(
- ceejayoz 14y ago
  You're not over-simplifying, you're just wrong. S3 and CloudFront have supported custom subdomains for years.
Inufu 14y ago
You could always have that, but now you can use just "yourdomain.com" instead of a subdomain.

gregr 14y ago

Interesting. Root level cnames go against the RFC spec, yet it seems this solution is technically on par. Looking at the linked site, users mark the root as an A record and then select a new option in Route53 called 'Alias'. Since many DNS providers enforce actual A records, sounds like route53 will often be required for this feature (not a show stopper for most, just checking). FTA: "In the Amazon Route 53 Management Console, create two records for your domain. Create an A (alias) record in the domain's DNS hosted zone, mark it as an Alias, then choose the value that corresponds to your root domain name. Create a CNAME record and set the value to the S3 website endpoint for the first bucket."

isb 14y ago
Route 53 alias feature is RFC compliant - we will return an A record for an apex alias. Aliases can be used at non-apex too to avoid following CNAME chains. So, for e.g., the configuration in docs can be improved upon by having www.example be an alias to example.com instead of a CNAME.

saurik 14y ago

Why do people host static websites on S3 at all? It really isn't designed for that: it is an object store. Yes: it has a URL structure accessible in a way that makes it look like static hosting, and Amazon caved pretty early to people wanting to use it that way by adding features to make it more reasonable, but it doesn't fix the underlying problem. Specifically, it is both allowed to--and often does--return 50x errors to requests. The documentation for the S3 API states you should immediately retry; that's fine for an API where I can code that logic into my client library, but is simply an unacceptable solution on the web. Maybe there are one or two exceptions to this, but I have simply never seen a web browser retry these requests: the result is you just get broken images, broken stylesheets, or even entire broken pages. Back when Twitter used to serve user avatar pictures directly from S3 the issue was downright endemic (as you'd often load a page with 30 small images to request from S3, so every few pages you'd come across a dud). Sure, it only happens to some small percentage of requests, but for a popular website that can be a lot of people (and even for an unpopular one, every user counts), and it is orders of magnitude higher of a random error rate than I've experience with my own hosting on EC2; it is also irritating because it is random: when my own hosting fails, it fails in its entirely: I don't have some tiny fraction of requests for users all over the world failing. Regardless, I actually have an administrative need to stop hosting a specific x.com to www.x.com redirect on some non-AWS hosting I have (the DNS is hosted by Route53, etc., but I was left with a dinky HTTP server in Kentucky somewhere handling the 301), and I figured "well, if it doesn't have to actually request through to an underlying storage system, maybe I won't run into problems; I mean, how hard is it to take a URL and just immediately return a 301?", but after just a few minutes of playing with it I managed to get a test request that was supposed to return a 301 returning a 500 error instead. :( HTTP/1.1 500 Internal Server Error x-amz-request-id: 1A631406498520D6 x-amz-id-2: hXQ1YXyu0gxaiGITKvcB+P8+tgPsP3UITX/Or4emyjZtaL16ULAyHFx2ROT4QPXY Content-Type: text/html; charset=utf-8 Content-Length: 354 Date: Fri, 28 Dec 2012 07:19:24 GMT Connection: close Server: AmazonS3 This wasn't just a single-time problem either: I've setup a loop requesting random files (based on the timestamp of the test run and a sequence number from the test) off this pure-redirect bucket that I've left running for a few minutes, and some of the S3 nodes I'm talking to (72.21.194.13 being a great example) are just downright unreliable, often returning 500 errors in small clumps (that one node is giving me a 2% failure rate!!). S3 is simply not an appropriate mechanism for site hosting, and it is a shame that Amazon is encouraging people to misuse it in this fashion. (edit: Great, and now someone downvoted me: would you like more evidence that this is a problem?)

aneth4 14y ago

You can also use a service like dnsimple that offers more generalized aliases to any domain. As I understand it, you can only alias to AWS services with Route 53, but that may have changed. I've been using Dnsimple for quite a while to host root domains on heroku.

donavanm 14y ago
Sad face indeed. I wish route 53 had generic "alias" support like other providers. IIRC Route 53 can ALIAS to 1) your own hosted zone 2) an ELB instance 3) an S3 website bucket.

akulbe 14y ago

Given Amazon's recent outages (and the fact that they happen a lot more frequently)... hearing about this new service offering doesn't instill a lot of confidence.

ceejayoz 14y ago
Essentially all outages in the last few years have been EBS related. S3 doesn't have an EBS dependency, and has been pretty rock-solid for half a decade now.

magoon 14y ago

I host a web site for my small community on S3 because it's nearly free to do so and, for such a small site with no server-side programming, it performs better and is more fault tolerant than a cheap VPS.

weakwire 14y ago

Time for heroku to implement root domain redirection!

asc76 14y ago

AWS just keeps on getting better and better. Although, this particular release isn't that exciting and press-worthy.

api 14y ago

Combined with the ability to set headers to permit AJAX requests to another site, it would be possible to host a webapp on S3 and have the "live" portion be a JSON API only.

bradgessler 14y ago

Check out shart at https://github.com/polleverywhere/shart https://github.com/polleverywhere/shart if you're looking for an easy way to deploy static websites, like Middleman, to S3.

philip1209 14y ago

You can do similar hosting on the Rackspace/Akamai CDN: http://www.rackspace.com/blog/running-jekyll-on-rackspace-cloud-files/ http://www.rackspace.com/blog/running-jekyll-on-rackspace-cl... That's how the Obama fundraising website was hosted

WestCoastJustin 14y ago
As far as I know, Rackspace was not involved with the Obama fundraising website (other than this blog post which talks about Jekyll). Obama fundraising website was Jekyll/Akamai/AWS[EC2,S3,RDS] [1]. You can also watch a video from most of the Obama ops team about the stack on AWS [2]. [1] http://net.tutsplus.com/articles/interviews/chatting-with-obama-for-americas-director-of-frontend-development-daniel-ryan/ http://net.tutsplus.com/articles/interviews/chatting-with-ob... [2] https://www.youtube.com/watch?v=X1tJAT7ioEg https://www.youtube.com/watch?v=X1tJAT7ioEg
- miles932 14y ago
  I can confirm that.

mleonhard 14y ago

If you want hosted root domain redirects without switching to Route 53, you can use https://www.RootRedirect.com/ https://www.RootRedirect.com/ . I run it on multiple EC2 instances and EIPs. Notable customers include Pizza Hut Australia and Airport Nürnberg. My 301 responses are cached on the client so subsequent redirects are instantaneous.

saurik 14y ago

These 301 redirects seem to be billed as "WebsiteGetObject,Requests-Tier2", which I believe is $0.01 per 10,000 (not including bandwidth, and with no decrease in price for large scale). I mean, its nice that its hosted in the cloud and all, but billing at the same rate as a much more complex S3 GET request (or "GetObject,Requests-Tier2") seems much too expensive (although I can imagine that if you have only a small number of requests; if you have any other server on which you can host the 301 redirect, you can do so with a marginal cost near 0, even at a load of many millions of redirects per day.