8 ms·

While I'm not going to complain about a price reduction, I'd honestly be more excited if S3 implemented support for additional headers and redirect rules. Right

While I'm not going to complain about a price reduction, I'd honestly be more excited if S3 implemented support for additional headers and redirect rules. Right now, anyone hosting a single page app (e.g. Angular/React) behind S3 and Cloudfront is going to get an F on securityheaders.io. And even worse, there is no way to prerender an SPA site for search engines without standing up an nginx proxy on ec2, which completely eliminates almost all of the benefits from Cloudfront. This is because right now S3 can only redirect based on a key prefix or error code, not based on a user agent like Googlebot or whatever. This means that even if you can technically drop a <meta name="fragment" content="!"> tag in your front end and then have S3 redirect on the key prefix '?_escaped_fragment_=', that will be a 301 redirect. This means that Google will ignore any <link rel="canonical" href="..."> tag on the prerendered page and will instead index https://api.yoursite.com https://api.yoursite.com or wherever your prerendered content is being hosted rather than your actual site. Not only is it a bunch of extra work to stand up an nginx proxy as a workaround, but it's also a whole extra set of security concerns, scaling concerns, etc. Not a good situation. edit: For more info on the prerendering issues, c.f.: https://github.com/prerender/prerender/issues/93 https://github.com/prerender/prerender/issues/93 https://gist.github.com/thoop/8165802 https://gist.github.com/thoop/8165802

ponyfleisch 10y ago
If i'm not wrong, cloudfront actually follows 302 redirects internally instead of passing the redirect on to the client. but since ?_escaped_fragment_= is a suffix, not a prefix, i don't think redirection rules help.
- Alex3917 10y ago
  Confusingly, _escaped_fragment_= is always a prefix if you have a #! in your url, otherwise I think it can come anywhere in the URL params.[1] Here is the exact system of redirects we were using, along with an explanation of why it didn't work: https://www.fwdeveryone.com/t/mdDouBesQwCv0za_o7GjCA/serving-prerendered-s3-sites https://www.fwdeveryone.com/t/mdDouBesQwCv0za_o7GjCA/serving... [1] https://prerender.io/js-seo/angularjs-seo-get-your-site-indexed-and-to-the-top-of-the-search-results/ https://prerender.io/js-seo/angularjs-seo-get-your-site-inde...
  - ponyfleisch 10y ago
    Ah, yes, if you use #!. In that case, can't you do a cloudfront custom behaviour on /?_escaped_fragment_=*? I havent't tried this.
tbrock 10y ago
Hmm, as someone who is about to move from using a webserver to hosting our single page app on S3 you may have just convinced me not to. What exactly are the benefits over simply setting up nginx if not simplicity? Yeah, it's great to just serve the asset from s3 but the complexity of what you just described negates it almost entirely.
- Alex3917 10y ago
  > What exactly are the benefits over simply setting up nginx if not simplicity? We weren't fully aware of these limitations when we decided to host our site on S3. Had we been, we may have just used nginx. There are obviously a ton of benefits to S3 and Cloudfront, it's just than in practice you can't really get them if you need Google to index your site. And while Google claims they can now execute javascript and include async content, in practice this isn't true for any real Angular or React site. And even if every search engine were to magically execute js correctly, you'd still need to prerender your site in order for Facebook and Twitter to populate the preview cards for your site with the proper headline, summary, and image.
  - scrollaway 10y ago
    You should keep in mind that S3 really is an object storage. It's named as such, advertised as such, priced as such and the limitations you're hitting are because it's built as such. It does work if you want to host a static site, and it's nice that they offer a bunch of extra niceties to help make those work... but expecting things like user-agent-specific redirects is a bit much for what's essentially a filesystem.
    - mgkimsal 10y ago
      Amen! Have been baffled by the urge to move stuff to S3 for app hosting. I understand the convenience (sort of) and the scalability aspects (mostly) but you seem to lose loads of functionality.
      - toomuchtodo 10y ago
eistrati 10y ago
> While I'm not going to complain about a price reduction, I'd honestly be more excited if S3 implemented support for additional headers and redirect rules. Right now, anyone hosting a single page app (e.g. Angular/React) behind S3 and Cloudfront is going to get an F on securityheaders.io. You can setup "Origin Custom Headers" in CloudFront ;)
- Alex3917 10y ago
  That's for sending headers to your web server or S3 though, not for sending headers to the user. There are a few extra headers you can send to the user, but not the security related ones.
jeffbarr 10y ago
Can you share your needs with me so I can pass them along to the S3 team?
- flyt 10y ago
  Tell the CloudFront team to support adding custom HTTP headers to client responses (not the currently supported headers added to the origin requests), such as HSTS. CloudFlare and others already support this.
- Alex3917 10y ago
  Thanks Jeff, I just sent you an email: https://www.fwdeveryone.com/t/QOU4DQDbS8e4tddfI22J0w/s3-prerendering-issue-hacker-news https://www.fwdeveryone.com/t/QOU4DQDbS8e4tddfI22J0w/s3-prer... Of course this won't yet show up in Google until we get that nginx proxy stood up or this feature gets implemented. :-)
  - scrollaway 10y ago
    OT: That fwd:everyone service is really cool. Damn :)
    - deleted 10y ago
      [deleted]
    - Alex3917 10y ago
      Thanks! Consider making an account, we're re-launching the site in the next couple weeks so there should be a lot of good content up there shortly.
      - 0xmohit 10y ago
        Looks really good. Will sign up shortly.
  - gnud 10y ago
    So, you're building a pre-rendering system, but only Google gets the benefit? Also, doesn't google penalize sites that serves different content to googlebot and other UAs?
flyt 10y ago
This is really something that needs to be setup/supported at the CloudFront layer (as CloudFlare and other CDNs already do) instead of S3. The danger is that somebody sets an HSTS or other security-related header on their bucket and breaks access for all other customers inadvertently for their customers that fetch from the S3 domain.
bobfunk 10y ago
Take a look at Netlify (disclaimer, I'm a co-founder), you'll get redirects, built-in prerendering, automated deployments, custom headers, etc. All with full cache-ability on our CDN. https://www.netlify.com https://www.netlify.com
- sorenbs 10y ago
  I can vouch for Netlify. We use them for all our stuff at Graphcool and it is awesome. Going from S3+CF to netlify provides a step improvement similar to going from EC2 instances to a managed docker cluster.

ponyfleisch 10y ago

If i'm not wrong, cloudfront actually follows 302 redirects internally instead of passing the redirect on to the client. but since ?_escaped_fragment_= is a suffix, not a prefix, i don't think redirection rules help.

Alex3917 10y ago
Confusingly, _escaped_fragment_= is always a prefix if you have a #! in your url, otherwise I think it can come anywhere in the URL params.[1] Here is the exact system of redirects we were using, along with an explanation of why it didn't work: https://www.fwdeveryone.com/t/mdDouBesQwCv0za_o7GjCA/serving-prerendered-s3-sites https://www.fwdeveryone.com/t/mdDouBesQwCv0za_o7GjCA/serving... [1] https://prerender.io/js-seo/angularjs-seo-get-your-site-indexed-and-to-the-top-of-the-search-results/ https://prerender.io/js-seo/angularjs-seo-get-your-site-inde...
- ponyfleisch 10y ago
  Ah, yes, if you use #!. In that case, can't you do a cloudfront custom behaviour on /?_escaped_fragment_=*? I havent't tried this.

tbrock 10y ago

Hmm, as someone who is about to move from using a webserver to hosting our single page app on S3 you may have just convinced me not to. What exactly are the benefits over simply setting up nginx if not simplicity? Yeah, it's great to just serve the asset from s3 but the complexity of what you just described negates it almost entirely.

Alex3917 10y ago
> What exactly are the benefits over simply setting up nginx if not simplicity? We weren't fully aware of these limitations when we decided to host our site on S3. Had we been, we may have just used nginx. There are obviously a ton of benefits to S3 and Cloudfront, it's just than in practice you can't really get them if you need Google to index your site. And while Google claims they can now execute javascript and include async content, in practice this isn't true for any real Angular or React site. And even if every search engine were to magically execute js correctly, you'd still need to prerender your site in order for Facebook and Twitter to populate the preview cards for your site with the proper headline, summary, and image.
- scrollaway 10y ago
  You should keep in mind that S3 really is an object storage. It's named as such, advertised as such, priced as such and the limitations you're hitting are because it's built as such. It does work if you want to host a static site, and it's nice that they offer a bunch of extra niceties to help make those work... but expecting things like user-agent-specific redirects is a bit much for what's essentially a filesystem.
  - mgkimsal 10y ago
    Amen! Have been baffled by the urge to move stuff to S3 for app hosting. I understand the convenience (sort of) and the scalability aspects (mostly) but you seem to lose loads of functionality.
    - toomuchtodo 10y ago

eistrati 10y ago

> While I'm not going to complain about a price reduction, I'd honestly be more excited if S3 implemented support for additional headers and redirect rules. Right now, anyone hosting a single page app (e.g. Angular/React) behind S3 and Cloudfront is going to get an F on securityheaders.io. You can setup "Origin Custom Headers" in CloudFront ;)

Alex3917 10y ago
That's for sending headers to your web server or S3 though, not for sending headers to the user. There are a few extra headers you can send to the user, but not the security related ones.

jeffbarr 10y ago

Can you share your needs with me so I can pass them along to the S3 team?

flyt 10y ago
Tell the CloudFront team to support adding custom HTTP headers to client responses (not the currently supported headers added to the origin requests), such as HSTS. CloudFlare and others already support this.
Alex3917 10y ago
Thanks Jeff, I just sent you an email: https://www.fwdeveryone.com/t/QOU4DQDbS8e4tddfI22J0w/s3-prerendering-issue-hacker-news https://www.fwdeveryone.com/t/QOU4DQDbS8e4tddfI22J0w/s3-prer... Of course this won't yet show up in Google until we get that nginx proxy stood up or this feature gets implemented. :-)
- scrollaway 10y ago
  OT: That fwd:everyone service is really cool. Damn :)
  - deleted 10y ago
    [deleted]
  - Alex3917 10y ago
    Thanks! Consider making an account, we're re-launching the site in the next couple weeks so there should be a lot of good content up there shortly.
    - 0xmohit 10y ago
      Looks really good. Will sign up shortly.
- gnud 10y ago
  So, you're building a pre-rendering system, but only Google gets the benefit? Also, doesn't google penalize sites that serves different content to googlebot and other UAs?

flyt 10y ago

This is really something that needs to be setup/supported at the CloudFront layer (as CloudFlare and other CDNs already do) instead of S3. The danger is that somebody sets an HSTS or other security-related header on their bucket and breaks access for all other customers inadvertently for their customers that fetch from the S3 domain.

bobfunk 10y ago

Take a look at Netlify (disclaimer, I'm a co-founder), you'll get redirects, built-in prerendering, automated deployments, custom headers, etc. All with full cache-ability on our CDN. https://www.netlify.com https://www.netlify.com

sorenbs 10y ago
I can vouch for Netlify. We use them for all our stuff at Graphcool and it is awesome. Going from S3+CF to netlify provides a step improvement similar to going from EC2 instances to a managed docker cluster.