6 ms·

Since this thread will likely turn into people asking about Cloudfront performance, does anyone have any real-world experience with CloudFront vs. Rackspace wit

by jcastro 13y ago

Since this thread will likely turn into people asking about Cloudfront performance, does anyone have any real-world experience with CloudFront vs. Rackspace with Akamai CDN? On paper the Rackspace one looks like a great performance/price alternative.

jcampbell1 13y ago
I am in the process of abandoning Cloudfront, because they have a serious bug when serving video files. They serve HTTP 206 (Range-Get) as HTTP 1.0 but 206 didn't exist in HTTP 1.0. Chrome and Firefox treat this as "uncacheable", thus media assets bypass the local media cache. Depending on the nature of the content, Cloudfront is not usable for video, particularly the kind where people seek around a bunch, like instructional videos and tutorials. Also, people with slow connections expect the video to buffer while paused. This doesn't happen if you serve the videos from cloudfront.
- deleted 13y ago
  [deleted]
- v13inc 13y ago
  Why would Cloudfront cause the player to stop buffering the video? Does it happen with a HTTP Download distribution, or the RTMP streaming distribution? I've used both CloudFront and Akamai for serving video, and I don't remember running into those issues. I have only tested it with normal HTTP Download distributions, however.
  - jcampbell1 13y ago
    Cloudfront serves an invalid HTTP response, thus Chrome/Firefox refuse to cache the data. The theory is, it is better to not risk caching bad data. Chrome will still play the video served from Cloudfront, but it wont write any of the data to the media cache. Thus when you pause and buffer, it will only buffer a few seconds of video in the current playback window. It never writes to the cache so buffering must stop. To see this in action, play a video with chrome://media-internals/ open. A video served from S3 will get saved to the media cache, and the full video can buffer when paused. A video served from cloudfront will only buffer a few seconds of video. Update: It is probably just easier to see for yourself: Here is a video served from S3. Pause the video in chrome, and you will see the whole video gets buffered: http://s3.amazonaws.com/jctest20080526/test/test.mp4 http://s3.amazonaws.com/jctest20080526/test/test.mp4 Here is the same video served from Cloudfront. Start the video, and pause it, and notice how it doesn't buffer more than a few seconds: http://d3oocspv43fsel.cloudfront.net/test/test.mp4 http://d3oocspv43fsel.cloudfront.net/test/test.mp4
    - v13inc 13y ago
      Thanks for the info, thats very informative. Can you recommend any CDNs that work well with online video? It would be great to find one that is as easy to configure as cloudfront that doesn't have this issue. I'm not a big fan of Akamai's control panel -- setting up new distributions is way to complicated (albeit more flexible), and configuration changes were taking almost a day to propogate.
      - jcampbell1 13y ago
        I haven't had the time to find another vendor, thus can't recommend one. Amazon promised a fix, but then back tracked. Their explanation was something along the lines of "an old version of Squid has the same problem, thus it is okay". I cannot comprehend the logic system they employ to think that is a good reason to live with the bug.
    - micro-ram 13y ago
      Both buffer while paused on Chrome iOS.
      - jcampbell1 13y ago
        Chrome on iOS is probably using Apple's AVPlayer, which makes sense because there is probably no other way to access the dedicated decoding hardware. Apple is not afraid to Cache an HTTP 1.0 206 response.
        nkoren 13y ago
        Both are buffering fine for me on Chrome for Android, too.
      - mikeash 13y ago
        Note that Chrome on iOS is basically unrelated to every other version of Chrome. Due to Apple's silly rules, it's essentially just a reskinned Safari.
    - StavrosK 13y ago
- modemuser 13y ago
  > HTTP 1.0 If that is the only problem, flipping one bit in every response seems like a really simple solution. Why hasn't Cloudfront fixed it yet, do they know about this?
  - jcampbell1 13y ago
    Yes they know about it. Below is the response from Amazon. The logic they employ is that since it is broken in an old version of Squid, it is fine for it to be broken on Cloudfront. https://forums.aws.amazon.com/thread.jspa?messageID=351384&#351384 https://forums.aws.amazon.com/thread.jspa?messageID=351384&#... - - - Hello, While we are aware of the issue with range request HTTP/1.0 206 responses and Chrome, we cannot provide an ETA for a fix. Since this issue is specific to range requests, an immediate workaround is to disable range requests on your origin server if this is possible for your use case. It is also worth mentioning that multiple web proxy and cache application vendors have using HTTP/1.0 as a de facto standard for many years, so you will probably sporadically get similar reports from your end users using Chrome, but not other browsers such as Firefox or Safari. For example, here is a discussion between a Chrome developer on the mailing list for the popular Squid web cache about a similar report: http://www.squid-cache.org/mail-archive/squid-dev/201204/0113.html http://www.squid-cache.org/mail-archive/squid-dev/201204/011... I am not saying that always returning HTTP/1.0 will stick around forever, but it is fairly common in real world situations today.
- jefe78 13y ago
  You're doing something horribly wrong. I work for a live streaming company and we make extensive use of Varnish. It can probably solve the problem you're describing.
  - jcampbell1 13y ago
    I'm not doing a damn thing wrong other than using Cloudfront. The problem is on their end, not mine. Thinking Varnish could solve this problem is utterly confused. Do you know what CDN does? CDNs have servers located around the world so files are loaded quickly and with low latency. Furthermore, your profile suggest you work for a pump and dump penny stock company (basically a scam). If your employer is paying you in something other than cash, you need to walk away asap.
    - deleted 13y ago
      [deleted]
      - jcampbell1 13y ago
        I am sincerely not being spiteful. I am trying to warn you. Your company's unaudited financials state that there is only $7,000 in the bank. http://www.sec.gov/Archives/edgar/data/1499274/000109635013000034/xumanii_draft01311310q11.htm http://www.sec.gov/Archives/edgar/data/1499274/0001096350130... I sign my own paycheck, so the notion of whether he is overpaying is a bit confusing.
        jefe78 13y ago
        I'm entirely aware of the financial situation. How is calling my employer a pump and dump scam NOT spiteful when I've worked on this project from the beginning? Edit: Also, I don't appreciate you posting that. It's completely off-topic. Keep it classy.
        jcampbell1 13y ago
        At this point, the article has left hackernews. I am writing to you as fellow hacker looking to help you out. You are involved in a stock fraud. The company you work for is a sham. If you live in the US, then you have a plausible defense that you have no understanding of the underlying business. In this case, you likely can't afford the lawyer to present this case. If you don't live in the US, then be careful. Imagine, ten years down the road, you are a successful engineer, and want to take your family to Disney world. Unfortunately, there is an outstanding bench warrant for your arrest, and rather than a nice family vacation that your wife wanted, you end up in a US prison.
- photorized 13y ago
  If you haven't found a good alternative yet, we'll be happy to help. /co-founder at Advection.NET
  - jcampbell1 13y ago
    I'll have a look. Here is what I need: CDN that support Range Requests correctly. I want to use S3 as the origin. I need an edge location in Australia. Also, there can be no 301/302 HTTP redirects, thus I have eliminated Google as a potential offering. My current spend is around $1000 a month, so it is a tiny account.
    - photorized 13y ago
      Don't have an edge location in Australia currently, but our Hong Kong POP could work with reasonable latency - we can test that. Let's discuss offline. malatortsev at advection dot net
currysausage 13y ago
I don't know about Cloudfront vs Rackspace+Akamai, however, based on some quick tests I did a year ago, I seem to recall that Rackspace+Akamai didn't deliver the same performance as professional Akamai services. In my experiments, Rackspace+Akamai used different servers, often located farther away, than big Akamai sites. (If anybody has background information about this, I would be very interested to hear.)
- noelwelsh 13y ago
  Rackspace CDN says this: "Rackspace uses 213 of Akamai's edge locations, selected especially for our customers' typical usage patterns, and designed to cover all major areas of the globe." http://www.rackspace.com/cloud/files/ http://www.rackspace.com/cloud/files/ Akamai has approximately 5 gazillion edge locations, so yes, it is a cut down version. It is still a lot more POPs than just about every other CDN, though this doesn't necessarily translate into performance.
  - currysausage 13y ago
    Ah, thank you. That info wasn't there when I conducted the tests: http://web.archive.org/web/20120915095343/http://www.rackspace.com/cloud/public/files/ http://web.archive.org/web/20120915095343/http://www.rackspa...? "though this doesn't necessarily translate into performance" - Yes, even when the files were already cached, I wasn't always satisfied with the performance. A detail that bugged me, by the way, was the high number of CNAME requests, although this should at worst affect the first view.

by jcastro 13y ago

jcampbell1 13y ago

I am in the process of abandoning Cloudfront, because they have a serious bug when serving video files. They serve HTTP 206 (Range-Get) as HTTP 1.0 but 206 didn't exist in HTTP 1.0. Chrome and Firefox treat this as "uncacheable", thus media assets bypass the local media cache. Depending on the nature of the content, Cloudfront is not usable for video, particularly the kind where people seek around a bunch, like instructional videos and tutorials. Also, people with slow connections expect the video to buffer while paused. This doesn't happen if you serve the videos from cloudfront.

deleted 13y ago
[deleted]
v13inc 13y ago
Why would Cloudfront cause the player to stop buffering the video? Does it happen with a HTTP Download distribution, or the RTMP streaming distribution? I've used both CloudFront and Akamai for serving video, and I don't remember running into those issues. I have only tested it with normal HTTP Download distributions, however.
- jcampbell1 13y ago
  Cloudfront serves an invalid HTTP response, thus Chrome/Firefox refuse to cache the data. The theory is, it is better to not risk caching bad data. Chrome will still play the video served from Cloudfront, but it wont write any of the data to the media cache. Thus when you pause and buffer, it will only buffer a few seconds of video in the current playback window. It never writes to the cache so buffering must stop. To see this in action, play a video with chrome://media-internals/ open. A video served from S3 will get saved to the media cache, and the full video can buffer when paused. A video served from cloudfront will only buffer a few seconds of video. Update: It is probably just easier to see for yourself: Here is a video served from S3. Pause the video in chrome, and you will see the whole video gets buffered: http://s3.amazonaws.com/jctest20080526/test/test.mp4 http://s3.amazonaws.com/jctest20080526/test/test.mp4 Here is the same video served from Cloudfront. Start the video, and pause it, and notice how it doesn't buffer more than a few seconds: http://d3oocspv43fsel.cloudfront.net/test/test.mp4 http://d3oocspv43fsel.cloudfront.net/test/test.mp4
  - v13inc 13y ago
    Thanks for the info, thats very informative. Can you recommend any CDNs that work well with online video? It would be great to find one that is as easy to configure as cloudfront that doesn't have this issue. I'm not a big fan of Akamai's control panel -- setting up new distributions is way to complicated (albeit more flexible), and configuration changes were taking almost a day to propogate.
    - jcampbell1 13y ago
      I haven't had the time to find another vendor, thus can't recommend one. Amazon promised a fix, but then back tracked. Their explanation was something along the lines of "an old version of Squid has the same problem, thus it is okay". I cannot comprehend the logic system they employ to think that is a good reason to live with the bug.
  - micro-ram 13y ago
    Both buffer while paused on Chrome iOS.
    - jcampbell1 13y ago
      Chrome on iOS is probably using Apple's AVPlayer, which makes sense because there is probably no other way to access the dedicated decoding hardware. Apple is not afraid to Cache an HTTP 1.0 206 response.
      - nkoren 13y ago
        Both are buffering fine for me on Chrome for Android, too.
    - mikeash 13y ago
      Note that Chrome on iOS is basically unrelated to every other version of Chrome. Due to Apple's silly rules, it's essentially just a reskinned Safari.
  - StavrosK 13y ago
modemuser 13y ago
> HTTP 1.0 If that is the only problem, flipping one bit in every response seems like a really simple solution. Why hasn't Cloudfront fixed it yet, do they know about this?
- jcampbell1 13y ago
  Yes they know about it. Below is the response from Amazon. The logic they employ is that since it is broken in an old version of Squid, it is fine for it to be broken on Cloudfront. https://forums.aws.amazon.com/thread.jspa?messageID=351384&#351384 https://forums.aws.amazon.com/thread.jspa?messageID=351384&#... - - - Hello, While we are aware of the issue with range request HTTP/1.0 206 responses and Chrome, we cannot provide an ETA for a fix. Since this issue is specific to range requests, an immediate workaround is to disable range requests on your origin server if this is possible for your use case. It is also worth mentioning that multiple web proxy and cache application vendors have using HTTP/1.0 as a de facto standard for many years, so you will probably sporadically get similar reports from your end users using Chrome, but not other browsers such as Firefox or Safari. For example, here is a discussion between a Chrome developer on the mailing list for the popular Squid web cache about a similar report: http://www.squid-cache.org/mail-archive/squid-dev/201204/0113.html http://www.squid-cache.org/mail-archive/squid-dev/201204/011... I am not saying that always returning HTTP/1.0 will stick around forever, but it is fairly common in real world situations today.
jefe78 13y ago
You're doing something horribly wrong. I work for a live streaming company and we make extensive use of Varnish. It can probably solve the problem you're describing.
- jcampbell1 13y ago
  I'm not doing a damn thing wrong other than using Cloudfront. The problem is on their end, not mine. Thinking Varnish could solve this problem is utterly confused. Do you know what CDN does? CDNs have servers located around the world so files are loaded quickly and with low latency. Furthermore, your profile suggest you work for a pump and dump penny stock company (basically a scam). If your employer is paying you in something other than cash, you need to walk away asap.
  - deleted 13y ago
    [deleted]
    - jcampbell1 13y ago
      I am sincerely not being spiteful. I am trying to warn you. Your company's unaudited financials state that there is only $7,000 in the bank. http://www.sec.gov/Archives/edgar/data/1499274/000109635013000034/xumanii_draft01311310q11.htm http://www.sec.gov/Archives/edgar/data/1499274/0001096350130... I sign my own paycheck, so the notion of whether he is overpaying is a bit confusing.
      - jefe78 13y ago
        I'm entirely aware of the financial situation. How is calling my employer a pump and dump scam NOT spiteful when I've worked on this project from the beginning? Edit: Also, I don't appreciate you posting that. It's completely off-topic. Keep it classy.
        jcampbell1 13y ago
        At this point, the article has left hackernews. I am writing to you as fellow hacker looking to help you out. You are involved in a stock fraud. The company you work for is a sham. If you live in the US, then you have a plausible defense that you have no understanding of the underlying business. In this case, you likely can't afford the lawyer to present this case. If you don't live in the US, then be careful. Imagine, ten years down the road, you are a successful engineer, and want to take your family to Disney world. Unfortunately, there is an outstanding bench warrant for your arrest, and rather than a nice family vacation that your wife wanted, you end up in a US prison.
photorized 13y ago
If you haven't found a good alternative yet, we'll be happy to help. /co-founder at Advection.NET
- jcampbell1 13y ago
  I'll have a look. Here is what I need: CDN that support Range Requests correctly. I want to use S3 as the origin. I need an edge location in Australia. Also, there can be no 301/302 HTTP redirects, thus I have eliminated Google as a potential offering. My current spend is around $1000 a month, so it is a tiny account.
  - photorized 13y ago
    Don't have an edge location in Australia currently, but our Hong Kong POP could work with reasonable latency - we can test that. Let's discuss offline. malatortsev at advection dot net

currysausage 13y ago

I don't know about Cloudfront vs Rackspace+Akamai, however, based on some quick tests I did a year ago, I seem to recall that Rackspace+Akamai didn't deliver the same performance as professional Akamai services. In my experiments, Rackspace+Akamai used different servers, often located farther away, than big Akamai sites. (If anybody has background information about this, I would be very interested to hear.)

noelwelsh 13y ago
Rackspace CDN says this: "Rackspace uses 213 of Akamai's edge locations, selected especially for our customers' typical usage patterns, and designed to cover all major areas of the globe." http://www.rackspace.com/cloud/files/ http://www.rackspace.com/cloud/files/ Akamai has approximately 5 gazillion edge locations, so yes, it is a cut down version. It is still a lot more POPs than just about every other CDN, though this doesn't necessarily translate into performance.
- currysausage 13y ago
  Ah, thank you. That info wasn't there when I conducted the tests: http://web.archive.org/web/20120915095343/http://www.rackspace.com/cloud/public/files/ http://web.archive.org/web/20120915095343/http://www.rackspa...? "though this doesn't necessarily translate into performance" - Yes, even when the files were already cached, I wasn't always satisfied with the performance. A detail that bugged me, by the way, was the high number of CNAME requests, although this should at worst affect the first view.