5 ms·

If fly.io is serious about their marketing, this is not the way to do it. Find a really slow website that sells something, give them your service for free and

by birken 9y ago

If fly.io is serious about their marketing, this is not the way to do it. Find a really slow website that sells something, give them your service for free and run half the traffic through your fast version vs their slow version. Show me how much money they made in increased sales vs the amount of time it took to implement and the ongoing costs of your service. I'm pretty confident the math won't work out, but I'd be open to being convinced. Trotting out the 10 year old Google and Amazon studies about sub-1% gains isn't the way to do it. For the vast majority of startups or websites, there should be 50 things on your to-do list that have the potential to increase your conversions more than 1-2%, and your time is almost certainly better spent working on those things than adopting a service like this. I know engineers like taking the same thing and making it faster, it is fun and satisfying, but it almost certainly isn't the best use of your time when it comes to the bottom line. And that is assuming making your site faster actually will increase conversions by any meaningful amount, which is a big if (very old and very specific studies be damned!).

mrkurt 9y ago
(disclaimer, I work on fly) This is really just meant to be decent content, not a sales pitch. We obviously want to grow our business so we write about stuff that's close to what we do. Most people are really uninformed about site performance (still), you'd be shocked how many people don't even know about the 10 year old studies. Also note, Amazon's -1% per 100ms of latency holds up for the ecommerce companies I've worked with. It's not a small number for any company, and we're actually really cheap for companies with a high value per user (like ecommerce! or saas!). You probably wouldn't serve billions of page views through fly with very cheap ads, though. That said, I pretty much agree with you. We'll only succeed if we can keep proving that we're valuable. It's part of why we give enough traffic away for free that you can legitimately test us out. I don't particularly want people using us if they don't get any value out of it.
- MBCook 9y ago
  That was the problem I had with this article as well. I've seen some of this stuff before, but I kept expecting to run into a graph showing various delays versus revenue on an e-commerce site. I just assumed it had been measured. Instead when it came to revenue it was all implied (more people will stay on your site, they'll browse more pages, etc.) or an old statistic from someone else.
- sokoloff 9y ago
  About 5 years ago, I undertook a significant web performance effort on an IR top 100 e-commerce property. We sped the site up significantly and, being the good data nerds that we were, split-ran the hell out of it and found a zero-point-zero (sub-noise) difference in average session value, conversion rate, and average order value. Believing those then 5-year old studies, we went in the opposite direction and intentionally slowed the site down in multi-variate testing. Out to 1000ms of intentional slowdown per dynamic page, we saw zero-point-zero change in ASV, CR%, and AOV. We kept the faster experience, because "hey, why not?" and we'd paid all the NRE for it anyway, but the project was a technical success but a big bust versus business case.
  - mrkurt 9y ago
    That's actually really interesting. I'd love to know more!
    - EGreg 9y ago
      Actually that's an interesting takeway. Before working hard to optimize speed, if you have a working site, try slowing it down and seeing if that makes a difference!
      - sokoloff 9y ago
        Agreed (naturally), but also be aware that the relationship may not be linear. There is quite possibly a "knee" in the curve where faster than some threshold will all be the same. However, it's a quite easy test to run, so may be worth doing even if you can't stake your life on the results applying to a hypothetical speedup.
  - gingerlime 9y ago
    That's really fascinating! What about things like Google ranking that's supposedly also affected by page speed? (Or maybe it's not? I have a very limited SEO knowledge)
    - 9y ago
nasalgoat 9y ago
I can say that 1-2% speed improvement helps conversions. We can see it in the graphs when the API slows down, people don't sign up. Perhaps the context of the service plays a role - for us were are convincing people who might be a little more reluctant to sign up while other services are in high demand and no amount of friction is going to prevent a sale or signup.
andy_ppp 9y ago
Interesting; I always assumed faster performance was going to mean (lots) more conversions. Makes me wonder if AMP on google serps actually increases bounce rate due to lack of investment in a page load. There’s probably loads of stuff going on here and this area still needs more research. YC must have internal info on the tests companies have done to increase conversion/engagement? If not it’d be good to try to collate different experiences and do some meta-analysis.
blantonl 9y ago
This second paragraph resonates with me and how I run my business. I was prospected for years that I should use a “said” managed DNS provider because my business would be doomed if DNS lookups for my web properties were under a certain ms timeframe. Never mind trying to convince improper behaving resolvers out there to honor a TTL. There are so many other things we need to focus on to convert users to paying subscribers. A 50ms DNS lookup, outside of what could be delivered by a zillion caches, is not on my list. Sorry,

by birken 9y ago

mrkurt 9y ago

(disclaimer, I work on fly) This is really just meant to be decent content, not a sales pitch. We obviously want to grow our business so we write about stuff that's close to what we do. Most people are really uninformed about site performance (still), you'd be shocked how many people don't even know about the 10 year old studies. Also note, Amazon's -1% per 100ms of latency holds up for the ecommerce companies I've worked with. It's not a small number for any company, and we're actually really cheap for companies with a high value per user (like ecommerce! or saas!). You probably wouldn't serve billions of page views through fly with very cheap ads, though. That said, I pretty much agree with you. We'll only succeed if we can keep proving that we're valuable. It's part of why we give enough traffic away for free that you can legitimately test us out. I don't particularly want people using us if they don't get any value out of it.

MBCook 9y ago
That was the problem I had with this article as well. I've seen some of this stuff before, but I kept expecting to run into a graph showing various delays versus revenue on an e-commerce site. I just assumed it had been measured. Instead when it came to revenue it was all implied (more people will stay on your site, they'll browse more pages, etc.) or an old statistic from someone else.
sokoloff 9y ago
About 5 years ago, I undertook a significant web performance effort on an IR top 100 e-commerce property. We sped the site up significantly and, being the good data nerds that we were, split-ran the hell out of it and found a zero-point-zero (sub-noise) difference in average session value, conversion rate, and average order value. Believing those then 5-year old studies, we went in the opposite direction and intentionally slowed the site down in multi-variate testing. Out to 1000ms of intentional slowdown per dynamic page, we saw zero-point-zero change in ASV, CR%, and AOV. We kept the faster experience, because "hey, why not?" and we'd paid all the NRE for it anyway, but the project was a technical success but a big bust versus business case.
- mrkurt 9y ago
  That's actually really interesting. I'd love to know more!
  - EGreg 9y ago
    Actually that's an interesting takeway. Before working hard to optimize speed, if you have a working site, try slowing it down and seeing if that makes a difference!
    - sokoloff 9y ago
      Agreed (naturally), but also be aware that the relationship may not be linear. There is quite possibly a "knee" in the curve where faster than some threshold will all be the same. However, it's a quite easy test to run, so may be worth doing even if you can't stake your life on the results applying to a hypothetical speedup.
- gingerlime 9y ago
  That's really fascinating! What about things like Google ranking that's supposedly also affected by page speed? (Or maybe it's not? I have a very limited SEO knowledge)
  - 9y ago

nasalgoat 9y ago

I can say that 1-2% speed improvement helps conversions. We can see it in the graphs when the API slows down, people don't sign up. Perhaps the context of the service plays a role - for us were are convincing people who might be a little more reluctant to sign up while other services are in high demand and no amount of friction is going to prevent a sale or signup.

andy_ppp 9y ago

Interesting; I always assumed faster performance was going to mean (lots) more conversions. Makes me wonder if AMP on google serps actually increases bounce rate due to lack of investment in a page load. There’s probably loads of stuff going on here and this area still needs more research. YC must have internal info on the tests companies have done to increase conversion/engagement? If not it’d be good to try to collate different experiences and do some meta-analysis.

blantonl 9y ago

This second paragraph resonates with me and how I run my business. I was prospected for years that I should use a “said” managed DNS provider because my business would be doomed if DNS lookups for my web properties were under a certain ms timeframe. Never mind trying to convince improper behaving resolvers out there to honor a TTL. There are so many other things we need to focus on to convert users to paying subscribers. A 50ms DNS lookup, outside of what could be delivered by a zillion caches, is not on my list. Sorry,