13 ms·
Launch HN: RankScience (YC W17) – Automated Split-Testing for SEO
- ryanb 10y agoRankScience automates split-testing for SEO to grow organic search traffic for businesses. 80% of clicks from Google go to organic results and yet most companies don't know how to improve their SEO, or can't effectively measure their efforts to do so. Because of the scale of SEO and the constant change of both Google's ranking algorithm and your competitors' SEO campaigns, the only way to succeed in the long-run is with software and continuous testing. We've built a CDN that enables our software to provide tactical SEO execution and run A/B testing experiments for SEO across millions of pages. Experiments typically take 14-21 days for Google to index and react to changes, and we use Bayesian Structural Time Series and Negative Binomial Regression models to determine the statistical significance of our experiments. Our software is 100% technical SEO, and doesn't do anything black-hat, spammy, or anything related to link-building. One of our goals is to bring transparency and shed light on what is largely considered a shady industry, but is so important to so many companies' revenue and growth. In fact, If SEO didn't have such a bad reputation, we think someone else would have built this a long time ago. SEO as an industry earned itself a stigma for being spammy: between buying links, creating low-quality pages stuffed with keywords and text intended for Google rather than humans, and the used car salesmen attitude that many SEOs have, many people have been conditioned to dismiss SEO as an invalid or illegitimate growth channel. We're software engineers-turned-SEO's, who have previously consulted for dozens of companies on SEO, from YC startups to Fortune 500 companies like Pfizer. We previously shared our case study with HN, where we increased search traffic to Coderwall with one A/B test: https://www.rankscience.com/coderwall-seo-split-test https://www.rankscience.com/coderwall-seo-split-test Ask us anything! We'd love to answer any questions you have about SEO, A/B testing, and RankScience.
- etler 10y agoCan you explain a bit more about the technical details of what the platform can do? Does it integrate with web master tools and google analytics? Does it help identify page populations that are stable and similar enough to be compared? How are the different test versions sent to you guys? Can we use our own CDN on top of your CDN? What about dynamic pages that change over time? How big of a sample size is needed to get statistically significant results?
- dillonforrest 10y agoHowdy etler! > Does it integrate with web master tools and google analytics? Yep! > Does it help identify page populations that are stable and similar enough to be compared? Yep! > How are the different test versions sent to you guys? For the most part, we own the design of treatments. Some of our customers have their own experiments, but that's a small minority of our customers. Happy to talk more in depth about this if you're interested! dillon@rankscience.com > Can we use our own CDN on top of your CDN? Yes, we encourage this too! > What about dynamic pages that change over time? We address this on a case-by-case basis, but our software is designed to be as un-invasive as possible in use cases that we do not expect. > How big of a sample size is needed to get statistically significant results? We have to consider several variables here. Sample size is a function of Google's crawl rate on your site, the number of pages you have, and the amount of traffic you receive.
- etler 10y agoThanks for the response! I know the sample size question is hard to answer because of the number of variables involved, but the reason I ask is that the CoderWall blog post example mentioned 20,000 pages, which is more than we have so I was curious about what a rough rule of thumb range might be.
- xiaoma 10y agoHow does your pricing work? Do you have a free tier?
- dillonforrest 10y agoCo-founder/CTO here. Please share any SEO questions!
- mhoad 10y agoHow does your product compare to Distilled's offering? https://www.distilledodn.com/ https://www.distilledodn.com/
- ryanb 10y agoOurs is automated and continuous, and there's is a self-service product. We like those guys though!
- austenallred 10y agoThat's an enormous difference for me, for what it's worth
- TomAnthony 10y agoWe allow self service but most of our customers use our team (currently 16 SEO Analysts and Consultants) to work with their team, helping ideate, manage and run tests on an ongoing basis. See https://www.distilledodn.com/learn-more/expert-insights/ https://www.distilledodn.com/learn-more/expert-insights/
- will_critchlow 10y agoI think it's fairly similar - I remember showing Ryan a demo of our tool back when he was at 7cupsoftea before RankScience got going. We just published an update on our first year of data: https://www.distilled.net/resources/distilled-odn-by-the-numbers/ https://www.distilled.net/resources/distilled-odn-by-the-num... (Distilled founder & CEO here)
- saycheese 10y agoWhy have you taken this approach to monetizing this SEO optimization method, instead of other options, for example PPC arbitrage? Why A/B testing and not a more complex statistical hypothesis testing method?
- sgslo 10y agoLooks great, congrats on the launch! Two points of feedback: (1) The idea of the product is clearly conveyed, but I'm confused on exactly how it works. The landing page mentions that title tags, headlines, meta-tags, etc get tweaked - exactly how is this done? Do I have to manually enter a bunch of alternative text, or are you using a big fancy thesaurus to switch out some key terms? (2) How do you evaluate performance of the product? Solely through click rates, or by search rankings? How often do google search results get updated? In short, how do I know the product is working?
- dillonforrest 10y agoThanks sgslo! (1) We use both humans and software to generate experiments. For customers, it's completely automated. (2) We look at all primary search metrics (clicks, impressions, CTR, and rankings), with clicks being our main metric. Search results get updated at a pace determined by Google's crawl rate, which varies per site depending on multiple factors including domain authority. We use bayesian structural time series and negative binomial regression models to measure impact and statistical significance to power our data-driven SEO recommendations.
- coolswan 10y agoThanks for the AMA! What's the best way to deal with Google algorithm changes? Also, at what point does site performance actually impact my SEO?
- ryanb 10y agoThe only way to deal with Google algorithm updates is through experimentation. A data-driven approach is the only approach. : ) Staying abreast of what both Google is saying publicly and what the SEO community is saying also helps (ie Google announced they're cracking down on intrusive pop-ups for mobile sites) Site performance is always relative to your competitors. Optimal server response time is around 200ms, and that's what folks should strive for, but I've seen sites have really slow pages and still get lots of traffic.
- BickNowstrom 10y agoThough using A/B testing to improve user experience or conversion rates is fine, I thought using A/B testing to reverse engineer the ranking algorithm was against the guidelines. Has this been updated? From https://support.google.com/webmasters/answer/7238431?hl=en https://support.google.com/webmasters/answer/7238431?hl=en > Best practices for website testing with Google Search > The amount of time required for a reliable test will vary depending on factors like your conversion rates, and how much traffic your website gets; a good testing tool should tell you when you’ve gathered enough data to draw a reliable conclusion. Once you’ve concluded the test, you should update your site with the desired content variation(s) and remove all elements of the test as soon as possible, such as alternate URLs or testing scripts and markup. If we discover a site running an experiment for an unnecessarily long time, we may interpret this as an attempt to deceive search engines and take action accordingly. This is especially true if you’re serving one content variant to a large percentage of your users. Next to this, the advice is to use rel="canonical" to avoid duplicate issues with Googlebot crawling your variations. When using rel="canonical" this should not show you how a variation influences ranking. > If you’re running an A/B test with multiple URLs, you can use the rel=“canonical” link attribute on all of your alternate URLs to indicate that the original URL is the preferred version. We recommend using rel=“canonical” rather than a noindex meta tag because it more closely matches your intent in this situation.
- ryanb 10y agoThis is link is related to Conversation Rate Optimization testing (like Optimizely). We don't do A/B testing on single pages, or do cloaking or anything of the sort, but we run experiments across groups of URLs, and then we sum up the results and run our analysis. Also, our goal is not to deceive Google in anyway - a lot of our tests are related to increasing CTR (which is testing humans) and on-page times. (again testing humans) Overall we're trying to make pages better according to Google guidelines -- which leads to a better experience for users. Some more explanation of how it works here: https://www.rankscience.com/how-it-works https://www.rankscience.com/how-it-works
- BickNowstrom 10y agoMy post was in response to the homepage copy: > RankScience sits next to your website, making thousands of experiments to tweak your HTML in order to improve your page rankings. This seems to me an attempt at reverse engineering the ranking algorithm. Is my interpretation correct? And if so, is this allowed / in scope of the guidelines?
- killion 10y agoWe've been using RankScience for months at Suiteness. They've given our indexed pages a consistent boost every week.
- dillonforrest 10y agoThanks Kyle! :)
- omarchowdhury 10y agoHow does your pricing work?
- ryanb 10y agoIt's based on how much traffic goes through our CDN. If you signup on our site or send me a note directly ryan@rankscience.com I can give you more details
- TomAnthony 10y agoHey guys! Congratulations on the launch! Very exciting! We've been seeing some great results with SEO testing, and have recently had our biggest test result (in terms of revenue impact). Looking forward to hearing more of what you guys are up to. :) I think the fact that DistilledODN, RankScience, Etsy, and Pinterest have all published SEO split-test results recently demonstrates the importance of this type of data-driven approach to SEO! Best of luck with everything! Tom, Distilled (Disclaimer: I run the https://www.distilledodn.com/ https://www.distilledodn.com/ team)
- ryanb 10y agoThanks Tom. Would love to buy you guys a beer sometime. : )
- hartator 10y agoCongrats on the launch. How do you check the effectiness of a SEO change? You check for Google rankings or traffic?
- ryanb 10y agoWe look at Clicks, Rankings and Impressions. We use Google Analytics, Webmaster Tools, and another third party data source. More clicks is obviously the ultimate goal, but sometimes we also have significant experiments where rankings or impressions going up are the key metric
- austenallred 10y agoA few questions about your product (I run SEO and digital marketing at LendUp - YCW12). This is very, very cool. * What types of optimizations does it do/do you test? * I assume tests take a while to run, waiting for Google to re-index etc. Do you essentially monitor rank changes, assume that Google has re-indexed at that point, and use that as the data to optimize on? Or do you have some smart way to monitor for when Google re-indexes? * I'm again assuming here - that a user will plug in a few variations of things to test, and let your software test them? Or are there automatic optimizations the software tries to make? * How many tests can one do in a given time period, without confounding test & control variants? It seems like they would take a while to run? * Is there a good way to control for non-technical/non-content-based changes (e.g. external, links)? For example we get hundreds of negative links pointed at us per week. Do we just hope/assume that's not the cause of rank changes?
- dillonforrest 10y agoHey there! We test everything on-page. This usually includes CTR stuff, like titles and meta descriptions and paragraph text that Google extracts for the meta descript, headers, images, calls to action, even conversion rates. Correct, we continually iterate based on feedback like rankings and clicks, and we're working on smarter ways to monitor Google's crawler too! Users are as hands-on or hands-off as they'd like. We usually own the experiments, but some of our customers design their own too. The number of tests depends on the number of pages and your traffic. Our largest customers have full factorial experiments with thousands of concurrent split tests every day.
- Mizza 10y agoHow does this compare with the other offerings in the space?
- dillonforrest 10y agoWe focus on being automated! Our software never stops working to grow your search traffic, even when you're asleep or if you don't want to think about SEO at all.
- jitbit 10y agoGuys, The idea looks great. Congrats on the launch. 2 SEO questions: 1) a CDN means "thousands of websites on one IP address". Google doesn't like it UNTIL it knows, the IP belongs to a well-known CDN like cloundfront/cloudflare etc. Please comment? 2) An A/B test might look like "cloaking" to googles. How exactly do you run it? I assume, not in parallel, but "variation A then variation B" - correct? If yes, does it mean I have to leave the website UNTOUCHED for 21 days so the test results are not distorted by my other activities? (adding new content, internal links etc)
- ryanb 10y agoThanks! Great questions 1) We actually work with CloudFlare/CloudFront etc if companies are already using them. In that case, we sit in between CloudFlare and your origin webserver. But if you don't already have a CDN in front of your website, our SEO CDN goes in front of your origin webserver, and we use AWS so as far as Google is concerned it looks like AWS. We also use different IPs for every customer. 2) We don't do cloaking - we're essentially testing somewhat small tweaks across groups of similar pages, and seeing if they have any impact on clicks or rankings. It is preferred if the pages remain largely the same, but they don't have to be totally untouched.
- jitbit 10y agoThanks for the answer. PS. whoa, exciting (the 'in-between cloudflare and your server' bit)
- birken 10y agoAs somebody who wrote an article about A/B testing title tags in 2011 before it was cool [1], this is an awesome idea. I've talked about SEO with many companies and coming up with the proper title tags and meta descriptions alone is often worth so much traffic for such little effort (once you get past the upfront cost of running the tests). However, I think a critical aspect of SEO is thinking about an entire site holistically. Not only because certain signals are site-wide, but because a key aspect of SEO is deciding what pages of your website are "good" for SEO and which ones aren't, and then focusing on making the "good" pages better and not worrying about the "bad" pages. Good and bad in quotes because it is often quite a bit of an art and not a science. How does RankScience play into this? You've nailed the on-page stuff but is there any world in which RankScience is able to talk about a site holistically and recommend which types of pages and content seem to be working most effectively (and maybe even suggesting pages that could be removed/de-indexed)? Or do you leave that to SEO consultants and you just nail the hell out of the on-page stuff. 1: https://www.thumbtack.com/engineering/seo-tip-titles-matter-probably-more-than-you-think/ https://www.thumbtack.com/engineering/seo-tip-titles-matter-...
- dillonforrest 10y agoIt's a true honor to speak with an SEO A/B testing OG! :) > How does RankScience play into this? That's a really great point. Right now, our software either focuses on only pages which are good for SEO, or it runs less frequent experiments on the "bad" pages. We've had a few success stories of turning "bad" pages into good ones and having them become big revenue generators for our customers, but that's not common as I'm sure you can imagine.
- 7cupsoftea 10y agoJust want to chime in here and share that I'm a very happy customer. Ryan, Dillon, and Chad are some super smart guys who deeply understand SEO. I use RankScience for 7 Cups and Edvance360 and I've seen a huge ROI. They always find time to meet with me and my other team members. The advice and feedback they provide is worth the cost of the service alone. Highly, highly, highly recommended!
- dillonforrest 10y agoThanks Glen!! :)
- chadburgess 10y agoyaya, thanks!
- deleted 10y ago[deleted]
- ortekk 10y agoThat looks great! How do you automate google's rank checking? Do you use proxies/swarm of VPS's?
- ryanb 10y agoWe use Google Analytics, Google Search Console, and third party services to supplement keyword rankings data -- we don't crawl Google SERPs. :) Plenty of other companies do, though. Moz has great keyword ranking data.
- gukov 10y agoIt seems like you integrate with the likes of Google (Webmaster Tools) and Cloudflare in ways not done before, so my assumptions here are probably a bit outdated. Let's say you want to test a new title. You collect stats for a week, change the title, collect stats for another week. The two datasets are then compared. Am I close? Does it mean that you can't AB test in parallel? If so, it's not optimal for time sensitive stuff like breaking news.
- ryanb 10y agoWe don't run experiments on a single page, but instead run them across groups of similar pages. The easiest way to think about this is running a test across a "product" template on an e-commerce site. A site could have thousands of product pages, but they're all on the same template. We would grab a set of a few thousand product pages, and split them into control and variant groups, execute the change on the variant group, and monitor how Google reacts to the groups in aggregate over a period of time. Depending on how many pages you have, we can run many tests in parallel.
- gukov 10y agoGotcha Thanks
- franciscassel 10y agoI consult with startups and tech companies (recently: SurveyMonkey) on SEO, so this is super-interesting... especially the automated aspect, which is pretty novel in this space. What are some of the specific types of automated tests that you run?
- ryanb 10y agoHere's a case study with a simple example of a test: https://www.rankscience.com/coderwall-seo-split-test https://www.rankscience.com/coderwall-seo-split-test * CTR on titles/meta descriptions * Any HTML changes * Design changes * Time on site
- franciscassel 10y agoThanks for the reply. Would you mind being more specific about what specific aspects of the "automated" tests are "automated"? E.g., in the CoderWall example, is it propagating of the title tag change to all the pages in the test group that's "automated"? At least to my ears, "automated" suggests that there are tests that are selected and run completely without human intervention. (Which is hard to imagine in the SEO space.) Is that in any way accurate?
- dillonforrest 10y agoCorrect! Our customers don't have to lift a finger, and our software continually iterates on their SEO.
- gargarplex 10y agoWhat's the minimum amount of traffic to benefit from this offering. At what point should a startup seriously consider working with you
- ryanb 10y agoNo real minimum - it's all case by case.
- gargarplex 10y agoWould you consider donating some services to Zidisha (YC W14)? If yes, please email me – zack@zackburt.com. Thanks.
- jjyoung135 10y agoWhen you are just starting out with a website, what is the 20% of effort that leads to 80% of the results in regard to SEO?
- dillonforrest 10y agoHey there! I know we just chatted on facebook about this, but just in case anybody else is wondering: we're working on this post. Stay tuned!
- 0bfus 10y agoYou guys mention < 25ms as the performance hit of having you guys in front of the client's application. Is there a more concrete SLA you guys provide? Do you have a sense of what that hit might cost in page rank (there's some theories out there that time to first byte is one of the features that search engines reward)?
- dillonforrest 10y agoHey 0bfus, we're working on a fancy performance metrics dashboard and status page to answer these questions! Also, we've done extensive research on pageload speed's effect on page rank, and we found that <25ms of incremental latency does not have any impact. We're working on a case study for the latter!
- sixQuarks 10y agoI wish I could use a tool like this and know that it won't affect my rankings negatively, but I feel that Google will eventually punish sites that do a lot of A/B testing.
- ijafri 10y agoI am bit wary of the idea, despite the fair intentions, but Google in particular tends to view anything remotely close to 'improving' search ranking using 3rd party 'software', as a threat to their own Ego-rithms.