6 ms·

All good advice. I'd add: 1 Speed matters. Google is on record about this. They want your site to load quickly. This factor is part of the 'caffeine' update th

by jsm386 17y ago

All good advice. I'd add: 1 Speed matters. Google is on record about this. They want your site to load quickly. This factor is part of the 'caffeine' update that Google is in the process of rolling out http://searchengineland.com/site-speed-googles-next-ranking-factor-29793 http://searchengineland.com/site-speed-googles-next-ranking-... 2 Don't stop with H1s. Think about the structure of your page. Subheaders should be H2, and so on. 3 Use Google Webmaster Tools to submit a sitemap. They'll regularly pull it down from your server so as you add pages to your site, Google will know to index them. They'll tell you how many pages you have submitted, and how many are indexed. While you're in Webmaster Tools, check out if your site has crawl errors. Check out the HTML suggestions. Google will point out the pages which are missing title tags, which have duplicate tags, short descriptions, etc. Lots of other information in there, including page load times.

andrewljohnson 17y ago
You don't need a site index unless you have a strange site. If you have text/HTML and you are linked from other sites, the site map won't matter.
- chaosmachine 17y ago
  That's been my experience, as well.
- jacquesm 17y ago
  A site map is an excellent way to make sure that it is your content that makes it in to google rather than someone that 'clones' you. It can give you a few days lead over the cloners and that's sometimes all you need.
  - andrewljohnson 17y ago
    I call bullshit. Have you actually ever had someone clone your site or tried to protect against it this way? I am all ears for your story... I have never heard of anything like that. Setting up a sitemap is usually just a placebo for finicky webmasters, and it's a waste of time to boot. The biggest sin at a start-up is wasting time, and this easily qualifies. The only time you want a sitemap is when you don't have a text front-end, and even then, you are way better off mimicking your JavaScript with a text layer for the search engines that implementing some sitemap. Even submitting a page in a sitemap doesn't ensure it gets crawled, and it certainly won't get crawled if you have a large site and don't set up your sitemap just right.
    - jacquesm 17y ago
      > I call bullshit. That's cool with me. > Have you actually ever had someone clone your site or tried to protect against it this way? Yes, otherwise I wouldn't have written that, now wouldn't I? > I am all ears for your story... I have never heard of anything like that. That you haven't heard of something is not a reason to assume that it doesn't exist, there are actually more things that you probably haven't heard about that aren't bullshit either. In fact, there are probably lots of those things. I recently did a fairly large project, well documented here on HN and in the press where I did just that. Some less than honorable characters were cloning the data as fast as it hit the server and turning it into 'mfa' fodder, so at some point I shut that down and submitted the site to google which then crawled it at it's leisure. Even today that crawl isn't 100% complete yet, and the only way you can reach those pages right now is by going through the google index. Long term I expect plenty of those pages to be linked again, both internally by linking pages that are related as well as externally by people that link to their content. I get about 50 emails daily confirming that the strategy worked, those people have lost their content and find it again through google and this is the only copy of it on the web, in spite of active attempts at making clones. The ip blocklist for that site is close to 10,000 IPs. > Even submitting a page in a sitemap doesn't ensure it gets crawled, and it certainly won't get crawled if you have a large site and don't set up your sitemap just right. No, if you don't set it up right then of course, it won't work, but that goes for most things in the technology world. Right now there are 197,000 pages indexed on that site according to google so I really can't complain, it seems to have worked very well.
      - andrewljohnson 17y ago
        Ok, well maybe bullshit is a strong term, so I apologize. Still, it seems really odd to be using a sitemap like that. That's certainly not the intended purpose, and if that were a reasonable defense, what's to stop the spammers from employing it as an offense? As far as I can tell, you use a sitemap if your site doesn't map in the normal way. Even then, it's not nearly as effective as a text site with any in-links - so much so that I wouldn't hinge any SEO strategy on a sitemap, and I wouldn't recommend setting one up to any new start-up founder. Even if your observations on this are correct, then you are a very special case. It's also worth pointing out that it seems your site map was no defense until you took other measures, which wasn't clear from your original post: "so at some point I shut that down and submitted the site to google" Responding to some of your points: "I get about 50 emails daily confirming that the strategy worked..." * This confirms that your site is indexed, but it doesn't confirm your strategy beat the spammers. It may be more of Google's algorithms deciding you are best, no? I can't say for sure obviously, but you might have fared better if you hadn't done this at all. "Even today that crawl isn't 100% complete yet" * Part of the reason for this is because your pages have a low-priority to be crawled, because they are submitted via a site map.
- stef25 17y ago
  The advantage of a sitemap, linked to from say a footer link on every page of your site, is that it's easier to find than a page 4 levels down. This actually makes all pages on your site "findeable" at 2 levels down (homepage > sitemap > target page). I think it should improve indexing on otherwise hard to find pages.

by jsm386 17y ago

andrewljohnson 17y ago

You don't need a site index unless you have a strange site. If you have text/HTML and you are linked from other sites, the site map won't matter.

chaosmachine 17y ago
That's been my experience, as well.
jacquesm 17y ago
A site map is an excellent way to make sure that it is your content that makes it in to google rather than someone that 'clones' you. It can give you a few days lead over the cloners and that's sometimes all you need.
- andrewljohnson 17y ago
  I call bullshit. Have you actually ever had someone clone your site or tried to protect against it this way? I am all ears for your story... I have never heard of anything like that. Setting up a sitemap is usually just a placebo for finicky webmasters, and it's a waste of time to boot. The biggest sin at a start-up is wasting time, and this easily qualifies. The only time you want a sitemap is when you don't have a text front-end, and even then, you are way better off mimicking your JavaScript with a text layer for the search engines that implementing some sitemap. Even submitting a page in a sitemap doesn't ensure it gets crawled, and it certainly won't get crawled if you have a large site and don't set up your sitemap just right.
  - jacquesm 17y ago
    > I call bullshit. That's cool with me. > Have you actually ever had someone clone your site or tried to protect against it this way? Yes, otherwise I wouldn't have written that, now wouldn't I? > I am all ears for your story... I have never heard of anything like that. That you haven't heard of something is not a reason to assume that it doesn't exist, there are actually more things that you probably haven't heard about that aren't bullshit either. In fact, there are probably lots of those things. I recently did a fairly large project, well documented here on HN and in the press where I did just that. Some less than honorable characters were cloning the data as fast as it hit the server and turning it into 'mfa' fodder, so at some point I shut that down and submitted the site to google which then crawled it at it's leisure. Even today that crawl isn't 100% complete yet, and the only way you can reach those pages right now is by going through the google index. Long term I expect plenty of those pages to be linked again, both internally by linking pages that are related as well as externally by people that link to their content. I get about 50 emails daily confirming that the strategy worked, those people have lost their content and find it again through google and this is the only copy of it on the web, in spite of active attempts at making clones. The ip blocklist for that site is close to 10,000 IPs. > Even submitting a page in a sitemap doesn't ensure it gets crawled, and it certainly won't get crawled if you have a large site and don't set up your sitemap just right. No, if you don't set it up right then of course, it won't work, but that goes for most things in the technology world. Right now there are 197,000 pages indexed on that site according to google so I really can't complain, it seems to have worked very well.
    - andrewljohnson 17y ago
      Ok, well maybe bullshit is a strong term, so I apologize. Still, it seems really odd to be using a sitemap like that. That's certainly not the intended purpose, and if that were a reasonable defense, what's to stop the spammers from employing it as an offense? As far as I can tell, you use a sitemap if your site doesn't map in the normal way. Even then, it's not nearly as effective as a text site with any in-links - so much so that I wouldn't hinge any SEO strategy on a sitemap, and I wouldn't recommend setting one up to any new start-up founder. Even if your observations on this are correct, then you are a very special case. It's also worth pointing out that it seems your site map was no defense until you took other measures, which wasn't clear from your original post: "so at some point I shut that down and submitted the site to google" Responding to some of your points: "I get about 50 emails daily confirming that the strategy worked..." * This confirms that your site is indexed, but it doesn't confirm your strategy beat the spammers. It may be more of Google's algorithms deciding you are best, no? I can't say for sure obviously, but you might have fared better if you hadn't done this at all. "Even today that crawl isn't 100% complete yet" * Part of the reason for this is because your pages have a low-priority to be crawled, because they are submitted via a site map.
stef25 17y ago
The advantage of a sitemap, linked to from say a footer link on every page of your site, is that it's easier to find than a page 4 levels down. This actually makes all pages on your site "findeable" at 2 levels down (homepage > sitemap > target page). I think it should improve indexing on otherwise hard to find pages.