4 ms·

Your first point is not correct. Crawlers can crawl JavaScript only websites using the Ajax crawling API: http://code.google.com/web/ajaxcrawling/docs/specifica

by maccman 15y ago

Your first point is not correct. Crawlers can crawl JavaScript only websites using the Ajax crawling API: http://code.google.com/web/ajaxcrawling/docs/specification.html http://code.google.com/web/ajaxcrawling/docs/specification.h... Your second point has merit. However I'd counter it thus: * You can load a HTML page first, containing a representation of the end content, so you can avoid the flash * You can show some sort of loading indicator on page load. Users are usually fine with waiting for the initial page load - it's only subsequent interactions that need be fast. In other words, there's no reason why the page need be blank.

sirn 15y ago
However by using hashbang URLs, you have to keep that little snippet of JavaScript forever to maintain permalink even after the web has moved on to another solution. It's not a future-proof solution.
- mattmanser 15y ago
  I'm not sure what you mean by little snippets of javascript, a link is a link. If you change why couldn't you just 301 them to your new format? It's a trivial redirect that you're keeping for the g-juice.
  - icebraining 15y ago
    The part after an hashbang is not sent to the server, so you need client side code to load the content. This is because the # in an URL is supposed to be an identifier of a element in the page, not an actual resource.
- fooandbarify 15y ago
  Hashbangs are only a stop-gap at this point - modern browsers ship with the HTML5 History API.
  - quanticle 15y ago
    Does IE8 ship with the History API?
    - akavlie 15y ago
      No, IE8 is not a modern browser by most definitions.
      - quanticle 15y ago
        Which definitions? I've seen definitions of "modern browser" that only exclude IE 6 and 7.
        dextorious 15y ago
        They are wrong. They use modern to mean "quite new and available to most clients" instead of "actually implementing the majority of the latest standards". IE 9 comes close, and can be considered modern.
- dextorious 15y ago
  Or you could just keep a tiny snippet of Javascript that searches for old style hashbangs and redirects all of them to the "another solution" handler to decide what to do with them. It can be just 3-4 lines of JS.
  - sirn 15y ago
    The problem is that you have to keep that snippet forever if you ever use the hashbangs. That's why I'm saying it's not a future-proof solution. Not that few bytes matters, though.
bad_user 15y ago
Only Google does that currently. And while people can build crawlers that can fetch Javascript rendered pages, these scripts will have to drive real browsers with rendering engines, not just simple HTTP clients. And this is a problem, because fetching millions of web pages will require serious infrastructure work, versus a crawler using libev that can fetch dozens of pages in parallel without a sweat. Also, that spec you gave is not exactly how Google operates. That's the advice Google gives for making web apps compliant, but the reality is that Google is smarter than that and it takes Google a good amount of effort to fix other people's screwups. Accessible to crawlers doesn't mean Google-only and you should be careful not to throw the baby along with the bathwater.
- jonknee 15y ago
  For most purposes, search engines seeing you = Googlebot seeing you. I have never had a competitor past single digits in percentage of search engine referrals.
  - bad_user 15y ago
    A crawler is not necessarily providing content to a classical search engine, as it can serve many purposes. Ultimately such crawlers may generate links to your website and while not bringing you too much referrals traffic, it may increase your ranking, which ultimately means more traffic from Google. Also, if search engine referrals is all you're interested in, if Google drops you from their index, your website might as well not exist, right? As I said, be careful not to throw the baby with the bathwater. Google is not what made the web great, but a Google dependency might break it and website authors can be blamed for this.
nupark2 15y ago
Your first point is not correct. Crawlers can crawl JavaScript only websites using the Ajax crawling API: http://code.google.com/web/ajaxcrawling/docs/specification.h.. http://code.google.com/web/ajaxcrawling/docs/specification.h.... This sounds like a huge imposition to implement -- so much so that I seriously doubt anyone will bother for anything even moderately complex. Not only do you have to produce an API for the client-side javascript to call, but you also have to render static HTML for Google to index.

by maccman 15y ago

sirn 15y ago

However by using hashbang URLs, you have to keep that little snippet of JavaScript forever to maintain permalink even after the web has moved on to another solution. It's not a future-proof solution.

mattmanser 15y ago
I'm not sure what you mean by little snippets of javascript, a link is a link. If you change why couldn't you just 301 them to your new format? It's a trivial redirect that you're keeping for the g-juice.
- icebraining 15y ago
  The part after an hashbang is not sent to the server, so you need client side code to load the content. This is because the # in an URL is supposed to be an identifier of a element in the page, not an actual resource.
fooandbarify 15y ago
Hashbangs are only a stop-gap at this point - modern browsers ship with the HTML5 History API.
- quanticle 15y ago
  Does IE8 ship with the History API?
  - akavlie 15y ago
    No, IE8 is not a modern browser by most definitions.
    - quanticle 15y ago
      Which definitions? I've seen definitions of "modern browser" that only exclude IE 6 and 7.
      - dextorious 15y ago
        They are wrong. They use modern to mean "quite new and available to most clients" instead of "actually implementing the majority of the latest standards". IE 9 comes close, and can be considered modern.
dextorious 15y ago
Or you could just keep a tiny snippet of Javascript that searches for old style hashbangs and redirects all of them to the "another solution" handler to decide what to do with them. It can be just 3-4 lines of JS.
- sirn 15y ago
  The problem is that you have to keep that snippet forever if you ever use the hashbangs. That's why I'm saying it's not a future-proof solution. Not that few bytes matters, though.

bad_user 15y ago

Only Google does that currently. And while people can build crawlers that can fetch Javascript rendered pages, these scripts will have to drive real browsers with rendering engines, not just simple HTTP clients. And this is a problem, because fetching millions of web pages will require serious infrastructure work, versus a crawler using libev that can fetch dozens of pages in parallel without a sweat. Also, that spec you gave is not exactly how Google operates. That's the advice Google gives for making web apps compliant, but the reality is that Google is smarter than that and it takes Google a good amount of effort to fix other people's screwups. Accessible to crawlers doesn't mean Google-only and you should be careful not to throw the baby along with the bathwater.

jonknee 15y ago
For most purposes, search engines seeing you = Googlebot seeing you. I have never had a competitor past single digits in percentage of search engine referrals.
- bad_user 15y ago
  A crawler is not necessarily providing content to a classical search engine, as it can serve many purposes. Ultimately such crawlers may generate links to your website and while not bringing you too much referrals traffic, it may increase your ranking, which ultimately means more traffic from Google. Also, if search engine referrals is all you're interested in, if Google drops you from their index, your website might as well not exist, right? As I said, be careful not to throw the baby with the bathwater. Google is not what made the web great, but a Google dependency might break it and website authors can be blamed for this.

nupark2 15y ago

Your first point is not correct. Crawlers can crawl JavaScript only websites using the Ajax crawling API: http://code.google.com/web/ajaxcrawling/docs/specification.h.. http://code.google.com/web/ajaxcrawling/docs/specification.h.... This sounds like a huge imposition to implement -- so much so that I seriously doubt anyone will bother for anything even moderately complex. Not only do you have to produce an API for the client-side javascript to call, but you also have to render static HTML for Google to index.