5 ms·

If you do this, crawlers can't index your pages. Also your users see the page loaded blank and being gradually populated with content, which is not a good UX.

by ntoshev 15y ago

maccman 15y ago
Your first point is not correct. Crawlers can crawl JavaScript only websites using the Ajax crawling API: http://code.google.com/web/ajaxcrawling/docs/specification.html http://code.google.com/web/ajaxcrawling/docs/specification.h... Your second point has merit. However I'd counter it thus: * You can load a HTML page first, containing a representation of the end content, so you can avoid the flash * You can show some sort of loading indicator on page load. Users are usually fine with waiting for the initial page load - it's only subsequent interactions that need be fast. In other words, there's no reason why the page need be blank.
- sirn 15y ago
  However by using hashbang URLs, you have to keep that little snippet of JavaScript forever to maintain permalink even after the web has moved on to another solution. It's not a future-proof solution.
  - mattmanser 15y ago
    I'm not sure what you mean by little snippets of javascript, a link is a link. If you change why couldn't you just 301 them to your new format? It's a trivial redirect that you're keeping for the g-juice.
    - icebraining 15y ago
      The part after an hashbang is not sent to the server, so you need client side code to load the content. This is because the # in an URL is supposed to be an identifier of a element in the page, not an actual resource.
  - fooandbarify 15y ago
    Hashbangs are only a stop-gap at this point - modern browsers ship with the HTML5 History API.
    - quanticle 15y ago
      Does IE8 ship with the History API?
      - akavlie 15y ago
        No, IE8 is not a modern browser by most definitions.
        quanticle 15y ago
        Which definitions? I've seen definitions of "modern browser" that only exclude IE 6 and 7.
        dextorious 15y ago
        They are wrong. They use modern to mean "quite new and available to most clients" instead of "actually implementing the majority of the latest standards". IE 9 comes close, and can be considered modern.
  - dextorious 15y ago
    Or you could just keep a tiny snippet of Javascript that searches for old style hashbangs and redirects all of them to the "another solution" handler to decide what to do with them. It can be just 3-4 lines of JS.
    - sirn 15y ago
      The problem is that you have to keep that snippet forever if you ever use the hashbangs. That's why I'm saying it's not a future-proof solution. Not that few bytes matters, though.
- bad_user 15y ago
  Only Google does that currently. And while people can build crawlers that can fetch Javascript rendered pages, these scripts will have to drive real browsers with rendering engines, not just simple HTTP clients. And this is a problem, because fetching millions of web pages will require serious infrastructure work, versus a crawler using libev that can fetch dozens of pages in parallel without a sweat. Also, that spec you gave is not exactly how Google operates. That's the advice Google gives for making web apps compliant, but the reality is that Google is smarter than that and it takes Google a good amount of effort to fix other people's screwups. Accessible to crawlers doesn't mean Google-only and you should be careful not to throw the baby along with the bathwater.
  - jonknee 15y ago
    For most purposes, search engines seeing you = Googlebot seeing you. I have never had a competitor past single digits in percentage of search engine referrals.
    - bad_user 15y ago
      A crawler is not necessarily providing content to a classical search engine, as it can serve many purposes. Ultimately such crawlers may generate links to your website and while not bringing you too much referrals traffic, it may increase your ranking, which ultimately means more traffic from Google. Also, if search engine referrals is all you're interested in, if Google drops you from their index, your website might as well not exist, right? As I said, be careful not to throw the baby with the bathwater. Google is not what made the web great, but a Google dependency might break it and website authors can be blamed for this.
- nupark2 15y ago
  Your first point is not correct. Crawlers can crawl JavaScript only websites using the Ajax crawling API: http://code.google.com/web/ajaxcrawling/docs/specification.h.. http://code.google.com/web/ajaxcrawling/docs/specification.h.... This sounds like a huge imposition to implement -- so much so that I seriously doubt anyone will bother for anything even moderately complex. Not only do you have to produce an API for the client-side javascript to call, but you also have to render static HTML for Google to index.
csomar 15y ago
Not true. If you are building such application: 1. You need to have a non-JavaScript version. That is your main page index.html is a full functioning page without JavaScript. When JavaScript is enabled, it turns the page into a Web App. 2. A simple loader will solve the problem. You might not be able to show the progress for the moment.
- tomjen3 15y ago
  Why does he need to have a non-javascript version? There is (thankfully) no law which say that your service must be usable for people who turn javascript of.
  - anjc 15y ago
    Yeah, there's no law which says your website must be accessible...
    - intranation 15y ago
      Working with Javascript off != accessibility. Most screenreaders (the common face of accessibility technology) run on top of regular browsers, so your JS experience needs to be accessible too if you are legally mandated to (or even better if you care).
      - anjc 15y ago
        If javascript being off makes your site not work, or not viewable at least, then that's the very definition of not being accessible (i'm not limiting 'accessible' to only mean 'available to people with disabilities' like you appear to be) Edit: Graceful degradation should be a tenet of web development. Having no provision for a person who doesn't have javascript enabled seems bizarre to me, and being bullish about the requirement seems even more bizarre...maybe i'm just getting cranky with age or something.
        scarmig 15y ago
BerislavLopac 15y ago
If it's a true application, with user interaction and everything (and not just a set of glorified Web pages), then the former is not an issue; you don't really want your "pages" to be indexed. And the latter is easily solved with a "loading" bar or any other of UI tools previously invented for the exact same purpose on the desktop.
robmcm 15y ago
Rember this is an application, would a search engine want to crawl the innerds of photoshop or outlook, or a web page promoting them? Just because it's on the web doesn't mean it's a web site that wants to be searchable.
- hello_moto 15y ago
  In most community-based, social-networking, the companies behind them do want more exposure. The more people who join, the more vibrant the community is. FYI.

If you do this, crawlers can't index your pages. Also your users see the page loaded blank and being gradually populated with content, which is not a good UX.

by ntoshev 15y ago

If you do this, crawlers can't index your pages. Also your users see the page loaded blank and being gradually populated with content, which is not a good UX.

maccman 15y ago

Your first point is not correct. Crawlers can crawl JavaScript only websites using the Ajax crawling API: http://code.google.com/web/ajaxcrawling/docs/specification.html http://code.google.com/web/ajaxcrawling/docs/specification.h... Your second point has merit. However I'd counter it thus: * You can load a HTML page first, containing a representation of the end content, so you can avoid the flash * You can show some sort of loading indicator on page load. Users are usually fine with waiting for the initial page load - it's only subsequent interactions that need be fast. In other words, there's no reason why the page need be blank.

sirn 15y ago
However by using hashbang URLs, you have to keep that little snippet of JavaScript forever to maintain permalink even after the web has moved on to another solution. It's not a future-proof solution.
- mattmanser 15y ago
  I'm not sure what you mean by little snippets of javascript, a link is a link. If you change why couldn't you just 301 them to your new format? It's a trivial redirect that you're keeping for the g-juice.
  - icebraining 15y ago
    The part after an hashbang is not sent to the server, so you need client side code to load the content. This is because the # in an URL is supposed to be an identifier of a element in the page, not an actual resource.
- fooandbarify 15y ago
  Hashbangs are only a stop-gap at this point - modern browsers ship with the HTML5 History API.
  - quanticle 15y ago
    Does IE8 ship with the History API?
    - akavlie 15y ago
      No, IE8 is not a modern browser by most definitions.
      - quanticle 15y ago
        Which definitions? I've seen definitions of "modern browser" that only exclude IE 6 and 7.
        dextorious 15y ago
        They are wrong. They use modern to mean "quite new and available to most clients" instead of "actually implementing the majority of the latest standards". IE 9 comes close, and can be considered modern.
- dextorious 15y ago
  Or you could just keep a tiny snippet of Javascript that searches for old style hashbangs and redirects all of them to the "another solution" handler to decide what to do with them. It can be just 3-4 lines of JS.
  - sirn 15y ago
    The problem is that you have to keep that snippet forever if you ever use the hashbangs. That's why I'm saying it's not a future-proof solution. Not that few bytes matters, though.
bad_user 15y ago
Only Google does that currently. And while people can build crawlers that can fetch Javascript rendered pages, these scripts will have to drive real browsers with rendering engines, not just simple HTTP clients. And this is a problem, because fetching millions of web pages will require serious infrastructure work, versus a crawler using libev that can fetch dozens of pages in parallel without a sweat. Also, that spec you gave is not exactly how Google operates. That's the advice Google gives for making web apps compliant, but the reality is that Google is smarter than that and it takes Google a good amount of effort to fix other people's screwups. Accessible to crawlers doesn't mean Google-only and you should be careful not to throw the baby along with the bathwater.
- jonknee 15y ago
  For most purposes, search engines seeing you = Googlebot seeing you. I have never had a competitor past single digits in percentage of search engine referrals.
  - bad_user 15y ago
    A crawler is not necessarily providing content to a classical search engine, as it can serve many purposes. Ultimately such crawlers may generate links to your website and while not bringing you too much referrals traffic, it may increase your ranking, which ultimately means more traffic from Google. Also, if search engine referrals is all you're interested in, if Google drops you from their index, your website might as well not exist, right? As I said, be careful not to throw the baby with the bathwater. Google is not what made the web great, but a Google dependency might break it and website authors can be blamed for this.
nupark2 15y ago
Your first point is not correct. Crawlers can crawl JavaScript only websites using the Ajax crawling API: http://code.google.com/web/ajaxcrawling/docs/specification.h.. http://code.google.com/web/ajaxcrawling/docs/specification.h.... This sounds like a huge imposition to implement -- so much so that I seriously doubt anyone will bother for anything even moderately complex. Not only do you have to produce an API for the client-side javascript to call, but you also have to render static HTML for Google to index.

csomar 15y ago

Not true. If you are building such application: 1. You need to have a non-JavaScript version. That is your main page index.html is a full functioning page without JavaScript. When JavaScript is enabled, it turns the page into a Web App. 2. A simple loader will solve the problem. You might not be able to show the progress for the moment.

tomjen3 15y ago
Why does he need to have a non-javascript version? There is (thankfully) no law which say that your service must be usable for people who turn javascript of.
- anjc 15y ago
  Yeah, there's no law which says your website must be accessible...
  - intranation 15y ago
    Working with Javascript off != accessibility. Most screenreaders (the common face of accessibility technology) run on top of regular browsers, so your JS experience needs to be accessible too if you are legally mandated to (or even better if you care).
    - anjc 15y ago
      If javascript being off makes your site not work, or not viewable at least, then that's the very definition of not being accessible (i'm not limiting 'accessible' to only mean 'available to people with disabilities' like you appear to be) Edit: Graceful degradation should be a tenet of web development. Having no provision for a person who doesn't have javascript enabled seems bizarre to me, and being bullish about the requirement seems even more bizarre...maybe i'm just getting cranky with age or something.
      - scarmig 15y ago

BerislavLopac 15y ago

If it's a true application, with user interaction and everything (and not just a set of glorified Web pages), then the former is not an issue; you don't really want your "pages" to be indexed. And the latter is easily solved with a "loading" bar or any other of UI tools previously invented for the exact same purpose on the desktop.

robmcm 15y ago

Rember this is an application, would a search engine want to crawl the innerds of photoshop or outlook, or a web page promoting them? Just because it's on the web doesn't mean it's a web site that wants to be searchable.

hello_moto 15y ago
In most community-based, social-networking, the companies behind them do want more exposure. The more people who join, the more vibrant the community is. FYI.