33 ms·

Broken Links

303 points by martey 16y ago

il 16y ago
It's interesting how many upvotes this is getting in a very short time. However, I don't think the average Twitter user cares about performance and URL elegance, so I doubt Twitter will change anything.
- deleted 16y ago
  [deleted]
- InfinityX0 16y ago
  You can't get to 1 million pageviews with making a few enemies that figure out a way to game the system occasionally. Not saying this is that, though. Just sayin'.
- macrael 16y ago
  Probably not, but that doesn't mean people who do care shouldn't discuss the implications, or that Twitter shouldn't think there is a problem.
- jamesjyu 16y ago
  I have seen performance issues and outright broken behavior with Twitter's hashbang ajax loading scheme. In that respect, regular users will care (they just won't necessarily know what is causing the issue).
- guelo 16y ago
  Considering that twitter is the main reason for the spread of the abomination that is URL shorteners you're probably right. They don't seem to care about the health of the web.
Bockit 16y ago
Can't it work both ways? Serve the #! links and provide canonical content located at the (almost) same uri sans #!. If you visit http://mysicksite.com/article/1 http://mysicksite.com/article/1 javascript changes all the links to the #! format. Then when the user clicks the links they enter #! land. Now the user copies a link from their address bar and puts it into the wild. Someone gets that link, http://mysicksite.com/#!/article/1 http://mysicksite.com/#!/article/1, and visits it. Rewrite with htaccess or whatever method you employ to serve the content at http://mysicksite.com/article/1 http://mysicksite.com/article/1, using javascript to change all the links to the #! format. I posted this in the reddit thread about the Gawker/lifehacker problems recently, but was too late for anyone to really give me a response. For those of you that have worked with these kind of systems before, would this solve the problem the original link was describing? EDIT: Ahh I think I get the problem now, of course after I post it. Server doesn't get the data from the uri trailing the #! I think?
- s0urceror 16y ago
  That is, indeed, the crux of the problem. Anything after the hash is client-only.
garrettgillas 16y ago
The point of mainstream sites indicating that the page has ajax with the URL path is to tell search engines. I have a feeling that what the author doesn't get is that it is very hard for search engines to tell the difference between ajax pages, static pages, and spammy keyword stuffed pages. To me, it seems that Google recommends indicating ajax content in the path in the same way that our government issues concealed weapon permits. Yes it okay to have concealed content that can loads on the fly as long as you are very clear of your intentions. Once again this is a usability issue that wouldn't be an issue if it weren't for spammers.
zaius 16y ago
I think people are missing a huge benefit of the hashbang syntax: readable and copy/paste-able URLs. Without them, it's impossible to have an ajax application with a decent URL scheme.
- jamesjyu 16y ago
  I think the real question here is whether the application should be loading the main content via AJAX in the first place. Tim argues here that it should not, for this use case.
  - zaius 16y ago
    Then that's what the article should say, instead of hating on the hashbang syntax, which is currently the best solution to the problem. I also don't see why these web pages (specifically twitter) shouldn't be an ajax application - I think loading a page statically, and then the data via ajax is a Good Thing™.
- masklinn 16y ago
  You don't need the hashbang for that. You never did. Hashbang only tells google "munge around this shit to get an actual page".
  - robin_reala 16y ago
    What the hashbang does is update the URI in such a way that it’s possible to bookmark/share a URI (assuming you’ve coded your app properly). It is however just a stopgap measure until pushstate turns up - that way we’ll be able to have URIs that encode the document/state reference in the proper part, not the fragment. Actually, you are right with regards to the bang pash of hashbang - that’s just a Google defined way of letting them access the content.
  - zaius 16y ago
    I'm curious what the other solutions are then. The only one I can think of is the History.pushState, and that's only supported in newer browsers. Let's say I'm writing a web based word processor, and a user clicks on a document. I want the URL to be a reference to that specific document. The only way to change the URL to be specific without requiring a whole-page refresh is to use the hashbang syntax.
othermaciej 16y ago
HTML5 "AJAX History", also known as History.pushState, can solve this problem. It allows a website to update its contents with AJAX, but change the URL to a real URL that will actually retrieve the proper resource direct from the server, while maintaining proper back-forward navigation. See <http://dev.w3.org/html5/spec/Overview.html#dom-history-pushstate&#62 http://dev.w3.org/html5/spec/Overview.html#dom-history-pushs...; for spec details. It's in Safari, Chrome and Firefox. While Opera and IE don't have it yet, it would be easy to use conditionally on browsers that support it. I'm a little surprised that more sites don't use it. EDIT: This chart shows what browsers it will work in: http://caniuse.com/history http://caniuse.com/history
- mjs 16y ago
  This solves two problems: (1) the only visible/bookmarkable URLs are those without a #!; and (2) initial page loads can be fulfilled by a single request to the server. It doesn't solve the problem of URL discovery, but two out of three ain't bad.
  - lyime 16y ago
    Not sure what you mean by URL discovery. Although link/hrefs should be the same/legacy. What you could do is progressively enhance normal links. Javascript could disable the default behavior of a link <a href="/about/team" data-remote="true">About Team</a> and check and see if browser supports History.pushState. If it does it would just request the appropriate content for /about/team and update it client side, then update the url. If pushState is no supported, it could just request that link /about/team normally. This would be the ideal way to support both regular and progressively/JS enhanced pages (for speed).
- JoachimSchipper 16y ago
  Well, it "solves" it - you still have to download and parse a ton of Javascript before you even begin downloading the data...
  - wahnfrieden 16y ago
alexkearns 16y ago
Yet another annoying pontificating article about hashbangs. Why can't people accept that there are more than one way of doing things on the web. Just because you don't like using hashbangs does not mean no-one else can. Sure, use of hashbangs might make seo of your site harder. Yes, it might make it harder for hackers who want to do curls of your site's pages. But maybe this is not your aim with your site. Maybe you want to give your users a slicker experience by not loading whole new pages but instead grabs bits of new content. The web is a place for experimentation and we as hackers should encourage such experimentation, rather than condemning it because it does not fit with how we think things should be done.
- tlack 16y ago
  You can still avoid loading whole new pages. You simply attach Javascript events to your anchor tags and do whatever Ajax content trickery you want that way. The page content itself is maximally flexible and useful to all agents if the URLs inside of it are actual URLs.
  - rimantas 16y ago
    aka "Hijax": http://domscripting.com/blog/display/41 http://domscripting.com/blog/display/41
  - joelanman 16y ago
    The only problem with that is you end up with a mix of both. If a spider collects all the non-ajax links, and shows that to a javascript-enabled browser, the user will end up on eg. /shop/shoes. If the site is ajax enabled for a slicker experience, then as the user browses from here they might get something like this in their address bar: /shop/shoes#!shop/socks or even /shop/shoes#!help/technical which starts to look really weird. The google hashbang spec at least fixes this problem. The spider understands the normal URLs of the app, and will dispatch users to them.
    - Isofarro 16y ago
      Can you not use JavaScript to figure out your URL is a mess and redirect accordingly? JavaScript for redirecting people to the homepage of websites have been available on dynamicdrive.com for at least a decade now. That's one redirect to the homepage (which you're already doing by 301-redirecting the JavaScript-free URLs anyway), so it's hardly going to be difficult. I'm puzzled, considering the haphazard redirects already going on for incoming links to hash-banged sites, why this isn't a trivial problem. Incoming link is to /shop/shoes#!shop/socks JavaScript right at the top of /shop/shoes that window.location to /#!shop/shoes
aamar 16y ago
The problem in this situation is that you have a smart technical person arguing for technical purity, while at the same time (seemingly) ignoring the mostly non-technical considerations of user-experience and economics. Yes, the old, conservative model of HTML is very simple, but when people use AJAX well, the user experience is enormously and materially improved. We're still early in the development of this medium, and many people will do it wrong. But even the people who do it right will probably seem inelegant and kludgey by the standards of the old model. And yes, you can get both AJAX and clean URLs via (still poorly-supported) HTML5 History API and/or other progressive enhancement methods, but these may require a significant amount of additional effort. Maybe worth it, maybe not. This topic reminds me of when sound was added to movies. "Tight coupling" and "hideous kludge" sound a lot like the arguments that were made against that too. The conventional wisdom was to make your talkie such that the story worked even without sound; one can still sometimes hear that, but it isn't, I think, a standard that we associate with the best movies being made today.
pilif 16y ago
With pushState not widely implemented, you have three choices: 1) don't use AJAX in response to actions that alter the page content in a significant way. This of course forces page reloads and prevents the cool emerging pattern that is to not serve dynamic HTML but just have a REST API and do the rendering client side. 2) you do the ajaxy stuff but you don't touch the URL. This leads to a nonworking back button and prevents users from bookmarking or sharing links to specific views. You can work around this google maps style with some explicit "link to this page" functionality, but I would guess, people just don't get that. 3) you do the fragment change thing which allows for ajaxy page content changes but also makes the back button work and keeps links inherently shareable and bookmarkable at the cost of that one redirect, at the cost of screen-scrapability and maybe confusing to techies (normal people probably don't care either way) pushState can look like a salvation, but keep one thing in mind: to keep the page working for browsers without JS (and screen scrapers), you will have to do your work twice and STILL render dynamic content on the server which is something people are now beginning to try to avoid. Finally, as pushState is yet another not widely deployed thing, for the next five to ten years, you would have to do all of this three times: dynamic HTML generation for the purists. pushState for the new browsers and fragment change for IE. Personally, I really feel that fragment change is a good compromise as it works with browsers and even in IE while still allowing the nice pattern of not rendering anything on the server and keeping the URLs shareable. Maybe this current uproar is based on a) techies not used to this (normal people don't notice) and b) badly broken JS that sometimes prevents views from rendering AT ALL, but this is not caused by an inherent problem with the technology: if I screw up the server side rendering the page will be as empty as it is if I screw up on the client side.
- othermaciej 16y ago
  pushState with non-hash URLs doesn't require you to do server-side HTML generation. You can just send a stub page which looks at the URL and loads the right data, just as with hash URLs. To deploy it incrementally, you only really need one code path with a slight fork depending on whether the current URL contains a #! and whether the current browser supports pushState.
  - 16y ago
rushabh 16y ago
I can't understand how hard would it be for someone writing a crawler to replace a hashbang (#!) with _escaped_fragment_ For developers of AJAX apps it: 1. Improves productivity 2. Improves user experience 3. Is more efficient on the server as it prevents a lot of initializing code. I think the old school needs to wake-up a bit!
nostrademons 16y ago
It's not really that bad. The people using hash-bangs are following a spec proposed by Google to make AJAX webpages crawlable: http://code.google.com/web/ajaxcrawling/docs/specification.html http://code.google.com/web/ajaxcrawling/docs/specification.h... So when you see the lifehacker URL in the article, you know that there's an equivalent non-AJAX URL available with the same content at: http://lifehacker.com/?_escaped_fragment_=5753509/hello-world-this-is-the-new-lifehacker http://lifehacker.com/?_escaped_fragment_=5753509/hello-worl... There's no need to execute all the JavaScript that comes back from the server - if they're following the spec, all you have to do is escape the fragment and toss it over to a CGI arg. Another option is progressive enhancement, where you make every link point to a valid page and then add onclick event handlers that override the click event to do whatever JavaScript you want it to. I think this is a far superior option in general, but it has various issues in latency and coding complexity, so a good portion of web developers didn't do it anyway.
- brown9-2 16y ago
  But as Tim says, the spec proposed by Google is only meant to fix some problems (can't be searched by search engines) caused by using this URL scheme. It isn't meant to be a one-guide-fits-all approach making AJAX content addressable. In other words the spec treats one of the symptoms, not the original problem.
- deleted 16y ago
  [deleted]
siddhant 16y ago
Facebook had an outage some time back (I think this one - http://www.facebook.com/note.php?note_id=431441338919 http://www.facebook.com/note.php?note_id=431441338919), and when everything got back to normal, the hash-banged URLs were gone. Was it related?
- robryan 16y ago
  They are still there in IE, so I guess they have started using push state where available.
Isofarro 16y ago
Ran into another interesting shortcoming of hash-bang URLs last night looking through my referrer log. Loads of referring URLs of http://gawker.com/ http://gawker.com/ and http://kotaku.com/ http://kotaku.com/ to my blogpost. But no mention at all of my blog-post or a link to it on the homepage. First I thought they were referrer log spamming, then it dawned on me that fragment identifiers get stripped out of HTTP referers, so making hash-bangs useless as a means of joining up distributed conversations on the web. Somewhere on those two Gawker media sites there's a conversation going on about the use of hash-bangs. But nobody outside knows about it. It's a big black hole.
bruceboughton 16y ago
Isn't the underlying problem that web applications are often displaying combinations of content that doesn't have a natural URL? Take New Twitter, for example. If I click on a tweet in my stream, it shows related tweets. If a drill down a few of those, at some point it becomes impossible to represent the address of the current state in a sane manner. I think URLs are particular to the web (desktop apps don't have them) because the web is traditionally about content. Web applications are increasingly breaking that. Perhaps web applications and URLs don't go together all that well. Don't get me wrong--I love URLs, and it's crazy for content sites like Lifehacker to break them for so little benefit. But maybe the reason for this hashbang trend is that URLs aren't expressive enough for some of these sites.
- prodigal_erik 16y ago
  In that case "web application" is a misnomer. If the current state has no natural URL, it's not a legitimate part of the World-Wide Web. Instead the authors are tunneling a proprietary protocol over AJAX to carry opaque content to a single-purpose GUI app, just like all the terrible client/server apps from the 90s only slower.
  - vannevar 16y ago
    If the current state has no natural URL, it's not a legitimate part of the World-Wide Web. But most of the popular content accessible via the web now fits this description. Look at Google's own homepage, it's a complex Javascript application that's completely opaque. @bruceboughton is right, the problem isn't that people aren't respecting the WWW specification, it's that the specification is no longer adequate to describe what the Web has become.
    - prodigal_erik 16y ago
      Google has competent web developers who practice progressive enhancement. Their search form and results have stable (even sensible) URLs and are perfectly usable without trusting their js.
zemaj 16y ago
Despite all the FUD around hashbangs, the genuine problem I see with them is that they optimise for internal page loads, not the entry into a website. For example with hashbangs, requests to twitter when logged in go like; 1) HTTP GET http://twitter.com/some_account http://twitter.com/some_account [~500ms for me] 2) 302 redirect -> HTTP GET http://twitter.com/ http://twitter.com/ [~600ms for me] 3) HTML tells browser to download some JS -> HTTP GET bundle.js [~500ms for me] (concurrently here we start getting CSS) 4) JS reads hashbang & request actual data -> HTTP GET data.json [~500ms for me] ... only after about 2 seconds can we start(!) rendering data. Now there's about another 2 seconds for all json data & CSS calls to complete. It takes upwards of 4 seconds for a twitter page to render for me (the Load event lies as it fires well before actual data shows. Try it yourself with your favourite browser inspector). When not using hashbangs, a single HTTP request can get all the data for the page and start rendering it. One blocking CSS call (possibly cached) is all that's needed for styling. Hence when I see an external link with a hashbang it frustrates me (barely perceptively) because I know that when I load the page it's going to take a longer than a normal HTTP request. Significantly longer. While subsequent page loads are faster, it's not these you want to optimise for if you care about bounce rates. This issue affects every new link you click into a website, so it affects an even larger number of requests than normal bounces. Hashbangs are a good solution to an important problem, but I don't see them as a tool to build entire websites upon. Fortunately I see the performance issue as one which will result in people voting with their browsers and choosing sites which only use hashbangs when they genuinely improve the user experience - especially since they're easily visible in the url.
- wmf 16y ago
  Basically, NewTwitter isn't a Web site, it's an app and you have to "launch" it before you can do anything.
  - mustpax 16y ago
    But once you do launch it, everything is faster than it would have been if you were performing full page loads at each step. For sites you "live" in, the application route makes a lot of sense. This is the way GMail works and people seem to like it a lot. Unfortunately, web applications and web pages are growing increasingly divergent. It is simply not feasible to take the performance of web apps to the next level without doing away with full page loads. This is why Facebeook, Twitter et al are going the #! route. That's the cold hard truth.
jvdongen 16y ago
[EDIT: never mind, missed this response, similar in style but 2h earlier ... http://news.ycombinator.com/item?id=2197064 http://news.ycombinator.com/item?id=2197064] May be I'm missing something, but it seems to me that there is a way to have your cake and eat it too in this case. Say we have a site with a page /contacts/ which lists various contacts. On this page there are completely normal links like '/contacts/john/', each link preceded by/wrapped by an anchor tag - <a href="john"> in this case. If you visit this site without javascript enabled (e.g. you happen to be a web crawler), you just follow the links and you get just regular pages as always. If however you've javascript enabled, onclick events on each url intercept a click on a link and fetch just the information about the contact you clicked on (using an alternate url, for example /contacts/john.json), cancels the default action and (re)renders the page. Then it does one of two things: - if pushState is supported it just updates the url - if pushState is not supported it adds '#john' to the url If someone visits '/contacts/#john' with javascript enabled, /contacts/ is retrieved and then john's data is loaded and displayed. If someone visits '/contacts/#john' without javascript enabled, he gets the full contact list, with the focus on the link to john's page, which he can then click. By using this scheme: - search engine and other non-javascript users can fully use the site and see completely normal urls - XHR page loads are supported - XHR loaded pages don't break the backbutton - XHR loaded pages are bookmarkable - Bookmarks to XHR loaded pages are fully shareable if the recipient has javascript enabled or pushState is supported, and at least not totally broken if not. The only drawback I can see is the 'sharing bookmarks with someone who has no javascript support' issue - is that a real biggie? In addition of course to the 'made error in javascript, now all stops working' issue - but that is something that has not so much do with the #! debate as well as with the 'is loading primary content via XHR a good idea' debate. To me it seems that current users of the #! technique have just gone overboard a bit by relying only on the #! technique instead of combining it in a progressively enhancing way with regular HTTP requests.
jcfrei 16y ago
Just a thought - but could a lot of people complaining about hashbangs still be browsing the web with lynx?
zachbeane 16y ago
This rant would be more effective and persuasive if also directed at the Google engineers who made this hashbang style pervasive in Google Groups. I didn't think it would be possible to get deep links to old articles even worse than before, but they managed it.
dtby 16y ago
Hi, HTML/HTTP are the second worse application delivery platform available. Try not to be shocked. Sorry, your other choice was #1.
vanessafox 16y ago
I posted more as a comment on the original story, but I have covered this issue in depth (from when Google initially proposed it, to when it was launched) here: http://searchengineland.com/google-proposes-to-make-ajax-crawlable-27408 http://searchengineland.com/google-proposes-to-make-ajax-cra... http://searchengineland.com/googles-proposal-for-crawling-ajax-may-be-live-34411 http://searchengineland.com/googles-proposal-for-crawling-aj... http://searchengineland.com/its-official-googles-proposal-for-crawling-ajax-urls-is-live-37298 http://searchengineland.com/its-official-googles-proposal-fo... Of course, a better solution is some type of progressive enhancement that ensures both that search engines can crawl the URLs and anyone using device without JavaScript support can view all of the content and navigate the site.

303 points by martey 16y ago

il 16y ago

It's interesting how many upvotes this is getting in a very short time. However, I don't think the average Twitter user cares about performance and URL elegance, so I doubt Twitter will change anything.

deleted 16y ago
[deleted]
InfinityX0 16y ago
You can't get to 1 million pageviews with making a few enemies that figure out a way to game the system occasionally. Not saying this is that, though. Just sayin'.
macrael 16y ago
Probably not, but that doesn't mean people who do care shouldn't discuss the implications, or that Twitter shouldn't think there is a problem.
jamesjyu 16y ago
I have seen performance issues and outright broken behavior with Twitter's hashbang ajax loading scheme. In that respect, regular users will care (they just won't necessarily know what is causing the issue).
guelo 16y ago
Considering that twitter is the main reason for the spread of the abomination that is URL shorteners you're probably right. They don't seem to care about the health of the web.

Bockit 16y ago

Can't it work both ways? Serve the #! links and provide canonical content located at the (almost) same uri sans #!. If you visit http://mysicksite.com/article/1 http://mysicksite.com/article/1 javascript changes all the links to the #! format. Then when the user clicks the links they enter #! land. Now the user copies a link from their address bar and puts it into the wild. Someone gets that link, http://mysicksite.com/#!/article/1 http://mysicksite.com/#!/article/1, and visits it. Rewrite with htaccess or whatever method you employ to serve the content at http://mysicksite.com/article/1 http://mysicksite.com/article/1, using javascript to change all the links to the #! format. I posted this in the reddit thread about the Gawker/lifehacker problems recently, but was too late for anyone to really give me a response. For those of you that have worked with these kind of systems before, would this solve the problem the original link was describing? EDIT: Ahh I think I get the problem now, of course after I post it. Server doesn't get the data from the uri trailing the #! I think?

s0urceror 16y ago
That is, indeed, the crux of the problem. Anything after the hash is client-only.

garrettgillas 16y ago

The point of mainstream sites indicating that the page has ajax with the URL path is to tell search engines. I have a feeling that what the author doesn't get is that it is very hard for search engines to tell the difference between ajax pages, static pages, and spammy keyword stuffed pages. To me, it seems that Google recommends indicating ajax content in the path in the same way that our government issues concealed weapon permits. Yes it okay to have concealed content that can loads on the fly as long as you are very clear of your intentions. Once again this is a usability issue that wouldn't be an issue if it weren't for spammers.

zaius 16y ago

I think people are missing a huge benefit of the hashbang syntax: readable and copy/paste-able URLs. Without them, it's impossible to have an ajax application with a decent URL scheme.

jamesjyu 16y ago
I think the real question here is whether the application should be loading the main content via AJAX in the first place. Tim argues here that it should not, for this use case.
- zaius 16y ago
  Then that's what the article should say, instead of hating on the hashbang syntax, which is currently the best solution to the problem. I also don't see why these web pages (specifically twitter) shouldn't be an ajax application - I think loading a page statically, and then the data via ajax is a Good Thing™.
masklinn 16y ago
You don't need the hashbang for that. You never did. Hashbang only tells google "munge around this shit to get an actual page".
- robin_reala 16y ago
  What the hashbang does is update the URI in such a way that it’s possible to bookmark/share a URI (assuming you’ve coded your app properly). It is however just a stopgap measure until pushstate turns up - that way we’ll be able to have URIs that encode the document/state reference in the proper part, not the fragment. Actually, you are right with regards to the bang pash of hashbang - that’s just a Google defined way of letting them access the content.
- zaius 16y ago
  I'm curious what the other solutions are then. The only one I can think of is the History.pushState, and that's only supported in newer browsers. Let's say I'm writing a web based word processor, and a user clicks on a document. I want the URL to be a reference to that specific document. The only way to change the URL to be specific without requiring a whole-page refresh is to use the hashbang syntax.

othermaciej 16y ago

HTML5 "AJAX History", also known as History.pushState, can solve this problem. It allows a website to update its contents with AJAX, but change the URL to a real URL that will actually retrieve the proper resource direct from the server, while maintaining proper back-forward navigation. See <http://dev.w3.org/html5/spec/Overview.html#dom-history-pushstate&#62 http://dev.w3.org/html5/spec/Overview.html#dom-history-pushs...; for spec details. It's in Safari, Chrome and Firefox. While Opera and IE don't have it yet, it would be easy to use conditionally on browsers that support it. I'm a little surprised that more sites don't use it. EDIT: This chart shows what browsers it will work in: http://caniuse.com/history http://caniuse.com/history

mjs 16y ago
This solves two problems: (1) the only visible/bookmarkable URLs are those without a #!; and (2) initial page loads can be fulfilled by a single request to the server. It doesn't solve the problem of URL discovery, but two out of three ain't bad.
- lyime 16y ago
  Not sure what you mean by URL discovery. Although link/hrefs should be the same/legacy. What you could do is progressively enhance normal links. Javascript could disable the default behavior of a link <a href="/about/team" data-remote="true">About Team</a> and check and see if browser supports History.pushState. If it does it would just request the appropriate content for /about/team and update it client side, then update the url. If pushState is no supported, it could just request that link /about/team normally. This would be the ideal way to support both regular and progressively/JS enhanced pages (for speed).
JoachimSchipper 16y ago
Well, it "solves" it - you still have to download and parse a ton of Javascript before you even begin downloading the data...
- wahnfrieden 16y ago

alexkearns 16y ago

Yet another annoying pontificating article about hashbangs. Why can't people accept that there are more than one way of doing things on the web. Just because you don't like using hashbangs does not mean no-one else can. Sure, use of hashbangs might make seo of your site harder. Yes, it might make it harder for hackers who want to do curls of your site's pages. But maybe this is not your aim with your site. Maybe you want to give your users a slicker experience by not loading whole new pages but instead grabs bits of new content. The web is a place for experimentation and we as hackers should encourage such experimentation, rather than condemning it because it does not fit with how we think things should be done.

tlack 16y ago
You can still avoid loading whole new pages. You simply attach Javascript events to your anchor tags and do whatever Ajax content trickery you want that way. The page content itself is maximally flexible and useful to all agents if the URLs inside of it are actual URLs.
- rimantas 16y ago
  aka "Hijax": http://domscripting.com/blog/display/41 http://domscripting.com/blog/display/41
- joelanman 16y ago
  The only problem with that is you end up with a mix of both. If a spider collects all the non-ajax links, and shows that to a javascript-enabled browser, the user will end up on eg. /shop/shoes. If the site is ajax enabled for a slicker experience, then as the user browses from here they might get something like this in their address bar: /shop/shoes#!shop/socks or even /shop/shoes#!help/technical which starts to look really weird. The google hashbang spec at least fixes this problem. The spider understands the normal URLs of the app, and will dispatch users to them.
  - Isofarro 16y ago
    Can you not use JavaScript to figure out your URL is a mess and redirect accordingly? JavaScript for redirecting people to the homepage of websites have been available on dynamicdrive.com for at least a decade now. That's one redirect to the homepage (which you're already doing by 301-redirecting the JavaScript-free URLs anyway), so it's hardly going to be difficult. I'm puzzled, considering the haphazard redirects already going on for incoming links to hash-banged sites, why this isn't a trivial problem. Incoming link is to /shop/shoes#!shop/socks JavaScript right at the top of /shop/shoes that window.location to /#!shop/shoes

aamar 16y ago

The problem in this situation is that you have a smart technical person arguing for technical purity, while at the same time (seemingly) ignoring the mostly non-technical considerations of user-experience and economics. Yes, the old, conservative model of HTML is very simple, but when people use AJAX well, the user experience is enormously and materially improved. We're still early in the development of this medium, and many people will do it wrong. But even the people who do it right will probably seem inelegant and kludgey by the standards of the old model. And yes, you can get both AJAX and clean URLs via (still poorly-supported) HTML5 History API and/or other progressive enhancement methods, but these may require a significant amount of additional effort. Maybe worth it, maybe not. This topic reminds me of when sound was added to movies. "Tight coupling" and "hideous kludge" sound a lot like the arguments that were made against that too. The conventional wisdom was to make your talkie such that the story worked even without sound; one can still sometimes hear that, but it isn't, I think, a standard that we associate with the best movies being made today.

pilif 16y ago

With pushState not widely implemented, you have three choices: 1) don't use AJAX in response to actions that alter the page content in a significant way. This of course forces page reloads and prevents the cool emerging pattern that is to not serve dynamic HTML but just have a REST API and do the rendering client side. 2) you do the ajaxy stuff but you don't touch the URL. This leads to a nonworking back button and prevents users from bookmarking or sharing links to specific views. You can work around this google maps style with some explicit "link to this page" functionality, but I would guess, people just don't get that. 3) you do the fragment change thing which allows for ajaxy page content changes but also makes the back button work and keeps links inherently shareable and bookmarkable at the cost of that one redirect, at the cost of screen-scrapability and maybe confusing to techies (normal people probably don't care either way) pushState can look like a salvation, but keep one thing in mind: to keep the page working for browsers without JS (and screen scrapers), you will have to do your work twice and STILL render dynamic content on the server which is something people are now beginning to try to avoid. Finally, as pushState is yet another not widely deployed thing, for the next five to ten years, you would have to do all of this three times: dynamic HTML generation for the purists. pushState for the new browsers and fragment change for IE. Personally, I really feel that fragment change is a good compromise as it works with browsers and even in IE while still allowing the nice pattern of not rendering anything on the server and keeping the URLs shareable. Maybe this current uproar is based on a) techies not used to this (normal people don't notice) and b) badly broken JS that sometimes prevents views from rendering AT ALL, but this is not caused by an inherent problem with the technology: if I screw up the server side rendering the page will be as empty as it is if I screw up on the client side.

othermaciej 16y ago
pushState with non-hash URLs doesn't require you to do server-side HTML generation. You can just send a stub page which looks at the URL and loads the right data, just as with hash URLs. To deploy it incrementally, you only really need one code path with a slight fork depending on whether the current URL contains a #! and whether the current browser supports pushState.
- 16y ago

rushabh 16y ago

I can't understand how hard would it be for someone writing a crawler to replace a hashbang (#!) with _escaped_fragment_ For developers of AJAX apps it: 1. Improves productivity 2. Improves user experience 3. Is more efficient on the server as it prevents a lot of initializing code. I think the old school needs to wake-up a bit!

nostrademons 16y ago

It's not really that bad. The people using hash-bangs are following a spec proposed by Google to make AJAX webpages crawlable: http://code.google.com/web/ajaxcrawling/docs/specification.html http://code.google.com/web/ajaxcrawling/docs/specification.h... So when you see the lifehacker URL in the article, you know that there's an equivalent non-AJAX URL available with the same content at: http://lifehacker.com/?_escaped_fragment_=5753509/hello-world-this-is-the-new-lifehacker http://lifehacker.com/?_escaped_fragment_=5753509/hello-worl... There's no need to execute all the JavaScript that comes back from the server - if they're following the spec, all you have to do is escape the fragment and toss it over to a CGI arg. Another option is progressive enhancement, where you make every link point to a valid page and then add onclick event handlers that override the click event to do whatever JavaScript you want it to. I think this is a far superior option in general, but it has various issues in latency and coding complexity, so a good portion of web developers didn't do it anyway.

brown9-2 16y ago
But as Tim says, the spec proposed by Google is only meant to fix some problems (can't be searched by search engines) caused by using this URL scheme. It isn't meant to be a one-guide-fits-all approach making AJAX content addressable. In other words the spec treats one of the symptoms, not the original problem.
deleted 16y ago
[deleted]

siddhant 16y ago

Facebook had an outage some time back (I think this one - http://www.facebook.com/note.php?note_id=431441338919 http://www.facebook.com/note.php?note_id=431441338919), and when everything got back to normal, the hash-banged URLs were gone. Was it related?

robryan 16y ago
They are still there in IE, so I guess they have started using push state where available.

Isofarro 16y ago

Ran into another interesting shortcoming of hash-bang URLs last night looking through my referrer log. Loads of referring URLs of http://gawker.com/ http://gawker.com/ and http://kotaku.com/ http://kotaku.com/ to my blogpost. But no mention at all of my blog-post or a link to it on the homepage. First I thought they were referrer log spamming, then it dawned on me that fragment identifiers get stripped out of HTTP referers, so making hash-bangs useless as a means of joining up distributed conversations on the web. Somewhere on those two Gawker media sites there's a conversation going on about the use of hash-bangs. But nobody outside knows about it. It's a big black hole.

bruceboughton 16y ago

Isn't the underlying problem that web applications are often displaying combinations of content that doesn't have a natural URL? Take New Twitter, for example. If I click on a tweet in my stream, it shows related tweets. If a drill down a few of those, at some point it becomes impossible to represent the address of the current state in a sane manner. I think URLs are particular to the web (desktop apps don't have them) because the web is traditionally about content. Web applications are increasingly breaking that. Perhaps web applications and URLs don't go together all that well. Don't get me wrong--I love URLs, and it's crazy for content sites like Lifehacker to break them for so little benefit. But maybe the reason for this hashbang trend is that URLs aren't expressive enough for some of these sites.

prodigal_erik 16y ago
In that case "web application" is a misnomer. If the current state has no natural URL, it's not a legitimate part of the World-Wide Web. Instead the authors are tunneling a proprietary protocol over AJAX to carry opaque content to a single-purpose GUI app, just like all the terrible client/server apps from the 90s only slower.
- vannevar 16y ago
  If the current state has no natural URL, it's not a legitimate part of the World-Wide Web. But most of the popular content accessible via the web now fits this description. Look at Google's own homepage, it's a complex Javascript application that's completely opaque. @bruceboughton is right, the problem isn't that people aren't respecting the WWW specification, it's that the specification is no longer adequate to describe what the Web has become.
  - prodigal_erik 16y ago
    Google has competent web developers who practice progressive enhancement. Their search form and results have stable (even sensible) URLs and are perfectly usable without trusting their js.

zemaj 16y ago

Despite all the FUD around hashbangs, the genuine problem I see with them is that they optimise for internal page loads, not the entry into a website. For example with hashbangs, requests to twitter when logged in go like; 1) HTTP GET http://twitter.com/some_account http://twitter.com/some_account [~500ms for me] 2) 302 redirect -> HTTP GET http://twitter.com/ http://twitter.com/ [~600ms for me] 3) HTML tells browser to download some JS -> HTTP GET bundle.js [~500ms for me] (concurrently here we start getting CSS) 4) JS reads hashbang & request actual data -> HTTP GET data.json [~500ms for me] ... only after about 2 seconds can we start(!) rendering data. Now there's about another 2 seconds for all json data & CSS calls to complete. It takes upwards of 4 seconds for a twitter page to render for me (the Load event lies as it fires well before actual data shows. Try it yourself with your favourite browser inspector). When not using hashbangs, a single HTTP request can get all the data for the page and start rendering it. One blocking CSS call (possibly cached) is all that's needed for styling. Hence when I see an external link with a hashbang it frustrates me (barely perceptively) because I know that when I load the page it's going to take a longer than a normal HTTP request. Significantly longer. While subsequent page loads are faster, it's not these you want to optimise for if you care about bounce rates. This issue affects every new link you click into a website, so it affects an even larger number of requests than normal bounces. Hashbangs are a good solution to an important problem, but I don't see them as a tool to build entire websites upon. Fortunately I see the performance issue as one which will result in people voting with their browsers and choosing sites which only use hashbangs when they genuinely improve the user experience - especially since they're easily visible in the url.

wmf 16y ago
Basically, NewTwitter isn't a Web site, it's an app and you have to "launch" it before you can do anything.
- mustpax 16y ago
  But once you do launch it, everything is faster than it would have been if you were performing full page loads at each step. For sites you "live" in, the application route makes a lot of sense. This is the way GMail works and people seem to like it a lot. Unfortunately, web applications and web pages are growing increasingly divergent. It is simply not feasible to take the performance of web apps to the next level without doing away with full page loads. This is why Facebeook, Twitter et al are going the #! route. That's the cold hard truth.

jvdongen 16y ago

[EDIT: never mind, missed this response, similar in style but 2h earlier ... http://news.ycombinator.com/item?id=2197064 http://news.ycombinator.com/item?id=2197064] May be I'm missing something, but it seems to me that there is a way to have your cake and eat it too in this case. Say we have a site with a page /contacts/ which lists various contacts. On this page there are completely normal links like '/contacts/john/', each link preceded by/wrapped by an anchor tag - <a href="john"> in this case. If you visit this site without javascript enabled (e.g. you happen to be a web crawler), you just follow the links and you get just regular pages as always. If however you've javascript enabled, onclick events on each url intercept a click on a link and fetch just the information about the contact you clicked on (using an alternate url, for example /contacts/john.json), cancels the default action and (re)renders the page. Then it does one of two things: - if pushState is supported it just updates the url - if pushState is not supported it adds '#john' to the url If someone visits '/contacts/#john' with javascript enabled, /contacts/ is retrieved and then john's data is loaded and displayed. If someone visits '/contacts/#john' without javascript enabled, he gets the full contact list, with the focus on the link to john's page, which he can then click. By using this scheme: - search engine and other non-javascript users can fully use the site and see completely normal urls - XHR page loads are supported - XHR loaded pages don't break the backbutton - XHR loaded pages are bookmarkable - Bookmarks to XHR loaded pages are fully shareable if the recipient has javascript enabled or pushState is supported, and at least not totally broken if not. The only drawback I can see is the 'sharing bookmarks with someone who has no javascript support' issue - is that a real biggie? In addition of course to the 'made error in javascript, now all stops working' issue - but that is something that has not so much do with the #! debate as well as with the 'is loading primary content via XHR a good idea' debate. To me it seems that current users of the #! technique have just gone overboard a bit by relying only on the #! technique instead of combining it in a progressively enhancing way with regular HTTP requests.

jcfrei 16y ago

Just a thought - but could a lot of people complaining about hashbangs still be browsing the web with lynx?

zachbeane 16y ago

This rant would be more effective and persuasive if also directed at the Google engineers who made this hashbang style pervasive in Google Groups. I didn't think it would be possible to get deep links to old articles even worse than before, but they managed it.

dtby 16y ago

Hi, HTML/HTTP are the second worse application delivery platform available. Try not to be shocked. Sorry, your other choice was #1.

vanessafox 16y ago

I posted more as a comment on the original story, but I have covered this issue in depth (from when Google initially proposed it, to when it was launched) here: http://searchengineland.com/google-proposes-to-make-ajax-crawlable-27408 http://searchengineland.com/google-proposes-to-make-ajax-cra... http://searchengineland.com/googles-proposal-for-crawling-ajax-may-be-live-34411 http://searchengineland.com/googles-proposal-for-crawling-aj... http://searchengineland.com/its-official-googles-proposal-for-crawling-ajax-urls-is-live-37298 http://searchengineland.com/its-official-googles-proposal-fo... Of course, a better solution is some type of progressive enhancement that ensures both that search engines can crawl the URLs and anyone using device without JavaScript support can view all of the content and navigate the site.