4 ms·

Show HN: Two-faced URLs

85 points by ipsin 15y ago

inportb 15y ago
Clever use of content negotiation!
ipsin 15y ago
This is a URL shortener that gives a link that sends "preview" processes one place and actual users somewhere else. It works for google plus and facebook and several other "preview" bots. Proof-of-concept because I don't like URL shorteners in general, but it links to a restricted list of sites.
makethetick 15y ago
Very clever/sneaky!
rmccue 15y ago
At a guess: HEAD requests give one URL, GET requests give another?
- MattBearman 15y ago
  I think it knows the user agent of specific bots and serves different pages based on that, as social networks tend to do a full GET request in order to grab a screenshot.
ipsin 15y ago
It's actually several things, including user agent strings, HTTP accept/accept language. The underlying script allows you to route between n different destinations based on header matching and netmasks. The shortened URL is just a hash of the rule set, with a sequence number appended in case of collisions.
- phillco 15y ago
  Clever. I assume the only way for Facebook and Twitter to circumvent this would be to fake all of the HTTP headers sent by, say, a modern version of Firefox. I don't think it's cheating - even though the tools are bots, their job is to preview the content their users are about to see. Even then, the author could probably find out which IP addresses are used by Twitter and Facebook's lookup engines and send them the innocuous version. However, I just did a test on my own server, and Facebook used two different IPs (69.171.228.246 and 69.171.224.245) for the same request. Pretty big range.
jasondavies 15y ago
To defend against this, social networking sites should simply cache the link that the preview bot sees, and send users there directly instead.
pak 15y ago
Yet another reason for URL shorteners to be banished from the web (not that this will ever happen). Besides the arcane input requirements of Twitter and the semi-retarded behavior of certain email clients, I haven't found a good use case for them that couldn't have been solved better with proper link formatting or better content presentation. When the web is just a sea of opaque pointers to pointers to pointers, with various hilarity like these URLs mixed in, we'll all wonder why the hell we stood by and let it happen.
- covati 15y ago
  I'm biased, since I have run more than one shortener/tracker myself. This isn't meant to be a rant, more of a defense for the uninformed. It's a bit naive to think that most websites are referenced by a simple link to static content. The web has become so complex and urls now reflect that. Short urls are useful for many people for many different purposes. For an example of how complex urls are, take a look at SEO'd blog post or newspaper urls. They have become a string on keyword rich terms that help to increase SEO. The content that is served is brought up dynamically and may in fact change over time. Ads, sidebars, and comments are brought in after the fact, often based on content or origin of the user or if they have a cookie. It's complicated - url redirection (which already may be happening in this flow) doesn't add much to it. Advocating for "proper link formatting or better content presentation" reflects the fact that you've never had to dig deep into the complex SEO, user, customization, and other general business requirements that arise when running a CMS. Short urls are valuable because they allow people an ability to simplify the increasingly long urls for use in many different mediums without the worry of breaking or losing portions of the url. Yes, they often do allow the creator to change them, but that's no different than what any website operator can do natively on their website. There are problematic situations that can arise, such as when a url shortener goes down. And this has happened, and it hasn't caused any catastrophic problems. It's inconvenient, but it often only affects old content on twitter or facebook. I'm not saying it's ideal, it definitely doesn't add value to the internet. But it's not that bad. Regarding the 'arcane' input requirements for twitter or facebook. Somehow I think you missed the point of these networks. Twitters 140 characters is part of what defines them. There will always be short message mediums, they are there because people have a desire for short bursts of information. They aren't going away. I hope that helps to explain why shorteners arose because there was a need, not just to add complexity or more evil to the internet.
lbarrow 15y ago
This is great! Lots of fun.
billpg 15y ago
That's one thing that mildly annoys me about twitter. It clearly knows where a shortened link is heading because it reveals the full URL on the mouseover. Given that information, why does it need the shortened link any more?
lawn 15y ago
I've been waiting for this since url shorteners came to be. Sadly I don't think shorteners will loose popularity anytime soon and I will continue to avoid them as I've always done.
bgarbiak 15y ago
This actually could be quite useful. Sometimes people need to post a nicely described link with an adequate thumbnail, yet the Facebook bot finds only some random image. With this little service one can prepare a dummy page for illustrative purposes and then let brokenthings.org handle the bot and the redirection.
TobbenTM 15y ago
This is not working for me.. Whenever I change the source link, Facebook gives me no preview..

85 points by ipsin 15y ago

Clever use of content negotiation!

ipsin 15y ago

This is a URL shortener that gives a link that sends "preview" processes one place and actual users somewhere else. It works for google plus and facebook and several other "preview" bots. Proof-of-concept because I don't like URL shorteners in general, but it links to a restricted list of sites.

makethetick 15y ago

Very clever/sneaky!

rmccue 15y ago

At a guess: HEAD requests give one URL, GET requests give another?

MattBearman 15y ago
I think it knows the user agent of specific bots and serves different pages based on that, as social networks tend to do a full GET request in order to grab a screenshot.

ipsin 15y ago

It's actually several things, including user agent strings, HTTP accept/accept language. The underlying script allows you to route between n different destinations based on header matching and netmasks. The shortened URL is just a hash of the rule set, with a sequence number appended in case of collisions.

phillco 15y ago
Clever. I assume the only way for Facebook and Twitter to circumvent this would be to fake all of the HTTP headers sent by, say, a modern version of Firefox. I don't think it's cheating - even though the tools are bots, their job is to preview the content their users are about to see. Even then, the author could probably find out which IP addresses are used by Twitter and Facebook's lookup engines and send them the innocuous version. However, I just did a test on my own server, and Facebook used two different IPs (69.171.228.246 and 69.171.224.245) for the same request. Pretty big range.

jasondavies 15y ago

To defend against this, social networking sites should simply cache the link that the preview bot sees, and send users there directly instead.

pak 15y ago

Yet another reason for URL shorteners to be banished from the web (not that this will ever happen). Besides the arcane input requirements of Twitter and the semi-retarded behavior of certain email clients, I haven't found a good use case for them that couldn't have been solved better with proper link formatting or better content presentation. When the web is just a sea of opaque pointers to pointers to pointers, with various hilarity like these URLs mixed in, we'll all wonder why the hell we stood by and let it happen.

covati 15y ago
I'm biased, since I have run more than one shortener/tracker myself. This isn't meant to be a rant, more of a defense for the uninformed. It's a bit naive to think that most websites are referenced by a simple link to static content. The web has become so complex and urls now reflect that. Short urls are useful for many people for many different purposes. For an example of how complex urls are, take a look at SEO'd blog post or newspaper urls. They have become a string on keyword rich terms that help to increase SEO. The content that is served is brought up dynamically and may in fact change over time. Ads, sidebars, and comments are brought in after the fact, often based on content or origin of the user or if they have a cookie. It's complicated - url redirection (which already may be happening in this flow) doesn't add much to it. Advocating for "proper link formatting or better content presentation" reflects the fact that you've never had to dig deep into the complex SEO, user, customization, and other general business requirements that arise when running a CMS. Short urls are valuable because they allow people an ability to simplify the increasingly long urls for use in many different mediums without the worry of breaking or losing portions of the url. Yes, they often do allow the creator to change them, but that's no different than what any website operator can do natively on their website. There are problematic situations that can arise, such as when a url shortener goes down. And this has happened, and it hasn't caused any catastrophic problems. It's inconvenient, but it often only affects old content on twitter or facebook. I'm not saying it's ideal, it definitely doesn't add value to the internet. But it's not that bad. Regarding the 'arcane' input requirements for twitter or facebook. Somehow I think you missed the point of these networks. Twitters 140 characters is part of what defines them. There will always be short message mediums, they are there because people have a desire for short bursts of information. They aren't going away. I hope that helps to explain why shorteners arose because there was a need, not just to add complexity or more evil to the internet.

lbarrow 15y ago

This is great! Lots of fun.

billpg 15y ago

That's one thing that mildly annoys me about twitter. It clearly knows where a shortened link is heading because it reveals the full URL on the mouseover. Given that information, why does it need the shortened link any more?

lawn 15y ago

I've been waiting for this since url shorteners came to be. Sadly I don't think shorteners will loose popularity anytime soon and I will continue to avoid them as I've always done.

bgarbiak 15y ago

This actually could be quite useful. Sometimes people need to post a nicely described link with an adequate thumbnail, yet the Facebook bot finds only some random image. With this little service one can prepare a dummy page for illustrative purposes and then let brokenthings.org handle the bot and the redirection.

TobbenTM 15y ago

This is not working for me.. Whenever I change the source link, Facebook gives me no preview..