7 ms·
To this day, my most public contribution to reddit is that I wrote the code to put the title of the post in the URL. That was done specifically for SEO purpose
by jedberg 3y ago
To this day, my most public contribution to reddit is that I wrote the code to put the title of the post in the URL. That was done specifically for SEO purposes.
It was pretty much the only SEO optimization we ever did (along with a few DOM changes), because shortly after that, Google basically dedicated engineering effort specifically to crawling reddit. So much so that we lost the "crawl rate" button in our SEO admin page on Google, it was just set to "Custom".
I had to stand up a fleet of app servers and caches and databases, and change the load balancers so that Google basically had their own infrastructure (although we would shunt all crawlers there). Crawler traffic was very different than regular traffic -- it looked at pages more than two days old, something humans rarely did at the time. It would blow out every cache (memory, database, disk, etc.). So we put them on their own infra to keep them from killing the rest of the site.
And now I will hang an onion on my belt, which was the style at the time.
- gary_0 3y ago> That was done specifically for SEO purposes. As a human I also like it when I can see what topic a URL points to (eg. when it's being shared on HN, or pasted into source code, etc). URLs that just have an opaque ID or terse filename are not very user-friendly.
- jedberg 3y agoOh yeah, it turned out to be great for humans. It just happened to be the one thing that we specifically did for SEO first. Funny enough, it turns out that most "SEO optimizations" are just "make this better for humans, especially disabled humans".
- deltarholamda 3y ago>SEO optimizations are just "make this better for humans, especially disabled humans" I'm gonna embroider that on a throw pillow and use it to hit webdevs.
- jedberg 3y agoPlease do! And please send me a picture if you actually do it. :)
- gary_0 3y agoI see. I suppose in a perfect world, devs would just follow general best practices and search engines would reward that, and nobody would be looking for evil voodoo to boost traffic. And I would have a pet unicorn!
- rasz 3y agoNowadays reddit hides it, I use this to revert back to full url for (let value of document.querySelectorAll('.first a.bylink.comments')) { let href = value.closest('.top-matter').querySelector('.title a.title.may-blank').href; if (href != value.href && ( href.startsWith('https://www.reddit.com/') || href.startsWith('https://v.redd.it')) ) { console.log (href, value.href); value.closest('.top-matter').querySelector('.title a.title.may-blank').href = value.href; } }
- deleted 3y ago[deleted]
- jedberg 3y agoWhat do you mean the hide it now? I still see it in the URL?
- rasz 3y agoThis is what I see https://v.redd.it/pzma14ni23vc1 https://v.redd.it/pzma14ni23vc1 for post titles, while comment link https://www.reddit.com/r/UkraineWarVideoReport/comments/1c6hddm/the_most_vulnerable_spot_for_the_ruzzian_t72_tank/ https://www.reddit.com/r/UkraineWarVideoReport/comments/1c6h... First one is also dynamic, so next ~day its different breaking one of the fundamental features of HTML - :visited link color.
- ammar2 3y agoSaying they hide it might be a bit of a stretch, that URL just 302's to the comment link with the title-in-url as usual. I guess you're referring to being able to right-click and copy the url directly?
- xsmasher 3y agoI agree; I wish JIRA did this. I actually asked for this feature in their support forum yers ago and got a chorus of "that's stupid, why would anyone want that?" in reply.
- mleo 3y agoThis is similar to when people create branches with just the Jira ID instead of using the ID and title slug as branch name. I’m often working on 3+ items at a time and need a quick way to know which branch is which; how do people that just use the ID as branch name know what the code changes are?
- antifa 3y agoJIRA is what comes to mind when I think of software that does really stupid things nobody would ever want. Want to middle click this one specific link to open it in a background tab? Instead of respecting default browser behavior the user is expecting, JIRA happily hijacks the mouse down event to open your background link in the foreground instead, which prevents you from opening multiple links in a convenient timely manor. A deployment happened on December 29th. It's January 2th. Suddenly, you need to know if the deployment happened 1 week ago or 1 month ago. You remember the deployments page usually says useful information such as X days ago or X weeks ago. Based on the facts, a rational normal person is expecting something like 4 days ago. JIRA says "last year".
- saurik 3y agoThe problem is you can't trust it: people can modify that part of the URL arbitrarily and so you think the URL has meaning, but it doesn't.
- kevincox 3y agoIt depends how it is implemented. After extracting the ID and looking up the post you can then check that the "slug" matches. Return a 404 if it doesn't. This may add a bit of complexity (if the slug data is mutable you need to allow any past versions or freeze it in creation) but isn't too hard if this is the approach that you want to take.
- saurik 3y agoOk, but we don't have to talk about this "depends" idea: that's not how reddit did it and I thought we were talking about reddit ;P.
- antifa 3y agoI'd still rather be saved from a bad click 99% of the time, rather than the 0% of the time an unlabeled ID link will give you.
- saurik 3y agoFWIW there was a good discussion of this on r/admin last month. https://www.reddit.com/r/admin/comments/1c6lp0z/link_slug_policy_updates/ https://www.reddit.com/r/admin/comments/1c6lp0z/link_slug_po...
- mark242 3y agoIt's mind-boggling to me that Google didn't create a spec to work with special partners whereby they could get syndicated data in a format that was easy to ingest, and easy for their partners to produce. Lighting dollars on fire just to serve up pages to Googlebot, when you could just periodically dump a journal of updates to Google, is just crazy imo. On edit: if only there were some way to do some kind of really simple syndication of your data.
- ivanjermakov 3y agoAlso, isn't this illegal? Either Google respected rate limits (which I doubt) or it was a form of a DDoS attack.
- dylan604 3y agoIt's only illegal if you poor people do it. For us rich corporations, we do what we want
- aeyes 3y agoI guess they wanted to be crawled because the Google crawler is easy to identify/block/rate limit.
- nosecreek 3y agoAre there specific laws that deal with rate limits? Honest question - I get that something too fast could be considered DDoS, but so long as it’s below a certain threshold wouldn’t it be okay (not sure how said threshold would be determined)?
- germinator 3y agoIn the US, CFAA prohibits causing "damage", which includes "impairment to the integrity or availability" of data or systems. But as with many other things in law, it boils down to the court trying to assess your intent, whether you could've reasonably anticipated the outcome, and what that outcome ended up being. There's no law that says "you can't send more than n packets per hour".
- asdff 3y agoNowadays the reddit engineers perverted the link system for old reddit users. New reddit users link to posts using a syntax that for old reddit users returns the submit a new post page. Also, just to twist the knife further, old reddit users can't open full resolution images hosted on reddit anymore. Click a link and you get thrown to this url: https://old.reddit.com/r/funny/comments/media/nice_hat/?url=https%3A%2F%2Fi.redd.it%2Flgvaz37c21vc1.png https://old.reddit.com/r/funny/comments/media/nice_hat/?url=...
- gloryjulio 3y agoYou can still view the the full sized image by click and open it in a different tab. It's a bit of hassle but at least it works
- harimau777 3y agoAt least for me that doesn't work. It still opens Reddit's weird view image page.
- Havoc 3y agoThey broke that too. Links to image just redirect to the Reddit UI for me Saving it still gets full res though..for now anyway
- mfjordvald 3y agoYou can bypass that by customising your accept header. There's a Chrome extension that does it for you https://chromewebstore.google.com/detail/reddit-load-images-direct/fpimmmjbglpnlpbfikgekaaeinminolo?pli=1 https://chromewebstore.google.com/detail/reddit-load-images-...
- legohead 3y agoI use RES and haven't had any problems opening images or videos... actually, I've noticed they load really slowly, often times I can watch the image loading like it was 56k days. I wonder if that's because I'm using old reddit+RES.
- the_real_cher 3y agoDo you use the new design or stick with old.reddit.com?
- jedberg 3y agoI mostly use old.reddit but sometimes use sh.reddit
- the_real_cher 3y agointeresting I don't know if it's because I grew up with it but I love simple web designs like hacker News and old Reddit.
- LopovJack 3y agoWhat is sh.reddit?
- oniony 3y agoHmm, what's that on your belt?
- encoderer 3y agoBasically we had the same infrastructure back in the day at Trulia - a dedicated “bot cluster” We had two charges from management: let google crawl as fast as it possibly can, and don’t let it disrupt users.
- jgalt212 3y agoIndeed. Human activity is cacheable as it respects Zipf's or a power law. Spider activity is not cacheable. Most pages are GETed within a few multiples of each other. Probably a good way to detect bot traffic is put a URL on a sitemap, and have no links to it anywhere else.