5 ms·

Exploring the Vastness of a Website

57 points by cookingoils 7y ago

aehtyb 7y ago
nice post making a page that offers tools to find random content on specific sites seems like a fun idea, maybe as a starting point for an even larger project
- cookingoils 7y ago
  Thanks! Yes it does. Let me know if you have any ideas : )
  - nerdponx 7y ago
    Reddit, Hacker News, and Lobsters seem like obvious places to start. Also randomization of existing long-lived blogs.
  - fernly 7y ago
    Graphically it should suggest the experience of wandering the stacks of a large library. Have you been in the stacks of a big University or city library? Narrow, ill-lit aisles, books shelved on both sides, from the floor to well above your head. Oooh! Oooh! you could model it after Terry Pratchett's Unseen University Library[1]. [1] https://wiki.lspace.org/mediawiki/Library https://wiki.lspace.org/mediawiki/Library
TrevorFSmith 7y ago
Indeed, the random article link on Wikipedia is a delight.
- cookingoils 7y ago
  I would completely agree. It is one of my favorite features. I decided to leave Wikipedia out of this post, but I think they've done an amazing job creating tools to explore the vastness of the site.
FillardMillmore 7y ago
It seems that most people today prefer to have their content curated and streamlined - with less need to "explore". With the Muxtape example - that's actually an interesting implementation of a random content selector. But how many other viable implementations could you think of? Perhaps I'm just not that creative, but I can't think of many.
- cookingoils 7y ago
  It's true, I don't know if I'd necessarily recommend a random content selector for Airbnb as an example. That said it could actually be interesting to see a selection of random places to visit around the world. Reminds me a bit of closing your eyes, opening an Atlas to a random page, and putting a finger down on a place. I understand your point though that there are times when random doesn't cut it or becomes vague. I also wonder if there are other simple ways we could improve explore pages beyond the go-to recommendations engine or random selector.
  - FillardMillmore 7y ago
    I guess that depends upon what our definition of "simple" is. If we have content that we are presenting to the user, in my eyes, it can be curated somehow (like a recommender system) or it could be uncurated (random selection would fall in this category). For companies that are trying to sell a product or are trying to present users with content relevant to them, that would seem to call for some kind of content curation. I think we would be surprised if we knew how many online retailers still do recommendations based solely on your buying history (and also, perhaps, the buying history of others who have bought items similar to you). I think to improve these recommendation engines, we should improve the underlying algorithms. To improve said algorithms, I think graph databases like Neo4j have an important role to play as they are built for this sort of thing. To present better and more relevant content, we'd need two primary things - more information about the user in question (and I'm all for transparency in both collecting this data and informing the user how it will be utilized) and better algorithms to leverage that information. With the response times of a queried result set originating from multiple levels of depth-traversal in a graph database compared to the response time of a similar query in a traditional RDBMS, I'd say that graph databases are an important and vital component to making this kind of thing better for everyone.
saagarjha 7y ago
Another place where randomization is useful is webcomics. I’ve gone through most of xkcd by just hitting the “Random” button, even tough I would never have the patience to sit through the comics one by one and keep track of how far I’d gotten.
- gwern 7y ago
  Would using ArchiveBinge (https://archivebinge.com/ https://archivebinge.com/), which feeds you n comics per day using RSS, work for that use-case?
  - saagarjha 7y ago
    It’s sad, but I don’t actually have RSS software on my computer anymore. In any case I would flip through xkcd to waste time, not so I could stay up-to-date (I have a Slack bot that posts new comics regularly, funnily enough).
    - gwern 7y ago
      There are RSS to email tools!
kingkool68 7y ago
I wish Amazon had a random button. Several years ago my friend wanted to randomly browse Amazon for things and I thought it would be easy to randomly generate URLs for products but I kept getting 404s. Figured I would need to use the API to scoop up product IDs and randomly go through those to randomly browse Amazon.
- cookingoils 7y ago
  Wow, that sounds like a cool idea. You should build it!
- Jaruzel 7y ago
  Alternatively, you could just have a dictionary file and randomly pick a word to search Amazon with: https://www.amazon.co.uk/s?k={RANDOM_WORD} https://www.amazon.co.uk/s?k={RANDOM_WORD} Seems much simpler.
  - AlexDragusin 7y ago
    The issue with this is that one would end up with a page of stuff related to the keyword as opposed to the desirable way of ending up with an actual product page itself.
AlexDragusin 7y ago
Love the "surprise me" on https://wiby.me/ https://wiby.me/
iod 7y ago
I have enjoyed the random feature of Reddit https://www.reddit.com/r/random https://www.reddit.com/r/random which I have a favorite link to in my browser. I think it follows what the author is trying to get at.
benbristow 7y ago
I made a wee microsite that I've never really shared with anyone. I'm half expecting it to go down if I post it here. https://domaingame.benbristow.co.uk/ https://domaingame.benbristow.co.uk/ Basically takes a random word from a dictionary of English words and adds a random extension to the end, checks if the site up and returns the domain else loops until it does. Found some weird & wonderful stuff through it!
- cookingoils 7y ago
  This is very cool. Thanks for sharing. Reminds me a little of this site: http://vacantstartup.brycewilner.com/ http://vacantstartup.brycewilner.com/
nicolia 7y ago
This article reminds me of the one truly explorative news aggregate app I used some years ago called Zite. Zite would not only pull news from the usual major sources, but it would also pull up obscure blogs and micro niche indie publications on the topic. It was my first time discovering blogs in their original form (not as posts on tumblr). Zite let my discover so much aggregate information from sources that only someone with a pre-existing niche interest would know about. A mudslide in a small Italian village where no one was harmed, but a boulder rolled itself into a villa's sitting room. A tiny Lutheran blog explaining the latin origins of St. Patrick with cited footnotes. A modern explorer's article on the greatest used bookstores they have found around the world (it was not a listicle). The intellectual stimulus was greatly missed once Zite was purchased by Flipboard and all of those nuggets from the vastness of the internet were overruled by the loudest and largest news publications.

57 points by cookingoils 7y ago

nice post making a page that offers tools to find random content on specific sites seems like a fun idea, maybe as a starting point for an even larger project

cookingoils 7y ago
Thanks! Yes it does. Let me know if you have any ideas : )
- nerdponx 7y ago
  Reddit, Hacker News, and Lobsters seem like obvious places to start. Also randomization of existing long-lived blogs.
- fernly 7y ago
  Graphically it should suggest the experience of wandering the stacks of a large library. Have you been in the stacks of a big University or city library? Narrow, ill-lit aisles, books shelved on both sides, from the floor to well above your head. Oooh! Oooh! you could model it after Terry Pratchett's Unseen University Library[1]. [1] https://wiki.lspace.org/mediawiki/Library https://wiki.lspace.org/mediawiki/Library

TrevorFSmith 7y ago

Indeed, the random article link on Wikipedia is a delight.

cookingoils 7y ago
I would completely agree. It is one of my favorite features. I decided to leave Wikipedia out of this post, but I think they've done an amazing job creating tools to explore the vastness of the site.

FillardMillmore 7y ago

It seems that most people today prefer to have their content curated and streamlined - with less need to "explore". With the Muxtape example - that's actually an interesting implementation of a random content selector. But how many other viable implementations could you think of? Perhaps I'm just not that creative, but I can't think of many.

cookingoils 7y ago
It's true, I don't know if I'd necessarily recommend a random content selector for Airbnb as an example. That said it could actually be interesting to see a selection of random places to visit around the world. Reminds me a bit of closing your eyes, opening an Atlas to a random page, and putting a finger down on a place. I understand your point though that there are times when random doesn't cut it or becomes vague. I also wonder if there are other simple ways we could improve explore pages beyond the go-to recommendations engine or random selector.
- FillardMillmore 7y ago
  I guess that depends upon what our definition of "simple" is. If we have content that we are presenting to the user, in my eyes, it can be curated somehow (like a recommender system) or it could be uncurated (random selection would fall in this category). For companies that are trying to sell a product or are trying to present users with content relevant to them, that would seem to call for some kind of content curation. I think we would be surprised if we knew how many online retailers still do recommendations based solely on your buying history (and also, perhaps, the buying history of others who have bought items similar to you). I think to improve these recommendation engines, we should improve the underlying algorithms. To improve said algorithms, I think graph databases like Neo4j have an important role to play as they are built for this sort of thing. To present better and more relevant content, we'd need two primary things - more information about the user in question (and I'm all for transparency in both collecting this data and informing the user how it will be utilized) and better algorithms to leverage that information. With the response times of a queried result set originating from multiple levels of depth-traversal in a graph database compared to the response time of a similar query in a traditional RDBMS, I'd say that graph databases are an important and vital component to making this kind of thing better for everyone.

saagarjha 7y ago

Another place where randomization is useful is webcomics. I’ve gone through most of xkcd by just hitting the “Random” button, even tough I would never have the patience to sit through the comics one by one and keep track of how far I’d gotten.

gwern 7y ago
Would using ArchiveBinge (https://archivebinge.com/ https://archivebinge.com/), which feeds you n comics per day using RSS, work for that use-case?
- saagarjha 7y ago
  It’s sad, but I don’t actually have RSS software on my computer anymore. In any case I would flip through xkcd to waste time, not so I could stay up-to-date (I have a Slack bot that posts new comics regularly, funnily enough).
  - gwern 7y ago
    There are RSS to email tools!

kingkool68 7y ago

I wish Amazon had a random button. Several years ago my friend wanted to randomly browse Amazon for things and I thought it would be easy to randomly generate URLs for products but I kept getting 404s. Figured I would need to use the API to scoop up product IDs and randomly go through those to randomly browse Amazon.

cookingoils 7y ago
Wow, that sounds like a cool idea. You should build it!
Jaruzel 7y ago
Alternatively, you could just have a dictionary file and randomly pick a word to search Amazon with: https://www.amazon.co.uk/s?k={RANDOM_WORD} https://www.amazon.co.uk/s?k={RANDOM_WORD} Seems much simpler.
- AlexDragusin 7y ago
  The issue with this is that one would end up with a page of stuff related to the keyword as opposed to the desirable way of ending up with an actual product page itself.

AlexDragusin 7y ago

Love the "surprise me" on https://wiby.me/ https://wiby.me/

iod 7y ago

I have enjoyed the random feature of Reddit https://www.reddit.com/r/random https://www.reddit.com/r/random which I have a favorite link to in my browser. I think it follows what the author is trying to get at.

benbristow 7y ago

I made a wee microsite that I've never really shared with anyone. I'm half expecting it to go down if I post it here. https://domaingame.benbristow.co.uk/ https://domaingame.benbristow.co.uk/ Basically takes a random word from a dictionary of English words and adds a random extension to the end, checks if the site up and returns the domain else loops until it does. Found some weird & wonderful stuff through it!

cookingoils 7y ago
This is very cool. Thanks for sharing. Reminds me a little of this site: http://vacantstartup.brycewilner.com/ http://vacantstartup.brycewilner.com/

nicolia 7y ago

This article reminds me of the one truly explorative news aggregate app I used some years ago called Zite. Zite would not only pull news from the usual major sources, but it would also pull up obscure blogs and micro niche indie publications on the topic. It was my first time discovering blogs in their original form (not as posts on tumblr). Zite let my discover so much aggregate information from sources that only someone with a pre-existing niche interest would know about. A mudslide in a small Italian village where no one was harmed, but a boulder rolled itself into a villa's sitting room. A tiny Lutheran blog explaining the latin origins of St. Patrick with cited footnotes. A modern explorer's article on the greatest used bookstores they have found around the world (it was not a listicle). The intellectual stimulus was greatly missed once Zite was purchased by Flipboard and all of those nuggets from the vastness of the internet were overruled by the loudest and largest news publications.