7 ms·

Reddit Search: A search engine for Reddit indexable by user, subreddit, and date

86 points by lawrenceyan 6y ago

thepangolino 6y ago
It’s a real pity Reddit plays so many tricks with its content: from preventing search engines to index it to flat out editing comments.
- hardwaresofton 6y ago
  Can you go into this? I'm vaguely aware of reddit having changed in this way but not super sure which rules of this ilk are now implace. This must be why they're login-blocking some big subreddits now, the web scraping precedent that went against LinkedIn must let them know that they can't build a walled garden without making people login first.
  - CodesInChaos 6y ago
    Since they only do that on the mobile website, I'd assume this is just another way of wanting to track users and push them to the mobile app.
  - Dma54rhs 6y ago
    They don't want you to use their mobile website, I don't remember any site being more hostile and anti user friendly even after saying 100 times I'm OK using browser, not an app.
    - guidopallemans 6y ago
      Facebook messenger (messenger.com) is way more hostile; you can open it with "view desktop site", but whenever you tap the input box it focuses and unfocuses immediately, making it mostly unusable. Is mbasic facebook still a thing (for messaging)? I'm in the fortunate position of not having to use fb messenger for most of my contacts. Edit: despite what I said, I think the mobile reddit browser experience is horrible indeed. Usually I change the url to go to i.reddit.com instead, which is much more usable.
      - thatguy0900 6y ago
        I use mbasic whenever my mom texts me that she sent something on Facebook, it's my only interaction with the site. It works fine, though.
latentdeepspace 6y ago
How is this different from https://redditsearch.io https://redditsearch.io ? If I understand it right, both uses the same PushShift API and the rest is just UI.
- beigeoak 6y ago
  redditsearch.io disabled searching by username last year. It was being used for targeted harassment of specific users.
mattowen_uk 6y ago
Ok this is interesting... A few months ago, I deleted all my Reddit comments and posts (one by one, using a bit of web automation), yet they all still show up in this search even thought they don't on Reddit. Proves the point, that once something is on the internet, it's there forever.
- doix 6y ago
  Yep, pushshift indexes all reddit comments as they come in. Unless you delete your comment fairly quickly, it'll be stored forever.
  - stefan_ 6y ago
    So it is a big copyright violation machine? I give reddit a license, not "pushshift".
    - supergirl 6y ago
      do you give google a license?
    - square_usual 6y ago
      The Pushshift maintainer is very responsive to requests to remove personal data. Being an opt-out mechanism, it's not perfect, but it's better than ignoring those requests, plus pushshift has provided incredible value to HCI researches who've been studying reddit.
    - ceejayoz 6y ago
      https://www.redditinc.com/policies/user-agreement-october-15-2020 https://www.redditinc.com/policies/user-agreement-october-15... > When Your Content is created with or submitted to the Services, you grant us a worldwide, royalty-free, perpetual, irrevocable, non-exclusive, transferable, and sublicensable license to use, copy, modify, adapt, prepare derivative works of, distribute, store, perform, and display Your Content and any name, username, voice, or likeness provided in connection with Your Content in all media formats and channels now known or later developed anywhere in the world. This license includes the right for us to make Your Content available for syndication, broadcast, distribution, or publication by other companies, organizations, or individuals who partner with Reddit.
ceilingcorner 6y ago
I can’t tell if this displays deleted comments, but if the other comment is correct, it does. That’s a good thing, as I’ve found over-active moderation to be a serious problem on certain subreddits. Especially “Ask” ones like /r/askhistorians. So many insightful or helpful comments are deleted because a trigger-happy moderator arbitrarily decides they don’t “fit the rules,” which of course are flexible for some but not others.
INTPenis 6y ago
I will give this a try. Can someone please tell me why does the reddit search engine default to searching globally? 99 out of 100 searches I make I am on a subreddit, want to search the subreddit, enter my search term and the first thing it does is search globally. I have to click the link r/subreddit to make it search locally. Most times the API doesn't even have time to return the results before I click the link and make it do another request to search locally. Isn't that wasteful? I am of course talking about the new UI.
- optimalsolver 6y ago
  It's up to the subreddit's mods whether the search bar does a local or global search. It's obviously in Reddit's interest for users to be able to search site-wide, but mods can change this behavior. Seems a fair setup to me.
  - thatguy0900 6y ago
    Very strange to put that in the mods control. Why would the mods have any idea or care for what users search for? Reddit has all the data about what users usually want and users know what they actually want, I don't see where mods come in
  - ketralnis 6y ago
    Is it? I'm pretty sure they just remember the setting from last time you searched, I don't see this setting anywhere in the subreddits I mod
- apple4ever 6y ago
  That drives me BONKERS and they need to fix it.
gilbraithVO 6y ago
God knows there needs to be a better way to search reddit
- drexlspivey 6y ago
  I just use google with site:reddit.com
  - tediousdemise 6y ago
    Adding site:reddit.com is a little too verbose for me, so ultimately I end up just appending/prepending "reddit" to my query and it seems to do well enough for most of my searches.
asdev 6y ago
I built this small chrome extension to add how long ago a reddit post was made, number of upvotes and comments to Google search results since I was sick of clicking into super old threads or threads with no comments: https://chrome.google.com/webstore/detail/reddit-search-helper/laladmlmbneplheiajfdpcephiehoifh?hl=en https://chrome.google.com/webstore/detail/reddit-search-help... unfortunately the friction of using another search engine is too much for me, this extension helps a little bit at least
- brw 6y ago
  I didn't even realize I needed this but that's actually insanely useful. Thanks!
tediousdemise 6y ago
Does anyone else append or prepend "reddit" to most of their Google searches? I mostly do it to see real feedback from real people about whatever I'm interested in, rather than an ad-sense driven list of search results organized in descending order by the highest paying company.
- speedgoose 6y ago
  I do. I also add news.ycombinator.com when it's about technology. It provides better discussions than reddit on average.
- willio58 6y ago
  Not most by any means but if I’m looking for a product, I prepend Reddit to see real people talking about it.
- jasonladuke0311 6y ago
  Nearly everything I search for these days.
- ecf 6y ago
  I used to do this a whole lot more until Reddit started screwing with the dates of their posts. Now trying to find something within the last year will see results from five years ago show up. Additionally, Reddit realizes they are being used this way and have made great strides in building out the astroturfing side of their business to allow brands to post ads anonymously.
- apple4ever 6y ago
  I do as well - I use Bing but I experience the same thing. Any search for advice type things is met with a bunch of worthless ad driven junk. Adding Reddit at least gets me something worthwhile.
- kgin 6y ago
  Almost always
BayAreaEscapee 6y ago
This is really quite an interesting search engine. I was able to find posts by people who had deleted their accounts by using their deleted username in the "author" field. It's a good reminder to watch what you say on the internet. Nothing ever really disappears.
- kQq9oHeAz6wLLS 6y ago
  > It's a good reminder to watch what you say on the internet. Nothing ever really disappears. That's why I always use my real name online; keeps me from saying something I wouldn't say in public.
  - codezero 6y ago
    Nice to meet you, how do you pronounce your name?
    - kQq9oHeAz6wLLS 6y ago
      "Stan"
nabakin 6y ago
I've done a lot of research on deleting reddit data recently and there are some important things the reddit community should know about reddit data and actually deleting it. The most important thing is that 99% of reddit deletion tools out there (in fact all I have found except for two) do not actually delete your reddit data. Most of these tools rely on the last 1000 posts and comments listed on your profile but do not delete any other comments or posts you have made and even then, most of these do not edit your comments and posts before deleting which means those 1000 posts and comments are still stored on reddit's servers. To completely remove your reddit data, I have found only one way and that is with reddit-shreddit [0]. Despite its name being extraordinarily similar to another popular reddit deletion tool called Shreddit [1] which doesn't actually work, it is entirely different. reddit-shreddit works by taking the data from a reddit GDPR data request which includes links to all your posts and comments and editing and deleting each and every one. With this method, reddit-shreddit does not need to scrape posts and comments from reddit to know what to delete. It can leverage the official GDPR-compliant data request from reddit ensuring all of your posts and comments are known and are deleted. If you wish to confirm they are actually deleted, you can submit another data request to verify all of your posts and comments are gone from reddit's servers. It may also be possible to remove your data from reddit with a GDPR request but I'm not sure. I haven't looked into that. Although you can remove your data from reddit's servers, that does not mean it is gone. Ever since reddit was created, people have been scraping and archiving it. The most popular reddit archiving service is called pushshift and it has archives for every year of reddit up to 2015 when it started archiving every month. Pushshift runs the API that powers this website and other popular reddit tools like removeddit and the former ceddit. Luckily the developer has been known to delete your data if you request it. I've seen it mentioned on reddit that you can email him or pm him on reddit and he will remove your data. Unfortunately all the archives from previous years are still floating around the internet outside of pushshift but at least your data will disappear from the largest source. [0] https://github.com/nixfu/reddit-shreddit https://github.com/nixfu/reddit-shreddit [1] https://github.com/x89/Shreddit https://github.com/x89/Shreddit
truth_ 6y ago
All I wanted was a sort by NSFW, and top.

86 points by lawrenceyan 6y ago

It’s a real pity Reddit plays so many tricks with its content: from preventing search engines to index it to flat out editing comments.

hardwaresofton 6y ago
Can you go into this? I'm vaguely aware of reddit having changed in this way but not super sure which rules of this ilk are now implace. This must be why they're login-blocking some big subreddits now, the web scraping precedent that went against LinkedIn must let them know that they can't build a walled garden without making people login first.
- CodesInChaos 6y ago
  Since they only do that on the mobile website, I'd assume this is just another way of wanting to track users and push them to the mobile app.
- Dma54rhs 6y ago
  They don't want you to use their mobile website, I don't remember any site being more hostile and anti user friendly even after saying 100 times I'm OK using browser, not an app.
  - guidopallemans 6y ago
    Facebook messenger (messenger.com) is way more hostile; you can open it with "view desktop site", but whenever you tap the input box it focuses and unfocuses immediately, making it mostly unusable. Is mbasic facebook still a thing (for messaging)? I'm in the fortunate position of not having to use fb messenger for most of my contacts. Edit: despite what I said, I think the mobile reddit browser experience is horrible indeed. Usually I change the url to go to i.reddit.com instead, which is much more usable.
    - thatguy0900 6y ago
      I use mbasic whenever my mom texts me that she sent something on Facebook, it's my only interaction with the site. It works fine, though.

latentdeepspace 6y ago

How is this different from https://redditsearch.io https://redditsearch.io ? If I understand it right, both uses the same PushShift API and the rest is just UI.

beigeoak 6y ago
redditsearch.io disabled searching by username last year. It was being used for targeted harassment of specific users.

mattowen_uk 6y ago

Ok this is interesting... A few months ago, I deleted all my Reddit comments and posts (one by one, using a bit of web automation), yet they all still show up in this search even thought they don't on Reddit. Proves the point, that once something is on the internet, it's there forever.

doix 6y ago
Yep, pushshift indexes all reddit comments as they come in. Unless you delete your comment fairly quickly, it'll be stored forever.
- stefan_ 6y ago
  So it is a big copyright violation machine? I give reddit a license, not "pushshift".
  - supergirl 6y ago
    do you give google a license?
  - square_usual 6y ago
    The Pushshift maintainer is very responsive to requests to remove personal data. Being an opt-out mechanism, it's not perfect, but it's better than ignoring those requests, plus pushshift has provided incredible value to HCI researches who've been studying reddit.
  - ceejayoz 6y ago
    https://www.redditinc.com/policies/user-agreement-october-15-2020 https://www.redditinc.com/policies/user-agreement-october-15... > When Your Content is created with or submitted to the Services, you grant us a worldwide, royalty-free, perpetual, irrevocable, non-exclusive, transferable, and sublicensable license to use, copy, modify, adapt, prepare derivative works of, distribute, store, perform, and display Your Content and any name, username, voice, or likeness provided in connection with Your Content in all media formats and channels now known or later developed anywhere in the world. This license includes the right for us to make Your Content available for syndication, broadcast, distribution, or publication by other companies, organizations, or individuals who partner with Reddit.

ceilingcorner 6y ago

I can’t tell if this displays deleted comments, but if the other comment is correct, it does. That’s a good thing, as I’ve found over-active moderation to be a serious problem on certain subreddits. Especially “Ask” ones like /r/askhistorians. So many insightful or helpful comments are deleted because a trigger-happy moderator arbitrarily decides they don’t “fit the rules,” which of course are flexible for some but not others.

INTPenis 6y ago

I will give this a try. Can someone please tell me why does the reddit search engine default to searching globally? 99 out of 100 searches I make I am on a subreddit, want to search the subreddit, enter my search term and the first thing it does is search globally. I have to click the link r/subreddit to make it search locally. Most times the API doesn't even have time to return the results before I click the link and make it do another request to search locally. Isn't that wasteful? I am of course talking about the new UI.

optimalsolver 6y ago
It's up to the subreddit's mods whether the search bar does a local or global search. It's obviously in Reddit's interest for users to be able to search site-wide, but mods can change this behavior. Seems a fair setup to me.
- thatguy0900 6y ago
  Very strange to put that in the mods control. Why would the mods have any idea or care for what users search for? Reddit has all the data about what users usually want and users know what they actually want, I don't see where mods come in
- ketralnis 6y ago
  Is it? I'm pretty sure they just remember the setting from last time you searched, I don't see this setting anywhere in the subreddits I mod
apple4ever 6y ago
That drives me BONKERS and they need to fix it.

gilbraithVO 6y ago

God knows there needs to be a better way to search reddit

drexlspivey 6y ago
I just use google with site:reddit.com
- tediousdemise 6y ago
  Adding site:reddit.com is a little too verbose for me, so ultimately I end up just appending/prepending "reddit" to my query and it seems to do well enough for most of my searches.

asdev 6y ago

I built this small chrome extension to add how long ago a reddit post was made, number of upvotes and comments to Google search results since I was sick of clicking into super old threads or threads with no comments: https://chrome.google.com/webstore/detail/reddit-search-helper/laladmlmbneplheiajfdpcephiehoifh?hl=en https://chrome.google.com/webstore/detail/reddit-search-help... unfortunately the friction of using another search engine is too much for me, this extension helps a little bit at least

brw 6y ago
I didn't even realize I needed this but that's actually insanely useful. Thanks!

tediousdemise 6y ago

Does anyone else append or prepend "reddit" to most of their Google searches? I mostly do it to see real feedback from real people about whatever I'm interested in, rather than an ad-sense driven list of search results organized in descending order by the highest paying company.

speedgoose 6y ago
I do. I also add news.ycombinator.com when it's about technology. It provides better discussions than reddit on average.
willio58 6y ago
Not most by any means but if I’m looking for a product, I prepend Reddit to see real people talking about it.
jasonladuke0311 6y ago
Nearly everything I search for these days.
ecf 6y ago
I used to do this a whole lot more until Reddit started screwing with the dates of their posts. Now trying to find something within the last year will see results from five years ago show up. Additionally, Reddit realizes they are being used this way and have made great strides in building out the astroturfing side of their business to allow brands to post ads anonymously.
apple4ever 6y ago
I do as well - I use Bing but I experience the same thing. Any search for advice type things is met with a bunch of worthless ad driven junk. Adding Reddit at least gets me something worthwhile.
kgin 6y ago
Almost always

BayAreaEscapee 6y ago

This is really quite an interesting search engine. I was able to find posts by people who had deleted their accounts by using their deleted username in the "author" field. It's a good reminder to watch what you say on the internet. Nothing ever really disappears.

kQq9oHeAz6wLLS 6y ago
> It's a good reminder to watch what you say on the internet. Nothing ever really disappears. That's why I always use my real name online; keeps me from saying something I wouldn't say in public.
- codezero 6y ago
  Nice to meet you, how do you pronounce your name?
  - kQq9oHeAz6wLLS 6y ago
    "Stan"

nabakin 6y ago

I've done a lot of research on deleting reddit data recently and there are some important things the reddit community should know about reddit data and actually deleting it. The most important thing is that 99% of reddit deletion tools out there (in fact all I have found except for two) do not actually delete your reddit data. Most of these tools rely on the last 1000 posts and comments listed on your profile but do not delete any other comments or posts you have made and even then, most of these do not edit your comments and posts before deleting which means those 1000 posts and comments are still stored on reddit's servers. To completely remove your reddit data, I have found only one way and that is with reddit-shreddit [0]. Despite its name being extraordinarily similar to another popular reddit deletion tool called Shreddit [1] which doesn't actually work, it is entirely different. reddit-shreddit works by taking the data from a reddit GDPR data request which includes links to all your posts and comments and editing and deleting each and every one. With this method, reddit-shreddit does not need to scrape posts and comments from reddit to know what to delete. It can leverage the official GDPR-compliant data request from reddit ensuring all of your posts and comments are known and are deleted. If you wish to confirm they are actually deleted, you can submit another data request to verify all of your posts and comments are gone from reddit's servers. It may also be possible to remove your data from reddit with a GDPR request but I'm not sure. I haven't looked into that. Although you can remove your data from reddit's servers, that does not mean it is gone. Ever since reddit was created, people have been scraping and archiving it. The most popular reddit archiving service is called pushshift and it has archives for every year of reddit up to 2015 when it started archiving every month. Pushshift runs the API that powers this website and other popular reddit tools like removeddit and the former ceddit. Luckily the developer has been known to delete your data if you request it. I've seen it mentioned on reddit that you can email him or pm him on reddit and he will remove your data. Unfortunately all the archives from previous years are still floating around the internet outside of pushshift but at least your data will disappear from the largest source. [0] https://github.com/nixfu/reddit-shreddit https://github.com/nixfu/reddit-shreddit [1] https://github.com/x89/Shreddit https://github.com/x89/Shreddit

truth_ 6y ago

All I wanted was a sort by NSFW, and top.