16 ms·

Searx – Privacy-respecting metasearch engine

299 points by cube00 5y ago

gnufx 5y ago
For what it's worth, it's used by by the de-Googled phone /e/ Foundation at https://spot.ecloud.global https://spot.ecloud.global and seems to work tolerably well.
mg 5y ago
Every time I stumble across a new search engine, I add it to my search engine comparison tool: https://www.gnod.com/search/ https://www.gnod.com/search/ Will add SearX now. It seems to provide reasonably good results. Update: It's on (under 'More Engines').
- fsflover 5y ago
  You should also add YaCy: https://yacy.net https://yacy.net.
  - mg 5y ago
    It seems to be not web based? When I click on "Try out the YaCy Demo Peer", I get "502 Bad Gateway".
    - fsflover 5y ago
      It's self-hosted and peer-to-peer. You could search for other public-facing instances, e.g., http://sokrates.homeunix.net:6060 http://sokrates.homeunix.net:6060. Ideally, you could run your own instance to show the world how it works.
- wenbin 5y ago
  You should add the podcast search engine https://www.listennotes.com/ https://www.listennotes.com/
  - mg 5y ago
    Thanks, added. Holy Moly, are there really over 100 million podcast episodes out there?
    - wenbin 5y ago
      Yes. some numbers: https://www.listennotes.com/podcast-stats/ https://www.listennotes.com/podcast-stats/ Listen Notes was started in early 2017 as a side project, when there were ~23 million episodes. I remembered seeing the number of web pages indexed by Google in early 1998 was ~25 million, then I thought that "ok, 23 million episodes might justify the existence of a podcast search engine" :)
  - autoexec 5y ago
    Too bad they force you to log in to view a result or do anything but search. They also share/sell your data to 3rd parties including Google
- imglorp 5y ago
  This is a nice compilation. It would be very interesting if it examined and compared results.
  - m-i-l 5y ago
    Yes, looks good, although I thought it was going to be a federated search, i.e. you enter your search term and it performs that search on all the sites selected. The simpler way of implementing a federated search would be to show separate results boxes from each site, although that wouldn't scale well to a large number of sites, and it can get quite complicated to try to combine the results.
- deleted 5y ago
  [deleted]
- 51stpage 5y ago
  There is also Marginalia https://search.marginalia.nu/ https://search.marginalia.nu/ which I don't see on your list.
  - marginalia_nu 5y ago
    It's currently undergoing its monthly maintenance just FYI. It's up and technically working, but with a drastically reduced index size.
  - sysadm1n 5y ago
    Tried Marginalia. So many plaintext http links which I avoid like the plague. That's my only gripe with it. Other than that, it's an awesome tool.
    - freediver 5y ago
      > So many plaintext http links which I avoid like the plague. Why? What you described appears to be the safest place on the web.
      - sysadm1n 5y ago
        I only browse HTTPS sites. I have the `HTTPS Everywhere` addon installed with the `EASE` / Encrypt All Sites Eligible turned on so I don't accidentally browse an unencrypted website. Something like 85%/90% of the web is encrypted now, and there's no excuse to be using outdated plaintext http anymore. It's a privacy and security risk. There are only few instances where I had to view a http site (I'm a freelancer and my client's webpage was still unencrypted, so I had to see it, so a rare exception to the rule).
        marginalia_nu 5y ago
        It's funny because I got like 70% HTTP in my index, so the whole "90% of the web is encrypted" seems to depend on which sample you are looking at. Google doesn't index HTTP at all, so that's not a good place to go looking for what's the most popular. That's in fact half the reason why I built this search engine in the first place, because they demand things of websites that some websites simply can't or wont comply with. A lot of servers still use HTTP, for various reasons. There are also some clients that can't use HTTPS.
- jwithington 5y ago
  where's you.com lol
  - mg 5y ago
    Thanks, added.
- ColinHayhurst 5y ago
  Nice work. If you have a twitter handle you might request to get added to these lists; either way they might be useful for you: https://twitter.com/SearchEngineMap/lists https://twitter.com/SearchEngineMap/lists
- Jaruzel 5y ago
  I have this REALLY old text file of search engine URLs: http://www.jaruzel.com/textfiles/Old%20Web%20Info/Internet%20Search%20Engines%20v2.61.txt http://www.jaruzel.com/textfiles/Old%20Web%20Info/Internet%2... Google basically killed almost off of them off :( It would be great to see some proper competition in the search space, especially around specialist search engines.
  - sixothree 5y ago
    I would love a search engine targeted towards developers. Searching for symbols seem to be a problem with google, not to mention all of the utterly crappy results they serve up.
    - ijr 5y ago
      Symbolhound does that.
      - stagas 5y ago
        Also grep.app for searching into repos really fast.
        culi 5y ago
        also https://searchcode.com/ https://searchcode.com/ was posted on a HN thread about niche search engines a couple days ago
- Minor49er 5y ago
  You should add Fireball. It's excellent https://fireball.de/ https://fireball.de/
- nix23 5y ago
  A yacy instance would be good too ;) https://yacy.net/ https://yacy.net/
- danskeren 5y ago
  Ask.Moe nona.de
- 1_player 5y ago
  Is anything supposed to happen when I enter something and press Enter? Nothing happens for me, FF on Windows, uBlock.
  - reayn 5y ago
    You are supposed to type something into the search field then click the engine you want to use, it will pass on whatever you entered. I agree that the creator could make that a little more clear somewhere on the page.
herodotus 5y ago
Could this be installed on a Raspberry Pi? I am very happy with my Raspberry Pi-hole: would not mind adding a second pi for searching.
ChrisArchitect 5y ago
Please keep title same as the actual post: Searx - moving away from DuckDuckGo It gives more context to the topic, as in it's not just a link to the search engine itself.
egberts1 5y ago
My search engine puts a Pi symbol in lower left corner of every web page I visited.
hans_castorp 5y ago
Qwant is another (non-US) alternative
- rahen 5y ago
  Doesn't it use Bing results, like DuckDuckGo does? I read it's hosted on MS Azure like Bing and DDG, so in the end it's somewhat just a rebranded Bing. Quite a shame for a European (Franco-German) search engine.
  - kaba0 5y ago
    Even if they were just a proxy behind bing, that would be still good, weren’t it?
    - rahen 5y ago
      Well, claiming to offer an alternative to the GAFAM while depending on their products / data / infrastructure is a bit misleading. So indeed they're doing okay privacy wise, but a lot of users feel cheated when they realize their "independent search engine" (DuckDuckGo) is just a Bing portal hosted on Azure.
GhettoComputers 5y ago
Isn’t hosting your own instance taking away every benefit of searx by revealing your IP? If you had a VPN you’d use it to mask your IP from tracking of search engines anyway, and if you used Tor for it, you’d probably move back soon since it’ll be so much worst with latency, like how many people go back to google because DDG sucks for results. I suggest just using instances you can find them at https://searx.space https://searx.space some are more reliable than others but none have been trouble free. There’s a lot of these instances for privacy, chrome has a privacy plugin with a white eye that uses nitter.net for Twitter, teddit for Reddit and other public instances. One instance of Reddit was even made completely in rust. ;) https://chrome.google.com/webstore/detail/privacy-redirect/pmcmeagblkinmogikoikkdjiligflglb?hl=en https://chrome.google.com/webstore/detail/privacy-redirect/p... To reply to the person under me, you’re always relying on a trust in something unverified and untrustworthy filters like VPNs anyway, you’re either revealing your IP using a wrapper that reveals it instantly, use a site that isn’t a search engine and might be using your data, using a VPN that is based on reputable and assumptions, usually based in another country you won’t visit or know much about aside from random reviewers, or using Tor, losing latency, reasonable speed image search, and still be possibly compromised.
- kleinsch 5y ago
  But if you’re using instances someone else is hosting, aren’t you hitting half the author’s objections? - They may be hosted in the US - They may be hosted on AWS - You have no idea if the maintainer of the instance is tracking you
  - jarvuschris 5y ago
    ^ This right here. This article is pretty hogwash IMO Points 1 and 3 aren't relevant if they aren't recording the data. Companies in other jurisdictions have no magic invulnerability you can trust to their data getting out (legally or illegally) if they're storing it. Points 2 and 5 are equally true of any open source project unless you run it yourself from source. There are _plenty_ of examples of users getting phished by maliciously built/hosted open source tools Point 4 is obviously not malicious tracking and a mistake any project could make At the end of the day though, unless you're going to run everything yourself (which most people aren't) you have to pick who to trust -- some random person running a server somewhere, or a company with hundreds of employees recruited under the premise of working on a privacy-centric search engine who could all turn whistleblower
jostillmanns 5y ago
Whoogle is another alternative, that focuses on Google search results
ced 5y ago
From the link: The CEO sold his previous company's data before founding DDG. His previous company (Names DB) was a surveillance capitalist service designed to coerce naive users to submit sensitive information about their friends. Is that a fair statement? Can someone provide more context?
- boomboomsubban 5y ago
  It was a failed social network to help you reconnect with old friends. It tried to get you to recruit your friends immediately after registering and had a typical social network license. I'd say that description is intentionally describing it in the worst possible light, but not wholly inaccurate.
  - deltree7 5y ago
    Yet, HN is down and ready to suck searx dick because hurr durr privacy. This is HN in a nutshell. Hint: If all you privacy-paranoid people show the exact same behavior, you are an advertiser dream. Sure, you won't be sold shampoo like general population, most advertisement companies know the exact kind of doomer-prepper items to sell if you come via VPN/DDG/whatever convoluted hack/concoction you come up to make your life inconvenient.
ColinHayhurst 5y ago
https://www.searchenginemap.com/ https://www.searchenginemap.com/
- account-5 5y ago
  That site is horrible on mobile, a good portion of the screen is taken up by the orange "download/view" infographic thing. Interesting though to see how connected the engines are, I would have thought DDG would be bigger with its bang option though I assume it's about what is natively included in the results.
  - ColinHayhurst 5y ago
    Yes, it is horrible on mobile. The size of all the syndicating search services are the same. An update is overdue. Complementary twitters list are maintained here: https://twitter.com/SearchEngineMap/lists https://twitter.com/SearchEngineMap/lists
- freediver 5y ago
  I believe rightdao.com is missing from the list. It has independent index (and also impressive speed). Also not sure what the criteria for inclusion is, but search.marginalia.nu and teclis.com both have their own indexes.
skerit 5y ago
Alright, going to try searx.be for a while then.
- jqpabc123 5y ago
  I heard it could be a honeypot for the CIA. But feel free to prove me wrong.
  - unixfox 5y ago
    If you don't want to use a single searx instance then feel free to use a random one automatically for each search thanks to this tool which can be used locally: https://searx.neocities.org/ https://searx.neocities.org/
    - jqpabc123 5y ago
      I heard these searx instances could be linked together in a honeypot network run by the CIA. But feel free to prove me wrong.
schleck8 5y ago
Metager is a non-profit, open-source search engine running fully on renewable energy. It also has a proxy for opening results anonymously https://metager.de https://metager.de
- hermitsings 5y ago
  https://metager.org/ https://metager.org/ for english users
- czechdeveloper 5y ago
  That seems quite exclusive to germany
kristianpaul 5y ago
Doesnt searx looks for results at duckduckgo and google for you anyway? Whats the difference from using ddg directly?
- sebow 5y ago
  I don't think it uses DDG directly.But anyways you can configure the sources for files, media, wiki,etc. Makes sense since the engine is open source, but then again it's not really a search engine itself but a metasearch one
- boudin 5y ago
  There's other sources available. It is a meta search engine, so it will always rely on other sources, but you can disable duckduck go and google backends.
- luciusdomitius 5y ago
  Isn't duckduckgo just an alternative frontend for bing with an integrated ad/tracking blocker. Or at least that's what they claim.
  - nicce 5y ago
    It is. They say they add some indexing on they own, but results are all the same.
- BeetleB 5y ago
  Searx can ping multiple search engines, including those not supported by DDG. For example, searx has a dedicated file search, which includes torrents.
ahtaarra 5y ago
I recently made the swtich from DDG from Searx simply bevause right-clicking on a search result to copy the url resulted in a referrer link to be copied rather than the link of the result destination.
- caskstrength 5y ago
  I use https://addons.mozilla.org/en-US/firefox/addon/clearurls/ https://addons.mozilla.org/en-US/firefox/addon/clearurls/ to automatically convert referrals to actual links on web pages.
  - ravenstine 5y ago
    I've been using ClearURLs for a while so I never even realized that DDG used referrer links. Have they always done this?
- southerntofu 5y ago
  I think enabling DoNotTrack header or disabling JavaScript prevents this behavior: i cannot reproduce on Tor Browser. But you are correct this is a worrying development.
- boomboomsubban 5y ago
  Isn't the referral link only on the ad results, which is clearly marked "AD"? That's what my quick test shows.
  - ahtaarra 5y ago
    Now I am not sure what exactly the issue is but this seems to be applicable to all results, at least on my particular setup. On LibreWolf browser, On mouse hover, the correct result urls show but then I right-click one, it shows something like: https://duckduckgo.com/l/?uddg=<destination_url_here>&notrut=duckduck_in https://duckduckgo.com/l/?uddg=<destination_url_here>&notrut... But this same behaviour cannot be observed on Google/Searx on the same browser. It also even isn't observable with DDG on a FF nightly build on the same system and a FF stable build on a separate one. Edit: url formatting EDIT2: this behaviour seems to be limited to my particular setup and even a clean LibreWolf profile seems not to suffer from this issue. I apologise for the misunderstanding.
drcongo 5y ago
As someone who uses Safari with its built in list of search providers, I'm rather stuck with DDG for address bar searches, but boy it has really started to suck over the last year or two.
- jeroenhd 5y ago
  > with its built in list of search providers TIL. That's just... terrible UX.
- boomboomsubban 5y ago
  You could always do !searx
  - drcongo 5y ago
    True! I'm experimenting with Ecosia right now, but also recently got on the beta for the kagi.com search engine which so far has proven to be vasty superior to any other that I've tried.
    - SyzygistSix 5y ago
      Nothing in their FAQ answers where they are located and they dance around the question of what info they track/keep by saying that they keep the minimum. That is a non-answer. I like the idea in theory but in practice I have no idea who I'm dealing with. They could be far more open about their processes. I like the idea of a paid browser though.
- ColinHayhurst 5y ago
  Yes, Safari is the worst offender when it comes to offering search choice on desktop. On iOS using a new app called Hyperweb you can the new Safari extensions to access and create a longer preferences list. https://hyperweb.app/ https://hyperweb.app/ We really shouldn't have to choose this or that, but should be able to easily use multiple choices in search. You can do that today as explained here, but you'll need to switch browsers. https://blog.mojeek.com/2021/09/multiple-choice-in-search.html https://blog.mojeek.com/2021/09/multiple-choice-in-search.ht...
ricardo81 5y ago
'Uses Amazon Web Services (AWS) as a cloud provider and Cloudfare CDS.' IIRC DDG uses Microsoft servers now exclusively. Makes sense given the volume of queries they're handling and all dependent on Bing API.
hermitsings 5y ago
I had been considering using Searx (which I had known about before) lately since I have to use DDG+Google for getting satisfactory results. Edit: I really like DDG bang and vim like nav keys tho
- visiblink 5y ago
  Someone here once posted a link to duckduckstart.com. I have it set up in my search bar now. If you search, it goes through Startpage (Google results, more privately). If you search with a bang, it goes through Duckduckgo. It's probably close to what you're looking for.
  - hermitsings 5y ago
    saw it just now. wow thanks
phantom_oracle 5y ago
I will add add a disclaimer to this comment that it is tinfoil-hat and just speculation(bordering on conspiracy) but many of these "we are a privacy-first company" might actually just be honeypots and fronts for 3-letter agencies. The comment is not wholly conspiratorial, considering the CIA owned Swiss crypto company: Crypto AG [1] It's within the realm of possibility that most of these privacy services could be owned by 3-letter agencies or small enough to be coerced into cooperation. [1] https://www.scmp.com/news/world/europe/article/3050193/crypto-spy-caper-cia-secretly-owned-swiss-company-sold-encryption https://www.scmp.com/news/world/europe/article/3050193/crypt...
- no_time 5y ago
  Same goes for VPN companies. I do feel bad for all the journalists and whistleblowers who will fall for these scams but as far as I'm concerned, as long as I can avoid for profit data collection companies like Google, it's good enough for me. If I can't avoid my data being collected, I will still try my best to make it as worthless as possible just out of spite.
- jqpabc123 5y ago
  Haven't you heard? The CIA has gone open source. They don't need to own a company anymore. They can just download the Searx source code; modify it as they see fit, and make it available on a server someplace. Can you prove that searx.be isn't run by a "3 letter agency"? Can you prove that the source code running at searx.be is the same as on Github? The point being --- unless you have full access to the server, open source means nothing with regard to privacy and security of any service. It actually means less than nothing --- it means it is super easy to build into a honeypot.
  - marc_abonce 5y ago
    Of course, there's no fool-proof solution to knowing what code is running in the server side, but https://searx.space https://searx.space at least shows if an instance modified their client-side code, which you can see in the HTMl column. To mitigate server-side code from identifying you, you can consume an instance from Tor. Of course, you could try to do that with any other search engine, but most of the other search engines either block exit nodes or provide incomplete functionality if you disable JS. It's not perfect, but it may be good enough depending on your threat model.
ojosilva 5y ago
Are there any opensource real internet search engines worth looking at? I think we should be working on disrupting search as a whole instead of depending on the Googles, Baidus and Bings of the world. I'm fully aware of the massive crawling and storage requirements, but opensource projects that can get search right can later 1) be hosted by the powerhouses of the cloud or non-profit parties, or 2) become a fully distributed hosting and crawling effort as in p2p and blockchain.
- vindarel 5y ago
  p2p: there's the Yacy effort. https://yacy.net/ https://yacy.net/ I… couldn't find a portal to try it out (I did years ago, the results need… to be discussed. It's anyways easy to install and to choose what part of the web to crawl.) > YaCy is free software for your own search engine. maybe they rebranded and don't aspire to be a complete web search engine?
- deleted 5y ago
  [deleted]
YXNjaGVyZWdlbgo 5y ago
I had a searx instance running for a long time and it's great when it works but the plugins for site specific searches break all the time and if you have more than 3-4 users with a high search frequency google blacklists your IP by throttling.
- kaba0 5y ago
  May I ask about what’s your idea of preserving privacy with self-hosting a website that searches for specific terms from a presumably fixed IP, where by 1/3-1/4 chance it can be attributed to you?
  - YXNjaGVyZWdlbgo 5y ago
    The instance ran on a mobile connection not associated with any private information. EDIT: It was located in a german street light 20km away from any of the users. Just to get the geolocation question out of the way. It was more of an experiment than anything else there will be a talk about it and other FreiFunk (Open Mesh Network in Germany) related stuff at the next virtual CCC congress.
    - dalf 5y ago
      Did you renew the mobile IP from time to time? Note: since the version 1.0, searx stops sending request for 1 day when a CAPTCHA is detected which might help a little. (I'm really interested by the results of your experiment)
      - YXNjaGVyZWdlbgo 5y ago
        We renewed every 12 hours in the end we came to the conclusion that google might discriminate traffic from eastern european states. The sim cards were mostly from poland, belarus and the ukraine. When we tested it against german, french and italian cards captchas were way more frequent on the eastern cards and showed way earlier. I will ping you as soon as the talk is online.
xwdv 5y ago
People don’t want privacy. They want results. Society programs us to think privacy is our top concern. Is it?
- InfiniteRand 5y ago
  I feel like privacy is a concern relative to the chance of my information being misused and the damage from the misuse of that information. If a search engine wants to collect random facts about me but that collection is extremely unlikely to affect me in any way I couldn’t care less
- marginalia_nu 5y ago
  I do think you are mostly correct. Some people really care about privacy, but for most people it isn't a huge concern. This doesn't make it any less important, but just means that if your main selling point is "we're the search engine that cares about privacy", then odds are you're not going to get a lot of users.
  - lardolan 5y ago
    I agree with most of your points. Although It may be matter of a time to grow that niche. IMO Pople who value it are willing to take sacrifice of less functional outputs. Privacy is most effective selling point when working with sensitive information.
    - marginalia_nu 5y ago
      If you are working with sensitive information, the last thing you probably want to do is broadcast that you are working with sensitive information.
      - kreeben 5y ago
        Au contraire, it's the first thing you should broadcast, unless you're trying to scam people out of their PII.
- deleted 5y ago
goldsteinq 5y ago
The problem with self-hosted search engine is that they make you _very_ unique: you’re the only client of the ”backend“ engine with that (static and non-NATed) IP. Furthermore, you’re now one of the small group of people with ”hosting“ IPs. Using self-hosted SearX may make you easier to track, not harder. Using SearX hosted by someone else is marginally better, but now you have to trust the owner of the server, which is probably not what you want for privacy-centered search engine.
- woodruffw 5y ago
  Could you clarify where the privacy concern is here? As I understand it, I'm sharing my IP with search engines anyways; the only difference with a self-hosted SearX instance is that I'm sharing my server's IP instead. Is the concern that the latter's IP isn't behind a NAT, and therefore is more unique? If so, I think that's the least concerning of the identifying datapoints that a search engine has access to -- my browser metadata is far more identifying. With SearX, that information doesn't get forwarded (IIUC).
  - mekster 5y ago
    Server IP can easily be differentiated from casual internet users' IP, and since you also don't usually get a server IP rotated, you're pretty much stuck on a single IP and you're easy to be categorized as an unique user no matter which device or location you use it from.
- randomsilence 5y ago
  When you click a link on your SearX instance and you don't use referrers, how can anybody track you? Nobody knows that you are coming from your "backend" engine. You just reveal your search queries to the hosting provider if he maliciously intercepts them.
- marc_abonce 5y ago
  If you don't want to expose your IP address, you can configure searx to proxy all the queries through Tor. This obviously makes the instance way slower and you'll have to disable some engines that block Tor exit nodes, so it's a trade-off.
jqpabc123 5y ago
You do not need to trust third parties to keep you private and not track your every move, which is awesome. The only way to avoid third parties to run your own server ... but this "metasearch engine" is basically just an aggregation proxy. So every search can still be tracked back to your proxy server by Google, Bing or whoever is providing the actual results.
jefc1111 5y ago
I have been trialling Swisscows (https://swisscows.com/ https://swisscows.com/) and have found it quite useful. I have not deeply researched their privacy claims, but for now I am just trying to not use Google or mainstream alternatives. Does anyone else have experience or comments on Swisscows' search engine? Seems like an interesting company all round.
AmosLightnin 5y ago
Privacy is important, but I also care about the terrible quality of search results I get from nearly all the major providers these days. Couldn't an aggregator like SearX host a machine learning layer that learns what results are more likely to be valuable to me, and ranks them higher in the results? Keeping the customization layer on my own server and improving search results would seem to be a big advantage both privacy and performance wise.
freediver 5y ago
I always liked the term "search engine client" better (vs 'metasearch engine'). In essence it is a product that can connect to different search indexes. An "email client" does exactly the same thing, connects to different email servers and we do not call it "metaemail". edit: just realized that with the current hype around metaverse, 'metasearch' will probably be more appropriate for something searching the metaverse in the future.
tandav 5y ago
Just searched a couple of queries like "opencv rectangle", "python regex" - and it returned nothing
- unixfox 5y ago
  Which instance did you use?
  - tandav 5y ago
    https://searx.thegpm.org/ https://searx.thegpm.org/
agluszak 5y ago
Great! I wish there was a possibility to blocklist certain domains (who wants to see Quora in their results...). This should be easily implementable on Searx's side. Another feature I often wish for is searching in a specific time period. It's so annoying, for example on Youtube, when I remember that a video was released in 2011, but there's simply no filter for it.
- dalf 5y ago
  > I wish there was a possibility to blocklist certain domains You can do that in this fork: https://github.com/searxng/searxng/blob/e839910f4c4b085463f1522b4191448935468b53/searx/settings.yml#L162-L170 https://github.com/searxng/searxng/blob/e839910f4c4b085463f1...
citizenpaul 5y ago
Slightly off topic but has anyone a good solution for removing content farm seach results in this or any engine. For example some worst offenders wikihow , forbes, business i sider.
- gxnxcxcx 5y ago
  uBlacklist fully supports blocking results from Google, Bing and DDG, and partially from Startpage and Ecosia [1]. It's been available for Firefox and Chromium derivatives for a while, and there seems to be a third-party Safari port as well [2][3]. There's a bunch of language-specific blocklists on GitHub focused on StackOverflow/GitHub mirrors and wonky machine translations, but I don't know of any mature curation effort yet. [1] https://github.com/iorate/uBlacklist#supported-search-engines https://github.com/iorate/uBlacklist#supported-search-engine... [2] https://github.com/HoneyLuka/uBlacklist/tree/safari-port/safari-project https://github.com/HoneyLuka/uBlacklist/tree/safari-port/saf... [3] https://apps.apple.com/us/app/ublacklist-for-safari/id1547912640?mt=12 https://apps.apple.com/us/app/ublacklist-for-safari/id154791...
analyte123 5y ago
If you're trying to be comprehensive, a few other suggestions in rough order of their usefulness: Gigablast.com - Has been improved recently. private.sh is supposed to be a private proxy for Gigablast, but it has been broken recently Exalead.com - run by a French defense contractor for some reason filepursuit.com - search for files only. Need to play around with it more. PeteyVid.com - multi-platform video search Wiby.me - focus on "classic" style web sites
abetusk 5y ago
Search engines have been coming up lately, so maybe this is a good a place as any to discuss some back of the envelope calculations. Let's say we wanted to recreate the web index made by Google. How much cost and engineering would it take? Estimating the size of the web from worldwidewebsize.com [0], this is estimated at around 50 billion (5010^9). The average web page size looks to be on the order of 1.5 Mb (1.510^6). The nominal cost of hard disk space is about $0.02 / Gb [2]. So, roughly, that's 75 exabytes of data (~7510^15). At a cost of $0.02 / Gb that gives roughly $1.5M just to buy the hardware to store (a significant fraction of?) the web. The Hutter prize exists [3], so maybe there's some confidence that we only need to actually store 1/10 of that, so around $150k in costs. For perspective, that's 10 multi millionaire silicon valley types donating about $150k each, 100 "engineer types" at $15k each or 1000 to 10,000 pro-active citizens at $1.5k to $150 each (just* for the hard disk space, discounting energy, bandwidth and other operating costs). If we try to extrapolate lowering hard disk space costs and take the price halving to be about 2.5 years with a current (pessimistic?) cost of $0.02/Gb, that's about 10-15 years before a petabyte scale hard drive is available to the consumer for $1000. From my perspective, I would ask "why hasn't a decentralized search index been created and/or is in wide use?". My guess is that figuring out a robust enough system that's cheap enough is still out of reach. $150 might not seem like a lot, but you have to convince 10k people to devote energy just to search. Put another way, when does the landscape change enough so that decentralized search is a viable option? My guess is that when people can store a significant fraction of the web locally for nominal cost is the determining factor. Maybe some great compression and/or AI sentiment analysis can be done to bootstrap and maybe some type of financial incentives can help solve this issue, but my bet these will only provide a light push in the right direction and the needed technology is the underlying cheap disk space. As a side note, the worldwidewebsize.com [0] shows the number of indexed pages by Google holding pretty constant over a five year period with a sharp decline somewhere in 2020. I wonder if this is the method of estimation or if Google has changed something significant in their back end to alter their search engine and storage. [0] https://www.worldwidewebsize.com/ https://www.worldwidewebsize.com/ [1] https://www.pingdom.com/blog/webpages-are-getting-larger-every-year-and-heres-why-it-matters/#:~:text=According%20to%20the%20HTTP%20Archive,steadily%20increased%20over%20the%20years https://www.pingdom.com/blog/webpages-are-getting-larger-eve.... [2] https://www.backblaze.com/blog/hard-drive-cost-per-gigabyte/ https://www.backblaze.com/blog/hard-drive-cost-per-gigabyte/ [3] http://prize.hutter1.net/ http://prize.hutter1.net/
m0zg 5y ago
Or you could use the only independent search engine that actually has its own index, and search result quality comparable to that of Google: https://search.brave.com/ https://search.brave.com/
itschristmas 5y ago
I run searx locally, have it set as my default search engine, and use it as a search engine client. It's nice to have the same interface across as a variety of search engines. Searx is provided as a service on NixOS, which makes this all simple to run. I do the same for Nitter, a Twitter frontend, which supports RSS and behaves well while logged out.

299 points by cube00 5y ago

For what it's worth, it's used by by the de-Googled phone /e/ Foundation at https://spot.ecloud.global https://spot.ecloud.global and seems to work tolerably well.

mg 5y ago

Every time I stumble across a new search engine, I add it to my search engine comparison tool: https://www.gnod.com/search/ https://www.gnod.com/search/ Will add SearX now. It seems to provide reasonably good results. Update: It's on (under 'More Engines').

fsflover 5y ago
You should also add YaCy: https://yacy.net https://yacy.net.
- mg 5y ago
  It seems to be not web based? When I click on "Try out the YaCy Demo Peer", I get "502 Bad Gateway".
  - fsflover 5y ago
    It's self-hosted and peer-to-peer. You could search for other public-facing instances, e.g., http://sokrates.homeunix.net:6060 http://sokrates.homeunix.net:6060. Ideally, you could run your own instance to show the world how it works.
wenbin 5y ago
You should add the podcast search engine https://www.listennotes.com/ https://www.listennotes.com/
- mg 5y ago
  Thanks, added. Holy Moly, are there really over 100 million podcast episodes out there?
  - wenbin 5y ago
    Yes. some numbers: https://www.listennotes.com/podcast-stats/ https://www.listennotes.com/podcast-stats/ Listen Notes was started in early 2017 as a side project, when there were ~23 million episodes. I remembered seeing the number of web pages indexed by Google in early 1998 was ~25 million, then I thought that "ok, 23 million episodes might justify the existence of a podcast search engine" :)
- autoexec 5y ago
  Too bad they force you to log in to view a result or do anything but search. They also share/sell your data to 3rd parties including Google
imglorp 5y ago
This is a nice compilation. It would be very interesting if it examined and compared results.
- m-i-l 5y ago
  Yes, looks good, although I thought it was going to be a federated search, i.e. you enter your search term and it performs that search on all the sites selected. The simpler way of implementing a federated search would be to show separate results boxes from each site, although that wouldn't scale well to a large number of sites, and it can get quite complicated to try to combine the results.
deleted 5y ago
[deleted]
51stpage 5y ago
There is also Marginalia https://search.marginalia.nu/ https://search.marginalia.nu/ which I don't see on your list.
- marginalia_nu 5y ago
  It's currently undergoing its monthly maintenance just FYI. It's up and technically working, but with a drastically reduced index size.
- sysadm1n 5y ago
  Tried Marginalia. So many plaintext http links which I avoid like the plague. That's my only gripe with it. Other than that, it's an awesome tool.
  - freediver 5y ago
    > So many plaintext http links which I avoid like the plague. Why? What you described appears to be the safest place on the web.
    - sysadm1n 5y ago
      I only browse HTTPS sites. I have the `HTTPS Everywhere` addon installed with the `EASE` / Encrypt All Sites Eligible turned on so I don't accidentally browse an unencrypted website. Something like 85%/90% of the web is encrypted now, and there's no excuse to be using outdated plaintext http anymore. It's a privacy and security risk. There are only few instances where I had to view a http site (I'm a freelancer and my client's webpage was still unencrypted, so I had to see it, so a rare exception to the rule).
      - marginalia_nu 5y ago
        It's funny because I got like 70% HTTP in my index, so the whole "90% of the web is encrypted" seems to depend on which sample you are looking at. Google doesn't index HTTP at all, so that's not a good place to go looking for what's the most popular. That's in fact half the reason why I built this search engine in the first place, because they demand things of websites that some websites simply can't or wont comply with. A lot of servers still use HTTP, for various reasons. There are also some clients that can't use HTTPS.
jwithington 5y ago
where's you.com lol
- mg 5y ago
  Thanks, added.
ColinHayhurst 5y ago
Nice work. If you have a twitter handle you might request to get added to these lists; either way they might be useful for you: https://twitter.com/SearchEngineMap/lists https://twitter.com/SearchEngineMap/lists
Jaruzel 5y ago
I have this REALLY old text file of search engine URLs: http://www.jaruzel.com/textfiles/Old%20Web%20Info/Internet%20Search%20Engines%20v2.61.txt http://www.jaruzel.com/textfiles/Old%20Web%20Info/Internet%2... Google basically killed almost off of them off :( It would be great to see some proper competition in the search space, especially around specialist search engines.
- sixothree 5y ago
  I would love a search engine targeted towards developers. Searching for symbols seem to be a problem with google, not to mention all of the utterly crappy results they serve up.
  - ijr 5y ago
    Symbolhound does that.
    - stagas 5y ago
      Also grep.app for searching into repos really fast.
      - culi 5y ago
        also https://searchcode.com/ https://searchcode.com/ was posted on a HN thread about niche search engines a couple days ago
Minor49er 5y ago
You should add Fireball. It's excellent https://fireball.de/ https://fireball.de/
nix23 5y ago
A yacy instance would be good too ;) https://yacy.net/ https://yacy.net/
danskeren 5y ago
Ask.Moe nona.de
1_player 5y ago
Is anything supposed to happen when I enter something and press Enter? Nothing happens for me, FF on Windows, uBlock.
- reayn 5y ago
  You are supposed to type something into the search field then click the engine you want to use, it will pass on whatever you entered. I agree that the creator could make that a little more clear somewhere on the page.

herodotus 5y ago

Could this be installed on a Raspberry Pi? I am very happy with my Raspberry Pi-hole: would not mind adding a second pi for searching.

ChrisArchitect 5y ago

Please keep title same as the actual post: Searx - moving away from DuckDuckGo It gives more context to the topic, as in it's not just a link to the search engine itself.

egberts1 5y ago

My search engine puts a Pi symbol in lower left corner of every web page I visited.

hans_castorp 5y ago

Qwant is another (non-US) alternative

rahen 5y ago
Doesn't it use Bing results, like DuckDuckGo does? I read it's hosted on MS Azure like Bing and DDG, so in the end it's somewhat just a rebranded Bing. Quite a shame for a European (Franco-German) search engine.
- kaba0 5y ago
  Even if they were just a proxy behind bing, that would be still good, weren’t it?
  - rahen 5y ago
    Well, claiming to offer an alternative to the GAFAM while depending on their products / data / infrastructure is a bit misleading. So indeed they're doing okay privacy wise, but a lot of users feel cheated when they realize their "independent search engine" (DuckDuckGo) is just a Bing portal hosted on Azure.

GhettoComputers 5y ago

Isn’t hosting your own instance taking away every benefit of searx by revealing your IP? If you had a VPN you’d use it to mask your IP from tracking of search engines anyway, and if you used Tor for it, you’d probably move back soon since it’ll be so much worst with latency, like how many people go back to google because DDG sucks for results. I suggest just using instances you can find them at https://searx.space https://searx.space some are more reliable than others but none have been trouble free. There’s a lot of these instances for privacy, chrome has a privacy plugin with a white eye that uses nitter.net for Twitter, teddit for Reddit and other public instances. One instance of Reddit was even made completely in rust. ;) https://chrome.google.com/webstore/detail/privacy-redirect/pmcmeagblkinmogikoikkdjiligflglb?hl=en https://chrome.google.com/webstore/detail/privacy-redirect/p... To reply to the person under me, you’re always relying on a trust in something unverified and untrustworthy filters like VPNs anyway, you’re either revealing your IP using a wrapper that reveals it instantly, use a site that isn’t a search engine and might be using your data, using a VPN that is based on reputable and assumptions, usually based in another country you won’t visit or know much about aside from random reviewers, or using Tor, losing latency, reasonable speed image search, and still be possibly compromised.

kleinsch 5y ago
But if you’re using instances someone else is hosting, aren’t you hitting half the author’s objections? - They may be hosted in the US - They may be hosted on AWS - You have no idea if the maintainer of the instance is tracking you
- jarvuschris 5y ago
  ^ This right here. This article is pretty hogwash IMO Points 1 and 3 aren't relevant if they aren't recording the data. Companies in other jurisdictions have no magic invulnerability you can trust to their data getting out (legally or illegally) if they're storing it. Points 2 and 5 are equally true of any open source project unless you run it yourself from source. There are _plenty_ of examples of users getting phished by maliciously built/hosted open source tools Point 4 is obviously not malicious tracking and a mistake any project could make At the end of the day though, unless you're going to run everything yourself (which most people aren't) you have to pick who to trust -- some random person running a server somewhere, or a company with hundreds of employees recruited under the premise of working on a privacy-centric search engine who could all turn whistleblower

jostillmanns 5y ago

Whoogle is another alternative, that focuses on Google search results

ced 5y ago

From the link: The CEO sold his previous company's data before founding DDG. His previous company (Names DB) was a surveillance capitalist service designed to coerce naive users to submit sensitive information about their friends. Is that a fair statement? Can someone provide more context?

boomboomsubban 5y ago
It was a failed social network to help you reconnect with old friends. It tried to get you to recruit your friends immediately after registering and had a typical social network license. I'd say that description is intentionally describing it in the worst possible light, but not wholly inaccurate.
- deltree7 5y ago
  Yet, HN is down and ready to suck searx dick because hurr durr privacy. This is HN in a nutshell. Hint: If all you privacy-paranoid people show the exact same behavior, you are an advertiser dream. Sure, you won't be sold shampoo like general population, most advertisement companies know the exact kind of doomer-prepper items to sell if you come via VPN/DDG/whatever convoluted hack/concoction you come up to make your life inconvenient.

ColinHayhurst 5y ago

https://www.searchenginemap.com/ https://www.searchenginemap.com/

account-5 5y ago
That site is horrible on mobile, a good portion of the screen is taken up by the orange "download/view" infographic thing. Interesting though to see how connected the engines are, I would have thought DDG would be bigger with its bang option though I assume it's about what is natively included in the results.
- ColinHayhurst 5y ago
  Yes, it is horrible on mobile. The size of all the syndicating search services are the same. An update is overdue. Complementary twitters list are maintained here: https://twitter.com/SearchEngineMap/lists https://twitter.com/SearchEngineMap/lists
freediver 5y ago
I believe rightdao.com is missing from the list. It has independent index (and also impressive speed). Also not sure what the criteria for inclusion is, but search.marginalia.nu and teclis.com both have their own indexes.

skerit 5y ago

Alright, going to try searx.be for a while then.

jqpabc123 5y ago
I heard it could be a honeypot for the CIA. But feel free to prove me wrong.
- unixfox 5y ago
  If you don't want to use a single searx instance then feel free to use a random one automatically for each search thanks to this tool which can be used locally: https://searx.neocities.org/ https://searx.neocities.org/
  - jqpabc123 5y ago
    I heard these searx instances could be linked together in a honeypot network run by the CIA. But feel free to prove me wrong.

schleck8 5y ago

Metager is a non-profit, open-source search engine running fully on renewable energy. It also has a proxy for opening results anonymously https://metager.de https://metager.de

hermitsings 5y ago
https://metager.org/ https://metager.org/ for english users
czechdeveloper 5y ago
That seems quite exclusive to germany

kristianpaul 5y ago

Doesnt searx looks for results at duckduckgo and google for you anyway? Whats the difference from using ddg directly?

sebow 5y ago
I don't think it uses DDG directly.But anyways you can configure the sources for files, media, wiki,etc. Makes sense since the engine is open source, but then again it's not really a search engine itself but a metasearch one
boudin 5y ago
There's other sources available. It is a meta search engine, so it will always rely on other sources, but you can disable duckduck go and google backends.
luciusdomitius 5y ago
Isn't duckduckgo just an alternative frontend for bing with an integrated ad/tracking blocker. Or at least that's what they claim.
- nicce 5y ago
  It is. They say they add some indexing on they own, but results are all the same.
BeetleB 5y ago
Searx can ping multiple search engines, including those not supported by DDG. For example, searx has a dedicated file search, which includes torrents.

ahtaarra 5y ago

I recently made the swtich from DDG from Searx simply bevause right-clicking on a search result to copy the url resulted in a referrer link to be copied rather than the link of the result destination.

caskstrength 5y ago
I use https://addons.mozilla.org/en-US/firefox/addon/clearurls/ https://addons.mozilla.org/en-US/firefox/addon/clearurls/ to automatically convert referrals to actual links on web pages.
- ravenstine 5y ago
  I've been using ClearURLs for a while so I never even realized that DDG used referrer links. Have they always done this?
southerntofu 5y ago
I think enabling DoNotTrack header or disabling JavaScript prevents this behavior: i cannot reproduce on Tor Browser. But you are correct this is a worrying development.
boomboomsubban 5y ago
Isn't the referral link only on the ad results, which is clearly marked "AD"? That's what my quick test shows.
- ahtaarra 5y ago
  Now I am not sure what exactly the issue is but this seems to be applicable to all results, at least on my particular setup. On LibreWolf browser, On mouse hover, the correct result urls show but then I right-click one, it shows something like: https://duckduckgo.com/l/?uddg=<destination_url_here>&notrut=duckduck_in https://duckduckgo.com/l/?uddg=<destination_url_here>&notrut... But this same behaviour cannot be observed on Google/Searx on the same browser. It also even isn't observable with DDG on a FF nightly build on the same system and a FF stable build on a separate one. Edit: url formatting EDIT2: this behaviour seems to be limited to my particular setup and even a clean LibreWolf profile seems not to suffer from this issue. I apologise for the misunderstanding.

drcongo 5y ago

As someone who uses Safari with its built in list of search providers, I'm rather stuck with DDG for address bar searches, but boy it has really started to suck over the last year or two.

jeroenhd 5y ago
> with its built in list of search providers TIL. That's just... terrible UX.
boomboomsubban 5y ago
You could always do !searx
- drcongo 5y ago
  True! I'm experimenting with Ecosia right now, but also recently got on the beta for the kagi.com search engine which so far has proven to be vasty superior to any other that I've tried.
  - SyzygistSix 5y ago
    Nothing in their FAQ answers where they are located and they dance around the question of what info they track/keep by saying that they keep the minimum. That is a non-answer. I like the idea in theory but in practice I have no idea who I'm dealing with. They could be far more open about their processes. I like the idea of a paid browser though.
ColinHayhurst 5y ago
Yes, Safari is the worst offender when it comes to offering search choice on desktop. On iOS using a new app called Hyperweb you can the new Safari extensions to access and create a longer preferences list. https://hyperweb.app/ https://hyperweb.app/ We really shouldn't have to choose this or that, but should be able to easily use multiple choices in search. You can do that today as explained here, but you'll need to switch browsers. https://blog.mojeek.com/2021/09/multiple-choice-in-search.html https://blog.mojeek.com/2021/09/multiple-choice-in-search.ht...

ricardo81 5y ago

'Uses Amazon Web Services (AWS) as a cloud provider and Cloudfare CDS.' IIRC DDG uses Microsoft servers now exclusively. Makes sense given the volume of queries they're handling and all dependent on Bing API.

hermitsings 5y ago

I had been considering using Searx (which I had known about before) lately since I have to use DDG+Google for getting satisfactory results. Edit: I really like DDG bang and vim like nav keys tho

visiblink 5y ago
Someone here once posted a link to duckduckstart.com. I have it set up in my search bar now. If you search, it goes through Startpage (Google results, more privately). If you search with a bang, it goes through Duckduckgo. It's probably close to what you're looking for.
- hermitsings 5y ago
  saw it just now. wow thanks

phantom_oracle 5y ago

I will add add a disclaimer to this comment that it is tinfoil-hat and just speculation(bordering on conspiracy) but many of these "we are a privacy-first company" might actually just be honeypots and fronts for 3-letter agencies. The comment is not wholly conspiratorial, considering the CIA owned Swiss crypto company: Crypto AG [1] It's within the realm of possibility that most of these privacy services could be owned by 3-letter agencies or small enough to be coerced into cooperation. [1] https://www.scmp.com/news/world/europe/article/3050193/crypto-spy-caper-cia-secretly-owned-swiss-company-sold-encryption https://www.scmp.com/news/world/europe/article/3050193/crypt...

no_time 5y ago
Same goes for VPN companies. I do feel bad for all the journalists and whistleblowers who will fall for these scams but as far as I'm concerned, as long as I can avoid for profit data collection companies like Google, it's good enough for me. If I can't avoid my data being collected, I will still try my best to make it as worthless as possible just out of spite.
jqpabc123 5y ago
Haven't you heard? The CIA has gone open source. They don't need to own a company anymore. They can just download the Searx source code; modify it as they see fit, and make it available on a server someplace. Can you prove that searx.be isn't run by a "3 letter agency"? Can you prove that the source code running at searx.be is the same as on Github? The point being --- unless you have full access to the server, open source means nothing with regard to privacy and security of any service. It actually means less than nothing --- it means it is super easy to build into a honeypot.
- marc_abonce 5y ago
  Of course, there's no fool-proof solution to knowing what code is running in the server side, but https://searx.space https://searx.space at least shows if an instance modified their client-side code, which you can see in the HTMl column. To mitigate server-side code from identifying you, you can consume an instance from Tor. Of course, you could try to do that with any other search engine, but most of the other search engines either block exit nodes or provide incomplete functionality if you disable JS. It's not perfect, but it may be good enough depending on your threat model.

ojosilva 5y ago

Are there any opensource real internet search engines worth looking at? I think we should be working on disrupting search as a whole instead of depending on the Googles, Baidus and Bings of the world. I'm fully aware of the massive crawling and storage requirements, but opensource projects that can get search right can later 1) be hosted by the powerhouses of the cloud or non-profit parties, or 2) become a fully distributed hosting and crawling effort as in p2p and blockchain.

vindarel 5y ago
p2p: there's the Yacy effort. https://yacy.net/ https://yacy.net/ I… couldn't find a portal to try it out (I did years ago, the results need… to be discussed. It's anyways easy to install and to choose what part of the web to crawl.) > YaCy is free software for your own search engine. maybe they rebranded and don't aspire to be a complete web search engine?
deleted 5y ago
[deleted]

YXNjaGVyZWdlbgo 5y ago

I had a searx instance running for a long time and it's great when it works but the plugins for site specific searches break all the time and if you have more than 3-4 users with a high search frequency google blacklists your IP by throttling.

kaba0 5y ago
May I ask about what’s your idea of preserving privacy with self-hosting a website that searches for specific terms from a presumably fixed IP, where by 1/3-1/4 chance it can be attributed to you?
- YXNjaGVyZWdlbgo 5y ago
  The instance ran on a mobile connection not associated with any private information. EDIT: It was located in a german street light 20km away from any of the users. Just to get the geolocation question out of the way. It was more of an experiment than anything else there will be a talk about it and other FreiFunk (Open Mesh Network in Germany) related stuff at the next virtual CCC congress.
  - dalf 5y ago
    Did you renew the mobile IP from time to time? Note: since the version 1.0, searx stops sending request for 1 day when a CAPTCHA is detected which might help a little. (I'm really interested by the results of your experiment)
    - YXNjaGVyZWdlbgo 5y ago
      We renewed every 12 hours in the end we came to the conclusion that google might discriminate traffic from eastern european states. The sim cards were mostly from poland, belarus and the ukraine. When we tested it against german, french and italian cards captchas were way more frequent on the eastern cards and showed way earlier. I will ping you as soon as the talk is online.

xwdv 5y ago

People don’t want privacy. They want results. Society programs us to think privacy is our top concern. Is it?

InfiniteRand 5y ago
I feel like privacy is a concern relative to the chance of my information being misused and the damage from the misuse of that information. If a search engine wants to collect random facts about me but that collection is extremely unlikely to affect me in any way I couldn’t care less
marginalia_nu 5y ago
I do think you are mostly correct. Some people really care about privacy, but for most people it isn't a huge concern. This doesn't make it any less important, but just means that if your main selling point is "we're the search engine that cares about privacy", then odds are you're not going to get a lot of users.
- lardolan 5y ago
  I agree with most of your points. Although It may be matter of a time to grow that niche. IMO Pople who value it are willing to take sacrifice of less functional outputs. Privacy is most effective selling point when working with sensitive information.
  - marginalia_nu 5y ago
    If you are working with sensitive information, the last thing you probably want to do is broadcast that you are working with sensitive information.
    - kreeben 5y ago
      Au contraire, it's the first thing you should broadcast, unless you're trying to scam people out of their PII.
deleted 5y ago

goldsteinq 5y ago

The problem with self-hosted search engine is that they make you _very_ unique: you’re the only client of the ”backend“ engine with that (static and non-NATed) IP. Furthermore, you’re now one of the small group of people with ”hosting“ IPs. Using self-hosted SearX may make you easier to track, not harder. Using SearX hosted by someone else is marginally better, but now you have to trust the owner of the server, which is probably not what you want for privacy-centered search engine.

woodruffw 5y ago
Could you clarify where the privacy concern is here? As I understand it, I'm sharing my IP with search engines anyways; the only difference with a self-hosted SearX instance is that I'm sharing my server's IP instead. Is the concern that the latter's IP isn't behind a NAT, and therefore is more unique? If so, I think that's the least concerning of the identifying datapoints that a search engine has access to -- my browser metadata is far more identifying. With SearX, that information doesn't get forwarded (IIUC).
- mekster 5y ago
  Server IP can easily be differentiated from casual internet users' IP, and since you also don't usually get a server IP rotated, you're pretty much stuck on a single IP and you're easy to be categorized as an unique user no matter which device or location you use it from.
randomsilence 5y ago
When you click a link on your SearX instance and you don't use referrers, how can anybody track you? Nobody knows that you are coming from your "backend" engine. You just reveal your search queries to the hosting provider if he maliciously intercepts them.
marc_abonce 5y ago
If you don't want to expose your IP address, you can configure searx to proxy all the queries through Tor. This obviously makes the instance way slower and you'll have to disable some engines that block Tor exit nodes, so it's a trade-off.

jqpabc123 5y ago

You do not need to trust third parties to keep you private and not track your every move, which is awesome. The only way to avoid third parties to run your own server ... but this "metasearch engine" is basically just an aggregation proxy. So every search can still be tracked back to your proxy server by Google, Bing or whoever is providing the actual results.

jefc1111 5y ago

I have been trialling Swisscows (https://swisscows.com/ https://swisscows.com/) and have found it quite useful. I have not deeply researched their privacy claims, but for now I am just trying to not use Google or mainstream alternatives. Does anyone else have experience or comments on Swisscows' search engine? Seems like an interesting company all round.

AmosLightnin 5y ago

Privacy is important, but I also care about the terrible quality of search results I get from nearly all the major providers these days. Couldn't an aggregator like SearX host a machine learning layer that learns what results are more likely to be valuable to me, and ranks them higher in the results? Keeping the customization layer on my own server and improving search results would seem to be a big advantage both privacy and performance wise.

freediver 5y ago

I always liked the term "search engine client" better (vs 'metasearch engine'). In essence it is a product that can connect to different search indexes. An "email client" does exactly the same thing, connects to different email servers and we do not call it "metaemail". edit: just realized that with the current hype around metaverse, 'metasearch' will probably be more appropriate for something searching the metaverse in the future.

tandav 5y ago

Just searched a couple of queries like "opencv rectangle", "python regex" - and it returned nothing

unixfox 5y ago
Which instance did you use?
- tandav 5y ago
  https://searx.thegpm.org/ https://searx.thegpm.org/

agluszak 5y ago

Great! I wish there was a possibility to blocklist certain domains (who wants to see Quora in their results...). This should be easily implementable on Searx's side. Another feature I often wish for is searching in a specific time period. It's so annoying, for example on Youtube, when I remember that a video was released in 2011, but there's simply no filter for it.

dalf 5y ago
> I wish there was a possibility to blocklist certain domains You can do that in this fork: https://github.com/searxng/searxng/blob/e839910f4c4b085463f1522b4191448935468b53/searx/settings.yml#L162-L170 https://github.com/searxng/searxng/blob/e839910f4c4b085463f1...

citizenpaul 5y ago

Slightly off topic but has anyone a good solution for removing content farm seach results in this or any engine. For example some worst offenders wikihow , forbes, business i sider.

gxnxcxcx 5y ago
uBlacklist fully supports blocking results from Google, Bing and DDG, and partially from Startpage and Ecosia [1]. It's been available for Firefox and Chromium derivatives for a while, and there seems to be a third-party Safari port as well [2][3]. There's a bunch of language-specific blocklists on GitHub focused on StackOverflow/GitHub mirrors and wonky machine translations, but I don't know of any mature curation effort yet. [1] https://github.com/iorate/uBlacklist#supported-search-engines https://github.com/iorate/uBlacklist#supported-search-engine... [2] https://github.com/HoneyLuka/uBlacklist/tree/safari-port/safari-project https://github.com/HoneyLuka/uBlacklist/tree/safari-port/saf... [3] https://apps.apple.com/us/app/ublacklist-for-safari/id1547912640?mt=12 https://apps.apple.com/us/app/ublacklist-for-safari/id154791...

analyte123 5y ago

If you're trying to be comprehensive, a few other suggestions in rough order of their usefulness: Gigablast.com - Has been improved recently. private.sh is supposed to be a private proxy for Gigablast, but it has been broken recently Exalead.com - run by a French defense contractor for some reason filepursuit.com - search for files only. Need to play around with it more. PeteyVid.com - multi-platform video search Wiby.me - focus on "classic" style web sites

abetusk 5y ago

Search engines have been coming up lately, so maybe this is a good a place as any to discuss some back of the envelope calculations. Let's say we wanted to recreate the web index made by Google. How much cost and engineering would it take? Estimating the size of the web from worldwidewebsize.com [0], this is estimated at around 50 billion (5010^9). The average web page size looks to be on the order of 1.5 Mb (1.510^6). The nominal cost of hard disk space is about $0.02 / Gb [2]. So, roughly, that's 75 exabytes of data (~7510^15). At a cost of $0.02 / Gb that gives roughly $1.5M just to buy the hardware to store (a significant fraction of?) the web. The Hutter prize exists [3], so maybe there's some confidence that we only need to actually store 1/10 of that, so around $150k in costs. For perspective, that's 10 multi millionaire silicon valley types donating about $150k each, 100 "engineer types" at $15k each or 1000 to 10,000 pro-active citizens at $1.5k to $150 each (just* for the hard disk space, discounting energy, bandwidth and other operating costs). If we try to extrapolate lowering hard disk space costs and take the price halving to be about 2.5 years with a current (pessimistic?) cost of $0.02/Gb, that's about 10-15 years before a petabyte scale hard drive is available to the consumer for $1000. From my perspective, I would ask "why hasn't a decentralized search index been created and/or is in wide use?". My guess is that figuring out a robust enough system that's cheap enough is still out of reach. $150 might not seem like a lot, but you have to convince 10k people to devote energy just to search. Put another way, when does the landscape change enough so that decentralized search is a viable option? My guess is that when people can store a significant fraction of the web locally for nominal cost is the determining factor. Maybe some great compression and/or AI sentiment analysis can be done to bootstrap and maybe some type of financial incentives can help solve this issue, but my bet these will only provide a light push in the right direction and the needed technology is the underlying cheap disk space. As a side note, the worldwidewebsize.com [0] shows the number of indexed pages by Google holding pretty constant over a five year period with a sharp decline somewhere in 2020. I wonder if this is the method of estimation or if Google has changed something significant in their back end to alter their search engine and storage. [0] https://www.worldwidewebsize.com/ https://www.worldwidewebsize.com/ [1] https://www.pingdom.com/blog/webpages-are-getting-larger-every-year-and-heres-why-it-matters/#:~:text=According%20to%20the%20HTTP%20Archive,steadily%20increased%20over%20the%20years https://www.pingdom.com/blog/webpages-are-getting-larger-eve.... [2] https://www.backblaze.com/blog/hard-drive-cost-per-gigabyte/ https://www.backblaze.com/blog/hard-drive-cost-per-gigabyte/ [3] http://prize.hutter1.net/ http://prize.hutter1.net/

m0zg 5y ago

Or you could use the only independent search engine that actually has its own index, and search result quality comparable to that of Google: https://search.brave.com/ https://search.brave.com/

itschristmas 5y ago

I run searx locally, have it set as my default search engine, and use it as a search engine client. It's nice to have the same interface across as a variety of search engines. Searx is provided as a service on NixOS, which makes this all simple to run. I do the same for Nitter, a Twitter frontend, which supports RSS and behaves well while logged out.