10 ms·
SearXNG is a free internet metasearch engine
- ozehlaw 3y agoInteresting. Google is going to shit these days.
- YvUuXJiO 3y agoi dont think so
- nickburns 3y agoi don't either. it's done been to shit for some time now. scraping its results can be highly effective though.
- mrexroad 3y agoWeb content itself had gone to shit these days, in order to win google’s SEO game to win google’s Adsense game. “Google going to shit” is just a second order effect (or third/forth depending how you look at it).
- prmoustache 3y agoThe good content has not disappeared. So it is still google going to shit if it can't make up what is good and what isn't, which was the reason people started using it in the first place 25 years ago.
- ijijijjij 3y agoGoogle search has gone to shit since Google+ .... or more precisely, when they removed the plus operator in Google search around 2011. And no, the quotes aren't as good. My bet is that Google will become "Google TV" and search won't be possible. They will just show you what they want. They'll probably frame it as "AI knows what you want to see". Maybe they should ban Google instead of TikTok (I don't use either though).
- bilegeek 3y agoOOC, does this support YaCy as another engine? Would be the best of all worlds if it did IMO.
- hagbard_c 3y agoIt does, I run it that way with an optional fan-out to my personal YaCy instance. Here's the relevant part of settings.yml: - name: yacy engine: yacy categories: general search_type: text base_url: https://yacy.searchlab.eu shortcut: ya disabled: true # required if you aren't using HTTPS for your local yacy instance # https://docs.searxng.org/dev/engines/online/yacy.html # enable_http: true # timeout: 3.0 # search_mode: 'global' Change 'disabled' to 'false' and point it at whatever YaCy instance you want to use. It can use the 'general' and 'images' categories.
- sitkack 3y agoRun this on your personal internet connection, risk not being able to use a search engine.
- mostlysimilar 3y agoElaborate?
- HeatrayEnjoyer 3y agoYour IP address will get burned
- marginalia_nu 3y agoVirtually all public search engine endpoints see an insane amount of bot activity, often several queries per second. If you delegate queries to e.g. google or bing at that rate, you'll be ip blocked in a heartbeat.
- mostlysimilar 3y agoAh duh, for some reason my mind didn't go to hosting the search instance locally and I misunderstood. btw thank you for Marginalia! The spirit of the small web is very important to me.
- deleted 3y ago[deleted]
- RaisingSpear 3y agoSearch engines: they scrape the web, but get narky when scraped themselves.
- marginalia_nu 3y agoDifference is a crawler paces the requests, respects robots.txt and rate limits, and doesn't typically invoke 50-100MB disk I/O per request. Like I don't mind automated access to my search engine, I even offer a public API to the effect, that you can in fact hook into SearXNG. What I mind is when one jabroni with a botnet decides their search traffic is more important than everyone else's and grabs all the compute for himself via a sybil attack.
- longitudinal93 3y agoBeen running thieves the default on all my devices for the past year and I couldn't be happier. Have only had it choke twice and it just needed to be updated to be back in business.
- xnx 3y agoThis takes me back. Before Google, meta search tools increased your odds of finding a decent answer between the spammy results from Alta Vista, Hotbot, Lycos, etc.
- 8ig8 3y agoDogpile was another.
- canadianfella 3y ago[dead]
- nickburns 3y agomemories. and now it's come back around some decades later in the name of digital privacy.
- rockskon 3y agoHow are they private though? Unless they relay your search it is the exact opposite of private.
- notpushkin 3y agoThey do indeed relay your query. How else would they work?
- yayr 3y agoI would assume that the relaying can strip the request from identifying information such as IP, cookies and other tracking mechanisms that you get when visiting e.g. google.com.
- sneela 3y agoIf you host your own instance: > SearXNG protects the privacy of its users in multiple ways regardless of the type of the instance (private, public). Removal of private data from search requests comes in three forms: > 1. removal of private data from requests going to search services > 2. not forwarding anything from a third party services through search services (e.g. advertisement) > 3. removal of private data from requests going to the result pages From: https://docs.searxng.org/own-instance.html#how-does-searxng-protect-privacy https://docs.searxng.org/own-instance.html#how-does-searxng-... The docs mention a caveat below at "What are the consequences of using public instances?": > If someone uses a public instance, they have to trust the administrator of that instance. This means that the user of the public instance does not know whether their requests are logged, aggregated and sent or sold to a third party.
- nickburns 3y agouse when i'm tired of picking obfuscated Fumo plushies or Minecraft screencaps on https://4get.ca/ https://4get.ca/. i don't even know what a Fumo plushie is, never mind six of them.
- arcastroe 3y agoThis is great. I wish there was a way to block certain domains from ever appearing on search results. EDIT: Looks like there's already an open issue: https://github.com/searxng/searxng/issues/2351 https://github.com/searxng/searxng/issues/2351
- squarefoot 3y agouBlacklist does just that with Google and some other search engines. I use it with Firefox to filter out pinterest junk from search results. Also available for Chrome and Safari. https://github.com/iorate/ublacklist https://github.com/iorate/ublacklist
- poulpy123 3y agojust installed it to try. For the people that want to give it a try also, I noticed that several of the public list contains legitimate websites such as canva or reddit
- BeetleB 3y agoKagi does that.
- arcastroe 3y agoKagi requires an account to use, which is not great for privacy. I understand Kagi is generally reputable, but I like the idea of a self-hosted alternative where you're in full control.
- lannisterstark 3y agoI trust myself a bit more than I trust someone else to run my queries sadly. I understand that they claim to store no user data or associations etc, but honestly, it's just their word.
- BeetleB 3y ago> I understand that they claim to store no user data or associations etc, but honestly, it's just their word. My guess is that if they are found to do so, then they open themselves up to lawsuits. Not collecting data isn't merely a perk - it's practically the reason Kagi exists.
- keepamovin 3y agoThat's clever. X-ING (like those 'crossing' roadsigns), so it's like Search-ching. There's quite some similarity between the CH and the X sound in English. But, as this is HN probably someone with a PhD in comparative phonetics will explain why this is a common and infuriating misunderstanding of layfolken.
- lannisterstark 3y ago^_^' Hah. FWIW, it's a fork of searX. https://github.com/searx/searx https://github.com/searx/searx
- keepamovin 3y agoHahaha! :) Good to know
- nickburns 3y agothe abbreviation "ng" in software development/forking/maintenance evolution denotes 'next generation.'
- yunohn 3y agoX is pronounced as ch in Catalan too. https://spanish.stackexchange.com/questions/16203/use-of-x-instead-of-ch-in-internetslang#:~:text=x%20is%20pronounced%20%22ch%22%20(,like%20k%20instead%20of%20qu https://spanish.stackexchange.com/questions/16203/use-of-x-i....
- DanielBen6 3y ago[flagged]
- lygten 3y agoTry this guy. Its not Kagi, but the search results are pretty good. Host it yourself on Docker. https://felladrin-minisearch.hf.space/
- keepamovin 3y agoIf anyone is interested in searches applied to the full text of every page in your browser history, or to only select pages that you bookmark, check out our project DownloadNet (formerly, and possibly, futurely: "DiskerNet"). It hooks into your browser to give you an augmented experience. The UI is pretty simple (think 1997 era google but without CSS haha), and we don't do anything super complex with search (but could in future), but it works not bad. Check it out!!! https://github.com/dosyago/DownloadNet https://github.com/dosyago/DownloadNet Oh, it also makes your content (again either everything you browsed or only what you booked) available offline. So if you work on an oil rig, or shipping, or long haul freight, can be a good way to browse as normal but save yer satellite bandwidth!!!
- DavideNL 3y agoI don't see any documentation at all, like what browsers are supported, etc. :/
- alexdeloy 3y agoMe neither, it really would benefit from a better documentation since I like the idea a lot. I just tried it out and it seems to be tied to Chrome. Since I use Firefox and Chromium as my daily drivers this does not work for my case. I understand that they probably rely on some Chrome internals to dig through the content, a SOCKS Proxy approach would have worked better and would have no need to switch between a "save" and "serve" mode. But then again I was only scraping the top of it because of the lack of browser support. Will keep an eye on this one though!
- outofpaper 3y agoWhile it lacks a search feature last I checked there's always https://github.com/davidfstr/webcrystal https://github.com/davidfstr/webcrystal One .py file. Only one dependency (urllib3).n with a little love the concept could become a full transparent proxy.
- nickburns 3y agoit's unclear to me why anyone, particularly anyone with even a passing interest in what the topic of this submission has to offer, would be even remotely interested in being the "master archivist of your own internet browsing." i don't need anything else archiving anything related to my internet browsing except for my human brain. and yes, that's just me... but how is the shameless plug of this not just therefore off-topic but diametrically-opposed-to-total-personal-privacy tool appropriate here?
- kristjank 3y agoIt's my favourite tool to average out the individually shitty results of mainstream search engines into something vaguely usable.
- RGamma 3y agoThere's also https://metager.org https://metager.org
- deleted 3y ago[deleted]
- cess11 3y agoBeen using SearX/SearXNG a lot over the years, in large part because I used and preferred the Dogpile meta search many, many years ago. Apparently Dogpile still exists, didn't expect that: https://en.wikipedia.org/wiki/Dogpile https://en.wikipedia.org/wiki/Dogpile
- ranguna 3y agoLots of people publicly host searx instances. There's a list of publicly available instances online, but if you are looking for a tool that randomly redirects you to an instance for every search you do on your browser's bar, you can use neocities: https://searx.neocities.org/changelog https://searx.neocities.org/changelog I use this all the time. A downside is that sometimes you land on an instance that doesn't provide any results or gives you really poor ones. This has been happening less frequently recently.
- fasa99 3y agoPeople always sell Sear, but myself, I'm a fan of presearch.com I have no affiliation with them whatsoever or financial interest. I have no interest in their crypto based business model. In fact I think their lack of google or bing style search result filtering is entirely due to lack of funding and/or prioritizing other things more important to success, not due to taking a stand on free speech or anything like this. And that's perhaps how it was in the early days of the internet, when maslow's hierarchy of corporate needs focused on trying to make the thing work versus public relations goodfeels and presenting only rightspeech. Anyway, if I'm looking for some topic I believe google would be known to filter heavily, or something esoteric, I take a look at presearch to get a second opinion. I'd also love to see archive.org do something similar, archive.org has an amazing collection of data, poorly indexed and poorly searchable.
- dalf 3y agoA few years ago, I remember someone conducted a study on the quality of SearX(NG) results using different Internet providers: mobile, fiber, and VPN. I'm not sure if this person is still active on HN, but I'm really curious about the results.
- skoocda 3y agoFirst few instances I tried are either returning no results, or only DDG results. Error! Engines cannot retrieve results: brave ( Suspended: too many requests ) google ( Suspended: too many requests ) qwant ( server API error )
- johnklos 3y agoI've run in to exactly that. By putting my SearXNG on a machine that also does NAT for a busy network, this can be avoided. This is definitely one instance where IPs from a colo are a bad thing and residential IPs are a good thing ;)
- johnklos 3y agoSearXNG is great, with a few caveats: Running it on a machine that also does NAT for many other machines helps to prevent getting blocked by upstream search engines like DuckDuckGo. It'd be good if access to certain upstream search engines could be sent through, say, a proxy set up elsewhere to prevent this very, very common problem, if you can't run it from an IP used for other things. I'd like to figure out how to have a mode where my search is 100% literal - where every word I type must be in the search results exactly as I type them. Perhaps that's the equivalent of putting a "+" in front of each word, and putting each word in quotes? It's annoying that my words are constantly getting changed for me because there aren't many results, which I expressly don't want. Like mentioned elsewhere, I want to be able to explicitly exclude certain domains. I get that SearXNG wants to be stateless, but I could either configure a separate URL for it or simply configure it for all searches. For instance, if I search for a PDF manual for something, I never, ever, ever want to see anything from "manualslib.com" and sites like it. Other than these things which'd be nice to address, I'd say running SearXNG and encouraging people to use it instead of Google has worked quite well :)