5 ms·
SearXNG: A free internet metasearch engine
- satvikpendem 4mo agoTinySearch wraps this and works well for agents. It's better than the native SearXNG MCP because it optimizes the context before it even gets to the agent so as to not waste tokens. https://github.com/MarcellM01/TinySearch https://github.com/MarcellM01/TinySearch
- deleted 4mo ago[deleted]
- ProofHouse 4mo agoProps
- drnick1 4mo agoSearXNG did not include a built-in MCP server, last time I checked.
- Havoc 4mo agoIt does have json response though so pretty trivial to get an LLM to make you a mcp
- satvikpendem 4mo agoYou're right, I was previously using "MCP-SearXNG" and must've thought it was official for some reason.
- dexterdog 4mo agoI've been self hosting this as my default engine across all of my searches for a few years now. I can't recommend it more highly.
- viviansolide 4mo agoSame experience
- ProofHouse 4mo agoI’ll have to try, I’ve only recently learned Exa pricing is a bit crazy (especially on searches where you source 30-40 sources)I just used it be default and then was like oh damn when I got hit
- tomnow 4mo ago[flagged]
- ManWith2Plans 4mo agoI've been using this for some projects. It's exceptional and I recommend it highly. I actually included a recipe to deploy it to kubernetes in typekro, my TypeScript infrastructure-as-code project for kubernetes: https://typekro.run/api/searxng/ https://typekro.run/api/searxng/
- arikrahman 4mo agoI have used SearXNG hosts like https://searx.be/ https://searx.be/ but stick with Brave search for the most part. Are there other good hosts people tend to use?
- vimredo 4mo agoPersonally, I self-host it myself. All the hosts I tried either errored often, or gave search results that were complete garbage.
- artooro 4mo agoIt works well if you connect it the Brave Search API, but using it a scraper is fairly unreliable. Google stopped working a few days ago.
- tomfow 4mo ago[flagged]
- salmonik 4mo agoI prefer 4get.
- another_twist 4mo agoBeen a fan of searX for a while. Not sure if this is the same thing but there were plenty of hosted versions too.
- tom6ow 4mo ago[flagged]
- tom6ow 4mo ago[flagged]
- fishgoesblub 4mo agoI've been using SearXNG for a few years now, however I've been trying out Degoog as a SearXNG alternative since I've had issues with engines constantly failing or being slow since day 1 of using SearXNG, but Degoog has worse results with the same engines. It's a shame since I'm having to pick between slower but better results, or very fast but worse results.
- tom9ow 4mo ago[flagged]
- noobcoder 4mo agohow do i configure which specific search engines SearXNG pulls its results from? Can we extend it to onyl search Stack Overflow and GitHub
- rcarmo 4mo agoYears of regular use here, has been great even before I started using it as an agent tool.
- lucasrufkahr 4mo agoYeah, I find that searx results are way more relevant to what I’m actually looking for than a single engine. There’s so much manipulation going on that if you don’t aggregate multiple engines, it’s near impossible to get what you want.
- goodroot 4mo agoThis appears to be a key tool for providing search to local models. I'm curious what setups folks use to provide this functionality. Since the quantized 24B parameter Gemma model came out, I've had good luck with tool calling on a 4070 Ti Super. Successful tool calling is what finally made the local experience useful. I should note this is for the general and not coding specific context.
- drnick1 4mo agoI am also interested in what a full local AI stack with web search and other tools looks like. As far as I can tell, SearX does not embed an MCP server, so it can't be directly called from llama-server for example. Open WebUI does have an integration for SearX and other providers, but the results I obtained weren't particularly impressive.
- c-hendricks 4mo agoI use Searxng through Onyx, both as regular search and Onyx's Deep Research mode. I also have https://github.com/ihor-sokoliuk/mcp-searxng https://github.com/ihor-sokoliuk/mcp-searxng to add search to coding agents. Haven't really had many issues with it.
- gardnr 4mo agoIt has a JSON mode that you need to enable in settings and then you can create a simple python script to interact with it or have the agent use `curl` and `jq` to interact with it. It's at the bottom of this page: https://docs.searxng.org/admin/settings/settings_search.html https://docs.searxng.org/admin/settings/settings_search.html
- jared0x90 4mo agoare you running a quant? i have a friend with a 4080 that is wanting to experiment with local models and those cards should be similar enough. can you give any more detail about your setup? ty!
- goodroot 4mo ago
- asciimoo 4mo agoOhi, I'm the original creator of Searx, but due to the limitations of the metasearch concept I'm not involved in the development anymore. My new search project is https://github.com/asciimoo/hister https://github.com/asciimoo/hister (https://hister.org/ https://hister.org/). Hister is a full text indexer for websites and local files which automatically saves all the visited pages rendered by your browser. Storing full page content allows serving offline result previews and the full page content via MCP. Take a look at how the MCP can be utilized: https://hister.org/posts/give-your-ai-assistant-a-private-memory https://hister.org/posts/give-your-ai-assistant-a-private-me...
- operatingthetan 4mo agoI installed this a while back and honestly I almost never touch it. It turns out that for me searching my history doesn't really replace a search engine at all. The built in extractor list is pretty limited and adding them seems like too much of an ordeal for me to bother.
- asciimoo 4mo agoSure, it cannot fully replace web search engines (yet), but it can reduce the dependence on these services more and more as your index grows. Hister is designed to support quickly falling back to traditional search engines with a single hotkey if no results found. I agree, we should add more extractors [1]. Can you recommend extractors you missed? [1] https://github.com/asciimoo/hister/issues/305 https://github.com/asciimoo/hister/issues/305
- deleted 4mo ago[deleted]
- kristianpaul 4mo agoIs this similar to fastcrw ?
- asciimoo 4mo agoBoth are search engines, but that's all the similarity. Hister has a traditional crawler, but its biggest strength is automatically indexing browser tabs as those are rendered. This way it bypasses authentication, CloudFlare, captchas and most of the annoying limitations of traditional crawlers. Hister also provides full offline result previews. Check out the small read-only demo: https://demo.hister.org/ https://demo.hister.org/
- exiguus 4mo agoSearXNG is my daily internet search now +5 years; with YaCY Backends and else as fallback. I also build internal document search or RAG applications with this setup (SearXNG also support json results). However, there are some downer I accept because of privacy: 1. Its slower and the results are not that good then with others. But fast and good enough for most of my queries. 2. From time to time you get blocked on the duckduckgo, brave or whatever search and you must solve some captures. You can prevent this by getting and using API-Keys from them. The nice thing about using your own backend is, that you can prio it in the results and for example, if I crawl the smallweb and other site important for myself, this sites come up first in the results.
- sunshine-o 4mo ago> SearXNG is my daily internet search now +5 years Same here > with YaCY Backends and else as fallback. Do you run your own "super fast" YaCy instance? or with specific settings? My experience with YaCy is it doesn't fit in the backend of SearX since YaCy kind of slowly stream results for about 30 seconds... I also have a local `kiwix-serve` serving ZIM files of wikipedia, wiktionary, gutemberg, archwiki, etc. but same problem the kiwix search engine [0] doesn't really fit as a backend for SearX as it returns too many results and pollute the SearX result page. What I haven't done yet is trying to plug SearX to a local Recoll instance [1]. But Recoll doesn't support indexing ZIM files... but could be useful for other archived html documents. I would be curious to know more about a working setup since search is hard to get right. - [0] https://kiwix-tools.readthedocs.io/en/latest/kiwix-serve.html#search https://kiwix-tools.readthedocs.io/en/latest/kiwix-serve.htm... - [1] https://docs.searxng.org/dev/engines/online/recoll.html https://docs.searxng.org/dev/engines/online/recoll.html
- exiguus 4mo agoI ran my own YaCY instances. Three of them to be specific, because they are "super fast" and "reboot" often. I crawl with them the smallweb, smallcomic and smallyt sites and also all feeds from my miniflux instance; getting them via the miniflux api. Beside that i have other static entries that i crawl. For wikibooks and wikipedia i tried and use also YaCY, but it use a lot of resources. So its only in one instance. I suggest >16GB RAM and 300GB+ HDD if you want to do this. To access wikimedia, gutemberg, archwiki or media.ccc.de directly, I use also SearXNG. Usually it takes 1-3 Seconds to get search results from YaCY in my setup. I run them in docker on aarch64 with ~6GB of RAM and 200GB HDD. The VPS it-self has 8GB RAM, 6 arm cores and 250GB HDD. If YaCY hang, i just restart it. This are my pretty good working docker deploy and java settings I use currently: environment: JAVA_OPTS: >- -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+ParallelRefProcEnabled -XX:+UseStringDeduplication -XX:InitiatingHeapOccupancyPercent=45 -XX:G1ReservePercent=15 -Xms1024m -Xmx3072m -XX:MaxMetaspaceSize=256m -XX:MaxDirectMemorySize=256m -XX:+ExitOnOutOfMemoryError -XX:G1HeapWastePercent=10 -XX:G1MixedGCCountTarget=4 deploy: resources: limits: cpus: "4.2" memory: 5.2G reservations: cpus: "2" memory: 2.5G healthcheck: test: | /bin/bash -c ' if ! timeout 55s wget --spider --no-verbose http://127.0.0.1:8090/yacysearch.html?query=exiguus; then exit 1 fi if ! timeout 55s yacy_search_server/bin/checkalive.sh; then exit 1 fi exit 0 ' interval: 120s timeout: 60s retries: 3 start_period: 240s That's the smallest I got it running mostly stable and self-healing with a index size of +100GB. I also avoid to use crawling by the build in tasks and use the API and cron jobs for weekly feed importing, because I found out, that kind of crawling eats up less resources then the usual. All-Over, to much running crawlers, make retrieving search results slow. For production use, I suggest to min. double the resources. If you do this, it becomes very stable. Thanks to pointing out kiwix. I'll give it a try.
- tosief 4mo ago[dead]
- chatmasta 4mo agoI’ve always liked this tool, but I’m of two minds regarding the privacy gained by sending my searches to 280 companies instead of just one.
- MrDrMcCoy 4mo agoFriendly reminder that if your user and traffic count is low, your traffic is still unique and able to be profiled. Love and use this project, though.
- ninjahawk1 4mo agoI’ve been a big fan of SearXNG for a while now. My distain for google has only grown, so having the ability to search and avoid things like yk, small AI models being installed on my PC without my consent, is awesome.
- queenkjuul 4mo agoHighly recommend this. I set up a self-hosted instance and been using it exclusively for months. It's better than DDG and i don't miss Google whatsoever. Image search is worse i guess mostly for lack of CDN so it's slow but whatever.
- jaygreat2020 4mo agoThis is really great!
- grigio 4mo agosearxng is fantastic for AI agents
- denysvitali 4mo agoI've built https://github.com/denysvitali/searxng-mcp https://github.com/denysvitali/searxng-mcp to use this as an MCP for coding agents. Works very well, until you get rate limited by the providers (e.g: DDG). It also needs a SearXNG server to run, so I recently pivoted towards a self-contained solution: https://github.com/denysvitali/search-mcp https://github.com/denysvitali/search-mcp
- nikvdp 4mo agoI built something similar ([1]) that you might find interesting. Similar to your project, but with the fun tweak that it bundles searxng inside itself, so you don't need to run or find a searxng instance to use it. [1]: https://github.com/nikvdp/searxng-ai-kit https://github.com/nikvdp/searxng-ai-kit
- RandyOrion 4mo agoI've been using searxng for several years now. I don't run my own instances because the inhumane network censorship imposed by GFW, and proxy detection enforced by search engines. Instead, I rely on public instances on the list [1] and libredirect [2]. Note that service from a single instance is not guaranteed, but you can always switch to other available instances with little cost within a minute. I won't say searxng can help you degoogle because metasearch engine calls other search engines, e.g., google, to collect results. However, if you try searxng, you can at least get rid of things like ai reviews in no time. In the end, thank you people after searxng project and public instances. [1] https://searx.space https://searx.space [2] https://github.com/libredirect/browser_extension https://github.com/libredirect/browser_extension
- baranul 4mo agoThe thing about the public instances, is now you often have to go through a lot of them to verify they work properly. SearXNG needs better quality control. Often have to go through the preferences to deselect search engines that don't work (often because of the instance being blocked) or select those that do work, because of reliability problems. Which engines are working, can be different for each public instance, so that even saving a preference hash doesn't always work. Would be great if SearXNG did automatic adjustment of presented search engines (or offered the option) based on reliability.
- RandyOrion 4mo agoYeah, searxng instance reliability problem goes deeper than simple uptime. The google response time function on public instance page [1], a good measure of the search engine availability of one instance, is broken for some time. Without real testing, people really cannot get the full view of the service status of a single instance. [1] https://searx.space https://searx.space
- deleted 4mo ago[deleted]
- brucejackson 4mo agoHave used it in my homelab for the past 2 years, can recommend it. Easy to run in docker and helps both get better search results and keeps your search history local and in your control.
- Havoc 4mo agoI've got SearXNG set up but lately I've been using uruky instead. For those not familiar think EU-ish Kagi...but crucially their plans comes with very legit API limits too. Their recent show HN: https://news.ycombinator.com/item?id=48396004 https://news.ycombinator.com/item?id=48396004 Not on the same wavelength as searxng stallman wise but I prefer it for AI use. Faster, better quality (subjective) and much less tokens per search - 3.5k vs 23k. Searxng returns a bunch of extra crap that just pollutes AI context. e.g. Link to the thumbnail of the page. I guess you can build a proxy to clean it up but meh. [not affiliated]