6 ms·
Marginalia – A search engine that prioritizes non-commercial content
- hatefulmoron 2y agoI don't have anything unique to say, but I love when this project comes up on HN. The project exemplifies what a patient and loving hand can do.
- nostradumbasp 2y agosame. I was thinking about doing something similar at some point but this exceeds my expectations. Seriously a great job. Let us know if theres anyway to contribute.
- marginalia_nu 2y agoWell it's open source and contributions are welcome, though it's a fairly sprawling java project that's probably not the easiest to get into still (through no lack of effort making it more accessible): https://github.com/MarginaliaSearch/MarginaliaSearch https://github.com/MarginaliaSearch/MarginaliaSearch If you have spare dollars but not time, you can also contribute to the war chest: https://about.marginalia-search.com/article/supporting/ https://about.marginalia-search.com/article/supporting/
- wetpaws 2y ago[dead]
- yurisalazar 2y agoWow, this project has been going on for 3 years, amazing stuff, I love reading devlogs
- disqard 2y agoIf somebody hasn't been following this, it's one person's passion project, and it doesn't aim to compete with Google/Bing/DDG. It's more of a way to find interesting things on the "small web". Please keep that in mind when you check this out. (I'm not the creator)
- lolinder 2y agoI don't know if something has changed since then, but in 2023 the author did quit their job to start working on the search engine full time [0]. So yes, it's a passion project, but it's also not just a side project. [0] https://www.marginalia.nu/log/83_full_time/ https://www.marginalia.nu/log/83_full_time/
- barbs 2y agoThat's really cool, although I'm curious about their plans to monetize the site, if any.
- ChemTails 2y agoAccording to the creator, $200/month will maintain operations. He doesn’t mention plans to monetize.
- marginalia_nu 2y agoThe plan as much as there is one is to have very little costs, provide outsized value and somehow scrape by on grants, public donations. I'm currently set for about 1-2 more years of full time work (had 6 months' funds + a 1 year grant when I left my job 18 months ago).
- mannycalavera42 2y agoI simply LOVE and LOVE and LOVE marginalia. Keep rocking mate
- eitland 2y ago> If somebody hasn't been following this, it's one person's passion project, and it doesn't aim to compete with Google/Bing/DDG. That is correct. But I used[1] to find that in certain niches (Linux and open source software, history) I'd often get significantly better results with Marginalia than with Google or DDG. This seems to be related both to - the input stupidifier that seems to be in use with mainstream search engines (if I search for anything unusual it assumes I misspelled or mis-remembered and replace my search with something I didn't search for before sending it to the backend) - and mainstream engines going out of their way to prefer corporate media over original, authorative content > It's more of a way to find interesting things on the "small web". Yes, he keeps saying that, but it was[2] still not just interesting but useful for me. [1][2]: past tense because I switched to Kagi almost 3 years ago and now I don't have to maintain all the hacks (like separate search engines for separate niche topics) that I used to have. Full disclaimer: I know certain other people frequently want to contest this, saying they get great results with Google or bad results with Kagi, to which all I have to say is I have documented history of Google consistently failing even simple queries going more than a decade back and if Google works for you, more power to you, but it hasn't work reliably for me for over a decade and I am fed up.
- metadat 2y agoMy personal website has 0-ads or anything malicious, 100% original content, yet it still doesn't show up in the first 5 pages when searching marginalia for `<first-name> <last-name>'. There are some marginally noteworthy folks who share my name, but it's basically some artist nobody cares about, a Floridian pest control expert who I regularly get mis-addressed email replies from, and a low-ranked pro football player with an unexceptional web presence nobody appears to care about. I thought I could at least eek out a out #4 or #3, but no? I imagined my ad-free, www-free, firstname-lastname domain would be welcome on marginalia, but it seems deranked just like what Google has inexplicably done since I bought the domain. Despite following best practices and webmaster console recommendations. A squatter owned the domain for some time before 2010, but there's no evidence of interesting nefarious activity. My first name-lastname GitHub account, linked from my site homepage, has a noteworthy number of open-source contributions to all sorts of projects. P.s. marginalia_nu: Your Java and architectural design is beautiful, you're a true craftsman. <3 Best in show.
- roywiggins 2y agoAre you sure you're in the index? https://github.com/MarginaliaSearch/submit-site-to-marginalia-search https://github.com/MarginaliaSearch/submit-site-to-marginali...
- metadat 2y agoDo all sites manually submit themselves via PR? I'm happy to do it, never encountered this information previously.
- stebalien 2y agoIf and only if your site isn't already referenced by some other site in the index (see option A).
- marginalia_nu 2y ago(Creator of the search engine) It's not required, they will be discovered eventually, though it's a way of speeding up the process, as the index is anything but exhaustive still.
- rossdavidh 2y agoDecided to repeat my last few google searches ("celery", "Wardian case", and "ferry from Italy to Greece", if you're curious), and it came up with reasonable answers as the top one in every case! I will give it a try.
- ajdude 2y agoI really like the search engine, and I am absolutely adding it to my rotation when searching for interesting things
- ycombinete 2y agoI use it whenever I'm searching for cooking recipes. It's how I found the Cooking for Engineers forums. https://www.cookingforengineers.com/forums https://www.cookingforengineers.com/forums
- rafaelgoncalves 2y agowhoa, didn't know this project, looks really incredible, will add to my others search engines and give a try, thanks for the devs/creators and hope for success to you all!
- Levitating 2y agoI recently wrote an essay about this search engine, and its ranking algorithms. Initially Marginalia used an interesting variant of PageRank discussed in the original paper, called Personal Pagerank.[1] Currently pages are ranked with BM25. I think Personalized PageRank is still used for a new feature of Marginalia which is ranking pages based on similarity. I think this is already integrated into the website but there used to only be this testing page: https://explore2.marginalia.nu/ https://explore2.marginalia.nu/ In any case I have a lot of respect for the creator. Marginalia has seen a lot of growth and it's been interesting reading the blogposts.[2] [1]: https://www.marginalia.nu/log/26-personalized-pagerank/ https://www.marginalia.nu/log/26-personalized-pagerank/ [2]: https://www.marginalia.nu/log/ https://www.marginalia.nu/log/
- _emacsomancer_ 2y agowhere is the recent essay?
- Levitating 2y agoIt was for a university assignment, I am not sure if I could or should share it.
- hooli_gan 2y agoIf you don't share the task description or other materials provided by your university, I don't see the problem with sharing your essay. (Not a lawyer)
- hmlwilliams 2y agoIt's a sure fire way to get your uni submission incorrectly flagged as plagiarism. How straightforward it is to prove you're the original author varies by institution.
- em-bee 2y ago
- BOOSTERHIDROGEN 2y agoI am aware that SearxNG has chosen to remove the engine; hopefully, a mutually beneficial agreement can be reached.
- marginalia_nu 2y agoSearxNG integration didn't work great, in part because they had some issues where they were hitting the search engine like a shotgun with 10+ identical requests, but also because the API has a public key that's nearly unusable due to bot spam, meaning anyone who wants to route traffic to marginalia search needs to email me and get a key. Not a super convenient system. I'm not opposed to seeing a return of the integration though it probably could do with some more design work.
- modzu 2y agovery cool, but wish i could remember the name
- dang 2y agoMuch discussed under its previous domain (marginalia.nu): Phrase matching in Marginalia Search - https://news.ycombinator.com/item?id=41696046 https://news.ycombinator.com/item?id=41696046 - Sept 2024 (24 comments) Marginalia: 3 Years - https://news.ycombinator.com/item?id=39501061 https://news.ycombinator.com/item?id=39501061 - Feb 2024 (44 comments) Interview with Viktor Lofgren from Marginalia Search - https://news.ycombinator.com/item?id=38470832 https://news.ycombinator.com/item?id=38470832 - Nov 2023 (21 comments) Moving Marginalia to a new server - https://news.ycombinator.com/item?id=37800753 https://news.ycombinator.com/item?id=37800753 - Oct 2023 (39 comments) Marginalia.nu API - https://news.ycombinator.com/item?id=35871186 https://news.ycombinator.com/item?id=35871186 - May 2023 (22 comments) Marginalia: DIY search engine that focuses on non-commercial content - https://news.ycombinator.com/item?id=35611923 https://news.ycombinator.com/item?id=35611923 - April 2023 (193 comments) Marginalia Search has received an NLNet grant - https://news.ycombinator.com/item?id=34945541 https://news.ycombinator.com/item?id=34945541 - Feb 2023 (17 comments) A Theoretical Justification (2021) - https://news.ycombinator.com/item?id=32586273 https://news.ycombinator.com/item?id=32586273 - Aug 2022 (22 comments) The Evolution of Marginalia's Crawling - https://news.ycombinator.com/item?id=32565052 https://news.ycombinator.com/item?id=32565052 - Aug 2022 (22 comments) Marginalia Goes Open Source - https://news.ycombinator.com/item?id=31536626 https://news.ycombinator.com/item?id=31536626 - May 2022 (72 comments) Uncertain Future for Marginalia Search - https://news.ycombinator.com/item?id=31200319 https://news.ycombinator.com/item?id=31200319 - April 2022 (37 comments) Marginalia Search: 1 Year - https://news.ycombinator.com/item?id=30823481 https://news.ycombinator.com/item?id=30823481 - March 2022 (29 comments) Show HN: Marginalia – Exploration Mode - https://news.ycombinator.com/item?id=30047455 https://news.ycombinator.com/item?id=30047455 - Jan 2022 (53 comments) A search engine that favors text-heavy sites and punishes modern web design - https://news.ycombinator.com/item?id=28550764 https://news.ycombinator.com/item?id=28550764 - Sept 2021 (717 comments) https://news.ycombinator.com/user?id=marginalia_nu https://news.ycombinator.com/user?id=marginalia_nu
- Funes- 2y agoHell yeah. The very first one was submitted by yours truly. I love this project and hope to work on similar ones in the near future.
- dredmorbius 2y agoI'm surprised to see that, as of now at least, there's apparently no Marginalia bang search at DDG: <https://duckduckgo.com/bangs?q=marginalia https://duckduckgo.com/bangs?q=marginalia> I've submitted it as a suggestion.
- cratermoon 2y agoI'm less than impressed with the quality of results. Sure, the sites linked are generally not commercial, although various stackexchange.com posts show up, as to a lot of wordpress.com and blogspot.com blogs. Some sketchy results seemed highly ranked, too, include conspiracy theorists and fringe ideas.
- benreesman 2y agoI don’t have much to add other than to say that this is the most pleasant and useful and wholesome thing I’ve seen on the web in a long time. The technology looks extremely solid and the experience is flawless. To the author, thank you so much for this trip to the glory days of the Internet.
- vekker 2y agoAwesome! The name is just a bit funny if your native tongue is Dutch.
- jorams 2y agoIs it? The word means the same in English as it does in Dutch.
- miki123211 2y agoI tried a few relatively simple terms in the areas I have an interest in, e.g. ("attention is all you need paper", "sip rfc", "klatt speech synthesizer", "crafting interpreters book", "Scott Alexander Substack"), and none of them actually produced the results I expected. Marginalia seems to work okay-ish if you want to learn about something, but not if you want to find something. If you want to read what people think about Scott Alexander's Substack, you will get some decent results, but not if you want to find the newsletter itself.
- marginalia_nu 2y agoYeah navigational queries are a bit hit and miss, but I feel google does them very well as is so it's not a huge priority area. I've mostly been focusing my efforts on the stuff Google is bad at, finding human content about some topic.
- marginalia_nu 2y ago(Creator here) I recently moved the website from search.marginalia.nu to marginalia-search.com and gave it a bit of a visual touch up, on the basis that I felt it was working too well to be a weird subdomain outgrowth off my blog. It's still the same search engine :-)
- alberth 2y agoAny reason why you don’t use a shorter domain name? (Amazing what you built, btw)
- marginalia_nu 2y agoHard to find one that makes sense and isn't inscrutable like mxyzp.tlk. If I ever do find one that's appropriate I might set up a CNAME record to point to marginalia-search.com
- alberth 2y agoFYI - https://www.dynadot.com/market/search https://www.dynadot.com/market/search has a nice way you can filter their Aftermarket domains for sales by Character Count, TLD, No IDN/Number/Hyphens. I've found over the years plenty of <= 5 character domains that have a meaningful name just from using their search filters. (I'm not affiliated in anyway to them)
- axiomdata316 2y agoIt looks like the "random" button on the old version no longer works. Did you remove that feature?
- marginalia_nu 2y agoHmm, I think it's just a broken redirect. It works if you search for browse:random . I'll try to fix it later today.
- stackghost 2y ago
- Llamamoe 2y agoI haven't had much luck using it as a search engine, but I ADORE using the "explore random websites" feature[1], and just browsing random personal blogs and sites, or kitsch neocity pages. All of it feels so human and is a joy to view, in contrast to the raw garbage Google spews at me whenever I bother to use it. [1] https://search.marginalia.nu/explore/random https://search.marginalia.nu/explore/random
- biohcacker84 2y agoI've noticed an uptick of new search engines. Likely because of the enshitification of google. But what is most on my mind, is how the enshitification of google might have affected the global economy..... Allow me to try and explain. Market efficiency is deeply tied to market information. And early google was fantastic at providing accurate and valuable information. It's hard to put into words, and I am not aware of any studies, but it felt like it sped up and/or made the whole economy more efficient. And today it feels like that has been lost. And I wonder if there could ever be formal accounting to detect how much peak google contributed to.... not so much GDP, but how the economy functions. And compare to today.
- marginalia_nu 2y agoI honestly think this is as much the enshittification of google as it is the enshittification of the web. Early google was great in large part because the early web had a much better signal to noise ratio. Going on for well over a decade we've been putting far more slop on the web than useful content. It really adds up over time. I also feel this is largely why the GPTs have been so successful. They're if nothing else a lot less annoying than the web search experience has become.
- plagiarist 2y agoI agree, though I think Google and all the adtech companies share a lot of the blame for the structure of the modern web. The result is current web search returns static output from smaller models in the form of affiliate marketing listicles. Using a larger LLM directly on your query is simply better. There is a semantic/vector search happening with them such that there might be relevant terms or concepts even in a false answer. That's a step forward towards real information at least. That doesn't happen with the listicles.
- eitland 2y ago> I honestly think this is as much the enshittification of google as it is the enshittification of the web. Early google was great in large part because the early web had a much better signal to noise ratio. Hi again! Lots of respect for your work. This thing I considered for a while but I am now certain that it cannot be the whole explanation. Why? Because for me Kagi has consistently outperformed Google to the point where it feels like having Google 2009 back, just with some extra improvements: It finds the things I ask for and prioritise in a good way so I rarely have to look for page 2 or anything.
- webscout 2y ago[dead]
- black_puppydog 2y agoThis is for English content only, right? I'm trying with some specifically French queries and it only gives me (relevant, but) English results.
- marginalia_nu 2y agoYeah, for now I'm focusing on doing English search well before I tackle additional languages. It's some effort to get it working well for any given language, and I don't think a search engine that doesn't work well is going to make anyone happy.
- AndrewStephens 2y agoThis is great - as a site owner I especially appreciated that you publish discovered backlinks. I discovered a few that I didn't know about thanks to your service.
- Timwi 2y agoI looked at the backlinks for a website of mine and none of them actually linked to my site, so I was very confused...
- marginalia_nu 2y agoHmm, can you share the domain name? Either here or email me? I can look into it. (Though it's most likely suffix-list related, there are some outstanding issues in that regard.)
- Timwi 2y agoI owe you an apology; I clicked on the ↗ icons, not realizing that those take me to the domain’s homepage instead of the specific URL containing the link. I have now checked them all and they do in fact link to the domain. (Which, for the record, is ktane.timwi.de, a repository of documents relating to a video game.)
- fsflover 2y agoLooks similar to https://wiby.me https://wiby.me.