5 ms·

Marginalia Search: 1 Year

140 points by georgehill 5y ago

marginalia_nu 5y ago
Not really sure why this feverish ramble deserves the front page, but sure, I'll take it.
- elefanten 5y ago
  First reaction: Wow! That’s an insanely harsh reaction to someone celebrating a 1yr project anni. Thankfully, I checked poster name. Grats!
- georgeoliver 5y ago
  I appreciate those who put the hacker back in Hacker News.
- RistrettoMike 5y ago
  Thanks so much for your project! It’s one of the things I turn to when looking to uncover actual human writing these days.
- exolymph 5y ago
  I'll admit, I automatically upvote anything Marginalia-related. We just like you, man! Keep fighting the good fight <3
frogulis 5y ago
I really like this search engine. Particularly when I want to jump face-first into researching a new topic that I know almost nothing about, the signal-to-noise ratio is often so much better than Google.
- marginalia_nu 5y ago
  Honestly it's not working quite as I want it to right at this moment, it's gotten a bit slow due to a thing I'm testing, and it feels like it's not always finding the best results. Still works decently but still a bit wonkier than usual right now.
  - cubano 5y ago
    Only you, as it's sole (or soul even) developer, could possibly be aware of its current state and result sets and how they may or may not be differing from your ideal vision for it. For the rest of us, it's a really cool project and I, for one, am glad your efforting it...so thanks!
nonrandomstring 5y ago
It's one of my smorgasbord search engines run from a script (which collectively do so much better than Google or any of the legacy "big-tech" engines) Why it matters One of the great American philosophical orators, Rick Roderick (a Texan cowboy philosopher and academic twin of Bill Hicks), spoke about the importance of "the margins": Rick notices that in history certain people do not get a voice; Women. The Poor. Old and young people. The "mad" and disabled. Race minorities within any group. Thinkers more than a decade ahead of their time. There's even a thing called "marginal analysis" in hermeneutics - not the economic concept - the analysis of things that have been left out of history, and why, and by whom (victors writing and re-writing history etc). Erasure leaves a trace that tells its own tale. These things have become the grist of social justice today (with all it's positive and negative sides). But "injustice" is not really Roderick's point. It hardly needs mentioning that most people get a bad deal, don't get heard, and their lives fade into obscure irrelevance. What matters is the enormous value and importance of marginal contributions. These are lost. Almost every great discovery, it turns out on deeper analysis, was made by an (or often, simultaneously several) obscure nobody years earlier, who didn't provide a recognised proof in the right journal, or didn't patent the thing. In Plato and Shakespeare, Roderick says, "It is always the fool who delivers the important news". Things don't start at the center, they begin on the margins and diffuse in until they find legitimacy in an acceptable voice. A marginal search engine engine is, almost by definition, acting counter to the status quo, as the Internet Archive stands in opposition to fickle entropy of bitrot and ephemeral culture. Well done on this project and good luck.
- UncleMeat 5y ago
  While this is all true and all good, “marginalia” is a completely different word for a completely different thing. It refers to handwritten content in the margins of books.
  - marginalia_nu 5y ago
    The word is a Latin adjective and just means marginal. It's most frequently used for marginal notes, but I don't see why you couldn't use it in other contexts as well.
dang 5y ago
Related: Show HN: Marginalia – Exploration Mode - https://news.ycombinator.com/item?id=30047455 https://news.ycombinator.com/item?id=30047455 - Jan 2022 (53 comments) A search engine that favors text-heavy sites and punishes modern web design - https://news.ycombinator.com/item?id=28550764 https://news.ycombinator.com/item?id=28550764 - Sept 2021 (717 comments)
hakko 5y ago
Marginalia is a breath of fresh air in an internet that seems to be...disappearing. I actually like it's philosophy of not doing "useful" search queries - it's great just to see what's lost behind Google's wall of Reddit/Wikipedia/SEO Blogspam results. Funnily, after playing with it for a bit my gripe is that there isn't a way to exclude results from vanity domain names which are consistently the lowest quality results.
- marginalia_nu 5y ago
  Try the blog search profile, and if that's still no good, add set it to deny javascript. That reduces quite a lot of noise.
- h0h0h0h0111 5y ago
  >I actually like it's philosophy of not doing "useful" search queries Generalising this a little, it sometimes feels like the world has become all about optimisation and productivity, and the beauty of uselessness has gone missing. Many of my favourite posts/projects on HN are from people who have set out to build something utterly useless in the name of fun - and sometimes they turn out to be useful after all!
jhoechtl 5y ago
Apparently, if you omit all the we-got-you-and-lock-you-in-your-ad-induced-information-bubble, you can get quite far with even modest hardware: https://memex.marginalia.nu/projects/gemini-server.gmi https://memex.marginalia.nu/projects/gemini-server.gmi > I put the machine together mostly for a search engine, Because I didn't want an actual rack making noise and heat in my living room, the server is made out of consumer hardware: > Ryzen 9 3900X > 128 Gb RAM > 4x4 Gb IronWolf ZFS > A bunch of SSDs index lookups
- marginalia_nu 5y ago
  Some of that is a bit stale, I'm not using ZFS anymore because it eats too much memory, and the SQL database containing metadata is on an Optane 900P, but it's still largely the same machine. But yeah, turns out, if you mind what you are doing when you write your software, modern consumer hardware can be pretty good. I also think a lot of the architecture paradigms that are common today have a very high constant cost, which may be the admittance fee for high scalability, but I don't have anything to scale up to anyway so my problem is how to do the most with fixed hardware. If I had written this like a cloud application, there is just no way it would run on the same machine. I'd need ten times more hardware. I'd also not be anywhere near finished. Data locality is slept on in terms of performance.

140 points by georgehill 5y ago

marginalia_nu 5y ago

Not really sure why this feverish ramble deserves the front page, but sure, I'll take it.

elefanten 5y ago
First reaction: Wow! That’s an insanely harsh reaction to someone celebrating a 1yr project anni. Thankfully, I checked poster name. Grats!
georgeoliver 5y ago
I appreciate those who put the hacker back in Hacker News.
RistrettoMike 5y ago
Thanks so much for your project! It’s one of the things I turn to when looking to uncover actual human writing these days.
exolymph 5y ago
I'll admit, I automatically upvote anything Marginalia-related. We just like you, man! Keep fighting the good fight <3

frogulis 5y ago

I really like this search engine. Particularly when I want to jump face-first into researching a new topic that I know almost nothing about, the signal-to-noise ratio is often so much better than Google.

marginalia_nu 5y ago
Honestly it's not working quite as I want it to right at this moment, it's gotten a bit slow due to a thing I'm testing, and it feels like it's not always finding the best results. Still works decently but still a bit wonkier than usual right now.
- cubano 5y ago
  Only you, as it's sole (or soul even) developer, could possibly be aware of its current state and result sets and how they may or may not be differing from your ideal vision for it. For the rest of us, it's a really cool project and I, for one, am glad your efforting it...so thanks!

nonrandomstring 5y ago

It's one of my smorgasbord search engines run from a script (which collectively do so much better than Google or any of the legacy "big-tech" engines) Why it matters One of the great American philosophical orators, Rick Roderick (a Texan cowboy philosopher and academic twin of Bill Hicks), spoke about the importance of "the margins": Rick notices that in history certain people do not get a voice; Women. The Poor. Old and young people. The "mad" and disabled. Race minorities within any group. Thinkers more than a decade ahead of their time. There's even a thing called "marginal analysis" in hermeneutics - not the economic concept - the analysis of things that have been left out of history, and why, and by whom (victors writing and re-writing history etc). Erasure leaves a trace that tells its own tale. These things have become the grist of social justice today (with all it's positive and negative sides). But "injustice" is not really Roderick's point. It hardly needs mentioning that most people get a bad deal, don't get heard, and their lives fade into obscure irrelevance. What matters is the enormous value and importance of marginal contributions. These are lost. Almost every great discovery, it turns out on deeper analysis, was made by an (or often, simultaneously several) obscure nobody years earlier, who didn't provide a recognised proof in the right journal, or didn't patent the thing. In Plato and Shakespeare, Roderick says, "It is always the fool who delivers the important news". Things don't start at the center, they begin on the margins and diffuse in until they find legitimacy in an acceptable voice. A marginal search engine engine is, almost by definition, acting counter to the status quo, as the Internet Archive stands in opposition to fickle entropy of bitrot and ephemeral culture. Well done on this project and good luck.

UncleMeat 5y ago
While this is all true and all good, “marginalia” is a completely different word for a completely different thing. It refers to handwritten content in the margins of books.
- marginalia_nu 5y ago
  The word is a Latin adjective and just means marginal. It's most frequently used for marginal notes, but I don't see why you couldn't use it in other contexts as well.

dang 5y ago

Related: Show HN: Marginalia – Exploration Mode - https://news.ycombinator.com/item?id=30047455 https://news.ycombinator.com/item?id=30047455 - Jan 2022 (53 comments) A search engine that favors text-heavy sites and punishes modern web design - https://news.ycombinator.com/item?id=28550764 https://news.ycombinator.com/item?id=28550764 - Sept 2021 (717 comments)

hakko 5y ago

Marginalia is a breath of fresh air in an internet that seems to be...disappearing. I actually like it's philosophy of not doing "useful" search queries - it's great just to see what's lost behind Google's wall of Reddit/Wikipedia/SEO Blogspam results. Funnily, after playing with it for a bit my gripe is that there isn't a way to exclude results from vanity domain names which are consistently the lowest quality results.

marginalia_nu 5y ago
Try the blog search profile, and if that's still no good, add set it to deny javascript. That reduces quite a lot of noise.
h0h0h0h0111 5y ago
>I actually like it's philosophy of not doing "useful" search queries Generalising this a little, it sometimes feels like the world has become all about optimisation and productivity, and the beauty of uselessness has gone missing. Many of my favourite posts/projects on HN are from people who have set out to build something utterly useless in the name of fun - and sometimes they turn out to be useful after all!

jhoechtl 5y ago

Apparently, if you omit all the we-got-you-and-lock-you-in-your-ad-induced-information-bubble, you can get quite far with even modest hardware: https://memex.marginalia.nu/projects/gemini-server.gmi https://memex.marginalia.nu/projects/gemini-server.gmi > I put the machine together mostly for a search engine, Because I didn't want an actual rack making noise and heat in my living room, the server is made out of consumer hardware: > Ryzen 9 3900X > 128 Gb RAM > 4x4 Gb IronWolf ZFS > A bunch of SSDs index lookups

marginalia_nu 5y ago
Some of that is a bit stale, I'm not using ZFS anymore because it eats too much memory, and the SQL database containing metadata is on an Optane 900P, but it's still largely the same machine. But yeah, turns out, if you mind what you are doing when you write your software, modern consumer hardware can be pretty good. I also think a lot of the architecture paradigms that are common today have a very high constant cost, which may be the admittance fee for high scalability, but I don't have anything to scale up to anyway so my problem is how to do the most with fixed hardware. If I had written this like a cloud application, there is just no way it would run on the same machine. I'd need ten times more hardware. I'd also not be anywhere near finished. Data locality is slept on in terms of performance.