4 ms·

search.marginalia.nu consisted of 1 Swedish engineer and 1 server last I heard and still managed to outclass DDG in a number of query types relevant to me. In

search.marginalia.nu consisted of 1 Swedish engineer and 1 server last I heard and still managed to outclass DDG in a number of query types relevant to me. In fact, back when I used DDG I think I fell back to marginalia more often than I fell back to Google, partly because of my dislike for Google and partly because Google doesn't respect my queries - which of course is a contributing reason for my dislike for them. Let's say 100 of the Brave employees are engineers and 50 of them work on the Chromium skin, that still leaves 50 to work on search and related efforts. If 5 of them are as good as the marginalia guy and they are allowed to work with equally clear direction, lack of interruptions and more funding, I think that could almost explain a working search engine. Remember: In the first 20 years of Google existence (or in any 20 years of the semiconductor age until recently) Moores law had over 13 cycles. A lot of what used to be hard problems before isn't anynore.

riku_iki 4y ago
that niche engine looks like many magnitudes simpler problem than building competitive modern search.
- eitland 4y ago
  Marginalia is a niche engine for now, yes, but what it does, it does correctly, unlike Google that has been silently rewriting my queries since before 2013: https://techinorg.blogspot.com/2013/03/what-is-going-on-with-search-results.html?m=1 https://techinorg.blogspot.com/2013/03/what-is-going-on-with...
  - riku_iki 4y ago
    someone in google according to some metrics decided it is something majority of users wants.
jraph 4y ago
> Remember: In the first 20 years of Google existence (or in any 20 years of the semiconductor age until recently) Moores law had over 13 cycles. A lot of what used to be hard problems before isn't anynore. The web also grow tremendously, and probably user expectations too (in the beginning of the century we were told in school not to speak to search engine like we would to a human, but with keywords and operators). We also do and search for many more kinds of things
- eitland 4y ago
  If anything my expectations sunk massively between 2009 and the introduction of Kagi. To be blunt: for me, mainstream search engines Google and Bing very much feel like the things they replaced, Altavista and Yahoo, just with some fancy bolt ons like maps etc. I understand some people like to be able to write sentences to their search engines, but as long as the results have worse quality than they had 15 - 20 years ago, that "understanding" is just another fancy bolt on feature. The only things that exist today that could threaten Google quality is Kagi which has gone all in on quality and ChatGPT (and other similar solutions) which finally have produced a working "answer machine" instead of breaking perfect or at least working search engines.
  - senko 4y ago
    This will probably be of interest: https://labs.kagi.com/fastgpt https://labs.kagi.com/fastgpt
    - eitland 4y ago
      You were correct! It is really impressive and up to date. It feels a little less precise than ChatGPT but given the fact that it is up to date means it is a tradeoff I guess I will often want to take.
marginalia_nu 4y ago
> search.marginalia.nu consisted of 1 Swedish engineer and 1 server last I heard and still managed to outclass DDG in a number of query types relevant to me. Still just 1 dinky lil' consumer hardware server in my living room. I think what is limiting the project the most is the hardware. Like I can definitely squeeze more out of it, but I could probably do 100 times more if I had any sort of operational budget. But at least the development is funded for the moment. We'll see where I am when the NLnet money runs out...
- kragen 4y ago
  Does this mean you have more engineers working on it now? That was my reading of the grandparent comment's implications. BTW, thank you very much for offering this service, even though I don't use it often.
  - marginalia_nu 4y ago
    No I'm still solo developing. Would having a few more people working on it make stuff move faster? Undeniably. But I don't think lack of manpower is an obstacle at the moment in the same lack of hardware. It's reasonable to expect there's an upper limit to how big a project you can manage as a single developer, but I don't think I'm there yet.
    - reitanqild 4y ago
      I really wish there was someone reading this who could donate some hardware. I would love to but I am not in a position to do it, at least not right now.
      - marginalia_nu 4y ago
        I view it as a long march. Right now I've got about two years runway, which isn't a lot but it also isn't nothing. Considering the project had barely started two years ago, and considering the opportunities it's already created, hopefully more options will make themselves available along the way somehow.
z3t4 4y ago
All it takes is one good engineer and he can build anything. The difficult part is to get people to use it.
- eitland 4y ago
  Brave already has access to a large pool of (from what I can see) enthusiastic users because of their browser : )

riku_iki 4y ago

that niche engine looks like many magnitudes simpler problem than building competitive modern search.

eitland 4y ago
Marginalia is a niche engine for now, yes, but what it does, it does correctly, unlike Google that has been silently rewriting my queries since before 2013: https://techinorg.blogspot.com/2013/03/what-is-going-on-with-search-results.html?m=1 https://techinorg.blogspot.com/2013/03/what-is-going-on-with...
- riku_iki 4y ago
  someone in google according to some metrics decided it is something majority of users wants.

jraph 4y ago

> Remember: In the first 20 years of Google existence (or in any 20 years of the semiconductor age until recently) Moores law had over 13 cycles. A lot of what used to be hard problems before isn't anynore. The web also grow tremendously, and probably user expectations too (in the beginning of the century we were told in school not to speak to search engine like we would to a human, but with keywords and operators). We also do and search for many more kinds of things

eitland 4y ago
If anything my expectations sunk massively between 2009 and the introduction of Kagi. To be blunt: for me, mainstream search engines Google and Bing very much feel like the things they replaced, Altavista and Yahoo, just with some fancy bolt ons like maps etc. I understand some people like to be able to write sentences to their search engines, but as long as the results have worse quality than they had 15 - 20 years ago, that "understanding" is just another fancy bolt on feature. The only things that exist today that could threaten Google quality is Kagi which has gone all in on quality and ChatGPT (and other similar solutions) which finally have produced a working "answer machine" instead of breaking perfect or at least working search engines.
- senko 4y ago
  This will probably be of interest: https://labs.kagi.com/fastgpt https://labs.kagi.com/fastgpt
  - eitland 4y ago
    You were correct! It is really impressive and up to date. It feels a little less precise than ChatGPT but given the fact that it is up to date means it is a tradeoff I guess I will often want to take.

marginalia_nu 4y ago

> search.marginalia.nu consisted of 1 Swedish engineer and 1 server last I heard and still managed to outclass DDG in a number of query types relevant to me. Still just 1 dinky lil' consumer hardware server in my living room. I think what is limiting the project the most is the hardware. Like I can definitely squeeze more out of it, but I could probably do 100 times more if I had any sort of operational budget. But at least the development is funded for the moment. We'll see where I am when the NLnet money runs out...

kragen 4y ago
Does this mean you have more engineers working on it now? That was my reading of the grandparent comment's implications. BTW, thank you very much for offering this service, even though I don't use it often.
- marginalia_nu 4y ago
  No I'm still solo developing. Would having a few more people working on it make stuff move faster? Undeniably. But I don't think lack of manpower is an obstacle at the moment in the same lack of hardware. It's reasonable to expect there's an upper limit to how big a project you can manage as a single developer, but I don't think I'm there yet.
  - reitanqild 4y ago
    I really wish there was someone reading this who could donate some hardware. I would love to but I am not in a position to do it, at least not right now.
    - marginalia_nu 4y ago
      I view it as a long march. Right now I've got about two years runway, which isn't a lot but it also isn't nothing. Considering the project had barely started two years ago, and considering the opportunities it's already created, hopefully more options will make themselves available along the way somehow.

z3t4 4y ago

All it takes is one good engineer and he can build anything. The difficult part is to get people to use it.

eitland 4y ago
Brave already has access to a large pool of (from what I can see) enthusiastic users because of their browser : )