7 ms·

I've been trialling Marginalia Search a little and one thing that's struck me is the latency. The only other site I use with similar latency is HN; Marginalia s

I've been trialling Marginalia Search a little and one thing that's struck me is the latency. The only other site I use with similar latency is HN; Marginalia seems even lower despite being dynamic (HN has a much easier caching story). I wonder is it just down to having lower traffic. It's certainly a lot lower than many low-/zero-traffic blogs I've frequented though. I've had a look at the README[0] for the Java sourcecode, but it's highly focused on crawls, database & indexing (understandable for search); would be cool to see a front-end focused write-up. [0] https://github.com/MarginaliaSearch/MarginaliaSearch/blob/master/code/readme.md https://github.com/MarginaliaSearch/MarginaliaSearch/blob/ma...

marginalia_nu 3y ago
The blog is just hugo so it's 100% static files over nginx. The search engine is serverside-rendered mustache templates via handlebars[1], via served via spark[2]. It's basically all vanilla Java. I do raw SQL queries instead of ORM, which makes it quite a bit snappier than most Java applications. The sheer size of the database also mandates that basically every query is a primary key lookup. The code is written around that constraint. Although the search engine is a bit on the slow side since it's routed through cloudflare and I think I'm relatively far away from the closest datacenter so it adds like 100ms to the load times. [1] https://github.com/jknack/handlebars.java https://github.com/jknack/handlebars.java [2] https://sparkjava.com/ https://sparkjava.com/
- lucideer 3y ago
  > The blog is just hugo Yeah the static stuff being fast is less surprising - it was mainly the search results page that astounded me. > via served via spark[2] Had not heard of this Spark (only the other Apache one). Will definitely take a look. > Although the search engine is a bit on the slow side since it's routed through cloudflare and I think I'm relatively far away from the closest datacenter so it adds like 100ms to the load times. I've hit the CF loading screen which introduces a big delay, but when I don't see that the loading is really instantaneous.
  - marginalia_nu 3y ago
    I think overall the system is just really well optimized. It needs to be given I'm working with finite hardware.
    - lucideer 3y ago
      It's incredibly impressive. Well done.
  - playingalong 3y ago
    Watch out for Spark. If not dead, it went into some kind of hiatus. Little activity recently.
    - marginalia_nu 3y ago
      Dunno what I'd want to change though. If worse comes to worst, I'll fork it and keep the dependencies up to date.
      - lucideer 3y ago
        For anything handling user input I'd be concerned about maintenance status for fixes. Even beyond the codebase itself, even just maintaining an up to date pom.xml can be important - seems theirs was last updated in July of last year. Very brief manual browse of it shows potential exposure to things like https://nvd.nist.gov/vuln/detail/CVE-2022-25647 https://nvd.nist.gov/vuln/detail/CVE-2022-25647 - not sure if that's reachable in the codebase but there could be others.
    - brabel 3y ago
      Yes, and it was not that well designed to be honest... the successor is quite a lot nicer and it's called Javalin[1]. Same philosophy but just got things right where Spark, being the "first" (in the Java world, using the design inherited by Sinatra[2]) had a few design issues. [1] https://javalin.io/ https://javalin.io/ [2] https://sinatrarb.com/ https://sinatrarb.com/
    - 3y ago
- ericbarrett 3y ago
  > I do raw SQL queries instead of ORM Love it. I've seen so many cases where engineers with just basic SQL knowledge (like myself; I'm no JOIN god) can run circles around the queries ORMs generate.
  - brazzy 3y ago
    ORMs are great to spare you from writing heaps of error-prone boilerplate mapping code. Which is kinda what they are for, that's why they're called "Object-Relational Mappers". Not "Object-Relational Query Generators". Because they suck at the latter.
    - klysm 3y ago
      Usually they’re not even really relational mappers, but table mappers.
      - throwaway744678 3y ago
        Not sure what you mean: (in a rdms) a table is a relation.
        klysm 3y ago
        Yes exactly. ORMs are usually only good at representing tables, not general relations.
    - deleted 3y ago
      [deleted]
    - stolsvik 3y ago
      ORMs help you the first three weeks. After that it is a lead ball around both feets, dragging you down into dark pits of bad performance, incomprehensible relations, and impossible debugging. That anyone with more than 6 months of experience still drag Hibernate up from their chest is just absolutely beyond my comprehension.
- ignoramous 3y ago
  Curious: Do you have plans to bring in FOSS LLMs for summarization and Q&A style queries anytime in the coming months? Btw, I was half expecting that you quit because of FUTO grants (saw your post on their forums), but I guess it wasn't that. Either ways, rooting for you!
  - marginalia_nu 3y ago
    Not in the short term, if for no other reason than not having the hardware for it. Maybe down the line. Would be neat if someone who was into that sort of stuff wanted to integrate with Marginalia though. I've got a free API ;-) I do think LLMs has the potential to integrate well with search engines and I'd love to see a sort of open source search ecosystem emerge with different projects collaborating to exceed the sum of their parts. Yeah I was in talks with FUTO and they agreed to help the project out a bit, I won't say more until I have money in hand. It's taking a while but it's not on their end, I just need to sort out some legal stuff first.
- srazzaque 3y ago
  > It's basically all vanilla Java Have you considered rewriting it in rust? ;) In all seriousness, it's great to see something written in a "boring" language like Java, which seems to get a lot of hate in developer circles, hover at the top of HN. Java really can perform amazingly well, especially if you minimise the use of unneeded libraries and frameworks. Super curious to see how your stack evolves as you get more load. Best of luck to you on the journey! Ps there's truly a world of difference between "Spring Boot Developer" and "Software Engineer with Java experience". I suspect a lot of people who hate Java or think it performs badly have only worked with the former group of people.
  - marginalia_nu 3y ago
    I'm a believer in the conservation of cool. You can build something cool in a boring language, or build something mundane in a cool language. Java gets a lot of shit, some of it is merited but a lot of it isn't really fair to the language. There's a lot of Java developers that are kinda shit-tier copy-paste developers developing shit-tier copy-paste applications, because the language is so forgiving as to accommodate that, but it's also a competent language that you can do seriously impressive things with. You can be insanely productive in Java because it's extremely stable and mature. You almost never have to deal with library churn or other upstream changes that urgently needs to be fixed. I can think of exactly two instances in my professional life that's happened, migrating off Java 1.8 and the oh-shit moment of needing to patch log4shell.
    - bullen 3y ago
      Did you look at http://github.com/tinspin/rupy http://github.com/tinspin/rupy?
    - the-smug-one 3y ago
      Which version of Java are you running?
      - marginalia_nu 3y ago
        I'm on 18, but will upgrade to 21 when it drops. Not really felt the need to upgrade to 19 or 20 as they haven't provided any features I'm interested in.
adultSwim 3y ago
That's surprising to read. HN has always felt fast to me.
- BenjiWiebe 3y ago
  I believe they are saying that Marginalia is also very fast.
- lucideer 3y ago
  sibling comment is correct: I was saying that HN is very fast and the Marginalia search engine results page is even faster.
thewataccount 3y ago
How does HN's latency compare for you if you're logged in vs logged out?
- megalord 3y ago
  if you are logged in, it has to check if you upvoted some of those listed items (submission and comments). If you're logged out, it doesn't need to check anything - so it's faster
  - breakingcups 3y ago
    In fact, if I remember right, when logged out it can serve cached, pre-rendered pages. Sometimes when HN is down or underperforming, clearing your cookies or opening in incognito will still allow you to view the site because the cache is still present.
- lucideer 3y ago
  I'm always logged in so I haven't got a good impression of the difference but it seems lightning fast most of the time when logged in, so it can't be too much of a difference I guess.
ehnto 3y ago
Algolia has stunning latency and I assume a bucketload of traffic, I suspect they just have very competent infrastructure and fast as hell code and queries, perhaps thr same is true here.
- 59nadir 3y ago
  I appreciate that this is a glass half full or empty kind of situation, but I tend to see "fast" setups actually simply not doing a bunch of the mostly unnecessary stuff that other solutions are doing. This is especially true for code, where of course we're going to get bad results: We've over-abstracted everything with our own abstractions and/or libraries that are as bad as our own stuff and we're doing this in a language and runtime that is pretty much one big premature pessimisation to begin with because we could do significantly more with less. We have no idea what the GC is doing and when, and we care less about that than "using functional programming" or some other equally pointless-in-itself principle that you would be better taking very little from.
  - marginalia_nu 3y ago
    Yeah this is definitely true for Marginalia. It does exactly what it needs to do to serve the page and very little else. There's also no superfluous scripts in the frontend, no session cookie, no user tracking. That stuff does add up. Many applications do so many redundant calculations it's almost absurd. Professionally I've seen applications do enormous ORM lookups that fetch thousands of objects, stick them into a hash table by some key, and pick one value and compare against some parameter; and then do this operation in four different places in processing a single request. Gee I wonder why we have 800ms page loads...
  - ehnto 3y ago
    I totally agree, usually simple is fast. Get the simple stuff right and scale can be simple too. Of course sometimes a complex problem is difficult to solve simply, and sometimes being verbose is a good tradeoff to help maintainability.

marginalia_nu 3y ago

The blog is just hugo so it's 100% static files over nginx. The search engine is serverside-rendered mustache templates via handlebars[1], via served via spark[2]. It's basically all vanilla Java. I do raw SQL queries instead of ORM, which makes it quite a bit snappier than most Java applications. The sheer size of the database also mandates that basically every query is a primary key lookup. The code is written around that constraint. Although the search engine is a bit on the slow side since it's routed through cloudflare and I think I'm relatively far away from the closest datacenter so it adds like 100ms to the load times. [1] https://github.com/jknack/handlebars.java https://github.com/jknack/handlebars.java [2] https://sparkjava.com/ https://sparkjava.com/

lucideer 3y ago
> The blog is just hugo Yeah the static stuff being fast is less surprising - it was mainly the search results page that astounded me. > via served via spark[2] Had not heard of this Spark (only the other Apache one). Will definitely take a look. > Although the search engine is a bit on the slow side since it's routed through cloudflare and I think I'm relatively far away from the closest datacenter so it adds like 100ms to the load times. I've hit the CF loading screen which introduces a big delay, but when I don't see that the loading is really instantaneous.
- marginalia_nu 3y ago
  I think overall the system is just really well optimized. It needs to be given I'm working with finite hardware.
  - lucideer 3y ago
    It's incredibly impressive. Well done.
- playingalong 3y ago
  Watch out for Spark. If not dead, it went into some kind of hiatus. Little activity recently.
  - marginalia_nu 3y ago
    Dunno what I'd want to change though. If worse comes to worst, I'll fork it and keep the dependencies up to date.
    - lucideer 3y ago
      For anything handling user input I'd be concerned about maintenance status for fixes. Even beyond the codebase itself, even just maintaining an up to date pom.xml can be important - seems theirs was last updated in July of last year. Very brief manual browse of it shows potential exposure to things like https://nvd.nist.gov/vuln/detail/CVE-2022-25647 https://nvd.nist.gov/vuln/detail/CVE-2022-25647 - not sure if that's reachable in the codebase but there could be others.
  - brabel 3y ago
    Yes, and it was not that well designed to be honest... the successor is quite a lot nicer and it's called Javalin[1]. Same philosophy but just got things right where Spark, being the "first" (in the Java world, using the design inherited by Sinatra[2]) had a few design issues. [1] https://javalin.io/ https://javalin.io/ [2] https://sinatrarb.com/ https://sinatrarb.com/
  - 3y ago
ericbarrett 3y ago
> I do raw SQL queries instead of ORM Love it. I've seen so many cases where engineers with just basic SQL knowledge (like myself; I'm no JOIN god) can run circles around the queries ORMs generate.
- brazzy 3y ago
  ORMs are great to spare you from writing heaps of error-prone boilerplate mapping code. Which is kinda what they are for, that's why they're called "Object-Relational Mappers". Not "Object-Relational Query Generators". Because they suck at the latter.
  - klysm 3y ago
    Usually they’re not even really relational mappers, but table mappers.
    - throwaway744678 3y ago
      Not sure what you mean: (in a rdms) a table is a relation.
      - klysm 3y ago
        Yes exactly. ORMs are usually only good at representing tables, not general relations.
  - deleted 3y ago
    [deleted]
  - stolsvik 3y ago
    ORMs help you the first three weeks. After that it is a lead ball around both feets, dragging you down into dark pits of bad performance, incomprehensible relations, and impossible debugging. That anyone with more than 6 months of experience still drag Hibernate up from their chest is just absolutely beyond my comprehension.
ignoramous 3y ago
Curious: Do you have plans to bring in FOSS LLMs for summarization and Q&A style queries anytime in the coming months? Btw, I was half expecting that you quit because of FUTO grants (saw your post on their forums), but I guess it wasn't that. Either ways, rooting for you!
- marginalia_nu 3y ago
  Not in the short term, if for no other reason than not having the hardware for it. Maybe down the line. Would be neat if someone who was into that sort of stuff wanted to integrate with Marginalia though. I've got a free API ;-) I do think LLMs has the potential to integrate well with search engines and I'd love to see a sort of open source search ecosystem emerge with different projects collaborating to exceed the sum of their parts. Yeah I was in talks with FUTO and they agreed to help the project out a bit, I won't say more until I have money in hand. It's taking a while but it's not on their end, I just need to sort out some legal stuff first.
srazzaque 3y ago
> It's basically all vanilla Java Have you considered rewriting it in rust? ;) In all seriousness, it's great to see something written in a "boring" language like Java, which seems to get a lot of hate in developer circles, hover at the top of HN. Java really can perform amazingly well, especially if you minimise the use of unneeded libraries and frameworks. Super curious to see how your stack evolves as you get more load. Best of luck to you on the journey! Ps there's truly a world of difference between "Spring Boot Developer" and "Software Engineer with Java experience". I suspect a lot of people who hate Java or think it performs badly have only worked with the former group of people.
- marginalia_nu 3y ago
  I'm a believer in the conservation of cool. You can build something cool in a boring language, or build something mundane in a cool language. Java gets a lot of shit, some of it is merited but a lot of it isn't really fair to the language. There's a lot of Java developers that are kinda shit-tier copy-paste developers developing shit-tier copy-paste applications, because the language is so forgiving as to accommodate that, but it's also a competent language that you can do seriously impressive things with. You can be insanely productive in Java because it's extremely stable and mature. You almost never have to deal with library churn or other upstream changes that urgently needs to be fixed. I can think of exactly two instances in my professional life that's happened, migrating off Java 1.8 and the oh-shit moment of needing to patch log4shell.
  - bullen 3y ago
    Did you look at http://github.com/tinspin/rupy http://github.com/tinspin/rupy?
  - the-smug-one 3y ago
    Which version of Java are you running?
    - marginalia_nu 3y ago
      I'm on 18, but will upgrade to 21 when it drops. Not really felt the need to upgrade to 19 or 20 as they haven't provided any features I'm interested in.

adultSwim 3y ago

That's surprising to read. HN has always felt fast to me.

BenjiWiebe 3y ago
I believe they are saying that Marginalia is also very fast.
lucideer 3y ago
sibling comment is correct: I was saying that HN is very fast and the Marginalia search engine results page is even faster.

thewataccount 3y ago

How does HN's latency compare for you if you're logged in vs logged out?

megalord 3y ago
if you are logged in, it has to check if you upvoted some of those listed items (submission and comments). If you're logged out, it doesn't need to check anything - so it's faster
- breakingcups 3y ago
  In fact, if I remember right, when logged out it can serve cached, pre-rendered pages. Sometimes when HN is down or underperforming, clearing your cookies or opening in incognito will still allow you to view the site because the cache is still present.
lucideer 3y ago
I'm always logged in so I haven't got a good impression of the difference but it seems lightning fast most of the time when logged in, so it can't be too much of a difference I guess.

ehnto 3y ago

Algolia has stunning latency and I assume a bucketload of traffic, I suspect they just have very competent infrastructure and fast as hell code and queries, perhaps thr same is true here.

59nadir 3y ago
I appreciate that this is a glass half full or empty kind of situation, but I tend to see "fast" setups actually simply not doing a bunch of the mostly unnecessary stuff that other solutions are doing. This is especially true for code, where of course we're going to get bad results: We've over-abstracted everything with our own abstractions and/or libraries that are as bad as our own stuff and we're doing this in a language and runtime that is pretty much one big premature pessimisation to begin with because we could do significantly more with less. We have no idea what the GC is doing and when, and we care less about that than "using functional programming" or some other equally pointless-in-itself principle that you would be better taking very little from.
- marginalia_nu 3y ago
  Yeah this is definitely true for Marginalia. It does exactly what it needs to do to serve the page and very little else. There's also no superfluous scripts in the frontend, no session cookie, no user tracking. That stuff does add up. Many applications do so many redundant calculations it's almost absurd. Professionally I've seen applications do enormous ORM lookups that fetch thousands of objects, stick them into a hash table by some key, and pick one value and compare against some parameter; and then do this operation in four different places in processing a single request. Gee I wonder why we have 800ms page loads...
- ehnto 3y ago
  I totally agree, usually simple is fast. Get the simple stuff right and scale can be simple too. Of course sometimes a complex problem is difficult to solve simply, and sometimes being verbose is a good tradeoff to help maintainability.