8 ms·

.NET Transactional Document DB and Event Store on PostgreSQL

78 points by c17r 6y ago

GordonS 6y ago
I've been using Marten for maybe 3 years now, and I absolutely love it! It's extensible too, so you can do just about anything with it - for example, I use it with views, multi tenancy, a base for aggregate queries and all sorts, and it works great. The core team is really responsive on both GitHub and Gitter too, and are happy to both accept and help with PRs - it's (almost) the model of a well run OSS project. One small thing tho... the original developer, Jeremy Miller, can be a bit... spiky; like you ask a question and he just assumes bad faith and will snark at you. I imagine this behaviour has put at least a few people off contributing and using Marten. I almost feel bad about mentioning this, because I'm a fan of his work, but OTOH I kind of hope Jeremy reads this comment and takes it constructively.
- c17r 6y ago
  I came across Marten on someone's blog where they were talking about their ups and downs with RavenDB. It looks interesting. Since you're using it, I'd like to know more about it's performance profile. Can't really find any performance/speed comparisons online. And what kind of scale are you running. I'm right at the beginning of a project for a client and I'm roughing out the foundational code with pgsql already, so it would be a perfect time to take a look at it. Thanks!
  - GordonS 6y ago
    We're not running at scale, at least not the kind where performance of the library is likely to be an issue. We have around 50 tenants with a handful of users each. Tenants have up to 100's of millions of rows in Postgres - not exactly big data levels, but pretty big still. I don't have perf numbers anymore, but I did some performance testing before I started using it a few years ago, and even back then it was much faster than EF/EFCore. The team have made many performance improvements since then. Considering the ease of use, and the fact you get NoSQL-style JSON documents that work alongside regular relational tables, basically the performance impact is as good as neglibible, IMO. If you want to work with JSON documents, I highly recommend Marten. It does have a Benchmarks project if you'd like to run some tests[0], and I'm sure the team would be happy to answer any questions over in Gitter[1]. [0] https://github.com/JasperFx/marten/blob/07d99ed80a1f0c436c4f1817421dd8831efe73c0/README.md https://github.com/JasperFx/marten/blob/07d99ed80a1f0c436c4f... [1] https://gitter.im/JasperFx/marten https://gitter.im/JasperFx/marten
- jeremydmiller 6y ago
  Jeremy finds your “comment” behind an anonymous name on a very public board to be extremely obnoxious and a prime example of the kind of online interaction that tends to sour me on OSS.
NicoJuicy 6y ago
I've been using this for a while now and absolutely love it. Just go through the documentation of MartenDb and I'll bet you'll like it. Note: If you are interested in DDD/microservices on. Net, you probably know Mass-Transit. But I would advice you to check out everything under JasperFX ( https://github.com/JasperFx https://github.com/JasperFx ), which MartenDb is a part of. Fyi: they hang out on gitter.me
nitinreddy88 6y ago
What I absolutely miss from Postgres is Column store support. For many analytics related workloads, columnstore beats traditional row store. I really wish they can add this soon.
- merb 6y ago
  https://github.com/citusdata/cstore_fdw https://github.com/citusdata/cstore_fdw
londondev45 6y ago
We used it for a production project which I inherited. Honestly I'm not a bit fan of anything Orm like, as such there is some magic, I believe out of the box it will literally just create tables and keys for you. For what it is, it's very fast for prototyping things out. I would personally just use the in built jsonb functionality of postgres and build out my own repositories. Saves confusion from all of the magic. My 2 pence
- NicoJuicy 6y ago
  Multi-tenancy, soft delete and duplicated properties for NoSql add a lot of value :)
ThePadawan 6y ago
We used EventStore [0] as an Event Store in a previous project (yes, this does get tedious to explain). I'm quite curious who this is aimed at. .NET shops are often so deeply invested in the Microsoft ecosystem. In that previous project, needing to run/administer PostgreSQL instead of SQL Server would have been a significant administrative detriment to getting the project greenlit. (The fact that EventStore doesn't use any RDBMS as a store at all paradoxically helped. At least that meant we weren't getting DBA split-brain problems between SQL dialects.) [0] https://eventstore.com/ https://eventstore.com/
- paranoidrobot 6y ago
  > .NET shops are often so deeply invested in the Microsoft ecosystem Nobody likes MS's SQL License fees, however. If you're going all cloud, then you can't get away with using unlicensed/insufficiently licensed RDS instances. The delta between Postgres an SQL Server instance pricing per hour doesn't take much to add up to a salary for a mid range DBA.
  - c17r 6y ago
    No kidding. I remember back in the .NET 4.5 days and SQL Server was per-socket pricing while Oracle and whatnot were per-core. It was an amazing deal. Then SQL Server went per-core and our pricing went up 8-fold. Been using PostgreSQL ever since.
    - paranoidrobot 6y ago
      SQLs socket/core pricing is a pain. We wanted some instances, had to get our hosting provider to dig up previous generation CPU machines so we could have a lower core count. They were like "We'll give you the extra cores for free", Until we explained that it was for SQL License reasons.
  - ThePadawan 6y ago
    > The delta between Postgres an SQL Server instance pricing per hour doesn't take much to add up to a salary for a mid range DBA. I agree with you, leadership didn't shrug. Keeping to as few technologies as possible is a more pricey, but also more conservative choice.
brightball 6y ago
Can somebody explain why this is only for .NET developers if it’s running on PG?
- np_tedious 6y ago
  There's a nuget package linked, so I'm guessing this is achieved via some kind of tight coupling with a clever db client library
- rraghur 6y ago
  It's a. Net client side only library
throwaway13337 6y ago
This is interesting in that it seems to organize tenants into their own database. I've built a few SaaS products with on the order of a few thousand customers per application. Performance can sometimes be a problem when each customer can have millions of rows but no relation between those customers. I believe this is pretty typical. What's the purpose of mixing different customer rows? You rarely aggregate them all together. A million rows is easy for a database but when there are billions together, it starts requiring deeper optimizations. The idea of each getting their own separate database makes intuitive sense to me but most tools for dB access aren't built with this in mind.
tehlike 6y ago
I have been hoping to find this for nodejs one day :( sad. I really miss my .net days

78 points by c17r 6y ago

I've been using Marten for maybe 3 years now, and I absolutely love it! It's extensible too, so you can do just about anything with it - for example, I use it with views, multi tenancy, a base for aggregate queries and all sorts, and it works great. The core team is really responsive on both GitHub and Gitter too, and are happy to both accept and help with PRs - it's (almost) the model of a well run OSS project. One small thing tho... the original developer, Jeremy Miller, can be a bit... spiky; like you ask a question and he just assumes bad faith and will snark at you. I imagine this behaviour has put at least a few people off contributing and using Marten. I almost feel bad about mentioning this, because I'm a fan of his work, but OTOH I kind of hope Jeremy reads this comment and takes it constructively.

c17r 6y ago
I came across Marten on someone's blog where they were talking about their ups and downs with RavenDB. It looks interesting. Since you're using it, I'd like to know more about it's performance profile. Can't really find any performance/speed comparisons online. And what kind of scale are you running. I'm right at the beginning of a project for a client and I'm roughing out the foundational code with pgsql already, so it would be a perfect time to take a look at it. Thanks!
- GordonS 6y ago
  We're not running at scale, at least not the kind where performance of the library is likely to be an issue. We have around 50 tenants with a handful of users each. Tenants have up to 100's of millions of rows in Postgres - not exactly big data levels, but pretty big still. I don't have perf numbers anymore, but I did some performance testing before I started using it a few years ago, and even back then it was much faster than EF/EFCore. The team have made many performance improvements since then. Considering the ease of use, and the fact you get NoSQL-style JSON documents that work alongside regular relational tables, basically the performance impact is as good as neglibible, IMO. If you want to work with JSON documents, I highly recommend Marten. It does have a Benchmarks project if you'd like to run some tests[0], and I'm sure the team would be happy to answer any questions over in Gitter[1]. [0] https://github.com/JasperFx/marten/blob/07d99ed80a1f0c436c4f1817421dd8831efe73c0/README.md https://github.com/JasperFx/marten/blob/07d99ed80a1f0c436c4f... [1] https://gitter.im/JasperFx/marten https://gitter.im/JasperFx/marten
jeremydmiller 6y ago
Jeremy finds your “comment” behind an anonymous name on a very public board to be extremely obnoxious and a prime example of the kind of online interaction that tends to sour me on OSS.

NicoJuicy 6y ago

I've been using this for a while now and absolutely love it. Just go through the documentation of MartenDb and I'll bet you'll like it. Note: If you are interested in DDD/microservices on. Net, you probably know Mass-Transit. But I would advice you to check out everything under JasperFX ( https://github.com/JasperFx https://github.com/JasperFx ), which MartenDb is a part of. Fyi: they hang out on gitter.me

nitinreddy88 6y ago

What I absolutely miss from Postgres is Column store support. For many analytics related workloads, columnstore beats traditional row store. I really wish they can add this soon.

merb 6y ago
https://github.com/citusdata/cstore_fdw https://github.com/citusdata/cstore_fdw

londondev45 6y ago

We used it for a production project which I inherited. Honestly I'm not a bit fan of anything Orm like, as such there is some magic, I believe out of the box it will literally just create tables and keys for you. For what it is, it's very fast for prototyping things out. I would personally just use the in built jsonb functionality of postgres and build out my own repositories. Saves confusion from all of the magic. My 2 pence

NicoJuicy 6y ago
Multi-tenancy, soft delete and duplicated properties for NoSql add a lot of value :)

ThePadawan 6y ago

We used EventStore [0] as an Event Store in a previous project (yes, this does get tedious to explain). I'm quite curious who this is aimed at. .NET shops are often so deeply invested in the Microsoft ecosystem. In that previous project, needing to run/administer PostgreSQL instead of SQL Server would have been a significant administrative detriment to getting the project greenlit. (The fact that EventStore doesn't use any RDBMS as a store at all paradoxically helped. At least that meant we weren't getting DBA split-brain problems between SQL dialects.) [0] https://eventstore.com/ https://eventstore.com/

paranoidrobot 6y ago
> .NET shops are often so deeply invested in the Microsoft ecosystem Nobody likes MS's SQL License fees, however. If you're going all cloud, then you can't get away with using unlicensed/insufficiently licensed RDS instances. The delta between Postgres an SQL Server instance pricing per hour doesn't take much to add up to a salary for a mid range DBA.
- c17r 6y ago
  No kidding. I remember back in the .NET 4.5 days and SQL Server was per-socket pricing while Oracle and whatnot were per-core. It was an amazing deal. Then SQL Server went per-core and our pricing went up 8-fold. Been using PostgreSQL ever since.
  - paranoidrobot 6y ago
    SQLs socket/core pricing is a pain. We wanted some instances, had to get our hosting provider to dig up previous generation CPU machines so we could have a lower core count. They were like "We'll give you the extra cores for free", Until we explained that it was for SQL License reasons.
- ThePadawan 6y ago
  > The delta between Postgres an SQL Server instance pricing per hour doesn't take much to add up to a salary for a mid range DBA. I agree with you, leadership didn't shrug. Keeping to as few technologies as possible is a more pricey, but also more conservative choice.

brightball 6y ago

Can somebody explain why this is only for .NET developers if it’s running on PG?

np_tedious 6y ago
There's a nuget package linked, so I'm guessing this is achieved via some kind of tight coupling with a clever db client library
rraghur 6y ago
It's a. Net client side only library

throwaway13337 6y ago

This is interesting in that it seems to organize tenants into their own database. I've built a few SaaS products with on the order of a few thousand customers per application. Performance can sometimes be a problem when each customer can have millions of rows but no relation between those customers. I believe this is pretty typical. What's the purpose of mixing different customer rows? You rarely aggregate them all together. A million rows is easy for a database but when there are billions together, it starts requiring deeper optimizations. The idea of each getting their own separate database makes intuitive sense to me but most tools for dB access aren't built with this in mind.

tehlike 6y ago

I have been hoping to find this for nodejs one day :( sad. I really miss my .net days