9 ms·

Show HN: Mongita is to MongoDB as SQLite is to SQL

126 points by scottrogowski 5y ago

rvn1045 5y ago
This would be super nice to have when developing mobile apps. I’ve used sqllite when developing with react-native, but can see myself using a mongita type solution.
- rad_gruchalski 5y ago
  It’s written in python so I assume that’s a no go.
- mr_toad 5y ago
  If you’re already using JavaScript and you want an embeddable document database you could look at PouchDB.
teryyy 5y ago
In memory, how does perf compare to https://github.com/nodkz/mongodb-memory-server https://github.com/nodkz/mongodb-memory-server ?
- scottrogowski 5y ago
  That's a good question. I'll try adding it to the benchmarks.
- nerdponx 5y ago
  Does this require the enterprise edition of Mongo for its in-memory storage feature?
ruffrey 5y ago
There used to be something similar called NeDB
- xtracto 5y ago
  And PStore for Ruby and trivialDB for Javascript
- lttlrck 5y ago
  Lungo in Go. https://github.com/256dpi/lungo https://github.com/256dpi/lungo
latenightcoding 5y ago
> Mongita is to MongoDB as SQLite is to SQL it really isn't if it's written in python
- OJFord 5y ago
  I don't think it is whatever it's written in, because I can't imagine what it would mean? Unless there's a DBMS implementation called simply 'SQL' that I'm not aware of? (Or if Mongo calls its query language 'MongoDB'?) Seems like '.. to (My|Postgre)SQL' would be a better fit?
  - emteycz 5y ago
    I guess the idea relates to "a SQL database" - might be technically incorrect, but with Mongo it's sort of embedded in NoSQL - that also originally meant "no SQL [language]", but has turned to mean "no SQL and not a classical RDBMS with databases, tables and columns, rows, views, procedures..."
  - tracker1 5y ago
    SQLite is a portable DBMS library that exposes a SQL language interface that is built and runs under every OS and language under the sun, for the most part. It is probably the single-most distributed library on the planet (in aggregate). And likely one of the most used libraries (top 10) on just about every given OS and platform/language. Mongita is only useful for Python developers.
    - ipaddr 5y ago
      I like sqlite and agree with your it should run everywhere to be compared. I question that it is the most distributed library on the planet. Is there something backing that up? It would be cool but very shocking.
      - arthur2e5 5y ago
        I can't say it's the most, but it does get used a lot. Chromium and descendants, Firefox, bunch of chat apps (Telegram, WeChat), and so on. You probably have multiple copies per machine. The SQLite people are quite proud of their ubiquity: https://sqlite.org/mostdeployed.html https://sqlite.org/mostdeployed.html
saurik 5y ago
So, by their own benchmarks, unless you are doing totally random lookups of documents by identifier--and mostly reads, with very few writes--you should absolutely use SQLite with JSON values, which absolutely destroy this project in performance?...
- abrookewood 5y ago
  The project isn't about performance, it's about providing an embedded version of MongoDB - it clearly states that if you grow too large, then you can easily migrate to the full MongoDB. It also clearly states not to use it if you want a relational database.
  - remram 5y ago
    Then the comparison with SQLite is totally wrong. SQLite is a production-grade DBMS: though it has fewer features compared to e.g. PostgreSQL, if those feature suffice, you can throw very sizable workloads at it.
    - tracker1 5y ago
      Came to say largely the same... especially considering this is really Python only, where SQLite is available pretty much anywhere you run code.
    - deleted 5y ago
      [deleted]
    - nerdponx 5y ago
      The comparison is apt in that it's a library, not a separate server, and it can store data in memory, which is normally only available in Mongo enterprise.
  - scottrogowski 5y ago
    Author here. That's exactly it. I think it would be very difficult to completely beat SQLite with a Python library. My goal wasn't to beat it but to have performance that's within an order of magnitude which I think I've achieved. In my opinion, the MongoDB interface has a lot of advantages over SQL that make sense in a lot of use cases. Certainly, there are times when a traditional relational database is the right choice. But I do think Mongita fills a niche. Separately, the JSON1 extension, which the top-level comment refers to, is nice technically but has a challenging interface IMHO https://www.sqlite.org/json1.html https://www.sqlite.org/json1.html
hardwaresofton 5y ago
Cool project but I have to admit I sure wish this was written on top of SQLite, rather than just mentioned it, implementing a query language shim for MongoDB on SQLite would be an amazing project. In the absence of such a project though, this is a pretty great alternative to have.
- scottrogowski 5y ago
  Thank you! I actually did consider doing exactly what you said. Early on, I decided one my goals for the project would be to make it easy to swap back and forth between Mongita and PyMongo/MongoDB. For me, that meant getting as close as possible to their implementation and using things like BSON, ObjectIds, etc. For that, I sacrificed some performance but I think most people who would use something like Mongita would prefer a faithful reproduction over a few clock cycles. I could be wrong on that though.
  - hardwaresofton 5y ago
    Oh thanks for the insight, that makes a lot of sense -- pretty sure mongita is one of the first projects I've seen even attempt this, and it makes sense to target the platform that fits your use case (and obviously everyone who uses PyMongo) the tightest. Your project looks very high quality -- benchmarks, tests, and comparisons are basically an indicator in my mind. Looking through the code you've also already left me (or someone else) space to try doing the SQLite as long as we implement it as an Engine[0] -- am I understanding that right? If I trace the code from database.py to engines/.py it looks like that. I really like the balance you've picked between pragmatism and space for expansion/modification. A couple questions: - How do you feel about type annotations in* the code (as opposed to just the comments, as far as I can see) - PyPy? I wonder if you'd get a ~free speedup [0]: https://github.com/scottrogowski/mongita/blob/master/mongita/engines/disk_engine.py https://github.com/scottrogowski/mongita/blob/master/mongita...
    - scottrogowski 5y ago
      Thanks for the kind words :) I like your idea. As I understand it, since SQLite performs better on most benchmarks anyway, why not have an engine subclass that utilizes SQLite itself? This would be a little trickier than your envisioning. The engine class is the lowest layer but doesn't handle the slow bits like the finds, managing the indicies, etc. So to really get the benefit of SQLite you would have to pull those slow bits in. I do really like the idea of offering a third option so that you have - Memory (fastest) - SQLite (almost as fast but compromises on perfect MongoDB reproduction) - Disk (slower but is faithful to MongoDB) Happy to discuss it more if you want to email me. > How do you feel about type annotations in* the code (as opposed to just the comments, as far as I can see) I used type annotations for a while when they came out but didn't find them to offer advantages over docstrings. The world might have moved over to type annotations though and I've not been aware. > PyPy? I wonder if you'd get a ~free speedup I'm getting a lot of suspicion right now from other commenters who are convinced I have my thumb on the benchmarks and I don't want to give them any more excuses :tears-of-joy:. But joking aside, you're probably right and it's not something I had considered.
elviejo 5y ago
I'm looking for 'X' is to Neo4j as Sqlite is to SQL any suggestions for what project is X?
- havernator 5y ago
  Assuming the main thing you want out of it is the Cypher language (surely that's the main thing?) it looks like there's nothing like that. https://www.opencypher.org/projects https://www.opencypher.org/projects None of those appear to be a SQLite-like file-based embeddable database system.
- m0th87 5y ago
  For rust, I manage a graph database that can be embedded as a library: https://github.com/indradb/indradb/ https://github.com/indradb/indradb/
axiosgunnar 5y ago
How can the benchmark show Mongita be around as fast as SQLite if SQLite is written in C and Mongita in Python? Genuine question!
- scottrogowski 5y ago
  It's a good question and to be accurate, depending on the benchmark, Mongita is about the same speed at SQLite to several-times slower. There is less happening algorithmically than you would think. Where the tricky slow bits do exist, they have largely fallen into the happy-path of fast data structures in the Python language/stdlib. I also use sortedcontainers for indexes which helped quite a bit (http://www.grantjenks.com/docs/sortedcontainers/ http://www.grantjenks.com/docs/sortedcontainers/). If you're curious, the benchmark code is in the repo: https://github.com/scottrogowski/mongita/blob/master/benchmark_tests/benchmark.py https://github.com/scottrogowski/mongita/blob/master/benchma...
macintux 5y ago
It's telling how many killer features SQLite has, that so many people are complaining about the comparison for different reasons.
kabes 5y ago
Ok, if we forget the comparison with sqlite, it's a nice and useful library for python devs Thanks for making it available.
catchmeifyoucan 5y ago
Anyway to integrate it with a NodeJs project? Seems like a good fit for Electron apps!
- ogre_codes 5y ago
  Seems unlikely, it's written in Python. Might be possible to bodge something together, but interop would be crap.
- fulafel 5y ago
  PyNode should work.
bvrmn 5y ago
It's sad that author did not mentioned that disk engine basically stores copy of data in memory. And it seems from benchmark code that reading doesn't hit disk at all.
- scottrogowski 5y ago
  Hey, your first statement is true but your second is not. The disk engine does store data in memory which is part of the design. You wouldn't want to use a database that doesn't utilize caching. The last benchmark panel, https://raw.githubusercontent.com/scottrogowski/mongita/master/assets/performance_comparison_cold_starts.svg https://raw.githubusercontent.com/scottrogowski/mongita/mast..., shows cold starts where I test it without cache. So in that, it does hit the disk.
  - lelanthran 5y ago
    > The last benchmark panel, https://raw.githubusercontent.com/scottrogowski/mongita/mast https://raw.githubusercontent.com/scottrogowski/mongita/mast..., shows cold starts where I test it without cache. So in that, it does hit the disk. Okay, so looking at the first two tests - "Retrieve all documents" and "Get 1000 documents by ID" ... If you switch the order around, does it make a difference to the benchmark? Because I suspect that the first test preloads all records into RAM, and the second test simply searches RAM, which is not what we usually do with SQLite. We don't cache all records before searching. Switch those first two tests around, and lets see if it makes a difference.
    - scottrogowski 5y ago
      You're right. Fixed. SQLite is more-so the winner but Mongita appears to squeak ahead in id lookups https://github.com/scottrogowski/mongita/blob/master/assets/performance_comparison_cold_starts.svg https://github.com/scottrogowski/mongita/blob/master/assets/... (to be fair, it might be the thin translation layer I built). Regular MongoDB struggles a lot and I don't think I'm being unfair to it in any way afaik. Thank you for pointing that all out.
  - 5y ago
ogre_codes 5y ago
"Mongita is to MongoDB as SQLite is to SQL" SQLite is a embeddable SQL implementation which has been ported to dozens of platforms with no requirements. Mongita is a Python library. I like Python as much as the next guy, but the comparison is pretty far off whack. SQLite is popular because it embeds everywhere easily. This doesn't. I can't use this on my iPhone app. It's likely way too fat for Android and awkward at best on Android.
deknos 5y ago
what i always think about stuff like this: if stuff like mongita and sqlite would exist for all kinds of databases (graph,kv,xml; as for document and sql it already exists), couldn't we "just make distributed versions" if we put stuff ontop of it? like with dqlite/rqlite with sqlite? or does there have to be some inherent mechanisms withIN the database to support distributed versions?
- scottrogowski 5y ago
  I think as I understand it, your question is whether we can't just put a distributed layer on top of basic embedded databases. This is really interesting and is something I came across while writing this. It turns out that concurrency is actually quite difficult because either you have global locks, which means only one process can write to the database/indicies at once and slows things down considerably, or you have to do a lot of clever things to avoid those locks.
  - deknos 5y ago
    > I think as I understand it, your question is whether we can't just put a distributed layer on top of basic embedded databases. Exactly! > This is really interesting and is something I came across while writing this. It turns out that concurrency is actually quite difficult because either you have global locks, which means only one process can write to the database/indicies at once and slows things down considerably, or you have to do a lot of clever things to avoid those locks. well, that would be also the case with traditional db services, the question can they have more granular mechanisms for more granular locking than embedded databases. but perhaps they even can have only less granular locking?
- ezrast 5y ago
  There are generic distributed consensus algorithms out there. The most famous are Paxos and Raft. In theory, you can jam those on top of any system you like, as long as it has well-defined state transitions. Making it fast - or usable at all in the presence of heavy contention - is another story. Distributing a write-heavy workload over a cluster is useless if the cluster ends up rejecting most updates because they get preempted by some other write. Solving that problem usually means analyzing the underlying system to figure out which parts need to be truly atomic and which you can get away with doing in parallel. That job is a) really complex and b) filled with opportunities to make significant performance gains in exchange for weaker safety guarantees, like losing committed writes in a crash, or allowing individual nodes to reorder independent writes. You should check out http://jepsen.io/analyses http://jepsen.io/analyses if this stuff interests you.
syntonym2 5y ago
I mostly use SQLite for persistence - put anything in and it's resistant against crashes and corruption. What is the persistence story with Mongita?
jbverschoor 5y ago
SQL is a language and specification. How can I trust something that makes a comparison that's not right, or a product that compares itself to mongodb
- kevincox 5y ago
  SQL is a query language, MongoDB has a query language. I think that the tagline is reasonable and is made more precise in the first sentence of the README. > Mongita is a lightweight embedded document database that implements a commonly-used subset of the MongoDB/PyMongo interface.
jbverschoor 5y ago
afaict, the disk_engine does not lock the file when reading / writing. Also, the storage engine is bson and relies on cached offsets. There's also this non-atomic defrag method. I dunno.. wouldn't touch it with a pole wearing a hazmat suit, sorry. Using sqlite for storage and querying would've been better. Heck, that would be pretty great for moving a few smalller (server) applications off of mongodb. Although they're using ruby
giorgioz 5y ago
Very cool, I'm looking for MongoDB to be able to run on AWS Lambda and be more serverless. This seem like a step in the good direction. Personally I think JS would have been a better choice to implement this in than Python given all the mongo queries are in javascript db.collection('users').find({_id:'AN_ID'}) Also having it made in javascript would have open the option to embed it in the browser later on.
kragen 5y ago
"Mongita" is a misspelling according to the DLE; the standard spelling is "monjita". Literally, the word means "little nun", but is the common name for flycatchers of the genus Xolmis. https://dle.rae.es/?formList=form&w=monjita# https://dle.rae.es/?formList=form&w=monjita# https://es.wikipedia.org/wiki/Xolmis_rubetra https://es.wikipedia.org/wiki/Xolmis_rubetra Unfortunately every time I see the misspelling in this thread I involuntarily cringe. I suppose "MongoDB" is named after a slur used to insult people with Down syndrome, so maybe calling this project the Spanish equivalent of "Magolia", "Mamalian", or "Meercat" is a clever reversal of the insult into a form of self-deprecation on the part of the author, who is wittily feigning illiteracy? Or perhaps it is intended to ridicule the speling of Spainards and other speekers of Spansh? Or programmers who decided to yoke their applications to fake open source? Even if correctly spelled, perhaps the name would be more appropriate to a debugging tool than to a hash table implementation.
- rpastuszak 5y ago
  FYI the name MongoDB comes from "humongous". Thanks for the wiki links, though.
  - kragen 5y ago
    You're welcome! I hope the information is useful in helping scottrogowski carefully consider how many people he wants to insult. However, the DLE is not a Wiki.
- davisoneee 5y ago
  To be fair, one suggested etymology I've seen for for 'mongo' is a simplification of 'humongous'. Whether you believe this claim or not, it's entirely down to whether you want to take a charitable or negative guess at the original intention.
  - kragen 5y ago
    Anyone who reads Spanish fluently will react to the misspelling "Mongita"—whether with distaste, amusement, pity, or defensiveness—long before they have time to think through the possible motivations of the MongoDB developers.
justsomeuser 5y ago
I think you should change your tag line: - From: "Mongita is to MongoDB as SQLite is to SQL" - To: "Mongita is to MongoDB as SQLite is to MySQL" When I see "SQL" I think of the textual query language (not a server SQL process/engine).
null4bl3 5y ago
The valid use cases for non SQL databases can be counted on one hand. Not sure this project understands that

126 points by scottrogowski 5y ago

This would be super nice to have when developing mobile apps. I’ve used sqllite when developing with react-native, but can see myself using a mongita type solution.

rad_gruchalski 5y ago
It’s written in python so I assume that’s a no go.
mr_toad 5y ago
If you’re already using JavaScript and you want an embeddable document database you could look at PouchDB.

teryyy 5y ago

In memory, how does perf compare to https://github.com/nodkz/mongodb-memory-server https://github.com/nodkz/mongodb-memory-server ?

scottrogowski 5y ago
That's a good question. I'll try adding it to the benchmarks.
nerdponx 5y ago
Does this require the enterprise edition of Mongo for its in-memory storage feature?

ruffrey 5y ago

There used to be something similar called NeDB

xtracto 5y ago
And PStore for Ruby and trivialDB for Javascript
lttlrck 5y ago
Lungo in Go. https://github.com/256dpi/lungo https://github.com/256dpi/lungo

latenightcoding 5y ago

> Mongita is to MongoDB as SQLite is to SQL it really isn't if it's written in python

OJFord 5y ago
I don't think it is whatever it's written in, because I can't imagine what it would mean? Unless there's a DBMS implementation called simply 'SQL' that I'm not aware of? (Or if Mongo calls its query language 'MongoDB'?) Seems like '.. to (My|Postgre)SQL' would be a better fit?
- emteycz 5y ago
  I guess the idea relates to "a SQL database" - might be technically incorrect, but with Mongo it's sort of embedded in NoSQL - that also originally meant "no SQL [language]", but has turned to mean "no SQL and not a classical RDBMS with databases, tables and columns, rows, views, procedures..."
- tracker1 5y ago
  SQLite is a portable DBMS library that exposes a SQL language interface that is built and runs under every OS and language under the sun, for the most part. It is probably the single-most distributed library on the planet (in aggregate). And likely one of the most used libraries (top 10) on just about every given OS and platform/language. Mongita is only useful for Python developers.
  - ipaddr 5y ago
    I like sqlite and agree with your it should run everywhere to be compared. I question that it is the most distributed library on the planet. Is there something backing that up? It would be cool but very shocking.
    - arthur2e5 5y ago
      I can't say it's the most, but it does get used a lot. Chromium and descendants, Firefox, bunch of chat apps (Telegram, WeChat), and so on. You probably have multiple copies per machine. The SQLite people are quite proud of their ubiquity: https://sqlite.org/mostdeployed.html https://sqlite.org/mostdeployed.html

saurik 5y ago

So, by their own benchmarks, unless you are doing totally random lookups of documents by identifier--and mostly reads, with very few writes--you should absolutely use SQLite with JSON values, which absolutely destroy this project in performance?...

abrookewood 5y ago
The project isn't about performance, it's about providing an embedded version of MongoDB - it clearly states that if you grow too large, then you can easily migrate to the full MongoDB. It also clearly states not to use it if you want a relational database.
- remram 5y ago
  Then the comparison with SQLite is totally wrong. SQLite is a production-grade DBMS: though it has fewer features compared to e.g. PostgreSQL, if those feature suffice, you can throw very sizable workloads at it.
  - tracker1 5y ago
    Came to say largely the same... especially considering this is really Python only, where SQLite is available pretty much anywhere you run code.
  - deleted 5y ago
    [deleted]
  - nerdponx 5y ago
    The comparison is apt in that it's a library, not a separate server, and it can store data in memory, which is normally only available in Mongo enterprise.
- scottrogowski 5y ago
  Author here. That's exactly it. I think it would be very difficult to completely beat SQLite with a Python library. My goal wasn't to beat it but to have performance that's within an order of magnitude which I think I've achieved. In my opinion, the MongoDB interface has a lot of advantages over SQL that make sense in a lot of use cases. Certainly, there are times when a traditional relational database is the right choice. But I do think Mongita fills a niche. Separately, the JSON1 extension, which the top-level comment refers to, is nice technically but has a challenging interface IMHO https://www.sqlite.org/json1.html https://www.sqlite.org/json1.html

hardwaresofton 5y ago

Cool project but I have to admit I sure wish this was written on top of SQLite, rather than just mentioned it, implementing a query language shim for MongoDB on SQLite would be an amazing project. In the absence of such a project though, this is a pretty great alternative to have.

scottrogowski 5y ago
Thank you! I actually did consider doing exactly what you said. Early on, I decided one my goals for the project would be to make it easy to swap back and forth between Mongita and PyMongo/MongoDB. For me, that meant getting as close as possible to their implementation and using things like BSON, ObjectIds, etc. For that, I sacrificed some performance but I think most people who would use something like Mongita would prefer a faithful reproduction over a few clock cycles. I could be wrong on that though.
- hardwaresofton 5y ago
  Oh thanks for the insight, that makes a lot of sense -- pretty sure mongita is one of the first projects I've seen even attempt this, and it makes sense to target the platform that fits your use case (and obviously everyone who uses PyMongo) the tightest. Your project looks very high quality -- benchmarks, tests, and comparisons are basically an indicator in my mind. Looking through the code you've also already left me (or someone else) space to try doing the SQLite as long as we implement it as an Engine[0] -- am I understanding that right? If I trace the code from database.py to engines/.py it looks like that. I really like the balance you've picked between pragmatism and space for expansion/modification. A couple questions: - How do you feel about type annotations in* the code (as opposed to just the comments, as far as I can see) - PyPy? I wonder if you'd get a ~free speedup [0]: https://github.com/scottrogowski/mongita/blob/master/mongita/engines/disk_engine.py https://github.com/scottrogowski/mongita/blob/master/mongita...
  - scottrogowski 5y ago
    Thanks for the kind words :) I like your idea. As I understand it, since SQLite performs better on most benchmarks anyway, why not have an engine subclass that utilizes SQLite itself? This would be a little trickier than your envisioning. The engine class is the lowest layer but doesn't handle the slow bits like the finds, managing the indicies, etc. So to really get the benefit of SQLite you would have to pull those slow bits in. I do really like the idea of offering a third option so that you have - Memory (fastest) - SQLite (almost as fast but compromises on perfect MongoDB reproduction) - Disk (slower but is faithful to MongoDB) Happy to discuss it more if you want to email me. > How do you feel about type annotations in* the code (as opposed to just the comments, as far as I can see) I used type annotations for a while when they came out but didn't find them to offer advantages over docstrings. The world might have moved over to type annotations though and I've not been aware. > PyPy? I wonder if you'd get a ~free speedup I'm getting a lot of suspicion right now from other commenters who are convinced I have my thumb on the benchmarks and I don't want to give them any more excuses :tears-of-joy:. But joking aside, you're probably right and it's not something I had considered.

elviejo 5y ago

I'm looking for 'X' is to Neo4j as Sqlite is to SQL any suggestions for what project is X?

havernator 5y ago
Assuming the main thing you want out of it is the Cypher language (surely that's the main thing?) it looks like there's nothing like that. https://www.opencypher.org/projects https://www.opencypher.org/projects None of those appear to be a SQLite-like file-based embeddable database system.
m0th87 5y ago
For rust, I manage a graph database that can be embedded as a library: https://github.com/indradb/indradb/ https://github.com/indradb/indradb/

axiosgunnar 5y ago

How can the benchmark show Mongita be around as fast as SQLite if SQLite is written in C and Mongita in Python? Genuine question!

scottrogowski 5y ago
It's a good question and to be accurate, depending on the benchmark, Mongita is about the same speed at SQLite to several-times slower. There is less happening algorithmically than you would think. Where the tricky slow bits do exist, they have largely fallen into the happy-path of fast data structures in the Python language/stdlib. I also use sortedcontainers for indexes which helped quite a bit (http://www.grantjenks.com/docs/sortedcontainers/ http://www.grantjenks.com/docs/sortedcontainers/). If you're curious, the benchmark code is in the repo: https://github.com/scottrogowski/mongita/blob/master/benchmark_tests/benchmark.py https://github.com/scottrogowski/mongita/blob/master/benchma...

macintux 5y ago

It's telling how many killer features SQLite has, that so many people are complaining about the comparison for different reasons.

kabes 5y ago

Ok, if we forget the comparison with sqlite, it's a nice and useful library for python devs Thanks for making it available.

catchmeifyoucan 5y ago

Anyway to integrate it with a NodeJs project? Seems like a good fit for Electron apps!

ogre_codes 5y ago
Seems unlikely, it's written in Python. Might be possible to bodge something together, but interop would be crap.
fulafel 5y ago
PyNode should work.

bvrmn 5y ago

It's sad that author did not mentioned that disk engine basically stores copy of data in memory. And it seems from benchmark code that reading doesn't hit disk at all.

scottrogowski 5y ago
Hey, your first statement is true but your second is not. The disk engine does store data in memory which is part of the design. You wouldn't want to use a database that doesn't utilize caching. The last benchmark panel, https://raw.githubusercontent.com/scottrogowski/mongita/master/assets/performance_comparison_cold_starts.svg https://raw.githubusercontent.com/scottrogowski/mongita/mast..., shows cold starts where I test it without cache. So in that, it does hit the disk.
- lelanthran 5y ago
  > The last benchmark panel, https://raw.githubusercontent.com/scottrogowski/mongita/mast https://raw.githubusercontent.com/scottrogowski/mongita/mast..., shows cold starts where I test it without cache. So in that, it does hit the disk. Okay, so looking at the first two tests - "Retrieve all documents" and "Get 1000 documents by ID" ... If you switch the order around, does it make a difference to the benchmark? Because I suspect that the first test preloads all records into RAM, and the second test simply searches RAM, which is not what we usually do with SQLite. We don't cache all records before searching. Switch those first two tests around, and lets see if it makes a difference.
  - scottrogowski 5y ago
    You're right. Fixed. SQLite is more-so the winner but Mongita appears to squeak ahead in id lookups https://github.com/scottrogowski/mongita/blob/master/assets/performance_comparison_cold_starts.svg https://github.com/scottrogowski/mongita/blob/master/assets/... (to be fair, it might be the thin translation layer I built). Regular MongoDB struggles a lot and I don't think I'm being unfair to it in any way afaik. Thank you for pointing that all out.
- 5y ago

ogre_codes 5y ago

"Mongita is to MongoDB as SQLite is to SQL" SQLite is a embeddable SQL implementation which has been ported to dozens of platforms with no requirements. Mongita is a Python library. I like Python as much as the next guy, but the comparison is pretty far off whack. SQLite is popular because it embeds everywhere easily. This doesn't. I can't use this on my iPhone app. It's likely way too fat for Android and awkward at best on Android.

deknos 5y ago

what i always think about stuff like this: if stuff like mongita and sqlite would exist for all kinds of databases (graph,kv,xml; as for document and sql it already exists), couldn't we "just make distributed versions" if we put stuff ontop of it? like with dqlite/rqlite with sqlite? or does there have to be some inherent mechanisms withIN the database to support distributed versions?

scottrogowski 5y ago
I think as I understand it, your question is whether we can't just put a distributed layer on top of basic embedded databases. This is really interesting and is something I came across while writing this. It turns out that concurrency is actually quite difficult because either you have global locks, which means only one process can write to the database/indicies at once and slows things down considerably, or you have to do a lot of clever things to avoid those locks.
- deknos 5y ago
  > I think as I understand it, your question is whether we can't just put a distributed layer on top of basic embedded databases. Exactly! > This is really interesting and is something I came across while writing this. It turns out that concurrency is actually quite difficult because either you have global locks, which means only one process can write to the database/indicies at once and slows things down considerably, or you have to do a lot of clever things to avoid those locks. well, that would be also the case with traditional db services, the question can they have more granular mechanisms for more granular locking than embedded databases. but perhaps they even can have only less granular locking?
ezrast 5y ago
There are generic distributed consensus algorithms out there. The most famous are Paxos and Raft. In theory, you can jam those on top of any system you like, as long as it has well-defined state transitions. Making it fast - or usable at all in the presence of heavy contention - is another story. Distributing a write-heavy workload over a cluster is useless if the cluster ends up rejecting most updates because they get preempted by some other write. Solving that problem usually means analyzing the underlying system to figure out which parts need to be truly atomic and which you can get away with doing in parallel. That job is a) really complex and b) filled with opportunities to make significant performance gains in exchange for weaker safety guarantees, like losing committed writes in a crash, or allowing individual nodes to reorder independent writes. You should check out http://jepsen.io/analyses http://jepsen.io/analyses if this stuff interests you.

syntonym2 5y ago

I mostly use SQLite for persistence - put anything in and it's resistant against crashes and corruption. What is the persistence story with Mongita?

jbverschoor 5y ago

SQL is a language and specification. How can I trust something that makes a comparison that's not right, or a product that compares itself to mongodb

kevincox 5y ago
SQL is a query language, MongoDB has a query language. I think that the tagline is reasonable and is made more precise in the first sentence of the README. > Mongita is a lightweight embedded document database that implements a commonly-used subset of the MongoDB/PyMongo interface.

jbverschoor 5y ago

afaict, the disk_engine does not lock the file when reading / writing. Also, the storage engine is bson and relies on cached offsets. There's also this non-atomic defrag method. I dunno.. wouldn't touch it with a pole wearing a hazmat suit, sorry. Using sqlite for storage and querying would've been better. Heck, that would be pretty great for moving a few smalller (server) applications off of mongodb. Although they're using ruby

giorgioz 5y ago

Very cool, I'm looking for MongoDB to be able to run on AWS Lambda and be more serverless. This seem like a step in the good direction. Personally I think JS would have been a better choice to implement this in than Python given all the mongo queries are in javascript db.collection('users').find({_id:'AN_ID'}) Also having it made in javascript would have open the option to embed it in the browser later on.

kragen 5y ago

"Mongita" is a misspelling according to the DLE; the standard spelling is "monjita". Literally, the word means "little nun", but is the common name for flycatchers of the genus Xolmis. https://dle.rae.es/?formList=form&w=monjita# https://dle.rae.es/?formList=form&w=monjita# https://es.wikipedia.org/wiki/Xolmis_rubetra https://es.wikipedia.org/wiki/Xolmis_rubetra Unfortunately every time I see the misspelling in this thread I involuntarily cringe. I suppose "MongoDB" is named after a slur used to insult people with Down syndrome, so maybe calling this project the Spanish equivalent of "Magolia", "Mamalian", or "Meercat" is a clever reversal of the insult into a form of self-deprecation on the part of the author, who is wittily feigning illiteracy? Or perhaps it is intended to ridicule the speling of Spainards and other speekers of Spansh? Or programmers who decided to yoke their applications to fake open source? Even if correctly spelled, perhaps the name would be more appropriate to a debugging tool than to a hash table implementation.

rpastuszak 5y ago
FYI the name MongoDB comes from "humongous". Thanks for the wiki links, though.
- kragen 5y ago
  You're welcome! I hope the information is useful in helping scottrogowski carefully consider how many people he wants to insult. However, the DLE is not a Wiki.
davisoneee 5y ago
To be fair, one suggested etymology I've seen for for 'mongo' is a simplification of 'humongous'. Whether you believe this claim or not, it's entirely down to whether you want to take a charitable or negative guess at the original intention.
- kragen 5y ago
  Anyone who reads Spanish fluently will react to the misspelling "Mongita"—whether with distaste, amusement, pity, or defensiveness—long before they have time to think through the possible motivations of the MongoDB developers.

justsomeuser 5y ago

I think you should change your tag line: - From: "Mongita is to MongoDB as SQLite is to SQL" - To: "Mongita is to MongoDB as SQLite is to MySQL" When I see "SQL" I think of the textual query language (not a server SQL process/engine).

null4bl3 5y ago

The valid use cases for non SQL databases can be counted on one hand. Not sure this project understands that