4 ms·
I'm building a Node.js app where I was thinking of using MongoDB but I'm reconsidering it after reading the comments here. The database's only job is to keep p
by Kiro 10y ago
I'm building a Node.js app where I was thinking of using MongoDB but I'm reconsidering it after reading the comments here.
The database's only job is to keep persistence between server restarts. On server start I fetch the a collection from the database and populate a variable with it, which I use for any direct access.
When a user updates something it will update the variable as well as updating the collection in the database.
Something like this:
var foo = readFromDB();
socket.on('connection', function() {
this.emit(foo);
});
socket.on('update', function(data) {
foo[data.id] = data.val;
updateDB(data.id, data.val);
});
(In my real application each collection has sub-arrays etc and not just key-value like in the example.)
What should I use instead of MongoDB for this? Or is my approach fundamentally flawed?
- Fannon 10y agoBasically any database should do for this usecase. A simple key-value store probably best. I'd aim for keeping the technological complexity as low as possible. What is the simplest tool to do the job? Or maybe better yet: Use something that's already (or will be), in your technology stack to avoid having one more thing in it.
- Kiro 10y agoThanks! > What is the simplest tool to do the job? For me it's definitely MongoDB but I'm afraid of data loss etc. My example was simplified. In reality I want to update a sub-array within the collection, like this: collection['some_identifier']['items'][3] = 123. Overwriting the whole 'some_identifier' document seems wasteful. With MongoDB it's easy though to update nested sub-data.
- elmigranto 10y agoIf you ever only need to read or update single document in a single collection, Mongo's fine. When you need anything more (which is probably "soon" if not "already"), it'll bite one way or the other.
- threeseed 10y agoThis absolutely isn't true. If you batch operations it is extremely fast at updates across multiple documents.
- elmigranto 10y agoPoor wording on my part. My point isn't about performance, it's about API. Other than selecting documents from single collection (.find), Mongo's queries are shockingly wierd and inconsistent to me. Compare aggregation framework to what SQL has, for example: https://docs.mongodb.com/manual/reference/sql-aggregation-comparison/ https://docs.mongodb.com/manual/reference/sql-aggregation-co... Or even within Mongo, querying and aggregating are astonishingly different things, that you need to learn separately. And then there are JOINS. And stale reads with other serialization issues.
- bflesch 10y agoWe've been running node+postgres in production for a couple of years and it's been great so far. If you have advanced questions, #postgresql on freenode is always there for help, and the node-pg libraries are very mature.
- Kiro 10y agoCool! Thanks a lot. I will give it a try.
- twblalock 10y agoYou could even use Redis for this. You really need a cache or key-value store, not a database.
- Kiro 10y agoThanks! In my real application I want to be able to update a sub-array of the document, like this: collection['some_identifier']['items'][3] = 123. Just setting the value for the key (e.g. overwriting the whole document) seems wasteful since I only want to update a portion of it. With MongoDB it's easy to set sub-data so I guess I'm looking for something where it's as easy but without the downsides.
- amdavidson 10y agoIn redis couldn't your key be something like "someIdentifier_items_3" and your value be 123? Then if you wanted to make it 456 you would only be updating "someIdentifier_items_3".
- orf 10y agoWhat's wrong with using a plain file?
- Kiro 10y agoIt would probably work but in my real application it's a bit more complicated since I only want to update a sub-array of the document, like this: collection['some_identifier']['items'][3] = 123. I guess using a plain file would mean I have to overwrite everything on each write which seems wasteful.
- orf 10y agoSure, it depends on your use case though. For infrequent updates I don't think it's wasteful to just overwrite the file, it seems less wasteful to me than running a whole database. But it all depends on your setup :) Also I think updating a sub array in a collection triggers a whole write to that collection in mongodb. In any case I would avoid mongodb if you can, it seems redis might be a better fit for you.
- elmigranto 10y agoThis should read "plain SQLite file". At least it'll handle serialization and FS failures for you.
- weddpros 10y agoIf you ever want to scale your app to more than one process (CPU core), or more than one server, you'll have a problem... Using Redis could be a solution (if Redis can accommodate your read pattern), or you'll need a pub/sub system (like that of redis) to propagate db changes from one process to the other...
- Kiro 10y agoThanks! I'm building a real-time app using WebSockets so a pub/sub system sounds like a good solution.
- elmigranto 10y agoPostgres has it with LISTEN/NOTIFY. It also can read from and write to Redis using Foreign Data Wrappers. Pretty neat, if you want your DB to be single place you store data, but something faster for reads or cache.
- lmm 10y agoIf you want to do updates you want a traditional transactional database. If you really need massive scalability (which seems unlikely) you should figure out a way to store your data in a log-structured/append-only way, and then use Cassandra or Riak.