7 ms·
I'm very curious about this. I feel like the community at large seems to agree mongo is a dangerous choice. But then I read articles like this. At what scale do
by whalesalad 14y ago
I'm very curious about this. I feel like the community at large seems to agree mongo is a dangerous choice. But then I read articles like this. At what scale do you replace Mongo? Should it only be used for bootstrapping something? Is it something that 80% of apps will be fine on?
I'm in an interesting spot myself right now. I've been hacking on Django since 0.96. I love the ORM, and hate everything else. Python on the other hand is a wonderful language. Rails... I've been spending 8 hours a day for the past month building a rails project. I love a lot of things about rails, hate the ORM, and dislike a lot of the magic.
On the flip side: I've been a Javascript/front-end developer since before jQuery existed, and I love coffeescript. I understand callback-style evented programming and therefore node.js apps make perfect sense to me.
My co-founder and I have just finished some heavy duty design/wireframe/conceptual work on a new project of ours. It's time to build the API.
I am COMPLETELY torn with what framework/language/database to use for this project.
At first I thought mongodb and coffeescript+node would be fun and reliable. But the community seems to think mongo is unreliable and almost dangerous. So I thought, hmm, Riak looks like it will let me throw whatever data I want into it (like mongo) but it's rock solid. I later dismissed that since I figured I'd need to do a lot of heavy lifting on my own. Rails keeps poking me in the back of my mind but I honest-to-god hate ActiveRecord and am afraid to use DataMapper for fear it won't play nice with a lot of the popular ruby goodies out there. I've modeled most of the project now in Django and i'm starting to play with Tastypie for the REST component. It feels too kludgy.
I'm a wreck. Mommy?
- ihsw 14y agoOne of the main sticking point for serious mongodb infrastructure is the global lock -- yes, write operations cause the entire mongodb instance (every database, every read operation) to block. This is clearly one of the main issues surrounding high-concurrent mongodb development. Thankfully with mongodb-2.2 there is now instead a database lock which mitigates the issue. Personally I feel redis more closely resembles language primitives with simple key-value pairs, hashes, lists (ordered, repeating values allowed), sets (un-ordered, no repeating values), and sorted sets (more interesting).
- taligent 14y agoRedis has no clustering (yet) so is best used when your data is transient and you are able to repopulate.
- j-kidd 14y agoI have no experience with Redis or MongoDB, but your statement is only true if the storage layer is transient. Some of us use servers that don't go away upon shutdown, backed by a SAN.
- mh- 14y agoHave you tested how long it would take your redis instance to be usable again after it finishes replaying the commitlog (AOF)?
- deleted 14y ago[deleted]
- WALoeIII 14y agoYou are thinking too hard. Just pick one, you can always change later. All of the options you have presented are "Very Good" - they do so many things right. To steal a pithy quote from my co-worker: "Its a balloon squeeze." This decision will not determine the success or failure of your business, move along. I pick Rails & Ruby.
- deleted 14y ago[deleted]
- chanks 14y agoI strongly recommend Sequel for Ruby: http://sequel.rubyforge.org http://sequel.rubyforge.org It provides an ActiveModel-compliant interface, so it works fine with Rails. It also has a lot less magic (associations don't use proxy objects, for example) and has a lot of features that ActiveRecord is sorely missing (such as complete support for composite primary keys). I don't know what your other problems with ActiveRecord are, but Sequel is definitely worth a look. I'm very happy with it.
- jameskilton 14y agoIMO you're worrying about the wrong thing right now. Build your API and application without any backend persistence. Make sure your API works. This will result in a nice data API for your backend that you can then plug into various tools. Use this to test and play with and empirically figure out what the best persistence layer is for your app. There's no possible way you know at the beginning of the app what persistence is best for your app, so don't make that decision yet.
- whalesalad 14y agoI agree with your approach. I feel though that the angle to take would be something like mongo... where I do not need to worry about a schema/migrations etc... and rather focus on the API logic itself.
- lusr 14y agoI'm a big fan of relational databases. The problem with them is the high upfront cost of developing a new product: maintaining your schemas, updating data types, getting bogged down in data mapping code and worrying about what the most efficient SQL implementation is going to be, etc. I like the fact I can whip something up quickly with MongoDB, get everything working and then once my schemas and services operating thereupon have stabilised then I can start looking at swapping my implementations out for a SQL-based back-end if I think it's worthwhile.
- brown9-2 14y agohow about Flask and SQLAlchemy?
- psgibbs 14y agoI'm surprised this is buried by people suggesting Rails or Postgres or what have you. SQLAlchemy as good a SQL ORM as there is, and it isn't married to a framework, or a SQL backend. If you like Python and want to use a SQL backend, SQLAlchemy is a good place to start.
- zzzeek 14y agohe says he loves the Django ORM. This is typically predictive of a poor response to SQLAlchemy's model :)
- whalesalad 14y agoI've looked at SQL Alchemy and think it's pretty radical actually. I think that this evening/weekend I am actually going to begin doing this in flask+sqlalchemy. It was a strong contender from the beginning. Here is my first flask app ever, https://github.com/whalesalad/arbesko-files https://github.com/whalesalad/arbesko-files No authentication or session management or anything above basic request/response, but it was fun and serves it's purpose inside of Arbesko well.
- slurgfest 14y agoSQLAlchemy is great, but you cannot say that it is simple :)
- taligent 14y agoThis idea that MongoDB is dangerous is just crazy and seems to often come from the same people who endlessly tout PostgreSQL as the world's most perfect database. The fact is that very large companies are using MongoDB in production environments. Yes. MongoDB still has some issues in particular the write locks but the situation is improving pretty rapidly and almost exclusively affects very high end users. So the answer to your question is simple. Use what matches your data model. MySQL if your data is more relational as it is proven and well known. MongoDB is perfect when you treat it like a document store. For example I fetch one User object (with all the user's data self contained) from the database and do everything client side. So in my case MongoDB works perfect. But either way. Do whatever gets you up to speed the fastest.
- Avshalom 14y ago> The fact is that very large companies are using MongoDB in production environments. Not as a criticism of Mongo but the thing about very large companies is that they can devote a team to just making sure some bit of technology works. Almost anything is suitable for a large production environment if you can afford to have a couple people elbows deep in it tuning and troubleshooting.
- deleted 14y ago[deleted]
- rdtsc 14y ago> This idea that MongoDB is dangerous is just crazy The idea that a database was shipping with a mode where it would return after a write without waiting for the result to be written to disk. Or a database where silent data corruption is an easy possibility and yet still called itself a database is what is "crazy"
- taligent 14y agoFirstly that mode is completely valid for many use cases e.g. logging, metrics. Secondly all the drivers handle this behaviour behind the scenes for you. Thirdly it is trivial to set the write concern to specify how many instances of the replica set you want the write to persist to as well. And silent data corruption is an easy possibility ?
- Argorak 14y agoI don't know what kind of project it is - but have you actually considered just using Postgres? Its not hip and takes a bit to setup, but after you are done, you have reliable workhorse with all the stuff you need (access control, schemas, etc. pp.) It has a vast amount of documentation and reading about its advanced features never gets old. There is a lot of talent around that knows how to use it. If you still consider Ruby but don't like ActiveRecord magic, don't forget to have a look at Sequel. I would recommend against DataMapper. It is nicely constructed, but has some weird issues. Also, DataMapper 1 is going to be replaced by DM 2, which is a completely different beast (but better, hopefully). Don't be fooled by ease of setup of the DB too much. It should be rather straight-forward, but even if a DB can just be downloaded and ran to dabble with it, always remember that this just shifts the moment at which you really have to dig into the details. You don't want to do that 2 days before going to production. Be on the lookout for red flags, though: unreasonable and weird configurations that have to be flipped for no understandable reasons at all before going into production. That shows that the project is not maintained very well or the developers have lost track. Also try to figure out how long it takes you to solve a new problem with the database, with documentation reading and all. Pick the one that you can wrap your head around best.
- taligent 14y agoEasy of setup matters. Because unless you are using a hosted database then you will be managing and maintaing that database day to day. Which is why I would never recommend PostgreSQL for startups. It is far, far too convoluted for many of the basic tasks you will be doing everyday. That's why everyone chooses MySQL as a relational database. It is easy to setup, easy to maintain and EVERY problem has been solved and searchable online.
- whalesalad 14y agoWait... what?! I don't see any major differences between MySQL and PostgreSQL in terms of setup/maintenance. They have slightly different authorization styles, but Postgres can be configured to behave just like mysql.
- Argorak 14y ago
- izak30 14y agoTastypie developer checking in. What's don't you like about it? Personally I like how it separates resources from ModelResources and lays out your API (that's why I got involved with the project in the first place).
- hasenj 14y agoI sincerely advice against using node. (I'll get to that in a bit). For the DB, I personally like CouchDB. It's easy to get started with, and it lends itself very well to horizontal scaling. As for why node is a bad choice: you can think of 2 categories of languages: 1) Those which allow you to get things done quickly but in which large code bases are hard to maintain because you need to carry too much information in your head, and they possibly don't scale very well because they tend to run in an interpreter (python, ruby, lisp). 2) Those which require some amount of boilerplate to get simple things done, but provide structure and modularity so it's easier to maintain large code bases, and they possibly produce high-performance code because they tend to be compiled or almost-compiled (C#, Java, Go). There's a trade off between "getting up to speed quickly" and "maintainable performant code". Node is bad because it fails at getting you up to speed quickly AND it doesn't make your code maintainable at large scale. This is because async-style programming is not natural to how we think. The "performance" should not be taken as a plus for Node because you can get it in other languages/platforms (Go, for example) without sacrificing maintainability.
- aaronblohowiak 14y agoThe node debugging experience is quite nice. Unfortunately, I found myself having to use it. The issues with carrying around information in your head is true IFF you have a large monolithic app. SOA, which is advocated with compiled languages as well, helps to avoid this issue because each service is limited in scope. Then again, SOA itself is counter to "hello world" latency... I think this is important because if you find yourself dealing with a (example) rails app that is experiencing cognitive burden overload issues, splitting it out into a SOA can be a nice transition (eventually replacing parts with other languages if it is required.) Also, if you care about performance and quick execution and want a scripting environment, luvit is much faster than node, much lower memory usage and IIRC you can use lua continuations instead of async style. IMHO, node.js as an edge server providing connection handling and templating with backend services giving it JSON seems like a reasonable 3-tier architecture, especially because your edge tier can then be maintained by people with front-end experience.
- hasenj 14y ago
- rhizome 14y agoIf you have no basis upon which to decide what database to use, you go with the one with the most domain expertise floating around. This means you use the most popular one, or the one that your tech people have used before. Conceptualizers are the last people on the decision list here, sorry, and there's nothing wrong with ActiveRecord outside of academe. tl;dr: Reflect upon your assumptions before making a decision more complicated than it might need to be.
- JulianMorrison 14y agoIf you are thinking of them like they are interchangeable, you need to stop, use a SQL database and quit messing. Seriously, they are not at all the same category of thing. Mongo is an update-in-place structure store with powerful atomic operations. Mongo will not grow storage if you are just changing the value of existing fields. Couch is a compare-and-swap document snapshot store with multi-master replication. In replicated mode it's eventually-consistent with manual conflict resolution. Riak is an indexable, eventually consistent key-value store with tuneable tradeoffs for speed versus safety, and no central point of failure. Cassandra is a horizontally scalable, eventually consistent key-value column hybrid designed for huge data. And so on... they are not "databases", they are not something where you replace one with the other, they are specific tools for specific tasks, usually much more so than unsexy but adaptable SQL.
- e12e 14y agoJust a quick note -- if you're working in python, I recommend having a look at zodb[1]. Then you won't need an ORM, because it is an actual object database. The main caveat with zope and zodb is that the move to python3/pypy is very slow, and not highly prioritized. Another, real world, object database is gemstone -- and can be used with smalltalk or ruby (among other things)[2]. [1] See eg: http://www.ibm.com/developerworks/aix/library/au-zodb/ http://www.ibm.com/developerworks/aix/library/au-zodb/ and/or: http://www.zodb.org/zodbbook/installing.html http://www.zodb.org/zodbbook/installing.html [2] http://maglev.github.com/ http://maglev.github.com/ and http://seaside.gemstone.com/ http://seaside.gemstone.com/
- nodeekee 14y agoIf the main thing you like about Django is ORM yet are considering a DB layer that doesn't require an ORM (like mongo) - have you looked at Pyramid (previously Pylons)? Besides giving you only the tools you want, it works very well with a driver like PyMongo. If you're starting out small you can also get free developer hosting with MongoHQ to get you up and running quickly. Oh and you get to keep doing things in Python which is very nice. I've also had no problems with MongoDB for my web needs and I've worked at a few places that use it without the scary data corruption stories told to developers in order to make them keep drinking their Java.