9 ms·
Great to see the Pick database reinvented bit by bit. Takes me back to 1980. Forget the past, doomed to repeat, etc.
by gregjor 14y ago
Great to see the Pick database reinvented bit by bit. Takes me back to 1980. Forget the past, doomed to repeat, etc.
- julien_c 14y agoCan you elaborate for those of use who weren't around in 1980?
- tomsthumb 14y agoOr 1990.....
- shin_lao 14y agoBasically the guys from MongoDB are rebuilding databases as they existed in the 1980s with the motto "we will do better than relational databases!" or "Sybase, Oracle, prepare to die!". It's not clear which problem MongoDB is trying to solve or if it is an improvement over existing technology (this is my personal opinion).
- base698 14y agoI think it's pretty clear to anyone who's used it. It allows for rapid, low overhead changes to your data model in the very early stages of building something new. Of course, the further along you get those changes are no longer low overhead, but at the start of the new project it's very easy to get up and running.
- gtaylor 14y agoI guess I just don't find schema that big a deal even in the early goings. A relational DB schema isn't necessarily hard to change and adapt on the fly. I don't feel like this is the best thing for Mongo to hang its hat on, and I'm not sure they really are. I think its users tend to point at this as a big feature more than they should. I feel like the other reasons to use Mongo should show up more in these discussions than this "It's easy to make schema changes". Perhaps these are easier scaling/sharding, I don't know, I don't use Mongo. I just see the schema part of this as a sidepoint (albeit an important one from an application design perspective). There are legitimate cases where supremely simple schema flexibility is desirable, but I suspect a lot of people who think they need this really don't (there are easy ways to do this with relational DBs), and are in fact making some things more complicated as a result. While it IS very easy to change your schema with Mongo, let's not forget that you're going to need to enforce certain schema-related rules in your codebase instead of the DB now.
- pkulak 14y agoAt the start of a project it's also really easy to change an SQL schema.
- gtaylor 14y agoThis. I don't understand why this isn't mentioned more often. Developer convenience and productivity are important things, but the future integrity of your application shouldn't be compromised for either. There are legitimate usage cases for a non-relational document store like MongoDB, but I suspect there are a non-trivial number of people using it that would be better served by a relational DB. Or even something like Postgres that has both relational and non-relational capabilities.
- base698 14y agoIt's not easier than nothing. You don't have to do ANYTHING in mongo. In any reasonable sized web app with a well designed ERD changing is going to result in you stopping coding and doing some ALTER TABLE, DROP INDEX, etc commands before you can start back with the coding. Also when you want to create a report you can just write a script aggregates the data and fills another collection on the fly without creating the table first. You can even try out 6 different reports at the same time, without issuing separate create table statements.
- gregjor 14y agoQED.
- gregjor 14y agoIt's easy to change a relational schema if you understand relational databases. If you don't then Mongo et al. look like good ideas because they solve an actual developer problem: ignorance.
- rimantas 14y agoMaking problem less visible (for a while) is hardly a solution.
- deleted 14y ago[deleted]
- themgt 14y agoWhich RDBMS can I use to do capped collection with entries streaming live in and out, supported in the client libs with a few lines of code?
- ivanb 14y agoPostgresql http://stackoverflow.com/questions/10896699/mongodb-capped-collection-equivalent-in-postgresql http://stackoverflow.com/questions/10896699/mongodb-capped-c...
- tracker1 14y agoWhat comes into play for me, is that if you are working with data, that in a traditional RDBMS would be split across several tables, where the majority of said data really isn't queried on, or even used other than as a whole, MongoDB works out better. Example, where I work we have classified ads for cars... having a structure for a listing.. { listinginfo1:...,listinginfoN:..., vehicleinfo: {...}, sellerinfo: {...} } The above example sees sellerinfo as a duplicate, but the listing/vehicle info would be broken up into several tables for an ACID/SQL database, and is mostly read. Querying against a single dataset, returning singular results shaped how you need them in the application has huge performance advantages. (Note: the old database still used for edits/updates, then exported to mongo for search/display requires over 20 joins, and +N calls to get the shape of data for a search results display. I created a view with all the base data for easier export, it's too slow to actually search against (there was a "search" table updated every half hour that was being used), and the single record for each listing in MongoDB fits better, and searches as fast as the old version on a smaller server. Another example, is when you have to pull data from many different sources, with similar shapes.. for example tracking health issues, and chemical exposure data... the core data who/what/where would be similar, but additional details may vary. Allowing for this dynamic shaping of data is difficult in most RDBMS systems. Though Postgres does have inherited tables, you would still want your structure broken out, or too flat for proper structure in an SQL db. That doesn't even go into the option in support of sharding, and replication that's far easier to configure and use than what you get with other DBs, let alone for the free version, without commercial addons. Is it perfect, absolutely not.. is it the best solution for everything, no. But the above are just a couple of examples where a non-relational database can be a better fit than SQL.
- shin_lao 14y agoIt was my understanding that MongoDB had poor scalability.
- tracker1 14y agoIt depends on your data, and how you are using it... Worst case, it should perform similar to SQL or slightly worse. If you can reliably shard your data, and don't need result sets larger than 16MB (not sure if they raised this in 2.4), you can do pretty well. Again, it depends on your data, structure and use. In some cases a traditional RDBMS is a better call. In others, for mongo, having an additional collection with your indexed subset may be the best approach. It really does depend. I don't consider it best for all cases, but do consider MongoDB a really good fit for a lot of web based cases.
- mindcrime 14y agoThe Wikipedia article on Pick[1] has some good background info. This stuff is pretty "old school" but is still in use in places. Funny thing... I did an A.S. in Computer Programming back in the early 90's and the community college I attended had a Prime mainframe in the IT department, and ran Pick databases. I did some "work study" stuff with the IT manager there, but never really had a chance to get deeply into the Pick stuff. I was always curious about it though, so seeing this story (and this comment) brings back memories... [1]: http://en.wikipedia.org/wiki/Pick_operating_system http://en.wikipedia.org/wiki/Pick_operating_system
- redwood 14y agoReminds me of Bill Gates recent IAMA where he talked about how they were close but didn't make it to market with a similar product, e.g. he seemed to be saying "MSFT was there first...!" I guess not, if this has been around since 1980 :) http://www.reddit.com/r/IAmA/comments/18bhme/im_bill_gates_cochair_of_the_bill_melinda_gates/c8dbamx http://www.reddit.com/r/IAmA/comments/18bhme/im_bill_gates_c...
- gregjor 14y agoPick-style databases were a dominant technology from around 1973 until relational databases (led by Oracle) marginalized them, more or less. Pick databases are still widely used and I run into legacy applications all the time, especially in publishing. There's a decent Wikipedia article on the Pick operating system if you're interested.
- spf13 14y agoWhile MongoDB was certainly built upon the shoulders of the giants that went before, so has been every product ever. To call it a reinvention of the Pick Database is flattering but highly inaccurate. Perhaps more accurate is learn from the past, improve the future.
- gregjor 14y agoMy point is that the Pick OS was stable, fast, and very good at maintaining data integrity. It ultimately lost favor (to relational database systems) because typical Pick applications had all of the database logic in application code, and that logic ended up all over the place. Simple things like enforcing referential integrity, transactions, and so on. RDBMSs do that in the database, where it belongs. MongoDB and NoSQL are not new, they are backwards steps into the pre-relational world. They seem new to anyone who doesn't know how database management was done pre-MySQL. That isn't to say that everything has to be in a relational database, or that there's never a use case for a non-relational data store like key/value or flat file, but at least recognize that those are not new ideas -- they are the prehistory of database management.
- ssmoot 14y agoRelational Databases are great, until you have downtime. PostgreSQL has the slowest database restore procedure the world has ever seen. So backups are basically useless for anything bigger than "trivial". Master/Slave replication is death for HA. MySQL has Master/Master, and is actually reasonably supported "in the cloud", so that's probably the only realistic option if you don't want to run your own hardware, need HA, and don't want to pay Oracle or MS. AFAIK Oracle and MS both have decent HA solutions, but you're probably looking at $100K starting prices, and again, you're probably going to have to run your own hardware. My #1 database of any type would be PostgreSQL. If it was HA. And if you could run a decent sized deployment (hundreds of GB) without having to run your own hardware. I think this more than anything is probably why NoSQL is so popular with this (HN) crowd. Because in-Memory "free" databases are a good fit for people who don't have $100K to throw at a vendor in licensing, and who don't want to lease their own hardware. RDBMSs are way behind the curve on that. Even for people like me who genuinely like them. Unless you're willing to go with MySQL I guess. But even then, do you trust RDS to be dependable and fast? Without a dedicated DBA? I can goto Cloudant and get a database that's very fast, has relatively modest costs, I don't have to run my own hardware, and BigCouch (derivative) has a far better HA story than any RDBMS I'm aware of outside of maybe VoltDB (which I don't have any practical experience with). I make some compromises on ACID, which I'm learning to be OK with. I lose transactions (which really hurts some things, but rolling your own 2PC on a case-by-case basis works passably when you really need the semantics). And I have a mostly-pleasant, sometimes very fragmented set of tools to help me work with my data. It's a living. But I'm not responsible for the hardware. It's much cheaper. And it does HA. When PostgreSQL steps up and gets that right, I'll switch back. Not to mention, there's no excuse for the amount of tuning a decent PostgreSQL install takes. Speaking as a guy who's built several 80K-TPS systems, the barrier to entry is needlessly high. Try MS-SQL sometime and contrast. Then again, PostgreSQL is free. Plus the free RDBMSs need enforced clustered indexes. This crontab stuff is for chumps. 99% of the time an enforced cluster would be preferable to increased insert performance for 99% of web-apps. /end-long-rambling-rantish-comment. :)
- newishuser 14y agoYeah, your right, MongoHQ hasn't brought anything positive to web development and there's no way they're going to make a business supporting their db. It definitely needs to be put in it's place and we should all take the time to talk about just how cool we are for predicting it's downfall. If only the whole world had enough insight to just stop before they tried to do something awesome. After all, someone may have tried it before.
- fusiongyro 14y agoI suppose answering snark with snark is fair play, but our industry has a real unhealthy preoccupation with the new. The early releases of MongoDB were great proof that the engineers had failed to learn many of the lessons of their predecessors, and I think this is why I and other database-y programmers distrust it to this day. Sexiness is not a prerequisite for reliability. Your unspoken message reflects the widespread idea in the industry that a detailed knowledge of old things is incompatible with doing new and exciting things. I think this notion is naive and also contributes to ageism, and I hope our industry recovers from it in my lifetime.
- newishuser 14y agoI feel like you're more concerned with making sure people understand where they sit on the totem pole than encouraging progress or exciting developers. While learning from history is certainly paramount in software development, raw experimentation, naive excitement, and continuing in the face of nay-sayers is what pushes all industries forward. People are allowed to make mistakes, tons of them, and the software industry is one of the best industries to make mistakes in. You get quality feedback almost instantly and can fail faster than anywhere else. No one is claiming that "detailed knowledge of old things" is on it's way out. We are engulfed in systems and code that are decades old. I don't see the point in nay-saying.
- fennecfoxen 14y agoYou have a point. But we can meaningfully compare and contrast newer database technologies as well. I think it's pretty clear that Mongo is the MySQL of NoSQL, and I do mean that in the most pejorative way imaginable. Architecture decisions like "let the kernel do all the swapping" decision, a casual approach to durability, the very mindset that leads to inclusion of SSL without certificate validation... ick. They'll gradually polish their stuff up so that it's quasi-usable and maybe make lots of money, and good for their pocketbooks, but there are plenty of alternative data stores out there which have characteristics which render them more suitable for storing data for most if not all projects.
- eddieroger 14y agoMy understanding of Pick databases was that they were typeless. Mongo isn't. There's a difference between being typeless and schemaless, and Mongo is a good mix for my usecases.
- gregjor 14y agoYou don't understand Pick databases very well then. They supported types and could enforce them. Some Pick-style databases were weaker in this regard than others but there's nothing inherently typeless about the Pick database. I wouldn't call Pick databases schema-less, either. Pick databases called their schema a dictionary. In a Pick database the dictionary is less strictly enforced than in a SQL database but it's not schema-less.
- gregjor 14y agoAnd before anyone goes off, yes, I know that Pick systems usually stored everything as a string. That was done for reasons that made sense in the 1970s, and because the Pick OS was implemented on several different hardware architectures. That isn't the same as type-less though it looks that way at first glance. Although a number might be stored as a string of digits on disk the dictionary allowed for strict validation on the way in (Input CONVersions) and formatting on the way out (Output CONVersions). In my extensive experience with Pick OS I found that most programmers preferred doing that stuff in application code, the features of the dictionary were not much used, and as a result Pick code was harder to work with and maintain than necessary. I find the same thing when working with SQL databases -- instead of putting validation and enforcement in the database schema the rules are enforced in application code, usually in more than one place.