40 ms·

Amazon DocumentDB, with MongoDB compatibility

519 points by ifcologne 8y ago

vaer-k 8y ago
Finally an AWS service where the name makes sense and describes what it is. I hope this is the start of a trend.
- rjbwork 8y ago
  I love Azure for this. The names are almost all extremely straightforward. There are a handful that have made the jump from confusing to straight forward, and a handful that have made the jump from straightforward to confusing (CosmosDB, formerly DocumentDB, chiefly comes to mind).
  - Someone1234 8y ago
    Agreed. Too bad the Azure portal is the polar opposite. AWS, for all its faults, is mostly just a boring HTML portal but it works. Azure tried to get fancy, with side sliding panels all over the place, and it is barely useable. The nicest thing I can say is it is "quirky." It isn't really productive however, particularly not on my 1080p monitor at Windows 10's default 125% DPI. I literally quit Azure's Application Insights and went back to Google Analytics simply because I hated the Azure UI with a burning passion of a thousand suns. The concept of writing queries is good, but if that's the only way you can get at your data you better make it damn easy, and they didn't. I'm sure for full time data pros it is a dream however.
    - dharmab 8y ago
      Azure Portal feels like if someone tried to make the Xbox 360 blade interface into an admin tool, without first asking the admins what they needed.
    - rjbwork 8y ago
      That's interesting. I actually quite like it. I can build monitoring dashboards for our various services an see how something I don't need to monitor is doing just by going to the panel for it. To each his own I suppose.
  - aynsof 8y ago
    Except for the fact that Azure names seem to change once per year. Our Azure SA was giving us a presentation and actually got confused himself. "So that's TFS... I mean VSTS... Actually wait, it's Azure DevOps now?"
    - coredog64 8y ago
      Just wait until next year when it’s called TFSHub.
    - innocentoldguy 8y ago
      I worked at Microsoft for a while and I swear most of their "upgrades" are nothing more than renaming things and juggling menu items around so people can't find them.
  - titanix2 8y ago
    Yes the names are good but they still suffer for the MS illness: they change every few years. I also agree with other comments about the portal UI. Heck, it's supposed to be a professional tool...
- napsterbr 8y ago
  Surprised they didn't go with a 3-letter acronym. AWS DDB. Recently I made a typo on a formal document. Wrote "AMI" when I meant to to write "IAM". Oops.
  - tj-teej 8y ago
    DDB is typically used to denote DynamoDB
    - stevehawk 8y ago
      i honestly thought that was his joke
  - jkchu 8y ago
    I would have preferred AWS D2B.
    - Corrado 8y ago
      My brain keeps trying to parse this into either DB2 or some Star Wars reference (R2DB, RD2B, etc.)
- kerng 8y ago
  Yeah, they copied Azure basically. DocumentDB was the name of an Azure service in past, interestingly it offered MongoDB, Gremlin and other API gateway options. Its called Azure CosmosDB now.
notmyname 8y ago
seems an obvious response to https://www.mongodb.com/press/mongodb-issues-new-server-side-public-license-for-mongodb-community-server https://www.mongodb.com/press/mongodb-issues-new-server-side...
- dantiberian 8y ago
  I think it is likely it was the other way around. MongoDB caught wind of what AWS was about to release and changed the license.
  - Slippery_John 8y ago
    Does that even apply? This is api-compatible, but doesn't appear to be using any actual MongoDB code.
    - dantiberian 8y ago
      It seems like Amazon may think so, with this line: > Amazon DocumentDB implements the Apache 2.0 open source MongoDB 3.6 API by emulating the responses that a MongoDB client expects from a MongoDB server
- tootie 8y ago
  Azure has been running a Mongo-compliant DB under their Cosmos umbrella for quite a while. It's not clear to me that either Azure or AWS are actually running Mongo software under the hood or rather a proprietary DB that uses the Mongo wire protocol. https://docs.microsoft.com/en-us/azure/cosmos-db/mongodb-introduction https://docs.microsoft.com/en-us/azure/cosmos-db/mongodb-int...
  - Analemma_ 8y ago
    Doesn’t Azure also have a not-Mongo service also called DocumentDB? Is this the same code? These cloud services are confusing enough when they don’t borrow each other’s names.
jchw 8y ago
Seems like this is likely to be the real result of licenses like the SSPL. Not even a terrible outcome if the different implementations remain relatively compatible.
- wmf 8y ago
  SSPL style licenses only work if the software can't be cloned but that's a difficult assumption.
  - earenndil 8y ago
    Operating systems, compilers, and web browsers come to mind. There are currently: * 4 independently-developed competitive compilers (gcc, clang, msvc, icc) * 4 independently-developed competitive operating systems (windows, macos, linux, and bsd --I'm grouping the BSDs as one since their source code has a common ancestor) * 3 independently-developed competitive browser engines, soon-to-be 2 (edgehtml, gecko, webkit) And it's been that way for a few decades now; doesn't look like anyone is interested in taking the resources to make another one of those.
    - johncolanduoni 8y ago
      Throwing Chrome/Blink under WebKit is a pretty hard sell at this point. They’ve diverged enough that supporting one far from guarantees you’ll support the other. You might as well replace WebKit with KHTML in your list.
      - earenndil 8y ago
        For the purposes of licensing and simple inertia (what it takes to start a project from scratch) -- that work was done once, with khtml, sure.
- kemitchell 8y ago
  I’d be surprised to learn that AWS started and launched this project since the SSPL announcement. I suspect they began when latest Mongo was still AGPL, with no sign of impeding change.
deleted 8y ago
[deleted]
StreamBright 8y ago
I hope MongoDB's users are really keen on on-prem otherwise this might be a challenging moment for them.
- threeseed 8y ago
  Not sure what this means. MongoDB has had Atlas which is a cloud hosted solution for a while now.
  - thawkins 8y ago
    But you cant host your data inside your aws account with atlas, non starter for some
  - ec109685 8y ago
    Poster means that their cloud business could be impacted, but on-prem (non aws) would not be.
mark_l_watson 8y ago
Looks good. I used to use both MongoDB and CouchDB (for very different use cases) a lot but haven’t touched them in a long while.
- anonytrary 8y ago
  Out of curiosity, what use cases did you have for CouchDB that weren't feasible with MongoDB?
  - gorbypark 8y ago
    The best part of CouchDB is PouchDB, in my opinion. It's kinda cool having an in browser database that just syncs "almost effortlessly" with the cloud hosted version.
  - WorldMaker 8y ago
    In my case, at least, PouchDB is still a lot handier of an offline-first mobile-capable DB than just about anything else, and because it speaks the CouchDB replication/sync API that still leaves a lot of use cases where CouchDB is more feasible. (It would be great to have more document DBs converge on a replication/sync API for offline-first applications. I've voted on the UserVoice suggestions to CosmosDB on the idea.)
    - karmelapple 8y ago
      Same here. We love how replication is built in to CouchDB from the ground up. I wish MongoDB, which seems to still have more love for it in the world than CouchDB/Cloudant, would have that replication built-in.
      - metheus 8y ago
        Wait... do you mean something other than https://docs.mongodb.com/manual/replication/ https://docs.mongodb.com/manual/replication/ ?
anonytrary 8y ago
AWS lacked something like this before, which is why I used MongoDB Atlas (MongoDB's own managed mongo service). How is this different than DynamoDB (other than the obvious API differences)?
- dfischer 8y ago
  DynamoDB has specific requirements on how data is partitioned. You have to do upfront planning on how you access your data based on keys. Mongodb is more flexible in this regard. I’m sure there’s other big differences but this is the biggest to my experience with both.
  - anonytrary 8y ago
    I was asking about DocumentDB, not MongoDB, but maybe I missed something. From their site: > Amazon DocumentDB implements the Apache 2.0 open source MongoDB 3.6 API by emulating the responses that a MongoDB client expects from a MongoDB server, allowing you to use your existing MongoDB drivers and tools with Amazon DocumentDB So, is this a "managed MongoDB" or is it a NOSQL AmazonDB that just implements the same API as MongoDB? If the latter, I don't think we can assume this is the same thing as MongoDB, because the internals might be completely different.
    - Slippery_John 8y ago
      I would imagine this doesn't use any MongoDB code, or else the SSPL license would brutalize them. That said, I would be surprised if the experience is much worse.
    - k__ 8y ago
      Sounds like the latter to me, they probably implemented it on top of one of their other DBs.
      - anonytrary 8y ago
        Lmao I was a bit afraid this might be some surface-level API wrapper around DynamoDB, in which case this is less cool than I thought. It doesn't seem to be the case at a first glance but I wish it was a bit more clear on the site exactly how this relates to DynamoDB, if at all. Maybe I'll just have to read more about it. Hopefully the internal architecture gets away from DynamoDB-style partitioning.
whalesalad 8y ago
Serious question: is there any real reason to use Mongo over Elasticsearch?
- a13n 8y ago
  I run a startup that uses both of these. Mongo is a heck of a lot easier to configure and develop with, and works great as a general database with a rapidly changing schema. Elasticsearch is great at solving specific problems like searching for items in a specific way, but it's got quite a learning curve and is pretty painful to host and configure compared to Mongo where you can have a prod instance going in seconds.
- ajsharp 8y ago
  Yes. Elasticsearch is not nearly as easy to use and is not really designed to be a transactional database.
- alrs 8y ago
  Elasticsearch is a distributed index, not a reliable document store. You can expect availability problems and data loss. This is fine, because you can rebuild the index from your source of truth.
  - rafaelturk 8y ago
    True.
  - plasma 8y ago
    In my experience with ES, definitely don't treat it as a source of truth; allow it to be rebuilt.
- StreamBright 8y ago
  Is it a good idea to use a database instead of a search engine?
- throwaway204 8y ago
  ES grew out of Lucene, which provided an inverted index of all the text in a document, with a bunch of NLP related features bolted onto that. While ostensibly designed to be developer friendly, ES had and still has a horribly hacked together API with bugs and mis-documented misfeatures all over the place. In my experience it's anything but developer friendly. But if all you need is a text index on a document store with a static schema, it does its job reasonably well. Mongo started as a very developer-friendly data store with lots of overstated claims to being a database. While earlier versions of Mongo were wildly dangerous to use as a business critical database, it has since matured and is now quite good at being a developer-friendly document-centric database. In my experience Mongo truly is developer-friendly as long as you don't try to use it as a full-blown transactional database with lots of complex data shapes and indexes. I would not trust ES with anything but text search on a document store, and I would not trust Mongo with anything resembling multi-document transactions. With that said, they are both good at specific, different things. MySQL and Postgres have their own baggage that makes them pretty terrible in some aspects. IMHO a JSON-over-HTTP API should really be table stakes for a database to be considered developer-friendly nowadays. (But please don't butcher HTTP like ES did and then claim to have that.)
dstaley 8y ago
If I'm reading the pricing page correctly, DocumentDB would run a _minimum_ of $200/month. That's for the smallest instance and no storage or I/O. Kind of steep if you ask me.
- imaginenore 8y ago
  AWS pricing is so strange. They give you 25GB of DynamoDB for free and charge 25 cents per million reads (under 4K, i think). $200 minimum means almost no indie developer will even try it.
- a13n 8y ago
  Yeah, that's pricy. They're definitely not going after early-stage startups then. But if you have a medium-sized data set (eg. 50+ GB), this is definitely competitively priced. More RAM, storage, compute than Mongo Atlas and Compose for less money. Here's hoping they introduce cheaper options!
  - orf 8y ago
    50GB is a really small data set.
    - deleted 8y ago
      [deleted]
    - a13n 8y ago
      Eh, by what measure? Realistically it's probably bigger than 90% of all Mongo datasets. It's tiny if you're a massive company and it's massive if you're a tiny startup.
      - q3k 8y ago
        50GB easily fits in RAM. It's a small dataset.
      - orf 8y ago
        If you can run the dataset comfortably on a Macbook then it's very, very small. Heck, you can even just use grep over 50GB reading straight from disk. It's tiny.
misframer 8y ago
Sounds like this runs on the same storage service as Aurora.
- misframer 8y ago
  Not sure why I'm getting downvoted. The characteristics sound exactly like Aurora. - "replicates six copies of your data across three AWS Availability Zones (AZs)" [0] - "Amazon DocumentDB uses a distributed, fault-tolerant, self-healing storage system that auto-scales up to 64 TB per database cluster." [0] - "When writing to storage, Amazon DocumentDB only persists a write-ahead logs, and does not need to write full buffer page syncs." [1] [0] https://aws.amazon.com/documentdb/ https://aws.amazon.com/documentdb/ [1] https://aws.amazon.com/documentdb/faqs/ https://aws.amazon.com/documentdb/faqs/
  - MrTonyD 8y ago
    Yeah, downvoting is totally broken on Hacker News. Seems like anything that isn't immediately agreed-with gets downvoted. I don't know where all the small-minded people come from, but they seem to have found their home here.
    - ma2rten 8y ago
      There is just some random noise. It might just have gotten one downvote. Now it's the top-comment.
  - jkchu 8y ago
    I'm guessing if you would have included this reasoning in your original comment, then it wouldn't have been downvoted.
- ec109685 8y ago
  Good article on Aurora's design: https://www.allthingsdistributed.com/files/p1041-verbitski.pdf https://www.allthingsdistributed.com/files/p1041-verbitski.p...
orestes910 8y ago
Is it really THAT hard to to run on prem/cloud because of the "complexity that comes with setting up and managing MongoDB clusters at scale"? The documentation is strong, it was built with horizontal scalability in mind. I don't see the struggle.
- skywhopper 8y ago
  If you don't see the value, then no need to pay for it.
- uberduper 8y ago
  Yes. Sharded mongo has many pitfalls that can challenge your devs, dbas and devops people. Mongo api without all the mongo scaling problems is pretty awesome.
  - chx 8y ago
    I must have used a different API, then. Well, obviously not but it could happen because there are more than one ... I agree with https://www.linkedin.com/pulse/mongodb-frankenstein-monster-nosql-databases-john-de-goes/ https://www.linkedin.com/pulse/mongodb-frankenstein-monster-... the contents of this article (and reserve critique for its author). Even the basic query language, in particular $elemMatch is atrocious which has a certain irony to it because Eliot added it when we asked for the functionality (https://jira.mongodb.org/browse/SERVER-377 https://jira.mongodb.org/browse/SERVER-377), so long ago. We have been one of their first commercial support clients, my boss needed to pressure them to accept money for support, they didn't want to...
deleted 8y ago
[deleted]
ahachete 8y ago
My bet is that it is built on top of Aurora PostgreSQL. By looking at the "Limits" section (https://docs.aws.amazon.com/documentdb/latest/developerguide/limits.html https://docs.aws.amazon.com/documentdb/latest/developerguide...), identifiers are limited to 63 characters and the same characters that PostgreSQL limits identifiers to; and a collection size limit of 32TB, coincidentally maximum PostgreSQL table size. Edit: I can confirm: does not allow the UTF-8 null character in strings: https://docs.aws.amazon.com/documentdb/latest/developerguide/functional-differences.html https://docs.aws.amazon.com/documentdb/latest/developerguide... ... It is written on top of PostgreSQL.
- deleted 8y ago
  [deleted]
- misframer 8y ago
  I think they're built on a common storage system, just like the MySQL compatible version too.
- erikig 8y ago
  It would be nice if Amazon provided an API to access the data via SQL alongside the MongoDB API; I've seen quite a number of organizations migrate from mongo to Postgres once they get out of the rapid development phase. This would make that transition butter smooth.
  - ahachete 8y ago
    That would make the internal representation used an "API" and thus won't be able to change it in the future. Apparently, they are using a 1:1 mapping between a collection and a table. Either by flattening the document or by using jsonb or equivalent. I'm not a big believer this is good for performance reasons, at least compared to a more normalized approach like the one we did for https://www.torodb.com https://www.torodb.com But they may change it in the future --if they don't expose the SQL API to their internal representation.
  - scarface74 8y ago
    Anecdote: I led a C# project where we could seamlessly switch back and forth between Mongo and SQL Server without changing the underlying LINQ expressions. We sent the expressions to the Mongo driver and they got translated to MongoQuery we sent the expressions to Entity Framework and they got translated to Sql Server.
deleted 8y ago
[deleted]
openloop 8y ago
https://www.rackspace.com/en-us/cloud/files https://www.rackspace.com/en-us/cloud/files
abrookewood 8y ago
I was reading a post [0] by Brian Cantrill that predicted this would be the result of licences like the SSPL. I instinctively disagreed with him, but it turns out he was right: "The cloud services providers are currently reproprietarizing all of computing — they are making their own CPUs for crying out loud! — reimplementing the bits of your software that they need in the name of the service that their customers want (and will pay for!) won’t even move the needle in terms of their effort." [0] http://dtrace.org/blogs/bmc/2018/12/14/open-source-confronts-its-midlife-crisis/ http://dtrace.org/blogs/bmc/2018/12/14/open-source-confronts...
- dman 8y ago
  What is the way out? Would love to hear from people.
  - bilbo0s 8y ago
    Not sure this would fall under SSPL in any case. It's clear that what Amazon is doing is using Postgres under the hood, not really mongo. So I'm not sure how that would work if you make an interface shim to make postgres look like mongo, are you then subject to the mongo license? the postgres license? the apache 2.0 mongo api license? all of them? what if clauses of them are mutually exclusive? etc etc etc. Just at a cursory glance it certainly seems like only the apache 2.0 mongo api license would apply. But I guess mongo could try to force the sspl on amazon?
  - SpicyLemonZest 8y ago
    There's no secret formula to stop people from competing with you. If MongoDB Inc is successful, it should be because they run a good document-database-as-a-service people want to use, not because they earn indefinite seigniorage from launching a popular open source project.
    - KaoruAoiShiho 8y ago
      But isn't it wrong to place all economic value in the hosting layer rather than the software layer?
luhn 8y ago
Why is it so expensive? Not only is the entry point $200/m, the instances are twice the price of their EC2 equivalents. (At first I thought perhaps the price included multi-AZ, but it doesn't.)
ISV_Damocles 8y ago
Now I kinda hope Oracle decides to buy out MongoDB and integrate it into their own cloud. Then Oracle can decide to pull the same bullshit that they did with Google over the Java APIs with the MongoDB APIs but now against their current enemy Amazon (and Microsoft, too). Then a combined Google + Amazon + Microsoft may finally be able to reverse the API Copyright insanity that is hovering ominously over the tech industry, and Oracle can continue to be a shining city upon a hill of shitty technologies you should never allow your business to adopt.
- wmf 8y ago
  AWS is preemptively defensive about API licensing claims: "Amazon DocumentDB implements the Apache 2.0 open source MongoDB 3.6 API".
  - christkv 8y ago
    I think they are referencing the drivers which are licensed under Apache 2.0.
- giorgioz 8y ago
  I've always seen Google+Android as the good guys that gave Java new life while I saw Oracle has the bad guys that bought Sun and killed Java.
XorNot 8y ago
I'm pretty sure this is going to kill Mongo as a company dead. With this in existence there's literally no reason to use Atlas. If they wanted to twist the knife they should get to work implementing a pass through migration option.
- _wmd 8y ago
  MongoDB still have a strong hand - control of the client and particularly its exposed featureset - due to that, also control of the protocol and the ability to, for example, insert legally protected strings in the style of the Apple SMC signature into the handshake. - ability to gate new features on the presence of an object like a copyrighted text, trademark, or even a crypto signature - ownership of the name. AWS are pissing in Mongo's pool marketing themselves as compatible, and there are a variety of ways it could be made to backfire, if it were in Mongo's interests to encourage that outcome - AWS focuses on breadth and very rarely nails any particular service. Their hosted Postgres for example still does not expose core features years later - Following from that, AWS services on the whole are rarely best-in-class in terms of raw performance. I imagine Mongo could continue to easily compete on benchmark results running on AWS own infrastructure I think this is a really interesting case, far more interesting than the technical minutia of Just Yet Another AWS service. It does not sit well with me whatsoever that they're basically ripping off a much smaller company's core tech while simultaneously borrowing their trademark (in a legally acceptable manner) as part of the marketing, but I also find it hard not to see a ton of potential upside from this for Mongo
  - ec109685 8y ago
    The client change would never work. The client is licensed as lgpl, so if they tried to pull any funny business like that, it would be instantly forked and if’d out.
    - _wmd 8y ago
      As the person suggesting it, it's difficult to imagine how it could never work considering I haven't managed to figure out all the possible combinations in which such a strategy could be applied. Finally, it is quite exasperating to call this kind of strategy "funny business" in a thread about their core tech being ripped off by a megacorp
outworlder 8y ago
How the heck was this approved by legal? AWS is linking to MongoDB's own documentation. https://docs.aws.amazon.com/documentdb/latest/developerguide/mongo-apis.html https://docs.aws.amazon.com/documentdb/latest/developerguide... Go to any operation, it links to docs.mongodb.com !
- chrisseaton 8y ago
  Is linking to documentation illegal?
  - outworlder 8y ago
    The 'legal' department doesn't only vet things that are obviously illegal, they are there for other things too, like EULAs, contracts and intellectual property in general. In any case, they are implementing the competitor's APIs and the documentation is already licensed as Creative Commons. Can't they at least host their own copy?
    - deleted 8y ago
      [deleted]
    - deleted 8y ago
      [deleted]
  - deleted 8y ago
    [deleted]
- manigandham 8y ago
  What is there to approve? Links are public.
- kerng 8y ago
  Nothing novel here by Amazon. Azure had MongoDB gateways (and many others) for their NoSQL offering.
koolba 8y ago
Jeff Bezos continues his quest to eat all open lunch boxes.
- ec109685 8y ago
  He isn’t using any of their code though.
- 0xFFFF0000 8y ago
  Microsoft was first with a MongoDB layer, like some 3 years ago I think. Nothing novel here from Amazon.
manigandham 8y ago
So now AWS DocumentDB, Azure CosmosDB, and even Apple's FoundationDB have a MongoDB compatible API. I expect other multimodal databases to offer the same soon enough. Strange turn of events for MongoDB but I guess that's what happens when the interface is open and anyone can build a backend to it, especially a relatively simple document-store.
- ec109685 8y ago
  And when they have a restrictive license.
markwolfe 8y ago
This looks great for teams getting started in AWS, being able to reuse idioms, libraries and knowledge in managed services to remove some of the operational load. A note this is also nothing new, Azure Cosmos DB has had this for a while.
0xfffff 8y ago
Wasn't DocumentDB the old name of Azure Cosmos DB? just marketing wise this seems like a bad choice for a name.
miaklesp 8y ago
Of all possible names they managed to choose already obsolete name for a similar product from Azure? Azure in 2017: forget DocumentDB, Cosmos DB is the new thing now! AWS in 2019: We just released DocumentDB! Are they so geniuses or so idiots?
- 0xFFFF0000 8y ago
  I had the same observation (although I'd reconsider your wording, that's not very polite). It doesn't seem like a good choice in naming - especially because many of the Data Platform folks at Amazon (like their VP) came from Microsoft's data platform group.
- miaklesp 8y ago
  UPDATE: They are idiots.
morpheuskafka 8y ago
I really wish this was serverless. Azure CosmosDB offers SQL and MariaDB interface against a serverless, pay for what you use database and DynamoDB is the only product of that class Amazon has. Even Aurora "serverless" appears to be little more than autoscale, and it requires an elastic IP which slows launch of lambdas since they have to connect to VPC.
- iamed2 8y ago
  > it requires an elastic IP which slows launch of lambdas since they have to connect to VPC Do you have any other info/links related to this?
  - gazzini 8y ago
    There's an enlightening graph in this post: https://medium.freecodecamp.org/lambda-vpc-cold-starts-a-latency-killer-5408323278dd https://medium.freecodecamp.org/lambda-vpc-cold-starts-a-lat... I was surprised to learn this. When working in Lambda, you have to choose between a relational database & a responsive API. It seems inevitable that AWS will fix this soon, but apparently this is a significant architectural problem. As I understand it, RDS instances should (must?) be accessed from within a VPC, and anything inside of a VPC needs an IP address, so the Lambda function has to wait ~10 seconds on cold-start for the Elastic IP service. The only workaround I've heard of is to setup a service, such as CloudWatch, to call your Lambda function every ~25 secs to keep it "warm", but this seems anti-thetical to the value proposition of serverless architecture in the first place. Of course, you could "just" use DyanmoDB, but IMO the query language is really limited, and I'm not sure I fully grasp the problem (why doesn't DynamoDB need to be accessed from within a VPC?)
    - athrun 8y ago
      > I'm not sure I fully grasp the problem (why doesn't DynamoDB need to be accessed from within a VPC?) This is due to the fact that DynamoDB's query API is a standard AWS API which means granular internal/external access can be provided through IAM mechanisms (ie: roles, temporary tokens, federation, etc.). On the contrary, to access RDS, Redshift or DocumentDB you would use standard ODBC/JDBC/Mongo facilities, which do not rely on IAM mechanisms, leaving VPC/Security Groups as the only isolation option.
KaoruAoiShiho 8y ago
Can someone do a comparison on pricing vs Atlas and vs DynamoDB and Amazon's managed postgresql?
social_quotient 8y ago
I really wish this were priced more along the lines of https://www.compose.com/pricing https://www.compose.com/pricing - a $200/m floor is a tough dB cost to absorb on smaller yet important projects. Suppose an app has a few mb of data and maybe one day hits 100mb of awesomeness I really have to pay $200/m here? I get it, I love Aws, just wish this was priced differently.
- illumin8 8y ago
  If you need a cheap/free document store for a small app, just use DynamoDB. It's free (forever, not just the first year) up to 25GB of storage and enough read/write capacity units to handle up to 200M requests per month: https://aws.amazon.com/free/?awsf.Free%20Tier%20Types=categories%23alwaysfree https://aws.amazon.com/free/?awsf.Free%20Tier%20Types=catego...
  - privateSFacct 8y ago
    Exactly, this probably isn't targeting you. A few mb of data? Host an instance? Use sqlite? Atlas can run 3-5k/month as the minimum. This is going to be 10x cheaper (which amazon seems to try and shoot for).
- Corrado 8y ago
  Another thought along these same lines is how do you host non-production (Dev, QA, Demo, etc.) environments without spending a fortune? Sure my production workload is 15TB but my development system is only running 15GB and I want to develop against what I deploy to.
  - GordonS 8y ago
    If you can create schemas, that could be one solution. I do this with Azure SQL Server instances, so a single instance can host all the 'non-critical' environments (dev, test, QA, demo) - works great!
0xFFFF0000 8y ago
Weird question: Could Microsoft sue Amazon here for infringing on the DocumentDB name? I mean Microsoft's DocumentDB was among the first to even have such a MongoDB layer also) and that was like 3 years ago. Given that current Amazon leaders actually came from Microsoft's data platform group this leaves a bit of a bad taste behind. I'm not working for either company.
- jon-wood 8y ago
  My assumption is that DocumentDB falls into a category of being so generic you can't trademark it, or otherwise claim exclusivity to it. Its literally just describing the fact this is a database for documents.
  - SmellyGeekBoy 8y ago
    Bear in mind we're talking about the company that trademarked "Word" and "Excel" here.
- amyjess 8y ago
  For all any of us know, Amazon's lawyers already talked to Microsoft's lawyers about it and got permission beforehand. See: Apple licensing the iOS name from Cisco before announcing the name change.
swoorup 8y ago
I cant use this crap locally?
talawahdotnet 8y ago
I am pretty frustrated that DB services like Aurora, and now DocumentDB are still limited to last-gen instance types like r4 instead of the latest instances like r5 and t3 which have marked improvements in terms of CPU and networking performance. I wonder if it is that they just have a so much r4 inventory left that they are forcing us to use it or if they haven't fully integrated/validated the latest instance types with their custom storage backend.
LoSboccacc 8y ago
> Text Index: No as MS Cosmo DB prior, the compatibility list leaves quite a bit out.
hendry 8y ago
Atlas is $0.03 per server hour, Amazon starts at $0.277 Do I have that right?
- manigandham 8y ago
  Starting prices maybe, but you have to compare hardware and performance capacity, not just flat numbers.
hguhghuff 8y ago
Is this serverless? Pricing?
sauravt 8y ago
How is it better than buying a MongoDb subscription from the AWS marketplace? https://aws.amazon.com/marketplace/search/results?x=0&y=0&searchTerms=mongodb https://aws.amazon.com/marketplace/search/results?x=0&y=0&se...
christkv 8y ago
There is a serious amount of weasel words in that statement. Implementing the Apache 2.0 API I think is just weasel words for "Works more or less with the MongoDB drivers" which is the only thing Apache 2.0 licensed that I know off and is obviously not the API which is the combination of the MongoDB wireprotocol and the commands supported by the MongoDB server.
buremba 8y ago
I don't understand why people are reacting to it so aggressively. That's basically how AWS works, they did the same to Apach Kafka with Kinesis, Prestodb with Athena, PostgreSQL and MySQL with Aurora, Redis with ElastiCache and many others over the last 4 years so it's not new. It took too long for the open-source community to figure out that the cloud providers are killing them, now it's too late. Well played, AWS.
- QuestionMarking 8y ago
  It's because nerds love to lose their shit any time a big company does anything.
- konschubert 8y ago
  Well, people are angry about that, that’s why they react aggressively.
  - buremba 8y ago
    Open-source gave AWS the ability to monetize their software so the software companies should be careful enough to prevent any other big company to steal their software and use their name to make money. I think that it's too late considering AWS already did that to most of the industries but here is Hazelcast's take: https://www.linkedin.com/pulse/open-source-needs-protect-itself-from-service-wrappers-greg-luck/ https://www.linkedin.com/pulse/open-source-needs-protect-its...
- driverdan 8y ago
  > It took too long for the open-source community to figure out that the cloud providers are killing them How are service providers killing FOSS? That doesn't make sense. Permissive FOSS licensing allows anyone to use their software, regardless of how it's used, and that's how it should be.
  - keepper 8y ago
    Do you get to see AWS's source code for these services? No... That's how it's killing "FOSS". Extend and Extinguish. This is not a new playbook.
borplk 8y ago
While we are on the topic, are there cloud providers out there that provide "hosted FoundationDB as a service"? And if you are curious, no I don't have any special need or use case for it, just wondering.
craig_peacock 8y ago
I would actually like to thank AWS for focusing on supporting Open APIs. Let the community design the standards and operations and let the vendors compete to implement them, open source included.
vemv 8y ago
I hope AWS realises that harming OSS authors, even indirectly, is counterproductive to AWS' interests. If Mongo dies, so does its userbase, and so does DocumentDB. How to fix it classily? Contribute back with code maintenance or whatever customizations were developed in-house. Also you get to save face - you are not perceived as a leech anymore.
jonplackett 8y ago
Shame there isn’t a free tier equivalent. I’d like to try some projects using mongo but $200 a month minimum does not work for Indy projects.
- metheus 8y ago
  Try the free tier on Atlas.
bnjmn 8y ago
Looking through the supported APIs (https://docs.aws.amazon.com/documentdb/latest/developerguide/mongo-apis.html https://docs.aws.amazon.com/documentdb/latest/developerguide...), it appears DocumentDB has no support for Mongo's oplog (https://docs.mongodb.com/manual/core/replica-set-oplog/ https://docs.mongodb.com/manual/core/replica-set-oplog/), or change streams (https://docs.mongodb.com/manual/changeStreams https://docs.mongodb.com/manual/changeStreams), which I guess is no surprise because change streams were introduced in Mongo 4, whereas DocumentDB copied the 3.6 API. So DocumentDB seems much less useful as a reactive data store than MongoDB. In other words, DocumentDB is only a drop-in replacement for MongoDB if you weren't using any of the features Amazon decided not to support. Happy to be corrected if I'm misreading the documentation!
- ahachete 8y ago
  I agree with you. Having said that, when we were working on https://www.torodb.com https://www.torodb.com we discussed how we'd implement the oplog. And actually, based on PostgreSQL's logical decoding (LD), it wouldn't have been a great deal (there are some gotchas, but LD brings much of what you need. So I won't be surprised if this would be implemented sooner than later.
- indogooner 8y ago
  Interesting. I think the sole purpose of this product is to wean existing Mongo customers (3.6-). And only those customers who are happy with Mongo API but not MongoDB itself. Is that such a huge market? Would be curious to see how this solution is adopted.
- InspiredIdiot 8y ago
  Also the aggregation pipeline is seriously hobbled with way more No-s than Yes-es over here https://docs.aws.amazon.com/documentdb/latest/developerguide/mongo-apis-aggregation-pipeline.html https://docs.aws.amazon.com/documentdb/latest/developerguide...
keepper 8y ago
The current biggest threat to Free and Open Source software is cloud computing. Plain and simple.[0] I know this is a blunt and harsh statement to make, but when you sell a service, you have zero native incentives to Open Source the way your system works. It just opens up Competition. This is not unique to AWS/Amazon. But their success gives them the power to have wide OSS damage. This is, to me, the biggest reason why cloud portability should be something that every customer of a cloud service should have in their plans. Amazon as a company has shown no timidness in both "embracing, extending, extinguishing" their competition. OSS literally built the internet and opened up the wold wild communication age, let's not be so short sighted that we don't see proliferation of cloud services ( specifically one having so much dominance), for what it really is. [0] http://dtrace.org/blogs/bmc/2018/12/14/open-source-confronts-its-midlife-crisis/ http://dtrace.org/blogs/bmc/2018/12/14/open-source-confronts...
- SilasX 8y ago
  Cloud computing was arguably an outgrowth of inability to prevent piracy and everyone being encouraged to open source everything.
  - keepper 8y ago
    No, it was an outgrowth of infrastructure work being a niche trade, and capacity management by startups ( and mature companies), being a hard task. This point is well known, and pretty much in every cloud providers marketing material.
    - SilasX 8y ago
      So, even though it's impossible to prevent piracy, effectively forcing you to hide the code behind an internet API, the real reason to do so is something else, as proven by marketing literature? If I could re-engineer MongoDB so that a monkey could administer, you'd recommend I still use the cloud model rather than sell binaries?
- cthalupa 8y ago
  This comment is pretty bizarre when taken into account with the link you referenced. Unless I'm completely misreading what Cantrill is saying in that blog, I don't think he agrees with you. >And while they’re at it, it would be great if they could please stop making outlandish threats about the demise of open source >Adam’s fundamental optimism serves to remind us, too, that any perceived “danger” to open source is overblown: open source is going to endure >and in the end, open source will survive its midlife questioning just as people in midlife get through theirs: by returning to its core values and by finding rejuvenation in its communities
codepopacy 8y ago
Dj from MongoDB here. We have, obviously, been keeping up with this and other threads, but we've also been busy testing out Amazon DocumentDB's correctness and performance. While we're getting that together to bring you an official response in a few days, complete with test results and methodology, I'd like to pick up on a couple of points and some inaccuracies that have been repeated in various threads: This move shows MongoDB’s approach to document databases is compelling. We’ve thought so for a long time. A cloud-hosted, truly global and managed MongoDB, MongoDB Atlas, has existed for the last two and a half years and has been serving more and more satisfied users every day with some massive workloads. MongoDB Atlas runs the full implementation of MongoDB in the cloud. Many features of MongoDB are documented as not being implemented by DocumentDB: these include change streams, many aggregation operators including $lookup and $graphlookup. But beyond that, well let’s just say we’ve been staggered by how many tests DocumentDB has failed (no spoilers!). The MongoDB API is not under an Apache license. MongoDB drivers are still under the Apache license. The MongoDB server used to be licensed under AGPL and is now licensed under SSPL. The source code is open to all, as it has always been, at https://github.com/mongodb/mongo https://github.com/mongodb/mongo DocumentDB is not cheaper than MongoDB Atlas. Preliminary estimates show this to only be the case with very large collections and very, very high read/write workloads. There’ll be more next week over on the MongoDB blogs. Dj
- snissn 8y ago
  SSPL feels a lot like a bait and switch to me
- therealdrag0 8y ago
  Any idea when Atlas will expand support for Sharding configurations and taggable zones? My impression is Atlas ONLY supports a 2 field shard, and the first shard MUST be location. Also it's impossible for clients to set write-concern to tags, because you don't support custom tags as MongoDB itself does.

519 points by ifcologne 8y ago

vaer-k 8y ago

Finally an AWS service where the name makes sense and describes what it is. I hope this is the start of a trend.

rjbwork 8y ago
I love Azure for this. The names are almost all extremely straightforward. There are a handful that have made the jump from confusing to straight forward, and a handful that have made the jump from straightforward to confusing (CosmosDB, formerly DocumentDB, chiefly comes to mind).
- Someone1234 8y ago
  Agreed. Too bad the Azure portal is the polar opposite. AWS, for all its faults, is mostly just a boring HTML portal but it works. Azure tried to get fancy, with side sliding panels all over the place, and it is barely useable. The nicest thing I can say is it is "quirky." It isn't really productive however, particularly not on my 1080p monitor at Windows 10's default 125% DPI. I literally quit Azure's Application Insights and went back to Google Analytics simply because I hated the Azure UI with a burning passion of a thousand suns. The concept of writing queries is good, but if that's the only way you can get at your data you better make it damn easy, and they didn't. I'm sure for full time data pros it is a dream however.
  - dharmab 8y ago
    Azure Portal feels like if someone tried to make the Xbox 360 blade interface into an admin tool, without first asking the admins what they needed.
  - rjbwork 8y ago
    That's interesting. I actually quite like it. I can build monitoring dashboards for our various services an see how something I don't need to monitor is doing just by going to the panel for it. To each his own I suppose.
- aynsof 8y ago
  Except for the fact that Azure names seem to change once per year. Our Azure SA was giving us a presentation and actually got confused himself. "So that's TFS... I mean VSTS... Actually wait, it's Azure DevOps now?"
  - coredog64 8y ago
    Just wait until next year when it’s called TFSHub.
  - innocentoldguy 8y ago
    I worked at Microsoft for a while and I swear most of their "upgrades" are nothing more than renaming things and juggling menu items around so people can't find them.
- titanix2 8y ago
  Yes the names are good but they still suffer for the MS illness: they change every few years. I also agree with other comments about the portal UI. Heck, it's supposed to be a professional tool...
napsterbr 8y ago
Surprised they didn't go with a 3-letter acronym. AWS DDB. Recently I made a typo on a formal document. Wrote "AMI" when I meant to to write "IAM". Oops.
- tj-teej 8y ago
  DDB is typically used to denote DynamoDB
  - stevehawk 8y ago
    i honestly thought that was his joke
- jkchu 8y ago
  I would have preferred AWS D2B.
  - Corrado 8y ago
    My brain keeps trying to parse this into either DB2 or some Star Wars reference (R2DB, RD2B, etc.)
kerng 8y ago
Yeah, they copied Azure basically. DocumentDB was the name of an Azure service in past, interestingly it offered MongoDB, Gremlin and other API gateway options. Its called Azure CosmosDB now.

notmyname 8y ago

seems an obvious response to https://www.mongodb.com/press/mongodb-issues-new-server-side-public-license-for-mongodb-community-server https://www.mongodb.com/press/mongodb-issues-new-server-side...

dantiberian 8y ago
I think it is likely it was the other way around. MongoDB caught wind of what AWS was about to release and changed the license.
- Slippery_John 8y ago
  Does that even apply? This is api-compatible, but doesn't appear to be using any actual MongoDB code.
  - dantiberian 8y ago
    It seems like Amazon may think so, with this line: > Amazon DocumentDB implements the Apache 2.0 open source MongoDB 3.6 API by emulating the responses that a MongoDB client expects from a MongoDB server
tootie 8y ago
Azure has been running a Mongo-compliant DB under their Cosmos umbrella for quite a while. It's not clear to me that either Azure or AWS are actually running Mongo software under the hood or rather a proprietary DB that uses the Mongo wire protocol. https://docs.microsoft.com/en-us/azure/cosmos-db/mongodb-introduction https://docs.microsoft.com/en-us/azure/cosmos-db/mongodb-int...
- Analemma_ 8y ago
  Doesn’t Azure also have a not-Mongo service also called DocumentDB? Is this the same code? These cloud services are confusing enough when they don’t borrow each other’s names.

jchw 8y ago

Seems like this is likely to be the real result of licenses like the SSPL. Not even a terrible outcome if the different implementations remain relatively compatible.

wmf 8y ago
SSPL style licenses only work if the software can't be cloned but that's a difficult assumption.
- earenndil 8y ago
  Operating systems, compilers, and web browsers come to mind. There are currently: * 4 independently-developed competitive compilers (gcc, clang, msvc, icc) * 4 independently-developed competitive operating systems (windows, macos, linux, and bsd --I'm grouping the BSDs as one since their source code has a common ancestor) * 3 independently-developed competitive browser engines, soon-to-be 2 (edgehtml, gecko, webkit) And it's been that way for a few decades now; doesn't look like anyone is interested in taking the resources to make another one of those.
  - johncolanduoni 8y ago
    Throwing Chrome/Blink under WebKit is a pretty hard sell at this point. They’ve diverged enough that supporting one far from guarantees you’ll support the other. You might as well replace WebKit with KHTML in your list.
    - earenndil 8y ago
      For the purposes of licensing and simple inertia (what it takes to start a project from scratch) -- that work was done once, with khtml, sure.
kemitchell 8y ago
I’d be surprised to learn that AWS started and launched this project since the SSPL announcement. I suspect they began when latest Mongo was still AGPL, with no sign of impeding change.

deleted 8y ago

[deleted]

StreamBright 8y ago

I hope MongoDB's users are really keen on on-prem otherwise this might be a challenging moment for them.

threeseed 8y ago
Not sure what this means. MongoDB has had Atlas which is a cloud hosted solution for a while now.
- thawkins 8y ago
  But you cant host your data inside your aws account with atlas, non starter for some
- ec109685 8y ago
  Poster means that their cloud business could be impacted, but on-prem (non aws) would not be.

mark_l_watson 8y ago

Looks good. I used to use both MongoDB and CouchDB (for very different use cases) a lot but haven’t touched them in a long while.

anonytrary 8y ago
Out of curiosity, what use cases did you have for CouchDB that weren't feasible with MongoDB?
- gorbypark 8y ago
  The best part of CouchDB is PouchDB, in my opinion. It's kinda cool having an in browser database that just syncs "almost effortlessly" with the cloud hosted version.
- WorldMaker 8y ago
  In my case, at least, PouchDB is still a lot handier of an offline-first mobile-capable DB than just about anything else, and because it speaks the CouchDB replication/sync API that still leaves a lot of use cases where CouchDB is more feasible. (It would be great to have more document DBs converge on a replication/sync API for offline-first applications. I've voted on the UserVoice suggestions to CosmosDB on the idea.)
  - karmelapple 8y ago
    Same here. We love how replication is built in to CouchDB from the ground up. I wish MongoDB, which seems to still have more love for it in the world than CouchDB/Cloudant, would have that replication built-in.
    - metheus 8y ago
      Wait... do you mean something other than https://docs.mongodb.com/manual/replication/ https://docs.mongodb.com/manual/replication/ ?

anonytrary 8y ago

AWS lacked something like this before, which is why I used MongoDB Atlas (MongoDB's own managed mongo service). How is this different than DynamoDB (other than the obvious API differences)?

dfischer 8y ago
DynamoDB has specific requirements on how data is partitioned. You have to do upfront planning on how you access your data based on keys. Mongodb is more flexible in this regard. I’m sure there’s other big differences but this is the biggest to my experience with both.
- anonytrary 8y ago
  I was asking about DocumentDB, not MongoDB, but maybe I missed something. From their site: > Amazon DocumentDB implements the Apache 2.0 open source MongoDB 3.6 API by emulating the responses that a MongoDB client expects from a MongoDB server, allowing you to use your existing MongoDB drivers and tools with Amazon DocumentDB So, is this a "managed MongoDB" or is it a NOSQL AmazonDB that just implements the same API as MongoDB? If the latter, I don't think we can assume this is the same thing as MongoDB, because the internals might be completely different.
  - Slippery_John 8y ago
    I would imagine this doesn't use any MongoDB code, or else the SSPL license would brutalize them. That said, I would be surprised if the experience is much worse.
  - k__ 8y ago
    Sounds like the latter to me, they probably implemented it on top of one of their other DBs.
    - anonytrary 8y ago
      Lmao I was a bit afraid this might be some surface-level API wrapper around DynamoDB, in which case this is less cool than I thought. It doesn't seem to be the case at a first glance but I wish it was a bit more clear on the site exactly how this relates to DynamoDB, if at all. Maybe I'll just have to read more about it. Hopefully the internal architecture gets away from DynamoDB-style partitioning.

whalesalad 8y ago

Serious question: is there any real reason to use Mongo over Elasticsearch?

a13n 8y ago
I run a startup that uses both of these. Mongo is a heck of a lot easier to configure and develop with, and works great as a general database with a rapidly changing schema. Elasticsearch is great at solving specific problems like searching for items in a specific way, but it's got quite a learning curve and is pretty painful to host and configure compared to Mongo where you can have a prod instance going in seconds.
ajsharp 8y ago
Yes. Elasticsearch is not nearly as easy to use and is not really designed to be a transactional database.
alrs 8y ago
Elasticsearch is a distributed index, not a reliable document store. You can expect availability problems and data loss. This is fine, because you can rebuild the index from your source of truth.
- rafaelturk 8y ago
  True.
- plasma 8y ago
  In my experience with ES, definitely don't treat it as a source of truth; allow it to be rebuilt.
StreamBright 8y ago
Is it a good idea to use a database instead of a search engine?
throwaway204 8y ago
ES grew out of Lucene, which provided an inverted index of all the text in a document, with a bunch of NLP related features bolted onto that. While ostensibly designed to be developer friendly, ES had and still has a horribly hacked together API with bugs and mis-documented misfeatures all over the place. In my experience it's anything but developer friendly. But if all you need is a text index on a document store with a static schema, it does its job reasonably well. Mongo started as a very developer-friendly data store with lots of overstated claims to being a database. While earlier versions of Mongo were wildly dangerous to use as a business critical database, it has since matured and is now quite good at being a developer-friendly document-centric database. In my experience Mongo truly is developer-friendly as long as you don't try to use it as a full-blown transactional database with lots of complex data shapes and indexes. I would not trust ES with anything but text search on a document store, and I would not trust Mongo with anything resembling multi-document transactions. With that said, they are both good at specific, different things. MySQL and Postgres have their own baggage that makes them pretty terrible in some aspects. IMHO a JSON-over-HTTP API should really be table stakes for a database to be considered developer-friendly nowadays. (But please don't butcher HTTP like ES did and then claim to have that.)

dstaley 8y ago

If I'm reading the pricing page correctly, DocumentDB would run a _minimum_ of $200/month. That's for the smallest instance and no storage or I/O. Kind of steep if you ask me.

imaginenore 8y ago
AWS pricing is so strange. They give you 25GB of DynamoDB for free and charge 25 cents per million reads (under 4K, i think). $200 minimum means almost no indie developer will even try it.
a13n 8y ago
Yeah, that's pricy. They're definitely not going after early-stage startups then. But if you have a medium-sized data set (eg. 50+ GB), this is definitely competitively priced. More RAM, storage, compute than Mongo Atlas and Compose for less money. Here's hoping they introduce cheaper options!
- orf 8y ago
  50GB is a really small data set.
  - deleted 8y ago
    [deleted]
  - a13n 8y ago
    Eh, by what measure? Realistically it's probably bigger than 90% of all Mongo datasets. It's tiny if you're a massive company and it's massive if you're a tiny startup.
    - q3k 8y ago
      50GB easily fits in RAM. It's a small dataset.
    - orf 8y ago
      If you can run the dataset comfortably on a Macbook then it's very, very small. Heck, you can even just use grep over 50GB reading straight from disk. It's tiny.

misframer 8y ago

Sounds like this runs on the same storage service as Aurora.

misframer 8y ago
Not sure why I'm getting downvoted. The characteristics sound exactly like Aurora. - "replicates six copies of your data across three AWS Availability Zones (AZs)" [0] - "Amazon DocumentDB uses a distributed, fault-tolerant, self-healing storage system that auto-scales up to 64 TB per database cluster." [0] - "When writing to storage, Amazon DocumentDB only persists a write-ahead logs, and does not need to write full buffer page syncs." [1] [0] https://aws.amazon.com/documentdb/ https://aws.amazon.com/documentdb/ [1] https://aws.amazon.com/documentdb/faqs/ https://aws.amazon.com/documentdb/faqs/
- MrTonyD 8y ago
  Yeah, downvoting is totally broken on Hacker News. Seems like anything that isn't immediately agreed-with gets downvoted. I don't know where all the small-minded people come from, but they seem to have found their home here.
  - ma2rten 8y ago
    There is just some random noise. It might just have gotten one downvote. Now it's the top-comment.
- jkchu 8y ago
  I'm guessing if you would have included this reasoning in your original comment, then it wouldn't have been downvoted.
ec109685 8y ago
Good article on Aurora's design: https://www.allthingsdistributed.com/files/p1041-verbitski.pdf https://www.allthingsdistributed.com/files/p1041-verbitski.p...

orestes910 8y ago

Is it really THAT hard to to run on prem/cloud because of the "complexity that comes with setting up and managing MongoDB clusters at scale"? The documentation is strong, it was built with horizontal scalability in mind. I don't see the struggle.

skywhopper 8y ago
If you don't see the value, then no need to pay for it.
uberduper 8y ago
Yes. Sharded mongo has many pitfalls that can challenge your devs, dbas and devops people. Mongo api without all the mongo scaling problems is pretty awesome.
- chx 8y ago
  I must have used a different API, then. Well, obviously not but it could happen because there are more than one ... I agree with https://www.linkedin.com/pulse/mongodb-frankenstein-monster-nosql-databases-john-de-goes/ https://www.linkedin.com/pulse/mongodb-frankenstein-monster-... the contents of this article (and reserve critique for its author). Even the basic query language, in particular $elemMatch is atrocious which has a certain irony to it because Eliot added it when we asked for the functionality (https://jira.mongodb.org/browse/SERVER-377 https://jira.mongodb.org/browse/SERVER-377), so long ago. We have been one of their first commercial support clients, my boss needed to pressure them to accept money for support, they didn't want to...

deleted 8y ago

[deleted]

ahachete 8y ago

My bet is that it is built on top of Aurora PostgreSQL. By looking at the "Limits" section (https://docs.aws.amazon.com/documentdb/latest/developerguide/limits.html https://docs.aws.amazon.com/documentdb/latest/developerguide...), identifiers are limited to 63 characters and the same characters that PostgreSQL limits identifiers to; and a collection size limit of 32TB, coincidentally maximum PostgreSQL table size. Edit: I can confirm: does not allow the UTF-8 null character in strings: https://docs.aws.amazon.com/documentdb/latest/developerguide/functional-differences.html https://docs.aws.amazon.com/documentdb/latest/developerguide... ... It is written on top of PostgreSQL.

deleted 8y ago
[deleted]
misframer 8y ago
I think they're built on a common storage system, just like the MySQL compatible version too.
erikig 8y ago
It would be nice if Amazon provided an API to access the data via SQL alongside the MongoDB API; I've seen quite a number of organizations migrate from mongo to Postgres once they get out of the rapid development phase. This would make that transition butter smooth.
- ahachete 8y ago
  That would make the internal representation used an "API" and thus won't be able to change it in the future. Apparently, they are using a 1:1 mapping between a collection and a table. Either by flattening the document or by using jsonb or equivalent. I'm not a big believer this is good for performance reasons, at least compared to a more normalized approach like the one we did for https://www.torodb.com https://www.torodb.com But they may change it in the future --if they don't expose the SQL API to their internal representation.
- scarface74 8y ago
  Anecdote: I led a C# project where we could seamlessly switch back and forth between Mongo and SQL Server without changing the underlying LINQ expressions. We sent the expressions to the Mongo driver and they got translated to MongoQuery we sent the expressions to Entity Framework and they got translated to Sql Server.

deleted 8y ago

[deleted]

openloop 8y ago

https://www.rackspace.com/en-us/cloud/files https://www.rackspace.com/en-us/cloud/files

abrookewood 8y ago

I was reading a post [0] by Brian Cantrill that predicted this would be the result of licences like the SSPL. I instinctively disagreed with him, but it turns out he was right: "The cloud services providers are currently reproprietarizing all of computing — they are making their own CPUs for crying out loud! — reimplementing the bits of your software that they need in the name of the service that their customers want (and will pay for!) won’t even move the needle in terms of their effort." [0] http://dtrace.org/blogs/bmc/2018/12/14/open-source-confronts-its-midlife-crisis/ http://dtrace.org/blogs/bmc/2018/12/14/open-source-confronts...

dman 8y ago
What is the way out? Would love to hear from people.
- bilbo0s 8y ago
  Not sure this would fall under SSPL in any case. It's clear that what Amazon is doing is using Postgres under the hood, not really mongo. So I'm not sure how that would work if you make an interface shim to make postgres look like mongo, are you then subject to the mongo license? the postgres license? the apache 2.0 mongo api license? all of them? what if clauses of them are mutually exclusive? etc etc etc. Just at a cursory glance it certainly seems like only the apache 2.0 mongo api license would apply. But I guess mongo could try to force the sspl on amazon?
- SpicyLemonZest 8y ago
  There's no secret formula to stop people from competing with you. If MongoDB Inc is successful, it should be because they run a good document-database-as-a-service people want to use, not because they earn indefinite seigniorage from launching a popular open source project.
  - KaoruAoiShiho 8y ago
    But isn't it wrong to place all economic value in the hosting layer rather than the software layer?

luhn 8y ago

Why is it so expensive? Not only is the entry point $200/m, the instances are twice the price of their EC2 equivalents. (At first I thought perhaps the price included multi-AZ, but it doesn't.)

ISV_Damocles 8y ago

Now I kinda hope Oracle decides to buy out MongoDB and integrate it into their own cloud. Then Oracle can decide to pull the same bullshit that they did with Google over the Java APIs with the MongoDB APIs but now against their current enemy Amazon (and Microsoft, too). Then a combined Google + Amazon + Microsoft may finally be able to reverse the API Copyright insanity that is hovering ominously over the tech industry, and Oracle can continue to be a shining city upon a hill of shitty technologies you should never allow your business to adopt.

wmf 8y ago
AWS is preemptively defensive about API licensing claims: "Amazon DocumentDB implements the Apache 2.0 open source MongoDB 3.6 API".
- christkv 8y ago
  I think they are referencing the drivers which are licensed under Apache 2.0.
giorgioz 8y ago
I've always seen Google+Android as the good guys that gave Java new life while I saw Oracle has the bad guys that bought Sun and killed Java.

XorNot 8y ago

I'm pretty sure this is going to kill Mongo as a company dead. With this in existence there's literally no reason to use Atlas. If they wanted to twist the knife they should get to work implementing a pass through migration option.

_wmd 8y ago
MongoDB still have a strong hand - control of the client and particularly its exposed featureset - due to that, also control of the protocol and the ability to, for example, insert legally protected strings in the style of the Apple SMC signature into the handshake. - ability to gate new features on the presence of an object like a copyrighted text, trademark, or even a crypto signature - ownership of the name. AWS are pissing in Mongo's pool marketing themselves as compatible, and there are a variety of ways it could be made to backfire, if it were in Mongo's interests to encourage that outcome - AWS focuses on breadth and very rarely nails any particular service. Their hosted Postgres for example still does not expose core features years later - Following from that, AWS services on the whole are rarely best-in-class in terms of raw performance. I imagine Mongo could continue to easily compete on benchmark results running on AWS own infrastructure I think this is a really interesting case, far more interesting than the technical minutia of Just Yet Another AWS service. It does not sit well with me whatsoever that they're basically ripping off a much smaller company's core tech while simultaneously borrowing their trademark (in a legally acceptable manner) as part of the marketing, but I also find it hard not to see a ton of potential upside from this for Mongo
- ec109685 8y ago
  The client change would never work. The client is licensed as lgpl, so if they tried to pull any funny business like that, it would be instantly forked and if’d out.
  - _wmd 8y ago
    As the person suggesting it, it's difficult to imagine how it could never work considering I haven't managed to figure out all the possible combinations in which such a strategy could be applied. Finally, it is quite exasperating to call this kind of strategy "funny business" in a thread about their core tech being ripped off by a megacorp

outworlder 8y ago

How the heck was this approved by legal? AWS is linking to MongoDB's own documentation. https://docs.aws.amazon.com/documentdb/latest/developerguide/mongo-apis.html https://docs.aws.amazon.com/documentdb/latest/developerguide... Go to any operation, it links to docs.mongodb.com !

chrisseaton 8y ago
Is linking to documentation illegal?
- outworlder 8y ago
  The 'legal' department doesn't only vet things that are obviously illegal, they are there for other things too, like EULAs, contracts and intellectual property in general. In any case, they are implementing the competitor's APIs and the documentation is already licensed as Creative Commons. Can't they at least host their own copy?
  - deleted 8y ago
    [deleted]
  - deleted 8y ago
    [deleted]
- deleted 8y ago
  [deleted]
manigandham 8y ago
What is there to approve? Links are public.
kerng 8y ago
Nothing novel here by Amazon. Azure had MongoDB gateways (and many others) for their NoSQL offering.

koolba 8y ago

Jeff Bezos continues his quest to eat all open lunch boxes.

ec109685 8y ago
He isn’t using any of their code though.
0xFFFF0000 8y ago
Microsoft was first with a MongoDB layer, like some 3 years ago I think. Nothing novel here from Amazon.

manigandham 8y ago

So now AWS DocumentDB, Azure CosmosDB, and even Apple's FoundationDB have a MongoDB compatible API. I expect other multimodal databases to offer the same soon enough. Strange turn of events for MongoDB but I guess that's what happens when the interface is open and anyone can build a backend to it, especially a relatively simple document-store.

ec109685 8y ago
And when they have a restrictive license.

markwolfe 8y ago

This looks great for teams getting started in AWS, being able to reuse idioms, libraries and knowledge in managed services to remove some of the operational load. A note this is also nothing new, Azure Cosmos DB has had this for a while.

0xfffff 8y ago

Wasn't DocumentDB the old name of Azure Cosmos DB? just marketing wise this seems like a bad choice for a name.

miaklesp 8y ago

Of all possible names they managed to choose already obsolete name for a similar product from Azure? Azure in 2017: forget DocumentDB, Cosmos DB is the new thing now! AWS in 2019: We just released DocumentDB! Are they so geniuses or so idiots?

0xFFFF0000 8y ago
I had the same observation (although I'd reconsider your wording, that's not very polite). It doesn't seem like a good choice in naming - especially because many of the Data Platform folks at Amazon (like their VP) came from Microsoft's data platform group.
miaklesp 8y ago
UPDATE: They are idiots.

morpheuskafka 8y ago

I really wish this was serverless. Azure CosmosDB offers SQL and MariaDB interface against a serverless, pay for what you use database and DynamoDB is the only product of that class Amazon has. Even Aurora "serverless" appears to be little more than autoscale, and it requires an elastic IP which slows launch of lambdas since they have to connect to VPC.

iamed2 8y ago
> it requires an elastic IP which slows launch of lambdas since they have to connect to VPC Do you have any other info/links related to this?
- gazzini 8y ago
  There's an enlightening graph in this post: https://medium.freecodecamp.org/lambda-vpc-cold-starts-a-latency-killer-5408323278dd https://medium.freecodecamp.org/lambda-vpc-cold-starts-a-lat... I was surprised to learn this. When working in Lambda, you have to choose between a relational database & a responsive API. It seems inevitable that AWS will fix this soon, but apparently this is a significant architectural problem. As I understand it, RDS instances should (must?) be accessed from within a VPC, and anything inside of a VPC needs an IP address, so the Lambda function has to wait ~10 seconds on cold-start for the Elastic IP service. The only workaround I've heard of is to setup a service, such as CloudWatch, to call your Lambda function every ~25 secs to keep it "warm", but this seems anti-thetical to the value proposition of serverless architecture in the first place. Of course, you could "just" use DyanmoDB, but IMO the query language is really limited, and I'm not sure I fully grasp the problem (why doesn't DynamoDB need to be accessed from within a VPC?)
  - athrun 8y ago
    > I'm not sure I fully grasp the problem (why doesn't DynamoDB need to be accessed from within a VPC?) This is due to the fact that DynamoDB's query API is a standard AWS API which means granular internal/external access can be provided through IAM mechanisms (ie: roles, temporary tokens, federation, etc.). On the contrary, to access RDS, Redshift or DocumentDB you would use standard ODBC/JDBC/Mongo facilities, which do not rely on IAM mechanisms, leaving VPC/Security Groups as the only isolation option.

KaoruAoiShiho 8y ago

Can someone do a comparison on pricing vs Atlas and vs DynamoDB and Amazon's managed postgresql?

social_quotient 8y ago

I really wish this were priced more along the lines of https://www.compose.com/pricing https://www.compose.com/pricing - a $200/m floor is a tough dB cost to absorb on smaller yet important projects. Suppose an app has a few mb of data and maybe one day hits 100mb of awesomeness I really have to pay $200/m here? I get it, I love Aws, just wish this was priced differently.

illumin8 8y ago
If you need a cheap/free document store for a small app, just use DynamoDB. It's free (forever, not just the first year) up to 25GB of storage and enough read/write capacity units to handle up to 200M requests per month: https://aws.amazon.com/free/?awsf.Free%20Tier%20Types=categories%23alwaysfree https://aws.amazon.com/free/?awsf.Free%20Tier%20Types=catego...
- privateSFacct 8y ago
  Exactly, this probably isn't targeting you. A few mb of data? Host an instance? Use sqlite? Atlas can run 3-5k/month as the minimum. This is going to be 10x cheaper (which amazon seems to try and shoot for).
Corrado 8y ago
Another thought along these same lines is how do you host non-production (Dev, QA, Demo, etc.) environments without spending a fortune? Sure my production workload is 15TB but my development system is only running 15GB and I want to develop against what I deploy to.
- GordonS 8y ago
  If you can create schemas, that could be one solution. I do this with Azure SQL Server instances, so a single instance can host all the 'non-critical' environments (dev, test, QA, demo) - works great!

0xFFFF0000 8y ago

Weird question: Could Microsoft sue Amazon here for infringing on the DocumentDB name? I mean Microsoft's DocumentDB was among the first to even have such a MongoDB layer also) and that was like 3 years ago. Given that current Amazon leaders actually came from Microsoft's data platform group this leaves a bit of a bad taste behind. I'm not working for either company.

jon-wood 8y ago
My assumption is that DocumentDB falls into a category of being so generic you can't trademark it, or otherwise claim exclusivity to it. Its literally just describing the fact this is a database for documents.
- SmellyGeekBoy 8y ago
  Bear in mind we're talking about the company that trademarked "Word" and "Excel" here.
amyjess 8y ago
For all any of us know, Amazon's lawyers already talked to Microsoft's lawyers about it and got permission beforehand. See: Apple licensing the iOS name from Cisco before announcing the name change.

swoorup 8y ago

I cant use this crap locally?

talawahdotnet 8y ago

I am pretty frustrated that DB services like Aurora, and now DocumentDB are still limited to last-gen instance types like r4 instead of the latest instances like r5 and t3 which have marked improvements in terms of CPU and networking performance. I wonder if it is that they just have a so much r4 inventory left that they are forcing us to use it or if they haven't fully integrated/validated the latest instance types with their custom storage backend.

LoSboccacc 8y ago

> Text Index: No as MS Cosmo DB prior, the compatibility list leaves quite a bit out.

hendry 8y ago

Atlas is $0.03 per server hour, Amazon starts at $0.277 Do I have that right?

manigandham 8y ago
Starting prices maybe, but you have to compare hardware and performance capacity, not just flat numbers.

hguhghuff 8y ago

Is this serverless? Pricing?

sauravt 8y ago

How is it better than buying a MongoDb subscription from the AWS marketplace? https://aws.amazon.com/marketplace/search/results?x=0&y=0&searchTerms=mongodb https://aws.amazon.com/marketplace/search/results?x=0&y=0&se...

christkv 8y ago

There is a serious amount of weasel words in that statement. Implementing the Apache 2.0 API I think is just weasel words for "Works more or less with the MongoDB drivers" which is the only thing Apache 2.0 licensed that I know off and is obviously not the API which is the combination of the MongoDB wireprotocol and the commands supported by the MongoDB server.

buremba 8y ago

I don't understand why people are reacting to it so aggressively. That's basically how AWS works, they did the same to Apach Kafka with Kinesis, Prestodb with Athena, PostgreSQL and MySQL with Aurora, Redis with ElastiCache and many others over the last 4 years so it's not new. It took too long for the open-source community to figure out that the cloud providers are killing them, now it's too late. Well played, AWS.

QuestionMarking 8y ago
It's because nerds love to lose their shit any time a big company does anything.
konschubert 8y ago
Well, people are angry about that, that’s why they react aggressively.
- buremba 8y ago
  Open-source gave AWS the ability to monetize their software so the software companies should be careful enough to prevent any other big company to steal their software and use their name to make money. I think that it's too late considering AWS already did that to most of the industries but here is Hazelcast's take: https://www.linkedin.com/pulse/open-source-needs-protect-itself-from-service-wrappers-greg-luck/ https://www.linkedin.com/pulse/open-source-needs-protect-its...
driverdan 8y ago
> It took too long for the open-source community to figure out that the cloud providers are killing them How are service providers killing FOSS? That doesn't make sense. Permissive FOSS licensing allows anyone to use their software, regardless of how it's used, and that's how it should be.
- keepper 8y ago
  Do you get to see AWS's source code for these services? No... That's how it's killing "FOSS". Extend and Extinguish. This is not a new playbook.

borplk 8y ago

While we are on the topic, are there cloud providers out there that provide "hosted FoundationDB as a service"? And if you are curious, no I don't have any special need or use case for it, just wondering.

craig_peacock 8y ago

I would actually like to thank AWS for focusing on supporting Open APIs. Let the community design the standards and operations and let the vendors compete to implement them, open source included.

vemv 8y ago

I hope AWS realises that harming OSS authors, even indirectly, is counterproductive to AWS' interests. If Mongo dies, so does its userbase, and so does DocumentDB. How to fix it classily? Contribute back with code maintenance or whatever customizations were developed in-house. Also you get to save face - you are not perceived as a leech anymore.

jonplackett 8y ago

Shame there isn’t a free tier equivalent. I’d like to try some projects using mongo but $200 a month minimum does not work for Indy projects.

metheus 8y ago
Try the free tier on Atlas.

bnjmn 8y ago

Looking through the supported APIs (https://docs.aws.amazon.com/documentdb/latest/developerguide/mongo-apis.html https://docs.aws.amazon.com/documentdb/latest/developerguide...), it appears DocumentDB has no support for Mongo's oplog (https://docs.mongodb.com/manual/core/replica-set-oplog/ https://docs.mongodb.com/manual/core/replica-set-oplog/), or change streams (https://docs.mongodb.com/manual/changeStreams https://docs.mongodb.com/manual/changeStreams), which I guess is no surprise because change streams were introduced in Mongo 4, whereas DocumentDB copied the 3.6 API. So DocumentDB seems much less useful as a reactive data store than MongoDB. In other words, DocumentDB is only a drop-in replacement for MongoDB if you weren't using any of the features Amazon decided not to support. Happy to be corrected if I'm misreading the documentation!

ahachete 8y ago
I agree with you. Having said that, when we were working on https://www.torodb.com https://www.torodb.com we discussed how we'd implement the oplog. And actually, based on PostgreSQL's logical decoding (LD), it wouldn't have been a great deal (there are some gotchas, but LD brings much of what you need. So I won't be surprised if this would be implemented sooner than later.
indogooner 8y ago
Interesting. I think the sole purpose of this product is to wean existing Mongo customers (3.6-). And only those customers who are happy with Mongo API but not MongoDB itself. Is that such a huge market? Would be curious to see how this solution is adopted.
InspiredIdiot 8y ago
Also the aggregation pipeline is seriously hobbled with way more No-s than Yes-es over here https://docs.aws.amazon.com/documentdb/latest/developerguide/mongo-apis-aggregation-pipeline.html https://docs.aws.amazon.com/documentdb/latest/developerguide...

keepper 8y ago

The current biggest threat to Free and Open Source software is cloud computing. Plain and simple.[0] I know this is a blunt and harsh statement to make, but when you sell a service, you have zero native incentives to Open Source the way your system works. It just opens up Competition. This is not unique to AWS/Amazon. But their success gives them the power to have wide OSS damage. This is, to me, the biggest reason why cloud portability should be something that every customer of a cloud service should have in their plans. Amazon as a company has shown no timidness in both "embracing, extending, extinguishing" their competition. OSS literally built the internet and opened up the wold wild communication age, let's not be so short sighted that we don't see proliferation of cloud services ( specifically one having so much dominance), for what it really is. [0] http://dtrace.org/blogs/bmc/2018/12/14/open-source-confronts-its-midlife-crisis/ http://dtrace.org/blogs/bmc/2018/12/14/open-source-confronts...

SilasX 8y ago
Cloud computing was arguably an outgrowth of inability to prevent piracy and everyone being encouraged to open source everything.
- keepper 8y ago
  No, it was an outgrowth of infrastructure work being a niche trade, and capacity management by startups ( and mature companies), being a hard task. This point is well known, and pretty much in every cloud providers marketing material.
  - SilasX 8y ago
    So, even though it's impossible to prevent piracy, effectively forcing you to hide the code behind an internet API, the real reason to do so is something else, as proven by marketing literature? If I could re-engineer MongoDB so that a monkey could administer, you'd recommend I still use the cloud model rather than sell binaries?
cthalupa 8y ago
This comment is pretty bizarre when taken into account with the link you referenced. Unless I'm completely misreading what Cantrill is saying in that blog, I don't think he agrees with you. >And while they’re at it, it would be great if they could please stop making outlandish threats about the demise of open source >Adam’s fundamental optimism serves to remind us, too, that any perceived “danger” to open source is overblown: open source is going to endure >and in the end, open source will survive its midlife questioning just as people in midlife get through theirs: by returning to its core values and by finding rejuvenation in its communities

codepopacy 8y ago

Dj from MongoDB here. We have, obviously, been keeping up with this and other threads, but we've also been busy testing out Amazon DocumentDB's correctness and performance. While we're getting that together to bring you an official response in a few days, complete with test results and methodology, I'd like to pick up on a couple of points and some inaccuracies that have been repeated in various threads: This move shows MongoDB’s approach to document databases is compelling. We’ve thought so for a long time. A cloud-hosted, truly global and managed MongoDB, MongoDB Atlas, has existed for the last two and a half years and has been serving more and more satisfied users every day with some massive workloads. MongoDB Atlas runs the full implementation of MongoDB in the cloud. Many features of MongoDB are documented as not being implemented by DocumentDB: these include change streams, many aggregation operators including $lookup and $graphlookup. But beyond that, well let’s just say we’ve been staggered by how many tests DocumentDB has failed (no spoilers!). The MongoDB API is not under an Apache license. MongoDB drivers are still under the Apache license. The MongoDB server used to be licensed under AGPL and is now licensed under SSPL. The source code is open to all, as it has always been, at https://github.com/mongodb/mongo https://github.com/mongodb/mongo DocumentDB is not cheaper than MongoDB Atlas. Preliminary estimates show this to only be the case with very large collections and very, very high read/write workloads. There’ll be more next week over on the MongoDB blogs. Dj

snissn 8y ago
SSPL feels a lot like a bait and switch to me
therealdrag0 8y ago
Any idea when Atlas will expand support for Sharding configurations and taggable zones? My impression is Atlas ONLY supports a 2 field shard, and the first shard MUST be location. Also it's impossible for clients to set write-concern to tags, because you don't support custom tags as MongoDB itself does.