10 ms·

DocumentDB

300 points by daigoba66 12y ago

yxhuvud 12y ago
It would have been nice to see some actual details of how it works so that it can be compared to the competition.
- orf 12y ago
  First link on the page: "introduction to DocumentDB". http://azure.microsoft.com/en-us/documentation/articles/documentdb-introduction/ http://azure.microsoft.com/en-us/documentation/articles/docu...
cvburgess 12y ago
Does anyone know how this compares to AWS DynamoDB[1] ? [1] https://aws.amazon.com/dynamodb/ https://aws.amazon.com/dynamodb/
- luuio 12y ago
  This allows you to use SQL queries, and built in index support
  - cvburgess 12y ago
    Very cool features, thanks!
- balls187 12y ago
  DynamoDB really isn't document oriented storage. Instead it's a key-value wide column store, with 1st and 2nd level index support. I haven't delved into the MSFT pricing model, but DynamoDB is pay-for-through put. You provision your table with a certain amount of read/write performance that is guaranteed, and you pay for that.
  - cvburgess 12y ago
    I'd assuming pricing will be similar and incredibly competitive... and thanks, didn't realize how limited dynamoDB was
- ryanfitz 12y ago
  I've been using DynamoDB since it was released. I wrote a nodejs driver[1] and a nodejs data mapper for it[2], so I have a decent bit of experience with it. Browsing the DocumentDB docs the two services seem to be very different. DynamoDB is really just a key value store with some very nice properties, but also a lot of tradeoffs. One such tradeoff is querying data is very limited in DynamoDB. In Dynamo you can only query data by its primary hash key and optional range key. These keys you must specify upfront when you create your table and cannot be changed afterwards. DocumentDB seems much more similar to mongodb and appears to have a very flexible query ability. In my opinion, one of the best features with DynamoDB is you can tune the number of reads/writes each individual table requires. This lets you scale up and down your database and greatly helps keep costs down. This is a feature that only a hosted database service can offer. I haven't yet read any pricing info on DocumentDB, but hopefully they offer a similar feature as this is really where a hosted database service can shine. [1]https://github.com/Wantworthy/dynode https://github.com/Wantworthy/dynode [2]https://github.com/ryanfitz/vogels https://github.com/ryanfitz/vogels
  - cvburgess 12y ago
    Thanks, yeah I can see how this is more like hosted mongoDB - and I've seen dynode on npm, nice work!
pokstad 12y ago
Sounds very similar to CouchDB. Server side Javascript written by the user, and an HTTP interface. The ability to adjust consistency is really neat.
- Goranek 12y ago
  similar to cloudantdb a little bit more because of searching and indexing.
- andrewguenther 12y ago
  The ability to adjust consistency isn't really a new concept. The first I saw it was in Amazon's DynamoDB paper in 2007. Cassandra uses the model outlined there as well, and I'm sure there are others out there that I'm less familiar with.
- ahoge 12y ago
  CouchDB doesn't support ad hoc queries. CouchDB is all about MapReduce and heavy caching. It's very rigid.
bkeroack 12y ago
...and MS goes after MongoDB. It would be nice to see an on-premises version, if only to compare performance/consistency with Mongo.
- codereflection 12y ago
  Not really so far, it's Azure only.
- taude 12y ago
  I have to believe it's coming, since a lot of MSFTs core business is still selling server-licenses for business who host their own data centers. I for one would love to use something like this, and our company is one that's not comfortable with third-party open-source tools, and we have to host in our own data center...
- reubenbond 12y ago
  DocDB is built atop a very well battle-tested distributed systems framework with replication based on a multi-Paxos implementation. I haven't done any benchmarks, but the replication model is far superior to MongoDB's "journal journal" model for performance. http://daprlabs.com/blog/blog/2014/08/22/azure-documentdb/ http://daprlabs.com/blog/blog/2014/08/22/azure-documentdb/
  - dougws 12y ago
    Your objections to MongoDB's model seem reasonable, but I don't see any evidence in either this comment or the linked blog post that DocumentDB is better (especially in the absence of benchmarks). What is this "battle-tested distributed systems framework"? Several of your complaints about MongoDB have to do with the interaction between persistence to disk and replication to the network; as the Multi-Paxos algorithm does not specify when data should be written to disk (much less what the format should be), what reason is there to believe that DocumentDB does this any better? I'm totally willing to believe that DocumentDB beats the pants of MongoDB on just about every axis (in fact, that seems pretty likely) but it's going to take some actual numbers and a better description of the internals.
jnardiello 12y ago
And that is a creative product name. Well played MS.
- talles 12y ago
  Just like SQL Server
- programminggeek 12y ago
  It's really smart. It's a document database, so they call it DocumentDB. It's better than calling it something like Dolphin and then explaining that it's a document database. Simple and clear is a good thing.
  - allegory 12y ago
    Err, Apple: Airport, Bootcamp, Safari, Time Machine, FaceTime, Grand Central Dispatch, QuickTime... and I won't even start on the open source side of things...
  - talles 12y ago
    But it's also kinda selfish don't you think?
    - bdcravens 12y ago
      No more than AWS's SimpleDB.
  - tomsthumb 12y ago
    "Microsoft has DocumentDB service now." "Oh cool, which document database is it? MongoDB? CouchDB? Cassandra?" "It's DocumentDB!" "...."
    - jaredmcateer 12y ago
      Just like trying to get help around libraries of Prototype or Async in regards to Javascript. 90% of the results (I'm exagerating) are jQuery
jpalomaki 12y ago
Ad hoc queries using SQL like syntax. No need to define indexes. Javascript execution within database. Stored procedures, triggers and functions can be written with Javascript. "All JavaScript logic is executed within an ambient ACID transaction with snapshot isolation. During the course of its execution, if the JavaScript throws an exception, then the entire transaction is aborted." Pricing is based on "capacity units". Starts with $22.50 per month (this includes 50% preview period discount). One capacity unit (CU) gives 10GB of storage and can perform 2000 reads per second, 500 insert/replace/delete, 1000 simple queries returning one doc. In order to see pricing details, change the region to "US West": http://azure.microsoft.com/en-us/pricing/details/documentdb/ http://azure.microsoft.com/en-us/pricing/details/documentdb/ Very interesting addition to Microsoft offering. I was actually just yesterday wondering if they have any plans for this kind of service. Table Storage is quite primitive and Azure SQL on the other hand gets expensive when you have lots of data. One potential "problem" with this is the bundling of storage capacity and processing power. If I understand this correctly, I would need to buy 10 CUs per month to store 100GB of data even if I'm not very actively using that data.
- jnardiello 12y ago
  That is actually good to start with, yet scaling will cost A LOT.
talles 12y ago
Can I use DocumentDB out of Azure (hook my own)?
- twotwotwo 12y ago
  You know, it's not a simple question, but there's some case for Microsoft just up and releasing some scale-out DB for free (as in beer) or at least cheap. My thinking is, scale-out deployments aren't all that likely to pay at SQL-Server-like rates per CPU anyway, and you're helping give the Windows world better parity with the horde of Linux options for folks that need scale-out (or, just as commonly, hope to need it someday). On the flipside, by doing that you might forego some sales of SQL Server (though I suspect that's limited; most folks that need SQL Server really need SQL Server) or sales on Azure that could theoretically help recover the dev costs. But greatly improving the scale-out-on-Windows story seems like a big deal, the kind of thing that might justify going to lots of effort to make a DB product then giving it away.
  - aikah 12y ago
    > You know, it's not a simple question Yes it is a simple question, and the answer is NO.
    - deleted 12y ago
      [deleted]
ceejayoz 12y ago
http://azure.microsoft.com/en-us/documentation/articles/documentdb-introduction/ http://azure.microsoft.com/en-us/documentation/articles/docu... > Want to edit or suggest changes to this content? You can edit and submit changes to this article using GitHub. Pretty remarkable given Microsoft's approach to open source in the 1990s that they're now using a service built around Linus's bespoke open source version control system to allow people to suggest changes to their documentation.
- pjmlp 12y ago
  Microsoft is the new IBM. :)
  - braum 12y ago
    Agree and I've been saying it for ten damn years, havn't I been saying it!
    - scholia 12y ago
      See my comment above. Actually, there is one way that Microsoft did become the new IBM. Long ago, I went to a talk by a senior IBMer and he said "We used to be the Evil Empire. Somebody else has that job now."
  - scholia 12y ago
    Nothing remotely like it, in terms of power, range, degree of market control, or willingness to be commercially evil. IBM used to be more than twice as big as every other IT company put together, and in some years, made more than 100% of the industry's profit. Even today, after disposing of numerous divisions, it's still a $100bn company, ie still larger than Microsoft. (So you pay IBM more than you pay MS every year, and you always have done.) However, it is true that Microsoft is the IBM of PC software, just as Google is the IBM of the web, Intel is the IBM of processors, Facebook is the IBM of social networking, Cisco is the IBM of routers, Oracle is trying to be the IBM of server software, Apple is the IBM of hipster status symbols... Well, you get the idea. IBM used to be the IBM of everything in IT, and started getting into other areas (telephone switches, copiers, cash machines etc) before the wheels started to come off. Today, Google is a lot more powerful than Microsoft, much scarier, and much more ambitious in IBM-like ways (self-driving cars, robots, superbrains etc).
andrea_s 12y ago
Am I alone in thinking that sql-like syntax is actually a step backwards from building query documents programmatically (MongoDB style)?
- haxor1232123 12y ago
  yes
- balls187 12y ago
  Not necessarily. I would say it allows flexibility in thinking about your storage and access patterns, which in turn allows flexibility in use. Hopefully without the expense of performance, and integrity.
- luuio 12y ago
  how? you're just doing the same thing with MongoDB in their own json-like syntax instead of the T-SQL that everyone knows
  - andrea_s 12y ago
    For simple queries there is virtually no difference (and clearly one should favor what he knows best). But there are cases where you need to build the query object programmatically (and dynamically), and I believe it's a bit awkward to do that to obtain an SQL statement. But possibly it's just my personal taste :)
- reubenbond 12y ago
  You can still build documents programmatically. The SQL-esque syntax is optional. The primary interface is HTTP.
  - andrea_s 12y ago
    Oh, I see, I've missed this detail on the first read. Thanks!
  - ahoge 12y ago
    > The SQL-esque syntax is optional. The primary interface is HTTP. Then the introduction needs some work. "DocumentDB enables complex ad hoc queries using the SQL dialect" "Azure DocumentDB offers the following key capabilities and benefits: Ad hoc queries with familiar SQL syntax" The "Query DocumentDB" article also seems to be focused on that SQL dialect.
streptomycin 12y ago
DocumentDB utilizes a highly concurrent, lock free, log structured indexing technology to automatically index all document content. This enables rich real-time queries without the need to specify schema hints, secondary indexes or views. How does that work? Isn't that going to incur a major performance hit? If not, why don't other databases get rid of indexes? Also, if anyone from MS is reading, http://azure.microsoft.com/en-us/documentation/articles/documentdb-introduction/ http://azure.microsoft.com/en-us/documentation/articles/docu... links to http://azure.microsoft.com/en-us/documentation/articles/documentdb-concepts/ http://azure.microsoft.com/en-us/documentation/articles/docu... which is a 404 error.
- andrea_s 12y ago
  One way to approach a behind the curtain automatic index generation would be to gather data about collection usage, and use such data to enable targeted indexes. I don't think it would be feasible to have "indexes for everything", as the numbers involved would scale geometrically.
  - arthurcolle 12y ago
    Forgive my ignorance but why would it scale geometrically? I tried to quickly think of a complexity analysis and I don't see where you are coming from.
    - andrea_s 12y ago
      Well, ordering matter in indexes - and clearly building a dedicated index for every field would not work, simplest queries aside. So if I have three fields - a, b and c - and by hypothesis I want to map all possible queries in a naive way, I will need to build, at the very least, the following indexes: [a, b, c] [a, c, b] [b, a, c] [b, c, a] [c, a, b] [c, b, a] all of which represent a different and valid way of querying my data. Anything less than this would leave some valid query uncovered by the indexes. These are 6 indexes, which is 3! (factorial). Add a fourth field and we have 12 indexes, and so on. Hence, geometrical growth (to be fair, factorial growth is even greater than geometrical). Which is why I'm thinking that either (i) indexes get created automatically based on the most frequent / heavy queries, or (ii) indexing works differently for DocumentDB and they are actually able to map the document space in a more efficient way (but I'd say that we lack the technical details to jump at this conclusion, at the moment).
utunga 12y ago
Another case of Not Invented Here syndrome from Microsoft. One wonders why they couldn't just take the open source and very well architected RavenDB http://ravendb.net http://ravendb.net .Net Document DB and provide first class support for that within Azure.
- Maarten88 12y ago
  RavenHQ was recently added as an Azure Add-On, you can use it now. But RavenHQ seems much more expensive than DocumentDB (even when doubling the "introduction pricing"), and it's not really integrated in Azure (I understand that it runs on Amazon, in a US datacenter only) But it will be very interesting to compare features, Raven really has a lot of advanced features.
- allegory 12y ago
  Because that's just as much of a ghetto, Microsoft can't control it as they'd have to buy it and I doubt Oren would sell it to them. Not only that, there are better products that are free.
- pistle 12y ago
  I'm going to posit licensing/cost issues as the easy downfall. However, something about the closed sourcing against the context of the open source movement from MS has me wondering about specific implementation details they feel are differentiating or expositions of brilliance/hacks?
allegory 12y ago
No local installation. No banana. I wouldn't tie a product to a single cloud vendor.
- kolev 12y ago
  So you're questioning AWS's success then?
- cheepin 12y ago
  nor one as expensive as azure
- threeseed 12y ago
  Absolutely. However there is an overwhelming trend towards hosting key parts of your infrastructure including data storage. Managing a database is surprisingly difficult in particular at scale.
- mariusmg 12y ago
  Yes, because all products today are designed and for multiple cloud vendors. Come on.......
  - weego 12y ago
    As the only source of DocumentDB is as an instance within Azure you are limited to that vendor. You can obviously take redis/monogo/couch and put them on any 'cloud' providers infrastructure as needs be. Not sure if your interpretation of his point was wilfully incorrect for some reason, but it was quite obvious what he meant.
MichaelGG 12y ago
I wonder what the search story is. One technology that really does deliver, and has totally impressed me, is Lucene/ElasticSearch. I'm used to all sorts of hyperbolic claims, but holy shit, ElasticSearch just delivers. We tossed in about 40M documents from a SQL Server DB, and not only did it require less resources (a 30%? reduction in size), the queries are beyond anything that'd be approachable using SQL Server. And I've only touched the surface, using it as a pure plug-n-play setup. With DocumentDB, not having a local version severely limits what I'd consider this for. Losing that flexibility is a big deal. Maybe this is just a limited preview and they haven't build the management side for local installs.
- curiousDog 12y ago
  They released a full-text search service as well today: http://azure.microsoft.com/en-us/services/search/ http://azure.microsoft.com/en-us/services/search/
- nemothekid 12y ago
  I doubt MS will ever release a non-hosted version. I see this product as a competitive offering to Amazon's Dynamo and Google's BigQuery on AppEngine.
  - bigtones 12y ago
    I think you mean a local version, or a non-hosted version.
    - nemothekid 12y ago
      Right, edited the parent post.
  - spicyj 12y ago
    BigQuery isn't really in the same space; it's meant for analytics queries where speed isn't as important (queries take seconds or minutes). Google Cloud Datastore is a better parallel.
deleted 12y ago
[deleted]
luuio 12y ago
A quick comparison between DocumentDB vs MongoDB: http://daprlabs.com/blog/blog/2014/08/22/azure-documentdb/ http://daprlabs.com/blog/blog/2014/08/22/azure-documentdb/
- spiderPig888 12y ago
  Wow, how'd you get one done so fast? Were you a private preview customer?
  - jeroen 12y ago
    OP did not write that. See here: https://news.ycombinator.com/item?id=8209525 https://news.ycombinator.com/item?id=8209525
cbsmith 12y ago
'cause what the world needs is another proprietary NoSQL solution.
- PerfectElement 12y ago
  As long as it offers something new, I don't see anything wrong with that. I've been working with MongoDB for the last year and DocumentDB features look very interesting to me.
  - cbsmith 12y ago
    Yeah, once you go down that road for a while, you'll understand how proprietary NoSQL is a giant PITA.
- james2vegas 12y ago
  'cause what the world needs is another pXrXoXpXrXiXeXtXaXrXyX NoSQL solution.
  - cbsmith 12y ago
    Are those X's supposed ^H's.
- hippich 12y ago
  I don't think it is about world. I would expect it is about http://en.wikipedia.org/wiki/Embrace,_extend_and_extinguish http://en.wikipedia.org/wiki/Embrace,_extend_and_extinguish And looking at their diagram[1] I already see where they can make this strategy work. While their collection/document/attachment structure is quite straightforward to move to another document DB, their UDF, sprocs and triggers I bet will be microsoft-specific. And microsoft will make everything possible to "optimize data fetching by running processing code close to data" and lure developers into using these. And in the end locking software solution to microsoft platform. [1] http://azure.microsoft.com/en-us/documentation/articles/documentdb-introduction/ http://azure.microsoft.com/en-us/documentation/articles/docu...
petilon 12y ago
So does it run on a cluster? If so which of Consistency, Availability and Partition tolerance does it NOT offer? (See CAP theorem)?
- semaphoreP 12y ago
  I don't think the CAP theorem is so black and white: http://en.wikipedia.org/wiki/CAP_theorem#2012 http://en.wikipedia.org/wiki/CAP_theorem#2012
- falsestprophet 12y ago
  It supports tunable consistency http://azure.microsoft.com/en-us/documentation/articles/documentdb-introduction/ http://azure.microsoft.com/en-us/documentation/articles/docu...
- curiousDog 12y ago
  They can all be provided to varying degrees. But usually most distributed systems sacrifice some availability to guarantee consistency (i.e avoid data loss at all costs)
chippy 12y ago
Spatial queries and indexing. Most data has some location component. I didn't see anything with this. Is it in there, or planned?
seanp2k2 12y ago
Interesting: https://github.com/Azure/azure-documentdb-python https://github.com/Azure/azure-documentdb-python (it's empty for the moment, but glad to see first-party support for Python)
- luuio 12y ago
  The .NET & JS repos are empty, too. I think they just have not published the code. It's still in Preview after all. https://github.com/Azure/azure-documentdb-net https://github.com/Azure/azure-documentdb-net
deleted 12y ago
[deleted]
lubos 12y ago
What are the limits of DocumentDB? You know, like max size of database, max size of document, max number of documents per database, max. number of attributes per document, max. number of databases per DocumentDB account. What's the max. duration of database query, max size of query result. What kind of performance can be expected, does it decrease as the size of database increases or it remains constant? I'm going to wait a few days until hype settles.
- reubenbond 12y ago
  Great suggestion, please vote here to make it a reality: http://feedback.azure.com/forums/263030-documentdb/suggestions/6329547-publish-documentdb-limits-and-thresholds-document http://feedback.azure.com/forums/263030-documentdb/suggestio...
- majorsc2noob 12y ago
  Most of this is documented since start. Did you actually read the docs? http://azure.microsoft.com/en-us/documentation/articles/documentdb-limits/ http://azure.microsoft.com/en-us/documentation/articles/docu...
  - lubos 12y ago
    I did read documentation but couldn't find it. I'm still not sure how did you find it other than going into their GitHub repository. Great find though. Thanks.
Nux 12y ago
A new "cool", locked-in service served on a silver platter by Microsoft to the brainwashed. Everybody else uses open source on premises or their cloud of choice.
poolpool 12y ago
I wonder if this is built on JetDB
- james2vegas 12y ago
  On Jet Blue (ESE)? It wouldn't be the first, RavenDB http://ravendb.net http://ravendb.net is based on ESE. I remember awhle back doing work on a system that (ab)used MS Exchange Server 5 as a database, mostly because of the Outlook integration.
gamesbrainiac 12y ago
I find it surprising that DocumentDB wasn't already a copyrighted name. ;)
reubenbond 12y ago
The @DocumentDB twitter links to a tutorial on DocDB: http://www.documentdb.com/sql/tutorial http://www.documentdb.com/sql/tutorial
fineline 12y ago
"All JavaScript logic is executed within an ambient ACID transaction with snapshot isolation. During the course of its execution, if the JavaScript throws an exception, then the entire transaction is aborted." Have I missed something, or have MS delivered a novel and valuable feature? I'm not aware of support for transactions across documents in other NoSQL platforms. I'd be grateful if someone has any experience or better information in that regard, thanks.
- int64 12y ago
  You are missing AmisaDB. Supports Multistatement transactions. http://www.amisalabs.com/ http://www.amisalabs.com/
  - fineline 12y ago
    No I didn't know about that, thanks. I guess I was more thinking about run-on-your-own-hardware/cloud - which I know is not equivalent to the MS offering. I see orands post about FoundationDB, which also looks interesting. Seems like it would be a killer feature for a Mongo/Rethink/Couch type system.
  - tristanz 12y ago
    It appears that AmisaDB is in-memory only, only supports read committed isolation, and has limited transactions semantics (no arbitrary JS). This is pretty far from what DocumentDB seems to offer. http://www.amisalabs.com/AmisaDB_Docs.html http://www.amisalabs.com/AmisaDB_Docs.html
- nikon 12y ago
  RavenDB http://ravendb.net/docs/client-api/advanced/transaction-support http://ravendb.net/docs/client-api/advanced/transaction-supp...
- savecastel 12y ago
  FoundationDB https://foundationdb.com/ https://foundationdb.com/ Quite an amazing piece of technology, check how they test cluster failures, there is a blog post about it :)
orand 12y ago
If I understand correctly, their multi-document ACID transaction support is a big deal. The only other NoSQL/NewSQL systems I'm aware of with that ability are FoundationDB and Google Spanner/F1.
- int64 12y ago
  So does AmisaDB. http://www.amisalabs.com/ http://www.amisalabs.com/
- platform 12y ago
  yes, it is a big deal MarkLogic does this too (I think they have been doing it since '03 or something like that)( http://www.marklogic.com/blog/acid-transactions-check/ http://www.marklogic.com/blog/acid-transactions-check/
- nikon 12y ago
  RavenDB http://ravendb.net/docs/client-api/advanced/transaction-support http://ravendb.net/docs/client-api/advanced/transaction-supp...
hackerkushal 12y ago
THIS THING IS A BEAST!! It is absolutely bad ass
whalesalad 12y ago
I liked everything about it until I saw the API for the Python client. What a catastrophe. I pray Microsoft is looking for Python developers: https://gist.github.com/whalesalad/2142f0075c6896f4547c https://gist.github.com/whalesalad/2142f0075c6896f4547c
- ayrx 12y ago
  Wow, the function naming alone is terrible. for i in range(len(path_parts) - 1, -1, -1): if not path_parts[i] in resource_types and path_parts[i] in resource_tokens: return resource_tokens[path_parts[i]] This bit of iteration is painful to look at as well.
  - jkldotio 12y ago
    Some optimisations to get it to web-scale on TempleOS were obviously necessary.
- maaaats 12y ago
  I didn't compare the code, but maybe it's autogenerated from a more C-like language, making a bit awkward syntax for a language like Python?
- blablabla123 12y ago
  The Client APIs are kind of free: https://github.com/Azure/azure-sdk-for-python/blob/master/LICENSE.txt https://github.com/Azure/azure-sdk-for-python/blob/master/LI...
mallipeddi 12y ago
What are the size limits on a collection? Docs mention transaction support is offered only within a collection. Is a collection essentially limited to a single physical machine in the background or does it span across machines? It looks like in Standard Preview, the max collection size is 10GB.
nandkishiee 12y ago
Sick! Love it
sarciszewski 12y ago
Leave it to Microsoft to give it the most generic sounding name possible.
- josteink 12y ago
  Yeah. Instead of giving it a reasonable, obvious name they should have gone with something hipstery and vague instead, like UberDcmntor.io. No thanks. I think this will do just fine.
  - rtkwe 12y ago
    No no no, it also has to be just one common word so that it's completely ungoogleable by itself.

DocumentDB

300 points by daigoba66 12y ago

yxhuvud 12y ago

It would have been nice to see some actual details of how it works so that it can be compared to the competition.

orf 12y ago
First link on the page: "introduction to DocumentDB". http://azure.microsoft.com/en-us/documentation/articles/documentdb-introduction/ http://azure.microsoft.com/en-us/documentation/articles/docu...

cvburgess 12y ago

Does anyone know how this compares to AWS DynamoDB[1] ? [1] https://aws.amazon.com/dynamodb/ https://aws.amazon.com/dynamodb/

luuio 12y ago
This allows you to use SQL queries, and built in index support
- cvburgess 12y ago
  Very cool features, thanks!
balls187 12y ago
DynamoDB really isn't document oriented storage. Instead it's a key-value wide column store, with 1st and 2nd level index support. I haven't delved into the MSFT pricing model, but DynamoDB is pay-for-through put. You provision your table with a certain amount of read/write performance that is guaranteed, and you pay for that.
- cvburgess 12y ago
  I'd assuming pricing will be similar and incredibly competitive... and thanks, didn't realize how limited dynamoDB was
ryanfitz 12y ago
I've been using DynamoDB since it was released. I wrote a nodejs driver[1] and a nodejs data mapper for it[2], so I have a decent bit of experience with it. Browsing the DocumentDB docs the two services seem to be very different. DynamoDB is really just a key value store with some very nice properties, but also a lot of tradeoffs. One such tradeoff is querying data is very limited in DynamoDB. In Dynamo you can only query data by its primary hash key and optional range key. These keys you must specify upfront when you create your table and cannot be changed afterwards. DocumentDB seems much more similar to mongodb and appears to have a very flexible query ability. In my opinion, one of the best features with DynamoDB is you can tune the number of reads/writes each individual table requires. This lets you scale up and down your database and greatly helps keep costs down. This is a feature that only a hosted database service can offer. I haven't yet read any pricing info on DocumentDB, but hopefully they offer a similar feature as this is really where a hosted database service can shine. [1]https://github.com/Wantworthy/dynode https://github.com/Wantworthy/dynode [2]https://github.com/ryanfitz/vogels https://github.com/ryanfitz/vogels
- cvburgess 12y ago
  Thanks, yeah I can see how this is more like hosted mongoDB - and I've seen dynode on npm, nice work!

pokstad 12y ago

Sounds very similar to CouchDB. Server side Javascript written by the user, and an HTTP interface. The ability to adjust consistency is really neat.

Goranek 12y ago
similar to cloudantdb a little bit more because of searching and indexing.
andrewguenther 12y ago
The ability to adjust consistency isn't really a new concept. The first I saw it was in Amazon's DynamoDB paper in 2007. Cassandra uses the model outlined there as well, and I'm sure there are others out there that I'm less familiar with.
ahoge 12y ago
CouchDB doesn't support ad hoc queries. CouchDB is all about MapReduce and heavy caching. It's very rigid.

bkeroack 12y ago

...and MS goes after MongoDB. It would be nice to see an on-premises version, if only to compare performance/consistency with Mongo.

codereflection 12y ago
Not really so far, it's Azure only.
taude 12y ago
I have to believe it's coming, since a lot of MSFTs core business is still selling server-licenses for business who host their own data centers. I for one would love to use something like this, and our company is one that's not comfortable with third-party open-source tools, and we have to host in our own data center...
reubenbond 12y ago
DocDB is built atop a very well battle-tested distributed systems framework with replication based on a multi-Paxos implementation. I haven't done any benchmarks, but the replication model is far superior to MongoDB's "journal journal" model for performance. http://daprlabs.com/blog/blog/2014/08/22/azure-documentdb/ http://daprlabs.com/blog/blog/2014/08/22/azure-documentdb/
- dougws 12y ago
  Your objections to MongoDB's model seem reasonable, but I don't see any evidence in either this comment or the linked blog post that DocumentDB is better (especially in the absence of benchmarks). What is this "battle-tested distributed systems framework"? Several of your complaints about MongoDB have to do with the interaction between persistence to disk and replication to the network; as the Multi-Paxos algorithm does not specify when data should be written to disk (much less what the format should be), what reason is there to believe that DocumentDB does this any better? I'm totally willing to believe that DocumentDB beats the pants of MongoDB on just about every axis (in fact, that seems pretty likely) but it's going to take some actual numbers and a better description of the internals.

jnardiello 12y ago

And that is a creative product name. Well played MS.

talles 12y ago
Just like SQL Server
programminggeek 12y ago
It's really smart. It's a document database, so they call it DocumentDB. It's better than calling it something like Dolphin and then explaining that it's a document database. Simple and clear is a good thing.
- allegory 12y ago
  Err, Apple: Airport, Bootcamp, Safari, Time Machine, FaceTime, Grand Central Dispatch, QuickTime... and I won't even start on the open source side of things...
- talles 12y ago
  But it's also kinda selfish don't you think?
  - bdcravens 12y ago
    No more than AWS's SimpleDB.
- tomsthumb 12y ago
  "Microsoft has DocumentDB service now." "Oh cool, which document database is it? MongoDB? CouchDB? Cassandra?" "It's DocumentDB!" "...."
  - jaredmcateer 12y ago
    Just like trying to get help around libraries of Prototype or Async in regards to Javascript. 90% of the results (I'm exagerating) are jQuery

jpalomaki 12y ago

Ad hoc queries using SQL like syntax. No need to define indexes. Javascript execution within database. Stored procedures, triggers and functions can be written with Javascript. "All JavaScript logic is executed within an ambient ACID transaction with snapshot isolation. During the course of its execution, if the JavaScript throws an exception, then the entire transaction is aborted." Pricing is based on "capacity units". Starts with $22.50 per month (this includes 50% preview period discount). One capacity unit (CU) gives 10GB of storage and can perform 2000 reads per second, 500 insert/replace/delete, 1000 simple queries returning one doc. In order to see pricing details, change the region to "US West": http://azure.microsoft.com/en-us/pricing/details/documentdb/ http://azure.microsoft.com/en-us/pricing/details/documentdb/ Very interesting addition to Microsoft offering. I was actually just yesterday wondering if they have any plans for this kind of service. Table Storage is quite primitive and Azure SQL on the other hand gets expensive when you have lots of data. One potential "problem" with this is the bundling of storage capacity and processing power. If I understand this correctly, I would need to buy 10 CUs per month to store 100GB of data even if I'm not very actively using that data.

jnardiello 12y ago
That is actually good to start with, yet scaling will cost A LOT.

talles 12y ago

Can I use DocumentDB out of Azure (hook my own)?

twotwotwo 12y ago
You know, it's not a simple question, but there's some case for Microsoft just up and releasing some scale-out DB for free (as in beer) or at least cheap. My thinking is, scale-out deployments aren't all that likely to pay at SQL-Server-like rates per CPU anyway, and you're helping give the Windows world better parity with the horde of Linux options for folks that need scale-out (or, just as commonly, hope to need it someday). On the flipside, by doing that you might forego some sales of SQL Server (though I suspect that's limited; most folks that need SQL Server really need SQL Server) or sales on Azure that could theoretically help recover the dev costs. But greatly improving the scale-out-on-Windows story seems like a big deal, the kind of thing that might justify going to lots of effort to make a DB product then giving it away.
- aikah 12y ago
  > You know, it's not a simple question Yes it is a simple question, and the answer is NO.
  - deleted 12y ago
    [deleted]

ceejayoz 12y ago

http://azure.microsoft.com/en-us/documentation/articles/documentdb-introduction/ http://azure.microsoft.com/en-us/documentation/articles/docu... > Want to edit or suggest changes to this content? You can edit and submit changes to this article using GitHub. Pretty remarkable given Microsoft's approach to open source in the 1990s that they're now using a service built around Linus's bespoke open source version control system to allow people to suggest changes to their documentation.

pjmlp 12y ago
Microsoft is the new IBM. :)
- braum 12y ago
  Agree and I've been saying it for ten damn years, havn't I been saying it!
  - scholia 12y ago
    See my comment above. Actually, there is one way that Microsoft did become the new IBM. Long ago, I went to a talk by a senior IBMer and he said "We used to be the Evil Empire. Somebody else has that job now."
- scholia 12y ago
  Nothing remotely like it, in terms of power, range, degree of market control, or willingness to be commercially evil. IBM used to be more than twice as big as every other IT company put together, and in some years, made more than 100% of the industry's profit. Even today, after disposing of numerous divisions, it's still a $100bn company, ie still larger than Microsoft. (So you pay IBM more than you pay MS every year, and you always have done.) However, it is true that Microsoft is the IBM of PC software, just as Google is the IBM of the web, Intel is the IBM of processors, Facebook is the IBM of social networking, Cisco is the IBM of routers, Oracle is trying to be the IBM of server software, Apple is the IBM of hipster status symbols... Well, you get the idea. IBM used to be the IBM of everything in IT, and started getting into other areas (telephone switches, copiers, cash machines etc) before the wheels started to come off. Today, Google is a lot more powerful than Microsoft, much scarier, and much more ambitious in IBM-like ways (self-driving cars, robots, superbrains etc).

andrea_s 12y ago

Am I alone in thinking that sql-like syntax is actually a step backwards from building query documents programmatically (MongoDB style)?

haxor1232123 12y ago
yes
balls187 12y ago
Not necessarily. I would say it allows flexibility in thinking about your storage and access patterns, which in turn allows flexibility in use. Hopefully without the expense of performance, and integrity.
luuio 12y ago
how? you're just doing the same thing with MongoDB in their own json-like syntax instead of the T-SQL that everyone knows
- andrea_s 12y ago
  For simple queries there is virtually no difference (and clearly one should favor what he knows best). But there are cases where you need to build the query object programmatically (and dynamically), and I believe it's a bit awkward to do that to obtain an SQL statement. But possibly it's just my personal taste :)
reubenbond 12y ago
You can still build documents programmatically. The SQL-esque syntax is optional. The primary interface is HTTP.
- andrea_s 12y ago
  Oh, I see, I've missed this detail on the first read. Thanks!
- ahoge 12y ago
  > The SQL-esque syntax is optional. The primary interface is HTTP. Then the introduction needs some work. "DocumentDB enables complex ad hoc queries using the SQL dialect" "Azure DocumentDB offers the following key capabilities and benefits: Ad hoc queries with familiar SQL syntax" The "Query DocumentDB" article also seems to be focused on that SQL dialect.

streptomycin 12y ago

DocumentDB utilizes a highly concurrent, lock free, log structured indexing technology to automatically index all document content. This enables rich real-time queries without the need to specify schema hints, secondary indexes or views. How does that work? Isn't that going to incur a major performance hit? If not, why don't other databases get rid of indexes? Also, if anyone from MS is reading, http://azure.microsoft.com/en-us/documentation/articles/documentdb-introduction/ http://azure.microsoft.com/en-us/documentation/articles/docu... links to http://azure.microsoft.com/en-us/documentation/articles/documentdb-concepts/ http://azure.microsoft.com/en-us/documentation/articles/docu... which is a 404 error.

andrea_s 12y ago
One way to approach a behind the curtain automatic index generation would be to gather data about collection usage, and use such data to enable targeted indexes. I don't think it would be feasible to have "indexes for everything", as the numbers involved would scale geometrically.
- arthurcolle 12y ago
  Forgive my ignorance but why would it scale geometrically? I tried to quickly think of a complexity analysis and I don't see where you are coming from.
  - andrea_s 12y ago
    Well, ordering matter in indexes - and clearly building a dedicated index for every field would not work, simplest queries aside. So if I have three fields - a, b and c - and by hypothesis I want to map all possible queries in a naive way, I will need to build, at the very least, the following indexes: [a, b, c] [a, c, b] [b, a, c] [b, c, a] [c, a, b] [c, b, a] all of which represent a different and valid way of querying my data. Anything less than this would leave some valid query uncovered by the indexes. These are 6 indexes, which is 3! (factorial). Add a fourth field and we have 12 indexes, and so on. Hence, geometrical growth (to be fair, factorial growth is even greater than geometrical). Which is why I'm thinking that either (i) indexes get created automatically based on the most frequent / heavy queries, or (ii) indexing works differently for DocumentDB and they are actually able to map the document space in a more efficient way (but I'd say that we lack the technical details to jump at this conclusion, at the moment).

utunga 12y ago

Another case of Not Invented Here syndrome from Microsoft. One wonders why they couldn't just take the open source and very well architected RavenDB http://ravendb.net http://ravendb.net .Net Document DB and provide first class support for that within Azure.

Maarten88 12y ago
RavenHQ was recently added as an Azure Add-On, you can use it now. But RavenHQ seems much more expensive than DocumentDB (even when doubling the "introduction pricing"), and it's not really integrated in Azure (I understand that it runs on Amazon, in a US datacenter only) But it will be very interesting to compare features, Raven really has a lot of advanced features.
allegory 12y ago
Because that's just as much of a ghetto, Microsoft can't control it as they'd have to buy it and I doubt Oren would sell it to them. Not only that, there are better products that are free.
pistle 12y ago
I'm going to posit licensing/cost issues as the easy downfall. However, something about the closed sourcing against the context of the open source movement from MS has me wondering about specific implementation details they feel are differentiating or expositions of brilliance/hacks?

allegory 12y ago

No local installation. No banana. I wouldn't tie a product to a single cloud vendor.

kolev 12y ago
So you're questioning AWS's success then?
cheepin 12y ago
nor one as expensive as azure
threeseed 12y ago
Absolutely. However there is an overwhelming trend towards hosting key parts of your infrastructure including data storage. Managing a database is surprisingly difficult in particular at scale.
mariusmg 12y ago
Yes, because all products today are designed and for multiple cloud vendors. Come on.......
- weego 12y ago
  As the only source of DocumentDB is as an instance within Azure you are limited to that vendor. You can obviously take redis/monogo/couch and put them on any 'cloud' providers infrastructure as needs be. Not sure if your interpretation of his point was wilfully incorrect for some reason, but it was quite obvious what he meant.

MichaelGG 12y ago

I wonder what the search story is. One technology that really does deliver, and has totally impressed me, is Lucene/ElasticSearch. I'm used to all sorts of hyperbolic claims, but holy shit, ElasticSearch just delivers. We tossed in about 40M documents from a SQL Server DB, and not only did it require less resources (a 30%? reduction in size), the queries are beyond anything that'd be approachable using SQL Server. And I've only touched the surface, using it as a pure plug-n-play setup. With DocumentDB, not having a local version severely limits what I'd consider this for. Losing that flexibility is a big deal. Maybe this is just a limited preview and they haven't build the management side for local installs.

curiousDog 12y ago
They released a full-text search service as well today: http://azure.microsoft.com/en-us/services/search/ http://azure.microsoft.com/en-us/services/search/
nemothekid 12y ago
I doubt MS will ever release a non-hosted version. I see this product as a competitive offering to Amazon's Dynamo and Google's BigQuery on AppEngine.
- bigtones 12y ago
  I think you mean a local version, or a non-hosted version.
  - nemothekid 12y ago
    Right, edited the parent post.
- spicyj 12y ago
  BigQuery isn't really in the same space; it's meant for analytics queries where speed isn't as important (queries take seconds or minutes). Google Cloud Datastore is a better parallel.

deleted 12y ago

[deleted]

luuio 12y ago

A quick comparison between DocumentDB vs MongoDB: http://daprlabs.com/blog/blog/2014/08/22/azure-documentdb/ http://daprlabs.com/blog/blog/2014/08/22/azure-documentdb/

spiderPig888 12y ago
Wow, how'd you get one done so fast? Were you a private preview customer?
- jeroen 12y ago
  OP did not write that. See here: https://news.ycombinator.com/item?id=8209525 https://news.ycombinator.com/item?id=8209525

cbsmith 12y ago

'cause what the world needs is another proprietary NoSQL solution.

PerfectElement 12y ago
As long as it offers something new, I don't see anything wrong with that. I've been working with MongoDB for the last year and DocumentDB features look very interesting to me.
- cbsmith 12y ago
  Yeah, once you go down that road for a while, you'll understand how proprietary NoSQL is a giant PITA.
james2vegas 12y ago
'cause what the world needs is another pXrXoXpXrXiXeXtXaXrXyX NoSQL solution.
- cbsmith 12y ago
  Are those X's supposed ^H's.
hippich 12y ago
I don't think it is about world. I would expect it is about http://en.wikipedia.org/wiki/Embrace,_extend_and_extinguish http://en.wikipedia.org/wiki/Embrace,_extend_and_extinguish And looking at their diagram[1] I already see where they can make this strategy work. While their collection/document/attachment structure is quite straightforward to move to another document DB, their UDF, sprocs and triggers I bet will be microsoft-specific. And microsoft will make everything possible to "optimize data fetching by running processing code close to data" and lure developers into using these. And in the end locking software solution to microsoft platform. [1] http://azure.microsoft.com/en-us/documentation/articles/documentdb-introduction/ http://azure.microsoft.com/en-us/documentation/articles/docu...

petilon 12y ago

So does it run on a cluster? If so which of Consistency, Availability and Partition tolerance does it NOT offer? (See CAP theorem)?

semaphoreP 12y ago
I don't think the CAP theorem is so black and white: http://en.wikipedia.org/wiki/CAP_theorem#2012 http://en.wikipedia.org/wiki/CAP_theorem#2012
falsestprophet 12y ago
It supports tunable consistency http://azure.microsoft.com/en-us/documentation/articles/documentdb-introduction/ http://azure.microsoft.com/en-us/documentation/articles/docu...
curiousDog 12y ago
They can all be provided to varying degrees. But usually most distributed systems sacrifice some availability to guarantee consistency (i.e avoid data loss at all costs)

chippy 12y ago

Spatial queries and indexing. Most data has some location component. I didn't see anything with this. Is it in there, or planned?

seanp2k2 12y ago

Interesting: https://github.com/Azure/azure-documentdb-python https://github.com/Azure/azure-documentdb-python (it's empty for the moment, but glad to see first-party support for Python)

luuio 12y ago
The .NET & JS repos are empty, too. I think they just have not published the code. It's still in Preview after all. https://github.com/Azure/azure-documentdb-net https://github.com/Azure/azure-documentdb-net

deleted 12y ago

[deleted]

lubos 12y ago

What are the limits of DocumentDB? You know, like max size of database, max size of document, max number of documents per database, max. number of attributes per document, max. number of databases per DocumentDB account. What's the max. duration of database query, max size of query result. What kind of performance can be expected, does it decrease as the size of database increases or it remains constant? I'm going to wait a few days until hype settles.

reubenbond 12y ago
Great suggestion, please vote here to make it a reality: http://feedback.azure.com/forums/263030-documentdb/suggestions/6329547-publish-documentdb-limits-and-thresholds-document http://feedback.azure.com/forums/263030-documentdb/suggestio...
majorsc2noob 12y ago
Most of this is documented since start. Did you actually read the docs? http://azure.microsoft.com/en-us/documentation/articles/documentdb-limits/ http://azure.microsoft.com/en-us/documentation/articles/docu...
- lubos 12y ago
  I did read documentation but couldn't find it. I'm still not sure how did you find it other than going into their GitHub repository. Great find though. Thanks.

Nux 12y ago

A new "cool", locked-in service served on a silver platter by Microsoft to the brainwashed. Everybody else uses open source on premises or their cloud of choice.

poolpool 12y ago

I wonder if this is built on JetDB

james2vegas 12y ago
On Jet Blue (ESE)? It wouldn't be the first, RavenDB http://ravendb.net http://ravendb.net is based on ESE. I remember awhle back doing work on a system that (ab)used MS Exchange Server 5 as a database, mostly because of the Outlook integration.

gamesbrainiac 12y ago

I find it surprising that DocumentDB wasn't already a copyrighted name. ;)

reubenbond 12y ago

The @DocumentDB twitter links to a tutorial on DocDB: http://www.documentdb.com/sql/tutorial http://www.documentdb.com/sql/tutorial

fineline 12y ago

"All JavaScript logic is executed within an ambient ACID transaction with snapshot isolation. During the course of its execution, if the JavaScript throws an exception, then the entire transaction is aborted." Have I missed something, or have MS delivered a novel and valuable feature? I'm not aware of support for transactions across documents in other NoSQL platforms. I'd be grateful if someone has any experience or better information in that regard, thanks.

int64 12y ago
You are missing AmisaDB. Supports Multistatement transactions. http://www.amisalabs.com/ http://www.amisalabs.com/
- fineline 12y ago
  No I didn't know about that, thanks. I guess I was more thinking about run-on-your-own-hardware/cloud - which I know is not equivalent to the MS offering. I see orands post about FoundationDB, which also looks interesting. Seems like it would be a killer feature for a Mongo/Rethink/Couch type system.
- tristanz 12y ago
  It appears that AmisaDB is in-memory only, only supports read committed isolation, and has limited transactions semantics (no arbitrary JS). This is pretty far from what DocumentDB seems to offer. http://www.amisalabs.com/AmisaDB_Docs.html http://www.amisalabs.com/AmisaDB_Docs.html
nikon 12y ago
RavenDB http://ravendb.net/docs/client-api/advanced/transaction-support http://ravendb.net/docs/client-api/advanced/transaction-supp...
savecastel 12y ago
FoundationDB https://foundationdb.com/ https://foundationdb.com/ Quite an amazing piece of technology, check how they test cluster failures, there is a blog post about it :)

orand 12y ago

If I understand correctly, their multi-document ACID transaction support is a big deal. The only other NoSQL/NewSQL systems I'm aware of with that ability are FoundationDB and Google Spanner/F1.

int64 12y ago
So does AmisaDB. http://www.amisalabs.com/ http://www.amisalabs.com/
platform 12y ago
yes, it is a big deal MarkLogic does this too (I think they have been doing it since '03 or something like that)( http://www.marklogic.com/blog/acid-transactions-check/ http://www.marklogic.com/blog/acid-transactions-check/
nikon 12y ago
RavenDB http://ravendb.net/docs/client-api/advanced/transaction-support http://ravendb.net/docs/client-api/advanced/transaction-supp...

hackerkushal 12y ago

THIS THING IS A BEAST!! It is absolutely bad ass

whalesalad 12y ago

I liked everything about it until I saw the API for the Python client. What a catastrophe. I pray Microsoft is looking for Python developers: https://gist.github.com/whalesalad/2142f0075c6896f4547c https://gist.github.com/whalesalad/2142f0075c6896f4547c

ayrx 12y ago
Wow, the function naming alone is terrible. for i in range(len(path_parts) - 1, -1, -1): if not path_parts[i] in resource_types and path_parts[i] in resource_tokens: return resource_tokens[path_parts[i]] This bit of iteration is painful to look at as well.
- jkldotio 12y ago
  Some optimisations to get it to web-scale on TempleOS were obviously necessary.
maaaats 12y ago
I didn't compare the code, but maybe it's autogenerated from a more C-like language, making a bit awkward syntax for a language like Python?
blablabla123 12y ago
The Client APIs are kind of free: https://github.com/Azure/azure-sdk-for-python/blob/master/LICENSE.txt https://github.com/Azure/azure-sdk-for-python/blob/master/LI...

mallipeddi 12y ago

What are the size limits on a collection? Docs mention transaction support is offered only within a collection. Is a collection essentially limited to a single physical machine in the background or does it span across machines? It looks like in Standard Preview, the max collection size is 10GB.

nandkishiee 12y ago

Sick! Love it

sarciszewski 12y ago

Leave it to Microsoft to give it the most generic sounding name possible.

josteink 12y ago
Yeah. Instead of giving it a reasonable, obvious name they should have gone with something hipstery and vague instead, like UberDcmntor.io. No thanks. I think this will do just fine.
- rtkwe 12y ago
  No no no, it also has to be just one common word so that it's completely ungoogleable by itself.