4 ms·

Well, Azure has been offering a similar NoSQL service for a few years now. http://sigops.org/sosp/sosp11/current/2011-Cascais/printable/11-calder.pdf http://si

by balakk 15y ago

Well, Azure has been offering a similar NoSQL service for a few years now. http://sigops.org/sosp/sosp11/current/2011-Cascais/printable/11-calder.pdf http://sigops.org/sosp/sosp11/current/2011-Cascais/printable...

ridruejo 15y ago
No. They offer the equivalent of Amazon S3, SimpleDB and SQS, but nothing comparable to this
- balakk 15y ago
  Can you elaborate how this is different and not comparable? Azure's table service offers the same automatic partition management, unlimited per-table scalability, composite keys, range queries, and availability guarantees. The linked paper goes into more details.
  - BradCalder 15y ago
    Balakk is correct. There are a lot of similarities between Windows Azure Tables and DynamoDB, and the release of DynamoDB validates the Data Model we have provided for a few years now with Azure Tables • They both are NoSQL schema-less table stores, where a table can store entities with completely different properties • They have a two attribute (property) composite primary key.One property that is used for partitioning and the other property is for optimizing range based operations within a partition • Both of them have just a single index based on their composite primary key • Both are built for effectively unlimited table size, seamlessly auto scale out with hands off management • Similar CRUD operations How Windows Azure Tables is implemented can be found in this SOSP paper and talk: http://blogs.msdn.com/b/windowsazurestorage/archive/2011/11/20/windows-azure-storage-a-highly-available-cloud-storage-service-with-strong-consistency.aspx http://blogs.msdn.com/b/windowsazurestorage/archive/2011/11/... As mentioned by someone else, one difference is that DynamoDB stores its data completely in SSDs, whereas, in Azure Storage our writes are committed via journaling (to either SSD or a dedicated journal drive) and reads are served from disks or memory if the data page is cached in memory. Therefore, the latency for single entity small writes are typically below 10ms due to our journaling approach (described in the above SOSP paper). Then single entity read times for small entities are typically under 40ms, which is shown in the results here: http://blogs.msdn.com/b/windowsazurestorage/archive/2010/11/06/how-to-get-most-out-of-windows-azure-tables.aspx http://blogs.msdn.com/b/windowsazurestorage/archive/2010/11/... Once and awhile we see someone saying that they see 100+ms latencies for small single entity reads and that is usually because they need to turn Nagle off, as described here: http://blogs.msdn.com/b/windowsazurestorage/archive/2010/06/25/nagle-s-algorithm-is-not-friendly-towards-small-requests.aspx http://blogs.msdn.com/b/windowsazurestorage/archive/2010/06/...
  - iampims 15y ago
    This is running on SSDs and it makes a HUGE difference.
  - vyrotek 15y ago
    Besides the points I already complained about before... How about 200ms response times even when performing a query using the Row & Partition Keys. I'm not sure if by composite keys you were referring to something other than the RK & PK because those are the only indexes you get.
    - balakk 15y ago
      ATS response times within the Azure data center are pretty impressive in my experience. Your partition keys can be composite, have a look here: http://blogs.msdn.com/b/windowsazurestorage/archive/2010/11/06/how-to-get-most-out-of-windows-azure-tables.aspx http://blogs.msdn.com/b/windowsazurestorage/archive/2010/11/... I agree with your other pain points - in terms of not being able to get counts, secondary indices etc. However, you can easily simulate some of those - maintain your own summary tables, indices and so on. These ought to emerge as platform features pretty soon though. It's not perfect, but its feature set is close to Dynamo. As for Mongo DB, I guess this service has been built from ground-up to provide the availability guarantees and automatic partition management features. I don't know if Mongo provides those. You could run Mongo yourself on Azure if you wanted to; there's even a supported solution done recently.
      - vyrotek 15y ago
        Hmm, I guess when I think about composite keys I think of ways to indicate a specific field/column as being part of the key. Data duplication along with string concatenation aren't really an elegant way to do it. If I remember right you also can't update the key values once the record has been saved. This is coming from a big SQL guy though :)
- mattcole 15y ago
  How is this vastly different to Azure Tables?
  - rojabuck 15y ago
    The cost per transaction, performance & ease. Reads per $0.01 = (50.60).60 = 180000 Writes per $0.01 = (10.60).60 = 36000 Assuming that you hit your usage is at 100% capacity then from a read prospective DynamoDB is half the price. Writes are much more expensive but many applications are heavily read oriented. DynamoDB claims single digit millisecond reads, azure tables does not (from my experience.) Azure tables have a maximum performance over a given partition table of 500 requests per second and over the whole account of 5,000 requests per second. DynamoDB does not state this. http://blogs.msdn.com/b/windowsazurestorage/archive/2010/05/10/windows-azure-storage-abstractions-and-their-scalability-targets.aspx http://blogs.msdn.com/b/windowsazurestorage/archive/2010/05/... To put this into context: Assume a system with 5000 writes per second and 50000 reads here are the costs: AWS Reads: $240 AWS Writes: $120 Aws Total: $360 Azure Reads: $4320 Azure Writes: $432 Azure Total: $4752 Seems like quite a difference for a decent sized read heavy application.
    - balakk 15y ago
      Can you please explain your math? AFAIK Azure txns are not paid by the hour - they are a flat cost of $.01 per 10000 storage txns. If you do batched GETs and PUTs you make only 550 txns (55000/100 entites/batch). http://www.windowsazure.com/en-us/home/tour/storage/ http://www.windowsazure.com/en-us/home/tour/storage/ I agree that Dynamo's provisioned throughput capacity is a very useful feature though. Azure does not provide any such performance guarantee; the throughput limit is also a guideline as far as i know, not an absolute barrier.
      - rojabuck 15y ago
        I should have explained that my costs were calculated on a "per day" assumption. Thus the costs are for: 5000 x 60 x 60 x 24 = 432000000 Writes 50000 x 60 x 60 x 24 = 4320000000 Reads (432000000/10000) x 0.01 = $432 (4320000000/10000) x 0.01 = $4320 Azure Total Cost For One Days Use: $4752 ((5000/10) x 0.01) x 24 = $120 ((50000/50) x 0.01) x 24 = $240 AWS Total Cost For One Days Use: $360 You are right that I don't take into account the bulk feature of azure reads & writes but this is down to bulk requests only being possible on a single partition at a time which in my personal experience (not exhaustive) is non-trivial to take advantage of.
vyrotek 15y ago
I've been using Azure Table Storage since the beginning and this doesn't seem to be the same. Like others have mentioned TS is more similar to SimpleDB. Now, I would love for someone to give me a tl;dr on the the feature set of DynamoDB so I can make an accurate comparison. Table Storage does not allow any other indexes other than the main primary ones (Row Key and Partition Key). You also cannot store complex object within fields and use them in a query. You basically just serialize the data and stuff it into the field. The dynamic schema is very nice if you can leverage it but the actually query support is TERRIBLE. (Sorry Microsoft, I'm a fanboy but you blew it here). There is no Order By or event Count support which makes a lot of things very difficult. Want to know how many "color=green" records there are? Guess what, you're going to retrieve all those rows and then count them yourself. They're starting to listen to the community and have just recently introduced upserts and projection (select). I would love to see them adopt something like MongoDB instead :) For more issues check out: http://www.mygreatwindowsazureidea.com/forums/34192-windows-azure-feature-voting/suggestions/415607-add-tablestorage-linq-query-support-for-select-co http://www.mygreatwindowsazureidea.com/forums/34192-windows-... Edit - For what its worth. We've moved more things to SQL Azure now that it has Federation support. Scalability with the power of SQL. http://blogs.msdn.com/b/windowsazure/archive/2011/12/13/building-large-scale-elastic-database-tiers-with-sql-azure-introducing-federations.aspx http://blogs.msdn.com/b/windowsazure/archive/2011/12/13/buil...

by balakk 15y ago

ridruejo 15y ago

No. They offer the equivalent of Amazon S3, SimpleDB and SQS, but nothing comparable to this

balakk 15y ago
Can you elaborate how this is different and not comparable? Azure's table service offers the same automatic partition management, unlimited per-table scalability, composite keys, range queries, and availability guarantees. The linked paper goes into more details.
- BradCalder 15y ago
  Balakk is correct. There are a lot of similarities between Windows Azure Tables and DynamoDB, and the release of DynamoDB validates the Data Model we have provided for a few years now with Azure Tables • They both are NoSQL schema-less table stores, where a table can store entities with completely different properties • They have a two attribute (property) composite primary key.One property that is used for partitioning and the other property is for optimizing range based operations within a partition • Both of them have just a single index based on their composite primary key • Both are built for effectively unlimited table size, seamlessly auto scale out with hands off management • Similar CRUD operations How Windows Azure Tables is implemented can be found in this SOSP paper and talk: http://blogs.msdn.com/b/windowsazurestorage/archive/2011/11/20/windows-azure-storage-a-highly-available-cloud-storage-service-with-strong-consistency.aspx http://blogs.msdn.com/b/windowsazurestorage/archive/2011/11/... As mentioned by someone else, one difference is that DynamoDB stores its data completely in SSDs, whereas, in Azure Storage our writes are committed via journaling (to either SSD or a dedicated journal drive) and reads are served from disks or memory if the data page is cached in memory. Therefore, the latency for single entity small writes are typically below 10ms due to our journaling approach (described in the above SOSP paper). Then single entity read times for small entities are typically under 40ms, which is shown in the results here: http://blogs.msdn.com/b/windowsazurestorage/archive/2010/11/06/how-to-get-most-out-of-windows-azure-tables.aspx http://blogs.msdn.com/b/windowsazurestorage/archive/2010/11/... Once and awhile we see someone saying that they see 100+ms latencies for small single entity reads and that is usually because they need to turn Nagle off, as described here: http://blogs.msdn.com/b/windowsazurestorage/archive/2010/06/25/nagle-s-algorithm-is-not-friendly-towards-small-requests.aspx http://blogs.msdn.com/b/windowsazurestorage/archive/2010/06/...
- iampims 15y ago
  This is running on SSDs and it makes a HUGE difference.
- vyrotek 15y ago
  Besides the points I already complained about before... How about 200ms response times even when performing a query using the Row & Partition Keys. I'm not sure if by composite keys you were referring to something other than the RK & PK because those are the only indexes you get.
  - balakk 15y ago
    ATS response times within the Azure data center are pretty impressive in my experience. Your partition keys can be composite, have a look here: http://blogs.msdn.com/b/windowsazurestorage/archive/2010/11/06/how-to-get-most-out-of-windows-azure-tables.aspx http://blogs.msdn.com/b/windowsazurestorage/archive/2010/11/... I agree with your other pain points - in terms of not being able to get counts, secondary indices etc. However, you can easily simulate some of those - maintain your own summary tables, indices and so on. These ought to emerge as platform features pretty soon though. It's not perfect, but its feature set is close to Dynamo. As for Mongo DB, I guess this service has been built from ground-up to provide the availability guarantees and automatic partition management features. I don't know if Mongo provides those. You could run Mongo yourself on Azure if you wanted to; there's even a supported solution done recently.
    - vyrotek 15y ago
      Hmm, I guess when I think about composite keys I think of ways to indicate a specific field/column as being part of the key. Data duplication along with string concatenation aren't really an elegant way to do it. If I remember right you also can't update the key values once the record has been saved. This is coming from a big SQL guy though :)
mattcole 15y ago
How is this vastly different to Azure Tables?
- rojabuck 15y ago
  The cost per transaction, performance & ease. Reads per $0.01 = (50.60).60 = 180000 Writes per $0.01 = (10.60).60 = 36000 Assuming that you hit your usage is at 100% capacity then from a read prospective DynamoDB is half the price. Writes are much more expensive but many applications are heavily read oriented. DynamoDB claims single digit millisecond reads, azure tables does not (from my experience.) Azure tables have a maximum performance over a given partition table of 500 requests per second and over the whole account of 5,000 requests per second. DynamoDB does not state this. http://blogs.msdn.com/b/windowsazurestorage/archive/2010/05/10/windows-azure-storage-abstractions-and-their-scalability-targets.aspx http://blogs.msdn.com/b/windowsazurestorage/archive/2010/05/... To put this into context: Assume a system with 5000 writes per second and 50000 reads here are the costs: AWS Reads: $240 AWS Writes: $120 Aws Total: $360 Azure Reads: $4320 Azure Writes: $432 Azure Total: $4752 Seems like quite a difference for a decent sized read heavy application.
  - balakk 15y ago
    Can you please explain your math? AFAIK Azure txns are not paid by the hour - they are a flat cost of $.01 per 10000 storage txns. If you do batched GETs and PUTs you make only 550 txns (55000/100 entites/batch). http://www.windowsazure.com/en-us/home/tour/storage/ http://www.windowsazure.com/en-us/home/tour/storage/ I agree that Dynamo's provisioned throughput capacity is a very useful feature though. Azure does not provide any such performance guarantee; the throughput limit is also a guideline as far as i know, not an absolute barrier.
    - rojabuck 15y ago
      I should have explained that my costs were calculated on a "per day" assumption. Thus the costs are for: 5000 x 60 x 60 x 24 = 432000000 Writes 50000 x 60 x 60 x 24 = 4320000000 Reads (432000000/10000) x 0.01 = $432 (4320000000/10000) x 0.01 = $4320 Azure Total Cost For One Days Use: $4752 ((5000/10) x 0.01) x 24 = $120 ((50000/50) x 0.01) x 24 = $240 AWS Total Cost For One Days Use: $360 You are right that I don't take into account the bulk feature of azure reads & writes but this is down to bulk requests only being possible on a single partition at a time which in my personal experience (not exhaustive) is non-trivial to take advantage of.

vyrotek 15y ago

I've been using Azure Table Storage since the beginning and this doesn't seem to be the same. Like others have mentioned TS is more similar to SimpleDB. Now, I would love for someone to give me a tl;dr on the the feature set of DynamoDB so I can make an accurate comparison. Table Storage does not allow any other indexes other than the main primary ones (Row Key and Partition Key). You also cannot store complex object within fields and use them in a query. You basically just serialize the data and stuff it into the field. The dynamic schema is very nice if you can leverage it but the actually query support is TERRIBLE. (Sorry Microsoft, I'm a fanboy but you blew it here). There is no Order By or event Count support which makes a lot of things very difficult. Want to know how many "color=green" records there are? Guess what, you're going to retrieve all those rows and then count them yourself. They're starting to listen to the community and have just recently introduced upserts and projection (select). I would love to see them adopt something like MongoDB instead :) For more issues check out: http://www.mygreatwindowsazureidea.com/forums/34192-windows-azure-feature-voting/suggestions/415607-add-tablestorage-linq-query-support-for-select-co http://www.mygreatwindowsazureidea.com/forums/34192-windows-... Edit - For what its worth. We've moved more things to SQL Azure now that it has Federation support. Scalability with the power of SQL. http://blogs.msdn.com/b/windowsazure/archive/2011/12/13/building-large-scale-elastic-database-tiers-with-sql-azure-introducing-federations.aspx http://blogs.msdn.com/b/windowsazure/archive/2011/12/13/buil...