9 ms·

After trying CosmosDB I put it down pretty quickly. I can't say I'd recommend it to anyone. Shockingly poor perf. Is CosmosDB really that bad or did we have it

After trying CosmosDB I put it down pretty quickly. I can't say I'd recommend it to anyone. Shockingly poor perf. Is CosmosDB really that bad or did we have it misconfigured? We're not sure, the docs didn't help us understand. Random failures when connecting. Random errors (or worse, no errors but unexpected results) when querying. Largely undocumented. Painful interop with non-azure proprietary offerings. Almost zero developer community. Most of the coverage of CosmosDB isn't by developers, it's by various MS affiliated blogs such as this one, which are more advertisements than resources. Unfortunately CosmosDB seems to fit the Azure MO of investing more into sales/marketing than into engineering/support. I don't know about the general atmosphere, but in the circles where I work, Azure and its various proprietary components are losing developer goodwill at a staggering pace.

deleted 8y ago
[deleted]
suff 8y ago
Putting it down quickly was your finely tuned technical instincts saving you from disaster.
m_fayer 8y ago
Same experience here. Also found it to be hideously expensive while setting all dials to minimum for experimentation and exploration. There may be a way to configure it for an affordable trial run, but it didn't motivate me to stay and figure it out.
- rnehme 8y ago
  Hi, I am from Azure Cosmos DB Engineering Team. For development (before going into production) with Azure Cosmos DB, we suggest a number of options (also documented here: https://docs.microsoft.com/en-us/azure/cosmos-db/optimize-dev-test https://docs.microsoft.com/en-us/azure/cosmos-db/optimize-de...): [1] Cosmos DB emulator (Cosmos DB Local version) - https://docs.microsoft.com/en-us/azure/cosmos-db/local-emulator https://docs.microsoft.com/en-us/azure/cosmos-db/local-emula.... The Azure Cosmos DB Emulator provides a local environment that emulates the Azure Cosmos DB service for development purposes. Using the Azure Cosmos DB Emulator, you can develop and test your application locally, without creating an Azure subscription or incurring any costs. When you're satisfied with how your application is working in the Azure Cosmos DB Emulator, you can switch to using an Azure Cosmos DB account in the cloud. Support for Cosmos DB emulator on Linux is coming. [2] We provide Try Cosmos DB for free experience - https://azure.microsoft.com/try/cosmosdb/ https://azure.microsoft.com/try/cosmosdb/. There is Try Azure Cosmos DB for free experience which gives you a time-limited full-service experience with Azure Cosmos DB. It is designed for trying out Cosmos DB, doing a tutorial, a demo, doing a quick start, a lab without requiring an Azure account or a credit card. [3] We provide a free tier (Cosmos DB is a part of free account) - https://azure.microsoft.com/free https://azure.microsoft.com/free. In Azure, we support a free tier with 12 month access to all popular services in Azure. As a part of free tier specifically for Cosmos DB you get 5GB in storage and 400RUs. Thanks so much!
foobarbazetc 8y ago
I can say this exact thing about most Azure services. Nothing really works as documented and everything is more expensive than it should be. I always feel like I’m missing something.
- tracker1 8y ago
  I've found that the "Azure Storage" services (blob, queues, table) are pretty decent (for what they are). If you don't have huge throughput or size needs that is. Different beasts, but definitely useful imho. If I need really high load/throughput, I'd probably reach for Cassandra on DO for best bang for the buck. YMMV though, and not too big on Cassandra unless you really need it.
- tgtweak 8y ago
  It's hard to get behind cloud services that require you to send an email to support engineers to reboot your instance.
  - rad_gruchalski 8y ago
    Wow, is that really the case? The last “cloud provider” I remember this to be the case was SoftLayer. Is Azure really that bad?
    - chrischen 8y ago
      Believe it or not Azure is the #2 cloud provider (by revenue?).
    - tgtweak 8y ago
      With AKS (at least in early days, not beta or anything mind you) the kubernetes coordinator node was managed by Microsoft, after a some arbitrary amount of days/weeks it would become unresponsive and the recommended action was to open a ticket and wait for them it restart it.
      - verst 8y ago
maltalex 8y ago
To the brave, I recommend trying to make a copy of a CosmosDb collection. The only solution that doesn't require spinning up a VM, reading all the data and writing it back again is the Azure Data Factory [0]. Which according to Azure's own benchmarks [1] manages to copy only 2MBps between two CosmosDb instances each with 100,000 RUs ("Resource Units"). That's more than half a day for a small 100GB collection. And in case you're wondering whether 100K RUs is a lot, consider the cost: 5840$/Month for single-region writes. 11680$/Month for multi-region writes [2]. Oh, and as a bonus - can you guess the official procedure for restoring a collection from a backup? [3]: > If you have accidentally deleted or corrupted your data, you should contact Azure support within 8 hours so that the Azure Cosmos DB team can help you restore the data from the backups. And yeah, it's 8 hours because they backup the database every 4 hours and keep the last two backup copies [3] [0]: https://docs.microsoft.com/en-us/azure/data-factory/introduction https://docs.microsoft.com/en-us/azure/data-factory/introduc... [1]: https://docs.microsoft.com/en-us/azure/data-factory/copy-activity-performance https://docs.microsoft.com/en-us/azure/data-factory/copy-act... [2]: https://azure.microsoft.com/en-us/pricing/details/cosmos-db/ https://azure.microsoft.com/en-us/pricing/details/cosmos-db/ [3]: https://docs.microsoft.com/en-us/azure/cosmos-db/online-backup-and-restore#automatic-and-online-backups https://docs.microsoft.com/en-us/azure/cosmos-db/online-back...
- llcoolv 8y ago
  I am sorry to have to argue, but the multi-region writes are calculated N + 1, where N is the number of regions. So it would actually set you back $17520 [0]. And yes, I know that even their own documentation and calculator conflict on this, but they do charge the higher number. 0. https://docs.microsoft.com/bg-bg/azure/cosmos-db/optimize-cost-regions https://docs.microsoft.com/bg-bg/azure/cosmos-db/optimize-co...
  - maltalex 8y ago
    Even better.
  - rnehme 8y ago
    Hi, I am from Azure Cosmos DB Engineering Team. We are in the process of updating the Azure pricing calculator to provide various options (e.g., # of regions, single vs. multi-master, etc.). The updates to the calculator are coming shortly. In the meantime, please feel free to ping us at AskCosmosDB@microsoft.com, if you need any help with cost estimates or have any questions.
ResidentSleeper 8y ago
That's sad to hear. I've had a lot of fun playing around with Azure's table/blob storage services. Never deployed a large production app with them, but the performance seemed decent (eg. when using table storage as a json store). I did look at Cosmos DB but decided not to give it a try when I saw the pricing.
- ui-explorer12 8y ago
  I like their table/blob and queue storage, but maybe it's because it's relatively mature and the domain is well understood. Most of the comparable services from other providers are generic analogs. I'm not sure I feel much different across Azure Cosmos and Aws Dynamo. It seems super-easy to get locked into a relatively expensive proprietary offering in each case.
- equasar 8y ago
  I second this, Azure Table Storage is an amazing product, but would be a more powerful one if they enable indexing on all columns and also any automatic backup options, which are only available on CosmosDB Table, which makes me thing that Azure wants to push me to use CosmosDB if I want those features, since they don't have a roadmap for this for the Azure Table Storage.
  - maltalex 8y ago
    These are two very different products at two very different price points. 10TB of Table storage is 655$ 10TB of CosmosDb is 2500$
  - manigandham 8y ago
    Azure Table storage is a key/value store similar to Hbase, Cassandra, GCP Bigtable, AWS DynamoDB and others. It's a very stable and scalable design which is why everyone has some version of it. There's no way to add indexes to this data model, it's all done by just writing your data in another table using the key you want and the value pointing to the original table's row. Perhaps Azure could offer secondary indexes like AWS DynamoDB does but it's probably too complex of a feature to add at this point, which is why it took DynamoDB several years to get to working GSI.
tasubotadas 8y ago
You also forgot to mention that it is crazy expensive if you have lots of collections
- unoti 8y ago
  I usually put multiple document types into a single collection and have a “type” field to get around this. The fields are indexed so this is efficient. To decide when to use a different collection I look at the scalability and other settings at the collection level, and group things that have similar needs together.
  - tracker1 8y ago
    If you only have a few "type" options, that may be a poorly performing index unless it's used as a secondary to a query that narrows results more.
- llcoolv 8y ago
  It actually supports throughput provisioning on database-level (or container-level as they sometimes call it). The only catch is that you'll have to recreate the db (on creation is the only time when you can enable it) and data migration is not exactly quick.
- rnehme 8y ago
  Hi, I am from Azure Cosmos DB Engineering Team. @llcoolv has responded and the response is correct. For scenarios where you create a lot of collections/tables, we recommend to provision throughput on a database and share the provisioned throughput among all the containers – to save on costs. You can change the throughput at database exactly like how you will change the throughput at container level. It is the same API, it is supported via CLI, SDK, Rest API and Portal as well. The entry point for database level throughout is 400RUs $24 per month and you can add any number of collections/tables that can share this throughput. You can scale up and down in the increments of 100RUs (or $6/month). Please see the following links for details: [1] https://docs.microsoft.com/en-us/azure/cosmos-db/set-throughput https://docs.microsoft.com/en-us/azure/cosmos-db/set-through... [2] https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-provision-container-throughput https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-prov... [3] https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-provision-database-throughput https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-prov... Please see the sample here for moving data: [4] https://github.com/Azure/azure-documentdb-dotnet/tree/master/samples/ChangeFeedMigrationTool https://github.com/Azure/azure-documentdb-dotnet/tree/master... Thanks!
victor106 8y ago
I can second this. We evaluated AWS, Azure and GCP for a very large healthcare company and found that Azure was the worst cloud services provider for anything. This was in spite a big push by leadership to encourage Azure, developers just hated it Between AWS and GCP even though AWS come out on top, GCP wins in quite a few services.
- llcoolv 8y ago
  Well Azure comes with the biggest compliance/certification package - of course upper management will love it.
- libria 8y ago
  What tips in AWS's favor, roughly speaking? My general feel is GCP is slightly cheaper but is missing some offerings, while AWS is a mature 1-stop shop that's well documented.
- usmannk 8y ago
  Good luck with GCP support...
- dekayed 8y ago
  Could you go more into this? How did you go about the evaluation and what metrics were you using? How did you capture the 'developers just hated it' in a way that leadership understood? I'm not disputing the outcome, just feel like it would be useful for others in a similar situation.
  - pojzon 8y ago
    Gonna second this. We are also in a situation where a few of the higher management were clearly bribed to push our company into Azure direction but the amount of problems we see may destroy our company..
enitihas 8y ago
That sounds really bad. I wonder how did Azure gain marketshare with a core offering (database) being so broken.
- llcoolv 8y ago
  The core database is SQL Server and they also have PostgreSQL. This is what most people are using.
  - enitihas 8y ago
    I should have been more precise. Cosmo DB is their core distributed database offering. Neither of SQL server of PostgreSQL is a distributed database. Both require applications to deal with the presence of multiple database nodes (sharding).
llcoolv 8y ago
Unfortunately, I could only second this opinion and add a few points of mine. 1. Many unexpected behaviours which you would not encounter in any other software - for example when you're blocked by its firewall, it would let you connect on all 7 OSI layers, then kick you out with "Command failed with error 13: 'Not Authenticated' on server xyz:10255'". By the way even Azure Portal itself doesn't know how to handle this behaviour correctly and when not included in the firewall whitelist just blinks like crazy. 2. Many seemingly basic features commonly found on other database systems are for some reason missing here - it doesn't support users as such, just issuing resource tokens[1] (which you must implement yourself). Thus, most CDB users just go straight with the master key even on production. 3. It is not fully (nor at least nearly fully) compatible with any of the advertised APIs - SQL doesn't support joins, MongoDB misses quite a few features and adds a requirement for the shardkey to be part of the query. Which is not supported by Spring Data, so you must do low-level queries for everything but select (think updates field by field with null checks). 4. The tooling is horrible - as it is not compatible with neither mongodb nor traditional sql, you would need to include the "CosmosDB Emulator" in your build in order to have at least some predictability. The problem? It only runs on docker4win and a Windows VM would need to be running by all developers on the team and workers of the CI/CD. As of April 2019 5. It is CRAZY expensive and the pricing is not really transparent - in our case we had 20 MB of data, 4 1000RUs collections and multi-master write in Western and Northern Europe. The monthly bill? 300 EUR even though Azure Portal was displaying 1/6 of that cost - apparently for multi-master you multiply by the number of regions + 1 (2+1 = 3) and for geo-redundancy only by the number of regions (2), thus the 6x higher cost. The result of that was that we were paying several times more to manage 20MB of data than for all the remaining data - hundreds of GB. Maybe if you really, really need the planetary scale, the master-master replication and the <10ms SLAs it would make sense and the hassle and cost would be worth it (though I seriously doubt it), but in any other case it is a massive mistake to go this route. 1. https://docs.microsoft.com/en-us/azure/cosmos-db/secure-access-to-data https://docs.microsoft.com/en-us/azure/cosmos-db/secure-acce...
- pm90 8y ago
  > It is CRAZY expensive and the pricing is not really transparent - in our case we had 20 MB of data, 4 1000RUs collections and multi-master write in Western and Northern Europe. The monthly bill? 300 EUR even though Azure Portal was displaying 1/6 of that cost - apparently for multi-master you multiply by the number of regions + 1 (2+1 = 3) and for geo-redundancy only by the number of regions (2), thus the 6x higher cost. The result of that was that we were paying several times more to manage 20MB of data than for all the remaining data - hundreds of GB. I get your point and they should definitely fix either the cost calculation, or paste a disclaimer for multi-region pricing. But I want to (sincerely) ask how you could assume that multi region would _not_ cost more than single region.
m0zg 8y ago
I've used it briefly internally at Microsoft 2009, I had almost the same experience. Super slow, takes a while to even start the query, non-standard query language. For internal users there were even more hurdles: you had to get approval and quota before you were able to use it at all, and the default answer to both requests was "no", you really had to justify it. As an aside, at MS "no" was the answer to any request you make of Legal. Lots of bicycles were reinvented from scratch, with lots bugs, just because Legal would not approve the use of BSD or MIT licensed code. Imagine my shock when I started at Google a year later and discovered Dremel (now BigQuery) where I could just run a query and get the results back in seconds over tens of terabytes of data, with no quotas or approvals. I think having a system like Dremel is a huge competitive advantage, because you don't have to guess blindly about things, you can just query them and get results immediately. If Cosmos is still the best MS has internally, they need to roll their own Dremel asap. Lots of great people from SQL Server team work at Google now, but MS still has plenty of people who could do a competent petascale columnar DB with one hand tied behind their back.
- CurtHagenlocher 8y ago
  It sounds like you're describing the internal big data platform "Cosmos", something which has been made publicly-available as "Azure Data Lake Analytics" and which has no relationship to "CosmosDB" other than a similar name.
  - metildaa 8y ago
    Sounds like a branding mishap, Microsoft created the same type of hard to understand mess with the Microsoft Dynamics branding.
    - int_19h 8y ago
      Like he said, the public product is called Azure Data Lake, and internal product names aren't brands (that's kind of the whole point of them).
  - deleted 8y ago
    [deleted]
swebs 8y ago
>Shockingly poor perf >Random failures >Largely undocumented >Painful interop with non-azure proprietary offerings >Almost zero developer community >Most of the coverage ... isn't by developers, it's by various MS affiliated blogs such as this one, which are more advertisements than resources. Welcome to Azure!
rnehme 8y ago
Hi, I am from Azure Cosmos DB Engineering Team. Please feel free to send us the details of your workload that suffered from poor performance to AskCosmosDB@microsoft.com (it is a direct way to reach the engineering team and it does not cost anything), and we can help determine why and provide with the recommendations. For best practices and performance optimizations, the following links will help: https://docs.microsoft.com/en-us/azure/cosmos-db/sql-api-query-metrics https://docs.microsoft.com/en-us/azure/cosmos-db/sql-api-que... https://docs.microsoft.com/en-us/azure/cosmos-db/performance-tips https://docs.microsoft.com/en-us/azure/cosmos-db/performance... https://docs.microsoft.com/en-us/azure/cosmos-db/performance-tips-java https://docs.microsoft.com/en-us/azure/cosmos-db/performance... https://docs.microsoft.com/en-us/azure/cosmos-db/performance-testing https://docs.microsoft.com/en-us/azure/cosmos-db/performance... Also, the following conceptual docs will help: https://docs.microsoft.com/en-us/azure/cosmos-db/partitioning-overview https://docs.microsoft.com/en-us/azure/cosmos-db/partitionin... https://docs.microsoft.com/en-us/azure/cosmos-db/partition-data https://docs.microsoft.com/en-us/azure/cosmos-db/partition-d... https://docs.microsoft.com/en-us/azure/cosmos-db/request-units https://docs.microsoft.com/en-us/azure/cosmos-db/request-uni... We are doing a full refresh of our docs (rewriting a lot of the older content and adding new technical content). We welcome feedback on how we can make the docs better and invite everyone to contribute via public PR/Github. If there are any topics that are missing in our docs or are not easily discoverable, please let us know. We will make sure to address it. We are committed to building a great community. As a relatively young database, we have ton of work ahead, but we are super committed to making sure we make developers successful.
mcv 8y ago
That's good to know. My team is currently using Neo4j, but may be forced to switch to either Neptune or Cosmos. I figured those were probably pretty similar, but this sounds like Cosmos might be a really bad idea.
- onedataguy 8y ago
  I am a Cosmos DB graph engineer and would encourage you to take the service for a spin and build your opinion. This thread unfortunately has more emotions than truth to it. We build the service with best intentions in mind. Feel free to reach out to us at askcosmosdb@microsoft.com any time with any questions.
  - llcoolv 8y ago
    Could you be a bit more specific on the what exactly is untrue?

deleted 8y ago

[deleted]

suff 8y ago

Putting it down quickly was your finely tuned technical instincts saving you from disaster.

m_fayer 8y ago

Same experience here. Also found it to be hideously expensive while setting all dials to minimum for experimentation and exploration. There may be a way to configure it for an affordable trial run, but it didn't motivate me to stay and figure it out.

rnehme 8y ago
Hi, I am from Azure Cosmos DB Engineering Team. For development (before going into production) with Azure Cosmos DB, we suggest a number of options (also documented here: https://docs.microsoft.com/en-us/azure/cosmos-db/optimize-dev-test https://docs.microsoft.com/en-us/azure/cosmos-db/optimize-de...): [1] Cosmos DB emulator (Cosmos DB Local version) - https://docs.microsoft.com/en-us/azure/cosmos-db/local-emulator https://docs.microsoft.com/en-us/azure/cosmos-db/local-emula.... The Azure Cosmos DB Emulator provides a local environment that emulates the Azure Cosmos DB service for development purposes. Using the Azure Cosmos DB Emulator, you can develop and test your application locally, without creating an Azure subscription or incurring any costs. When you're satisfied with how your application is working in the Azure Cosmos DB Emulator, you can switch to using an Azure Cosmos DB account in the cloud. Support for Cosmos DB emulator on Linux is coming. [2] We provide Try Cosmos DB for free experience - https://azure.microsoft.com/try/cosmosdb/ https://azure.microsoft.com/try/cosmosdb/. There is Try Azure Cosmos DB for free experience which gives you a time-limited full-service experience with Azure Cosmos DB. It is designed for trying out Cosmos DB, doing a tutorial, a demo, doing a quick start, a lab without requiring an Azure account or a credit card. [3] We provide a free tier (Cosmos DB is a part of free account) - https://azure.microsoft.com/free https://azure.microsoft.com/free. In Azure, we support a free tier with 12 month access to all popular services in Azure. As a part of free tier specifically for Cosmos DB you get 5GB in storage and 400RUs. Thanks so much!

foobarbazetc 8y ago

I can say this exact thing about most Azure services. Nothing really works as documented and everything is more expensive than it should be. I always feel like I’m missing something.

tracker1 8y ago
I've found that the "Azure Storage" services (blob, queues, table) are pretty decent (for what they are). If you don't have huge throughput or size needs that is. Different beasts, but definitely useful imho. If I need really high load/throughput, I'd probably reach for Cassandra on DO for best bang for the buck. YMMV though, and not too big on Cassandra unless you really need it.
tgtweak 8y ago
It's hard to get behind cloud services that require you to send an email to support engineers to reboot your instance.
- rad_gruchalski 8y ago
  Wow, is that really the case? The last “cloud provider” I remember this to be the case was SoftLayer. Is Azure really that bad?
  - chrischen 8y ago
    Believe it or not Azure is the #2 cloud provider (by revenue?).
  - tgtweak 8y ago
    With AKS (at least in early days, not beta or anything mind you) the kubernetes coordinator node was managed by Microsoft, after a some arbitrary amount of days/weeks it would become unresponsive and the recommended action was to open a ticket and wait for them it restart it.
    - verst 8y ago

maltalex 8y ago

To the brave, I recommend trying to make a copy of a CosmosDb collection. The only solution that doesn't require spinning up a VM, reading all the data and writing it back again is the Azure Data Factory [0]. Which according to Azure's own benchmarks [1] manages to copy only 2MBps between two CosmosDb instances each with 100,000 RUs ("Resource Units"). That's more than half a day for a small 100GB collection. And in case you're wondering whether 100K RUs is a lot, consider the cost: 5840$/Month for single-region writes. 11680$/Month for multi-region writes [2]. Oh, and as a bonus - can you guess the official procedure for restoring a collection from a backup? [3]: > If you have accidentally deleted or corrupted your data, you should contact Azure support within 8 hours so that the Azure Cosmos DB team can help you restore the data from the backups. And yeah, it's 8 hours because they backup the database every 4 hours and keep the last two backup copies [3] [0]: https://docs.microsoft.com/en-us/azure/data-factory/introduction https://docs.microsoft.com/en-us/azure/data-factory/introduc... [1]: https://docs.microsoft.com/en-us/azure/data-factory/copy-activity-performance https://docs.microsoft.com/en-us/azure/data-factory/copy-act... [2]: https://azure.microsoft.com/en-us/pricing/details/cosmos-db/ https://azure.microsoft.com/en-us/pricing/details/cosmos-db/ [3]: https://docs.microsoft.com/en-us/azure/cosmos-db/online-backup-and-restore#automatic-and-online-backups https://docs.microsoft.com/en-us/azure/cosmos-db/online-back...

llcoolv 8y ago
I am sorry to have to argue, but the multi-region writes are calculated N + 1, where N is the number of regions. So it would actually set you back $17520 [0]. And yes, I know that even their own documentation and calculator conflict on this, but they do charge the higher number. 0. https://docs.microsoft.com/bg-bg/azure/cosmos-db/optimize-cost-regions https://docs.microsoft.com/bg-bg/azure/cosmos-db/optimize-co...
- maltalex 8y ago
  Even better.
- rnehme 8y ago
  Hi, I am from Azure Cosmos DB Engineering Team. We are in the process of updating the Azure pricing calculator to provide various options (e.g., # of regions, single vs. multi-master, etc.). The updates to the calculator are coming shortly. In the meantime, please feel free to ping us at AskCosmosDB@microsoft.com, if you need any help with cost estimates or have any questions.

ResidentSleeper 8y ago

That's sad to hear. I've had a lot of fun playing around with Azure's table/blob storage services. Never deployed a large production app with them, but the performance seemed decent (eg. when using table storage as a json store). I did look at Cosmos DB but decided not to give it a try when I saw the pricing.

ui-explorer12 8y ago
I like their table/blob and queue storage, but maybe it's because it's relatively mature and the domain is well understood. Most of the comparable services from other providers are generic analogs. I'm not sure I feel much different across Azure Cosmos and Aws Dynamo. It seems super-easy to get locked into a relatively expensive proprietary offering in each case.
equasar 8y ago
I second this, Azure Table Storage is an amazing product, but would be a more powerful one if they enable indexing on all columns and also any automatic backup options, which are only available on CosmosDB Table, which makes me thing that Azure wants to push me to use CosmosDB if I want those features, since they don't have a roadmap for this for the Azure Table Storage.
- maltalex 8y ago
  These are two very different products at two very different price points. 10TB of Table storage is 655$ 10TB of CosmosDb is 2500$
- manigandham 8y ago
  Azure Table storage is a key/value store similar to Hbase, Cassandra, GCP Bigtable, AWS DynamoDB and others. It's a very stable and scalable design which is why everyone has some version of it. There's no way to add indexes to this data model, it's all done by just writing your data in another table using the key you want and the value pointing to the original table's row. Perhaps Azure could offer secondary indexes like AWS DynamoDB does but it's probably too complex of a feature to add at this point, which is why it took DynamoDB several years to get to working GSI.

tasubotadas 8y ago

You also forgot to mention that it is crazy expensive if you have lots of collections

unoti 8y ago
I usually put multiple document types into a single collection and have a “type” field to get around this. The fields are indexed so this is efficient. To decide when to use a different collection I look at the scalability and other settings at the collection level, and group things that have similar needs together.
- tracker1 8y ago
  If you only have a few "type" options, that may be a poorly performing index unless it's used as a secondary to a query that narrows results more.
llcoolv 8y ago
It actually supports throughput provisioning on database-level (or container-level as they sometimes call it). The only catch is that you'll have to recreate the db (on creation is the only time when you can enable it) and data migration is not exactly quick.
rnehme 8y ago
Hi, I am from Azure Cosmos DB Engineering Team. @llcoolv has responded and the response is correct. For scenarios where you create a lot of collections/tables, we recommend to provision throughput on a database and share the provisioned throughput among all the containers – to save on costs. You can change the throughput at database exactly like how you will change the throughput at container level. It is the same API, it is supported via CLI, SDK, Rest API and Portal as well. The entry point for database level throughout is 400RUs $24 per month and you can add any number of collections/tables that can share this throughput. You can scale up and down in the increments of 100RUs (or $6/month). Please see the following links for details: [1] https://docs.microsoft.com/en-us/azure/cosmos-db/set-throughput https://docs.microsoft.com/en-us/azure/cosmos-db/set-through... [2] https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-provision-container-throughput https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-prov... [3] https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-provision-database-throughput https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-prov... Please see the sample here for moving data: [4] https://github.com/Azure/azure-documentdb-dotnet/tree/master/samples/ChangeFeedMigrationTool https://github.com/Azure/azure-documentdb-dotnet/tree/master... Thanks!

victor106 8y ago

I can second this. We evaluated AWS, Azure and GCP for a very large healthcare company and found that Azure was the worst cloud services provider for anything. This was in spite a big push by leadership to encourage Azure, developers just hated it Between AWS and GCP even though AWS come out on top, GCP wins in quite a few services.

llcoolv 8y ago
Well Azure comes with the biggest compliance/certification package - of course upper management will love it.
libria 8y ago
What tips in AWS's favor, roughly speaking? My general feel is GCP is slightly cheaper but is missing some offerings, while AWS is a mature 1-stop shop that's well documented.
usmannk 8y ago
Good luck with GCP support...
dekayed 8y ago
Could you go more into this? How did you go about the evaluation and what metrics were you using? How did you capture the 'developers just hated it' in a way that leadership understood? I'm not disputing the outcome, just feel like it would be useful for others in a similar situation.
- pojzon 8y ago
  Gonna second this. We are also in a situation where a few of the higher management were clearly bribed to push our company into Azure direction but the amount of problems we see may destroy our company..

enitihas 8y ago

That sounds really bad. I wonder how did Azure gain marketshare with a core offering (database) being so broken.

llcoolv 8y ago
The core database is SQL Server and they also have PostgreSQL. This is what most people are using.
- enitihas 8y ago
  I should have been more precise. Cosmo DB is their core distributed database offering. Neither of SQL server of PostgreSQL is a distributed database. Both require applications to deal with the presence of multiple database nodes (sharding).

llcoolv 8y ago

Unfortunately, I could only second this opinion and add a few points of mine. 1. Many unexpected behaviours which you would not encounter in any other software - for example when you're blocked by its firewall, it would let you connect on all 7 OSI layers, then kick you out with "Command failed with error 13: 'Not Authenticated' on server xyz:10255'". By the way even Azure Portal itself doesn't know how to handle this behaviour correctly and when not included in the firewall whitelist just blinks like crazy. 2. Many seemingly basic features commonly found on other database systems are for some reason missing here - it doesn't support users as such, just issuing resource tokens[1] (which you must implement yourself). Thus, most CDB users just go straight with the master key even on production. 3. It is not fully (nor at least nearly fully) compatible with any of the advertised APIs - SQL doesn't support joins, MongoDB misses quite a few features and adds a requirement for the shardkey to be part of the query. Which is not supported by Spring Data, so you must do low-level queries for everything but select (think updates field by field with null checks). 4. The tooling is horrible - as it is not compatible with neither mongodb nor traditional sql, you would need to include the "CosmosDB Emulator" in your build in order to have at least some predictability. The problem? It only runs on docker4win and a Windows VM would need to be running by all developers on the team and workers of the CI/CD. As of April 2019 5. It is CRAZY expensive and the pricing is not really transparent - in our case we had 20 MB of data, 4 1000RUs collections and multi-master write in Western and Northern Europe. The monthly bill? 300 EUR even though Azure Portal was displaying 1/6 of that cost - apparently for multi-master you multiply by the number of regions + 1 (2+1 = 3) and for geo-redundancy only by the number of regions (2), thus the 6x higher cost. The result of that was that we were paying several times more to manage 20MB of data than for all the remaining data - hundreds of GB. Maybe if you really, really need the planetary scale, the master-master replication and the <10ms SLAs it would make sense and the hassle and cost would be worth it (though I seriously doubt it), but in any other case it is a massive mistake to go this route. 1. https://docs.microsoft.com/en-us/azure/cosmos-db/secure-access-to-data https://docs.microsoft.com/en-us/azure/cosmos-db/secure-acce...

pm90 8y ago
> It is CRAZY expensive and the pricing is not really transparent - in our case we had 20 MB of data, 4 1000RUs collections and multi-master write in Western and Northern Europe. The monthly bill? 300 EUR even though Azure Portal was displaying 1/6 of that cost - apparently for multi-master you multiply by the number of regions + 1 (2+1 = 3) and for geo-redundancy only by the number of regions (2), thus the 6x higher cost. The result of that was that we were paying several times more to manage 20MB of data than for all the remaining data - hundreds of GB. I get your point and they should definitely fix either the cost calculation, or paste a disclaimer for multi-region pricing. But I want to (sincerely) ask how you could assume that multi region would _not_ cost more than single region.

m0zg 8y ago

I've used it briefly internally at Microsoft 2009, I had almost the same experience. Super slow, takes a while to even start the query, non-standard query language. For internal users there were even more hurdles: you had to get approval and quota before you were able to use it at all, and the default answer to both requests was "no", you really had to justify it. As an aside, at MS "no" was the answer to any request you make of Legal. Lots of bicycles were reinvented from scratch, with lots bugs, just because Legal would not approve the use of BSD or MIT licensed code. Imagine my shock when I started at Google a year later and discovered Dremel (now BigQuery) where I could just run a query and get the results back in seconds over tens of terabytes of data, with no quotas or approvals. I think having a system like Dremel is a huge competitive advantage, because you don't have to guess blindly about things, you can just query them and get results immediately. If Cosmos is still the best MS has internally, they need to roll their own Dremel asap. Lots of great people from SQL Server team work at Google now, but MS still has plenty of people who could do a competent petascale columnar DB with one hand tied behind their back.

CurtHagenlocher 8y ago
It sounds like you're describing the internal big data platform "Cosmos", something which has been made publicly-available as "Azure Data Lake Analytics" and which has no relationship to "CosmosDB" other than a similar name.
- metildaa 8y ago
  Sounds like a branding mishap, Microsoft created the same type of hard to understand mess with the Microsoft Dynamics branding.
  - int_19h 8y ago
    Like he said, the public product is called Azure Data Lake, and internal product names aren't brands (that's kind of the whole point of them).
- deleted 8y ago
  [deleted]

swebs 8y ago

>Shockingly poor perf >Random failures >Largely undocumented >Painful interop with non-azure proprietary offerings >Almost zero developer community >Most of the coverage ... isn't by developers, it's by various MS affiliated blogs such as this one, which are more advertisements than resources. Welcome to Azure!

rnehme 8y ago

Hi, I am from Azure Cosmos DB Engineering Team. Please feel free to send us the details of your workload that suffered from poor performance to AskCosmosDB@microsoft.com (it is a direct way to reach the engineering team and it does not cost anything), and we can help determine why and provide with the recommendations. For best practices and performance optimizations, the following links will help: https://docs.microsoft.com/en-us/azure/cosmos-db/sql-api-query-metrics https://docs.microsoft.com/en-us/azure/cosmos-db/sql-api-que... https://docs.microsoft.com/en-us/azure/cosmos-db/performance-tips https://docs.microsoft.com/en-us/azure/cosmos-db/performance... https://docs.microsoft.com/en-us/azure/cosmos-db/performance-tips-java https://docs.microsoft.com/en-us/azure/cosmos-db/performance... https://docs.microsoft.com/en-us/azure/cosmos-db/performance-testing https://docs.microsoft.com/en-us/azure/cosmos-db/performance... Also, the following conceptual docs will help: https://docs.microsoft.com/en-us/azure/cosmos-db/partitioning-overview https://docs.microsoft.com/en-us/azure/cosmos-db/partitionin... https://docs.microsoft.com/en-us/azure/cosmos-db/partition-data https://docs.microsoft.com/en-us/azure/cosmos-db/partition-d... https://docs.microsoft.com/en-us/azure/cosmos-db/request-units https://docs.microsoft.com/en-us/azure/cosmos-db/request-uni... We are doing a full refresh of our docs (rewriting a lot of the older content and adding new technical content). We welcome feedback on how we can make the docs better and invite everyone to contribute via public PR/Github. If there are any topics that are missing in our docs or are not easily discoverable, please let us know. We will make sure to address it. We are committed to building a great community. As a relatively young database, we have ton of work ahead, but we are super committed to making sure we make developers successful.

mcv 8y ago

That's good to know. My team is currently using Neo4j, but may be forced to switch to either Neptune or Cosmos. I figured those were probably pretty similar, but this sounds like Cosmos might be a really bad idea.

onedataguy 8y ago
I am a Cosmos DB graph engineer and would encourage you to take the service for a spin and build your opinion. This thread unfortunately has more emotions than truth to it. We build the service with best intentions in mind. Feel free to reach out to us at askcosmosdb@microsoft.com any time with any questions.
- llcoolv 8y ago
  Could you be a bit more specific on the what exactly is untrue?