4 ms·

This rhymes with my overall impression of Cosmos. It took us a while to see through the smokescreen because when talking to Microsoft support and representative

by dagss 4y ago

This rhymes with my overall impression of Cosmos. It took us a while to see through the smokescreen because when talking to Microsoft support and representatives it is the Best Thing Ever and they sound so confident about it. But it really is a beta demo product sold with an alpha premium price tag. If your traffic pattern is exactly right, and you always scale traffic up and never ever down and do not have spikes, I guess it is probably OK. The main problem is the docs are (or, at least were 2 years ago) not clear about all the caveats and restrictions but pretend it is a generic database that just works. So one has to discover all the caveats oneself. Microsoft thinks the exact workings of the partitioning is something that should work so well you don't need to know it in detail. But, if your usecase is slightly off you end up really needing to know. I know at least one team who routinely copy all their data from one Cosmos instance to another and switch over traffic to the copy just to get a partitioning reset; it is one thing to have to do it; another to discover in production yourself it has to be done with no prior warning.. Also: The ipython+portal+Cosmos security meltdown from 1 1/2 years ago alone should be reason to look elsewhere. (No, not a competitor, just have spent way way way too much engineering time moving first on and then off Cosmos and yes I am bitter)

MrBuddyCasino 4y ago
I used it a few months ago, it still is a half-baked piece of shit. Code quality of client libs even worse than AWS. It could have been easy. We could have used Postgres.
- sofixa 4y ago
  > It could have been easy. We could have used Postgres. And then all that would have been left is just to make that Postgres in a globally distributed DB and manage it (with something like Citus). Postgres' native scaling out (not even talking about globally distributed) capabilities are basically nonexistent so you need third party tooling.
  - MrBuddyCasino 4y ago
    We didn't need a "globally distributed DB". This was an internal tool for a BigCorp.
nobodyandproud 4y ago
Is this analogous to NTFS? For you young uns, back in the 1990s Microsoft was so convinced that NTFS made file fragmentation impossible that they didn’t provide a way to defrag for a very long time.
- dspillett 4y ago
  In fairness NTFS does usually make fragmentation a non issue away from pathological cases¹, much like ext2/3/4² which didn't have defrag tools³ for a long time either. FAT12/FAT16/FAT32's earliest-free-block-no-matter-what allocation method⁴ trained people to believe that fragmentation is a universally rampant problem, but with a better designed allocation strategy it really isn't. ---- [1] for instance simultaneously growing files on near-full volumes [2] more so ext4 with delayed allocation turned on [3] at least not ones without attendant "are you really sure you want to use this on data you care about?" warnings [4] I wonder if anyone retrofitted a brighter heuristic into an implementation of these or exFAT. Could be a (not massively useful but) interesting learning exercise for a budding OS developer.
PaulWaldman 4y ago
>(No, not a competitor, just have spent way way way too much engineering time moving first on and then off Cosmos and yes I am bitter) Can you share what you migrated onto and the results?
- dagss 4y ago
  We migrated to Azure SQL Hyperscale and are very happy about that.
  - jychang 4y ago
    Ah. That explains a lot of comments in this thread. Cosmos is used because some tables we use are larger than 64TB. It’s decently useful for large chunks of data.
    - dagss 4y ago
      Well, even if WE do not have 64TB tables (where one need low latency for the entire table), I believe there are plenty of other mature NoSQL offerings out there? Whether you go to the other cloud vendors or self host. Cosmos is at best unfinished. I used Google Data Store for years and it was finished in the sense that it always did what it says on the tin and didn't cause lots of problems for you (although I think higher latency than Cosmos, so may not be for every usecase -- there is also BigTable and Spanner). And I don't believe Google would ever have done something like deploying a shared multitenant Jupyter system with full access to all customer's DB. Microsoft actually did that. As I said the main problem is mainly how Cosmos is marketed, as a general purpose NoSQL. If it was marketed with all the caveats you learn about after you go live, and people only used it if they really had to, it would be a bit different. Though realistically it would drive most people to other cloud providers.
VWWHFSfQ 4y ago
After suffering through the AWS SimpleDB disaster 10 years ago I will never use any of the cloud providers' hairbrained databases ever again. I'll use bog-standard Postgres or MySQL if they host it but nothing else.
- Godel_unicode 4y ago
  What if I told you there’s now CosmosDB for PostgreSQL. https://devblogs.microsoft.com/cosmosdb/distributed-postgresql-comes-to-azure-cosmos-db/ https://devblogs.microsoft.com/cosmosdb/distributed-postgres...
  - SigmundA 4y ago
    Which is just managed PG Citus with the word Cosmos in the name it seems.
    - sixdimensional 4y ago
      This was both shocking and not surprising at all, all at the same time. It’s worth noting how cloud vendors have fallen back on tried and true databases such as Postgres (mind you, often replacing the guts of those databases with new implementations.. but still).
      - sofixa 4y ago
        > It’s worth noting how cloud vendors have fallen back on tried and true databases such as Postgres (mind you, often replacing the guts of those databases with new implementations.. but still). Have they? Amazon created Aurora from scratch, with compatibility layers for different database engines (MySQL, Postgres), and GCP did the same with Spanner, which i wouldn't call "fallen back".
        sixdimensional 4y ago
        Yes, that's true - those products are definitely more of trying to displace the open source tech by borrowing the interface. I still think it is a plain admission to the fact of the popularity of the original tech (e.g. MySQL and Postgres). For sure, credit due where it is due for implementing entirely new implementations - but it's such a common pattern to "borrow" the open source interfaces and reimplement the core engine. That said, you didn't mention things like RDS or Google Cloud SQL, which are cloud-based versions of standard open source dbs.
maltalex 4y ago
This matches my (fairly extensive) experience with CosmosDb. It’s truly an awfully engineered product. I cannot comprehend what organizational process lead not only to its creation, but also to its continued existence. Is existence is a constant reminder that the road to hell is paved with good intentions, and that you better do your due diligence before adopting any piece of tech that’s not easy to replace. Even if it comes with a pedigree. I wrote up some of the horrors stories a while back on HN: https://news.ycombinator.com/item?id=29295871 https://news.ycombinator.com/item?id=29295871
Twirrim 4y ago
> Microsoft thinks the exact workings of the partitioning is something that should work so well you don't need to know it in detail. But, if your usecase is slightly off you end up really needing to know. I know at least one team who routinely copy all their data from one Cosmos instance to another and switch over traffic to the copy just to get a partitioning reset; it is one thing to have to do it; another to discover in production yourself it has to be done with no prior warning.. AWS DynamoDB used to do that. When I was working for a team in AWS we ended up discovering an unexpected bad hash choice, had a hot partition and ended up with a crazy number of partitions and a need to have really high r/w allocation on it. The only option at the time was to roll over to a new table (which we could do and fix our hash choice, thankfully, without too much hassle). They fixed that stuff a few years ago now so most (all?) of my "here be dragons" concerns about DynamoDB have been addressed.
danjc 4y ago
Spoke with a customer last week who said their cost for cosmos is on the order of 10x what it would be for SQL. I’ve learned to stay away from services that are high abstractions. Huge lock in, expensive and much more risk of it being eol’d. In our platform we’ve actually dropped SQL in favor of Az blob and table storage. It’s going to reduce our database costs by 5x.

by dagss 4y ago

MrBuddyCasino 4y ago

I used it a few months ago, it still is a half-baked piece of shit. Code quality of client libs even worse than AWS. It could have been easy. We could have used Postgres.

sofixa 4y ago
> It could have been easy. We could have used Postgres. And then all that would have been left is just to make that Postgres in a globally distributed DB and manage it (with something like Citus). Postgres' native scaling out (not even talking about globally distributed) capabilities are basically nonexistent so you need third party tooling.
- MrBuddyCasino 4y ago
  We didn't need a "globally distributed DB". This was an internal tool for a BigCorp.

nobodyandproud 4y ago

Is this analogous to NTFS? For you young uns, back in the 1990s Microsoft was so convinced that NTFS made file fragmentation impossible that they didn’t provide a way to defrag for a very long time.

dspillett 4y ago
In fairness NTFS does usually make fragmentation a non issue away from pathological cases¹, much like ext2/3/4² which didn't have defrag tools³ for a long time either. FAT12/FAT16/FAT32's earliest-free-block-no-matter-what allocation method⁴ trained people to believe that fragmentation is a universally rampant problem, but with a better designed allocation strategy it really isn't. ---- [1] for instance simultaneously growing files on near-full volumes [2] more so ext4 with delayed allocation turned on [3] at least not ones without attendant "are you really sure you want to use this on data you care about?" warnings [4] I wonder if anyone retrofitted a brighter heuristic into an implementation of these or exFAT. Could be a (not massively useful but) interesting learning exercise for a budding OS developer.

PaulWaldman 4y ago

>(No, not a competitor, just have spent way way way too much engineering time moving first on and then off Cosmos and yes I am bitter) Can you share what you migrated onto and the results?

dagss 4y ago
We migrated to Azure SQL Hyperscale and are very happy about that.
- jychang 4y ago
  Ah. That explains a lot of comments in this thread. Cosmos is used because some tables we use are larger than 64TB. It’s decently useful for large chunks of data.
  - dagss 4y ago
    Well, even if WE do not have 64TB tables (where one need low latency for the entire table), I believe there are plenty of other mature NoSQL offerings out there? Whether you go to the other cloud vendors or self host. Cosmos is at best unfinished. I used Google Data Store for years and it was finished in the sense that it always did what it says on the tin and didn't cause lots of problems for you (although I think higher latency than Cosmos, so may not be for every usecase -- there is also BigTable and Spanner). And I don't believe Google would ever have done something like deploying a shared multitenant Jupyter system with full access to all customer's DB. Microsoft actually did that. As I said the main problem is mainly how Cosmos is marketed, as a general purpose NoSQL. If it was marketed with all the caveats you learn about after you go live, and people only used it if they really had to, it would be a bit different. Though realistically it would drive most people to other cloud providers.

VWWHFSfQ 4y ago

After suffering through the AWS SimpleDB disaster 10 years ago I will never use any of the cloud providers' hairbrained databases ever again. I'll use bog-standard Postgres or MySQL if they host it but nothing else.

Godel_unicode 4y ago
What if I told you there’s now CosmosDB for PostgreSQL. https://devblogs.microsoft.com/cosmosdb/distributed-postgresql-comes-to-azure-cosmos-db/ https://devblogs.microsoft.com/cosmosdb/distributed-postgres...
- SigmundA 4y ago
  Which is just managed PG Citus with the word Cosmos in the name it seems.
  - sixdimensional 4y ago
    This was both shocking and not surprising at all, all at the same time. It’s worth noting how cloud vendors have fallen back on tried and true databases such as Postgres (mind you, often replacing the guts of those databases with new implementations.. but still).
    - sofixa 4y ago
      > It’s worth noting how cloud vendors have fallen back on tried and true databases such as Postgres (mind you, often replacing the guts of those databases with new implementations.. but still). Have they? Amazon created Aurora from scratch, with compatibility layers for different database engines (MySQL, Postgres), and GCP did the same with Spanner, which i wouldn't call "fallen back".
      - sixdimensional 4y ago
        Yes, that's true - those products are definitely more of trying to displace the open source tech by borrowing the interface. I still think it is a plain admission to the fact of the popularity of the original tech (e.g. MySQL and Postgres). For sure, credit due where it is due for implementing entirely new implementations - but it's such a common pattern to "borrow" the open source interfaces and reimplement the core engine. That said, you didn't mention things like RDS or Google Cloud SQL, which are cloud-based versions of standard open source dbs.

maltalex 4y ago

This matches my (fairly extensive) experience with CosmosDb. It’s truly an awfully engineered product. I cannot comprehend what organizational process lead not only to its creation, but also to its continued existence. Is existence is a constant reminder that the road to hell is paved with good intentions, and that you better do your due diligence before adopting any piece of tech that’s not easy to replace. Even if it comes with a pedigree. I wrote up some of the horrors stories a while back on HN: https://news.ycombinator.com/item?id=29295871 https://news.ycombinator.com/item?id=29295871

Twirrim 4y ago

> Microsoft thinks the exact workings of the partitioning is something that should work so well you don't need to know it in detail. But, if your usecase is slightly off you end up really needing to know. I know at least one team who routinely copy all their data from one Cosmos instance to another and switch over traffic to the copy just to get a partitioning reset; it is one thing to have to do it; another to discover in production yourself it has to be done with no prior warning.. AWS DynamoDB used to do that. When I was working for a team in AWS we ended up discovering an unexpected bad hash choice, had a hot partition and ended up with a crazy number of partitions and a need to have really high r/w allocation on it. The only option at the time was to roll over to a new table (which we could do and fix our hash choice, thankfully, without too much hassle). They fixed that stuff a few years ago now so most (all?) of my "here be dragons" concerns about DynamoDB have been addressed.

danjc 4y ago

Spoke with a customer last week who said their cost for cosmos is on the order of 10x what it would be for SQL. I’ve learned to stay away from services that are high abstractions. Huge lock in, expensive and much more risk of it being eol’d. In our platform we’ve actually dropped SQL in favor of Az blob and table storage. It’s going to reduce our database costs by 5x.