3 ms·
I've used it briefly internally at Microsoft 2009, I had almost the same experience. Super slow, takes a while to even start the query, non-standard query langu
by m0zg 8y ago
I've used it briefly internally at Microsoft 2009, I had almost the same experience. Super slow, takes a while to even start the query, non-standard query language. For internal users there were even more hurdles: you had to get approval and quota before you were able to use it at all, and the default answer to both requests was "no", you really had to justify it. As an aside, at MS "no" was the answer to any request you make of Legal. Lots of bicycles were reinvented from scratch, with lots bugs, just because Legal would not approve the use of BSD or MIT licensed code.
Imagine my shock when I started at Google a year later and discovered Dremel (now BigQuery) where I could just run a query and get the results back in seconds over tens of terabytes of data, with no quotas or approvals.
I think having a system like Dremel is a huge competitive advantage, because you don't have to guess blindly about things, you can just query them and get results immediately. If Cosmos is still the best MS has internally, they need to roll their own Dremel asap. Lots of great people from SQL Server team work at Google now, but MS still has plenty of people who could do a competent petascale columnar DB with one hand tied behind their back.
- CurtHagenlocher 8y agoIt sounds like you're describing the internal big data platform "Cosmos", something which has been made publicly-available as "Azure Data Lake Analytics" and which has no relationship to "CosmosDB" other than a similar name.
- metildaa 8y agoSounds like a branding mishap, Microsoft created the same type of hard to understand mess with the Microsoft Dynamics branding.
- int_19h 8y agoLike he said, the public product is called Azure Data Lake, and internal product names aren't brands (that's kind of the whole point of them).
- deleted 8y ago[deleted]
- dtrailin 8y agoThat's impressive, considering that this CosmosDB development only started in 2014! You are talking about Cosmos which is still around but is now used as the basis of Azure Data lake [0]. However, I think the closest MS has to Dremel is actually Kusto [1] which has a custom query language but works pretty well. [0] https://azure.microsoft.com/en-ca/solutions/data-lake/ https://azure.microsoft.com/en-ca/solutions/data-lake/ [1] https://docs.microsoft.com/en-us/azure/kusto/query/ https://docs.microsoft.com/en-us/azure/kusto/query/
- m0zg 8y agoYou're probably right. However, having worked at MS for nearly a decade in the past, I'd be stunned if Cosmos did not serve, at least in part, as the foundation, and CosmosDB did not inherit a good chunk of the team. Dremel now uses standard SQL and unlike Kusto it supports multiway joins and a bunch of other things expected of a more "general purpose" analytical DB. Ultimately MS efforts will be stymied by the fact that their underlying storage story is nowhere near as good as Google's Colossus. I won't disclose the details, but that's another thing that blew my mind: I could do linear reads (which is what columnar DBs do nearly all of the time) faster than I could process the data, from hundreds, or even thousands of workers at the same time. The IO throughput there is truly immense, which is especially impressive given that all of the storage is remote.
- CurtHagenlocher 8y agoCosmos and CosmosDB have next-to-nothing in common and were not built by the same people.
- manigandham 8y agoWhy would you be stunned? They are completely separate systems. It's like saying BigQuery and BigTable share something because they both have "Big" in their name.
- curiousDog 8y agoYou're conflating CosmosDB and Cosmos (which has been publicly released as Azure Data lake Analytics and botched imo).