4 ms·

> My understanding is that Cosmos has gotten a lot better, but man that left a sour taste in my mouth. A couple of years ago I stumbled upon a Azure project wh

> My understanding is that Cosmos has gotten a lot better, but man that left a sour taste in my mouth. A couple of years ago I stumbled upon a Azure project which started off using the old timey Cosmos DB. Looking at the repository history from those days, I saw a bunch of Entity Framework configurations and navigations and arcane wizardry that would take an engineer months to really figure out. Then there was an update to CosmosSDK, and all that EF noise was replaced by single CRUD operations that took the unserialized object, id and partition key as input. That's it. Worlds of difference, and a simple key-value query takes ~10ms to do. Yes, it's worlds of difference.

mort96 2y ago
> Worlds of difference, and a simple key-value query takes ~10ms to do. Unless that query goes over the Internet to another continent, that's a really long time isn't it?
- maccard 2y ago
  Yes it’s ages but not multi continent long. Us east 1 to us east 2 is about 10-15ms in my experience.
  - mort96 2y ago
    You're right, I exaggerated unnecessarily there. Sorry
    - maccard 2y ago
      No reason to apologise - I assumed and tried to give a better reference. It is an absolute eternity though. A KV lookup is fractions of a microsecond in a managed language like c#. A http request is in the microseconds range on localhost and a smidge more on a performant local network. A poorly behaved local network (busy wifi on an ISP router) is 1-2ms, and I can do a round trip to my nearest AWS region in 10-15ms from my home network. It’s an absolute eternity, and when thinking about this stuff and scaling, think “is it worth slowing down the normal case by 1000x to introduce an external service”
      - neonsunset 2y ago
        Some numbers no one asked for :) ConcurrentDictionary<K, V> read latency is going to be around 7-15ns for the data hot in the cache, scaling with key length if it is string-based, and anywhere between 75ns and 150ns for reading that out of RAM. Alternate implementations like NonBlocking.ConcurrentDictionary can reach down to 3.5-5ns for integer-based keys given all data is in L1 and the branch predictor is fully primed, on modern x86_64 cores.
        tombert 2y ago
        I've gotten in arguments over overuse of Redis for this very reason. Redis is cool but people use it too willy nilly. It's expensive to run, and you have to pay a cost of (de)serialization, and network latency. Sometimes that's necessary, especially if you need to share stuff across multiple nodes and you're afraid of hammering the database too much, but a lot of the time, I'd say even most of the time, you can get away with just a big ol' global ConcurrentDictionary (or whatever the equivalent is in your language of choice). Basically any thread-safe dictionary is fine, because no matter what kind of locking strategy they're using, it will almost certainly be faster than anything that has to hit the network, by several orders of magnitude, and you have one less thing to manage in your application. You can figure out the best way to invalidate old entries, or find a library to do it (e.g. something like a Guava cache), and you'll likely much better performance than arbitrarily farming to Redis.
        motorest 2y ago
        > (...) but a lot of the time, I'd say even most of the time, you can get away with just a big ol' global ConcurrentDictionary (or whatever the equivalent is in your language of choice). That's fine if all you're doing is caching limited data independently on each node and you have no requirement for consistency or even durability. Most of the usecases you stumble upon do not fit that scenario. That's why "a lot of the time" suggesting a dictionary is plain wrong.
        maccard 2y ago
- motorest 2y ago
  > Unless that query goes over the Internet to another continent, that's a really long time isn't it? If you're hosting a service in a cloud provider and you implement your services so that your call to the cloud provider's database goes over the internet to another continent, you have serious problems but none of them are caused by the cloud provider. Also, CosmosDB is globally distributed.
  - pbalau 2y ago
    They meant 10ms is slow.
    - motorest 2y ago
      > They meant 10ms is slow. I don't think anyone making that sort of claim knows anything about cloud services. A single roundtrip of a no-op request within the same data center takes 0.5ms. Add querying across multiple partitions and data seeking, and you don't find cloud providers doing better. To frame how oblivious that claim is, DynamoDB is lauded for it's response times being sub-20ms. https://stackoverflow.com/questions/34552625/how-to-get-sub-10ms-response-times-from-aws-dynamodb https://stackoverflow.com/questions/34552625/how-to-get-sub-...
      - mort96 2y ago
        No that's exactly what I mean: I expect the round-trip within a datacenter to take roughly 0.5ms, I expect the key lookup to take about that time or less, so 10ms is roughly an order of magnitude more than I would expect a "simple key-value query" to take
- jvans 2y ago
  For comparison if you run your own hardware and do a memcached KV lookup with a different server on the same rack, p99 times are slightly under 1ms. Given the guarantees of cosmosdb ~10ms isn't that bad for a p100

mort96 2y ago

> Worlds of difference, and a simple key-value query takes ~10ms to do. Unless that query goes over the Internet to another continent, that's a really long time isn't it?

maccard 2y ago
Yes it’s ages but not multi continent long. Us east 1 to us east 2 is about 10-15ms in my experience.
- mort96 2y ago
  You're right, I exaggerated unnecessarily there. Sorry
  - maccard 2y ago
    No reason to apologise - I assumed and tried to give a better reference. It is an absolute eternity though. A KV lookup is fractions of a microsecond in a managed language like c#. A http request is in the microseconds range on localhost and a smidge more on a performant local network. A poorly behaved local network (busy wifi on an ISP router) is 1-2ms, and I can do a round trip to my nearest AWS region in 10-15ms from my home network. It’s an absolute eternity, and when thinking about this stuff and scaling, think “is it worth slowing down the normal case by 1000x to introduce an external service”
    - neonsunset 2y ago
      Some numbers no one asked for :) ConcurrentDictionary<K, V> read latency is going to be around 7-15ns for the data hot in the cache, scaling with key length if it is string-based, and anywhere between 75ns and 150ns for reading that out of RAM. Alternate implementations like NonBlocking.ConcurrentDictionary can reach down to 3.5-5ns for integer-based keys given all data is in L1 and the branch predictor is fully primed, on modern x86_64 cores.
      - tombert 2y ago
        I've gotten in arguments over overuse of Redis for this very reason. Redis is cool but people use it too willy nilly. It's expensive to run, and you have to pay a cost of (de)serialization, and network latency. Sometimes that's necessary, especially if you need to share stuff across multiple nodes and you're afraid of hammering the database too much, but a lot of the time, I'd say even most of the time, you can get away with just a big ol' global ConcurrentDictionary (or whatever the equivalent is in your language of choice). Basically any thread-safe dictionary is fine, because no matter what kind of locking strategy they're using, it will almost certainly be faster than anything that has to hit the network, by several orders of magnitude, and you have one less thing to manage in your application. You can figure out the best way to invalidate old entries, or find a library to do it (e.g. something like a Guava cache), and you'll likely much better performance than arbitrarily farming to Redis.
        motorest 2y ago
        > (...) but a lot of the time, I'd say even most of the time, you can get away with just a big ol' global ConcurrentDictionary (or whatever the equivalent is in your language of choice). That's fine if all you're doing is caching limited data independently on each node and you have no requirement for consistency or even durability. Most of the usecases you stumble upon do not fit that scenario. That's why "a lot of the time" suggesting a dictionary is plain wrong.
        maccard 2y ago
motorest 2y ago
> Unless that query goes over the Internet to another continent, that's a really long time isn't it? If you're hosting a service in a cloud provider and you implement your services so that your call to the cloud provider's database goes over the internet to another continent, you have serious problems but none of them are caused by the cloud provider. Also, CosmosDB is globally distributed.
- pbalau 2y ago
  They meant 10ms is slow.
  - motorest 2y ago
    > They meant 10ms is slow. I don't think anyone making that sort of claim knows anything about cloud services. A single roundtrip of a no-op request within the same data center takes 0.5ms. Add querying across multiple partitions and data seeking, and you don't find cloud providers doing better. To frame how oblivious that claim is, DynamoDB is lauded for it's response times being sub-20ms. https://stackoverflow.com/questions/34552625/how-to-get-sub-10ms-response-times-from-aws-dynamodb https://stackoverflow.com/questions/34552625/how-to-get-sub-...
    - mort96 2y ago
      No that's exactly what I mean: I expect the round-trip within a datacenter to take roughly 0.5ms, I expect the key lookup to take about that time or less, so 10ms is roughly an order of magnitude more than I would expect a "simple key-value query" to take
jvans 2y ago
For comparison if you run your own hardware and do a memcached KV lookup with a different server on the same rack, p99 times are slightly under 1ms. Given the guarantees of cosmosdb ~10ms isn't that bad for a p100