17 ms·
This is a great list. The one that I see people ignore the most is the fact that spinning disk access is 20x the cost of a datacenter roundtrip. If you need l
by pcmonk 10y ago
This is a great list. The one that I see people ignore the most is the fact that spinning disk access is 20x the cost of a datacenter roundtrip. If you need low-latency persistence, something like Apache Kafka is a much better solution than a disk (as long as your scale justifies it).
- dleibovic 10y agoKafka is backed by spinning disks though. Am I missing something?
- yellowapple 10y agoI'm not familiar with Kafka specifically, but I reckon caching in memory is the way to work around that.
- AnimalMuppet 10y agoI think it's assuming that Kafka caches a lot more than your local machine, and therefore is less likely to need to access disk. Note that it also assumes low network latency, though; if you have congestion within the data center, things change.
- hadsed 10y agoGenerally Kafka takes advantage of the OS page cache and the fact that its data is sequential. The reason you can back Kafka effectively with spinning disks is because Kafka is just an immutable log. OTOH you won't always have sequential access so services that store data in memory cover some of those other cases (eg a KV store like Redis).
- edblarney 10y agoThere's networking overhead, hand-shaking, latency etc that I don't think is included in that.
- Declanomous 10y agoA big thing that is missing is dropped packets. Our IT director moved everything offsite with the same justification, and now we constantly have issues because our front-end software does not handle dropped packets well. Edit: Also, in 10,000 ns, light travels about 3 km in a vacuum. I know latency is addressed a bit in there, but our closest DC is about 70 km away from our main office. That's a pretty substantial hit on latency, especially considering our servers still have spinning discs in them.