2 ms·
It is relatively common for companies to not use open source for this type of application above a certain scale, even if they use open source for most other thi
by jandrewrogers 7y ago
It is relatively common for companies to not use open source for this type of application above a certain scale, even if they use open source for most other things. I've seen it happen at multiple companies big and small. There are two major reasons for this.
The first reason is that open source platforms struggle beyond a certain scale due to architectural weaknesses, which becomes an ongoing operational headache. Most companies just deal with it but it gets worse as the workload grows.
The second reason is that it is expensive to run the open source platforms due to their very low efficiency. I've seen companies reduce their hardware footprint by a factor of 10 by rolling their own metrics/time-series implementations due solely to superior software design. When you are running a petabyte of metrics per day through these systems, that adds up to a lot of money.
tl;dr: it is technically straightforward for a company to design their own metrics infrastructure that massively outperforms the open source tooling, and the limitations of the open source implementations are often painful enough as the data models scale up that many companies do.