3 ms·
We use MR using Pig (data in cassandra/CFS) with a 6 node hadoop cluster to process timeseries data. The events contain user metrics like which view was tapped,
by zengr 13y ago
We use MR using Pig (data in cassandra/CFS) with a 6 node hadoop cluster to process timeseries data. The events contain user metrics like which view was tapped, user behavior, search result, clicks etc.
We process these events to use it downstream for our search relevancy, internal metrics, see top products.
We did this on mysql for a long time but things went really slow. We could have optimized mysql for performance but cassandra was an easier way to go about it and it works for us for now.
- oacgnol 13y agoCan you estimate how much faster your processing is now vs. before on MySQL? I find it interesting that your cluster is only 6 nodes - relatively small compared to what I've seen and from what I've read about. It'd be interesting to know the benefits of small-scale usage of big data tech.
- alecco 13y agoDid you try a columnar database first? Vertica, InfinyDB, MonetDB... There are many.