3 ms·
Just an idea - but when your event collectors start to become unresponsive, why not redirect your load balancer to a SnowPlow-style "no moving parts" collector,
by alexatkeplar 14y ago
Just an idea - but when your event collectors start to become unresponsive, why not redirect your load balancer to a SnowPlow-style "no moving parts" collector, e.g. CloudFront + S3? You would have to merge the S3 log data into your main data store after the outage was over, but at least there would be no data loss (at least, not for events logged by GETs - POSTs are not supported by CloudFront), and webpages wouldn't be hanging.
The SnowPlow architecture diagram is here: https://github.com/snowplow/snowplow/wiki/Technical-architecture https://github.com/snowplow/snowplow/wiki/Technical-architec...