Another possible approach: https://cloud.google.com/blog/big-data/2017/06/how-qubit-deduplicates-streaming-data-at-scale-with-google-cloud-platform https://clou

Another possible approach: https://cloud.google.com/blog/big-data/2017/06/how-qubit-deduplicates-streaming-data-at-scale-with-google-cloud-platform https://cloud.google.com/blog/big-data/2017/06/how-qubit-ded...