4 ms·
I would store the reservation date and truck length as separate data objects in the batch layer. You would tie them together with the reservation id. It isn't a
by nathanmarz 15y ago
I would store the reservation date and truck length as separate data objects in the batch layer. You would tie them together with the reservation id. It isn't a big deal to do those joins since the batch system is good at that kind of large-scale computation and you've already accepted high latency in the batch layer.
I wrote an article about graph schemas on the batch layer here:
http://nathanmarz.com/blog/thrift-graphs-strong-flexible-schemas-on-hadoop.html http://nathanmarz.com/blog/thrift-graphs-strong-flexible-sch...
The realtime db only accounts for the last few hours of data. Unless you're doing a query that only needs the last few hours of data, you need to do a merge between the batch view and the realtime view. We constantly flush old data out of our realtime db so as to keep it small and fast.