3 ms·
partitioned state that the runtime then handles for the user Yes, when "wiring up" a bolt to a spout or another bolt, you can say that it should receive tuples
by samstokes 15y ago
partitioned state that the runtime then handles for the user
Yes, when "wiring up" a bolt to a spout or another bolt, you can say that it should receive tuples grouped by one or more of the tuple fields. (e.g. tuples [user, num_events] grouped by "user".) Then Storm takes care of the consistent hashing needed to ensure that each instance of the bolt receives all of the tuples in a given group (e.g. instance 1 gets all events for user 42, instance 2 gets all events for user 53).
I had assumed there was no arbitrary state because of the replay semantics.
Storm itself doesn't implement replay; it relies on the ability of an external event source (e.g. a message queue) to replay tuples if needed. It just provides the ability to notify the source of whether replay is needed (by tracking the tuples as they flow through the topology). So whether or not you replay tuples is an application choice.
But yes, if you have stateful bolts and replay then you do need to make sure that processing a given tuple is idempotent.