public interface MessageChooser
Consider the case of a Samza task consuming multiple streams, where some streams may be from live systems that have stricter SLA requirements and must always be prioritized over other streams that may be from batch systems. MessageChooser allows developers to inject message prioritization logic into the SamzaContainer.
In general, the MessageChooser can be used to prioritize certain systems, streams or partitions over others. It can also be used to throttle certain partitions, by choosing not to return messages even though they are available. The MessageChooser can also throttle the entire SamzaContainer by performing a blocking operation, such as Thread.sleep.
The manner in which MessageChooser is used is:
Since the MessageChooser only receives one message at a time per
SystemStreamPartition, it can be used to order messages between different
SystemStreamPartitions, but it can't be used to re-order messages within a
single SystemStreamPartition (a buffered sort). This must be done within a
The contract between the MessageChooser and the SystemConsumers is:
update(IncomingMessageEnvelope)can be called multiple times before
choose()returns null, that means no envelopes should be processed at the moment.
choose()is called, even if unprocessed messages have been given by the update method.
|Modifier and Type||Method and Description|
The choose method is invoked when the SamzaContainer is ready to process a new message.
Called before start, to let the chooser know that it will be handling envelopes from the given SystemStreamPartition.
Called after all SystemStreamPartitions have been registered.
Called when the chooser is about to be discarded.
Notify the chooser that a new envelope is available for a processing.
void register(SystemStreamPartition systemStreamPartition, java.lang.String offset)
systemStreamPartition- A SystemStreamPartition that envelopes will be coming from.
offset- The offset of the first message expected for the system/stream/partition that's being registered. If "7" were supplied as the offset, then the MessageChooser can expect the first message it is updated with for the system/stream/partition will have an offset of "7".
void update(IncomingMessageEnvelope envelope)
envelope- An unprocessed envelope.