Package org.apache.storm.topology.base
Class BaseStatefulWindowedBolt<T extends State>
java.lang.Object
org.apache.storm.topology.base.BaseWindowedBolt
org.apache.storm.topology.base.BaseStatefulWindowedBolt<T>
- All Implemented Interfaces:
Serializable,IComponent,IStatefulComponent<T>,IStatefulWindowedBolt<T>,IWindowedBolt
public abstract class BaseStatefulWindowedBolt<T extends State>
extends BaseWindowedBolt
implements IStatefulWindowedBolt<T>
- See Also:
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.storm.topology.base.BaseWindowedBolt
BaseWindowedBolt.Count, BaseWindowedBolt.Duration -
Field Summary
Fields inherited from class org.apache.storm.topology.base.BaseWindowedBolt
timestampExtractor, windowConfiguration -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionbooleanIf the stateful windowed bolt should have its windows persisted in state and maintain a subset of events in memory.longThe maximum number of window events to keep in memory.voidpreCommit(long txid) This is a hook for the component to perform some actions just before the framework commits its state.voidprePrepare(long txid) This is a hook for the component to perform some actions just before the framework prepares its state.voidThis is a hook for the component to perform some actions just before the framework rolls back the prepared state.withLag(BaseWindowedBolt.Duration duration) Specify the maximum time lag of the tuple timestamp in milliseconds.withLateTupleStream(String streamName) Specify a stream id on which late tuples are going to be emitted.withMaxEventsInMemory(long maxEventsInMemory) The maximum number of window events to keep in memory.withMessageIdField(String fieldName) Specify the name of the field in the tuple that holds the message id.If set, the stateful windowed bolt would use the backend state for window persistence and only keep a sub-set of events in memory as specified bywithMaxEventsInMemory(long).withTimestampExtractor(TimestampExtractor timestampExtractor) Specify the timestamp extractor implementation.withTimestampField(String fieldName) Specify a field in the tuple that represents the timestamp as a long value.A count based tumbling window.withTumblingWindow(BaseWindowedBolt.Duration duration) A time duration based tumbling window.Specify the watermark event generation interval.withWindow(BaseWindowedBolt.Count windowLength) A tuple count based window that slides with every incoming tuple.withWindow(BaseWindowedBolt.Count windowLength, BaseWindowedBolt.Count slidingInterval) Tuple count based sliding window configuration.withWindow(BaseWindowedBolt.Count windowLength, BaseWindowedBolt.Duration slidingInterval) Tuple count and time duration based sliding window configuration.withWindow(BaseWindowedBolt.Duration windowLength) A time duration based window that slides with every incoming tuple.withWindow(BaseWindowedBolt.Duration windowLength, BaseWindowedBolt.Count slidingInterval) Time duration and count based sliding window configuration.withWindow(BaseWindowedBolt.Duration windowLength, BaseWindowedBolt.Duration slidingInterval) Time duration based sliding window configuration.Methods inherited from class org.apache.storm.topology.base.BaseWindowedBolt
cleanup, declareOutputFields, getComponentConfiguration, getTimestampExtractor, prepareMethods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitMethods inherited from interface org.apache.storm.topology.IComponent
declareOutputFields, getComponentConfigurationMethods inherited from interface org.apache.storm.topology.IStatefulComponent
initStateMethods inherited from interface org.apache.storm.topology.IWindowedBolt
cleanup, execute, getTimestampExtractor, prepare
-
Constructor Details
-
BaseStatefulWindowedBolt
public BaseStatefulWindowedBolt()
-
-
Method Details
-
withWindow
public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Count windowLength, BaseWindowedBolt.Count slidingInterval) Tuple count based sliding window configuration.- Overrides:
withWindowin classBaseWindowedBolt- Parameters:
windowLength- the number of tuples in the windowslidingInterval- the number of tuples after which the window slides
-
withWindow
public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Count windowLength, BaseWindowedBolt.Duration slidingInterval) Tuple count and time duration based sliding window configuration.- Overrides:
withWindowin classBaseWindowedBolt- Parameters:
windowLength- the number of tuples in the windowslidingInterval- the time duration after which the window slides
-
withWindow
public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Duration windowLength, BaseWindowedBolt.Count slidingInterval) Time duration and count based sliding window configuration.- Overrides:
withWindowin classBaseWindowedBolt- Parameters:
windowLength- the time duration of the windowslidingInterval- the number of tuples after which the window slides
-
withWindow
public BaseStatefulWindowedBolt<T> withWindow(BaseWindowedBolt.Duration windowLength, BaseWindowedBolt.Duration slidingInterval) Time duration based sliding window configuration.- Overrides:
withWindowin classBaseWindowedBolt- Parameters:
windowLength- the time duration of the windowslidingInterval- the time duration after which the window slides
-
withWindow
A tuple count based window that slides with every incoming tuple.- Overrides:
withWindowin classBaseWindowedBolt- Parameters:
windowLength- the number of tuples in the window
-
withWindow
A time duration based window that slides with every incoming tuple.- Overrides:
withWindowin classBaseWindowedBolt- Parameters:
windowLength- the time duration of the window
-
withTumblingWindow
A count based tumbling window.- Overrides:
withTumblingWindowin classBaseWindowedBolt- Parameters:
count- the number of tuples after which the window tumbles
-
withTumblingWindow
A time duration based tumbling window.- Overrides:
withTumblingWindowin classBaseWindowedBolt- Parameters:
duration- the time duration after which the window tumbles
-
withTimestampField
Specify a field in the tuple that represents the timestamp as a long value. If this field is not present in the incoming tuple, anIllegalArgumentExceptionwill be thrown. The field MUST contain a timestamp in milliseconds- Overrides:
withTimestampFieldin classBaseWindowedBolt- Parameters:
fieldName- the name of the field that contains the timestamp
-
withTimestampExtractor
Specify the timestamp extractor implementation.- Overrides:
withTimestampExtractorin classBaseWindowedBolt- Parameters:
timestampExtractor- theTimestampExtractorimplementation
-
withLateTupleStream
Specify a stream id on which late tuples are going to be emitted. They are going to be accessible via theWindowedBoltExecutor.LATE_TUPLE_FIELDfield. It must be defined on a per-component basis, and in conjunction with theBaseWindowedBolt.withTimestampField(java.lang.String), otherwiseIllegalArgumentExceptionwill be thrown.- Overrides:
withLateTupleStreamin classBaseWindowedBolt- Parameters:
streamName- the name of the stream used to emit late tuples on
-
withLag
Specify the maximum time lag of the tuple timestamp in milliseconds. It means that the tuple timestamps cannot be out of order by more than this amount.- Overrides:
withLagin classBaseWindowedBolt- Parameters:
duration- the max lag duration
-
withWatermarkInterval
Specify the watermark event generation interval. For tuple based timestamps, watermark events are used to track the progress of time- Overrides:
withWatermarkIntervalin classBaseWindowedBolt- Parameters:
interval- the interval at which watermark events are generated
-
withMessageIdField
Specify the name of the field in the tuple that holds the message id. This is used to track the windowing boundaries and re-evaluating the windowing operation during recovery of IStatefulWindowedBolt- Parameters:
fieldName- the name of the field that contains the message id
-
withPersistence
If set, the stateful windowed bolt would use the backend state for window persistence and only keep a sub-set of events in memory as specified bywithMaxEventsInMemory(long). -
withMaxEventsInMemory
The maximum number of window events to keep in memory. This is meaningful only ifwithPersistence()is also set. As the number of events in memory grows close to the maximum, the events that are less likely to be used again are evicted and persisted. The default value for this is1,000,000.- Parameters:
maxEventsInMemory- the maximum number of window events to keep in memory
-
isPersistent
public boolean isPersistent()Description copied from interface:IStatefulWindowedBoltIf the stateful windowed bolt should have its windows persisted in state and maintain a subset of events in memory.The default is to keep all the window events in memory.
- Specified by:
isPersistentin interfaceIStatefulWindowedBolt<T extends State>- Returns:
- true if the windows should be persisted
-
maxEventsInMemory
public long maxEventsInMemory()Description copied from interface:IStatefulWindowedBoltThe maximum number of window events to keep in memory.- Specified by:
maxEventsInMemoryin interfaceIStatefulWindowedBolt<T extends State>
-
preCommit
public void preCommit(long txid) Description copied from interface:IStatefulComponentThis is a hook for the component to perform some actions just before the framework commits its state.- Specified by:
preCommitin interfaceIStatefulComponent<T extends State>
-
prePrepare
public void prePrepare(long txid) Description copied from interface:IStatefulComponentThis is a hook for the component to perform some actions just before the framework prepares its state.- Specified by:
prePreparein interfaceIStatefulComponent<T extends State>
-
preRollback
public void preRollback()Description copied from interface:IStatefulComponentThis is a hook for the component to perform some actions just before the framework rolls back the prepared state.- Specified by:
preRollbackin interfaceIStatefulComponent<T extends State>
-