Getting Started

Learn

Community

Contribute

Archive

Monitors

Samza REST supports the ability to add Monitors to the service. Monitors are essentially tasks that can be scheduled to run periodically. It provides the capability to the users to define configurations that are specific to individual Monitors. These configurations are injected into the monitor instances through the Config instances.

Monitor configuration

All of the configuration keys for the monitors should be prefixed with monitor.{monitorName}. Since each monitor is expected to have an unique name, these prefixes provide the namespacing across the monitor configurations.

The following configurations are required for each of the monitors.

Name Default Description
monitor.monitorName.scheduling.interval.ms This defines the periodic scheduling interval in milliseconds for a monitor named monitorName. If this configuration is not defined, it is defaulted to 60 seconds.
monitor.monitorName.factory.class Required: This should contain a fully qualified name of a class that implements the MonitorFactory interface. Monitors that are instantiated by the factory implementation will be scheduled for periodic execution. Custom implementations of the MonitorFactory interface are expected to inject the Config and MetricsRegistry instances available in the createMonitor method into the Monitors.

For example, configurations for two monitors named NMTaskMonitor and RMTaskMonitor should be defined as follows.

monitor.RMTaskMonitor.factory.class=org.apache.samza.monitor.RMTaskMonitor

  monitor.RMTaskMonitor.scheduling.interval.ms=1000

  monitor.RMTaskMonitor.custom.config.key1=configValue1

  monitor.NMTaskMonitor.factory.class=org.apache.samza.monitor.NMTaskMonitor

  monitor.NMTaskMonitor.scheduling.interval.ms=2000

  monitor.NMTaskMonitor.custom.config.key2=configValue2

Implementing a New Monitor

Implement the Monitor interface with some behavior that should be executed periodically. The Monitor is Java code that invokes some method on the SAMZA Rest Service, runs a bash script to restart a failed NodeManager, or cleans old RocksDB sst files left by Host Affinity, for example.

Implement the MonitorFactory interface, which will be used to instantiate your Monitor. Each Monitor implementation should have a associated MonitorFactory implementation, which is responsible for instantiating the monitors.

Adding a New Monitor to the Samza REST Service

Add the fully-qualified class name of the MonitorFactory implementation to the monitor.monitorName.factory.class property in the service config. Set the config key monitor.monitorName.scheduling.interval.ms to the scheduling interval in milliseconds.

The configuration key monitor.monitorName.scheduling.interval.ms defines the periodic scheduling interval of the monitor() method in milli seconds.

Resource Reference »