org.apache.samza.storage

Interface TaskBackupManager

public interface TaskBackupManager

TaskBackupManager is the interface that must be implemented for any remote system that Samza persists its state to during the task commit operation. snapshot(CheckpointId) will be evoked synchronous to task processing and get a snapshot of the stores state to be persisted for the commit. upload(CheckpointId, Map) will then use the snapshotted state to persist to the underlying backup system and will be asynchronous to task processing.

The interface will be evoked in the following way:

• Snapshot will be called before Upload.
• persistToFilesystem will be called after Upload is completed
• Cleanup is only called after Upload and persistToFilesystem has successfully completed

Method Summary

Modifier and Type	Method and Description
java.util.concurrent.CompletableFuture<java.lang.Void>	cleanUp(CheckpointId checkpointId, java.util.Map<java.lang.String,java.lang.String> stateCheckpointMarkers) Cleanup any local or remote state for checkpoint information that is older than the provided checkpointId This operation is required to be idempotent.
void	close() Shutdown hook the backup manager to cleanup any allocated resources
void	init(Checkpoint checkpoint) Initializes the TaskBackupManager instance.
java.util.Map<java.lang.String,java.lang.String>	snapshot(CheckpointId checkpointId) Snapshot is used to capture the current state of the stores in order to persist it to the backup manager in the upload(CheckpointId, Map) (CheckpointId, Map)} phase.
java.util.concurrent.CompletableFuture<java.util.Map<java.lang.String,java.lang.String>>	upload(CheckpointId checkpointId, java.util.Map<java.lang.String,java.lang.String> stateCheckpointMarkers) Upload is used to persist the state provided by the snapshot(CheckpointId) to the underlying backup system.

Method Detail

init

void init(@Nullable
          Checkpoint checkpoint)

Initializes the TaskBackupManager instance.

Parameters:

checkpoint - last recorded checkpoint from the CheckpointManager or null if no last checkpoint was found

snapshot

java.util.Map<java.lang.String,java.lang.String> snapshot(CheckpointId checkpointId)

Snapshot is used to capture the current state of the stores in order to persist it to the backup manager in the upload(CheckpointId, Map) (CheckpointId, Map)} phase. Performs the commit operation that is synchronous to processing. Returns the per store name state checkpoint markers to be used in upload.

Parameters:

checkpointId - CheckpointId of the current commit

Returns:

a map of store name to state checkpoint markers for stores managed by this state backend

upload

java.util.concurrent.CompletableFuture<java.util.Map<java.lang.String,java.lang.String>> upload(CheckpointId checkpointId,
                                                                                                java.util.Map<java.lang.String,java.lang.String> stateCheckpointMarkers)

Upload is used to persist the state provided by the snapshot(CheckpointId) to the underlying backup system. Commit operation that is asynchronous to message processing and returns a CompletableFuture containing the successfully uploaded state checkpoint markers .

Parameters:

checkpointId - CheckpointId of the current commit

stateCheckpointMarkers - the map of storename to state checkpoint markers returned by snapshot(CheckpointId)

Returns:

a CompletableFuture containing a map of store name to state checkpoint markers after the upload is complete

cleanUp

java.util.concurrent.CompletableFuture<java.lang.Void> cleanUp(CheckpointId checkpointId,
                                                               java.util.Map<java.lang.String,java.lang.String> stateCheckpointMarkers)

Cleanup any local or remote state for checkpoint information that is older than the provided checkpointId This operation is required to be idempotent.

Parameters:

checkpointId - the CheckpointId of the last successfully committed checkpoint

stateCheckpointMarkers - a map of store name to state checkpoint markers returned by upload(CheckpointId, Map) (CheckpointId, Map)} upload}

close

void close()

Shutdown hook the backup manager to cleanup any allocated resources