Package com.gengoai.stream.spark

Class SparkMAccumulator<IN,OUT>

java.lang.Object

com.gengoai.stream.spark.SparkMAccumulator<IN,OUT>

Type Parameters:

IN - the type parameter for what is being accumulated

OUT - the type parameter for the result of the accumulation

All Implemented Interfaces:

MAccumulator<IN,OUT>, Serializable

Direct Known Subclasses:

SparkMCounterAccumulator, SparkMDoubleAccumulator, SparkMLongAccumulator, SparkMMapAccumulator, SparkMMultiCounterAccumulator, SparkMStatisticsAccumulator

public class SparkMAccumulator<IN,OUT>
extends Object
implements MAccumulator<IN,OUT>

Generic accumulator for distributed Spark streams.

Author:

David B. Bracewell

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field	Description
protected org.apache.spark.util.AccumulatorV2<IN,OUT>	accumulatorV2	The AccumulatorV2 being used.

Constructor Summary

Constructors

Constructor	Description
SparkMAccumulator(LocalMAccumulator<IN,OUT> localMAccumulator)	Instantiates a new Spark m accumulator by wrapping a local accumulator.
SparkMAccumulator(org.apache.spark.util.AccumulatorV2<IN,OUT> accumulatorV2)	Instantiates a new Spark m accumulator.

Method Summary

Modifier and Type	Method	Description
void	add(IN in)	Adds an item to the accumulator
boolean	isZero()	Determines if the accumulator is a zero value
void	merge(MAccumulator<IN,OUT> other)	Merges another accumulator with this one
Optional<String>	name()	The name of the accumulator
void	register()	Registers the accumulator.
void	reset()	Resets the accumulator to its zero-value.
OUT	value()	The value of the accumulator.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.gengoai.stream.MAccumulator

report

Field Detail

accumulatorV2

protected final org.apache.spark.util.AccumulatorV2<IN,OUT> accumulatorV2

The AccumulatorV2 being used.

Constructor Detail

SparkMAccumulator

public SparkMAccumulator(org.apache.spark.util.AccumulatorV2<IN,OUT> accumulatorV2)

Instantiates a new Spark m accumulator.

Parameters:

accumulatorV2 - the accumulator v 2

SparkMAccumulator

public SparkMAccumulator(LocalMAccumulator<IN,OUT> localMAccumulator)

Instantiates a new Spark m accumulator by wrapping a local accumulator.

Parameters:

localMAccumulator - the local accumulator to wrap

Method Detail

add

public void add(IN in)

Description copied from interface: MAccumulator

Adds an item to the accumulator

Specified by:

add in interface MAccumulator<IN,OUT>

Parameters:

in - the item to add

merge

public void merge(MAccumulator<IN,OUT> other)

Description copied from interface: MAccumulator

Merges another accumulator with this one

Specified by:

merge in interface MAccumulator<IN,OUT>

Parameters:

other - the other accumulator to merge

name

public Optional<String> name()

Description copied from interface: MAccumulator

The name of the accumulator

Specified by:

name in interface MAccumulator<IN,OUT>

Returns:

the optional name of the accumulator

reset

public void reset()

Description copied from interface: MAccumulator

Resets the accumulator to its zero-value.

Specified by:

reset in interface MAccumulator<IN,OUT>

value

public OUT value()

Description copied from interface: MAccumulator

The value of the accumulator.

Specified by:

value in interface MAccumulator<IN,OUT>

Returns:

the result of the accumulator

isZero

public boolean isZero()

Description copied from interface: MAccumulator

Determines if the accumulator is a zero value

Specified by:

isZero in interface MAccumulator<IN,OUT>

Returns:

True if the accumulator is in a zero state

register

public void register()

Description copied from interface: MAccumulator

Registers the accumulator.

Specified by:

register in interface MAccumulator<IN,OUT>