Package com.gengoai.stream.local

Class AbstractLocalMPairStream<K,V>

java.lang.Object

com.gengoai.stream.local.AbstractLocalMPairStream<K,V>

All Implemented Interfaces:

MPairStream<K,V>, Serializable, AutoCloseable

Direct Known Subclasses:

LocalDefaultMPairStream

public abstract class AbstractLocalMPairStream<K,V>
extends Object
implements MPairStream<K,V>, Serializable

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor	Description
AbstractLocalMPairStream()

Method Summary

Modifier and Type	Method	Description
protected abstract MStream<Map.Entry<K,V>>	asMStream()
MPairStream<K,V>	cache()	Caches the stream.
void	close()
List<Map.Entry<K,V>>	collectAsList()	Collects the items in the stream as a list
Map<K,V>	collectAsMap()	Collects the items in the stream as a map
long	count()	The number of items in the stream
MPairStream<K,V>	filter(@NonNull SerializableBiPredicate<? super K,? super V> predicate)	Filters the stream.
MPairStream<K,V>	filterByKey(@NonNull SerializablePredicate<K> predicate)	Filters the stream by key.
MPairStream<K,V>	filterByValue(@NonNull SerializablePredicate<V> predicate)	Filters the stream by value.
<R,V1> MPairStream<R,V1>	flatMapToPair(SerializableBiFunction<? super K,? super V,Stream<Map.Entry<? extends R,? extends V1>>> function)	Maps the key-value pairs to one or more new key-value pairs
void	forEach(@NonNull SerializableBiConsumer<? super K,? super V> consumer)	Performs an operation on each item in the stream
void	forEachLocal(@NonNull SerializableBiConsumer<? super K,? super V> consumer)	Performs an operation on each item in the stream ensuring that is done locally and not distributed.
StreamingContext	getContext()	Gets the context used to create the stream
MPairStream<K,Iterable<V>>	groupByKey()	Group bys the items in the stream by the key
boolean	isEmpty()	Determines if the stream is empty or not
Stream<Map.Entry<K,V>>	javaStream()
<V1> MPairStream<K,Map.Entry<V,V1>>	join(@NonNull MPairStream<? extends K,? extends V1> other)	Performs an inner join between this stream and another using the key to match.
MStream<K>	keys()	Returns a stream over the keys in this pair stream
<V1> MPairStream<K,Map.Entry<V,V1>>	leftOuterJoin(@NonNull MPairStream<? extends K,? extends V1> other)	Performs a left outer join between this stream and another using the key to match.
<R> MStream<R>	map(@NonNull SerializableBiFunction<? super K,? super V,? extends R> function)	Maps the key-value pairs to a new object
MDoubleStream	mapToDouble(@NonNull SerializableToDoubleBiFunction<? super K,? super V> function)	Maps the key-value pairs to doubles
<R,V1> MPairStream<R,V1>	mapToPair(@NonNull SerializableBiFunction<? super K,? super V,? extends Map.Entry<? extends R,? extends V1>> function)	Maps the key-value pairs to new key-value pairs
Optional<Map.Entry<K,V>>	max(@NonNull SerializableComparator<Map.Entry<K,V>> comparator)	Returns the max item in the stream using the given comparator to compare items.
Optional<Map.Entry<K,V>>	min(SerializableComparator<Map.Entry<K,V>> comparator)	Returns the min item in the stream requiring that the items be comparable.
MPairStream<K,V>	onClose(@NonNull SerializableRunnable closeHandler)	Sets the handler to call when the stream is closed.
MPairStream<K,V>	parallel()	Ensures that the stream is parallel or distributed.
MPairStream<K,V>	persist(@NonNull StorageLevel storageLevel)
MPairStream<K,V>	reduceByKey(@NonNull SerializableBinaryOperator<V> operator)	Performs a reduction by key on the elements of this stream using the given binary operator.
MPairStream<K,V>	repartition(int numPartitions)	Repartitions the stream to the given number of partitions.
<V1> MPairStream<K,Map.Entry<V,V1>>	rightOuterJoin(@NonNull MPairStream<? extends K,? extends V1> other)	Performs a right outer join between this stream and another using the key to match.
MPairStream<K,V>	sample(boolean withReplacement, long number)	Randomly samples number items from the stream.
MPairStream<K,V>	shuffle(@NonNull Random random)	Shuffles the items in the string using the given Random object.
MPairStream<K,V>	sortByKey(@NonNull SerializableComparator<K> comparator)	Sorts the items in the stream by key using the given comparator.
MPairStream<K,V>	union(@NonNull MPairStream<? extends K,? extends V> other)	Unions this stream with another.
MStream<V>	values()	Returns a stream of values

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.gengoai.stream.MPairStream

isDistributed, isReusable, maxByKey, maxByKey, maxByValue, maxByValue, minByKey, minByKey, minByValue, minByValue, shuffle, sortByKey, updateConfig

Constructor Detail

AbstractLocalMPairStream

public AbstractLocalMPairStream()

Method Detail

asMStream

protected abstract MStream<Map.Entry<K,V>> asMStream()

javaStream

public Stream<Map.Entry<K,V>> javaStream()

Specified by:

javaStream in interface MPairStream<K,V>

cache

public MPairStream<K,V> cache()

Description copied from interface: MPairStream

Caches the stream.

Specified by:

cache in interface MPairStream<K,V>

Returns:

the cached stream

collectAsList

public List<Map.Entry<K,V>> collectAsList()

Description copied from interface: MPairStream

Collects the items in the stream as a list

Specified by:

collectAsList in interface MPairStream<K,V>

Returns:

the list of items in the stream

collectAsMap

public Map<K,V> collectAsMap()

Description copied from interface: MPairStream

Collects the items in the stream as a map

Specified by:

collectAsMap in interface MPairStream<K,V>

Returns:

the map of items in the stream

count

public long count()

Description copied from interface: MPairStream

The number of items in the stream

Specified by:

count in interface MPairStream<K,V>

Returns:

the number of items in the stream

filter

public MPairStream<K,V> filter(@NonNull
                                     @NonNull SerializableBiPredicate<? super K,? super V> predicate)

Description copied from interface: MPairStream

Filters the stream.

Specified by:

filter in interface MPairStream<K,V>

Parameters:

predicate - the predicate to use to determine which objects are kept

Returns:

the new stream

filterByKey

public MPairStream<K,V> filterByKey(@NonNull
                                          @NonNull SerializablePredicate<K> predicate)

Description copied from interface: MPairStream

Filters the stream by key.

Specified by:

filterByKey in interface MPairStream<K,V>

Parameters:

predicate - the predicate to apply to keys in order to determine which objects are kept

Returns:

the new stream

filterByValue

public MPairStream<K,V> filterByValue(@NonNull
                                            @NonNull SerializablePredicate<V> predicate)

Description copied from interface: MPairStream

Filters the stream by value.

Specified by:

filterByValue in interface MPairStream<K,V>

Parameters:

predicate - the predicate to apply to values in order to determine which objects are kept

Returns:

the new stream

flatMapToPair

public <R,V1> MPairStream<R,V1> flatMapToPair(SerializableBiFunction<? super K,? super V,Stream<Map.Entry<? extends R,? extends V1>>> function)

Description copied from interface: MPairStream

Maps the key-value pairs to one or more new key-value pairs

Specified by:

flatMapToPair in interface MPairStream<K,V>

Type Parameters:

R - the new key type parameter

V1 - the new value type parameter

Parameters:

function - the function to map key-value pairs

Returns:

the new pair stream

forEach

public void forEach(@NonNull
                    @NonNull SerializableBiConsumer<? super K,? super V> consumer)

Description copied from interface: MPairStream

Performs an operation on each item in the stream

Specified by:

forEach in interface MPairStream<K,V>

Parameters:

consumer - the consumer action to perform

forEachLocal

public void forEachLocal(@NonNull
                         @NonNull SerializableBiConsumer<? super K,? super V> consumer)

Description copied from interface: MPairStream

Performs an operation on each item in the stream ensuring that is done locally and not distributed.

Specified by:

forEachLocal in interface MPairStream<K,V>

Parameters:

consumer - the consumer action to perform

getContext

public StreamingContext getContext()

Description copied from interface: MPairStream

Gets the context used to create the stream

Specified by:

getContext in interface MPairStream<K,V>

Returns:

the context

groupByKey

public MPairStream<K,Iterable<V>> groupByKey()

Description copied from interface: MPairStream

Group bys the items in the stream by the key

Specified by:

groupByKey in interface MPairStream<K,V>

Returns:

the new stream

persist

public MPairStream<K,V> persist(@NonNull
                                      @NonNull StorageLevel storageLevel)

Specified by:

persist in interface MPairStream<K,V>

isEmpty

public boolean isEmpty()

Description copied from interface: MPairStream

Determines if the stream is empty or not

Specified by:

isEmpty in interface MPairStream<K,V>

Returns:

True if empty, False otherwise

join

public <V1> MPairStream<K,Map.Entry<V,V1>> join(@NonNull
                                                            @NonNull MPairStream<? extends K,? extends V1> other)

Description copied from interface: MPairStream

Performs an inner join between this stream and another using the key to match.

Specified by:

join in interface MPairStream<K,V>

Type Parameters:

V1 - the value type parameter of the stream to join with

Parameters:

other - the other stream to inner join with

Returns:

the new stream

keys

public MStream<K> keys()

Description copied from interface: MPairStream

Returns a stream over the keys in this pair stream

Specified by:

keys in interface MPairStream<K,V>

Returns:

the key stream

leftOuterJoin

public <V1> MPairStream<K,Map.Entry<V,V1>> leftOuterJoin(@NonNull
                                                                     @NonNull MPairStream<? extends K,? extends V1> other)

Description copied from interface: MPairStream

Performs a left outer join between this stream and another using the key to match.

Specified by:

leftOuterJoin in interface MPairStream<K,V>

Type Parameters:

V1 - the value type parameter of the stream to join with

Parameters:

other - the other stream to left outer join with

Returns:

the new stream

map

public <R> MStream<R> map(@NonNull
                          @NonNull SerializableBiFunction<? super K,? super V,? extends R> function)

Description copied from interface: MPairStream

Maps the key-value pairs to a new object

Specified by:

map in interface MPairStream<K,V>

Type Parameters:

R - the type parameter being mapped to

Parameters:

function - the function to map from key-value pairs to objects of type R

Returns:

the new stream

mapToDouble

public MDoubleStream mapToDouble(@NonNull
                                 @NonNull SerializableToDoubleBiFunction<? super K,? super V> function)

Description copied from interface: MPairStream

Maps the key-value pairs to doubles

Specified by:

mapToDouble in interface MPairStream<K,V>

Parameters:

function - the function to map from key-value pairs to double values

Returns:

the new double stream

mapToPair

public <R,V1> MPairStream<R,V1> mapToPair(@NonNull
                                                      @NonNull SerializableBiFunction<? super K,? super V,? extends Map.Entry<? extends R,? extends V1>> function)

Description copied from interface: MPairStream

Maps the key-value pairs to new key-value pairs

Specified by:

mapToPair in interface MPairStream<K,V>

Type Parameters:

R - the new key type parameter

V1 - the new value type parameter

Parameters:

function - the function to map key-value pairs

Returns:

the new pair stream

max

public Optional<Map.Entry<K,V>> max(@NonNull
                                          @NonNull SerializableComparator<Map.Entry<K,V>> comparator)

Description copied from interface: MPairStream

Returns the max item in the stream using the given comparator to compare items.

Specified by:

max in interface MPairStream<K,V>

Parameters:

comparator - the comparator to use to compare values in the stream

Returns:

the optional containing the max value

min

public Optional<Map.Entry<K,V>> min(SerializableComparator<Map.Entry<K,V>> comparator)

Description copied from interface: MPairStream

Returns the min item in the stream requiring that the items be comparable.

Specified by:

min in interface MPairStream<K,V>

Returns:

the optional containing the min value

onClose

public MPairStream<K,V> onClose(@NonNull
                                      @NonNull SerializableRunnable closeHandler)

Description copied from interface: MPairStream

Sets the handler to call when the stream is closed. Typically, this is to clean up any open resources, such as file handles.

Specified by:

onClose in interface MPairStream<K,V>

Parameters:

closeHandler - the handler to run when the stream is closed.

parallel

public MPairStream<K,V> parallel()

Description copied from interface: MPairStream

Ensures that the stream is parallel or distributed.

Specified by:

parallel in interface MPairStream<K,V>

Returns:

the new stream

reduceByKey

public MPairStream<K,V> reduceByKey(@NonNull
                                          @NonNull SerializableBinaryOperator<V> operator)

Description copied from interface: MPairStream

Performs a reduction by key on the elements of this stream using the given binary operator.

Specified by:

reduceByKey in interface MPairStream<K,V>

Parameters:

operator - the binary operator used to combine two objects

Returns:

the new stream containing keys and reduced values

repartition

public MPairStream<K,V> repartition(int numPartitions)

Description copied from interface: MPairStream

Repartitions the stream to the given number of partitions. This may be a no-op for some streams, i.e. Local Streams.

Specified by:

repartition in interface MPairStream<K,V>

Parameters:

numPartitions - the number of partitions the stream should have

Returns:

the new stream

rightOuterJoin

public <V1> MPairStream<K,Map.Entry<V,V1>> rightOuterJoin(@NonNull
                                                                      @NonNull MPairStream<? extends K,? extends V1> other)

Description copied from interface: MPairStream

Performs a right outer join between this stream and another using the key to match.

Specified by:

rightOuterJoin in interface MPairStream<K,V>

Type Parameters:

V1 - the value type parameter of the stream to join with

Parameters:

other - the other stream to right outer join with

Returns:

the new stream

sample

public MPairStream<K,V> sample(boolean withReplacement,
                                     long number)

Description copied from interface: MPairStream

Randomly samples number items from the stream.

Specified by:

sample in interface MPairStream<K,V>

Parameters:

withReplacement - true allow a single item to be represented in the sample multiple times, false allow a single item to only be picked once.

number - the number of items desired in the sample

Returns:

the new stream

shuffle

public MPairStream<K,V> shuffle(@NonNull
                                      @NonNull Random random)

Description copied from interface: MPairStream

Shuffles the items in the string using the given Random object.

Specified by:

shuffle in interface MPairStream<K,V>

Parameters:

random - the random number generator

Returns:

the new stream

sortByKey

public MPairStream<K,V> sortByKey(@NonNull
                                        @NonNull SerializableComparator<K> comparator)

Description copied from interface: MPairStream

Sorts the items in the stream by key using the given comparator.

Specified by:

sortByKey in interface MPairStream<K,V>

Parameters:

comparator - The comparator to use to comapre keys

Returns:

the new stream

union

public MPairStream<K,V> union(@NonNull
                                    @NonNull MPairStream<? extends K,? extends V> other)

Description copied from interface: MPairStream

Unions this stream with another.

Specified by:

union in interface MPairStream<K,V>

Parameters:

other - the other stream to add to this one.

Returns:

the new stream

values

public MStream<V> values()

Description copied from interface: MPairStream

Returns a stream of values

Specified by:

values in interface MPairStream<K,V>

Returns:

the new stream of values

close

public void close()
           throws Exception

Specified by:

close in interface AutoCloseable

Throws:

Exception