AbstractTokenStream (com.gengoai:mango 1.1-SNAPSHOT API)

java.lang.Object
- com.gengoai.parsing.AbstractTokenStream

All Implemented Interfaces:

TokenStream, Serializable
```
public abstract class AbstractTokenStream
extends Object
implements TokenStream, Serializable
```
Abstract base token stream providing extracted tokens from a Lexer.

Author:

David B. Bracewell

See Also:

Serialized Form

Field Summary
- Fields inherited from interface com.gengoai.parsing.TokenStream
  EOF, EOF_TOKEN

Constructor Summary

Constructors
Constructor Description

AbstractTokenStream()

Method Summary

All Methods Instance Methods Abstract Methods Concrete Methods
Modifier and Type	Method	Description
`ParserToken`	`consume()`	Consumes a token from the stream.
`protected abstract List<ParserToken>`	`next()`	Gets the next available token.
`ParserToken`	`peek()`	Peeks at the next token on the stream.
`ParserToken`	`token()`	Returns the last token extracted via the call to consume.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.gengoai.parsing.TokenStream
consume, hasNext

- Constructor Detail
  - AbstractTokenStream
```
public AbstractTokenStream()
```
- Method Detail
  - consume
```
public final ParserToken consume()
```
    Description copied from interface: TokenStream
    
    Consumes a token from the stream.
    
    Specified by:
    
    consume in interface TokenStream
    
    Returns:
    
    the consumed token.
  - next
```
protected abstract List<ParserToken> next()
```
    Gets the next available token.
    
    Returns:
    
    the next ParserToken
  - peek
```
public final ParserToken peek()
```
    Description copied from interface: TokenStream
    
    Peeks at the next token on the stream.
    
    Specified by:
    
    peek in interface TokenStream
    
    Returns:
    
    the next token on the stream (special EOF token if no more tokens exists)
  - token
```
public final ParserToken token()
```
    Description copied from interface: TokenStream
    
    Returns the last token extracted via the call to consume.
    
    Specified by:
    
    token in interface TokenStream
    
    Returns:
    
    the last extracted token via consume