ENTokenizer (com.gengoai:hermes 1.1-SNAPSHOT API)

java.lang.Object
- com.gengoai.hermes.en.ENTokenizer

All Implemented Interfaces:

Tokenizer, Serializable
```
public class ENTokenizer
extends Object
implements Tokenizer, Serializable
```
English language tokenizer

Author:

David B. Bracewell

See Also:

Serialized Form

- Nested Class Summary
  - Nested classes/interfaces inherited from interface com.gengoai.hermes.morphology.Tokenizer
    Tokenizer.Token
- Constructor Summary
  
  Constructors
  Constructor Description
  
  ENTokenizer()
  Instantiates a new English tokenizer.
- Method Summary
  
  All Methods Instance Methods Concrete Methods
  Modifier and Type Method Description
  
  Iterable<Tokenizer.Token> tokenize(@NonNull Reader reader)
  Tokenizes an given reader into tokens.
  - Methods inherited from class java.lang.Object
    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
  - Methods inherited from interface com.gengoai.hermes.morphology.Tokenizer
    tokenize

- Constructor Detail
  - ENTokenizer
```
public ENTokenizer()
```
    Instantiates a new English tokenizer.
- Method Detail
  - tokenize
```
public Iterable<Tokenizer.Token> tokenize(@NonNull
                                          @NonNull Reader reader)
```
    Description copied from interface: Tokenizer
    
    Tokenizes an given reader into tokens. All IO errors should be rethrown as runtime exceptions.
    
    Specified by:
    
    tokenize in interface Tokenizer
    
    Parameters:
    
    reader - the reader
    
    Returns:
    
    an iterable of tokens.