Class DefaultTokenAnnotator

  • All Implemented Interfaces:
    Serializable

    public class DefaultTokenAnnotator
    extends Annotator
    Default token annotator that uses the Tokenizer registered with the token's language to perform tokenization.
    Author:
    David B. Bracewell
    See Also:
    Serialized Form
    • Constructor Detail

      • DefaultTokenAnnotator

        public DefaultTokenAnnotator()
    • Method Detail

      • annotateImpl

        protected void annotateImpl​(Document document)
        Description copied from class: Annotator
        Annotates a document with one or more AnnotatableType defined in satisfies().
        Specified by:
        annotateImpl in class Annotator
        Parameters:
        document - The document to annotate
      • getProvider

        public String getProvider​(Language language)
        Description copied from class: Annotator
        Gets the provider information for this annotator.. The provider information should relate to a version number, model used, or something else to identify the settings of the annotator.
        Overrides:
        getProvider in class Annotator
        Returns:
        the provider
      • satisfies

        public Set<AnnotatableType> satisfies()
        Description copied from class: Annotator
        The set of annotation types that this annotator satisfies by this annotator
        Specified by:
        satisfies in class Annotator
        Returns:
        the set of satisfied annotation types