Uses of Class
org.apache.lucene.analysis.TokenStream

Packages that use TokenStream
org.apache.lucene.analysis API and code to convert text into indexable tokens. 
org.apache.lucene.analysis.de Support for indexing and searching of German text. 
org.apache.lucene.analysis.ru Support for indexing and searching Russian text. 
org.apache.lucene.analysis.standard A grammar-based tokenizer constructed with JavaCC.  
org.apache.lucene.search.highlight The highlight package contains classes to provide "keyword in context" features typically used to highlight search terms in the text of results pages.  
 

Uses of TokenStream in org.apache.lucene.analysis
 

Subclasses of TokenStream in org.apache.lucene.analysis
 class CharTokenizer
          An abstract base class for simple, character-oriented tokenizers.
 class LetterTokenizer
          A LetterTokenizer is a tokenizer that divides text at non-letters.
 class LowerCaseFilter
          Normalizes token text to lower case.
 class LowerCaseTokenizer
          LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together.
 class PorterStemFilter
          Transforms the token stream as per the Porter stemming algorithm.
 class StopFilter
          Removes stop words from a token stream.
 class TokenFilter
          A TokenFilter is a TokenStream whose input is another token stream.
 class Tokenizer
          A Tokenizer is a TokenStream whose input is a Reader.
 class WhitespaceTokenizer
          A WhitespaceTokenizer is a tokenizer that divides text at whitespace.
 

Fields in org.apache.lucene.analysis declared as TokenStream
protected  TokenStream TokenFilter.input
          The source of tokens for this filter.
 

Methods in org.apache.lucene.analysis that return TokenStream
 TokenStream StopAnalyzer.tokenStream(String fieldName, Reader reader)
          Filters LowerCaseTokenizer with StopFilter.
 TokenStream SimpleAnalyzer.tokenStream(String fieldName, Reader reader)
           
 TokenStream PerFieldAnalyzerWrapper.tokenStream(String fieldName, Reader reader)
           
 TokenStream Analyzer.tokenStream(String fieldName, Reader reader)
          Creates a TokenStream which tokenizes all the text in the provided Reader.
 TokenStream Analyzer.tokenStream(Reader reader)
          Deprecated. use tokenStream(String, Reader) instead.
 TokenStream WhitespaceAnalyzer.tokenStream(String fieldName, Reader reader)
           
 

Constructors in org.apache.lucene.analysis with parameters of type TokenStream
StopFilter(TokenStream in, String[] stopWords)
          Constructs a filter which removes words from the input TokenStream that are named in the array of words.
StopFilter(TokenStream in, Hashtable stopTable)
          Deprecated. Use StopFilter.StopFilter(TokenStream, Set) instead
StopFilter(TokenStream in, Set stopWords)
          Constructs a filter which removes words from the input TokenStream that are named in the Set.
TokenFilter(TokenStream input)
          Construct a token stream filtering the given input.
PorterStemFilter(TokenStream in)
           
LowerCaseFilter(TokenStream in)
           
 

Uses of TokenStream in org.apache.lucene.analysis.de
 

Subclasses of TokenStream in org.apache.lucene.analysis.de
 class GermanStemFilter
          A filter that stems German words.
 

Methods in org.apache.lucene.analysis.de that return TokenStream
 TokenStream GermanAnalyzer.tokenStream(String fieldName, Reader reader)
          Creates a TokenStream which tokenizes all the text in the provided Reader.
 

Constructors in org.apache.lucene.analysis.de with parameters of type TokenStream
GermanStemFilter(TokenStream in)
           
GermanStemFilter(TokenStream in, Hashtable exclusiontable)
          Deprecated. Use GermanStemFilter.GermanStemFilter(org.apache.lucene.analysis.TokenStream, java.util.Set) instead.
GermanStemFilter(TokenStream in, Set exclusionSet)
          Builds a GermanStemFilter that uses an exclusiontable.
 

Uses of TokenStream in org.apache.lucene.analysis.ru
 

Subclasses of TokenStream in org.apache.lucene.analysis.ru
 class RussianLetterTokenizer
          A RussianLetterTokenizer is a tokenizer that extends LetterTokenizer by additionally looking up letters in a given "russian charset".
 class RussianLowerCaseFilter
          Normalizes token text to lower case, analyzing given ("russian") charset.
 class RussianStemFilter
          A filter that stems Russian words.
 

Methods in org.apache.lucene.analysis.ru that return TokenStream
 TokenStream RussianAnalyzer.tokenStream(String fieldName, Reader reader)
          Creates a TokenStream which tokenizes all the text in the provided Reader.
 

Constructors in org.apache.lucene.analysis.ru with parameters of type TokenStream
RussianStemFilter(TokenStream in, char[] charset)
           
RussianLowerCaseFilter(TokenStream in, char[] charset)
           
 

Uses of TokenStream in org.apache.lucene.analysis.standard
 

Subclasses of TokenStream in org.apache.lucene.analysis.standard
 class StandardFilter
          Normalizes tokens extracted with StandardTokenizer.
 class StandardTokenizer
          A grammar-based tokenizer constructed with JavaCC.
 

Methods in org.apache.lucene.analysis.standard that return TokenStream
 TokenStream StandardAnalyzer.tokenStream(String fieldName, Reader reader)
          Constructs a StandardTokenizer filtered by a StandardFilter, a LowerCaseFilter and a StopFilter.
 

Constructors in org.apache.lucene.analysis.standard with parameters of type TokenStream
StandardFilter(TokenStream in)
          Construct filtering in.
 

Uses of TokenStream in org.apache.lucene.search.highlight
 

Methods in org.apache.lucene.search.highlight that return TokenStream
static TokenStream TokenSources.getAnyTokenStream(IndexReader reader, int docId, String field, Analyzer analyzer)
          A convenience method that tries a number of approaches to getting a token stream.
static TokenStream TokenSources.getTokenStream(TermPositionVector tpv)
           
static TokenStream TokenSources.getTokenStream(TermPositionVector tpv, boolean tokenPositionsGuaranteedContiguous)
          Low level api.
static TokenStream TokenSources.getTokenStream(IndexReader reader, int docId, String field)
           
static TokenStream TokenSources.getTokenStream(IndexReader reader, int docId, String field, Analyzer analyzer)
           
 

Methods in org.apache.lucene.search.highlight with parameters of type TokenStream
 String Highlighter.getBestFragment(TokenStream tokenStream, String text)
          Highlights chosen terms in a text, extracting the most relevant section.
 String[] Highlighter.getBestFragments(TokenStream tokenStream, String text, int maxNumFragments)
          Highlights chosen terms in a text, extracting the most relevant sections.
 TextFragment[] Highlighter.getBestTextFragments(TokenStream tokenStream, String text, boolean mergeContiguousFragments, int maxNumFragments)
          Low level api to get the most relevant (formatted) sections of the document.
 String Highlighter.getBestFragments(TokenStream tokenStream, String text, int maxNumFragments, String separator)
          Highlights terms in the text , extracting the most relevant sections and concatenating the chosen fragments with a separator (typically "...").
 



Copyright © 2000-2005 Apache Software Foundation. All Rights Reserved.