sparknlp_jsl.annotator.qa.beam_search_params#
Module Contents#
Classes#
- class BeamSearchParams#
Bases:
object- doSample#
- ignoreTokenIds#
- maxContextLength#
- maxNewTokens#
- noRepeatNgramSize#
- topK#
- setDoSample(value)#
Sets whether or not to use sampling, use greedy decoding otherwise.
- Parameters:
value (bool) – Whether or not to use sampling; use greedy decoding otherwise
- setIgnoreTokenIds(value)#
A list of token ids which are ignored in the decoder’s output.
- Parameters:
value (List[int]) – The words to be filtered out
- setMaxContextLength(value)#
Sets maximum length of output text.
- Parameters:
value (int) – Maximum length of output text
- setMaxNewTokens(value)#
Sets the maximum number of new tokens to be generated
- Parameters:
value (int) – the maximum number of new tokens to be generated
- setNoRepeatNgramSize(value)#
Sets size of n-grams that can only occur once.
If set to int > 0, all ngrams of that size can only occur once.
- Parameters:
value (int) – N-gram size can only occur once
- setRandomSeed(seed)#
Sets random seed.
- Parameters:
seed (int) – Random seed
- setTopK(value)#
Sets the number of highest probability vocabulary tokens to consider
- Parameters:
value (int) – Number of highest probability vocabulary tokens to consider