Packages

t

com.johnsnowlabs.nlp.annotators.matcher

TextMatcherInternalParams

trait TextMatcherInternalParams extends Params with HasFeatures

Trait containing parameters and helper methods for the TextMatcherInternal and TextMatcherInternalModel components.

Linear Supertypes
HasFeatures, Params, Serializable, Serializable, Identifiable, AnyRef, Any
Known Subclasses
TextMatcherInternal, TextMatcherInternalModel
Ordering
  1. Grouped
  2. Alphabetic
  3. By Inheritance
Inherited
  1. TextMatcherInternalParams
  2. HasFeatures
  3. Params
  4. Serializable
  5. Serializable
  6. Identifiable
  7. AnyRef
  8. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def copy(extra: ParamMap): Params
    Definition Classes
    Params
  2. abstract val uid: String
    Definition Classes
    Identifiable

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def $[T](param: Param[T]): T
    Attributes
    protected
    Definition Classes
    Params
  4. def $$[T](feature: StructFeature[T]): T
    Attributes
    protected
    Definition Classes
    HasFeatures
  5. def $$[K, V](feature: MapFeature[K, V]): Map[K, V]
    Attributes
    protected
    Definition Classes
    HasFeatures
  6. def $$[T](feature: SetFeature[T]): Set[T]
    Attributes
    protected
    Definition Classes
    HasFeatures
  7. def $$[T](feature: ArrayFeature[T]): Array[T]
    Attributes
    protected
    Definition Classes
    HasFeatures
  8. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  9. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  10. def cartesianTokenVariants(tokens: Seq[Annotation], lemmaDictionary: Map[String, String]): Seq[Seq[String]]
    Attributes
    protected
  11. val caseSensitive: BooleanParam

    Whether to match regardless of case (Default: true)

  12. val cleanKeywords: StringArrayParam

    A parameter defining additional keywords to be removed during text processing, in addition to the standard stopwords.

    A parameter defining additional keywords to be removed during text processing, in addition to the standard stopwords.

    These keywords are appended to the default stopwords list and will be excluded from the text when cleanStopWords is enabled.

    By default, this parameter is an empty array, meaning no additional keywords are filtered unless specified.

  13. val cleanStopWords: BooleanParam

    Parameter indicating whether to clean stop words during text processing.

    Parameter indicating whether to clean stop words during text processing. Defaults to true.

  14. final def clear(param: Param[_]): TextMatcherInternalParams.this.type
    Definition Classes
    Params
  15. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  16. def copyValues[T <: Params](to: T, extra: ParamMap): T
    Attributes
    protected
    Definition Classes
    Params
  17. final def defaultCopy[T <: Params](extra: ParamMap): T
    Attributes
    protected
    Definition Classes
    Params
  18. val enableLemmatizer: BooleanParam

    A Boolean parameter that controls whether lemmatization should be applied during text processing.

    A Boolean parameter that controls whether lemmatization should be applied during text processing.

    Lemmatization is the process of reducing words to their base or dictionary form (lemma). When this parameter is set to true: - The incoming tokens (words from the input text) are lemmatized. - The predefined entities (the terms you want to match against) are also lemmatized.

    This allows for more flexible and accurate matching. For example, words like "running", "ran", or "runs" will all be reduced to "run", and can match consistently even if the exact form in the text differs.

    Default value is false, meaning lemmatization is disabled unless explicitly turned on.

  19. val enableStemmer: BooleanParam

    A Boolean parameter that controls whether stemming should be applied during text processing.

    A Boolean parameter that controls whether stemming should be applied during text processing.

    Stemming reduces words to their root forms (e.g., "running", "runs", and "runner" → "run"). This can help match different word forms more effectively in tasks such as keyword matching and entity recognition.

    When this parameter is set to true, stemming is applied in addition to the original form: - Input tokens are matched both in their original and stemmed forms. - Target entities can also be matched using their stemmed forms.

    This does not replace original matching — it complements it. Matching is performed using both the original and processed (stemmed) versions to improve recall and flexibility.

    Default value is false.

  20. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  21. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  22. val excludePunctuation: BooleanParam

    A parameter indicating whether punctuation marks should be removed during text processing.

    A parameter indicating whether punctuation marks should be removed during text processing.

    When set to true, most punctuation characters will be excluded from the processed text. This is typically used to clean text by removing non-word characters.

    Defaults to true, meaning punctuation is removed unless explicitly disabled. Some characters may be preserved if specifically handled by other parameters (e.g., safe keywords).

  23. val excludeRegexPatterns: StringArrayParam

    A parameter specifying regular expression patterns used to exclude matching chunks during text processing.

    A parameter specifying regular expression patterns used to exclude matching chunks during text processing.

    Each string in this array is a regex pattern. If a detected chunk matches any of these patterns, it will be discarded and excluded from the final output.

    This is useful for removing unwanted matches based on pattern rules (e.g., specific codes, formats, or noise). By default, this parameter is empty, meaning no chunks are dropped based on regex.

  24. def explainParam(param: Param[_]): String
    Definition Classes
    Params
  25. def explainParams(): String
    Definition Classes
    Params
  26. final def extractParamMap(): ParamMap
    Definition Classes
    Params
  27. final def extractParamMap(extra: ParamMap): ParamMap
    Definition Classes
    Params
  28. val features: ArrayBuffer[Feature[_, _, _]]
    Definition Classes
    HasFeatures
  29. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  30. def get[T](feature: StructFeature[T]): Option[T]
    Attributes
    protected
    Definition Classes
    HasFeatures
  31. def get[K, V](feature: MapFeature[K, V]): Option[Map[K, V]]
    Attributes
    protected
    Definition Classes
    HasFeatures
  32. def get[T](feature: SetFeature[T]): Option[Set[T]]
    Attributes
    protected
    Definition Classes
    HasFeatures
  33. def get[T](feature: ArrayFeature[T]): Option[Array[T]]
    Attributes
    protected
    Definition Classes
    HasFeatures
  34. final def get[T](param: Param[T]): Option[T]
    Definition Classes
    Params
  35. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  36. def getCleanKeywords: Array[String]

    Retrieves the list of keywords to be filtered out.

    Retrieves the list of keywords to be filtered out.

    returns

    an array of strings representing the keywords.

  37. def getCleanStopWords: Boolean

    Retrieves the current state of the cleanStopWords parameter.

    Retrieves the current state of the cleanStopWords parameter.

    returns

    true if the cleanStopWords option is enabled, false otherwise.

  38. final def getDefault[T](param: Param[T]): Option[T]
    Definition Classes
    Params
  39. def getEnableLemmatizer: Boolean

    Gets the current state of the lemmatizer enablement setting.

    Gets the current state of the lemmatizer enablement setting.

    returns

    true if the lemmatizer is enabled, false otherwise.

  40. def getEnableStemmer: Boolean

    Retrieves the current value of the enableStemmer parameter.

    Retrieves the current value of the enableStemmer parameter.

    returns

    true if stemming is enabled, false otherwise

  41. def getExcludeRegexPattern: Array[String]

    Retrieves the list of regex patterns used to exclude specific text matches during processing.

    Retrieves the list of regex patterns used to exclude specific text matches during processing.

    returns

    an array of strings representing the regex patterns to be excluded.

  42. final def getOrDefault[T](param: Param[T]): T
    Definition Classes
    Params
  43. def getParam(paramName: String): Param[Any]
    Definition Classes
    Params
  44. def getReturnChunks: String

    Retrieves the current value of the returnChunks parameter.

    Retrieves the current value of the returnChunks parameter.

    returns

    A string representing the configured value for the returnChunks setting.

  45. def getSafeKeywords: Array[String]

    Retrieves the list of keywords to be filtered out.

    Retrieves the list of keywords to be filtered out.

    returns

    an array of strings representing the keywords.

  46. def getSkipMatcherAugmentation: Boolean

    Gets whether augmentation for matcher patterns is skipped.

    Gets whether augmentation for matcher patterns is skipped.

    returns

    true if augmentation for matcher patterns is skipped, false otherwise.

  47. def getSkipSourceTextAugmentation: Boolean

    Gets whether augmentation for source text is skipped.

    Gets whether augmentation for source text is skipped.

    returns

    true if augmentation for source text is skipped, false otherwise.

  48. def getStopWords: Array[String]

    Retrieves the list of stop words used within the text matching process.

    Retrieves the list of stop words used within the text matching process.

    returns

    an array of strings representing the stop words.

  49. def getTokenVariants(token: Annotation, lemmaDictionary: Map[String, String]): Seq[String]
    Attributes
    protected
  50. final def hasDefault[T](param: Param[T]): Boolean
    Definition Classes
    Params
  51. def hasParam(paramName: String): Boolean
    Definition Classes
    Params
  52. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  53. final def isDefined(param: Param[_]): Boolean
    Definition Classes
    Params
  54. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  55. final def isSet(param: Param[_]): Boolean
    Definition Classes
    Params
  56. val lemmaDict: MapFeature[String, String]

    lemmaDict

  57. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  58. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  59. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  60. lazy val params: Array[Param[_]]
    Definition Classes
    Params
  61. val returnChunks: Param[String]

    A string parameter that defines which version of the matched chunks should be returned: "original" or "matched".

    A string parameter that defines which version of the matched chunks should be returned: "original" or "matched".

    - If set to "original" (default): the returned chunks reflect the exact text spans as they appeared in the original input. This ensures that the begin and end character indices accurately map to the source text.

    - If set to "matched": the returned chunks are based on the processed form that triggered the match, such as a stemmed or lemmatized version of the phrase. This can be useful to see which normalized entity was matched, but the character indices (begin, end) may not align correctly with the original input text.

    Use "original" if accurate text positioning is important (e.g., for highlighting), and "matched" if you want to inspect the normalized form used for the match.

  62. val safeKeywords: StringArrayParam

    A parameter representing an array of keywords that should be preserved during text cleaning, when stopword removal (cleanStopWords) is enabled.

    A parameter representing an array of keywords that should be preserved during text cleaning, when stopword removal (cleanStopWords) is enabled.

    When cleanStopWords is set to true, common stopwords are typically removed from the text. However, keywords specified in safeKeywords will be exempt from removal and retained in the processed text.

    By default, this parameter is an empty array, meaning no exceptions are made unless explicitly provided.

  63. lazy val safeLemmaDict: Map[String, String]
  64. def set[T](feature: StructFeature[T], value: T): TextMatcherInternalParams.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  65. def set[K, V](feature: MapFeature[K, V], value: Map[K, V]): TextMatcherInternalParams.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  66. def set[T](feature: SetFeature[T], value: Set[T]): TextMatcherInternalParams.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  67. def set[T](feature: ArrayFeature[T], value: Array[T]): TextMatcherInternalParams.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  68. final def set(paramPair: ParamPair[_]): TextMatcherInternalParams.this.type
    Attributes
    protected
    Definition Classes
    Params
  69. final def set(param: String, value: Any): TextMatcherInternalParams.this.type
    Attributes
    protected
    Definition Classes
    Params
  70. final def set[T](param: Param[T], value: T): TextMatcherInternalParams.this.type
    Definition Classes
    Params
  71. def setCleanKeywords(value: ArrayList[String]): TextMatcherInternalParams.this.type
  72. def setCleanKeywords(values: Array[String]): TextMatcherInternalParams.this.type

    Sets the list of keywords to be cleaned during text processing.

    Sets the list of keywords to be cleaned during text processing.

    returns

    This instance with the updated configuration for cleaning keywords.

  73. def setCleanStopWords(v: Boolean): TextMatcherInternalParams.this.type

    Sets whether to clean stop words during text processing.

    Sets whether to clean stop words during text processing.

    v

    Boolean value indicating whether to enable (true) or disable (false) the cleaning of stop words.

    returns

    This instance with the updated configuration for cleaning stop words.

  74. def setDefault[T](feature: StructFeature[T], value: () ⇒ T): TextMatcherInternalParams.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  75. def setDefault[K, V](feature: MapFeature[K, V], value: () ⇒ Map[K, V]): TextMatcherInternalParams.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  76. def setDefault[T](feature: SetFeature[T], value: () ⇒ Set[T]): TextMatcherInternalParams.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  77. def setDefault[T](feature: ArrayFeature[T], value: () ⇒ Array[T]): TextMatcherInternalParams.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  78. final def setDefault(paramPairs: ParamPair[_]*): TextMatcherInternalParams.this.type
    Attributes
    protected
    Definition Classes
    Params
  79. final def setDefault[T](param: Param[T], value: T): TextMatcherInternalParams.this.type
    Attributes
    protected[org.apache.spark.ml]
    Definition Classes
    Params
  80. def setEnableLemmatizer(value: Boolean): TextMatcherInternalParams.this.type

    Enables or disables the lemmatizer for text matching.

    Enables or disables the lemmatizer for text matching.

    value

    If true, the lemmatizer will be enabled; if false, it will be disabled.

    returns

    This TextMatcherInternal instance with the updated lemmatizer setting.

  81. def setEnableStemmer(value: Boolean): TextMatcherInternalParams.this.type

    Enables or disables the use of a stemmer for text processing.

    Enables or disables the use of a stemmer for text processing.

    value

    Boolean value indicating whether to enable (true) or disable (false) the stemmer.

    returns

    Instance of this class with updated configuration.

  82. def setExcludePunctuation(v: Boolean): TextMatcherInternalParams.this.type

    Sets the value of the excludePunctuation parameter used for text processing.

    Sets the value of the excludePunctuation parameter used for text processing.

    v

    A boolean value indicating whether to exclude punctuation.

    returns

    This instance with the updated excludePunctuation configuration.

  83. def setExcludeRegexPatterns(v: Array[String]): TextMatcherInternalParams.this.type

    Sets the regular expression patterns for excluding specific elements during text processing.

    Sets the regular expression patterns for excluding specific elements during text processing.

    v

    Array of strings where each string represents a regular expression pattern to be used for excluding matching text elements.

    returns

    This instance with the updated configuration for exclude regex patterns.

  84. def setLemmaDict(value: Map[String, String]): TextMatcherInternalParams.this.type

    Sets the internal dictionary used for lemmatization.

    Sets the internal dictionary used for lemmatization.

    value

    a map where keys are words and values are their corresponding lemmas.

    returns

    this

  85. def setReturnChunks(v: String): TextMatcherInternalParams.this.type

    Sets the value of the returnChunks parameter used for text processing.

    Sets the value of the returnChunks parameter used for text processing.

    v

    A string value that specifies the configuration for returning chunks.

    returns

    This instance with the updated returnChunks configuration.

  86. def setSafeKeywords(value: ArrayList[String]): TextMatcherInternalParams.this.type
  87. def setSafeKeywords(v: Array[String]): TextMatcherInternalParams.this.type

    Sets the list of safe keywords to be used in text processing.

    Sets the list of safe keywords to be used in text processing.

    v

    Array of strings representing the safe keywords.

    returns

    This instance with the updated configuration for safe keywords.

  88. def setSkipMatcherAugmentation(value: Boolean): TextMatcherInternalParams.this.type

    Sets whether to skip augmentation for matcher patterns.

    Sets whether to skip augmentation for matcher patterns.

    value

    If true, matcher patterns won't be augmented with lemmatization, stemming, etc. If false, matcher patterns will be augmented if the corresponding features are enabled.

    returns

    This instance with the updated configuration.

  89. def setSkipSourceTextAugmentation(value: Boolean): TextMatcherInternalParams.this.type

    Sets whether to skip augmentation for source text.

    Sets whether to skip augmentation for source text.

    value

    If true, source text won't be augmented with lemmatization, stemming, etc. If false, source text will be augmented if the corresponding features are enabled.

    returns

    This instance with the updated configuration.

  90. def setStopWords(value: ArrayList[String]): TextMatcherInternalParams.this.type
  91. def setStopWords(v: Array[String]): TextMatcherInternalParams.this.type

    Sets the list of stop words to be used in text processing.

    Sets the list of stop words to be used in text processing.

    v

    Array of strings representing the stop words.

    returns

    This instance with the updated stop words setting.

  92. val skipMatcherAugmentation: BooleanParam

    A Boolean parameter that controls whether to skip augmentation (lemmatization, stemming, etc.) for matcher patterns.

    A Boolean parameter that controls whether to skip augmentation (lemmatization, stemming, etc.) for matcher patterns.

    When set to true, the matcher patterns won't be augmented with lemmatization, stemming, stopword removal, etc., even if those features are enabled. This applies only to entities/patterns being matched, not the source text.

    Default value is false, meaning matcher patterns will be augmented if the corresponding features are enabled.

  93. val skipSourceTextAugmentation: BooleanParam

    A Boolean parameter that controls whether to skip augmentation (lemmatization, stemming, etc.) for the source text.

    A Boolean parameter that controls whether to skip augmentation (lemmatization, stemming, etc.) for the source text.

    When set to true, the source text won't be augmented with lemmatization, stemming, stopword removal, etc., even if those features are enabled. This applies only to the source text being analyzed, not the matcher patterns.

    Default value is false, meaning source text will be augmented if the corresponding features are enabled.

  94. val stopWords: StringArrayParam

    A parameter representing the list of stop words to be filtered out during text processing.

    A parameter representing the list of stop words to be filtered out during text processing.

    By default, it is set to the English stop words provided by Spark ML.

  95. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  96. def toString(): String
    Definition Classes
    Identifiable → AnyRef → Any
  97. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  98. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  99. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from HasFeatures

Inherited from Params

Inherited from Serializable

Inherited from Serializable

Inherited from Identifiable

Inherited from AnyRef

Inherited from Any

Parameters

A list of (hyper-)parameter keys this annotator can take. Users can set and get the parameter values through setters and getters, respectively.

Parameter setters

Parameter getters

Members