Packages

class TextMatcherInternalModel extends AnnotatorModel[TextMatcherInternalModel] with HasSimpleAnnotate[TextMatcherInternalModel] with CheckLicense

Instantiated model of the TextMatcherInternal. For usage and examples see the documentation of the main class.

Linear Supertypes
CheckLicense, HasSimpleAnnotate[TextMatcherInternalModel], AnnotatorModel[TextMatcherInternalModel], CanBeLazy, RawAnnotator[TextMatcherInternalModel], HasOutputAnnotationCol, HasInputAnnotationCols, HasOutputAnnotatorType, ParamsAndFeaturesWritable, HasFeatures, DefaultParamsWritable, MLWritable, Model[TextMatcherInternalModel], Transformer, PipelineStage, Logging, Params, Serializable, Serializable, Identifiable, AnyRef, Any
Ordering
  1. Grouped
  2. Alphabetic
  3. By Inheritance
Inherited
  1. TextMatcherInternalModel
  2. CheckLicense
  3. HasSimpleAnnotate
  4. AnnotatorModel
  5. CanBeLazy
  6. RawAnnotator
  7. HasOutputAnnotationCol
  8. HasInputAnnotationCols
  9. HasOutputAnnotatorType
  10. ParamsAndFeaturesWritable
  11. HasFeatures
  12. DefaultParamsWritable
  13. MLWritable
  14. Model
  15. Transformer
  16. PipelineStage
  17. Logging
  18. Params
  19. Serializable
  20. Serializable
  21. Identifiable
  22. AnyRef
  23. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new TextMatcherInternalModel()

    internal constructor for writabale annotator

  2. new TextMatcherInternalModel(uid: String)

    uid

    internally renquired UID to make it writable

Type Members

  1. type AnnotationContent = Seq[Row]
    Attributes
    protected
    Definition Classes
    AnnotatorModel
  2. type AnnotatorType = String
    Definition Classes
    HasOutputAnnotatorType

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def $[T](param: Param[T]): T
    Attributes
    protected
    Definition Classes
    Params
  4. def $$[T](feature: StructFeature[T]): T
    Attributes
    protected
    Definition Classes
    HasFeatures
  5. def $$[K, V](feature: MapFeature[K, V]): Map[K, V]
    Attributes
    protected
    Definition Classes
    HasFeatures
  6. def $$[T](feature: SetFeature[T]): Set[T]
    Attributes
    protected
    Definition Classes
    HasFeatures
  7. def $$[T](feature: ArrayFeature[T]): Array[T]
    Attributes
    protected
    Definition Classes
    HasFeatures
  8. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  9. def _transform(dataset: Dataset[_], recursivePipeline: Option[PipelineModel]): DataFrame
    Attributes
    protected
    Definition Classes
    AnnotatorModel
  10. def afterAnnotate(dataset: DataFrame): DataFrame
    Attributes
    protected
    Definition Classes
    AnnotatorModel
  11. def annotate(annotations: Seq[Annotation]): Seq[Annotation]

    Searches entities and stores them in the annotation.

    Searches entities and stores them in the annotation. Defines annotator phrase matching depending on whether we are using SBD or not

    returns

    Extracted Entities

    Definition Classes
    TextMatcherInternalModel → HasSimpleAnnotate
  12. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  13. def beforeAnnotate(dataset: Dataset[_]): Dataset[_]
    Attributes
    protected
    Definition Classes
    AnnotatorModel
  14. val buildFromTokens: BooleanParam

    Whether the TextMatcherInternal should take the CHUNK from TOKEN or not

  15. val caseSensitive: BooleanParam

    Whether to match regardless of case (Default: true)

  16. final def checkSchema(schema: StructType, inputAnnotatorType: String): Boolean
    Attributes
    protected
    Definition Classes
    HasInputAnnotationCols
  17. def checkValidEnvironment(spark: Option[SparkSession], scopes: Seq[String]): Unit
    Definition Classes
    CheckLicense
  18. def checkValidScope(scope: String): Unit
    Definition Classes
    CheckLicense
  19. def checkValidScopeAndEnvironment(scope: String, spark: Option[SparkSession], checkLp: Boolean): Unit
    Definition Classes
    CheckLicense
  20. def checkValidScopesAndEnvironment(scopes: Seq[String], spark: Option[SparkSession], checkLp: Boolean): Unit
    Definition Classes
    CheckLicense
  21. final def clear(param: Param[_]): TextMatcherInternalModel.this.type
    Definition Classes
    Params
  22. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  23. final def collapse(rs: List[(Int, Int)], sep: List[(Int, Int)] = Nil): List[(Int, Int)]
    Attributes
    protected
    Annotations
    @tailrec()
  24. def copy(extra: ParamMap): TextMatcherInternalModel
    Definition Classes
    RawAnnotator → Model → Transformer → PipelineStage → Params
  25. def copyValues[T <: Params](to: T, extra: ParamMap): T
    Attributes
    protected
    Definition Classes
    Params
  26. final def defaultCopy[T <: Params](extra: ParamMap): T
    Attributes
    protected
    Definition Classes
    Params
  27. val delimiter: Param[String]

    Value for the delimiter between Phrase, Entity in the entities file (Default: ,)

  28. def dfAnnotate: UserDefinedFunction
    Definition Classes
    HasSimpleAnnotate
  29. val entityValue: Param[String]

    Value for the entity metadata field

  30. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  31. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  32. def explainParam(param: Param[_]): String
    Definition Classes
    Params
  33. def explainParams(): String
    Definition Classes
    Params
  34. def extraValidate(structType: StructType): Boolean
    Attributes
    protected
    Definition Classes
    RawAnnotator
  35. def extraValidateMsg: String
    Attributes
    protected
    Definition Classes
    RawAnnotator
  36. final def extractParamMap(): ParamMap
    Definition Classes
    Params
  37. final def extractParamMap(extra: ParamMap): ParamMap
    Definition Classes
    Params
  38. val features: ArrayBuffer[Feature[_, _, _]]
    Definition Classes
    HasFeatures
  39. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  40. def get[T](feature: StructFeature[T]): Option[T]
    Attributes
    protected
    Definition Classes
    HasFeatures
  41. def get[K, V](feature: MapFeature[K, V]): Option[Map[K, V]]
    Attributes
    protected
    Definition Classes
    HasFeatures
  42. def get[T](feature: SetFeature[T]): Option[Set[T]]
    Attributes
    protected
    Definition Classes
    HasFeatures
  43. def get[T](feature: ArrayFeature[T]): Option[Array[T]]
    Attributes
    protected
    Definition Classes
    HasFeatures
  44. final def get[T](param: Param[T]): Option[T]
    Definition Classes
    Params
  45. def getBuildFromTokens: Boolean

    Getter for buildFromTokens param

  46. def getCaseSensitive: Boolean

    Setter for case sensitivity (Default: true)

  47. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  48. final def getDefault[T](param: Param[T]): Option[T]
    Definition Classes
    Params
  49. def getDelimiter: String

    Get the Phrase, Entity delimiter in the entities file (Default: ,)

  50. def getEntityValue: String

    Getter for Value for the entity metadata field

  51. def getInputCols: Array[String]
    Definition Classes
    HasInputAnnotationCols
  52. def getLazyAnnotator: Boolean
    Definition Classes
    CanBeLazy
  53. def getLicenseScopes: Seq[String]
    Attributes
    protected
  54. def getMergeOverlapping: Boolean

    Whether to merge overlapping matched chunks.

    Whether to merge overlapping matched chunks. Defaults false

  55. final def getOrDefault[T](param: Param[T]): T
    Definition Classes
    Params
  56. final def getOutputCol: String
    Definition Classes
    HasOutputAnnotationCol
  57. def getParam(paramName: String): Param[Any]
    Definition Classes
    Params
  58. final def hasDefault[T](param: Param[T]): Boolean
    Definition Classes
    Params
  59. def hasParam(paramName: String): Boolean
    Definition Classes
    Params
  60. def hasParent: Boolean
    Definition Classes
    Model
  61. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  62. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  63. def initializeLogIfNecessary(isInterpreter: Boolean): Unit
    Attributes
    protected
    Definition Classes
    Logging
  64. val inputAnnotatorTypes: Array[AnnotatorType]

    input annotator type : DOCUMENT, TOKEN

    input annotator type : DOCUMENT, TOKEN

    Definition Classes
    TextMatcherInternalModel → HasInputAnnotationCols
  65. final val inputCols: StringArrayParam
    Attributes
    protected
    Definition Classes
    HasInputAnnotationCols
  66. final def isDefined(param: Param[_]): Boolean
    Definition Classes
    Params
  67. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  68. final def isSet(param: Param[_]): Boolean
    Definition Classes
    Params
  69. def isTraceEnabled(): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  70. val lazyAnnotator: BooleanParam
    Definition Classes
    CanBeLazy
  71. def log: Logger
    Attributes
    protected
    Definition Classes
    Logging
  72. def logDebug(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  73. def logDebug(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  74. def logError(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  75. def logError(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  76. def logInfo(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  77. def logInfo(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  78. def logName: String
    Attributes
    protected
    Definition Classes
    Logging
  79. def logTrace(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  80. def logTrace(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  81. def logWarning(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  82. def logWarning(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  83. def merge(rs: List[(Int, Int)]): List[(Int, Int)]
    Attributes
    protected
  84. val mergeOverlapping: BooleanParam

    whether to merge overlapping matched chunks.

    whether to merge overlapping matched chunks. Defaults false

  85. def msgHelper(schema: StructType): String
    Attributes
    protected
    Definition Classes
    HasInputAnnotationCols
  86. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  87. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  88. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  89. def onWrite(path: String, spark: SparkSession): Unit
    Attributes
    protected
    Definition Classes
    ParamsAndFeaturesWritable
  90. val optionalInputAnnotatorTypes: Array[String]
    Definition Classes
    HasInputAnnotationCols
  91. val outputAnnotatorType: AnnotatorType

    Output annotator type : CHUNK

    Output annotator type : CHUNK

    Definition Classes
    TextMatcherInternalModel → HasOutputAnnotatorType
  92. final val outputCol: Param[String]
    Attributes
    protected
    Definition Classes
    HasOutputAnnotationCol
  93. lazy val params: Array[Param[_]]
    Definition Classes
    Params
  94. var parent: Estimator[TextMatcherInternalModel]
    Definition Classes
    Model
  95. def save(path: String): Unit
    Definition Classes
    MLWritable
    Annotations
    @Since( "1.6.0" ) @throws( ... )
  96. val searchTries: MapFeature[String, SearchTrie]

    searchTries for Searching words

  97. def set[T](feature: StructFeature[T], value: T): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  98. def set[K, V](feature: MapFeature[K, V], value: Map[K, V]): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  99. def set[T](feature: SetFeature[T], value: Set[T]): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  100. def set[T](feature: ArrayFeature[T], value: Array[T]): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  101. final def set(paramPair: ParamPair[_]): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    Params
  102. final def set(param: String, value: Any): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    Params
  103. final def set[T](param: Param[T], value: T): TextMatcherInternalModel.this.type
    Definition Classes
    Params
  104. def setBuildFromTokens(v: Boolean): TextMatcherInternalModel.this.type

    Setter for buildFromTokens param

  105. def setCaseSensitive(v: Boolean): TextMatcherInternalModel.this.type

    Setter for case sensitivity (Default: true)

  106. def setDefault[T](feature: StructFeature[T], value: () ⇒ T): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  107. def setDefault[K, V](feature: MapFeature[K, V], value: () ⇒ Map[K, V]): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  108. def setDefault[T](feature: SetFeature[T], value: () ⇒ Set[T]): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  109. def setDefault[T](feature: ArrayFeature[T], value: () ⇒ Array[T]): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    HasFeatures
  110. final def setDefault(paramPairs: ParamPair[_]*): TextMatcherInternalModel.this.type
    Attributes
    protected
    Definition Classes
    Params
  111. final def setDefault[T](param: Param[T], value: T): TextMatcherInternalModel.this.type
    Attributes
    protected[org.apache.spark.ml]
    Definition Classes
    Params
  112. def setDelimiter(v: String): TextMatcherInternalModel.this.type

    Set the Phrase, Entity delimiter in the entities file (Default: ,)

  113. def setEntityValue(v: String): TextMatcherInternalModel.this.type

    Setter for Value for the entity metadata field

  114. final def setInputCols(value: String*): TextMatcherInternalModel.this.type
    Definition Classes
    HasInputAnnotationCols
  115. def setInputCols(value: Array[String]): TextMatcherInternalModel.this.type
    Definition Classes
    HasInputAnnotationCols
  116. def setLazyAnnotator(value: Boolean): TextMatcherInternalModel.this.type
    Definition Classes
    CanBeLazy
  117. def setMergeOverlapping(v: Boolean): TextMatcherInternalModel.this.type

    Whether to merge overlapping matched chunks.

    Whether to merge overlapping matched chunks. Defaults false

  118. final def setOutputCol(value: String): TextMatcherInternalModel.this.type
    Definition Classes
    HasOutputAnnotationCol
  119. def setParent(parent: Estimator[TextMatcherInternalModel]): TextMatcherInternalModel
    Definition Classes
    Model
  120. def setSearchTries(value: Map[String, SearchTrie]): TextMatcherInternalModel.this.type

    SearchTrie of Tokens

  121. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  122. def toString(): String
    Definition Classes
    Identifiable → AnyRef → Any
  123. final def transform(dataset: Dataset[_]): DataFrame
    Definition Classes
    AnnotatorModel → Transformer
  124. def transform(dataset: Dataset[_], paramMap: ParamMap): DataFrame
    Definition Classes
    Transformer
    Annotations
    @Since( "2.0.0" )
  125. def transform(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): DataFrame
    Definition Classes
    Transformer
    Annotations
    @Since( "2.0.0" ) @varargs()
  126. final def transformSchema(schema: StructType): StructType
    Definition Classes
    RawAnnotator → PipelineStage
  127. def transformSchema(schema: StructType, logging: Boolean): StructType
    Attributes
    protected
    Definition Classes
    PipelineStage
    Annotations
    @DeveloperApi()
  128. val uid: String
    Definition Classes
    TextMatcherInternalModel → Identifiable
  129. def validate(schema: StructType): Boolean
    Attributes
    protected
    Definition Classes
    RawAnnotator
  130. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  131. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  132. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  133. def wrapColumnMetadata(col: Column): Column
    Attributes
    protected
    Definition Classes
    RawAnnotator
  134. def write: MLWriter
    Definition Classes
    ParamsAndFeaturesWritable → DefaultParamsWritable → MLWritable

Inherited from CheckLicense

Inherited from HasSimpleAnnotate[TextMatcherInternalModel]

Inherited from AnnotatorModel[TextMatcherInternalModel]

Inherited from CanBeLazy

Inherited from RawAnnotator[TextMatcherInternalModel]

Inherited from HasOutputAnnotationCol

Inherited from HasInputAnnotationCols

Inherited from HasOutputAnnotatorType

Inherited from ParamsAndFeaturesWritable

Inherited from HasFeatures

Inherited from DefaultParamsWritable

Inherited from MLWritable

Inherited from Model[TextMatcherInternalModel]

Inherited from Transformer

Inherited from PipelineStage

Inherited from Logging

Inherited from Params

Inherited from Serializable

Inherited from Serializable

Inherited from Identifiable

Inherited from AnyRef

Inherited from Any

Parameters

A list of (hyper-)parameter keys this annotator can take. Users can set and get the parameter values through setters and getters, respectively.

Annotator types

Required input and expected output annotator types

Members

Parameter setters

Parameter getters