Class

com.johnsnowlabs.nlp.annotators.resolution

SentenceEntityResolverApproach

Related Doc: package resolution

Permalink

class SentenceEntityResolverApproach extends AnnotatorApproach[SentenceEntityResolverModel] with SentenceResolverParams with HasCaseSensitiveProperties with Licensed

Linear Supertypes
Licensed, HasCaseSensitiveProperties, ParamsAndFeaturesWritable, HasFeatures, SentenceResolverParams, AnnotatorApproach[SentenceEntityResolverModel], CanBeLazy, DefaultParamsWritable, MLWritable, HasOutputAnnotatorType, HasOutputAnnotationCol, HasInputAnnotationCols, Estimator[SentenceEntityResolverModel], PipelineStage, Logging, Params, Serializable, Serializable, Identifiable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SentenceEntityResolverApproach
  2. Licensed
  3. HasCaseSensitiveProperties
  4. ParamsAndFeaturesWritable
  5. HasFeatures
  6. SentenceResolverParams
  7. AnnotatorApproach
  8. CanBeLazy
  9. DefaultParamsWritable
  10. MLWritable
  11. HasOutputAnnotatorType
  12. HasOutputAnnotationCol
  13. HasInputAnnotationCols
  14. Estimator
  15. PipelineStage
  16. Logging
  17. Params
  18. Serializable
  19. Serializable
  20. Identifiable
  21. AnyRef
  22. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new SentenceEntityResolverApproach()

    Permalink
  2. new SentenceEntityResolverApproach(uid: String)

    Permalink

Type Members

  1. type AnnotatorType = String

    Permalink
    Definition Classes
    HasOutputAnnotatorType

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def $[T](param: Param[T]): T

    Permalink
    Attributes
    protected
    Definition Classes
    Params
  4. def $$[T](feature: StructFeature[T]): T

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  5. def $$[K, V](feature: MapFeature[K, V]): Map[K, V]

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  6. def $$[T](feature: SetFeature[T]): Set[T]

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  7. def $$[T](feature: ArrayFeature[T]): Array[T]

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  8. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  9. def _fit(dataset: Dataset[_], recursiveStages: Option[PipelineModel]): SentenceEntityResolverModel

    Permalink
    Attributes
    protected
    Definition Classes
    AnnotatorApproach
  10. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  11. val auxLabelCol: Param[String]

    Permalink
  12. val auxLabelMap: StructFeature[Map[String, String]]

    Permalink
  13. def beforeTraining(spark: SparkSession): Unit

    Permalink
    Definition Classes
    AnnotatorApproach
  14. val caseSensitive: BooleanParam

    Permalink
    Definition Classes
    HasCaseSensitiveProperties
  15. final def checkSchema(schema: StructType, inputAnnotatorType: String): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    HasInputAnnotationCols
  16. final def clear(param: Param[_]): SentenceEntityResolverApproach.this.type

    Permalink
    Definition Classes
    Params
  17. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  18. val confidenceFunction: Param[String]

    Permalink
    Definition Classes
    SentenceResolverParams
  19. final def copy(extra: ParamMap): Estimator[SentenceEntityResolverModel]

    Permalink
    Definition Classes
    AnnotatorApproach → Estimator → PipelineStage → Params
  20. def copyValues[T <: Params](to: T, extra: ParamMap): T

    Permalink
    Attributes
    protected
    Definition Classes
    Params
  21. final def defaultCopy[T <: Params](extra: ParamMap): T

    Permalink
    Attributes
    protected
    Definition Classes
    Params
  22. val description: String

    Permalink
    Definition Classes
    SentenceEntityResolverApproach → AnnotatorApproach
  23. val distanceFunction: Param[String]

    Permalink

    what distance function to use for KNN: 'EUCLIDEAN' or 'COSINE'

    what distance function to use for KNN: 'EUCLIDEAN' or 'COSINE'

    Definition Classes
    SentenceResolverParams
  24. lazy val embeddingsColumnName: String

    Permalink
  25. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  26. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  27. def explainParam(param: Param[_]): String

    Permalink
    Definition Classes
    Params
  28. def explainParams(): String

    Permalink
    Definition Classes
    Params
  29. def extractAuxLabelMap(dataset: Dataset[_]): Map[String, String]

    Permalink

    Extracts a Map[String,String] where key=label and value=auxLabel from a dataset.

    Extracts a Map[String,String] where key=label and value=auxLabel from a dataset. If either of one columns does not exist, it will return an empty map

    dataset

    from whichw e extract the column

    returns

    a Map[String,String]

  30. final def extractParamMap(): ParamMap

    Permalink
    Definition Classes
    Params
  31. final def extractParamMap(extra: ParamMap): ParamMap

    Permalink
    Definition Classes
    Params
  32. val features: ArrayBuffer[Feature[_, _, _]]

    Permalink
    Definition Classes
    HasFeatures
  33. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  34. final def fit(dataset: Dataset[_]): SentenceEntityResolverModel

    Permalink
    Definition Classes
    AnnotatorApproach → Estimator
  35. def fit(dataset: Dataset[_], paramMaps: Array[ParamMap]): Seq[SentenceEntityResolverModel]

    Permalink
    Definition Classes
    Estimator
    Annotations
    @Since( "2.0.0" )
  36. def fit(dataset: Dataset[_], paramMap: ParamMap): SentenceEntityResolverModel

    Permalink
    Definition Classes
    Estimator
    Annotations
    @Since( "2.0.0" )
  37. def fit(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): SentenceEntityResolverModel

    Permalink
    Definition Classes
    Estimator
    Annotations
    @Since( "2.0.0" ) @varargs()
  38. def get[T](feature: StructFeature[T]): Option[T]

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  39. def get[K, V](feature: MapFeature[K, V]): Option[Map[K, V]]

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  40. def get[T](feature: SetFeature[T]): Option[Set[T]]

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  41. def get[T](feature: ArrayFeature[T]): Option[Array[T]]

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  42. final def get[T](param: Param[T]): Option[T]

    Permalink
    Definition Classes
    Params
  43. def getAuxLabelCol(): Option[String]

    Permalink
  44. def getAuxLabelMap(): Map[String, String]

    Permalink
  45. def getCaseSensitive: Boolean

    Permalink
    Definition Classes
    HasCaseSensitiveProperties
  46. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  47. def getConfidenceFunction: String

    Permalink
    Definition Classes
    SentenceResolverParams
  48. final def getDefault[T](param: Param[T]): Option[T]

    Permalink
    Definition Classes
    Params
  49. def getDistanceFunction: String

    Permalink
    Definition Classes
    SentenceResolverParams
  50. def getInputCols: Array[String]

    Permalink
    Definition Classes
    HasInputAnnotationCols
  51. def getLabelCol: String

    Permalink
  52. def getLazyAnnotator: Boolean

    Permalink
    Definition Classes
    CanBeLazy
  53. def getMissAsEmpty: Boolean

    Permalink
    Definition Classes
    SentenceResolverParams
  54. def getNeighbours: Int

    Permalink
    Definition Classes
    SentenceResolverParams
  55. def getNormalizedCol: String

    Permalink
  56. final def getOrDefault[T](param: Param[T]): T

    Permalink
    Definition Classes
    Params
  57. final def getOutputCol: String

    Permalink
    Definition Classes
    HasOutputAnnotationCol
  58. def getParam(paramName: String): Param[Any]

    Permalink
    Definition Classes
    Params
  59. def getReturnAllKEmbeddings(): Boolean

    Permalink
  60. def getReturnCosineDistances: Boolean

    Permalink

    Get whether cosine distances should be calculated between a chunk and the k_candidates result embeddings

  61. def getThreshold: Double

    Permalink
    Definition Classes
    SentenceResolverParams
  62. def getUseAuxLabel(): Boolean

    Permalink
  63. final def hasDefault[T](param: Param[T]): Boolean

    Permalink
    Definition Classes
    Params
  64. def hasParam(paramName: String): Boolean

    Permalink
    Definition Classes
    Params
  65. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  66. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  67. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  68. val inputAnnotatorTypes: Array[String]

    Permalink
    Definition Classes
    SentenceEntityResolverApproach → HasInputAnnotationCols
  69. final val inputCols: StringArrayParam

    Permalink
    Attributes
    protected
    Definition Classes
    HasInputAnnotationCols
  70. final def isDefined(param: Param[_]): Boolean

    Permalink
    Definition Classes
    Params
  71. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  72. final def isSet(param: Param[_]): Boolean

    Permalink
    Definition Classes
    Params
  73. def isTraceEnabled(): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  74. val labelCol: Param[String]

    Permalink

    column name for the value we are trying to resolve

  75. lazy val labelColumnName: String

    Permalink
  76. val lazyAnnotator: BooleanParam

    Permalink
    Definition Classes
    CanBeLazy
  77. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  78. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  79. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  80. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  81. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  82. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  83. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  84. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  85. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  86. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  87. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  88. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  89. val missAsEmpty: BooleanParam

    Permalink

    whether or not to return an empty annotation on unmatched chunks

    whether or not to return an empty annotation on unmatched chunks

    Definition Classes
    SentenceResolverParams
  90. def msgHelper(schema: StructType): String

    Permalink
    Attributes
    protected
    Definition Classes
    HasInputAnnotationCols
  91. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  92. val neighbours: IntParam

    Permalink

    number of neighbours to consider in the KNN query to calculate WMD

    number of neighbours to consider in the KNN query to calculate WMD

    Definition Classes
    SentenceResolverParams
  93. val normalizedCol: Param[String]

    Permalink

    column name for the original, normalized description

  94. lazy val normalizedColumnName: String

    Permalink
  95. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  96. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  97. def onTrained(model: SentenceEntityResolverModel, spark: SparkSession): Unit

    Permalink
    Definition Classes
    AnnotatorApproach
  98. def onWrite(path: String, spark: SparkSession): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    ParamsAndFeaturesWritable
  99. val outputAnnotatorType: AnnotatorType

    Permalink
    Definition Classes
    SentenceEntityResolverApproach → HasOutputAnnotatorType
  100. final val outputCol: Param[String]

    Permalink
    Attributes
    protected
    Definition Classes
    HasOutputAnnotationCol
  101. lazy val params: Array[Param[_]]

    Permalink
    Definition Classes
    Params
  102. val returnAllKEmbeddings: BooleanParam

    Permalink
  103. val returnCosineDistances: BooleanParam

    Permalink
  104. def save(path: String): Unit

    Permalink
    Definition Classes
    MLWritable
    Annotations
    @Since( "1.6.0" ) @throws( ... )
  105. def set[T](feature: StructFeature[T], value: T): SentenceEntityResolverApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  106. def set[K, V](feature: MapFeature[K, V], value: Map[K, V]): SentenceEntityResolverApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  107. def set[T](feature: SetFeature[T], value: Set[T]): SentenceEntityResolverApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  108. def set[T](feature: ArrayFeature[T], value: Array[T]): SentenceEntityResolverApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  109. final def set(paramPair: ParamPair[_]): SentenceEntityResolverApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params
  110. final def set(param: String, value: Any): SentenceEntityResolverApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params
  111. final def set[T](param: Param[T], value: T): SentenceEntityResolverApproach.this.type

    Permalink
    Definition Classes
    Params
  112. def setAuxLabelCol(c: String): SentenceEntityResolverApproach.this.type

    Permalink

    Optional column with one extra label per document.

    Optional column with one extra label per document. This extra label will be outputted later on in an additional column

  113. def setAuxLabelMap(m: Map[String, String]): SentenceEntityResolverApproach.this.type

    Permalink

    Optional column with one extra label per document.

    Optional column with one extra label per document. This extra label will be outputted later on in an additional column

  114. def setCaseSensitive(value: Boolean): SentenceEntityResolverApproach.this.type

    Permalink
    Definition Classes
    HasCaseSensitiveProperties
  115. def setConfidenceFunction(v: String): SentenceEntityResolverApproach.this.type

    Permalink
    Definition Classes
    SentenceResolverParams
  116. def setDefault[T](feature: StructFeature[T], value: () ⇒ T): SentenceEntityResolverApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  117. def setDefault[K, V](feature: MapFeature[K, V], value: () ⇒ Map[K, V]): SentenceEntityResolverApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  118. def setDefault[T](feature: SetFeature[T], value: () ⇒ Set[T]): SentenceEntityResolverApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  119. def setDefault[T](feature: ArrayFeature[T], value: () ⇒ Array[T]): SentenceEntityResolverApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    HasFeatures
  120. final def setDefault(paramPairs: ParamPair[_]*): SentenceEntityResolverApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params
  121. final def setDefault[T](param: Param[T], value: T): SentenceEntityResolverApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params
  122. def setDistanceFunction(value: String): SentenceEntityResolverApproach.this.type

    Permalink
    Definition Classes
    SentenceResolverParams
  123. final def setInputCols(value: String*): SentenceEntityResolverApproach.this.type

    Permalink
    Definition Classes
    HasInputAnnotationCols
  124. final def setInputCols(value: Array[String]): SentenceEntityResolverApproach.this.type

    Permalink
    Definition Classes
    HasInputAnnotationCols
  125. def setLabelCol(value: String): SentenceEntityResolverApproach.this.type

    Permalink
  126. def setLazyAnnotator(value: Boolean): SentenceEntityResolverApproach.this.type

    Permalink
    Definition Classes
    CanBeLazy
  127. def setMissAsEmpty(v: Boolean): SentenceEntityResolverApproach.this.type

    Permalink
    Definition Classes
    SentenceResolverParams
  128. def setNeighbours(k: Int): SentenceEntityResolverApproach.this.type

    Permalink
    Definition Classes
    SentenceResolverParams
  129. def setNormalizedCol(value: String): SentenceEntityResolverApproach.this.type

    Permalink
  130. final def setOutputCol(value: String): SentenceEntityResolverApproach.this.type

    Permalink
    Definition Classes
    HasOutputAnnotationCol
  131. def setReturnAllKEmbeddings(b: Boolean): SentenceEntityResolverApproach.this.type

    Permalink
  132. def setReturnCosineDistances(value: Boolean): SentenceEntityResolverApproach.this.type

    Permalink

    Set whether cosine distances should be calculated between a chunk and the k_candidates result embeddings

  133. def setThreshold(dist: Double): SentenceEntityResolverApproach.this.type

    Permalink
    Definition Classes
    SentenceResolverParams
  134. def setUseAuxLabel(b: Boolean): SentenceEntityResolverApproach.this.type

    Permalink
  135. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  136. val threshold: DoubleParam

    Permalink

    threshold value for the aggregated distance

    threshold value for the aggregated distance

    Definition Classes
    SentenceResolverParams
  137. def toString(): String

    Permalink
    Definition Classes
    Identifiable → AnyRef → Any
  138. def train(dataset: Dataset[_], recursivePipeline: Option[PipelineModel]): SentenceEntityResolverModel

    Permalink

    Returns the ChunkEntityResolverModel Transformer, that can be used to transform input datasets

    Returns the ChunkEntityResolverModel Transformer, that can be used to transform input datasets

    The dataset provided to the fit method should have one chunk per row and contain the following columns: ChunkTokens, ChunkEmbeddings, ResolverLabel, [ResolverNormalized]

    The cardinality of the dataset should not exceed 100.000 data points since searching in such a big KD-tree becomes impractical

    This method is called inside the AnnotatorApproach's fit method

    dataset

    a Dataset containing ChunkTokens, ChunkEmbeddings, ClassifierLabel, ResolverLabel, [ResolverNormalized]

    returns

    a trained ChunkEntityResolverModel

    Definition Classes
    SentenceEntityResolverApproach → AnnotatorApproach
  139. final def transformSchema(schema: StructType): StructType

    Permalink
    Definition Classes
    AnnotatorApproach → PipelineStage
  140. def transformSchema(schema: StructType, logging: Boolean): StructType

    Permalink
    Attributes
    protected
    Definition Classes
    PipelineStage
    Annotations
    @DeveloperApi()
  141. val uid: String

    Permalink
    Definition Classes
    SentenceEntityResolverApproach → Identifiable
  142. val useAuxLabel: BooleanParam

    Permalink
  143. def validate(schema: StructType): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    AnnotatorApproach
  144. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  145. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  146. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  147. def write: MLWriter

    Permalink
    Definition Classes
    ParamsAndFeaturesWritable → DefaultParamsWritable → MLWritable

Inherited from Licensed

Inherited from HasCaseSensitiveProperties

Inherited from ParamsAndFeaturesWritable

Inherited from HasFeatures

Inherited from SentenceResolverParams

Inherited from AnnotatorApproach[SentenceEntityResolverModel]

Inherited from CanBeLazy

Inherited from DefaultParamsWritable

Inherited from MLWritable

Inherited from HasOutputAnnotatorType

Inherited from HasOutputAnnotationCol

Inherited from HasInputAnnotationCols

Inherited from Estimator[SentenceEntityResolverModel]

Inherited from PipelineStage

Inherited from Logging

Inherited from Params

Inherited from Serializable

Inherited from Serializable

Inherited from Identifiable

Inherited from AnyRef

Inherited from Any

Ungrouped