Class/Object

com.johnsnowlabs.nlp.annotators.assertion.logreg

AssertionLogRegApproach

Related Docs: object AssertionLogRegApproach | package logreg

Permalink

class AssertionLogRegApproach extends AnnotatorApproach[AssertionLogRegModel] with Windowing with Licensed

This is a classification method, which uses LogisticRegression algorithm Contains all the methods for training the LogisticRegression Approach, together with trainWithChunk, trainWithStartEnd

Linear Supertypes
Licensed, Windowing, AnnotatorApproach[AssertionLogRegModel], CanBeLazy, DefaultParamsWritable, MLWritable, HasOutputAnnotatorType, HasOutputAnnotationCol, HasInputAnnotationCols, Estimator[AssertionLogRegModel], PipelineStage, Logging, Params, Serializable, Serializable, Identifiable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. AssertionLogRegApproach
  2. Licensed
  3. Windowing
  4. AnnotatorApproach
  5. CanBeLazy
  6. DefaultParamsWritable
  7. MLWritable
  8. HasOutputAnnotatorType
  9. HasOutputAnnotationCol
  10. HasInputAnnotationCols
  11. Estimator
  12. PipelineStage
  13. Logging
  14. Params
  15. Serializable
  16. Serializable
  17. Identifiable
  18. AnyRef
  19. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new AssertionLogRegApproach()

    Permalink
  2. new AssertionLogRegApproach(uid: String)

    Permalink

    uid

    a unique identifier for the instanced AssertionDLApproach

Type Members

  1. type AnnotatorType = String

    Permalink
    Definition Classes
    HasOutputAnnotatorType
  2. case class VectorizedChunk(vector: Vector, begin: Int, end: Int, sentenceId: Int, chunkId: Int) extends Product with Serializable

    Permalink
    Attributes
    protected
    Definition Classes
    Windowing

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def $[T](param: Param[T]): T

    Permalink
    Attributes
    protected
    Definition Classes
    Params
  4. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  5. def _fit(dataset: Dataset[_], recursiveStages: Option[PipelineModel]): AssertionLogRegModel

    Permalink
    Attributes
    protected
    Definition Classes
    AnnotatorApproach
  6. lazy val after: Int

    Permalink
    Definition Classes
    AssertionLogRegApproachWindowing
  7. val afterParam: IntParam

    Permalink

    Amount of tokens from the context after the target

  8. def applyWindow(tokenizedSentence: WordpieceEmbeddingsSentence, s: Int, e: Int, embeddingsDim: Int): Array[Double]

    Permalink
    Definition Classes
    Windowing
  9. def applyWindowContext(tokenizedSentence: WordpieceEmbeddingsSentence, s: Int, e: Int, embeddingsDim: Int): (Array[Array[Float]], Array[Array[Float]], Array[Array[Float]])

    Permalink
    Definition Classes
    Windowing
  10. def applyWindowUdf(embeddingsDim: Int): UserDefinedFunction

    Permalink
    Definition Classes
    Windowing
  11. def applyWindowUdfChunk(embeddingsDim: Int): UserDefinedFunction

    Permalink
    Definition Classes
    Windowing
  12. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  13. lazy val before: Int

    Permalink
    Definition Classes
    AssertionLogRegApproachWindowing
  14. val beforeParam: IntParam

    Permalink

    Amount of tokens from the context before the target

  15. def beforeTraining(spark: SparkSession): Unit

    Permalink
    Definition Classes
    AnnotatorApproach
  16. final def checkSchema(schema: StructType, inputAnnotatorType: String): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    HasInputAnnotationCols
  17. final def clear(param: Param[_]): AssertionLogRegApproach.this.type

    Permalink
    Definition Classes
    Params
  18. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  19. final def copy(extra: ParamMap): Estimator[AssertionLogRegModel]

    Permalink
    Definition Classes
    AnnotatorApproach → Estimator → PipelineStage → Params
  20. def copyValues[T <: Params](to: T, extra: ParamMap): T

    Permalink
    Attributes
    protected
    Definition Classes
    Params
  21. final def defaultCopy[T <: Params](extra: ParamMap): T

    Permalink
    Attributes
    protected
    Definition Classes
    Params
  22. val description: String

    Permalink
    Definition Classes
    AssertionLogRegApproach → AnnotatorApproach
  23. val eNetParam: DoubleParam

    Permalink

    Elastic net parameter

  24. val endCol: Param[String]

    Permalink

    Column that contains the token number for the end of the target

  25. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  26. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  27. def explainParam(param: Param[_]): String

    Permalink
    Definition Classes
    Params
  28. def explainParams(): String

    Permalink
    Definition Classes
    Params
  29. final def extractParamMap(): ParamMap

    Permalink
    Definition Classes
    Params
  30. final def extractParamMap(extra: ParamMap): ParamMap

    Permalink
    Definition Classes
    Params
  31. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  32. final def fit(dataset: Dataset[_]): AssertionLogRegModel

    Permalink
    Definition Classes
    AnnotatorApproach → Estimator
  33. def fit(dataset: Dataset[_], paramMaps: Array[ParamMap]): Seq[AssertionLogRegModel]

    Permalink
    Definition Classes
    Estimator
    Annotations
    @Since( "2.0.0" )
  34. def fit(dataset: Dataset[_], paramMap: ParamMap): AssertionLogRegModel

    Permalink
    Definition Classes
    Estimator
    Annotations
    @Since( "2.0.0" )
  35. def fit(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): AssertionLogRegModel

    Permalink
    Definition Classes
    Estimator
    Annotations
    @Since( "2.0.0" ) @varargs()
  36. final def get[T](param: Param[T]): Option[T]

    Permalink
    Definition Classes
    Params
  37. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  38. final def getDefault[T](param: Param[T]): Option[T]

    Permalink
    Definition Classes
    Params
  39. def getInputCols: Array[String]

    Permalink
    Definition Classes
    HasInputAnnotationCols
  40. def getLazyAnnotator: Boolean

    Permalink
    Definition Classes
    CanBeLazy
  41. final def getOrDefault[T](param: Param[T]): T

    Permalink
    Definition Classes
    Params
  42. final def getOutputCol: String

    Permalink
    Definition Classes
    HasOutputAnnotationCol
  43. def getParam(paramName: String): Param[Any]

    Permalink
    Definition Classes
    Params
  44. final def hasDefault[T](param: Param[T]): Boolean

    Permalink
    Definition Classes
    Params
  45. def hasParam(paramName: String): Boolean

    Permalink
    Definition Classes
    Params
  46. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  47. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  48. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  49. val inputAnnotatorTypes: Array[String]

    Permalink
    Definition Classes
    AssertionLogRegApproach → HasInputAnnotationCols
  50. final val inputCols: StringArrayParam

    Permalink
    Attributes
    protected
    Definition Classes
    HasInputAnnotationCols
  51. final def isDefined(param: Param[_]): Boolean

    Permalink
    Definition Classes
    Params
  52. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  53. final def isSet(param: Param[_]): Boolean

    Permalink
    Definition Classes
    Params
  54. def isTraceEnabled(): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  55. def l2norm(xs: Array[Double]): Double

    Permalink
    Definition Classes
    Windowing
  56. val label: Param[String]

    Permalink

    Column with one label per document

  57. val lazyAnnotator: BooleanParam

    Permalink
    Definition Classes
    CanBeLazy
  58. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  59. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  60. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  61. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  62. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  63. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  64. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  65. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  66. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  67. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  68. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  69. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  70. val maxIter: IntParam

    Permalink

    Max number of iterations for algorithm

  71. def msgHelper(schema: StructType): String

    Permalink
    Attributes
    protected
    Definition Classes
    HasInputAnnotationCols
  72. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  73. def normalize(vec: Array[Double]): Array[Double]

    Permalink
    Definition Classes
    Windowing
  74. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  75. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  76. def onTrained(model: AssertionLogRegModel, spark: SparkSession): Unit

    Permalink
    Definition Classes
    AnnotatorApproach
  77. val outputAnnotatorType: AnnotatorType

    Permalink
    Definition Classes
    AssertionLogRegApproach → HasOutputAnnotatorType
  78. final val outputCol: Param[String]

    Permalink
    Attributes
    protected
    Definition Classes
    HasOutputAnnotationCol
  79. lazy val params: Array[Param[_]]

    Permalink
    Definition Classes
    Params
  80. val regParam: DoubleParam

    Permalink

    Regularization parameter

  81. def save(path: String): Unit

    Permalink
    Definition Classes
    MLWritable
    Annotations
    @Since( "1.6.0" ) @throws( ... )
  82. final def set(paramPair: ParamPair[_]): AssertionLogRegApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params
  83. final def set(param: String, value: Any): AssertionLogRegApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params
  84. final def set[T](param: Param[T], value: T): AssertionLogRegApproach.this.type

    Permalink
    Definition Classes
    Params
  85. def setAfter(a: Int): AssertionLogRegApproach.this.type

    Permalink
  86. def setBefore(b: Int): AssertionLogRegApproach.this.type

    Permalink
  87. final def setDefault(paramPairs: ParamPair[_]*): AssertionLogRegApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params
  88. final def setDefault[T](param: Param[T], value: T): AssertionLogRegApproach.this.type

    Permalink
    Attributes
    protected
    Definition Classes
    Params
  89. def setEndCol(end: String): AssertionLogRegApproach.this.type

    Permalink
  90. def setEnet(enet: Double): AssertionLogRegApproach.this.type

    Permalink
  91. final def setInputCols(value: String*): AssertionLogRegApproach.this.type

    Permalink
    Definition Classes
    HasInputAnnotationCols
  92. final def setInputCols(value: Array[String]): AssertionLogRegApproach.this.type

    Permalink
    Definition Classes
    HasInputAnnotationCols
  93. def setLabelCol(label: String): AssertionLogRegApproach.this.type

    Permalink
  94. def setLazyAnnotator(value: Boolean): AssertionLogRegApproach.this.type

    Permalink
    Definition Classes
    CanBeLazy
  95. def setMaxIter(max: Int): AssertionLogRegApproach.this.type

    Permalink
  96. final def setOutputCol(value: String): AssertionLogRegApproach.this.type

    Permalink
    Definition Classes
    HasOutputAnnotationCol
  97. def setReg(lambda: Double): AssertionLogRegApproach.this.type

    Permalink
  98. def setStartCol(start: String): AssertionLogRegApproach.this.type

    Permalink
  99. val startCol: Param[String]

    Permalink

    Column that contains the token number for the start of the target

  100. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  101. def toString(): String

    Permalink
    Definition Classes
    Identifiable → AnyRef → Any
  102. def tokenIndexToChunkIndex(doc: Array[TokenPieceEmbeddings], start: Int, end: Int): (Int, Int)

    Permalink
    Definition Classes
    Windowing
  103. def train(dataset: Dataset[_], recursivePipeline: Option[PipelineModel] = None): AssertionLogRegModel

    Permalink

    This is a main point of interest of this class.

    This is a main point of interest of this class. It trains the dataset with recursive pipeline and uses methods trainWithChunk() and trainwithStartEnd() The choice of training happens based on the startCol value of the DL Approach

    dataset

    a collection of inputs to train

    recursivePipeline

    an instance of PipelineModel

    returns

    an instance of trained AssertionLogRegModel

    Definition Classes
    AssertionLogRegApproach → AnnotatorApproach
  104. final def transformSchema(schema: StructType): StructType

    Permalink
    Definition Classes
    AnnotatorApproach → PipelineStage
  105. def transformSchema(schema: StructType, logging: Boolean): StructType

    Permalink
    Attributes
    protected
    Definition Classes
    PipelineStage
    Annotations
    @DeveloperApi()
  106. val uid: String

    Permalink

    a unique identifier for the instanced AssertionDLApproach

    a unique identifier for the instanced AssertionDLApproach

    Definition Classes
    AssertionLogRegApproach → Identifiable
  107. def validate(schema: StructType): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    AnnotatorApproach
  108. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  109. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  110. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  111. def write: MLWriter

    Permalink
    Definition Classes
    DefaultParamsWritable → MLWritable

Inherited from Licensed

Inherited from Windowing

Inherited from AnnotatorApproach[AssertionLogRegModel]

Inherited from CanBeLazy

Inherited from DefaultParamsWritable

Inherited from MLWritable

Inherited from HasOutputAnnotatorType

Inherited from HasOutputAnnotationCol

Inherited from HasInputAnnotationCols

Inherited from Estimator[AssertionLogRegModel]

Inherited from PipelineStage

Inherited from Logging

Inherited from Params

Inherited from Serializable

Inherited from Serializable

Inherited from Identifiable

Inherited from AnyRef

Inherited from Any

Ungrouped