Spark NLP 5.5.0 ScalaDoc - com.johnsnowlabs.finance.token

final def !=(arg0: Any): Boolean

Definition Classes: AnyRef → Any

final def ##(): Int

Definition Classes: AnyRef → Any

final def $[T](param: Param[T]): T

Attributes: protected
Definition Classes: Params

def $$[T](feature: StructFeature[T]): T

Attributes: protected
Definition Classes: HasFeatures

def $$[K, V](feature: MapFeature[K, V]): Map[K, V]

Attributes: protected
Definition Classes: HasFeatures

def $$[T](feature: SetFeature[T]): Set[T]

Attributes: protected
Definition Classes: HasFeatures

def $$[T](feature: ArrayFeature[T]): Array[T]

Attributes: protected
Definition Classes: HasFeatures

final def ==(arg0: Any): Boolean

Definition Classes: AnyRef → Any

def _transform(dataset: Dataset[_], recursivePipeline: Option[PipelineModel]): DataFrame

Attributes: protected
Definition Classes: AnnotatorModel

def afterAnnotate(dataset: DataFrame): DataFrame

Attributes: protected
Definition Classes: AnnotatorModel

final def asInstanceOf[T0]: T0

Definition Classes: Any

def batchAnnotate(batchedAnnotations: Seq[Array[Annotation]]): Seq[Seq[Annotation]]

Definition Classes: MedicalNerModel → HasBatchedAnnotate

def batchProcess(rows: Iterator[_]): Iterator[Row]

Definition Classes: HasBatchedAnnotate

val batchSize: IntParam

Definition Classes: HasBatchedAnnotate

def beforeAnnotate(dataset: Dataset[_]): Dataset[_]

Attributes: protected
Definition Classes: MedicalNerModel → AnnotatorModel

final def checkSchema(schema: StructType, inputAnnotatorType: String): Boolean

Attributes: protected
Definition Classes: HasInputAnnotationCols

def checkValidEnvironment(spark: Option[SparkSession], scopes: Seq[String]): Unit

Definition Classes: CheckLicense

def checkValidScope(scope: String): Unit

Definition Classes: CheckLicense

def checkValidScopeAndEnvironment(scope: String, spark: Option[SparkSession], checkLp: Boolean): Unit

Definition Classes: CheckLicense

def checkValidScopesAndEnvironment(scopes: Seq[String], spark: Option[SparkSession], checkLp: Boolean): Unit

Definition Classes: CheckLicense

val classes: StringArrayParam

Definition Classes: MedicalNerModel

final def clear(param: Param[_]): FinanceNerModel.this.type

Definition Classes: Params

def clone(): AnyRef

Attributes: protected[lang]
Definition Classes: AnyRef
Annotations: @throws( ... ) @native()

val configProtoBytes: IntArrayParam

ConfigProto from tensorflow, serialized into byte array.

ConfigProto from tensorflow, serialized into byte array. Get with config_proto.SerializeToString()

Definition Classes: MedicalNerParams

def copy(extra: ParamMap): MedicalNerModel

Definition Classes: RawAnnotator → Model → Transformer → PipelineStage → Params

def copyValues[T <: Params](to: T, extra: ParamMap): T

Attributes: protected
Definition Classes: Params

def createDatabaseConnection(database: Name): RocksDBConnection

Definition Classes: HasStorageRef

val datasetInfo: Param[String]

Descriptive information about the dataset being used.

Definition Classes: MedicalNerParams

val datasetParams: StructFeature[DatasetEncoderParams]

datasetParams

Definition Classes: MedicalNerModel

final def defaultCopy[T <: Params](extra: ParamMap): T

Attributes: protected
Definition Classes: Params

val doExceptionHandling: BooleanParam

If true, effective batchsize is 1 and exceptions are handled.

If true, effective batchsize is 1 and exceptions are handled. If exception causing data is passed to the model, a error annotation is emitted which has the exception message. Processing continues with the next batch. This comes with a performance penalty.

Definition Classes: MedicalNerModel

val dropout: FloatParam

Dropout coefficient, by default 0.5.

The coefficient of the dropout layer. The value should be between 0.0 and 1.0. Internally, it is used by Tensorflow as: rate = 1.0 - dropout when adding a dropout layer on top of the recurrent layers.

Definition Classes: MedicalNerParams

val earlyStoppingCriterion: FloatParam

If set, this param specifies the criterion to stop training if performance is not improving.

Default value is 0 which is means that early stopping is not used.

The criterion is set to F1-score if the validationSplit is greater than 0.0 (F1-socre on validation set) or testDataset is defined (F1-score on test set), otherwise it is set to model loss. The priority is as follows: - If testDataset is defined, then the criterion is set to F1-score on test set. - If validationSplit is greater than 0.0, then the criterion is set to F1-score on validation set. - Otherwise, the criterion is set to model loss.

Note that while the F1-score ranges from 0.0 to 1.0, the loss ranges from 0.0 to infinity. So, depending on which case you are in, the value you use for the criterion can be very different. For example, if validationSplit is 0.1, then a criterion of 0.01 means that if the F1-score on the validation set difference from last epoch is greater than 0.01, then the training should stop. However, if there is not validation or test set defined, then a criterion of 2.0 means that if the loss difference between the last epoch and the current one is less than 2.0, then training should stop.

Definition Classes: MedicalNerParams
See also: earlyStoppingPatience.

val earlyStoppingPatience: IntParam

Number of epochs to wait before early stopping if no improvement, by default 5.

Given the earlyStoppingCriterion, if the performance does not improve for the given number of epochs, then the training will stop. If the value is 0, then early stopping will occurs as soon as the criterion is met (no patience).

Definition Classes: MedicalNerParams
See also: earlyStoppingCriterion.

val enableMemoryOptimizer: BooleanParam

Whether to optimize for large datasets or not.

Whether to optimize for large datasets or not. Enabling this option can slow down training.

In practice, if set to true the training will iterate over the spark Data Frame and retrieve the batches from the Data Frame iterator. This can be slower than the default option as it has to collect the batches on evey bach for every epoch, but it can be useful if the dataset is too large to fit in memory.

It controls if we want the features collected and generated at once and then feed into the network batch by batch (False) or collected and generated by batch and then feed into the network in batches (True) .

If the training data can fit to memory, then it is recommended to set this option to False (default value).

Definition Classes: MedicalNerParams

final def eq(arg0: AnyRef): Boolean

Definition Classes: AnyRef

def equals(arg0: Any): Boolean

Definition Classes: AnyRef → Any

def explainParam(param: Param[_]): String

Definition Classes: Params

def explainParams(): String

Definition Classes: Params

def extraValidate(structType: StructType): Boolean

Attributes: protected
Definition Classes: RawAnnotator

def extraValidateMsg: String

Attributes: protected
Definition Classes: RawAnnotator

final def extractParamMap(): ParamMap

Definition Classes: Params

final def extractParamMap(extra: ParamMap): ParamMap

Definition Classes: Params

val features: ArrayBuffer[Feature[_, _, _]]

Definition Classes: HasFeatures

def finalize(): Unit

Attributes: protected[lang]
Definition Classes: AnyRef
Annotations: @throws( classOf[java.lang.Throwable] )

def get[T](feature: StructFeature[T]): Option[T]

Attributes: protected
Definition Classes: HasFeatures

def get[K, V](feature: MapFeature[K, V]): Option[Map[K, V]]

Attributes: protected
Definition Classes: HasFeatures

def get[T](feature: SetFeature[T]): Option[Set[T]]

Attributes: protected
Definition Classes: HasFeatures

def get[T](feature: ArrayFeature[T]): Option[Array[T]]

Attributes: protected
Definition Classes: HasFeatures

final def get[T](param: Param[T]): Option[T]

Definition Classes: Params

def getBatchSize: Int

Definition Classes: HasBatchedAnnotate

final def getClass(): Class[_]

Definition Classes: AnyRef → Any
Annotations: @native()

def getClasses: Array[String]

get the tags used to trained this MedicalNerModel

Definition Classes: MedicalNerModel

def getConfigProtoBytes: Option[Array[Byte]]

ConfigProto from tensorflow, serialized into byte array.

ConfigProto from tensorflow, serialized into byte array. Get with config_proto.SerializeToString()

Definition Classes: MedicalNerParams

def getConfigProtoBytesAsInt: Option[Array[Int]]

Definition Classes: MedicalNerModel

def getDatasetInfo: String

get descriptive information about the dataset being used

Definition Classes: MedicalNerParams

def getDatasetParams: DatasetEncoderParams

Definition Classes: MedicalNerModel

final def getDefault[T](param: Param[T]): Option[T]

Definition Classes: Params

def getDropout: Float

Dropout coefficient

Definition Classes: MedicalNerParams

def getEarlyStoppingCriterion: Float

Early stopping criterion

Definition Classes: MedicalNerParams

def getEarlyStoppingPatience: Int

Early stopping patience

Definition Classes: MedicalNerParams

def getEnableMemoryOptimizer: Boolean

Whether to optimize for large datasets or not.

Whether to optimize for large datasets or not. Enabling this option can slow down training.

Definition Classes: MedicalNerParams

def getIncludeAllConfidenceScores: Boolean

whether to include all confidence scores in annotation metadata or just the score of the predicted tag

Definition Classes: MedicalNerParams

def getIncludeConfidence: Boolean

whether to include confidence scores in annotation metadata

Definition Classes: MedicalNerParams

def getInferenceBatchSize: Int

get the number of sentences to process in a single batch during inference

Definition Classes: MedicalNerModel

def getInputCols: Array[String]

Definition Classes: HasInputAnnotationCols

def getLazyAnnotator: Boolean

Definition Classes: CanBeLazy

def getLicenseScopes: Seq[String]

Attributes: protected
Definition Classes: FinanceNerModel → MedicalNerModel

def getLr: Float

Learning Rate

Definition Classes: MedicalNerParams

def getMinProba: Float

Minimum probability.

Minimum probability. Used only if there is no CRF on top of LSTM layer.

Definition Classes: MedicalNerModel

def getModelIfNotSet: TensorflowMedicalNer

ConfigProto from tensorflow, serialized into byte array.

ConfigProto from tensorflow, serialized into byte array. Get with config_proto.SerializeToString()

Definition Classes: MedicalNerModel

final def getOrDefault[T](param: Param[T]): T

Definition Classes: Params

final def getOutputCol: String

Definition Classes: HasOutputAnnotationCol

def getOverrideExistingTags: Boolean

Whether to override already learned tags when using a pretrained model to initialize the new model.

Definition Classes: MedicalNerParams

def getParam(paramName: String): Param[Any]

Definition Classes: Params

def getPo: Float

Learning rate decay coefficient.

Learning rate decay coefficient. Real Learning Rage = lr / (1 + po * epoch)

Definition Classes: MedicalNerParams

def getRandomValidationSplitPerEpoch: Boolean

Checks if a random validation split is done after each epoch or at the beginning of training only.

Definition Classes: MedicalNerParams

def getSentenceTokenIndex: Boolean

whether to include the token index for each sentence in annotation metadata.

Definition Classes: MedicalNerParams

def getStorageRef: String

Definition Classes: HasStorageRef

def getTrainingClassDistribution: Map[String, Long]

Definition Classes: MedicalNerModel

def getTrainingClassDistributionJava: Map[String, Long]

Definition Classes: MedicalNerModel

def getUseBestModel: Boolean

useBestModel

Definition Classes: MedicalNerParams

def getUseContrib: Boolean

Whether to use contrib LSTM Cells.

Whether to use contrib LSTM Cells. Not compatible with Windows. Might slightly improve accuracy.

Definition Classes: MedicalNerParams

val graphFile: Param[String]

Path that contains the external graph file.

When specified, the provided file will be used, and no graph search will happen. The path can be a local file path, a distributed file path (HDFS, DBFS), or a cloud storage (S3).

Definition Classes: MedicalNerParams

val graphFolder: Param[String]

Folder path that contains external graph files.

The path can be a local file path, a distributed file path (HDFS, DBFS), or a cloud storage (S3).

When instantiating the Tensorflow model, uses this folder to search for the adequate Tensorflow graph. The search is done using the name of the .pb file, which should be in this format: blstn_{ntags}_{embedding_dim}_{lstm_size}_{nchars}.pb.

Then, the search follows these rules: - Embedding dimension should be exactly the same as the one used to train the model. - Number of unique tags should be greater than or equal to the number of unique tags in the training data. - Number of unique chars should be greater than or equal to the number of unique chars in the training data.

The returned file will be the first one that satisfies all the conditions.

If the name of the file is ill-formed, errors will occur during training.

Definition Classes: MedicalNerParams

final def hasDefault[T](param: Param[T]): Boolean

Definition Classes: Params

def hasParam(paramName: String): Boolean

Definition Classes: Params

def hasParent: Boolean

Definition Classes: Model

def hashCode(): Int

Definition Classes: AnyRef → Any
Annotations: @native()

val includeAllConfidenceScores: BooleanParam

Whether to include confidence scores for all tags in annotation metadata or just the score of the predicted tag, by default False.

Needs the includeConfidence parameter to be set to true.

Enabling this may slow down the inference speed.

Definition Classes: MedicalNerParams

val includeConfidence: BooleanParam

Whether to include confidence scores in annotation metadata, by default False.

Setting this parameter to True will add the confidence score to the metadata of the NAMED_ENTITY annotation. In addition, if includeAllConfidenceScores is set to true, then the confidence scores of all the tags will be added to the metadata, otherwise only for the predicted tag (the one with maximum score).

Definition Classes: MedicalNerParams

val inferenceBatchSize: IntParam

Number of sentences to process in a single batch during inference

Definition Classes: MedicalNerModel

def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

Attributes: protected
Definition Classes: Logging

def initializeLogIfNecessary(isInterpreter: Boolean): Unit

Attributes: protected
Definition Classes: Logging

val inputAnnotatorTypes: Array[String]

Required input Annotators coulumns, expects DOCUMENT, TOKEN, WORD_EMBEDDINGS

Definition Classes: MedicalNerModel → HasInputAnnotationCols

final val inputCols: StringArrayParam

Attributes: protected
Definition Classes: HasInputAnnotationCols

final def isDefined(param: Param[_]): Boolean

Definition Classes: Params

final def isInstanceOf[T0]: Boolean

Definition Classes: Any

final def isSet(param: Param[_]): Boolean

Definition Classes: Params

def isTraceEnabled(): Boolean

Attributes: protected
Definition Classes: Logging

val labelCasing: Param[String]

Set the tag to case sensitive or not.Setting all labels of the NER models upper/lower case.

Set the tag to case sensitive or not.Setting all labels of the NER models upper/lower case. values upper|lower

Definition Classes: MedicalNerModel

val lazyAnnotator: BooleanParam

Definition Classes: CanBeLazy

def log: Logger

Attributes: protected
Definition Classes: Logging

def logDebug(msg: ⇒ String, throwable: Throwable): Unit

Attributes: protected
Definition Classes: Logging

def logDebug(msg: ⇒ String): Unit

Attributes: protected
Definition Classes: Logging

def logError(msg: ⇒ String, throwable: Throwable): Unit

Attributes: protected
Definition Classes: Logging

def logError(msg: ⇒ String): Unit

Attributes: protected
Definition Classes: Logging

def logInfo(msg: ⇒ String, throwable: Throwable): Unit

Attributes: protected
Definition Classes: Logging

def logInfo(msg: ⇒ String): Unit

Attributes: protected
Definition Classes: Logging

def logName: String

Attributes: protected
Definition Classes: Logging

val logPrefix: Param[String]

A prefix that will be appended to every log, default value is empty.

Definition Classes: MedicalNerParams

def logTrace(msg: ⇒ String, throwable: Throwable): Unit

Attributes: protected
Definition Classes: Logging

def logTrace(msg: ⇒ String): Unit

Attributes: protected
Definition Classes: Logging

def logWarning(msg: ⇒ String, throwable: Throwable): Unit

Attributes: protected
Definition Classes: Logging

def logWarning(msg: ⇒ String): Unit

Attributes: protected
Definition Classes: Logging

val lr: FloatParam

Learning Rate, by default 0.001.

Definition Classes: MedicalNerParams

val minProba: FloatParam

Minimum probability.

Minimum probability. Used only if there is no CRF on top of LSTM layer.

Definition Classes: MedicalNerModel

def msgHelper(schema: StructType): String

Attributes: protected
Definition Classes: HasInputAnnotationCols

final def ne(arg0: AnyRef): Boolean

Definition Classes: AnyRef

final def notify(): Unit

Definition Classes: AnyRef
Annotations: @native()

final def notifyAll(): Unit

Definition Classes: AnyRef
Annotations: @native()

def onWrite(path: String, spark: SparkSession): Unit

Definition Classes: MedicalNerModel → ParamsAndFeaturesWritable

val optionalInputAnnotatorTypes: Array[String]

Definition Classes: HasInputAnnotationCols

val outputAnnotatorType: String

Output Annnotator type : NAMED_ENTITY

Definition Classes: MedicalNerModel → HasOutputAnnotatorType

final val outputCol: Param[String]

Attributes: protected
Definition Classes: HasOutputAnnotationCol

val overrideExistingTags: BooleanParam

Controls whether to override already learned tags when using a pretrained model to initialize the new model.

Controls whether to override already learned tags when using a pretrained model to initialize the new model. A value of true will override existing tags.

Definition Classes: MedicalNerParams

lazy val params: Array[Param[_]]

Definition Classes: Params

var parent: Estimator[MedicalNerModel]

Definition Classes: Model

val po: FloatParam

Learning rate decay coefficient (time-based).

This is used to calculate the decayed learning rate at each step as: lr = lr / (1 + po * epoch), meaning that the value of the learning rate is updated on each epoch. By default 0.005.

Definition Classes: MedicalNerParams

val pretrainedModelPath: Param[String]

Path to an already trained MedicalNerModel.

This pretrained model will be used as a starting point for training the new one. The path can be a local file path, a distributed file path (HDFS, DBFS), or a cloud storage (S3).

Definition Classes: MedicalNerParams

val randomValidationSplitPerEpoch: BooleanParam

Do a random validation split after each epoch rather than at the beginning of training only.

Definition Classes: MedicalNerParams

def save(path: String): Unit

Definition Classes: MLWritable
Annotations: @Since( "1.6.0" ) @throws( ... )

val sentenceTokenIndex: BooleanParam

whether to include the token index for each sentence in annotation metadata, by default false.

whether to include the token index for each sentence in annotation metadata, by default false. If the value is true, the process might be slowed down.

Definition Classes: MedicalNerParams

def set[T](feature: StructFeature[T], value: T): FinanceNerModel.this.type

Attributes: protected
Definition Classes: HasFeatures

def set[K, V](feature: MapFeature[K, V], value: Map[K, V]): FinanceNerModel.this.type

Attributes: protected
Definition Classes: HasFeatures

def set[T](feature: SetFeature[T], value: Set[T]): FinanceNerModel.this.type

Attributes: protected
Definition Classes: HasFeatures

def set[T](feature: ArrayFeature[T], value: Array[T]): FinanceNerModel.this.type

Attributes: protected
Definition Classes: HasFeatures

final def set(paramPair: ParamPair[_]): FinanceNerModel.this.type

Attributes: protected
Definition Classes: Params

final def set(param: String, value: Any): FinanceNerModel.this.type

Attributes: protected
Definition Classes: Params

final def set[T](param: Param[T], value: T): FinanceNerModel.this.type

Definition Classes: Params

def setBatchSize(size: Int): FinanceNerModel.this.type

Definition Classes: HasBatchedAnnotate

def setConfigProtoBytes(bytes: Array[Int]): FinanceNerModel.this.type

ConfigProto from tensorflow, serialized into byte array.

ConfigProto from tensorflow, serialized into byte array. Get with config_proto.SerializeToString()

Definition Classes: MedicalNerParams

def setDatasetInfo(value: String): FinanceNerModel.this.type

set descriptive information about the dataset being used

Definition Classes: MedicalNerParams

def setDatasetParams(params: DatasetEncoderParams): FinanceNerModel.this.type

datasetParams

Definition Classes: MedicalNerModel

def setDefault[T](feature: StructFeature[T], value: () ⇒ T): FinanceNerModel.this.type

Attributes: protected
Definition Classes: HasFeatures

def setDefault[K, V](feature: MapFeature[K, V], value: () ⇒ Map[K, V]): FinanceNerModel.this.type

Attributes: protected
Definition Classes: HasFeatures

def setDefault[T](feature: SetFeature[T], value: () ⇒ Set[T]): FinanceNerModel.this.type

Attributes: protected
Definition Classes: HasFeatures

def setDefault[T](feature: ArrayFeature[T], value: () ⇒ Array[T]): FinanceNerModel.this.type

Attributes: protected
Definition Classes: HasFeatures

final def setDefault(paramPairs: ParamPair[_]*): FinanceNerModel.this.type

Attributes: protected
Definition Classes: Params

final def setDefault[T](param: Param[T], value: T): FinanceNerModel.this.type

Attributes: protected[org.apache.spark.ml]
Definition Classes: Params

def setDoExceptionHandling(value: Boolean): FinanceNerModel.this.type

If true, effective batchsize is 1 and exceptions are handled.

If true, effective batchsize is 1 and exceptions are handled. If exception causing data is passed to the model, a error annotation is emitted which has the exception message. Processing continues with the next batch. This comes with a performance penalty.

Definition Classes: MedicalNerModel

def setDropout(dropout: Float): FinanceNerModel.this.type

Dropout coefficient

Definition Classes: MedicalNerParams

def setEarlyStoppingCriterion(value: Float): FinanceNerModel.this.type

Definition Classes: MedicalNerParams

def setEarlyStoppingPatience(value: Int): FinanceNerModel.this.type

Definition Classes: MedicalNerParams

def setEnableMemoryOptimizer(value: Boolean): FinanceNerModel.this.type

Definition Classes: MedicalNerParams

def setGraphFile(path: String): FinanceNerModel.this.type

Folder path that contain external graph files

Definition Classes: MedicalNerParams

def setGraphFolder(path: String): FinanceNerModel.this.type

Folder path that contain external graph files

Definition Classes: MedicalNerParams

def setIncludeAllConfidenceScores(value: Boolean): FinanceNerModel.this.type

Whether to include confidence scores in annotation metadata

Definition Classes: MedicalNerParams

def setIncludeConfidence(value: Boolean): FinanceNerModel.this.type

Whether to include confidence scores for all tags rather than just for the predicted one

Definition Classes: MedicalNerParams

def setInferenceBatchSize(value: Int): FinanceNerModel.this.type

set the number of sentences to process in a single batch during inference

Definition Classes: MedicalNerModel

final def setInputCols(value: String*): FinanceNerModel.this.type

Definition Classes: HasInputAnnotationCols

def setInputCols(value: Array[String]): FinanceNerModel.this.type

Definition Classes: HasInputAnnotationCols

def setLabelCasing(value: String): FinanceNerModel.this.type

Definition Classes: MedicalNerModel

def setLazyAnnotator(value: Boolean): FinanceNerModel.this.type

Definition Classes: CanBeLazy

def setLogPrefix(value: String): FinanceNerModel.this.type

a string prefix to be included in the logs

Definition Classes: MedicalNerParams

def setLr(lr: Float): FinanceNerModel.this.type

Learning Rate

Definition Classes: MedicalNerParams

def setMinProbability(minProba: Float): FinanceNerModel.this.type

Minimum probability.

Minimum probability. Used only if there is no CRF on top of LSTM layer.

Definition Classes: MedicalNerModel

def setModelIfNotSet(spark: SparkSession, tf: TensorflowWrapper): FinanceNerModel.this.type

Definition Classes: MedicalNerModel

final def setOutputCol(value: String): FinanceNerModel.this.type

Definition Classes: HasOutputAnnotationCol

def setOverrideExistingTags(value: Boolean): FinanceNerModel.this.type

Controls whether to override already learned tags when using a pretrained model to initialize the new model.

Controls whether to override already learned tags when using a pretrained model to initialize the new model. A value of true will override existing tags.

Definition Classes: MedicalNerParams

def setParent(parent: Estimator[MedicalNerModel]): MedicalNerModel

Definition Classes: Model

def setPo(po: Float): FinanceNerModel.this.type

Learning rate decay coefficient.

Learning rate decay coefficient. Real Learning Rage = lr / (1 + po * epoch)

Definition Classes: MedicalNerParams

def setPretrainedModelPath(path: String): FinanceNerModel.this.type

Set the location of an already trained MedicalNerModel, which is used as a starting point for training the new model.

Definition Classes: MedicalNerParams

def setRandomValidationSplitPerEpoch(value: Boolean): FinanceNerModel.this.type

Do a random validation split after each epoch rather than at the beginning of training only.

Definition Classes: MedicalNerParams

def setSentenceTokenIndex(value: Boolean): FinanceNerModel.this.type

whether to include the token index for each sentence in annotation metadata, by default false.

whether to include the token index for each sentence in annotation metadata, by default false. If the value is true, the process might be slowed down.

Definition Classes: MedicalNerParams

def setStorageRef(value: String): FinanceNerModel.this.type

Definition Classes: HasStorageRef

def setTagsMapping(mapping: Map[String, String]): FinanceNerModel.this.type

A map specifying how old tags are mapped to new ones.

A map specifying how old tags are mapped to new ones. Maps are specified either using a list of comma separated strings, e.g. ("OLDTAG1,NEWTAG1", "OLDTAG2,NEWTAG2", ...) or by a Map data structure.

Definition Classes: MedicalNerParams

def setTagsMapping(mapping: ArrayList[String]): FinanceNerModel.this.type

Definition Classes: MedicalNerParams

def setTagsMapping(mapping: Array[String]): FinanceNerModel.this.type

A map specifying how old tags are mapped to new ones.

A map specifying how old tags are mapped to new ones. Maps are specified either using a list of comma separated strings, e.g. ("OLDTAG1,NEWTAG1", "OLDTAG2,NEWTAG2", ...) or by a Map data structure. It only works if setOverrideExistingTags is false.

Definition Classes: MedicalNerParams

def setTrainingClassDistribution(value: Map[String, Long]): FinanceNerModel.this.type

set the number of sentences to process in a single batch during inference

Definition Classes: MedicalNerModel

def setUseBestModel(value: Boolean): FinanceNerModel.this.type

Definition Classes: MedicalNerParams

def setUseContrib(value: Boolean): FinanceNerModel.this.type

Whether to use contrib LSTM Cells.

Whether to use contrib LSTM Cells. Not compatible with Windows. Might slightly improve accuracy.

Definition Classes: MedicalNerParams

val storageRef: Param[String]

Definition Classes: HasStorageRef

final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes: AnyRef

def tag(tokenized: Array[Array[WordpieceEmbeddingsSentence]]): Seq[Array[NerTaggedSentence]]

Definition Classes: MedicalNerModel

val tagsMapping: MapFeature[String, String]

A map specifying how old tags are mapped to new ones.

It only works if overrideExistingTags is set to false.

Definition Classes: MedicalNerParams

def toString(): String

Definition Classes: Identifiable → AnyRef → Any

val trainingClassDistribution: MapFeature[String, Long]

Definition Classes: MedicalNerModel

final def transform(dataset: Dataset[_]): DataFrame

Definition Classes: AnnotatorModel → Transformer

def transform(dataset: Dataset[_], paramMap: ParamMap): DataFrame

Definition Classes: Transformer
Annotations: @Since( "2.0.0" )

def transform(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): DataFrame

Definition Classes: Transformer
Annotations: @Since( "2.0.0" ) @varargs()

final def transformSchema(schema: StructType): StructType

Definition Classes: RawAnnotator → PipelineStage

def transformSchema(schema: StructType, logging: Boolean): StructType

Attributes: protected
Definition Classes: PipelineStage
Annotations: @DeveloperApi()

val uid: String

Definition Classes: FinanceNerModel → MedicalNerModel → Identifiable

val useBestModel: BooleanParam

Whether to restore and use the model from the epoch that has achieved the best performance at the end of the training.

By default false (keep the model from the last trained epoch).

The best model depends on the earlyStoppingCriterion, which can be F1-score on test/validation dataset or the value of loss.

Definition Classes: MedicalNerParams

val useContrib: BooleanParam

whether to use contrib LSTM Cells.

whether to use contrib LSTM Cells. Not compatible with Windows. Might slightly improve accuracy. By default true.

Definition Classes: MedicalNerParams

def validate(schema: StructType): Boolean

Attributes: protected
Definition Classes: RawAnnotator

def validateStorageRef(dataset: Dataset[_], inputCols: Array[String], annotatorType: String): Unit

Definition Classes: HasStorageRef

final def wait(): Unit

Definition Classes: AnyRef
Annotations: @throws( ... )

final def wait(arg0: Long, arg1: Int): Unit

Definition Classes: AnyRef
Annotations: @throws( ... )

final def wait(arg0: Long): Unit

Definition Classes: AnyRef
Annotations: @throws( ... ) @native()

def wrapColumnMetadata(col: Column): Column

Attributes: protected
Definition Classes: RawAnnotator

def write: MLWriter

Definition Classes: ParamsAndFeaturesWritable → DefaultParamsWritable → MLWritable

def writeTensorflowHub(path: String, tfPath: String, spark: SparkSession, suffix: String): Unit

Definition Classes: WriteTensorflowModel

def writeTensorflowModel(path: String, spark: SparkSession, tensorflow: TensorflowWrapper, suffix: String, filename: String, configProtoBytes: Option[Array[Byte]]): Unit

Definition Classes: WriteTensorflowModel

def writeTensorflowModelV2(path: String, spark: SparkSession, tensorflow: TensorflowWrapper, suffix: String, filename: String, configProtoBytes: Option[Array[Byte]], savedSignatures: Option[Map[String, String]]): Unit

Definition Classes: WriteTensorflowModel

Packages

FinanceNerModel

Companion object FinanceNerModel

class FinanceNerModel extends MedicalNerModel

Instance Constructors

Type Members

Value Members

Inherited from MedicalNerModel

Inherited from CheckLicense

Inherited from HasStorageRef

Inherited from WriteTensorflowModel

Inherited from HasBatchedAnnotate[MedicalNerModel]

Inherited from MedicalNerParams

Inherited from AnnotatorModel[MedicalNerModel]

Inherited from CanBeLazy

Inherited from RawAnnotator[MedicalNerModel]

Inherited from HasOutputAnnotationCol

Inherited from HasInputAnnotationCols

Inherited from HasOutputAnnotatorType

Inherited from ParamsAndFeaturesWritable

Inherited from HasFeatures

Inherited from DefaultParamsWritable

Inherited from MLWritable

Inherited from Model[MedicalNerModel]

Inherited from Transformer

Inherited from PipelineStage

Inherited from Logging

Inherited from Params

Inherited from Serializable

Inherited from Serializable

Inherited from Identifiable

Inherited from AnyRef

Inherited from Any

anno

getParam

param

setParam

Ungrouped

Packages

FinanceNerModel 

Companion object FinanceNerModel

class FinanceNerModel extends MedicalNerModel

Instance Constructors

Type Members

Value Members

Inherited from MedicalNerModel

Inherited from CheckLicense

Inherited from HasStorageRef

Inherited from WriteTensorflowModel

Inherited from HasBatchedAnnotate[MedicalNerModel]

Inherited from MedicalNerParams

Inherited from AnnotatorModel[MedicalNerModel]

Inherited from CanBeLazy

Inherited from RawAnnotator[MedicalNerModel]

Inherited from HasOutputAnnotationCol

Inherited from HasInputAnnotationCols

Inherited from HasOutputAnnotatorType

Inherited from ParamsAndFeaturesWritable

Inherited from HasFeatures

Inherited from DefaultParamsWritable

Inherited from MLWritable

Inherited from Model[MedicalNerModel]

Inherited from Transformer

Inherited from PipelineStage

Inherited from Logging

Inherited from Params

Inherited from Serializable

Inherited from Serializable

Inherited from Identifiable

Inherited from AnyRef

Inherited from Any

anno

getParam

param

setParam

Ungrouped

FinanceNerModel