com.johnsnowlabs.nlp.annotators.classification
GenericLogRegClassifierModel
Companion object GenericLogRegClassifierModel
class GenericLogRegClassifierModel extends GenericClassifierModel with ParamsAndFeaturesWritable
Logistic regression classification
Please check out the Models Hub for available models.
- Grouped
- Alphabetic
- By Inheritance
- GenericLogRegClassifierModel
- GenericClassifierModel
- CheckLicense
- HasSafeAnnotate
- HandleExceptionParams
- HasSimpleAnnotate
- WriteTensorflowModel
- HasStorageRef
- GenericClassifierParams
- AnnotatorModel
- CanBeLazy
- RawAnnotator
- HasOutputAnnotationCol
- HasInputAnnotationCols
- HasOutputAnnotatorType
- ParamsAndFeaturesWritable
- HasFeatures
- DefaultParamsWritable
- MLWritable
- Model
- Transformer
- PipelineStage
- Logging
- Params
- Serializable
- Serializable
- Identifiable
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Parameters
-
val
batchSize: IntParam
Batch size
Batch size
- Definition Classes
- GenericClassifierParams
-
val
datasetInfo: Param[String]
Descriptive information about the dataset being used.
Descriptive information about the dataset being used.
- Definition Classes
- GenericClassifierParams
-
val
doExceptionHandling: BooleanParam
If true, exceptions are handled.
If true, exceptions are handled. If exception causing data is passed to the model, a error annotation is emitted which has the exception message. Processing continues with the next one. This comes with a performance penalty.
- Definition Classes
- HandleExceptionParams
-
val
dropout: FloatParam
Dropout coefficient
Dropout coefficient
- Definition Classes
- GenericClassifierParams
-
val
epochsN: IntParam
Maximum number of epochs to train
Maximum number of epochs to train
- Definition Classes
- GenericClassifierParams
-
val
featureScaling: Param[String]
Feature scaling method.
Feature scaling method. Possible values are 'zscore', 'minmax' or empty (no scaling)
- Definition Classes
- GenericClassifierParams
-
val
fixImbalance: BooleanParam
Fix the imbalance in the training set by replicating examples of under represented categories
Fix the imbalance in the training set by replicating examples of under represented categories
- Definition Classes
- GenericClassifierParams
-
val
labelColumn: Param[String]
Column with label per each document
Column with label per each document
- Definition Classes
- GenericClassifierParams
-
val
learningRate: FloatParam
Learning Rate
Learning Rate
- Definition Classes
- GenericClassifierParams
-
val
modelFile: Param[String]
Location of file of the model used for classification
Location of file of the model used for classification
- Definition Classes
- GenericClassifierParams
-
val
multiClass: BooleanParam
If multiClass is set, the model will return all the labels with corresponding scores.
If multiClass is set, the model will return all the labels with corresponding scores. By default, multiClass is false.
- Definition Classes
- GenericClassifierParams
-
val
outputLogsPath: Param[String]
Folder path to save training logs.
Folder path to save training logs. If no path is specified, the logs won't be stored in disk. The path can be a local file path, a distributed file path (HDFS, DBFS), or a cloud storage (S3).
- Definition Classes
- GenericClassifierParams
-
val
validationSplit: FloatParam
The proportion of training dataset to be used as validation set.
The proportion of training dataset to be used as validation set.
The model will be validated against this dataset on each Epoch and will not be used for training. The value should be between 0.0 and 1.0.
- Definition Classes
- GenericClassifierParams
Annotator types
Required input and expected output annotator types
-
val
inputAnnotatorTypes: Array[AnnotatorType]
Input annotator types : FEATURE_VECTOR
Input annotator types : FEATURE_VECTOR
- Definition Classes
- GenericLogRegClassifierModel → GenericClassifierModel → HasInputAnnotationCols
-
val
outputAnnotatorType: String
Output annotator types : CATEGORY
Output annotator types : CATEGORY
- Definition Classes
- GenericLogRegClassifierModel → GenericClassifierModel → HasOutputAnnotatorType
Members
-
type
AnnotatorType = String
- Definition Classes
- HasOutputAnnotatorType
-
def
annotate(annotations: Seq[Annotation]): Seq[Annotation]
takes a document and annotations and produces new annotations of this annotator's annotation type
takes a document and annotations and produces new annotations of this annotator's annotation type
- annotations
Annotations that correspond to inputAnnotationCols generated by previous annotators if any
- returns
any number of annotations processed for every input annotation. Not necessary one to one relationship
- Definition Classes
- GenericLogRegClassifierModel → GenericClassifierModel → HasSimpleAnnotate
-
def
checkValidEnvironment(spark: Option[SparkSession], scopes: Seq[String]): Unit
- Definition Classes
- CheckLicense
-
def
checkValidScope(scope: String): Unit
- Definition Classes
- CheckLicense
-
def
checkValidScopeAndEnvironment(scope: String, spark: Option[SparkSession], checkLp: Boolean): Unit
- Definition Classes
- CheckLicense
-
def
checkValidScopesAndEnvironment(scopes: Seq[String], spark: Option[SparkSession], checkLp: Boolean): Unit
- Definition Classes
- CheckLicense
-
val
classes: StringArrayParam
- Definition Classes
- GenericClassifierModel
-
final
def
clear(param: Param[_]): GenericLogRegClassifierModel.this.type
- Definition Classes
- Params
-
def
copy(extra: ParamMap): GenericClassifierModel
- Definition Classes
- RawAnnotator → Model → Transformer → PipelineStage → Params
-
def
createDatabaseConnection(database: Name): RocksDBConnection
- Definition Classes
- HasStorageRef
-
def
dfAnnotate: UserDefinedFunction
- Definition Classes
- HasSimpleAnnotate
-
def
explainParam(param: Param[_]): String
- Definition Classes
- Params
-
def
explainParams(): String
- Definition Classes
- Params
-
final
def
extractParamMap(): ParamMap
- Definition Classes
- Params
-
final
def
extractParamMap(extra: ParamMap): ParamMap
- Definition Classes
- Params
-
val
features: ArrayBuffer[Feature[_, _, _]]
- Definition Classes
- HasFeatures
-
final
def
get[T](param: Param[T]): Option[T]
- Definition Classes
- Params
-
def
getCategories(): Array[String]
- Definition Classes
- GenericClassifierModel
-
def
getCategoryName(id: Int): String
- Definition Classes
- GenericClassifierModel
-
final
def
getDefault[T](param: Param[T]): Option[T]
- Definition Classes
- Params
-
def
getEncoder: GenericClassifierDataEncoder
- Definition Classes
- GenericClassifierModel
-
def
getInputCols: Array[String]
- Definition Classes
- HasInputAnnotationCols
-
def
getLazyAnnotator: Boolean
- Definition Classes
- CanBeLazy
-
final
def
getOrDefault[T](param: Param[T]): T
- Definition Classes
- Params
-
final
def
getOutputCol: String
- Definition Classes
- HasOutputAnnotationCol
-
def
getParam(paramName: String): Param[Any]
- Definition Classes
- Params
-
def
getStorageRef: String
- Definition Classes
- HasStorageRef
-
final
def
hasDefault[T](param: Param[T]): Boolean
- Definition Classes
- Params
-
def
hasParam(paramName: String): Boolean
- Definition Classes
- Params
-
def
hasParent: Boolean
- Definition Classes
- Model
-
final
def
isDefined(param: Param[_]): Boolean
- Definition Classes
- Params
-
final
def
isSet(param: Param[_]): Boolean
- Definition Classes
- Params
-
val
lazyAnnotator: BooleanParam
- Definition Classes
- CanBeLazy
- def loadModel(sparkSession: SparkSession, tfModel: TensorflowWrapper, categories: Array[String], encoder: GenericClassifierDataEncoder): GenericLogRegClassifierModel.this.type
-
def
model: TensorflowGenericClassifier
- Definition Classes
- GenericClassifierModel
-
def
onWrite(path: String, spark: SparkSession): Unit
- Definition Classes
- GenericClassifierModel → ParamsAndFeaturesWritable
-
val
optionalInputAnnotatorTypes: Array[String]
- Definition Classes
- HasInputAnnotationCols
-
lazy val
params: Array[Param[_]]
- Definition Classes
- Params
-
var
parent: Estimator[GenericClassifierModel]
- Definition Classes
- Model
-
def
save(path: String): Unit
- Definition Classes
- MLWritable
- Annotations
- @Since( "1.6.0" ) @throws( ... )
-
final
def
set[T](param: Param[T], value: T): GenericLogRegClassifierModel.this.type
- Definition Classes
- Params
-
def
setCategoryNames(categoryNames: Array[String]): GenericLogRegClassifierModel.this.type
- Definition Classes
- GenericClassifierModel
-
def
setEncoder(encoder: GenericClassifierDataEncoder): GenericLogRegClassifierModel.this.type
- Definition Classes
- GenericClassifierModel
-
final
def
setInputCols(value: String*): GenericLogRegClassifierModel.this.type
- Definition Classes
- HasInputAnnotationCols
-
def
setInputCols(value: Array[String]): GenericLogRegClassifierModel.this.type
- Definition Classes
- HasInputAnnotationCols
-
def
setLazyAnnotator(value: Boolean): GenericLogRegClassifierModel.this.type
- Definition Classes
- CanBeLazy
-
final
def
setOutputCol(value: String): GenericLogRegClassifierModel.this.type
- Definition Classes
- HasOutputAnnotationCol
-
def
setParent(parent: Estimator[GenericClassifierModel]): GenericClassifierModel
- Definition Classes
- Model
-
def
setStorageRef(value: String): GenericLogRegClassifierModel.this.type
- Definition Classes
- HasStorageRef
-
def
setTensorflowModel(spark: SparkSession, tf: TensorflowWrapper): GenericLogRegClassifierModel.this.type
- Definition Classes
- GenericClassifierModel
-
val
storageRef: Param[String]
- Definition Classes
- HasStorageRef
-
def
toString(): String
- Definition Classes
- Identifiable → AnyRef → Any
-
final
def
transform(dataset: Dataset[_]): DataFrame
- Definition Classes
- AnnotatorModel → Transformer
-
def
transform(dataset: Dataset[_], paramMap: ParamMap): DataFrame
- Definition Classes
- Transformer
- Annotations
- @Since( "2.0.0" )
-
def
transform(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): DataFrame
- Definition Classes
- Transformer
- Annotations
- @Since( "2.0.0" ) @varargs()
-
final
def
transformSchema(schema: StructType): StructType
- Definition Classes
- RawAnnotator → PipelineStage
-
val
uid: String
- Definition Classes
- GenericLogRegClassifierModel → GenericClassifierModel → Identifiable
-
def
validateStorageRef(dataset: Dataset[_], inputCols: Array[String], annotatorType: String): Unit
- Definition Classes
- HasStorageRef
-
def
write: MLWriter
- Definition Classes
- ParamsAndFeaturesWritable → DefaultParamsWritable → MLWritable
-
def
writeTensorflowHub(path: String, tfPath: String, spark: SparkSession, suffix: String): Unit
- Definition Classes
- WriteTensorflowModel
-
def
writeTensorflowModel(path: String, spark: SparkSession, tensorflow: TensorflowWrapper, suffix: String, filename: String, configProtoBytes: Option[Array[Byte]]): Unit
- Definition Classes
- WriteTensorflowModel
-
def
writeTensorflowModelV2(path: String, spark: SparkSession, tensorflow: TensorflowWrapper, suffix: String, filename: String, configProtoBytes: Option[Array[Byte]], savedSignatures: Option[Map[String, String]]): Unit
- Definition Classes
- WriteTensorflowModel
Parameter setters
-
def
setBatchSize(batch: Int): GenericLogRegClassifierModel.this.type
Batch size
Batch size
- Definition Classes
- GenericClassifierParams
-
def
setDatasetInfo(value: String): GenericLogRegClassifierModel.this.type
set descriptive information about the dataset being used
set descriptive information about the dataset being used
- Definition Classes
- GenericClassifierParams
-
def
setDoExceptionHandling(value: Boolean): GenericLogRegClassifierModel.this.type
If true, exceptions are handled.
If true, exceptions are handled. If exception causing data is passed to the model, a error annotation is emitted which has the exception message. Processing continues with the next one. This comes with a performance penalty.
- Definition Classes
- HandleExceptionParams
-
def
setDropout(dropout: Float): GenericLogRegClassifierModel.this.type
Dropout coefficient
Dropout coefficient
- Definition Classes
- GenericClassifierParams
-
def
setEpochsNumber(epochs: Int): GenericLogRegClassifierModel.this.type
Maximum number of epochs to train
Maximum number of epochs to train
- Definition Classes
- GenericClassifierParams
-
def
setFeatureScaling(featureScaling: String): GenericLogRegClassifierModel.this.type
Set the feature scaling method.
Set the feature scaling method. Possible values are 'zscore', 'minmax' or empty (no scaling)
- Definition Classes
- GenericClassifierParams
-
def
setFixImbalance(fix: Boolean): GenericLogRegClassifierModel.this.type
Fix imbalance of training set
Fix imbalance of training set
- Definition Classes
- GenericClassifierParams
-
def
setLabelColumn(column: String): GenericLogRegClassifierModel.this.type
Column with label per each document
Column with label per each document
- Definition Classes
- GenericClassifierParams
-
def
setModelFile(modelFile: String): GenericLogRegClassifierModel.this.type
Set the model file name
Set the model file name
- Definition Classes
- GenericClassifierParams
-
def
setMultiClass(value: Boolean): GenericLogRegClassifierModel.this.type
Sets the model in multi class prediction mode
Sets the model in multi class prediction mode
- Definition Classes
- GenericClassifierParams
-
def
setOutputLogsPath(outputLogsPath: String): GenericLogRegClassifierModel.this.type
Set the output log path
Set the output log path
- Definition Classes
- GenericClassifierParams
-
def
setValidationSplit(validationSplit: Float): GenericLogRegClassifierModel.this.type
Choose the proportion of training dataset to be validated against the model on each Epoch.
Choose the proportion of training dataset to be validated against the model on each Epoch. The value should be between 0.0 and 1.0 and by default it is 0.0 and off.
- Definition Classes
- GenericClassifierParams
-
def
setlearningRate(lr: Float): GenericLogRegClassifierModel.this.type
Learning Rate
Learning Rate
- Definition Classes
- GenericClassifierParams
Parameter getters
-
def
getBatchSize: Int
Batch size
Batch size
- Definition Classes
- GenericClassifierParams
-
def
getDatasetInfo: String
get descriptive information about the dataset being used
get descriptive information about the dataset being used
- Definition Classes
- GenericClassifierParams
-
def
getDropout: Float
Dropout coefficient
Dropout coefficient
- Definition Classes
- GenericClassifierParams
-
def
getFeatureScaling: String
Get feature scaling method
Get feature scaling method
- Definition Classes
- GenericClassifierParams
-
def
getFixImbalance: Boolean
Fix imbalance in training set
Fix imbalance in training set
- Definition Classes
- GenericClassifierParams
-
def
getLabelColumn: String
Column with label per each document
Column with label per each document
- Definition Classes
- GenericClassifierParams
-
def
getLearningRate: Float
Learning Rate
Learning Rate
- Definition Classes
- GenericClassifierParams
-
def
getMaxEpochs: Int
Maximum number of epochs to train
Maximum number of epochs to train
- Definition Classes
- GenericClassifierParams
-
def
getModelFile: String
Model file name
Model file name
- Definition Classes
- GenericClassifierParams
-
def
getMultiClass: Boolean
Gets the model multi class prediction mode
Gets the model multi class prediction mode
- Definition Classes
- GenericClassifierParams
-
def
getOutputLogsPath: String
Get output logs path
Get output logs path
- Definition Classes
- GenericClassifierParams
-
def
getValidationSplit: Float
Choose the proportion of training dataset to be validated against the model on each Epoch.
Choose the proportion of training dataset to be validated against the model on each Epoch. The value should be between 0.0 and 1.0 and by default it is 0.0 and off.
- Definition Classes
- GenericClassifierParams