sparknlp_jsl.annotator.windowed.windowed_sentence
#
Module Contents#
Classes#
The WindowedSentenceModel class is used to combine a series of sentences based on specific window configurations. |
- class WindowedParams#
- glueString#
- windowSize#
- setGlueString(value)#
Sets string to use to join the neighboring elements together
- Parameters:
value (string) – string to use to join the neighboring elements together
- setWindowSize(value)#
Sets size of the sliding window
- Parameters:
value (int) – size of the sliding window
- class WindowedSentenceModel(classname='com.johnsnowlabs.nlp.annotators.windowed.WindowedSentenceModel', java_model=None)#
Bases:
sparknlp_jsl.common.AnnotatorModelInternal
,WindowedParams
- The WindowedSentenceModel class is used to combine a series of sentences based on specific window configurations.
Returns the joined results after windowing its inputs
Input Annotation types
Output Annotation type
DOCUMENT
DOCUMENT
- Parameters:
windowSize – size of the sliding window
glueString – string to use to join the neighboring elements together
Examples
>>> import sparknlp >>> from sparknlp.base import * >>> from sparknlp_jsl.common import * >>> from sparknlp.annotator import * >>> from sparknlp.training import * >>> import sparknlp_jsl >>> from sparknlp_jsl.base import * >>> from sparknlp_jsl.annotator import * >>> from pyspark.ml import Pipeline >>> documentAssembler = DocumentAssembler() ... .setInputCol("text") ... .setOutputCol("document") >>> sentenceDetector = SentenceDetector() ... .setInputCols("document") ... .setOutputCol("sentence") >>> windowedSentence = WindowedSentenceModel() ... .setWindowSize(1) ... .setInputCols("sentence") ... .setOutputCol("five") ... .setGlueString(":::") >>> flattener = Flattener() ... .setInputCols("five")
>>> pipeline = Pipeline(stages=[documentAssembler,sentenceDetector,windowedSentence,flattener])
>>> data = spark.createDataFrame([["A 28-year-old female with a history of gestational diabetes mellitus diagnosed eight years.Two weeks prior to presentation respiratory tract infection.She was on for HTG . She had been of presentation . examination benign with no or rigidity ."]]).toDF("text")
>>> model = pipeline.fit(data).transform(data) >>> model.show(truncate=False)
- getter_attrs = []#
- glueString#
- inputAnnotatorTypes#
- inputCols#
- lazyAnnotator#
- optionalInputAnnotatorTypes = []#
- outputAnnotatorType#
- outputCol#
- skipLPInputColsValidation = True#
- windowSize#
- clear(param)#
Clears a param from the param map if it has been explicitly set.
- copy(extra=None)#
Creates a copy of this instance with the same uid and some extra params. This implementation first calls Params.copy and then make a copy of the companion Java pipeline component with extra params. So both the Python wrapper and the Java pipeline component get copied.
- Parameters:
extra (dict, optional) – Extra parameters to copy to the new instance
- Returns:
Copy of this instance
- Return type:
JavaParams
- explainParam(param)#
Explains a single param and returns its name, doc, and optional default value and user-supplied value in a string.
- explainParams()#
Returns the documentation of all params with their optionally default values and user-supplied values.
- extractParamMap(extra=None)#
Extracts the embedded default param values and user-supplied values, and then merges them with extra values from input into a flat param map, where the latter value is used if there exist conflicts, i.e., with ordering: default param values < user-supplied values < extra.
- Parameters:
extra (dict, optional) – extra param values
- Returns:
merged param map
- Return type:
dict
- getInputCols()#
Gets current column names of input annotations.
- getLazyAnnotator()#
Gets whether Annotator should be evaluated lazily in a RecursivePipeline.
- getOrDefault(param)#
Gets the value of a param in the user-supplied param map or its default value. Raises an error if neither is set.
- getOutputCol()#
Gets output column name of annotations.
- getParam(paramName)#
Gets a param by its name.
- getParamValue(paramName)#
Gets the value of a parameter.
- Parameters:
paramName (str) – Name of the parameter
- hasDefault(param)#
Checks whether a param has a default value.
- hasParam(paramName)#
Tests whether this instance contains a param with a given (string) name.
- inputColsValidation(value)#
- isDefined(param)#
Checks whether a param is explicitly set by user or has a default value.
- isSet(param)#
Checks whether a param is explicitly set by user.
- classmethod load(path)#
Reads an ML instance from the input path, a shortcut of read().load(path).
- classmethod read()#
Returns an MLReader instance for this class.
- save(path)#
Save this ML instance to the given path, a shortcut of ‘write().save(path)’.
- set(param, value)#
Sets a parameter in the embedded param map.
- setForceInputTypeValidation(etfm)#
- setGlueString(value)#
Sets string to use to join the neighboring elements together
- Parameters:
value (string) – string to use to join the neighboring elements together
- setInputCols(*value)#
Sets column names of input annotations.
- Parameters:
*value (List[str]) – Input columns for the annotator
- setLazyAnnotator(value)#
Sets whether Annotator should be evaluated lazily in a RecursivePipeline.
- Parameters:
value (bool) – Whether Annotator should be evaluated lazily in a RecursivePipeline
- setOutputCol(value)#
Sets output column name of annotations.
- Parameters:
value (str) – Name of output column
- setParamValue(paramName)#
Sets the value of a parameter.
- Parameters:
paramName (str) – Name of the parameter
- setParams()#
- setWindowSize(value)#
Sets size of the sliding window
- Parameters:
value (int) – size of the sliding window
- transform(dataset, params=None)#
Transforms the input dataset with optional parameters.
New in version 1.3.0.
- Parameters:
dataset (
pyspark.sql.DataFrame
) – input datasetparams (dict, optional) – an optional param map that overrides embedded params.
- Returns:
transformed dataset
- Return type:
- write()#
Returns an MLWriter instance for this ML instance.