Type Members

1.  class AssertionDLApproach extends nlp.annotators.assertion.dl.AssertionDLApproach

   Contains all the methods for training an AssertionDLModel.

   Contains all the methods for training an AssertionDLModel. For pretrained models please use AssertionDLModel and see the Models Hub for available models.

   Example

   First, pipeline stages for pre-processing the dataset (containing columns for text and label) are defined.

   val document = new DocumentAssembler()
     .setInputCol("text")
     .setOutputCol("document")
   val chunk = new Doc2Chunk()
     .setInputCols("document")
     .setOutputCol("chunk")
   val token = new Tokenizer()
     .setInputCols("document")
     .setOutputCol("token")
   val embeddings = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")
     .setInputCols("document", "token")
     .setOutputCol("embeddings")

   Define AssertionDLApproach with parameters and start training

   val assertionStatus = new AssertionDLApproach()
     .setLabelCol("label")
     .setInputCols("document", "chunk", "embeddings")
     .setOutputCol("assertion")
     .setBatchSize(128)
     .setDropout(0.012f)
     .setLearningRate(0.015f)
     .setEpochs(1)
     .setStartCol("start")
     .setEndCol("end")
     .setMaxSentLen(250)
   
   val trainingPipeline = new Pipeline().setStages(Array(
     document,
     chunk,
     token,
     embeddings,
     assertionStatus
   ))
   
   val assertionModel = trainingPipeline.fit(data)
   val assertionResults = assertionModel.transform(data).cache()

   See also

   AssertionDLModel for using pretrained models

   AssertionLogRegModel for non deep learning based extraction

2.  class AssertionDLModel extends nlp.annotators.assertion.dl.AssertionDLModel

   This is a main class in AssertionDL family.

   This is a main class in AssertionDL family. A Deep Learning based approach is used to extract Assertion Status from extracted entities and text. AssertionDLModel requires DOCUMENT, CHUNK and WORD_EMBEDDINGS type annotator inputs, which can be obtained by e.g a DocumentAssembler, NerConverter and WordEmbeddingsModel. The result is an assertion status annotation for each recognized entity. Possible values include “present”, “absent”, “hypothetical”, “conditional”, “associated_with_other_person” etc.

   For pretrained models please see the Models Hub for available models.

   Example

   Define pipeline stages to extract NER chunks first

   val data = Seq(
     "Patient with severe fever and sore throat",
     "Patient shows no stomach pain",
     "She was maintained on an epidural and PCA for pain control.").toDF("text")
   val documentAssembler = new DocumentAssembler().setInputCol("text").setOutputCol("document")
   val sentenceDetector = new SentenceDetector().setInputCols("document").setOutputCol("sentence")
   val tokenizer = new Tokenizer().setInputCols("sentence").setOutputCol("token")
   val embeddings = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")
     .setOutputCol("embeddings")
   val nerModel = MedicalNerModel.pretrained("ner_clinical", "en", "clinical/models")
     .setInputCols("sentence", "token", "embeddings").setOutputCol("ner")
   val nerConverter = new NerConverter().setInputCols("sentence", "token", "ner").setOutputCol("ner_chunk")

   Then a pretrained AssertionDLModel is used to extract the assertion status

   val clinicalAssertion = AssertionDLModel.pretrained("assertion_dl", "en", "clinical/models")
     .setInputCols("sentence", "ner_chunk", "embeddings")
     .setOutputCol("assertion")
   
   val assertionPipeline = new Pipeline().setStages(Array(
     documentAssembler,
     sentenceDetector,
     tokenizer,
     embeddings,
     nerModel,
     nerConverter,
     clinicalAssertion
   ))
   
   val assertionModel = assertionPipeline.fit(data)

   Show results

   val result = assertionModel.transform(data)
   result.selectExpr("ner_chunk.result", "assertion.result").show(3, truncate=false)
   +--------------------------------+--------------------------------+
   |result                          |result                          |
   +--------------------------------+--------------------------------+
   |[severe fever, sore throat]     |[present, present]              |
   |[stomach pain]                  |[absent]                        |
   |[an epidural, PCA, pain control]|[present, present, hypothetical]|
   +--------------------------------+--------------------------------+

   See also

   AssertionDLApproach for training a custom AssertionDLModel

   AssertionLogRegModel for non deep learning based extraction