Type Members

1.  class ContextualAssertion extends AnnotatorModel[ContextualAssertion] with HasSimpleAnnotate[ContextualAssertion] with HandleExceptionParams with HasSafeAnnotate[ContextualAssertion] with CheckLicense

   An annotator model for contextual assertion analysis.

   An annotator model for contextual assertion analysis.

   This model identifies contextual cues within text data, such as negation, uncertainty, and assertion. It is used clinical assertion detection, etc. It annotates text chunks with assertions based on configurable rules, prefix and suffix patterns, and exception patterns.

   Example

   Define pipeline stages to extract entities

   val documentAssembler = new DocumentAssembler()
     .setInputCol("text")
     .setOutputCol("document")
   
   val sentenceDetector = new SentenceDetector()
     .setInputCols(Array("document"))
     .setOutputCol("sentences")
   
   val tokenizer = new Tokenizer()
     .setInputCols(Array("sentences"))
     .setOutputCol("tokens")
   
   val embedder = WordEmbeddingsModel
     .pretrained("embeddings_clinical", "en", "clinical/models")
     .setInputCols(Array("sentences", "tokens"))
     .setOutputCol("embeddings")
   
   val nerTagger = MedicalNerModel
     .pretrained("ner_clinical", "en", "clinical/models")
     .setInputCols(Array("sentences", "tokens", "embeddings"))
     .setOutputCol("nerTags")
   
   val nerConverter = new NerConverterInternal()
     .setInputCols(Array("sentences", "tokens", "nerTags"))
     .setOutputCol("nerChunks")

   Define Contextual Assertion rules that are to be searched for in the text

      val contextualAssertion = new ContextualAssertion()
       .setInputCols(Array("sentences", "tokens","nerChunks"))
       .setOutputCol("assertion")
       .setScopeWindow(2,2)
       .setPrefixRegexPatterns(Array("\\b(no|without|denies|never|none|free of|negative for|not include)\\b"))
       .setSuffixRegexPatterns(Array("\\b(free of|negative for|absence of|absence|not|neither|rule out)\\b"))
       .setPrefixPatterns(Array("not","never"))
       .setSuffixPatterns(Array("no","never"))
       .setCaseSensitive(false)
       .setAssertion("absent")
   
   val flattener = new Flattener()
     .setInputCols("assertion")
     .setExplodeSelectedFields(Map("assertion" -> Array("result",
                                                        "metadata.ner_chunk as ner_chunk ",
                                                        "metadata.ner_label as ner_label")))
   
   
   
   val pipeline = new Pipeline()
     .setStages(Array(documentAssembler,
                      sentenceDetector,
                      tokenizer,
                      embedder,
                      nerTagger,
                      nerConverter,
                      contextualAssertion,
                      flattener
                 )).fit(testDataSet)
   
   val testText = "Patient resting in bed. Patient given azithromycin without any difficulty. Patient has audible wheezing, states chest tightness." +
     " No evidence of hypertension. Patient denies nausea at this time. zofran declined. Patient is also having intermittent sweating " +
     "associated with pneumonia. Patient refused pain but tylenol still given. Neither substance abuse nor alcohol use however cocaine " +
     "once used in the last year. Alcoholism unlikely. Patient has headache and fever. Patient is not diabetic. Not clearly of diarrhea. " +
     "Lab reports confirm lymphocytopenia. Cardaic rhythm is Sinus bradycardia. Patient also has a history of cardiac injury." +
     "No kidney injury reported. No abnormal rashes or ulcers. Patient might not have liver disease. Confirmed absence of hemoptysis." +
     "Although patient has severe pneumonia and fever, test reports are negative for COVID-19 infection. COVID-19 viral infection absent."
   
   val testDataSet = Seq(testText).toDS.toDF("text")
   val dataSetResult = pipeline.transform(testDataSet)

   Show results

   result.show(truncate=false)
   
     +----------------+------------------+---------+
     |assertion_result|         ner_chunk|ner_label|
    +----------------+------------------+---------+
     |          absent|    any difficulty|  PROBLEM|
     |          absent|            nausea|  PROBLEM|
     |          absent|          diabetic|  PROBLEM|
     |          absent|     kidney injury|  PROBLEM|
     |          absent|   abnormal rashes|  PROBLEM|
     |          absent|     liver disease|  PROBLEM|
     |          absent|COVID-19 infection|  PROBLEM|
     +----------------+------------------+---------+

2.  trait ReadablePretrainedContextualAssertion extends DefaultParamsReadable[ContextualAssertion] with HasPretrained[ContextualAssertion]