Detect Assertion Status for Radiology

Description

Assign assertion status to clinical entities extracted by Radiology NER based on their context in the text.

Predicted Entities

Confirmed, Suspected, Negative.

Live Demo Open in Colab Download

How to use

Extract radiology entities using the radiology NER model in the pipeline and assign assertion status for them with assertion_dl_radiology pretrained model.

...
word_embeddings = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")\
  .setInputCols(["sentence", "token"])\
  .setOutputCol("embeddings")

radiology_ner = NerDLModel.pretrained("ner_radiology", "en", "clinical/models") \
  .setInputCols(["sentence", "token", "embeddings"]) \
  .setOutputCol("ner")

ner_converter = NerConverter() \
  .setInputCols(["sentence", "token", "ner"]) \
  .setOutputCol("ner_chunk")\
  .setWhiteList(["ImagingFindings"])

radiology_assertion = AssertionDLModel.pretrained("assertion_dl_radiology", "en", "clinical/models") \
    .setInputCols(["sentence", "ner_chunk", "embeddings"]) \
    .setOutputCol("assertion")
nlpPipeline = Pipeline(stages=[documentAssembler, sentenceDetector, tokenizer, word_embeddings, radiology_ner, ner_converter, radiology_assertion])

empty_data = spark.createDataFrame([[""]]).toDF("text")
model = nlpPipeline.fit(empty_data)
text = """Blunting of the left costophrenic angle on the lateral view posteriorly suggests a small left pleural effusion. No right-sided pleural effusion or pneumothorax is definitively seen. There are mildly displaced fractures of the left lateral 8th and likely 9th ribs."""

result = model.transform(spark.createDataFrame([[text]]).toDF("text"))
...
val word_embeddings = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")
  .setInputCols(Array("sentence", "token"))
  .setOutputCol("embeddings")

val radiology_ner = NerDLModel.pretrained("ner_radiology", "en", "clinical/models")
  .setInputCols(Array("sentence", "token", "embeddings"))
  .setOutputCol("ner")

val ner_converter = NerConverter() 
  .setInputCols(Array("sentence", "token", "ner")) 
  .setOutputCol("ner_chunk")
  .setWhiteList(Array("ImagingFindings"))

val radiology_assertion = AssertionDLModel.pretrained("assertion_dl_radiology", "en", "clinical/models")
    .setInputCols(Array("sentence", "ner_chunk", "embeddings"))
    .setOutputCol("assertion")

val nlpPipeline = new Pipeline().setStages(Array(documentAssembler,  sentenceDetector, tokenizer, word_embeddings, radiology_ner, ner_converter, radiology_assertion))

val result = pipeline.fit(Seq.empty["Blunting of the left costophrenic angle on the lateral view posteriorly suggests a small left pleural effusion. No right-sided pleural effusion or pneumothorax is definitively seen. There are mildly displaced fractures of the left lateral 8th and likely 9th ribs."].toDS.toDF("text")).transform(data)

Results

+---------------------------------------------------------------------------------------------------------------+-------------------+---------------+-------+---------+
|sentences                                                                                                      |chunk              |ner_label      |sent_id|assertion|
+---------------------------------------------------------------------------------------------------------------+-------------------+---------------+-------+---------+
|Blunting of the left costophrenic angle on the lateral view posteriorly suggests a small left pleural effusion.|Blunting           |ImagingFindings|0      |Confirmed|
|Blunting of the left costophrenic angle on the lateral view posteriorly suggests a small left pleural effusion.|effusion           |ImagingFindings|0      |Suspected|
|No right-sided pleural effusion or pneumothorax is definitively seen.                                          |effusion           |ImagingFindings|1      |Negative |
|No right-sided pleural effusion or pneumothorax is definitively seen.                                          |pneumothorax       |ImagingFindings|1      |Negative |
|There are mildly displaced fractures of the left lateral 8th and likely 9th ribs.                              |displaced fractures|ImagingFindings|2      |Confirmed|
+---------------------------------------------------------------------------------------------------------------+-------------------+---------------+-------+---------+

Model Information

Model Name: assertion_dl_radiology
Compatibility: Spark NLP 2.7.4+
License: Licensed
Edition: Official
Input Labels: [document, chunk, embeddings]
Output Labels: [assertion]
Language: en

Data Source

Custom internal labeled radiology dataset.

Benchmarking

label	 tp	 fp	 fn	 prec	 rec	 f1
Suspected	 629	 155	 159	 0.8022959	 0.7982234	 0.80025446
Negative	 417	 53	 36	 0.88723403	 0.9205298	 0.9035753
Confirmed	 2252	 173	 186	 0.9286598	 0.92370796	 0.92617726
tp: 3298 fp: 381 fn: 381 labels: 3
Macro-average	 prec: 0.87272996, rec: 0.88082033, f1: 0.8767565
Micro-average	 prec: 0.89643925, rec: 0.89643925, f1: 0.89643925