Explain Clinical Document - Granular

Description

This pipeline is designed to:

• extract clinical entities
• assign assertion status to the extracted entities
• establish relations between the extracted entities

from clinical texts. In this pipeline, ner_jsl NER model, assertion_jsl assertion model, and re_test_result_date relation extraction model were used to achieve those tasks. Here are the NER, assertion, and relation extraction labels this pipeline can extract.

• Clinical Entity Labels: Admission_Discharge, Age, Alcohol, Allergen, BMI, Birth_Entity, Blood_Pressure, Cerebrovascular_Disease, Clinical_Dept, Communicable_Disease, Date, Death_Entity, Diabetes, Diet, Direction, Disease_Syndrome_Disorder, Dosage, Drug_BrandName, Drug_Ingredient, Duration, EKG_Findings, Employment, External_body_part_or_region, Family_History_Header, Fetus_NewBorn, Form, Frequency, Gender, HDL, Heart_Disease, Height, Hyperlipidemia, Hypertension, ImagingFindings, Imaging_Technique, Injury_or_Poisoning, Internal_organ_or_component, Kidney_Disease, LDL, Labour_Delivery, Medical_Device, Medical_History_Header, Modifier, O2_Saturation, Obesity, Oncological, Overweight, Oxygen_Therapy, Pregnancy, Procedure, Psychological_Condition, Pulse, Race_Ethnicity, Relationship_Status, RelativeDate, RelativeTime, Respiration, Route, Section_Header, Sexually_Active_or_Sexual_Orientation, Smoking, Social_History_Header, Strength, Substance, Substance_Quantity, Symptom, Temperature, Test, Test_Result, Time, Total_Cholesterol, Treatment, Triglycerides, VS_Finding, Vaccine, Vaccine_Name, Vital_Signs_Header, Weight

• Assertion Status Labels: Hypothetical, Someoneelse, Past, Absent, Family, Planned, Possible, Present

• Relation Extraction Labels: is_finding_of, is_date_of, is_result_of, O

Predicted Entities

Admission_Discharge, Age, Alcohol, Allergen, BMI, Birth_Entity, Blood_Pressure, Cerebrovascular_Disease, Clinical_Dept, Communicable_Disease, Date, Death_Entity, Diabetes, Diet, Direction, Disease_Syndrome_Disorder, Dosage, Drug_BrandName, Drug_Ingredient, Duration, EKG_Findings, Employment, External_body_part_or_region, Family_History_Header, Fetus_NewBorn, Form, Frequency, Gender, HDL, Heart_Disease, Height, Hyperlipidemia, Hypertension, ImagingFindings, Imaging_Technique, Injury_or_Poisoning, Internal_organ_or_component, Kidney_Disease, LDL, Labour_Delivery, Medical_Device, Medical_History_Header, Modifier, O2_Saturation, Obesity, Oncological, Overweight, Oxygen_Therapy, Pregnancy, Procedure, Psychological_Condition, Pulse, Race_Ethnicity, Relationship_Status, RelativeDate, RelativeTime, Respiration, Route, Section_Header, Sexually_Active_or_Sexual_Orientation, Smoking, Social_History_Header, Strength, Substance, Substance_Quantity, Symptom, Temperature, Test, Test_Result, Time, Total_Cholesterol, Treatment, Triglycerides, VS_Finding, Vaccine, Vaccine_Name, Vital_Signs_Header, Weight

Live Demo Open in Colab Download Copy S3 URI

How to use

from sparknlp.pretrained import PretrainedPipeline

ner_pipeline = PretrainedPipeline("explain_clinical_doc_granular", "en", "clinical/models")

result = ner_pipeline.fullAnnotate("""The patient admitted for gastrointestinal pathology, under working treatment.
History of prior heart murmur with echocardiogram findings as above on March 1998.
According to the latest echocardiogram, basically revealed normal left ventricular function with left atrial enlargement .
Based on the above findings, we will treat her medically with ACE inhibitors and diuretics and see how she fares.""")

import com.johnsnowlabs.nlp.pretrained.PretrainedPipeline

val ner_pipeline = PretrainedPipeline("explain_clinical_doc_granular", "en", "clinical/models")

val result = ner_pipeline.fullAnnotate("""The patient admitted for gastrointestinal pathology, under working treatment.
History of prior heart murmur with echocardiogram findings as above on March 1998.
According to the latest echocardiogram, basically revealed normal left ventricular function with left atrial enlargement .
Based on the above findings, we will treat her medically with ACE inhibitors and diuretics and see how she fares.""")

Results

# ner

|    | chunks                     |   begin |   end | entities            |
|---:|:---------------------------|--------:|------:|:--------------------|
|  0 | admitted                   |      12 |    19 | Admission_Discharge |
|  1 | gastrointestinal pathology |      25 |    50 | Clinical_Dept       |
|  2 | heart murmur               |      95 |   106 | Heart_Disease       |
|  3 | echocardiogram             |     113 |   126 | Test                |
|  4 | March 1998                 |     149 |   158 | Date                |
|  5 | echocardiogram             |     185 |   198 | Test                |
|  6 | normal                     |     220 |   225 | Test_Result         |
|  7 | left ventricular function  |     227 |   251 | Test                |
|  8 | left atrial enlargement    |     258 |   280 | Heart_Disease       |
|  9 | her                        |     327 |   329 | Gender              |
| 10 | ACE inhibitors             |     346 |   359 | Drug_Ingredient     |
| 11 | diuretics                  |     365 |   373 | Drug_Ingredient     |
| 12 | she                        |     387 |   389 | Gender              |

# assertion

|sentence_id|begin|end|entity                   |label          |assertion_status|
+-----------+-----+---+-------------------------+---------------+----------------+
|1          |96   |107|heart murmur             |Heart_Disease  |Past            |
|1          |114  |127|echocardiogram           |Test           |Past            |
|2          |187  |200|echocardiogram           |Test           |Present         |
|2          |222  |227|normal                   |Test_Result    |Present         |
|2          |229  |253|left ventricular function|Test           |Present         |
|2          |260  |282|left atrial enlargement  |Heart_Disease  |Present         |
|3          |348  |361|ACE inhibitors           |Drug_Ingredient|Planned         |
|3          |367  |375|diuretics                |Drug_Ingredient|Planned         |


#relation

|sentence_id|relations    |relations_entity1|relations_chunk1         |relations_entity2|relations_chunk2         |
+-----------+-------------+-----------------+-------------------------+-----------------+-------------------------+
|1          |is_finding_of|Heart_Disease    |heart murmur             |Test             |echocardiogram           |
|1          |is_date_of   |Heart_Disease    |heart murmur             |Date             |March 1998               |
|1          |is_date_of   |Test             |echocardiogram           |Date             |March 1998               |
|2          |is_finding_of|Test             |echocardiogram           |Heart_Disease    |left atrial enlargement  |
|2          |is_result_of |Test_Result      |normal                   |Test             |left ventricular function|
|2          |is_finding_of|Test             |left ventricular function|Heart_Disease    |left atrial enlargement  |

Model Information

Included Models

• DocumentAssembler
• SentenceDetectorDLModel
• TokenizerModel
• WordEmbeddingsModel
• MedicalNerModel
• NerConverterInternalModel
• NerConverterInternalModel
• AssertionDLModel
• PerceptronModel
• DependencyParserModel
• RelationExtractionModel