Extract relations between problem, test, and findings in reports

Description

Find relations between diagnosis, tests and imaging findings in radiology reports. 1 : The two entities are related. 0 : The two entities are not related

Predicted Entities

0, 1

Live Demo Open in Colab Download

How to use

In the table below, re_test_problem_finding RE model, its labels, optimal NER model, and meaningful relation pairs are illustrated.

RE MODEL RE MODEL LABES NER MODEL RE PAIRS
re_test_problem_finding 0,1 ner_jsl [“test-cerebrovascular_disease”,
“cerebrovascular_disease-test”,
“test-communicable_disease”,
“communicable_disease-test”,
“test-diabetes”, “diabetes-test”,
“test-disease_syndrome_disorder”,
“disease_syndrome_disorder-test”,
“test-heart_disease”,
“heart_disease-test”,
“test-hyperlipidemia”,
“hyperlipidemia-test”,
“test-hypertension”,
“hypertension-test”,
“test-injury_or_poisoning”,
“injury_or_poisoning-test”,
“test-kidney_disease”,
“kidney_disease-test”,
“test-obesity”,
“obesity-test”,
“test-oncological”,
“oncological-test”,
“test-psychological_condition”,
“psychological_condition-test”,
“test-symptom”, “symptom-test”,
“ekg_findings-disease_syndrome_disorder”,
“disease_syndrome_disorder-ekg_findings”,
“ekg_findings-heart_disease”,
“heart_disease-ekg_findings”,
“ekg_findings-symptom”,
“symptom-ekg_findings”,
“imagingfindings-cerebrovascular_disease”,
“cerebrovascular_disease-imagingfindings”,
“imagingfindings-communicable_disease”,
“communicable_disease-imagingfindings”,
“imagingfindings-disease_syndrome_disorder”,
“disease_syndrome_disorder-imagingfindings”,
“imagingfindings-heart_disease”,
“heart_disease-imagingfindings”,
“imagingfindings-hyperlipidemia”,
“hyperlipidemia-imagingfindings”,
“imagingfindings-hypertension”,
“hypertension-imagingfindings”,
“imagingfindings-injury_or_poisoning”,
“injury_or_poisoning-imagingfindings”,
“imagingfindings-kidney_disease”,
“kidney_disease-imagingfindings”,
“imagingfindings-oncological”,
“oncological-imagingfindings”,
“imagingfindings-psychological_condition”,
“psychological_condition-imagingfindings”,
“imagingfindings-symptom”,
“symptom-imagingfindings”,
“vs_finding-cerebrovascular_disease”,
“cerebrovascular_disease-vs_finding”,
“vs_finding-communicable_disease”,
“communicable_disease-vs_finding”,
“vs_finding-diabetes”,
“diabetes-vs_finding”,
“vs_finding-disease_syndrome_disorder”,
“disease_syndrome_disorder-vs_finding”,
“vs_finding-heart_disease”,
“heart_disease-vs_finding”,
“vs_finding-hyperlipidemia”,
“hyperlipidemia-vs_finding”,
“vs_finding-hypertension”,
“hypertension-vs_finding”,
“vs_finding-injury_or_poisoning”,
“injury_or_poisoning-vs_finding”,
“vs_finding-kidney_disease”,
“kidney_disease-vs_finding”,
“vs_finding-obesity”,
“obesity-vs_finding”,
“vs_finding-oncological”,
“oncological-vs_finding”,
“vs_finding-overweight”,
“overweight-vs_finding”,
“vs_finding-psychological_condition”,
“psychological_condition-vs_finding”,
“vs_finding-symptom”, “symptom-vs_finding”]
ner_tagger = sparknlp.annotators.NerDLModel()\
    .pretrained('jsl_ner_wip_clinical',"en","clinical/models")\
    .setInputCols("sentences", "tokens", "embeddings")\
    .setOutputCol("ner_tags") 

re_model = RelationExtractionModel()\
    .pretrained("re_test_problem_finding", "en", 'clinical/models')\
    .setInputCols(["embeddings", "pos_tags", "ner_chunks", "dependencies"])\
    .setOutputCol("relations")\
    .setMaxSyntacticDistance(4)\ #default: 0
    .setPredictionThreshold(0.9)\ #default: 0.5
    .setRelationPairs(["procedure-symptom"]) # Possible relation pairs. Default: All Relations.

nlp_pipeline = Pipeline(stages=[ documenter, sentencer,tokenizer, words_embedder, pos_tagger,  clinical_ner_tagger,ner_chunker, dependency_parser,re_model])

light_pipeline = LightPipeline(nlp_pipeline.fit(spark.createDataFrame([['']]).toDF("text")))

annotations = light_pipeline.fullAnnotate(''''Targeted biopsy of this lesion for histological correlation should be considered.'')

Results

| index | relations    | entity1      | chunk1              | entity2      |  chunk2 |
|-------|--------------|--------------|---------------------|--------------|---------|
| 0     | 1            | PROCEDURE    | biopsy              | SYMPTOM      |  lesion | 

Model Information

Model Name: re_test_problem_finding
Type: re
Compatibility: Spark NLP for Healthcare 2.7.1+
License: Licensed
Edition: Official
Input Labels: [embeddings, pos_tags, train_ner_chunks, dependencies]
Output Labels: [relations]
Language: en

Data Source

Trained on internal datasets.