Description
This pipeline includes Named-Entity Recognition, Assertion Status, Relation Extraction and Entity Resolution models to extract information from oncology texts. This pipeline focuses on entities related to oncological diagnosis.
Predicted Entities
Adenopathy, Cancer_Dx, Cancer_Score, Direction, Grade, Histological_Type, Invasion, Lymph_Node, Lymph_Node_Modifier, Metastasis, Pathology_Result, Performance_Status, Site_Bone, Site_Brain, Site_Breast, Site_Liver, Site_Lung, Site_Lymph_Node, Site_Other_Body_Part, Staging, Tumor, Tumor_Description, Tumor_Finding, Tumor_Size
How to use
from sparknlp.pretrained import PretrainedPipeline
ner_pipeline = PretrainedPipeline("oncology_diagnosis_pipeline", "en", "clinical/models")
result = ner_pipeline.annotate("""Two years ago, the patient presented with a 4-cm tumor in her left breast. She was diagnosed with ductal carcinoma.
According to her last CT, she has no lung metastases.""")
import com.johnsnowlabs.nlp.pretrained.PretrainedPipeline
val ner_pipeline = PretrainedPipeline("oncology_diagnosis_pipeline", "en", "clinical/models")
val result = ner_pipeline.annotate("""Two years ago, the patient presented with a 4-cm tumor in her left breast. She was diagnosed with ductal carcinoma.
According to her last CT, she has no lung metastases.""")
Results
# ner_oncology_chunk
+----------+-----+---+-----------------+
|chunk |begin|end|ner_label |
+----------+-----+---+-----------------+
|4-cm |44 |47 |Tumor_Size |
|tumor |49 |53 |Tumor_Finding |
|left |62 |65 |Direction |
|breast |67 |72 |Site_Breast |
|ductal |98 |103|Histological_Type|
|carcinoma |105 |113|Cancer_Dx |
|lung |153 |156|Site_Lung |
|metastases|158 |167|Metastasis |
+----------+-----+---+-----------------+
# ner_oncology_diagnosis
+----------+-----+---+-----------------+
|chunk |begin|end|ner_label |
+----------+-----+---+-----------------+
|4-cm |44 |47 |Tumor_Size |
|tumor |49 |53 |Tumor_Finding |
|ductal |98 |103|Histological_Type|
|carcinoma |105 |113|Cancer_Dx |
|metastases|158 |167|Metastasis |
+----------+-----+---+-----------------+
# ner_oncology_tnm_chunk
+----------+-----+---+-----------------+
|chunk |begin|end|ner_label |
+----------+-----+---+-----------------+
|4-cm |44 |47 |Tumor_Description|
|tumor |49 |53 |Tumor |
|ductal |98 |103|Tumor_Description|
|carcinoma |105 |113|Cancer_Dx |
|metastases|158 |167|Metastasis |
+----------+-----+---+-----------------+
# assertion_oncology_wip
+----------+-----+---+-----------------+---------+
|chunk |begin|end|ner_label |assertion|
+----------+-----+---+-----------------+---------+
|tumor |49 |53 |Tumor_Finding |Present |
|ductal |98 |103|Histological_Type|Present |
|carcinoma |105 |113|Cancer_Dx |Present |
|metastases|158 |167|Metastasis |Absent |
+----------+-----+---+-----------------+---------+
# assertion_oncology_problem_wip
+----------+-----+---+-----------------+----------------------+
|chunk |begin|end|ner_label |assertion |
+----------+-----+---+-----------------+----------------------+
|tumor |49 |53 |Tumor_Finding |Medical_History |
|ductal |98 |103|Histological_Type|Medical_History |
|carcinoma |105 |113|Cancer_Dx |Medical_History |
|metastases|158 |167|Metastasis |Hypothetical_Or_Absent|
+----------+-----+---+-----------------+----------------------+
# re_oncology_wip
| chunk1 | entity1 | chunk2 | entity2 | relation | |
|-------:|--------:|--------------:|-----------:|--------------:|---------------|
| 0 | 4-cm | Tumor_Size | tumor | Tumor_Finding | is_related_to |
| 1 | 4-cm | Tumor_Size | carcinoma | Cancer_Dx | O |
| 2 | tumor | Tumor_Finding | breast | Site_Breast | is_related_to |
| 3 | breast | Site_Breast | carcinoma | Cancer_Dx | O |
| 4 | lung | Site_Lung | metastases | Metastasis | is_related_to |
# re_oncology_granular_wip
| | chunk1 | entity1 | chunk2 | entity2 | relation |
|--:|-------:|--------------:|-----------:|--------------:|---------------:|
| 0 | 4-cm | Tumor_Size | tumor | Tumor_Finding | is_size_of |
| 1 | 4-cm | Tumor_Size | carcinoma | Cancer_Dx | O |
| 2 | tumor | Tumor_Finding | breast | Site_Breast | is_location_of |
| 3 | breast | Site_Breast | carcinoma | Cancer_Dx | O |
| 4 | lung | Site_Lung | metastases | Metastasis | is_location_of |
# re_oncology_size_wip
| | chunk1 | entity1 | chunk2 | entity2 | relation |
|--:|-------:|-----------:|----------:|--------------:|-----------:|
| 0 | 4-cm | Tumor_Size | tumor | Tumor_Finding | is_size_of |
| 1 | 4-cm | Tumor_Size | carcinoma | Cancer_Dx | O |
# ICD-O resolver
+----------+-----+---+-----------------+------+-----------------+
|chunk |begin|end|ner_label |code |normalized_term |
+----------+-----+---+-----------------+------+-----------------+
|tumor |49 |53 |Tumor_Finding |8000/1|tumor |
|breast |67 |72 |Site_Breast |C50 |breast |
|ductal |98 |103|Histological_Type|8500/2|dcis |
|carcinoma |105 |113|Cancer_Dx |8010/3|carcinoma |
|lung |153 |156|Site_Lung |C34.9 |lung |
|metastases|158 |167|Metastasis |8000/6|tumor, metastatic|
+----------+-----+---+-----------------+------+-----------------+
Model Information
| Model Name: | oncology_diagnosis_pipeline |
| Type: | pipeline |
| Compatibility: | Healthcare NLP 5.2.0+ |
| License: | Licensed |
| Edition: | Official |
| Language: | en |
| Size: | 2.4 GB |
Included Models
- DocumentAssembler
- SentenceDetectorDLModel
- TokenizerModel
- WordEmbeddingsModel
- MedicalNerModel
- NerConverter
- MedicalNerModel
- NerConverter
- MedicalNerModel
- NerConverter
- ChunkMergeModel
- ChunkMergeModel
- AssertionDLModel
- AssertionDLModel
- PerceptronModel
- DependencyParserModel
- RelationExtractionModel
- RelationExtractionModel
- RelationExtractionModel
- ChunkMergeModel
- Chunk2Doc
- BertSentenceEmbeddings
- SentenceEntityResolverModel