Description
This pipeline includes Named-Entity Recognition, Assertion Status, Relation Extraction and Entity Resolution models to extract information from oncology texts. This pipeline focuses on entities related to oncological diagnosis.
Predicted Entities
Adenopathy, Cancer_Dx, Cancer_Score, Direction, Grade, Histological_Type, Invasion, Lymph_Node, Lymph_Node_Modifier, Metastasis, Pathology_Result, Performance_Status, Site_Bone, Site_Brain, Site_Breast, Site_Liver, Site_Lung, Site_Lymph_Node, Site_Other_Body_Part, Staging, Tumor, Tumor_Description, Tumor_Finding, Tumor_Size
How to use
from sparknlp.pretrained import PretrainedPipeline
pipeline = PretrainedPipeline("oncology_diagnosis_pipeline", "en", "clinical/models")
text = '''Two years ago, the patient presented with a 4-cm tumor in her left breast. She was diagnosed with ductal carcinoma.
According to her last CT, she has no lung metastases.'''
result = pipeline.fullAnnotate(text)
import com.johnsnowlabs.nlp.pretrained.PretrainedPipeline
val pipeline = new PretrainedPipeline("oncology_diagnosis_pipeline", "en", "clinical/models")
val text = "Two years ago, the patient presented with a 4-cm tumor in her left breast. She was diagnosed with ductal carcinoma.
According to her last CT, she has no lung metastases."
val result = pipeline.fullAnnotate(text)
import nlu
nlu.load("en.oncology_diagnosis.pipeline").predict("""Two years ago, the patient presented with a 4-cm tumor in her left breast. She was diagnosed with ductal carcinoma.
According to her last CT, she has no lung metastases.""")
Results
******************** ner_oncology_wip results ********************
| chunk | ner_label |
|:-----------|:------------------|
| 4-cm | Tumor_Size |
| tumor | Tumor_Finding |
| left | Direction |
| breast | Site_Breast |
| ductal | Histological_Type |
| carcinoma | Cancer_Dx |
| lung | Site_Lung |
| metastases | Metastasis |
******************** ner_oncology_diagnosis_wip results ********************
| chunk | ner_label |
|:-----------|:------------------|
| 4-cm | Tumor_Size |
| tumor | Tumor_Finding |
| ductal | Histological_Type |
| carcinoma | Cancer_Dx |
| metastases | Metastasis |
******************** ner_oncology_tnm_wip results ********************
| chunk | ner_label |
|:-----------|:------------------|
| 4-cm | Tumor_Description |
| tumor | Tumor |
| ductal | Tumor_Description |
| carcinoma | Cancer_Dx |
| metastases | Metastasis |
******************** assertion_oncology_wip results ********************
| chunk | ner_label | assertion |
|:-----------|:------------------|:------------|
| tumor | Tumor_Finding | Present |
| ductal | Histological_Type | Present |
| carcinoma | Cancer_Dx | Present |
| metastases | Metastasis | Absent |
******************** assertion_oncology_problem_wip results ********************
| chunk | ner_label | assertion |
|:-----------|:------------------|:-----------------------|
| tumor | Tumor_Finding | Medical_History |
| ductal | Histological_Type | Medical_History |
| carcinoma | Cancer_Dx | Medical_History |
| metastases | Metastasis | Hypothetical_Or_Absent |
******************** re_oncology_wip results ********************
| chunk1 | entity1 | chunk2 | entity2 | relation |
|:---------|:--------------|:-----------|:--------------|:--------------|
| 4-cm | Tumor_Size | tumor | Tumor_Finding | is_related_to |
| 4-cm | Tumor_Size | carcinoma | Cancer_Dx | O |
| tumor | Tumor_Finding | breast | Site_Breast | is_related_to |
| breast | Site_Breast | carcinoma | Cancer_Dx | O |
| lung | Site_Lung | metastases | Metastasis | is_related_to |
******************** re_oncology_granular_wip results ********************
| chunk1 | entity1 | chunk2 | entity2 | relation |
|:---------|:--------------|:-----------|:--------------|:---------------|
| 4-cm | Tumor_Size | tumor | Tumor_Finding | is_size_of |
| 4-cm | Tumor_Size | carcinoma | Cancer_Dx | O |
| tumor | Tumor_Finding | breast | Site_Breast | is_location_of |
| breast | Site_Breast | carcinoma | Cancer_Dx | O |
| lung | Site_Lung | metastases | Metastasis | is_location_of |
******************** re_oncology_size_wip results ********************
| chunk1 | entity1 | chunk2 | entity2 | relation |
|:---------|:-----------|:----------|:--------------|:-----------|
| 4-cm | Tumor_Size | tumor | Tumor_Finding | is_size_of |
| 4-cm | Tumor_Size | carcinoma | Cancer_Dx | O |
******************** ICD-O resolver results ********************
| chunk | ner_label | code | normalized_term |
|:-----------|:------------------|:-------|:------------------|
| tumor | Tumor_Finding | 8000/1 | tumor |
| breast | Site_Breast | C50 | breast |
| ductal | Histological_Type | 8500/2 | dcis |
| carcinoma | Cancer_Dx | 8010/3 | carcinoma |
| lung | Site_Lung | C34.9 | lung |
| metastases | Metastasis | 8000/6 | tumor, metastatic |
Model Information
| Model Name: | oncology_diagnosis_pipeline |
| Type: | pipeline |
| Compatibility: | Healthcare NLP 4.4.4+ |
| License: | Licensed |
| Edition: | Official |
| Language: | en |
| Size: | 2.3 GB |
Included Models
- DocumentAssembler
- SentenceDetectorDLModel
- TokenizerModel
- WordEmbeddingsModel
- MedicalNerModel
- NerConverter
- MedicalNerModel
- NerConverter
- MedicalNerModel
- NerConverter
- ChunkMergeModel
- ChunkMergeModel
- AssertionDLModel
- AssertionDLModel
- PerceptronModel
- DependencyParserModel
- RelationExtractionModel
- RelationExtractionModel
- RelationExtractionModel
- ChunkMergeModel
- Chunk2Doc
- BertSentenceEmbeddings
- SentenceEntityResolverModel