Description
Pretrained named entity recognition deep learning model for radiology related texts and reports.
Predicted Entities
ImagingTest, Imaging_Technique, ImagingFindings, OtherFindings, BodyPart, Direction, Test, Symptom, Disease_Syndrome_Disorder, Medical_Device, Procedure, Measurements, Units
How to use
Use as part of an NLP pipeline with the following stages: DocumentAssembler, SentenceDetector, Tokenizer, WordEmbeddingsModel, NerDLModel. Add the NerConverter to the end of the pipeline to convert entity tokens into full entity chunks.
...
radiology_ner = NerDLModel.pretrained("ner_radiology", "en", "clinical/models") \
.setInputCols(["sentence", "token", "embeddings"]) \
.setOutputCol("ner")
nlpPipeline = Pipeline(stages=[document_assembler, sentence_detector, tokenizer, word_embeddings, radiology_ner, ner_converter])
...
model = nlpPipeline.fit(spark.createDataFrame([['''Bilateral breast ultrasound was subsequently performed, which demonstrated an ovoid mass measuring approximately 0.5 x 0.5 x 0.4 cm in diameter located within the anteromedial aspect of the left shoulder. This mass demonstrates isoechoic echotexture to the adjacent muscle, with no evidence of internal color flow. This may represent benign fibrous tissue or a lipoma.''']]).toDF("text"))
results = model.transform(data)
...
val radiology_ner = NerDLModel().pretrained("ner_radiology", "en", "clinical/models")
.setInputCols(Array("sentence", "token", "embeddings"))
.setOutputCol("ner")
val nlpPipeline = new Pipeline().setStages(Array(document_assembler, sentence_detector, tokenizer, word_embeddings, radiology_ner, ner_converter))
val result = pipeline.fit(Seq.empty[String]).transform(data)
Results
| | chunks | entities |
|----|-----------------------|---------------------------|
| 0 | Bilateral | Direction |
| 1 | breast | BodyPart |
| 2 | ultrasound | ImagingTest |
| 3 | ovoid mass | ImagingFindings |
| 4 | 0.5 x 0.5 x 0.4 | Measurements |
| 5 | cm | Units |
| 6 | anteromedial aspect | Direction |
| 7 | left | Direction |
| 8 | shoulder | BodyPart |
| 9 | mass | ImagingFindings |
| 10 | isoechoic echotexture | ImagingFindings |
| 11 | muscle | BodyPart |
| 12 | internal color flow | ImagingFindings |
| 13 | benign fibrous tissue | ImagingFindings |
| 14 | lipoma | Disease_Syndrome_Disorder |
Model Information
| Model Name: | ner_radiology |
| Type: | ner |
| Compatibility: | Spark NLP 2.7.0+ |
| License: | Licensed |
| Edition: | Official |
| Input Labels: | [sentence, token, embeddings] |
| Output Labels: | [ner] |
| Language: | en |
| Dependencies: | embeddings_clinical |
Data Source
Trained on a custom dataset comprising of MIMIC-CXR and MT Radiology texts
Benchmarking
+--------------------+------+-----+-----+------+---------+------+------+
| entity| tp| fp| fn| total|precision|recall| f1|
+--------------------+------+-----+-----+------+---------+------+------+
| OtherFindings| 8.0| 15.0| 63.0| 71.0| 0.3478|0.1127|0.1702|
| Measurements| 481.0| 30.0| 15.0| 496.0| 0.9413|0.9698|0.9553|
| Direction| 650.0|137.0| 94.0| 744.0| 0.8259|0.8737|0.8491|
| ImagingFindings|1345.0|355.0|324.0|1669.0| 0.7912|0.8059|0.7985|
| BodyPart|1942.0|335.0|290.0|2232.0| 0.8529|0.8701|0.8614|
| Medical_Device| 236.0| 75.0| 64.0| 300.0| 0.7588|0.7867|0.7725|
| Test| 222.0| 41.0| 48.0| 270.0| 0.8441|0.8222| 0.833|
| Procedure| 269.0|117.0|116.0| 385.0| 0.6969|0.6987|0.6978|
| ImagingTest| 263.0| 50.0| 43.0| 306.0| 0.8403|0.8595|0.8498|
| Symptom| 498.0|101.0|132.0| 630.0| 0.8314|0.7905|0.8104|
|Disease_Syndrome_...|1180.0|258.0|200.0|1380.0| 0.8206|0.8551|0.8375|
| Units| 269.0| 10.0| 2.0| 271.0| 0.9642|0.9926|0.9782|
| Imaging_Technique| 140.0| 38.0| 25.0| 165.0| 0.7865|0.8485|0.8163|
+--------------------+------+-----+-----+------+---------+------+------+
+------------------+
| macro|
+------------------+
|0.7524248724038437|
+------------------+
+------------------+
| micro|
+------------------+
|0.8315240382681794|
+------------------+