Detect entities in radiology reports (ner_clinical_icdem)

Description

Extract entities related to pneumonia in radiology reports

Predicted Entities

Size, Positive1, Tests, Procedure1, Type, Extension, Bronchial1, Vascular1, Examined, Vascular, Laterality1, Resuts, Grade, Examined1, Laterality, Size1, pM, pN1, Parenchymal, Results, Resuts1, Localization, Tests1, "", OtherMargin, DcisMargin, Margins, Margins1, pT1, Diagnosis3, Localization1, Parenchymal1, Diagnosis10, Size2, Nuclear, Bronchial, Grade1, Procedure, Focality, Nuclear1, Localization2, pT, Results1, Positive, Type1, pN

Live Demo Open in Colab Download

How to use


...
embeddings_clinical = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")  .setInputCols(["sentence", "token"])  .setOutputCol("embeddings")
clinical_ner = MedicalNerModel.pretrained("ner_clinical_icdem", "en", "clinical/models")   .setInputCols(["sentence", "token", "embeddings"])   .setOutputCol("ner")
...
nlpPipeline = Pipeline(stages=[document_assembler, sentence_detector, tokenizer, embeddings_clinical, clinical_ner, ner_converter])
model = nlpPipeline.fit(spark.createDataFrame([[""]]).toDF("text"))
results = model.transform(spark.createDataFrame([["EXAMPLE_TEXT"]]).toDF("text"))

...
val embeddings_clinical = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")
  .setInputCols(Array("sentence", "token"))
  .setOutputCol("embeddings")
val ner = MedicalNerModel.pretrained("ner_clinical_icdem", "en", "clinical/models")
  .setInputCols(Array("sentence", "token", "embeddings"))
  .setOutputCol("ner")
...
val pipeline = new Pipeline().setStages(Array(document_assembler, sentence_detector, tokenizer, embeddings_clinical, ner, ner_converter))
val result = pipeline.fit(Seq.empty[""].toDS.toDF("text")).transform(data)

Model Information

Model Name: ner_clinical_icdem
Compatibility: Spark NLP for Healthcare 3.0.0+
License: Licensed
Edition: Official
Input Labels: [sentence, token, word_vecs]
Output Labels: [ner]
Language: en