Description
This model detects mentions of genes and human phenotypes (hp) in medical text.
Predicted Entities
: GENE, HP
Live Demo Open in Colab Download
How to use
...
word_embeddings = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")\
.setInputCols(["sentence", "token"])\
.setOutputCol("embeddings")
clinical_ner = MedicalNerModel.pretrained("ner_human_phenotype_gene_clinical", "en", "clinical/models") \
.setInputCols(["sentence", "token", "embeddings"]) \
.setOutputCol("ner")
...
nlp_pipeline = Pipeline(stages=[document_assembler, sentence_detector, tokenizer, word_embeddings, clinical_ner, ner_converter])
light_pipeline = LightPipeline(nlp_pipeline.fit(spark.createDataFrame([['']]).toDF("text")))
annotations = light_pipeline.fullAnnotate("Here we presented a case (BS type) of a 17 years old female presented with polyhydramnios, polyuria, nephrocalcinosis and hypokalemia, which was alleviated after treatment with celecoxib and vitamin D(3).")
...
val word_embeddings = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")
.setInputCols(Array("sentence", "token"))
.setOutputCol("embeddings")
val ner = MedicalNerModel.pretrained("ner_human_phenotype_gene_clinical", "en", "clinical/models")
.setInputCols("sentence", "token", "embeddings")
.setOutputCol("ner")
...
val pipeline = new Pipeline().setStages(Array(document_assembler, sentence_detector, tokenizer, word_embeddings, ner, ner_converter))
val result = pipeline.fit(Seq.empty["Here we presented a case (BS type) of a 17 years old female presented with polyhydramnios, polyuria, nephrocalcinosis and hypokalemia, which was alleviated after treatment with celecoxib and vitamin D(3)."].toDS.toDF("text")).transform(data)
Results
+----+------------------+---------+-------+----------+
| | chunk | begin | end | entity |
+====+==================+=========+=======+==========+
| 0 | BS type | 29 | 32 | GENE |
+----+------------------+---------+-------+----------+
| 1 | polyhydramnios | 75 | 88 | HP |
+----+------------------+---------+-------+----------+
| 2 | polyuria | 91 | 98 | HP |
+----+------------------+---------+-------+----------+
| 3 | nephrocalcinosis | 101 | 116 | HP |
+----+------------------+---------+-------+----------+
| 4 | hypokalemia | 122 | 132 | HP |
+----+------------------+---------+-------+----------+
Model Information
| Model Name: | ner_human_phenotype_gene_clinical |
| Compatibility: | Spark NLP for Healthcare 3.0.0+ |
| License: | Licensed |
| Edition: | Official |
| Input Labels: | [sentence, token, embeddings] |
| Output Labels: | [ner] |
| Language: | en |
Benchmarking
| | label | tp | fp | fn | prec | rec | f1 |
|---:|--------------:|------:|-----:|-----:|---------:|---------:|---------:|
| 0 | I-HP | 303 | 56 | 64 | 0.844011 | 0.825613 | 0.834711 |
| 1 | B-GENE | 1176 | 158 | 252 | 0.881559 | 0.823529 | 0.851557 |
| 2 | B-HP | 1078 | 133 | 96 | 0.890173 | 0.918228 | 0.903983 |
| 3 | Macro-average | 2557 | 347 | 412 | 0.871915 | 0.85579 | 0.863777 |
| 4 | Micro-average | 2557 | 347 | 412 | 0.88051 | 0.861233 | 0.870765 |