Description
This model detects chemical compounds/drugs and genes/proteins in medical text and research articles. Chemical compounds/drugs are labeled as CHEMICAL, genes/proteins are labeled as GENE and entity mentions of type GENE and of type CHEMICAL that overlap such as enzymes and small peptides are labeled as GENE_AND_CHEMICAL.
Predicted Entities
GENE, CHEMICAL, GENE_AND_CHEMICAL
Live Demo Open in Colab Copy S3 URI
How to use
...
document_assembler = DocumentAssembler()\
.setInputCol("text")\
.setOutputCol("document")
sentence_detector = SentenceDetector()\
.setInputCols(["document"])\
.setOutputCol("sentences")
tokenizer = Tokenizer()\
.setInputCols(["sentences"])\
.setOutputCol("tokens")
embeddings_clinical = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")\
.setInputCols(["sentences", "tokens"])\
.setOutputCol("embeddings")
clinical_ner = MedicalNerModel.pretrained("ner_drugprot_clinical", "en", "clinical/models")\
.setInputCols(["sentences", "tokens", "embeddings"])\
.setOutputCol("ner")
ner_converter = NerConverter()\
.setInputCols(["sentences", "tokens", "ner"])\
.setOutputCol("ner_chunks")
nlpPipeline = Pipeline(stages=[document_assembler, sentence_detector, tokenizer, embeddings_clinical, clinical_ner, ner_converter])
EXAMPLE_TEXT = "Anabolic effects of clenbuterol on skeletal muscle are mediated by beta 2-adrenoceptor activation."
data = spark.createDataFrame([[EXAMPLE_TEXT]]).toDF("text")
results = nlpPipeline.fit(data).transform(data)
...
val document_assembler = new DocumentAssembler()
.setInputCol("text")
.setOutputCol("document")
val sentence_detector = new SentenceDetector()
.setInputCols(Array("document"))
.setOutputCol("sentences")
val tokenizer = new Tokenizer()
.setInputCols(Array("sentences"))
.setOutputCol("tokens")
val embeddings_clinical = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")
.setInputCols(Array("sentences", "tokens"))
.setOutputCol("embeddings")
val clinical_ner = MedicalNerModel.pretrained("ner_drugprot_clinical", "en", "clinical/models")
.setInputCols(Array("sentences", "tokens", "embeddings"))
.setOutputCol("ner")
val ner_converter = new NerConverter()
.setInputCols(Array("sentences", "tokens", "ner"))
.setOutputCol("ner_chunks")
val pipeline = new Pipeline().setStages(Array(document_assembler, sentence_detector, tokenizer, embeddings_clinical, clinical_ner, ner_converter))
val data = Seq("""Anabolic effects of clenbuterol on skeletal muscle are mediated by beta 2-adrenoceptor activation.""").toDS.toDF("text")
val result = pipeline.fit(data).transform(data)
import nlu
nlu.load("en.med_ner.drugprot_clinical").predict("""Anabolic effects of clenbuterol on skeletal muscle are mediated by beta 2-adrenoceptor activation.""")
Results
+-------------------------------+---------+
|chunk |ner_label|
+-------------------------------+---------+
|clenbuterol |CHEMICAL |
|beta 2-adrenoceptor |GENE |
+-------------------------------+---------+
Model Information
| Model Name: | ner_drugprot_clinical |
| Compatibility: | Healthcare NLP 3.3.4+ |
| License: | Licensed |
| Edition: | Official |
| Input Labels: | [sentence, token, embeddings] |
| Output Labels: | [ner] |
| Language: | en |
| Size: | 14.7 MB |
| Dependencies: | embeddings_clinical |
Data Source
This model was trained on the DrugProt corpus.
Benchmarking
label tp fp fn total precision recall f1
GENE_AND_CHEMICAL 786.0 171.0 143.0 929.0 0.8213 0.8461 0.8335
CHEMICAL 8228.0 779.0 575.0 8803.0 0.9135 0.9347 0.924
GENE 7176.0 822.0 652.0 7828.0 0.8972 0.9167 0.9069
macro - - - - - - 0.88811683
micro - - - - - - 0.91156048