Description
Detect Drug, Dosage and administration instructions in text using pretraiend NER model.
Predicted Entities
FREQUENCY, DRUG, STRENGTH, FORM, DURATION, DOSAGE, ROUTE
Live Demo Open in Colab Download
How to use
...
embeddings_clinical = BertEmbeddings.pretrained("biobert_pubmed_base_cased").setInputCols(["sentence", "token"]).setOutputCol("embeddings")
clinical_ner = MedicalNerModel.pretrained("ner_posology_biobert", "en", "clinical/models").setInputCols(["sentence", "token", "embeddings"]).setOutputCol("ner")
...
nlpPipeline = Pipeline(stages=[document_assembler, sentence_detector, tokenizer, embeddings_clinical, clinical_ner, ner_converter])
model = nlpPipeline.fit(spark.createDataFrame([[""]]).toDF("text"))
results = model.transform(spark.createDataFrame([["EXAMPLE_TEXT"]]).toDF("text"))
...
val embeddings_clinical = BertEmbeddings.pretrained("biobert_pubmed_base_cased")
.setInputCols(Array("sentence", "token"))
.setOutputCol("embeddings")
val ner = MedicalNerModel.pretrained("ner_posology_biobert", "en", "clinical/models")
.setInputCols(Array("sentence", "token", "embeddings"))
.setOutputCol("ner")
...
val pipeline = new Pipeline().setStages(Array(document_assembler, sentence_detector, tokenizer, embeddings_clinical, ner, ner_converter))
val result = pipeline.fit(Seq.empty[String]).transform(data)
Model Information
| Model Name: | ner_posology_biobert |
| Compatibility: | Spark NLP for Healthcare 3.0.0+ |
| License: | Licensed |
| Edition: | Official |
| Input Labels: | [sentence, token, embeddings] |
| Output Labels: | [ner] |
| Language: | en |
Benchmarking
label precision recall f1-score support
B-DOSAGE 0.78 0.67 0.72 559
B-DRUG 0.93 0.94 0.94 3865
B-DURATION 0.79 0.81 0.80 331
B-FORM 0.90 0.87 0.88 1472
B-FREQUENCY 0.92 0.94 0.93 1577
B-ROUTE 0.94 0.85 0.89 772
B-STRENGTH 0.88 0.92 0.90 2519
I-DOSAGE 0.62 0.57 0.60 357
I-DRUG 0.81 0.89 0.85 1539
I-DURATION 0.80 0.89 0.84 796
I-FORM 0.58 0.54 0.56 142
I-FREQUENCY 0.86 0.93 0.89 2424
I-ROUTE 1.00 0.47 0.64 32
I-STRENGTH 0.85 0.91 0.88 2972
O 0.98 0.98 0.98 101134
accuracy - - 0.97 120491
macro-avg 0.84 0.81 0.82 120491
weighted-avg 0.97 0.97 0.97 120491