Description
Detect Drug, Dosage and administration instructions in text using pretraiend NER model.
Predicted Entities
FREQUENCY
, DRUG
, STRENGTH
, FORM
, DURATION
, DOSAGE
, ROUTE
Live Demo Open in Colab Download
How to use
...
embeddings_clinical = BertEmbeddings.pretrained("biobert_pubmed_base_cased").setInputCols(["sentence", "token"]).setOutputCol("embeddings")
clinical_ner = MedicalNerModel.pretrained("ner_posology_biobert", "en", "clinical/models").setInputCols(["sentence", "token", "embeddings"]).setOutputCol("ner")
...
nlpPipeline = Pipeline(stages=[document_assembler, sentence_detector, tokenizer, embeddings_clinical, clinical_ner, ner_converter])
model = nlpPipeline.fit(spark.createDataFrame([[""]]).toDF("text"))
results = model.transform(spark.createDataFrame([["EXAMPLE_TEXT"]]).toDF("text"))
...
val embeddings_clinical = BertEmbeddings.pretrained("biobert_pubmed_base_cased")
.setInputCols(Array("sentence", "token"))
.setOutputCol("embeddings")
val ner = MedicalNerModel.pretrained("ner_posology_biobert", "en", "clinical/models")
.setInputCols(Array("sentence", "token", "embeddings"))
.setOutputCol("ner")
...
val pipeline = new Pipeline().setStages(Array(document_assembler, sentence_detector, tokenizer, embeddings_clinical, ner, ner_converter))
val result = pipeline.fit(Seq.empty[String]).transform(data)
Model Information
Model Name: | ner_posology_biobert |
Compatibility: | Spark NLP for Healthcare 3.0.0+ |
License: | Licensed |
Edition: | Official |
Input Labels: | [sentence, token, embeddings] |
Output Labels: | [ner] |
Language: | en |
Benchmarking
label precision recall f1-score support
B-DOSAGE 0.78 0.67 0.72 559
B-DRUG 0.93 0.94 0.94 3865
B-DURATION 0.79 0.81 0.80 331
B-FORM 0.90 0.87 0.88 1472
B-FREQUENCY 0.92 0.94 0.93 1577
B-ROUTE 0.94 0.85 0.89 772
B-STRENGTH 0.88 0.92 0.90 2519
I-DOSAGE 0.62 0.57 0.60 357
I-DRUG 0.81 0.89 0.85 1539
I-DURATION 0.80 0.89 0.84 796
I-FORM 0.58 0.54 0.56 142
I-FREQUENCY 0.86 0.93 0.89 2424
I-ROUTE 1.00 0.47 0.64 32
I-STRENGTH 0.85 0.91 0.88 2972
O 0.98 0.98 0.98 101134
accuracy - - 0.97 120491
macro-avg 0.84 0.81 0.82 120491
weighted-avg 0.97 0.97 0.97 120491