Detect clinical concepts (jsl_ner_wip_modifier_clinical)

Description

Detect modifiers and other clinical entities using pretrained NER model.

Predicted Entities

Kidney_Disease, Height, Family_History_Header, RelativeTime, Hypertension, HDL, Alcohol, Test, Substance, Fetus_NewBorn, Diet, Substance_Quantity, Allergen, Form, Birth_Entity, Age, Race_Ethnicity, Modifier, Internal_organ_or_component, Hyperlipidemia, ImagingFindings, Psychological_Condition, Triglycerides, Cerebrovascular_Disease, Obesity, Duration, Weight, Date, Test_Result, Strength, VS_Finding, Respiration, Social_History_Header, Employment, Injury_or_Poisoning, Medical_History_Header, Death_Entity, Relationship_Status, Oxygen_Therapy, Blood_Pressure, Gender, Section_Header, Oncological, Drug, Labour_Delivery, Heart_Disease, LDL, Medical_Device, Temperature, Treatment, Female_Reproductive_Status, Total_Cholesterol, Time, Disease_Syndrome_Disorder, Communicable_Disease, EKG_Findings, Diabetes, Route, External_body_part_or_region, Pulse, Vital_Signs_Header, Direction, Admission_Discharge, Overweight, RelativeDate, O2_Saturation, BMI, Vaccine, Pregnancy, Sexually_Active_or_Sexual_Orientation, Procedure, Frequency, Dosage, Symptom, Clinical_Dept, Smoking

Live Demo Open in Colab Download

How to use


...
embeddings_clinical = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")  .setInputCols(["sentence", "token"])  .setOutputCol("embeddings")
clinical_ner = MedicalNerModel.pretrained("jsl_ner_wip_modifier_clinical", "en", "clinical/models")   .setInputCols(["sentence", "token", "embeddings"])   .setOutputCol("ner")
...
nlpPipeline = Pipeline(stages=[document_assembler, sentence_detector, tokenizer, embeddings_clinical, clinical_ner, ner_converter])
model = nlpPipeline.fit(spark.createDataFrame([[""]]).toDF("text"))
results = model.transform(spark.createDataFrame([["EXAMPLE_TEXT"]]).toDF("text"))

...
val embeddings_clinical = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")
  .setInputCols(Array("sentence", "token"))
  .setOutputCol("embeddings")
val ner = MedicalNerModel.pretrained("jsl_ner_wip_modifier_clinical", "en", "clinical/models")
  .setInputCols(Array("sentence", "token", "embeddings"))
  .setOutputCol("ner")
...
val pipeline = new Pipeline().setStages(Array(document_assembler, sentence_detector, tokenizer, embeddings_clinical, ner, ner_converter))
val result = pipeline.fit(Seq.empty[""].toDS.toDF("text")).transform(data)

Model Information

Model Name: jsl_ner_wip_modifier_clinical
Compatibility: Spark NLP for Healthcare 3.0.0+
License: Licensed
Edition: Official
Input Labels: [sentence, token, embeddings]
Output Labels: [ner]
Language: en