Description
This model is a few-shot classification model designed to identify and classify the age group of a person mentioned in health documents. Age of the person may or may not be mentioned explicitly in the training dataset. Utilizing the few-shot learning approach, it can effectively learn from a small number of labeled examples, making it highly adaptable to new and unseen classes.
Predicted Entities
How to use
document_assembler = DocumentAssembler() \
.setInputCol("text") \
.setOutputCol("document")
large_few_shot_classifier = LargeFewShotClassifierModel()\
.pretrained('large_fewshot_classifier_age_group', "en", "clinical/models")\
.setInputCols("document")\
.setOutputCol("prediction")
pipeline = Pipeline().setStages([
document_assembler,
large_few_shot_classifier
])
text_list = [
("A patient presented with complaints of chest pain and shortness of breath. The medical history revealed the patient had a smoking habit for over 30 years, and was diagnosed with hypertension two years ago. After a detailed physical examination, the doctor found a noticeable wheeze on lung auscultation and prescribed a spirometry test, which showed irreversible airway obstruction. The patient was diagnosed with Chronic obstructive pulmonary disease (COPD) caused by smoking.",),
("My 4.5 year old has been poorly this week with a cold and thismorning he saw the doctors because he was up in the night last night saying his ear hurt and she said he has an ear infection.She said it's most likely viral but could be bacterial.Mostly they clear on their own.Watch and see about antibiotics.So we held off yestetday and today to see how be went and he perkes up today but then has gone downhill this evening saying his ear hurts still and he just feels poorly.",),
("Hi have chronic gastritis from 4 month(confirmed by endoscopy).I do not have acid reflux.Only dull ache above abdomen and left side of chest.I am on reberprozole and librax.My question is whether chronic gastritis is curable or is it a lifetime condition?I am loosing hope because this dull ache is not going away.Please please reply",)
]
data = spark.createDataFrame(text_list, ["text"])
result = pipeline.fit(data).transform(data)
result.select("text", col("prediction.result").getItem(0).alias("result")).show(truncate=False)
val documentAssembler = new DocumentAssembler()
.setInputCol("text")
.setOutputCol("document")
val largeFewShotClassifier = LargeFewShotClassifierModel()
.pretrained("large_fewshot_classifier_age_group", "en", "clinical/models")
.setInputCols("document")
.setOutputCol("prediction")
val pipeline = new Pipeline().setStages(Array(
documentAssembler,
largeFewShotClassifier
))
val textList = Seq(
("A patient presented with complaints of chest pain and shortness of breath. The medical history revealed the patient had a smoking habit for over 30 years, and was diagnosed with hypertension two years ago. After a detailed physical examination, the doctor found a noticeable wheeze on lung auscultation and prescribed a spirometry test, which showed irreversible airway obstruction. The patient was diagnosed with Chronic obstructive pulmonary disease (COPD) caused by smoking.",),
("My 4.5 year old has been poorly this week with a cold and thismorning he saw the doctors because he was up in the night last night saying his ear hurt and she said he has an ear infection.She said it's most likely viral but could be bacterial.Mostly they clear on their own.Watch and see about antibiotics.So we held off yestetday and today to see how be went and he perkes up today but then has gone downhill this evening saying his ear hurts still and he just feels poorly.",),
("Hi have chronic gastritis from 4 month(confirmed by endoscopy).I do not have acid reflux.Only dull ache above abdomen and left side of chest.I am on reberprozole and librax.My question is whether chronic gastritis is curable or is it a lifetime condition?I am loosing hope because this dull ache is not going away.Please please reply.",)
)
val data = spark.createDataFrame(textList).toDF("text")
val result = pipeline.fit(data).transform(data)
result.select(col("text"), col("prediction.result").getItem(0).alias("result")).show(truncate = false)
Results
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
|text |result |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
|A patient presented with complaints of chest pain and shortness of breath. The medical history revealed the patient had a smoking habit for over 30 years, and was diagnosed with hypertension two years ago. After a detailed physical examination, the doctor found a noticeable wheeze on lung auscultation and prescribed a spirometry test, which showed irreversible airway obstruction. The patient was diagnosed with Chronic obstructive pulmonary disease (COPD) caused by smoking.|Adult |
|My 4.5 year old has been poorly this week with a cold and thismorning he saw the doctors because he was up in the night last night saying his ear hurt and she said he has an ear infection.She said it's most likely viral but could be bacterial.Mostly they clear on their own.Watch and see about antibiotics.So we held off yestetday and today to see how be went and he perkes up today but then has gone downhill this evening saying his ear hurts still and he just feels poorly. |Child |
|Hi have chronic gastritis from 4 month(confirmed by endoscopy).I do not have acid reflux.Only dull ache above abdomen and left side of chest.I am on reberprozole and librax.My question is whether chronic gastritis is curable or is it a lifetime condition?I am loosing hope because this dull ache is not going away.Please please reply. |Unknown|
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
Model Information
| Model Name: | large_fewshot_classifier_age_group |
| Compatibility: | Healthcare NLP 5.4.0+ |
| License: | Licensed |
| Edition: | Official |
| Language: | en |
| Size: | 407.1 MB |
| Case sensitive: | false |
References
This model has been trained using internal datasets.
Benchmarking
label precision recall f1-score support
Adult 0.7764 0.8132 0.7943 380
Child 0.9364 0.8663 0.9000 187
Unknown 0.7679 0.7588 0.7633 340
accuracy - - 0.8037 907
macro-avg 0.8269 0.8128 0.8192 907
weighted-avg 0.8062 0.8037 0.8045 907