Edit on GitHub
  • John Snow Labs
  • Nov 16, 2022

Mapping Abbreviations and Acronyms of Medical Regulatory Activities with Their Definitions and Categories

  • abbreviation
  • definition
  • category
  • licensed
  • en
  • clinical
  • chunk_mapper

Description

This pretrained model maps abbreviations and acronyms of medical regulatory activities with their definitions and categories. Predicted categories: general, problem, test, treatment, medical_condition, clinical_dept, drug, nursing, internal_organ_or_component, hospital_unit, drug_frequency, employment, procedure.

Important Note: Mappers extract additional information such as extended descriptions and categories related to Concept codes (such as RxNorm, ICD10, CPT, MESH, NDC, UMLS, etc.). They generally take Concept Codes, which are the outputs of EntityResolvers, as input. When creating a pipeline that contains ‘Mapper’, it is necessary to use the ChunkMapperModel after an EntityResolverModel.

Predicted Entities

definition, category

Open in Colab Copy S3 URI

How to use

document_assembler = DocumentAssembler()\
    .setInputCol("text")\
    .setOutputCol("document")

sentence_detector = SentenceDetector()\
    .setInputCols(["document"])\
    .setOutputCol("sentence")

tokenizer = Tokenizer()\
    .setInputCols("sentence")\
    .setOutputCol("token")

word_embeddings = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")\
    .setInputCols(["sentence", "token"])\
    .setOutputCol("embeddings")

abbr_ner = MedicalNerModel.pretrained("ner_abbreviation_clinical", "en", "clinical/models") \
    .setInputCols(["sentence", "token", "embeddings"]) \
    .setOutputCol("abbr_ner")

abbr_converter = NerConverter() \
    .setInputCols(["sentence", "token", "abbr_ner"]) \
    .setOutputCol("abbr_ner_chunk")\

chunkerMapper = ChunkMapperModel.pretrained("abbreviation_category_mapper", "en", "clinical/models")\
    .setInputCols(["abbr_ner_chunk"])\
    .setOutputCol("mappings")\
    .setRels(["definition", "category"])\

pipeline = Pipeline().setStages([
    document_assembler,
    sentence_detector,
    tokenizer, 
    word_embeddings,
    abbr_ner, 
    abbr_converter, 
    chunkerMapper])


text = ["""Gravid with estimated fetal weight of 6-6/12 pounds.
           LABORATORY DATA: Laboratory tests include a CBC which is normal. 
           VDRL: Nonreactive
           HIV: Negative. One-Hour Glucose: 117. Group B strep has not been done as yet."""]

data = spark.createDataFrame([text]).toDF("text")

result = pipeline.fit(data).transform(data)

val document_assembler = new DocumentAssembler()
    .setInputCol("text")
    .setOutputCol("document")

val sentence_detector = new SentenceDetector()
    .setInputCols(Array("document"))
    .setOutputCol("sentence")

val tokenizer = new Tokenizer()
    .setInputCols("sentence")
    .setOutputCol("token")

val word_embeddings = WordEmbeddingsModel.pretrained("embeddings_clinical", "en", "clinical/models")
    .setInputCols(Array("sentence", "token"))
    .setOutputCol("embeddings")

val abbr_ner = MedicalNerModel.pretrained("ner_abbreviation_clinical", "en", "clinical/models") 
    .setInputCols(Array("sentence", "token", "embeddings")) 
    .setOutputCol("abbr_ner")

val abbr_converter = new NerConverter() 
    .setInputCols(Array("sentence", "token", "abbr_ner")) 
    .setOutputCol("abbr_ner_chunk")

val chunkerMapper = ChunkMapperModel.pretrained("abbreviation_category_mapper", "en", "clinical/models")
    .setInputCols("abbr_ner_chunk")
    .setOutputCol("mappings")
    .setRels(Array("definition", "category"))


val pipeline = new Pipeline().setStages(Array(
    document_assembler,
    sentence_detector,
    tokenizer, 
    word_embeddings,
    abbr_ner, 
    abbr_converter, 
    chunkerMapper))


val sample_text = """Gravid with estimated fetal weight of 6-6/12 pounds.
           LABORATORY DATA: Laboratory tests include a CBC which is normal. 
           VDRL: Nonreactive
           HIV: Negative. One-Hour Glucose: 117. Group B strep has not been done as yet.""" 


val data = Seq(sample_text).toDS.toDF("text")

val result= pipeline.fit(data).transform(data)

import nlu
nlu.load("en.map_entity.abbreviation_category").predict("""Gravid with estimated fetal weight of 6-6/12 pounds.
           LABORATORY DATA: Laboratory tests include a CBC which is normal. 
           VDRL: Nonreactive
           HIV: Negative. One-Hour Glucose: 117. Group B strep has not been done as yet.""")

Results

|    | chunk   | category          | definition                             |
|---:|:--------|:------------------|:---------------------------------------|
|  0 | CBC     | general           | complete blood count                   |
|  1 | VDRL    | clinical_dept     | Venereal Disease Research Laboratories |
|  2 | HIV     | medical_condition | Human immunodeficiency virus           |

Model Information

Model Name: abbreviation_category_mapper
Compatibility: Healthcare NLP 4.2.1+
License: Licensed
Edition: Official
Input Labels: [abbr_ner_chunk]
Output Labels: [mappings]
Language: en
Size: 128.2 KB