ChunkResolver Loinc Clinical

Description

Entity Resolution model Based on KNN using Word Embeddings + Word Movers Distance.

Predicted Entities

LOINC Codes with clinical_embeddings.

Open in Colab Download

How to use

...    
loinc_resolver = ChunkEntityResolverModel.pretrained("chunkresolve_loinc_clinical", "en", "clinical/models") \
  .setInputCols(["token", "chunk_embeddings"]) \
  .setOutputCol("loinc_code") \
  .setDistanceFunction("COSINE") \
  .setNeighbours(5)
  
pipeline_loinc = Pipeline(stages = [documentAssembler, sentenceDetector, tokenizer, stopwords, word_embeddings, clinical_ner, ner_converter, chunk_embeddings, loinc_resolver])

model = pipeline_rxnorm.fit(spark.createDataFrame([["""A 28-year-old female with a history of gestational diabetes mellitus diagnosed eight years prior to presentation and subsequent type two diabetes mellitus (T2DM), one prior episode of HTG-induced pancreatitis three years prior to presentation, associated with an acute hepatitis, and obesity with a body mass index (BMI) of 33.5 kg/m2, presented with a one-week history of polyuria, polydipsia, poor appetite, and vomiting."""]]).toDF("text"))

results = model.transform(data)
...
val loinc_resolver = ChunkEntityResolverModel.pretrained("chunkresolve_loinc_clinical", "en", "clinical/models")
  .setInputCols(Array("token", "chunk_embeddings"))
  .setOutputCol("loinc_code")
  .setDistanceFunction("COSINE")
  .setNeighbours(5)
  
val pipeline = new Pipeline().setStages(Array(documentAssembler, sentenceDetector, tokenizer, stopwords, word_embeddings, clinical_ner, ner_converter, chunk_embeddings, loinc_resolver))

val result = pipeline.fit(Seq.empty["A 28-year-old female with a history of gestational diabetes mellitus diagnosed eight years prior to presentation and subsequent type two diabetes mellitus (T2DM), one prior episode of HTG-induced pancreatitis three years prior to presentation, associated with an acute hepatitis, and obesity with a body mass index (BMI) of 33.5 kg/m2, presented with a one-week history of polyuria, polydipsia, poor appetite, and vomiting."].toDS.toDF("text")).transform(data)

Results

                                      Chunk  loinc-Code

0             gestational diabetes mellitus  44877-9
1                type two diabetes mellitus  44877-9
2                                      T2DM  93692-2
3 prior episode of HTG-induced pancreatitis  85695-5
4        associated with an acute hepatitis  24363-4
5            obesity with a body mass index  47278-7
6                        BMI) of 33.5 kg/m2  47214-2
7                                  polyuria  35234-4
8                                polydipsia  25541-4
9                             poor appetite  50056-1
10                                 vomiting  34175-0

Model Information

Name: chunkresolve_loinc_clinical  
Type: ChunkEntityResolverModel  
Compatibility: Spark NLP 2.5.0+  
License: Licensed  
Edition: Official  
Input labels: [token, chunk_embeddings]  
Output labels: [entity]  
Language: en  
Case sensitive: True  
Dependencies: embeddings_clinical  

Data Source

Trained on LOINC dataset with embeddings_clinical. https://loinc.org/