Description
This model maps clinical entities and concepts to ICD10 CM codes using sentence bert embeddings. In this model, synonyms having low cosine similarity to unnormalized terms are dropped. It also returns the official resolution text within the brackets inside the metadata. The model is augmented with synonyms, and previous augmentations are flexed according to cosine distances to unnormalized terms (ground truths).
Predicted Entities
Outputs 7-digit billable ICD codes. In the result, look for aux_label parameter in the metadata to get HCC status. The HCC status can be divided to get further information: billable status, hcc status, and hcc score.For example, in the example shared below the billable status is 1, hcc status is 1, and hcc score is 11.
How to use
documentAssembler = DocumentAssembler()\
.setInputCol("text")\
.setOutputCol("ner_chunk")
sbert_embedder = BertSentenceEmbeddings\
.pretrained('sbert_jsl_medium_uncased', 'en','clinical/models')\
.setInputCols(["ner_chunk"])\
.setOutputCol("sbert_embeddings")
icd10_resolver = SentenceEntityResolverModel.pretrained("sbertresolve_icd10cm_slim_billable_hcc_med","en", "clinical/models") \
.setInputCols(["sbert_embeddings"]) \
.setOutputCol("icd10cm_code")\
.setDistanceFunction("EUCLIDEAN").setReturnCosineDistances(True)
bert_pipeline_icd = PipelineModel(stages = [document_assembler, sbert_embedder, icd10_resolver])
model = bert_pipeline_icd.fit(spark.createDataFrame([["bladder cancer"]]).toDF("text"))
results = model.transform(data)
val document_assembler = DocumentAssembler()
.setInputCol("text")
.setOutputCol("document")
val sbert_embedder = BertSentenceEmbeddings
.pretrained("sbert_jsl_medium_uncased","en","clinical/models")
.setInputCols("document")
.setOutputCol("sbert_embeddings")
val icd10_resolver = SentenceEntityResolverModel.pretrained("sbertresolve_icd10cm_slim_billable_hcc_med","en", "clinical/models")
.setInputCols(["sbert_embeddings"])
.setOutputCol("icd10cm_code")
.setDistanceFunction("EUCLIDEAN")
.setReturnCosineDistances(True)
val bert_pipeline_icd = new PipelineModel().setStages(Array(document_assembler, sbert_embedder, icd10_resolver))
val data = Seq("bladder cancer").toDS.toDF("text")
val result = bert_pipeline_icd.fit(data).transform(data)
import nlu
nlu.load("en.resolve.icd10cm.slim_billable_hcc_med").predict("""bladder cancer""")
Results
| | chunks | code | resolutions | all_codes | billable_hcc_status_score | all_distances |
|---:|:---------------|:--------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|--------------------------------------------------------------------------------------:|:----------------------------|:-----------------------------------------------------------------------------------------------------------------|
| 0 | bladder cancer | C671 |[bladder cancer, dome [Malignant neoplasm of dome of bladder], cancer of the urinary bladder [Malignant neoplasm of bladder, unspecified], prostate cancer [Malignant neoplasm of prostate], cancer of the urinary bladder, lateral wall [Malignant neoplasm of lateral wall of bladder], cancer of the urinary bladder, anterior wall [Malignant neoplasm of anterior wall of bladder], cancer of the urinary bladder, posterior wall [Malignant neoplasm of posterior wall of bladder], cancer of the urinary bladder, neck [Malignant neoplasm of bladder neck], cancer of the urinary bladder, ureteric orifice [Malignant neoplasm of ureteric orifice]]| [C671, C679, C61, C672, C673, C674, C675, C676, D090, Z126, D494, C670, Z8551, C7911] | ['1', '1', '11'] | [0.0894, 0.1051, 0.1184, 0.1180, 0.1200, 0.1204, 0.1255, 0.1375, 0.1357, 0.1452, 0.1469, 0.1513, 0.1500, 0.1575] |
Model Information
| Model Name: | sbertresolve_icd10cm_slim_billable_hcc_med |
| Compatibility: | Healthcare NLP 3.0.3+ |
| License: | Licensed |
| Edition: | Official |
| Input Labels: | [ner_chunk, sbert_embeddings] |
| Output Labels: | [icd10_code] |
| Language: | en |
| Case sensitive: | false |