Description
This model is trained on a list of clinical and biomedical datasets curated in-house
How to use
document_assembler = DocumentAssembler()\
.setInputCol("text")\
.setOutputCol("document")
mpnet_embedding = MPNetEmbeddings.pretrained("mpnet_embeddings_medical_assertion_i2b2", "en", "clinical/models")\
.setInputCols(["document"])\
.setOutputCol("mpnet_embeddings")
pipeline = Pipeline().setStages([document_assembler, mpnet_embedding])
text = [
["I feel a bit drowsy after taking an insulin."],
["Peter Parker is a nice lad and lives in New York"]
]
data = spark.createDataFrame(text).toDF("text")
result = pipeline.fit(data).transform(data)
val document_assembler = new DocumentAssembler()
.setInputCol("text")
.setOutputCol("document")
val mpnet_embedding = MPNetEmbeddings.pretrained("mpnet_embeddings_medical_assertion_i2b2", "en", "clinical/models")
.setInputCols(Array("document"))
.setOutputCol("mpnet_embeddings")
val pipeline = new Pipeline().setStages(Array(document_assembler, mpnet_embedding))
val result = pipeline.fit(data).transform(data)
Results
| | assertion_embedding |
|---:|:-------------------------------|
| 0 | [Row(annotatorType='sentence_embeddings', begin=0, end=43, result='I feel a bit drowsy after taking an insulin.', metadata={'sentence': '0'}, embeddings=[-0.02157330885529518, -0.05100712180137634, 0.043191660195589066, 0.035359036177396774, -0.04416131228208542, 0.036355987191200256, -0...])] |
| 1 | [Row(annotatorType='sentence_embeddings', begin=0, end=47, result='Peter Parker is a nice lad and lives in New York', metadata={'sentence': '0'}, embeddings=[-0.07660277187824249, -0.01287313923239708, 0.015349301509559155, 0.008208038285374641, 0.015206931158900261, -0.0321115218102932,...])] |
Model Information
| Model Name: | mpnet_embeddings_medical_assertion_i2b2 |
| Compatibility: | Healthcare NLP 5.3.2+ |
| License: | Licensed |
| Edition: | Official |
| Input Labels: | [document] |
| Output Labels: | [assertion_embedding] |
| Language: | en |
| Size: | 407.0 MB |
| Case sensitive: | false |