Description
This model is trained on a list of clinical and biomedical datasets curated in-house
How to use
document_assembler = DocumentAssembler()\
.setInputCol("text")\
.setOutputCol("document")
mpnet_embedding = MPNetEmbeddings.pretrained("mpnet_embeddings_medical_assertion_sdoh", "en", "clinical/models")\
.setInputCols(["document"])\
.setOutputCol("mpnet_embeddings")
pipeline = Pipeline().setStages([document_assembler, mpnet_embedding])
text = [
["I feel a bit drowsy after taking an insulin."],
["Peter Parker is a nice lad and lives in New York"]
]
data = spark.createDataFrame(text).toDF("text")
result = pipeline.fit(data).transform(data)
val document_assembler = new DocumentAssembler()
.setInputCol("text")
.setOutputCol("document")
val mpnet_embedding = MPNetEmbeddings.pretrained("mpnet_embeddings_medical_assertion_sdoh", "en", "clinical/models")
.setInputCols(Array("document"))
.setOutputCol("mpnet_embeddings")
val pipeline = new Pipeline().setStages(Array(document_assembler, mpnet_embedding))
val result = pipeline.fit(data).transform(data)
Results
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| assertion_embedding|
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|[{sentence_embeddings, 0, 43, I feel a bit drowsy after taking an insulin., {sentence -> 0}, [-0.09830807, 0.0137982415, -0.051585164, -0.0023749713, -0.017916167, 0.017543513, 0.025593378, 0.05106...|
|[{sentence_embeddings, 0, 47, Peter Parker is a nice lad and lives in New York, {sentence -> 0}, [-0.10453681, 0.010062916, -0.024983741, 0.009945293, -0.01242009, 0.018787898, 0.039723188, 0.04624...|
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Model Information
| Model Name: | mpnet_embeddings_medical_assertion_sdoh |
| Compatibility: | Healthcare NLP 5.3.2+ |
| License: | Licensed |
| Edition: | Official |
| Input Labels: | [document] |
| Output Labels: | [assertion_embedding] |
| Language: | en |
| Size: | 406.9 MB |
| Case sensitive: | false |