BERT Base Biolink Embeddings

Description

This embeddings component was trained on PubMed abstracts all along with citations link information. The embeddings were introduced in this paper. This model achieves state-of-the-art performance on several biomedical NLP benchmarks such as BLURB and MedQA-USMLE.

Predicted Entities

Download

How to use

embeddings = BertEmbeddings.pretrained("bert_biolink_base", "en")\
       .setInputCols(["sentence", "token"])\
       .setOutputCol("embeddings")
val embeddings = BertEmbeddings.pretrained("bert_biolink_base", "en")
       .setInputCols(Array("sentence", "token"))
       .setOutputCol("embeddings")

Model Information

Model Name: bert_biolink_base
Compatibility: Spark NLP 3.4.2+
License: Open Source
Edition: Official
Input Labels: [sentence, token]
Output Labels: [embeddings]
Language: en
Size: 406.4 MB
Case sensitive: true
Max sentence length: 512

References

https://pubmed.ncbi.nlm.nih.gov/

@InProceedings{yasunaga2022linkbert,
   author =  {Michihiro Yasunaga and Jure Leskovec and Percy Liang},
   title =   {LinkBERT: Pretraining Language Models with Document Links},
   year =    {2022},  
   booktitle = {Association for Computational Linguistics (ACL)},  
}

Benchmarking

Scores for several benchmark datasets :

 - BLURB : 83.39
 - PubMedQA : 70.2
 - BioASQ : 91.4
 - MedQA-USMLE : 40.0