BERT Biolink Embeddings


This embeddings component was trained on PubMed abstracts all along with citation link information. The model was introduced in this paper, achieving state-of-the-art performance on several biomedical NLP benchmarks such as BLURB and MedQA-USMLE.

Predicted Entities


How to use

embeddings = BertEmbeddings.pretrained("bert_biolink_large", "en")\
      .setInputCols(["sentence", "token"])\
val embeddings = BertEmbeddings.pretrained("bert_biolink_large", "en")
      .setInputCols(Array("sentence", "token"))

Model Information

Model Name: bert_biolink_large
Compatibility: Spark NLP 3.4.2+
License: Open Source
Edition: Official
Input Labels: [sentence, token]
Output Labels: [embeddings]
Language: en
Size: 1.3 GB
Case sensitive: true
Max sentence length: 512


  author =  {Michihiro Yasunaga and Jure Leskovec and Percy Liang},
  title =   {LinkBERT: Pretraining Language Models with Document Links},
  year =    {2022},  
  booktitle = {Association for Computational Linguistics (ACL)},  


Scores for several benchmark datasets :

- BLURB : 84.30
- PubMedQA : 72.2
- BioASQ : 94.8
- MedQA-USMLE : 44.6