Description
Pretrained Lemmatizer model (lemma_bdt) trained on Universal Dependencies 2.9 (UD_Basque-BDT) in Basque language.
Open in Colab Download Copy S3 URICopied!
How to use
document = DocumentAssembler()\
.setInputCol("text")\
.setOutputCol("document")
sentence = SentenceDetectorDLModel.pretrained("sentence_detector_dl", "xx")\
.setInputCols(["document"])\
.setOutputCol("sentence")
tokenizer = Tokenizer()\
.setInputCols(["sentence"])\
.setOutputCol("token")
lemma = LemmatizerModel.pretrained("lemma_bdt", "eu")\
.setInputCols(["token"])\
.setOutputCol("lemma")
pipeline = Pipeline(stages=[document, sentence, tokenizer, lemma])
data = spark.createDataFrame([["I love Spark NLP"]]).toDF("text")
result = pipeline.fit(data).transform(data)
Model Information
| Model Name: | lemma_bdt |
| Compatibility: | Spark NLP 3.4.3+ |
| License: | Open Source |
| Edition: | Official |
| Input Labels: | [form] |
| Output Labels: | [lemma] |
| Language: | eu |
| Size: | 261.5 KB |
References
Model is trained on Universal Dependencies (treebank 2.9) UD_Basque-BDT
https://github.com/UniversalDependencies/UD_Basque-BDT