Edit on GitHub
  • John Snow Labs
  • Apr 28, 2023

Legal NER for MAPA(Multilingual Anonymisation for Public Administrations)

  • sk
  • licensed
  • ner
  • legal
  • mapa

Description

The dataset consists of 12 documents taken from EUR-Lex, a multilingual corpus of court decisions and legal dispositions in the 24 official languages of the European Union.

This model extracts ADDRESS, AMOUNT, DATE, ORGANISATION, and PERSON entities from Slovak documents.

Predicted Entities

ADDRESS, AMOUNT, DATE, ORGANISATION, PERSON

Download Copy S3 URI

How to use

document_assembler = nlp.DocumentAssembler()\
        .setInputCol("text")\
        .setOutputCol("document")

sentence_detector = nlp.SentenceDetectorDLModel.pretrained("sentence_detector_dl", "xx")\
        .setInputCols(["document"])\
        .setOutputCol("sentence")

tokenizer = nlp.Tokenizer()\
        .setInputCols(["sentence"])\
        .setOutputCol("token")

embeddings = nlp.RoBertaEmbeddings.pretrained("roberta_base_slovak_legal","sk")\
        .setInputCols(["sentence", "token"])\
        .setOutputCol("embeddings")\
        .setMaxSentenceLength(512)\
        .setCaseSensitive(True)

ner_model = legal.NerModel.pretrained("legner_mapa", "sk", "legal/models")\
        .setInputCols(["sentence", "token", "embeddings"])\
        .setOutputCol("ner")

ner_converter = nlp.NerConverter()\
        .setInputCols(["sentence", "token", "ner"])\
        .setOutputCol("ner_chunk")

nlpPipeline = nlp.Pipeline(stages=[
        document_assembler,
        sentence_detector,
        tokenizer,
        embeddings,
        ner_model,
        ner_converter])

empty_data = spark.createDataFrame([[""]]).toDF("text")

model = nlpPipeline.fit(empty_data)

text = ["""Návrhom podaným 22. mája 2007 na Tribunale di Teramo ( súd v Terame, Taliansko ) požiadal pán Liberato o rozluku a o zverenie syna do svojej starostlivosti."""]

result = model.transform(spark.createDataFrame([text]).toDF("text"))

Results

+-------------+---------+
|chunk        |ner_label|
+-------------+---------+
|22. mája 2007|DATE     |
|Terame       |ADDRESS  |
|Taliansko    |ADDRESS  |
|pán Liberato |PERSON   |
+-------------+---------+

Model Information

Model Name: legner_mapa
Compatibility: Legal NLP 1.0.0+
License: Licensed
Edition: Official
Input Labels: [sentence, token, embeddings]
Output Labels: [ner]
Language: sk
Size: 1.4 MB

References

The dataset is available here.

Benchmarking

label         precision  recall  f1-score  support 
ADDRESS       0.88       0.85    0.86      26      
AMOUNT        1.00       1.00    1.00      4       
DATE          0.92       0.88    0.90      50      
ORGANISATION  0.79       0.61    0.69      31      
PERSON        0.66       0.86    0.75      44      
micro-avg     0.80       0.82    0.81      155     
macro-avg     0.85       0.84    0.84      155     
weighted-avg  0.81       0.82    0.81      155