Edit on GitHub
  • John Snow Labs
  • Apr 27, 2023

Legal NER for MAPA(Multilingual Anonymisation for Public Administrations)

  • nl
  • ner
  • licensed
  • legal
  • mapa

Description

The dataset consists of 12 documents taken from EUR-Lex, a multilingual corpus of court decisions and legal dispositions in the 24 official languages of the European Union.

This model extracts ADDRESS, DATE, ORGANISATION, and PERSON entities from Dutch documents.

Predicted Entities

ADDRESS, DATE, ORGANISATION, PERSON

Download Copy S3 URI

How to use

document_assembler = nlp.DocumentAssembler()\
        .setInputCol("text")\
        .setOutputCol("document")

sentence_detector = nlp.SentenceDetectorDLModel.pretrained("sentence_detector_dl", "xx")\
        .setInputCols(["document"])\
        .setOutputCol("sentence")

tokenizer = nlp.Tokenizer()\
        .setInputCols(["sentence"])\
        .setOutputCol("token")

embeddings = nlp.BertEmbeddings.pretrained("bert_embeddings_base_nl_cased", "nl")\
        .setInputCols(["sentence", "token"])\
        .setOutputCol("embeddings")\
        .setMaxSentenceLength(512)\
        .setCaseSensitive(True)

ner_model = legal.NerModel.pretrained("legner_mapa", "nl", "legal/models")\
        .setInputCols(["sentence", "token", "embeddings"])\
        .setOutputCol("ner")

ner_converter = nlp.NerConverter()\
        .setInputCols(["sentence", "token", "ner"])\
        .setOutputCol("ner_chunk")

nlpPipeline = nlp.Pipeline(stages=[
        document_assembler,
        sentence_detector,
        tokenizer,
        embeddings,
        ner_model,
        ner_converter])

empty_data = spark.createDataFrame([[""]]).toDF("text")

model = nlpPipeline.fit(empty_data)

text = ["""Liberato en Grigorescu zijn op 22 oktober 2005 in Rome ( Italië ) in het huwelijk getreden en hebben tot de geboorte van hun kind op 20 februari 2006 in die lidstaat samengewoond."""]

result = model.transform(spark.createDataFrame([text]).toDF("text"))

Results

+----------------+---------+
|chunk           |ner_label|
+----------------+---------+
|Liberato        |PERSON   |
|Grigorescu      |PERSON   |
|22 oktober 2005 |DATE     |
|Rome ( Italië ) |ADDRESS  |
|20 februari 2006|DATE     |
+----------------+---------+

Model Information

Model Name: legner_mapa
Compatibility: Legal NLP 1.0.0+
License: Licensed
Edition: Official
Input Labels: [sentence, token, embeddings]
Output Labels: [ner]
Language: nl
Size: 1.4 MB

References

The dataset is available here.

Benchmarking

label         precision  recall  f1-score  support 
ADDRESS       0.87       0.81    0.84      16      
DATE          0.98       0.98    0.98      54      
ORGANISATION  0.83       0.97    0.90      31      
PERSON        0.90       0.92    0.91      39      
macro-avg     0.91       0.94    0.93      140     
macro-avg     0.90       0.92    0.91      140     
weighted-avg  0.91       0.94    0.93      140