Financial NER on EDGAR Documents

Description

This Financial NER model extracts ORG, INST, LAW, COURT, PER, LOC, MISC, ALIAS, and TICKER entities from the US SEC EDGAR documents.

Predicted Entities

ALIAS, COURT, INST, LAW, LOC, MISC, ORG, PER, TICKER

Download Copy S3 URI

How to use

document_assembler = nlp.DocumentAssembler()\
      .setInputCol("text")\
      .setOutputCol("document")
        
sentence_detector = nlp.SentenceDetectorDLModel.pretrained("sentence_detector_dl", "en")\
      .setInputCols(["document"])\
      .setOutputCol("sentence")

tokenizer = nlp.Tokenizer()\
      .setInputCols(["sentence"])\
      .setOutputCol("token")

embeddings = nlp.BertEmbeddings.pretrained("bert_embeddings_sec_bert_base", "en") \
      .setInputCols("sentence", "token") \
      .setOutputCol("embeddings")\
      .setMaxSentenceLength(512)\
      .setCaseSensitive(True)

ner_model = finance.NerModel.pretrained("finner_sec_edgar", "en", "finance/models")\
      .setInputCols(["sentence", "token", "embeddings"])\
      .setOutputCol("ner")

ner_converter = nlp.NerConverter()\
      .setInputCols(["sentence", "token", "ner"])\
      .setOutputCol("ner_chunk")

nlpPipeline = nlp.Pipeline(stages=[
        document_assembler,
        sentence_detector,
        tokenizer,
        embeddings,
        ner_model,
        ner_converter])

empty_data = spark.createDataFrame([[""]]).toDF("text")

model = nlpPipeline.fit(empty_data)

text = ["""In our opinion, the accompanying consolidated balance sheets and the related consolidated statements of operations, of changes in stockholders' equity, and of cash flows present fairly, in all material respects, the financial position of SunGard Capital Corp. II and its subsidiaries ( SCC II ) at December 31, 2010, and 2009, and the results of their operations and their cash flows for each of the three years in the period ended December 31, 2010, in conformity with accounting principles generally accepted in the United States of America."""]

result = model.transform(spark.createDataFrame([text]).toDF("text"))

Results

+----------------------------------------+---------+
|chunk                                   |ner_label|
+----------------------------------------+---------+
|SunGard Capital Corp. II                |ORG      |
|SCC II                                  |ALIAS    |
|accounting principles generally accepted|LAW      |
|United States of America                |LOC      |
+----------------------------------------+---------+

Model Information

Model Name: finner_sec_edgar
Compatibility: Finance NLP 1.0.0+
License: Licensed
Edition: Official
Input Labels: [sentence, token, embeddings]
Output Labels: [ner]
Language: en
Size: 16.3 MB

References

In-house annotations

Benchmarking

label         precision  recall  f1-score  support 
ALIAS         0.88       0.93    0.90      84      
COURT         1.00       0.83    0.91      6       
INST          0.95       0.78    0.86      76      
LAW           0.90       0.90    0.90      166     
LOC           0.83       0.83    0.83      139     
MISC          0.83       0.81    0.82      226     
ORG           0.88       0.87    0.87      430     
PER           0.91       0.79    0.85      66      
TICKER        0.86       0.86    0.86      7       
micro-avg     0.87       0.85    0.86      1200    
macro-avg     0.89       0.84    0.87      1200    
weighted-avg  0.88       0.85    0.86      1200