Financial Form 10k summary Item Binary Classifier

Description

This model is a Binary Classifier (True, False) for the form_10k_summary item type of 10K Annual Reports. To use this model, make sure you provide enough context as an input. Adding Sentence Splitters to the pipeline will make the model see only sentences, not the whole text, so it’s better to skip it, unless you want to do Binary Classification as sentence level.

If you have big financial documents, and you want to look for clauses, we recommend you to split the documents using any of the techniques available in our Finance NLP Workshop Tokenization & Splitting Tutorial (link here), namely:

• Paragraph splitting (by multiline);
• Splitting by headers / subheaders;
• etc.

Take into consideration the embeddings of this model allows up to 512 tokens. If you have more than that, consider splitting in smaller pieces (you can also check the same tutorial link provided above).

Predicted Entities

other, form_10k_summary

Live Demo Open in Colab Download Copy S3 URI

How to use

documentAssembler = nlp.DocumentAssembler() \
     .setInputCol("text") \
     .setOutputCol("document")

useEmbeddings = nlp.UniversalSentenceEncoder.pretrained() \
    .setInputCols("document") \
    .setOutputCol("sentence_embeddings")

docClassifier = nlp.ClassifierDLModel.pretrained("finclf_form_10k_summary_item", "en", "finance/models")\
    .setInputCols(["sentence_embeddings"])\
    .setOutputCol("category")
    
nlpPipeline = nlp.Pipeline(stages=[
    documentAssembler, 
    useEmbeddings,
    docClassifier])
 
df = spark.createDataFrame([["YOUR TEXT HERE"]]).toDF("text")
model = nlpPipeline.fit(df)
result = model.transform(df)

Results

+-------+
| result|
+-------+
|[form_10k_summary]|
|[other]|
|[other]|
|[form_10k_summary]|

Model Information

References

Weak labelling on documents from Edgar database

Benchmarking

           label  precision    recall  f1-score   support
form_10k_summary       0.71      0.74      0.73       145
           other       0.76      0.73      0.74       162
        accuracy          -         -      0.74       307
       macro-avg       0.74      0.74      0.74       307
    weighted-avg       0.74      0.74      0.74       307