Emotion Detection Classifier

Description

Automatically identify Joy, Surprise, Fear, Sadness emotions in Tweets.

Predicted Entities

surprise, sadness, fear, joy

Live Demo Open in Colab Download

How to use

document_assembler = DocumentAssembler()\
  .setInputCol("text")\
  .setOutputCol("document")
use = UniversalSentenceEncoder.pretrained('tfhub_use', lang="en") \
  .setInputCols(["document"])\
  .setOutputCol("sentence_embeddings")
document_classifier = ClassifierDLModel.pretrained('classifierdl_use_emotion', 'en') \
  .setInputCols(["document", "sentence_embeddings"]) \
  .setOutputCol("class")
nlpPipeline = Pipeline(stages=[document_assembler, use, document_classifier])
light_pipeline = LightPipeline(nlp_pipeline.fit(spark.createDataFrame([['']]).toDF("text")))
annotations = light_pipeline.fullAnnotate('@Mira I just saw you on live t.v!!')
val documentAssembler = DocumentAssembler()
  .setInputCol("text")
  .setOutputCol("document")
val use = UniversalSentenceEncoder.pretrained(lang="en")
  .setInputCols(Array("document"))
  .setOutputCol("sentence_embeddings")
val document_classifier = ClassifierDLModel.pretrained("classifierdl_use_emotion", "en")
  .setInputCols(Array("document", "sentence_embeddings"))
  .setOutputCol("class")
val pipeline = new Pipeline().setStages(Array(documentAssembler, use, document_classifier))

val result = pipeline.fit(Seq.empty["@Mira I just saw you on live t.v!!"].toDS.toDF("text")).transform(data)
import nlu

text = ["""@Mira I just saw you on live t.v!!"""]
emotion_df = nlu.load('en.classify.emotion.use').predict(text, output_level='document')
emotion_df[["document", "emotion"]]

Results

+------------------------------------------------------------------------------------------------+------------+
|document                                                                                        |class       |
+------------------------------------------------------------------------------------------------+------------+
|@Mira I just saw you on live t.v!!                                                              | joy        |
+------------------------------------------------------------------------------------------------+------------+

Model Information

Model Name: classifierdl_use_emotion
Compatibility: Spark NLP 2.7.1+
License: Open Source
Edition: Official
Input Labels: [sentence_embeddings]
Output Labels: [class]
Language: en
Dependencies: tfhub_use

Data Source

This model is trained on multiple datasets inlcuding youtube comments, twitter and ISEAR dataset.

Benchmarking

        fear       0.78      0.67      0.72      2253
         joy       0.71      0.68      0.69      3000
     sadness       0.69      0.73      0.71      3075
    surprise       0.67      0.73      0.70      3067

    accuracy                           0.71     11395
   macro avg       0.71      0.70      0.71     11395
weighted avg       0.71      0.71      0.71     11395