Description
Automatically identify Joy, Surprise, Fear, Sadness in Tweets using out pretrained Spark NLP DL classifier.
Classified Labels
surprise
, sadness
, fear
, joy
.
Live Demo
Open in Colab
Download
How to use
documentAssembler = DocumentAssembler()\
.setInputCol("text")\
.setOutputCol("document")
use = UniversalSentenceEncoder.pretrained(lang="en") \
.setInputCols(["document"])\
.setOutputCol("sentence_embeddings")
document_classifier = ClassifierDLModel.pretrained('classifierdl_use_emotion', 'en') \
.setInputCols(["document", "sentence_embeddings"]) \
.setOutputCol("class")
nlpPipeline = Pipeline(stages=[documentAssembler, use, document_classifier])
light_pipeline = LightPipeline(nlp_pipeline.fit(spark.createDataFrame([['']]).toDF("text")))
annotations = light_pipeline.fullAnnotate('@Mira I just saw you on live t.v!!')
val documentAssembler = DocumentAssembler()
.setInputCol("text")
.setOutputCol("document")
val use = UniversalSentenceEncoder.pretrained(lang="en")
.setInputCols(Array("document"))
.setOutputCol("sentence_embeddings")
val document_classifier = ClassifierDLModel.pretrained("classifierdl_use_emotion", "en")
.setInputCols(Array("document", "sentence_embeddings"))
.setOutputCol("class")
val pipeline = new Pipeline().setStages(Array(documentAssembler, use, document_classifier))
val result = pipeline.fit(Seq.empty["@Mira I just saw you on live t.v!!"].toDS.toDF("text")).transform(data)
import nlu
text = ["""@Mira I just saw you on live t.v!!"""]
emotion_df = nlu.load('en.classify.emotion.use').predict(text, output_level='document')
emotion_df[["document", "emotion"]]
Results
+------------------------------------------------------------------------------------------------+------------+
|document |class |
+------------------------------------------------------------------------------------------------+------------+
|@Mira I just saw you on live t.v!! | joy |
+------------------------------------------------------------------------------------------------+------------+
Model Information
Model Name | classifierdl_use_emotion |
Model Class | ClassifierDLModel |
Spark Compatibility | 2.5.3 |
Spark NLP Compatibility | 2.4 |
License | open source |
Edition | public |
Input Labels | [document, sentence_embeddings] |
Output Labels | [class] |
Language | en |
Upstream Dependencies | tfhub_use |
Data Source
This model is trained on multiple datasets inlcuding youtube comments, twitter and ISEAR dataset.
PREVIOUSCyberbullying Classifier