Description
Determine if news articles are Real or Fake.
Predicted Entities
REAL, FAKE
Live Demo Open in Colab Download
How to use
document_assembler = DocumentAssembler()\
.setInputCol("text")\
.setOutputCol("document")
use = UniversalSentenceEncoder.pretrained('tfhub_use', lang="en") \
.setInputCols(["document"])\
.setOutputCol("sentence_embeddings")
document_classifier = ClassifierDLModel.pretrained('classifierdl_use_fakenews', 'en') \
.setInputCols(["document", "sentence_embeddings"]) \
.setOutputCol("class")
nlpPipeline = Pipeline(stages=[document_assembler, use, document_classifier])
light_pipeline = LightPipeline(nlp_pipeline.fit(spark.createDataFrame([['']]).toDF("text")))
annotations = light_pipeline.fullAnnotate('Donald Trump a KGB Spy? 11/02/2016 In today’s video, Christopher Greene of AMTV reports Hillary Clinton')
Results
+--------------------------------------------------------------------------------------------------------+------------+
|document |class |
+--------------------------------------------------------------------------------------------------------+------------+
|Donald Trump a KGB Spy? 11/02/2016 In today’s video, Christopher Greene of AMTV reports Hillary Clinton | FAKE |
+--------------------------------------------------------------------------------------------------------+------------+
Model Information
| Model Name: | classifierdl_use_fakenews |
| Compatibility: | Spark NLP 2.7.1+ |
| License: | Open Source |
| Edition: | Official |
| Input Labels: | [sentence_embeddings] |
| Output Labels: | [class] |
| Language: | en |
| Dependencies: | tfhub_use |
Data Source
This model is trained on the fake new classification challenge. https://raw.githubusercontent.com/joolsa/fake_real_news_dataset/master/fake_or_real_news.csv.zip
Benchmarking
precision recall f1-score support
FAKE 0.86 0.89 0.88 626
REAL 0.89 0.86 0.87 634
accuracy 0.87 1260
macro avg 0.88 0.87 0.87 1260
weighted avg 0.88 0.87 0.87 1260