# Bert for Sequence Classification (Question vs Statement)

## Description

Trained to add sentence classifying capabilities to distinguish between Question vs Statements.

This model was imported from Hugging Face (https://huggingface.co/shahrukhx01/question-vs-statement-classifier), and trained based on Haystack (https://github.com/deepset-ai/haystack/issues/611).

## Predicted Entities

question, statement

## How to use

documentAssembler = DocumentAssembler()\
.setInputCol("text")\
.setOutputCol("document")

sentenceDetector = SentenceDetectorDLModel.pretrained() \
.setInputCols(["document"]) \
.setOutputCol("sentence")

tokenizer = Tokenizer()\
.setInputCols("sentence")\
.setOutputCol("token")

seq = BertForSequenceClassification.pretrained('bert_sequence_classifier_question_statement', 'en')\
.setInputCols(["token", "sentence"])\
.setOutputCol("label")\
.setCaseSensitive(True)

pipeline = Pipeline(stages = [
documentAssembler,
sentenceDetector,
tokenizer,
seq])

test_sentences = ["""What feature in your car did you not realize you had until someone else told you about it?
Years ago, my Dad bought me a cute little VW Beetle. The first day I had it, me and my BFF were sitting in my car looking at everything.
When we opened the center console, we had quite the scare. Inside was a hollowed out, plastic phallic looking object with tiny spikes on it.
My friend and I literally screamed in horror. It was clear to us that somehow someone left their “toy” in my new car! We were shook, as they say.
This was my car, I had to do something. So, I used a pen to pick up the nasty looking thing and threw it out.
We freaked out about how gross it was and then we forgot about it… until my Dad called me.
My Dad said: How’s the new car? Have you seen the flower holder in the center console?
To summarize, we thought a flower vase was an XXX item…
In our defense, this is a picture of a VW Beetle flower holder."""]

res = p_model.transform(spark.createDataFrame(pd.DataFrame({'text': test_sentences})))

val documentAssembler = new DocumentAssembler()
.setInputCol("text")
.setOutputCol("document")

val sentenceDetector = SentenceDetectorDLModel.pretrained()
.setInputCols(Array("document"))
.setOutputCol("sentence")

val tokenizer = new Tokenizer()
.setInputCols("sentence")
.setOutputCol("token")

val seq = BertForSequenceClassification.pretrained("bert_sequence_classifier_question_statement", "en")
.setInputCols(Array("token", "sentence"))
.setOutputCol("label")
.setCaseSensitive(True)

val pipeline = new Pipeline().setStages(Array(
documentAssembler,
sentenceDetector,
tokenizer,
seq))

val test_sentences = "What feature in your car did you not realize you had until someone else told you about it?
Years ago, my Dad bought me a cute little VW Beetle. The first day I had it, me and my BFF were sitting in my car looking at everything.
When we opened the center console, we had quite the scare. Inside was a hollowed out, plastic phallic looking object with tiny spikes on it.
My friend and I literally screamed in horror. It was clear to us that somehow someone left their “toy” in my new car! We were shook, as they say.
This was my car, I had to do something. So, I used a pen to pick up the nasty looking thing and threw it out.
We freaked out about how gross it was and then we forgot about it… until my Dad called me.
My Dad said: How’s the new car? Have you seen the flower holder in the center console?
To summarize, we thought a flower vase was an XXX item…
In our defense, this is a picture of a VW Beetle flower holder."

val example = Seq(test_sentences).toDF("text")

val result = pipeline.fit(example).transform(example)

import nlu
Years ago, my Dad bought me a cute little VW Beetle. The first day I had it, me and my BFF were sitting in my car looking at everything.
When we opened the center console, we had quite the scare. Inside was a hollowed out, plastic phallic looking object with tiny spikes on it.
My friend and I literally screamed in horror. It was clear to us that somehow someone left their “toy” in my new car! We were shook, as they say.
This was my car, I had to do something. So, I used a pen to pick up the nasty looking thing and threw it out.
We freaked out about how gross it was and then we forgot about it… until my Dad called me.
My Dad said: How’s the new car? Have you seen the flower holder in the center console?
To summarize, we thought a flower vase was an XXX item…
In our defense, this is a picture of a VW Beetle flower holder.""")


## Results

+------------------------------------------------------------------------------------------+---------+
|                                                                                  sentence|    label|
+------------------------------------------------------------------------------------------+---------+
|What feature in your car did you not realize you had until someone else told you about it?| question|
|                                      Years ago, my Dad bought me a cute little VW Beetle.|statement|
|       The first day I had it, me and my BFF were sitting in my car looking at everything.|statement|
|                                When we opened the center console, we had quite the scare.|statement|
|         Inside was a hollowed out, plastic phallic looking object with tiny spikes on it.|statement|
|                                             My friend and I literally screamed in horror.|statement|
|                   It was clear to us that somehow someone left their “toy” in my new car!|statement|
|                                                               We were shook, as they say.|statement|
|                                                   This was my car, I had to do something.|statement|
|                     So, I used a pen to pick up the nasty looking thing and threw it out.|statement|
|We freaked out about how gross it was and then we forgot about it… until my Dad called me.|statement|
|                                                           My Dad said: How’s the new car?| question|
|                                    Have you seen the flower holder in the center console?| question|
|                                   To summarize, we thought a flower vase was an XXX item…|statement|
|                           In our defense, this is a picture of a VW Beetle flower holder.|statement|
+------------------------------------------------------------------------------------------+---------+


## Model Information

 Model Name: bert_sequence_classifier_question_statement Compatibility: Spark NLP 3.3.2+ License: Open Source Edition: Official Input Labels: [token, sentence] Output Labels: [label] Language: en Case sensitive: true

## Data Source

https://github.com/deepset-ai/haystack/issues/611

## Benchmarking

Extracted from https://github.com/deepset-ai/haystack/issues/611
precision    recall  f1-score   support

statement        0.94      0.94      0.94     16105
question        0.96      0.96      0.96     26198

accuracy                           0.95     42303
macro avg       0.95      0.95      0.95     42303
weighted avg       0.95      0.95      0.95     42303