Identify intent in general text - SNIPS dataset

Description

Understand general commands and recognise the intent.

Predicted Entities

AddToPlaylist, BookRestaurant, GetWeather, PlayMusic, RateBook, SearchCreativeWork, SearchScreeningEvent.

Live Demo Open in Colab Download

How to use

...

embeddings = UniversalSentenceEncoder.pretrained('tfhub_use', lang="en") \
.setInputCols(["document"])\
.setOutputCol("sentence_embeddings")

classifier = ClassifierDLModel.pretrained('classifierdl_use_snips').setInputCols(['sentence_embeddings']).setOutputCol('class')

nlp_pipeline = Pipeline(stages=[document_assembler, embeddings, classifier])

l_model = LightPipeline(nlp_pipeline.fit(spark.createDataFrame([['']]).toDF("text")))

annotations = l_model.fullAnnotate(["i want to bring six of us to a bistro in town that serves hot chicken sandwich that is within the same area", "show weather forcast for t h  stone memorial st  joseph peninsula state park on one hour from now"])
...
val embeddings = UniversalSentenceEncoder.pretrained("tfhub_use", lang="en") \
.setInputCols(Array("document"))\
.setOutputCol("sentence_embeddings")

val classifier = ClassifierDLModel.pretrained("classifierdl_use_snips", "en").setInputCols(Array("sentence_embeddings")).setOutputCol("class")

val pipeline = new Pipeline().setStages(Array(document_assembler, embeddings, classifier))

val data = Seq("i want to bring six of us to a bistro in town that serves hot chicken sandwich that is within the same area", "show weather forcast for t h  stone memorial st  joseph peninsula state park on one hour from now").toDF("text")
val result = pipeline.fit(data).transform(data)
import nlu
nlu.load("en.ner.snips").predict("""i want to bring six of us to a bistro in town that serves hot chicken sandwich that is within the same area""")

Results

+---------------------------------------------------------------------------------------------------------------+----------------+
| document                                                        										        | label          |
+---------------------------------------------------------------------------------------------------------------+----------------+
| i want to bring six of us to a bistro in town that serves hot chicken sandwich that is within the same area   | BookRestaurant |
| show weather forcast for t h  stone memorial st  joseph peninsula state park on one hour from now				| GetWeather	 |
+---------------------------------------------------------------------------------------------------------------+----------------+

Model Information

Model Name: classifierdl_use_snips
Compatibility: Spark NLP 2.7.3+
License: Open Source
Edition: Official
Input Labels: [sentence_embeddings]
Output Labels: [class]
Language: en

Data Source

This model is trained on the NLU Benchmark, SNIPS dataset https://github.com/MiuLab/SlotGated-SLU

Benchmarking

precision    recall  f1-score   support

AddToPlaylist       0.98      0.97      0.97       124
BookRestaurant       0.98      0.99      0.98        92
GetWeather       1.00      0.98      0.99       104
PlayMusic       0.85      0.95      0.90        86
RateBook       1.00      1.00      1.00        80
SearchCreativeWork       0.82      0.84      0.83       107
SearchScreeningEvent       0.95      0.85      0.90       107

accuracy                           0.94       700
macro avg       0.94      0.94      0.94       700
weighted avg       0.94      0.94      0.94       700