Identify intent in general text - SNIPS dataset

Description

Understand general commands and recognise the intent.

Predicted Entities

AddToPlaylist, BookRestaurant, GetWeather, PlayMusic, RateBook, SearchCreativeWork, SearchScreeningEvent.

Live Demo Open in Colab Download

How to use

...

embeddings = UniversalSentenceEncoder.pretrained('tfhub_use', lang="en") \
    .setInputCols(["document"])\
    .setOutputCol("sentence_embeddings")

classifier = ClassifierDLModel('classifierdl_use_snips').setInputCols(['sentence_embeddings']).setOutputCol('class')

nlp_pipeline = Pipeline(stages=[document_assembler, embeddings, classifier])

l_model = LightPipeline(nlp_pipeline.fit(spark.createDataFrame([['']]).toDF("text")))

annotations = l_model.fullAnnotate(["i want to bring six of us to a bistro in town that serves hot chicken sandwich that is within the same area", "show weather forcast for t h  stone memorial st  joseph peninsula state park on one hour from now"])
...
val embeddings = UniversalSentenceEncoder.pretrained('tfhub_use', lang="en") \
    .setInputCols(Array("document"))\
    .setOutputCol("sentence_embeddings")

val classifier = ClassifierDLModel('classifierdl_use_snips', 'en').setInputCols(Array('sentence_embeddings')).setOutputCol('class')

val pipeline = new Pipeline().setStages(Array(document_assembler, embeddings, classifier))

val data = Seq("i want to bring six of us to a bistro in town that serves hot chicken sandwich that is within the same area", "show weather forcast for t h  stone memorial st  joseph peninsula state park on one hour from now").toDF("text")
val result = pipeline.fit(data).transform(data)

Results

+---------------------------------------------------------------------------------------------------------------+----------------+
| document                                                        										        | label          |
+---------------------------------------------------------------------------------------------------------------+----------------+
| i want to bring six of us to a bistro in town that serves hot chicken sandwich that is within the same area   | BookRestaurant |
| show weather forcast for t h  stone memorial st  joseph peninsula state park on one hour from now				| GetWeather	 |
+---------------------------------------------------------------------------------------------------------------+----------------+

Model Information

Model Name: classifierdl_use_snips
Compatibility: Spark NLP 2.7.3+
License: Open Source
Edition: Official
Input Labels: [sentence_embeddings]
Output Labels: [class]
Language: en

Data Source

This model is trained on the NLU Benchmark, SNIPS dataset https://github.com/MiuLab/SlotGated-SLU

Benchmarking

                      precision    recall  f1-score   support

       AddToPlaylist       0.98      0.97      0.97       124
      BookRestaurant       0.98      0.99      0.98        92
          GetWeather       1.00      0.98      0.99       104
           PlayMusic       0.85      0.95      0.90        86
            RateBook       1.00      1.00      1.00        80
  SearchCreativeWork       0.82      0.84      0.83       107
SearchScreeningEvent       0.95      0.85      0.90       107

            accuracy                           0.94       700
           macro avg       0.94      0.94      0.94       700
        weighted avg       0.94      0.94      0.94       700