Packages

package vivekn

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. All

Type Members

  1. trait ReadablePretrainedVivekn extends ParamsAndFeaturesReadable[ViveknSentimentModel] with HasPretrained[ViveknSentimentModel]
  2. class ViveknSentimentApproach extends AnnotatorApproach[ViveknSentimentModel] with ViveknSentimentUtils

    Trains a sentiment analyser inspired by the algorithm by Vivek Narayanan https://github.com/vivekn/sentiment/.

    Trains a sentiment analyser inspired by the algorithm by Vivek Narayanan https://github.com/vivekn/sentiment/.

    The algorithm is based on the paper "Fast and accurate sentiment classification using an enhanced Naive Bayes model".

    The analyzer requires sentence boundaries to give a score in context. Tokenization is needed to make sure tokens are within bounds. Transitivity requirements are also required.

    The training data needs to consist of a column for normalized text and a label column (either "positive" or "negative").

    For extended examples of usage, see the Spark NLP Workshop and the ViveknSentimentTestSpec.

    Example

    import spark.implicits._
    import com.johnsnowlabs.nlp.base.DocumentAssembler
    import com.johnsnowlabs.nlp.annotators.Tokenizer
    import com.johnsnowlabs.nlp.annotators.Normalizer
    import com.johnsnowlabs.nlp.annotators.sda.vivekn.ViveknSentimentApproach
    import com.johnsnowlabs.nlp.Finisher
    import org.apache.spark.ml.Pipeline
    
    val document = new DocumentAssembler()
      .setInputCol("text")
      .setOutputCol("document")
    
    val token = new Tokenizer()
      .setInputCols("document")
      .setOutputCol("token")
    
    val normalizer = new Normalizer()
      .setInputCols("token")
      .setOutputCol("normal")
    
    val vivekn = new ViveknSentimentApproach()
      .setInputCols("document", "normal")
      .setSentimentCol("train_sentiment")
      .setOutputCol("result_sentiment")
    
    val finisher = new Finisher()
      .setInputCols("result_sentiment")
      .setOutputCols("final_sentiment")
    
    val pipeline = new Pipeline().setStages(Array(document, token, normalizer, vivekn, finisher))
    
    val training = Seq(
      ("I really liked this movie!", "positive"),
      ("The cast was horrible", "negative"),
      ("Never going to watch this again or recommend it to anyone", "negative"),
      ("It's a waste of time", "negative"),
      ("I loved the protagonist", "positive"),
      ("The music was really really good", "positive")
    ).toDF("text", "train_sentiment")
    val pipelineModel = pipeline.fit(training)
    
    val data = Seq(
      "I recommend this movie",
      "Dont waste your time!!!"
    ).toDF("text")
    val result = pipelineModel.transform(data)
    
    result.select("final_sentiment").show(false)
    +---------------+
    |final_sentiment|
    +---------------+
    |[positive]     |
    |[negative]     |
    +---------------+
    See also

    SentimentDetector for an alternative approach to sentiment detection

  3. class ViveknSentimentModel extends AnnotatorModel[ViveknSentimentModel] with HasSimpleAnnotate[ViveknSentimentModel] with ViveknSentimentUtils

    Sentiment analyser inspired by the algorithm by Vivek Narayanan https://github.com/vivekn/sentiment/.

    Sentiment analyser inspired by the algorithm by Vivek Narayanan https://github.com/vivekn/sentiment/.

    The algorithm is based on the paper "Fast and accurate sentiment classification using an enhanced Naive Bayes model".

    This is the instantiated model of the ViveknSentimentApproach. For training your own model, please see the documentation of that class.

    The analyzer requires sentence boundaries to give a score in context. Tokenization is needed to make sure tokens are within bounds. Transitivity requirements are also required.

    For extended examples of usage, see the Spark NLP Workshop and the ViveknSentimentTestSpec.

    See also

    SentimentDetector for an alternative approach to sentiment detection

  4. trait ViveknSentimentUtils extends AnyRef

Value Members

  1. object ViveknSentimentModel extends ReadablePretrainedVivekn with Serializable

    This is the companion object of ViveknSentimentModel.

    This is the companion object of ViveknSentimentModel. Please refer to that class for the documentation.

Ungrouped