Sentiment Analysis in R with {sentimentr} that handles Negation (Valence Shifters)

Sentiment Analysis is one of the most wanted and used NLP techniques. Companies like to see what their customers are talking about - like if there’s a new product launch then what’s the feedback about it. Whereever you’ve got Natural Language - like Social Media, Community Pages, Customer Support - Sentiment Analysis as a technique has found its home there.

While the technique itself is highly wanted, Sentiment Analysis is one of the NLP fields that’s far from super-accurate and the reason being is a lot of ways Humans talk. One of the aspects of it is called Valence Shifters like Negation that can flip the polarity of a sentence with one word.

“I’m happy” -> Positive “I’m not happy” -> Negative

Because of this, a lot of out-of-box Sentiment analysis packages and libraries fail at tasks like this. Kudos to Tyler Rinker’s sentimentr R package that handles this scenario very well. sentimentr is a lexicon-based Sentiment Analysis Package that’s one of the best out-of-box sentiment analysis solution (given you don’t want to build a Sentiment Classification or you don’t want to use a Paid API like Google Cloud API).

Youtube - https://www.youtube.com/watch?v=eQU8Zd1B9tM

library(sentimentr)
library(tidyverse)


text <- "This tutorial is awesome. The creator is not boring"

sentiment()

sentiment_by()

sentiment(text)

sentiment_by(text, by = NULL)

profanity(text)

debates <- presidential_debates_2012  


debates_with_pol <- debates %>% 
  get_sentences() %>% 
  sentiment() %>% 
  mutate(polarity_level = ifelse(sentiment < 0.2, "Negative",
                                 ifelse(sentiment > 0.2, "Positive","Neutral")))
           

debates_with_pol %>% filter(polarity_level == "Negative") %>% View()


debates_with_senti %>% 
  ggplot() + geom_boxplot(aes(y = person, x = sentiment))

debates$dialogue %>% 
  get_sentences() %>% 
  sentiment_by() %>% #View()
  highlight()
  


debates %>% 
  get_sentences() %>% 
  sentiment_by(by = NULL) %>% #View()
  ggplot() + geom_density(aes(ave_sentiment))