Sentiment Analysis in NLP

The reasons are obvious:

• 80% of the world’s data is unstructured. People increasingly tend to share their opinions on forums, blogs, social networks, and chats. It is very time-consuming to analyze and sort all these texts manually. So, the cost-effective way is to automate this process by using sentiment analysis systems.
• The second reason is that it is possible to provide real-time sentiment analysis. This property allows one to react quickly to specific situations in real-time, e.g. dealing with an angry customer and de-escalating the situation.

There are three types of sentiment analysis algorithms: rule-based, automatic, and hybrid (which combines the first two approaches). In a rule-based system, there are two lists of polarized words (keywords) or short phrases, like “bad, horrible, ugly, etc.” and “good, beautiful, amazing, high quality, etc.” Then, classic NLP techniques are used to find and count the numbers of positive and negative words in the text. The comparison of these two numbers gives us a positive, or negative, or neutral sentiment.

Automatic approaches use machine learning techniques. In this approach, it is important to prepare an accurate database (or use an existing one) with, e.g., user reviews already labeled as positive, negative, and neutral. Then, the classification model should be trained using this labeled dataset. The resulting model is able to predict tags (positive, negative, neutral) for unseen text.

In general, the precision of sentiment analysis algorithms is about 60-80%. What makes sentiment analysis so inaccurate? While there are many reasons, we have listed some of the main challenges encountered in sentiment analysis below:

• Complex sentence structure. People often express opinions in complex ways. For example, the sentence “I don’t like the movie that you love” has both positive and negative sentiment towards the movie. Most systems are unable to recognize the sentiment that was expressed in the sentence.
• Multiple topics. A review can contain several topics, e.g. a user can mention a few different products in one review, or a few different aspects of a product in one review. For example, a review may contain one sentence with a negative opinion of a product's price (“iPhone X is too expensive”) and another sentence with a positive opinion of the same product's camera ( “the camera of iPhone X performs really well”). Models which are able to manage reviews with several topics in one document are in high demand in sentiment analysis.
• Implicit sentiment and sentiment ambiguity. A sentence with a positive or negative phrase does not necessarily express any sentiment; and, vice versa, a sentence without an opinion marker may have some polarity. For example, the sentence “Can you recommend a good movie I could watch?” does not express any sentiment or opinion, but uses the positive sentiment word “good”. Conversely, the sentence “I went to eat, but the restaurant was closed” expresses negative polarity without sentiment word markers.
• Sarcasm, irony and metaphors. Sarcasm and irony are often used in reviews. Readers can easily assess a product’s quality by reading such reviews. Unfortunately, the same cannot be said about machines. It is often impossible to identify a sentence containing sarcasm or irony without knowing the context.

There is a wide range of potential uses for sentiment analysis. It has many applications in e-commerce, marketing, sociology and psychology research, as well as politics. Here are a few examples:

• Customer feedback. Analyzing customers’ reviews can help companies to avoid crises. A major motivator for many customers ending their relationship with a specific company is their first negative experience with a company service or product. Constant monitoring of reviews and outer social media is the key to major insights, e.g. whether users like the new version of the company app or not. Sentiment analysis provides quick reactions, which allow companies to tackle an issue promptly.
• Market research and analysis. Tracking customer sentiment helps to understand overall customer satisfaction, loyalty, and how likely they are to engage again in a day/week/month. The knowledge in these metrics may provide insights which are valuable to a company's marketing strategy.
• Sociology. Sociology is interested in idea propagation through groups. Opinions and reactions to ideas in a given society are relevant to the adoption of new ideas. Insights on this process can be taken from sentiment analysis of reactions on blogs, tweets, etc. For example, sentiment analysis can shed light on behaviors across cultures and inter-group identity meanings through analysis of news media coverage during the Arab Spring. Another application of sentiment analysis is in exploring how influence is spread between communities and individuals in the blogosphere.