Sentiment Analysis in NLP

Sentiment Analysis in NLP

Sentiment analysis has become a new trend in social media monitoring, brand monitoring, product analytics, and market research. Like most areas that use artificial intelligence, sentiment analysis (also known as Opinion Mining) is an interdisciplinary field spanning computer science, psychology, social sciences, linguistics, and cognitive science. The goal of sentiment analysis is to identify and extract attitudes, emotions, and opinions from a text. In other words, sentiment analysis is a mining of subjective impressions, but not facts, from users’ tweets, reviews, posts, etc.

Why is opinion mining necessary?

The reasons are obvious:

  • 80% of the world’s data is unstructured. People increasingly tend to share their opinions on forums, blogs, social networks, and chats. It is very time-consuming to analyze and sort all these texts manually. So, the cost-effective way is to automate this process by using sentiment analysis systems.
  • The second reason is that it is possible to provide real-time sentiment analysis. This property allows one to react quickly to specific situations in real-time, e.g. dealing with an angry customer and de-escalating the situation.

There are three types of sentiment analysis algorithms: rule-based, automatic, and hybrid (which combines the first two approaches). In a rule-based system, there are two lists of polarized words (keywords) or short phrases, like “bad, horrible, ugly, etc.” and “good, beautiful, amazing, high quality, etc.” Then, classic NLP techniques are used to find and count the numbers of positive and negative words in the text. The comparison of these two numbers gives us a positive, or negative, or neutral sentiment.

Automatic approaches use machine learning techniques. In this approach, it is important to prepare an accurate database (or use an existing one) with, e.g., user reviews already labeled as positive, negative, and neutral. Then, the classification model should be trained using this labeled dataset. The resulting model is able to predict tags (positive, negative, neutral) for unseen text.

In general, the precision of sentiment analysis algorithms is about 60-80%. What makes sentiment analysis so inaccurate? While there are many reasons, we have listed some of the main challenges encountered in sentiment analysis below:

  • Complex sentence structure. People often express opinions in complex ways. For example, the sentence “I don’t like the movie that you love” has both positive and negative sentiment towards the movie. Most systems are unable to recognize the sentiment that was expressed in the sentence.
  • Multiple topics. A review can contain several topics, e.g. a user can mention a few different products in one review, or a few different aspects of a product in one review. For example, a review may contain one sentence with a negative opinion of a product's price (“iPhone X is too expensive”) and another sentence with a positive opinion of the same product's camera ( “the camera of iPhone X performs really well”). Models which are able to manage reviews with several topics in one document are in high demand in sentiment analysis.
  • Implicit sentiment and sentiment ambiguity. A sentence with a positive or negative phrase does not necessarily express any sentiment; and, vice versa, a sentence without an opinion marker may have some polarity. For example, the sentence “Can you recommend a good movie I could watch?” does not express any sentiment or opinion, but uses the positive sentiment word “good”. Conversely, the sentence “I went to eat, but the restaurant was closed” expresses negative polarity without sentiment word markers.
  • Sarcasm, irony and metaphors. Sarcasm and irony are often used in reviews. Readers can easily assess a product’s quality by reading such reviews. Unfortunately, the same cannot be said about machines. It is often impossible to identify a sentence containing sarcasm or irony without knowing the context.

Applications of Sentiment Analysis

There is a wide range of potential uses for sentiment analysis. It has many applications in e-commerce, marketing, sociology and psychology research, as well as politics. Here are a few examples:

  • Customer feedback. Analyzing customers’ reviews can help companies to avoid crises. A major motivator for many customers ending their relationship with a specific company is their first negative experience with a company service or product. Constant monitoring of reviews and outer social media is the key to major insights, e.g. whether users like the new version of the company app or not. Sentiment analysis provides quick reactions, which allow companies to tackle an issue promptly.
  • Market research and analysis. Tracking customer sentiment helps to understand overall customer satisfaction, loyalty, and how likely they are to engage again in a day/week/month. The knowledge in these metrics may provide insights which are valuable to a company's marketing strategy.
  • Sociology. Sociology is interested in idea propagation through groups. Opinions and reactions to ideas in a given society are relevant to the adoption of new ideas. Insights on this process can be taken from sentiment analysis of reactions on blogs, tweets, etc. For example, sentiment analysis can shed light on behaviors across cultures and inter-group identity meanings through analysis of news media coverage during the Arab Spring. Another application of sentiment analysis is in exploring how influence is spread between communities and individuals in the blogosphere.
Let’s have talk
Let’s have talk

Interesting For You

Emotion Recognition

Emotion Recognition

It is obvious that emotions are peculiar to humans and some social animals, like apes, wolves, crows. Emotion recognition is an important part of the communication between people. The efficiency of humans’ interactions depends on how we can predict the behavior of the other person we are interacting with, and, as a result, adjust or change our behavior. Fear can indicate danger; satisfaction indicates that the conversation is successful. Emotion recognition is not an easy task, as the same emotion may be shown differently by different people. With this being said, most people have no trouble distinguishing basic emotions such as fear, anger, disgust, happiness, or surprise, to list a few examples. The question that arises here is whether we can teach a computer to recognize emotions. Because of the advancements made in recent years, the answer is yes. Automatic emotion recognition is a field of study in AI. It is a process of identifying human emotion by leveraging techniques from multiple areas, such as signal processing, machine learning, computer vision, natural language processing. But before we discuss automatic emotion recognition in detail, it is important to explore why this technology is necessary at all. Well, as we already mentioned above, emotions are a powerful source of information. Different surveys said that verbal components convey one-third of human communication, and nonverbal components convey two-thirds. So, successful human-computer interaction needs this channel of communication.

Read article

What is Natural Language Processing (NLP)?

What is Natural Language Processing (NLP)?

Natural Language Processing (NLP) focuses on using computers to understand and derive meaning from human languages. In this formulation, the challenge for NLP is an extremely difficult one. The average 20-year-old native speaker of American English knows 42,000 words (from 27,000 words for the lowest 5% to 52,000 for the highest 5%) and thousands of grammatical concepts. We need a large volume of linguistic knowledge for communication in a professional context, as well as writing books and articles, which we spend decades developing. On the other hand, in everyday life, our language needs are less complex; using a vocabulary of 3000 words is enough to cover around 95% of common texts, such as news items, blogs, tweets, and learning from a text context. This facilitates the process of meaning extraction for computers, especially in terms of performing “simple” tasks like summarization, relationship extraction, topic segmentation, etc.

Read article

Fraud Detection

Fraud Detection

Fraud losses are the subject of constant interest by organizations and individuals alike. Interest in this area is justified, given that in 2018, 49% of organizations said they had been victims of fraud and economic crime according to PwC. Worldwide card fraud losses totalled $24.26 billion in 2017 according to The Nilson Report. Fraud is a widespread, global issue. Organizations should always monitor their data in order to be fraud resistant. The automatization of this process can reduce costs and detect fraud faster. A powerful helper in fraud detection and understanding how fraud works is Data Science. In addition to detecting known types of fraud, data analysis techniques help to uncover new types of fraud.

Read article