What’s the sentiment on “sentiment analysis”?

“No takers for ‘Takers’”

I had already received three posts slamming the new movie ‘Takers’ on Facebook. I had determinedly ignored them and planned to watch the movie this weekend. However, this last post was from someone who I considered a movie buff with taste similar to mine. The decision was made. Instead, I watched South Africa massacre India in the first test.  ‘Takers’ could not have been much worse.

Information now-a-days is being created at a frightening pace. Over 30 billion pieces of content are shared every month of Facebook. There are 55 million tweets  a day and hundreds of millions of blogs and forums being read by billions of people. Personal opinions abound in the shape of reviews, ratings and recommendations.

This is a gold mine of information for analysts and businesses who want to feel the pulse of the collective consciousness of the world wide web.

• What do customers think of my product?
• Are they happy with the services?
• How do my policies or external events impact my customers’ perception of me?
• What do customers like about my competitors?

Online opinion is a powerful force that can make or break a product in the market place. Yet many companies struggle to make sense of this glut of bouquets and brick-bats that comes their way on the web. Is there a way to make sense of this jungle of information, find meaningful patterns and make fact-based decisions?

Sentiment analysis is a powerful, emerging field that attempts to analyze and measure human emotions and convert it into hard facts. On the web, it helps businesses monitor news articles, online forums and social networking sites for trends in opinions about their products and services or topics in the news.

“Stub hub” an online ticketing company used such monitoring tools to identify an upsurge of negative blog sentiment against the company after its decision to not refund tickets for a particular game. The company quickly went into damage control mode by offering discounts and credits to affected customers.

The US home land security has been working with a consortium of universities over the last 4 years to develop a software that would let the government monitor negative opinions of the US or its leaders in newspapers and other publications overseas.

Large companies like Wal-Mart use sentiment analysis to observe opinions about themselves as well as specific issues around them.

How does sentiment analysis work?

At a basic level sentiment analysis works by classifying the polarity of a given text, either in part or in full. The simplest algorithms work by scanning keywords to categorize a statement as negative or positive, based on a simple binary analysis. Example – “enjoyed” is good, “miserable” is bad.

However, this is just the first step. Businesses have realized that when opinions are aplenty, not all are equally important. Some opinions carry more weight than others. A negative review by Lady Gaga on Twitter will have a much greater impact on a restaurant than a tweet by an ordinary person. Sentiment analysis tools now allow users to generate ‘influence scores’ to identify people, blogs, forums etc. that are important to their company or industry.

A restaurant could leverage sentiment analysis to find people on twitter who have the highest percentage of tweets related to restaurants or that particular restaurant. Amongst these, a twitterer with higher number of followers would be graded higher on influence and then re-tweets would measure level of engagement from the audience. In this way, a business could identify opinion-makers that are relevant to them and assign them higher weights to calculate a more accurate indicator of sentiment.

Is sentiment analysis going to take off in 2011?

Sentiment analysis is a popular field with countless applications. However, the field is still in its infancy. The best tools available currently can achieve no better than 70-80% accuracy. There are many challenges in translating human emotions into binary variables. ‘Sinful’ or ‘Decadent; is a good thing when applied to a cake or a bakery. Irony, sarcasm, slang – how does a tool capture these accurately. Reliable sentiment analysis requires parsing many linguistic shades of grey.

As sentiment analysis algorithms and tools evolve, they would begin to yield more accurate results that may eventually be the way forward in this age of information overload.

Some interesting news and articles on this topic.

• Sentiment analysis comes of age
• Is the world ready for email sentiment analysis?
• IBM’s Watson Could Be Used For Sentiment Analysis In Social Media
• Seven Breakthrough Sentiment Analysis Scenarios

Related articles:

• Key Skills for a Successful Analytics Career
• Career in Analytics in a KPO
• Educating on Analytics
• 42 – the answer to everything!
• Books on Analytics