FreshNetworks Blog

This is the fifth post in our Social Media Monitoring – 2010 review series. In it we’ll be looking at the issue of duplication, one reason for some of the seemingly large differences between the seven leading social media monitoring tools under investigation – Alterian, Brandwatch, Biz360, Neilsen Buzzmetrics, Radian6, Scoutlabs and Sysomos.

We saw from the first test in our review that the different tools produced markedly different volumes for the search terms we were using – all associated with Starbucks. The smallest number of conversations was found by Biz360 and the largest by Radian6 – over 11x the difference.

So which tools are reflecting better the conversations and discussions about Starbucks? Is bigger necessarily better? Are the tools with the largest number of conversations the best? We don’t think so.

The difference in volumes is striking. If you were using Radian6 you would get the impression that eleven times as many conversations were going on about Starbucks and related terms than if you were using Biz360. There are many reasons for this and bigger is not in this case necessarily better.

Think about the following: retweets, spam, signatures, adverts. Should these be counted in your study or not? Different tools treat them in different ways and so, as we saw with the issue of location, the actual number of conversations is not always as it seems.

Firstly, there’s the source of the conversation – who did is start with? Is there more than one conversation around the same topic or is someone copying it? Is this the same tweet that’s been retweeted or is it a new conversation? These distinctions are important. If a Tweet contains certain keywords it is often retweeted automatically many many times by ‘bots’ which search Twitter for these terms and automatically reposts them.

How do you deal with spam and adverts – taking blog posts titles or key terms from them and posting them on other sites. Should these be included or not in your counts of conversations? Are they real conversations if they are automatically taken from your site and used on others? Are they important to understand if they are being used in spam sites. Or indeed sites or an unsavoury nature (you know the ones I mean!)?

How many times should a comment in a forum thread be counted? If a comment is repeated in different places or on different pages should it be counted as a new conversation? Indeed if one person posts their comment on multiple sites to try to drive traffic or showcase their point of view should all of these instances be counted as a new conversation?

You need a social media monitoring tool that deals with these and other situations. The tools that identify the most conversations are often not the most useful or accurate. They may include a range of conversations that are irrelevant, spam or double-counted. Whereas any organisation looking to understand what people are saying about your brand online wants a more accurate portrayal of what is being said.

Bigger is not necessarily better and duplication is a serious issue that needs to be addressed in any social media monitoring.

Next…

More detail on these tests, and the results,  can be found in our final report which will be available to download on Friday 16th April. We’re also holding a free social media monitoring breakfast seminar on 15th April in London, where we’ll be presenting the findings of our report, as well as giving practical tips and advice about social media monitoring and the best way to analyse results. This event is now fully booked but you can follow the results live as they are announced on Twitter from 08.30 (London time) on Thursday by following #smm10.

This is the fourth post in our Social Media Monitoring – 2010 review series. In it we’ll be looking at the  location of conversations.

In the last post in our series on social media monitoring we saw that the different tools being reviewed produced dramatically different results in terms of the number of conversations they identified for our key search terms associated with Starbucks. The smallest number of conversations was found by Biz360 and the largest by Radian6 – over 11x the difference.

When we were first analysing the results of our tests, it was pretty surprising to find that, using similar searches, we got such varied results from the different tools. Our research is intended to compare the tools without prejudice for those without prior knowledge of social media monitoring. And for an untrained eye, the reasons for these differences might not be clear.

One basic factor that needs to be considered when looking at social media monitoring is the location of the conversations about your brand. In fact, different social media monitoring tools classify the location of conversations in different ways using different approaches.

Location: Region, Geography or Language?!

Brands are generally interested in where the conversations are originating – a brand looking to understand sentiment about its product in the Argentinian market, for example, is probably less interested in conversations happening in the Philippines or Spain, even if they are all in the Spanish language. The social media monitoring tools use three main ways to determine the origin of the conversations, some more accurately that others:

1. Using the URL: It’s actually harder than you might think to use the URL of a site where conversations happen to determine the location of the conversation itself.  It’s not as simple as looking at .co.uk for the UK and .fr for France. Site URLs are not especially helpful in defining location.
2. The IP address: So each site has a unique IP address right? And these are allocated to counties? Well sometimes. However major social media platforms tend to be hosted in the US, whilst their users may be commenting from around the world. For example, Twitter gives the impression that all of its users are in the US.
3. Language being used: Some providers use the actual words in the conversations to identify the country.  This is pretty advanced and can work. But nuances in, for example, International English are extremely hard to infer location from.

Each of the tools addresses the issue of location in a different way. Looking through the conversations manually it’s easy to find examples that aren’t categorised correctly. As a result it makes the job of comparing the different tools much harder.

Next…

More detail on these tests, and the results, can be found in our final report which will be available to download on Friday 16th April. We’re also holding a free social media monitoring breakfast seminar on 15th April in London, where we’ll be presenting the findings of our report, as well as giving practical tips and advice about social media monitoring and the best way to analyse results. You can register for the event by clicking on the button below:

Read the other posts from our social media monitoring review 2010.

This is our second post from the Social Media Monitoring – 2010 review series.

In it we’ll be giving you an insight into how we have set up the comaprison of tools (which proved rather a challenge) and the volume of online conversations that each social media monitoring tool was able to uncover.

Setting up the search string
We decided to use Starbucks as a test brand for our social media monitoring because it’s a global brand that is frequently discussed online. Also, the word ‘Starbucks’ doesn’t have any other meaning or use other than being a brand/company name.

As well as tracking the word ‘Starbucks’, we also tracked the phrase ‘Flat White’, a new addition to the Starbucks coffee range which launched in December 2009 . We also tracked their new ready brew coffee, ‘Via’, which was released in the autumn of last year in the US and in March 2010 in the UK. We wanted to see what impact this new development was having on online conversations about the brand . Finally, because Starbucks is associated with its ‘Reward Card’ and the phrase ‘Fair Trade’ we tracked these subjects too. To keep things fair we created a similar search string for each tool.

It is important to note that some tools are capable of more sophisticated search strings than others. So we were testing to the lowest-common-denominator in this sense.

Comparison challenges
Although the tools are very different, we wanted to try and evaluate them all as fairly as possible. Thus the tools were used ‘out-of-the-box’, as they come, for the fairest comparison. Again, there are limitations with this approach. Some of the more sophisiticated options offered in some tools are only relevant to more experienced users. And some providers (e.g. Neilsen) are set up to provide a much greater level of analyst support than, for example more technology focussed firms like Radian6.

Our sense for the market is that most firms are still learning the art of social media monitoring and that tools are often managed day-to-day by people with only limited training in how to use them in anger. This drove our approach to the research.

Coverage
As the tools all have different coverage, whether it’s for different media or markets, we set up the same filters for each tool to create a comparable ‘universe’ of conversations for Starbucks. Our test was carried out using only the English language and for the same time period on each tool.

Sentiment analysis
One of the areas we wanted to test was the sentiment analysis accuracy of each tool. In order to compare the automated sentiment (ie, sentiment that is coded automatically positive or negative by the tool) with our own analysis we had to extract the conversations and manually code them. Some tools don’t allow you to extract certain conversations, others do. Where we weren’t able to extract sentiment for some reason, we’ve marked the tool:

Number of conversations
The seven tools gave very different results when looking at overall conversations -  the smallest number of conversations was found by Biz360 and the largest by Radian6 – over 11x the difference! But remember, more conversations is not necessarily better – there is often duplication.

*You can usually make arrangements with your account manager if you need more data.

Conversation types
When you compare the conversations by media type, again each of the tools shows quite a different result:

*Scoutlabs doesn’t allow you to extract Twitter conversations with sentiment. The tool does allow you to browse the latest twitter conversations though.

At this top-level, it’s clear the tools are each doing something quite different…

Next…
More detail on these tests, and the results,  can be found in our final report which will be available to download on Friday 16th April. We’re also holding a free social media monitoring breakfast seminar on 15th April in London, where we’ll be presenting the findings of our report, as well as giving practical tips and advice about social media monitoring and the best way to analyse results. You can register for the event by clicking on the button below:

Read the other posts from our social media monitoring review 2010.

This is our first post from the Social Media Monitoring – 2010 review series. In it we’ll cover the basics of social media monitoring.

Background

Social media tools make it possible for people to have conversations online. The uptake in conversations, comments and reviews has been explosive and the importance of these conversations is growing by the day. Among many other things, people are discussing brands, describing their purchase intentions and asking for assistance in making buying decisions or product support.

The opportunity for organisations is clear. They can now listen-in on the conversations of their customers, potential customers and other stakeholders in a way that was previously impossible. Through social media monitoring it is possible to gain insights from the conversations people are having online every day and to make improvements to products, customer service and marketing as a result.

Real-world ethnography has been around for a while – the process of analysing the context in which people act, usually researched by observing subjects in their natural habitat. It can teach us a lot about behaviour and influencing factors, however it is expensive and subject to The Observer’s Paradox (see also Schrödinger’s Cat).

Social media monitoring brings observational research to a mass audience. By tracking what is said in forums, on Twitter and in other social networks, brands can gain customer insight. But beyond getting geeky researcher’s excited, it can also offer very practical benefits to organisations. Customer service teams can listen out for customer issues online and then and resolve them. Competitor Intelligence departments can find out what customers are saying about competitors’ products. PR Managers can get early warning of pending PR disasters before they hit main-stream media and most of all, by listening first, companies can be better prepared to join online conversations and become social.

Social media monitoring clearly has tangible business benefits and as a result it’s a hot topic. Furthermore, the power and importance of what people are writing online is increasing. The reach an influential blogger can have is extraordinary. And according to Neilsen, consumer recommendations have now become the most powerful form of advertising (78% of people trust customer reviews). As a result companies need to monitor word-of-mouth more than ever.

Using search engines to monitor online conversations

Many online conversations can be accessed with ease and for free using Google or other search engines. Simply using your brand name as a search term, or using keywords that are associated with your brand (eg, for Starbucks you could search for “Starbucks” or “whole bean coffee”) you can find conversations that are related to your brand. Taking that process a step further you can set up Google Alerts so that you get an email when someone mentions your keywords.

However, if you search in this way you’ll probably end up with hundreds of thousands of returned results and a limited number of ways to analyse the data further. You will also get a mixture of professional and user generated content. It is possible to use some free buzz tracking tools to focus on certain areas. For example Omgili and Board Tracker are great ways to search forums. But until Google enters the social media monitoring market, the best way for enterprises to track social media is by using a paid-for tool.

The benefits of using social media monitoring tools

Social media monitoring tools deal with the two problems of searching and analysing the online conversation. The tools use similar web crawling technology to search engines in the way that they read online conversations. However, unlike search engines, the tools clean, de-duplicate and categorise the conversations and then store them in a database.  As our report and future posts will show, some tools do these things better than others.

Social media monitoring tools also allow you to enter search terms into the database so that you can customise the way you view the results. The tools count the conversations that contain your search terms and provide you with the ability to display this information in graphs and charts.  Most tools also allow you to divide by location or media type (eg, Twitter or blogs) and at the cutting edge, some social media monitoring tools provide workflow management process that can help you disseminate conversations within your organisation, others are starting to combine buzz tracking with CRM in a bid to create single-customer-view Social CRM. And there are some tools that allow you to respond to conversations across the web from a single dashboard.

One key feature that marketeers have been most keen on is sentiment analysis.

What is sentiment?

Sentiment is a thought, view, or attitude that is often based more on emotion than reason. In the context of social media monitoring, it is the concept of deciding whether a specific online conversation is positive or negative. This is really useful in helping you determine the themes and topics that are driving both good and bad conversations about your brand.

Sentiment can also allow you to track the overall impact of marketing campaigns or news about your brand. We suspect the main reason people have latched onto sentiment is because it gives the impression that the plethora of web conversations can be summarised in a single number. Businesses love to track numbers and sentiment is often the KPI of choice for social media.

This is dangerous. Sentiment is more nuanced than a single number and using an automated tool to assess how people feel puts too much faith in the today’s software. We don’t believe that the tools on the market have nailed sentiment analysis yet. The tools can be extremely valuable, but it is important to understand their limitations as it is to understand their capabilities.

One piece of advice - it’s not about the bike

The most important thing to bear in mind when choosing a social media monitoring tool is that ongoing human interaction and interpretation are essential to get real value. If there is one mistake that companies are making it’s that they buy into a dashboard expecting insights on a plate. Months later they look back and wonder why the dashboard hasn’t changed their business.

Buzz tracking opens up opportunities for insight, but it is worthless without sufficient people resource and internal processes to act on information.

I am biased. My background is research (FreshMinds, our sister firm, has been twice named UK Research Agency of the Year by the Market Research Society) and we’re not selling a tool. Rather we help companies select the right tool and help them get value out of it on an ongoing basis. But I think you’ll find most of the software vendors will concur that their happiest clients are the ones who have properly resourced the listening effort and invested sufficient time in interpretation, dissemination and action. After all, what’s the point in listening if you never act on what you hear?

The next blog post in our series will be about setting up each of the leading tools to get the most out of them.

Social media monitoring breakfast seminar

We’re also holding a free social media monitoring breakfast seminar on 15th April in London, where we’ll be presenting the findings of our report, as well as giving practical tips and advice about social media monitoring and the best way to analyse results. You can register for the event by clicking on the button below:

Read the other posts from our social media monitoring review 2010.

Over the last few weeks we’ve been carrying out detailed tests and analysis on 7 of the leading social media monitoring tools – Alterian, Brandwatch, Biz360, Neilsen Buzzmetrics, Radian6, Scoutlabs and Sysomos. Our aim is to provide an in-depth comparison of buzz tracking tools that accurately depicts their individual pros and cons.

We’ve put the tools to the test by tracking well-known international coffee company Starbucks. We compared over 19,000 online conversations, giving us some really unexpected results and highlighting some staggering differences in the way each tool performs.

Over the next few weeks we’ll be blogging about our findings, and at the end of the series you’ll be able to download the full report for free. We’ll cover:

• The basics of social media monitoring
• An overview of our results
• The location of online conversations
• Social media monitoring and duplication
• Data latency
• Sentiment analysis
• International/multilingual social media monitoring

We’re also holding a free social media monitoring breakfast seminar on 15th April in London, where we’ll be presenting our findings, as well as giving practical tips and advice about social media monitoring and the best way to analyse results. You can register for the event by clicking on the button below:

So what are social media monitoring tools?

In a nutshell, social media monitoring tools pretty much do what they say on the tin – they monitor online conversations taking place through social media.  They track anything that’s being said about your business or your brand on blogs, forums, Twitter and other social spaces. Each tool is different, varying in complexity and in the way they gather and process the information, as we will show you over the next few weeks.

Our sister company, FreshMinds Research, has been using social media tools to generate customer insights for years. As we’re a social media agency, we usually work with FreshMinds Research to conduct social media audits or monitoring when establishing a  social media strategy for clients. So over the next few weeks you’ll benefit from the unique findings of a research company working in collaboration with a social media agency.

We’ll start with the basics and work through our research step-by-step. If at any time you want us to explore a certain aspect in more detail, please let us know. Our next post will explore the basics of social media monitoring.

Read the other posts from our social media monitoring review 2010.