FreshNetworks Blog

Our sixth post from the social media monitoring review 2010 series will look at the issue of data latency.

While most tools prioritise key websites to ensure the fastest possible upload of conversations, we found that some tools can take several days before the  conversation that’s being held online is available in the tool. This delay is known as data latency.

Of the seven tools we tested, we found that Brandwatch was the fastest at searching for and processing new online data, while some parts of Nielsen Buzzmetrics proved the slowest at collecting up-to-date information.  It’s important to note that Brandwatch doesn’t cover as many geographies and conversation types as Nielsen does.

As mentioned in previous posts, one of the ways that social media monitoring tools gather data is by using similar web-crawlers to those that Google uses to produce its Search Engine Results Pages (SERPs). This is an automated process that copies content from a list of websites into the tool.

Once the conversations have been collected they are processed – sometimes by analysts but mostly by automated processes. The speed at which conversations are collected by a tool is limited by the frequency of the web-crawlers and the length of time it takes the tool to process the data.

This has an obvious impact on clients wanting to look at up-to-date conversations. It can also skew historical data as it’s possible to look at a conversation trends for the last few days, but then the next day more conversations may arrive from the previous day, changing the results.

People carrying out social media monitoring need to be aware of data latency and to keep it in mind when using the tools to track online conversations.

Tracking specific influencers

In the video below, Charlie, one of the Directors at FreshNetworks,  talks through an example of a client who needed to track a small number of key influencers. They could not afford to wait days or even hours for updates and had to find a unique social media monitoring solution.

The Hare and the Tortoise

Social media has driven a yearning for real-time information. A desperation to know exactly what’s going on right now. As a result you might believe that a faster tool is the better tool. However, as with the other comparative measures between the tools, it depends on your business need.

Our experience of the different social media monitoring tools suggested that high latency was often the result of more sophisticated data processing and de-duplication. Thus if your goal is to track what’s going on with minimal effort, or to see only the really important conversations, you may be better off with an apparently ‘slow’ tool because it will cut out more of the online waffle.

To give an example, some of us at FreshNetworks like to read the most important blog posts from all of AdAge’s 150 top blogs. To do this we use Postrank to filter out the most popular posts and then Feedburner to email them once a week. It can occasionally feel like you’re a day behind on the news, but it’s certain that you will be seeing the most important posts by the end of the week. This system only works because Postrank tracks whether other people think a post is great – a process that performs better after a post has been live for a couple of days. Hence, in this case, higher latency drives better results.

Our next post will look at sentiment analysis across the seven tools.

Read the other posts in our social media monitoring 2010 review series.