Three months ago we started a small experiment with one of our online communities at FreshNetworks. The site had been in seeding phase for about 10 weeks and was starting to get some activity and some real traction, there were loyal members and real conversations starting to grow. At this stage we wanted to grow the number of members and also to grow the quality of the contributions to the site. There are a number of ways you can do this, but one of the real benefits of online communities is the ability to drive natural, organic, search. We wanted to test this and to start to build an ROI model for brands.
We decided to put paid and organic search head-to-head. To devote a modest sum to Google AdWords and also to monitor the organic search on the site. We didn’t spend any more time than usual optimising the content on the site, with the community manager working to organise content and correct spelling but not to make sure that user-generated content was keyword-rich. We wanted to do a real test – putting real organic search against paid search.The experiment is to run for six months and we are now halfway through this.
My hypothesis was that organic search would take time to grow and that at the three-month stage we would still be seeing significantly better results for paid search – higher traffic, more conversions to signing up, greater loyalty and deeper, longer visits. Then during the second three months of the experiment I’d expect organic search to really take the lead and over the six months this to be the clear winner.
Things never work out quite like you expect them to. And in this case they are actually much more interesting.
We’re going to be working on this experiment for another three months so we don’t want to reveal too many details, but initial results are, in part, as expected, but in other areas really not:
More visits to the online community come from paid search than from organic search (although if we take just the last month then this is reversed). In total about 1.5 times as many visits came from paid search as from organic search in the last three months
But, those coming from organic search are more likely to sign-up to become members of the community. The difference is marginal at the moment, but there is a clear trend to higher conversion rates for organic search visits
Those coming from organic search are spending much more time on the site and visiting many more pages per visit. In fact time on site is almost three times as long for organic search as paid search, and they visit almost twice as many pages per visit
Finally, those users that first came to the site by organic search are more likely to become active Members of the online community.
So the results are not quite what I expected, but a clear trend is developing. Whilst paid search delivers more search traffic (although this balance is changing) the quality of organic search is significantly better. Traffic to the site is important, but in an online community people signing-up, spending time on the community and actively contributing is much more interesting. For this, organic search seems to be significantly better.
Stay tuned for the final results of the experiment in three months time when we should be able to show the real power of organic search. These initial results are exciting.
A new report out this week suggests that the biggest hindrance in the adoption of social media is a lack of knowledge. The statistics, from MarketingSherpa, look at the barriers that organisations cite as having stopped them from adopting social media marketing techniques in their firm.
Of the 1,886 firms interviewed who had not embraced social media as a marketing tool, 46% cited a lack of knowledgeable staff as being a significant reason for this. The second most popular response was an inability to measure ROI, with 43% of firms claiming that this was preventing them from adopting social media marketing.
That these two reasons are cited by firms are not a surprise to us at FreshNetworks. We know that clients sometimes need support getting up to speed with the latest changes and developments in social media and how it can help marketing. This isn’t surprising. Things are changing quickly in social media, as we’ve seen in the UK over the last two weeks where a number of popular TV and radio figures have started using Twitter as a part of their shows, thus propelling it into the mainstream. It can be difficult for firms who are busy focusing on their own plans, products and direction to step back and take in the changes as they are happening. Even more so in the current economic climate.
ROI is also a serious issue for marketers, again even more so now when they need to maximise the impact of every piece of spending. There is a lot of discussion of measurement in social media. At one end of the spectrum are people who think that measuring web traffic and statistics gives you a real insight into what people are doing in social media. At the other end are those who think that a firm should be more business-focused, establish a set of objectives for their use of social media and then measure against these. We tend to think that a mix of the two is the best solution. No business should embark on a social media marketing plan without some very clear business objectives in mind. One of our clients wants to increase repeat purchase, for example; another wants to create a target number of new ideas for their product team. Alongside these business objectives are a set of community measures and metrics that make sure we’re on track to achieve what we want.
So, it’s not a surprise that lack of knowledge and lack of clarity of ROI are the two biggest stumbling blocks for firms embarking on social media marketing. But they really shouldn’t be.
And if you’re one of these firms that’s thinking about adopting social media marketing, we’re here to help. We’ve got a wealth of articles, reading, case studies and posts on both topics so take a look and see if we can help.
We’ve written in the past about how to measure ROI in online communities. It’s a subject we return to often with our clients at FreshNetworks. The online communities that we build for them all tie into over-riding business aims, and so measuring the impact is important. We can, of course, measure specific insights that they get from the community, the benefit of the qualitative information internally, the benefit that support communities have or any uplift in sales from the community. But there is a holy grail in online communities and indeed across social media – measuring ROI at a granular level; identifying influential members, recognising that these may not be those who post most.
In previous posts, I’ve suggested that what we need to do is develop a weighting that could be applied to individual members showing how important and influential they are. An analysis of the quality (not quantity) of their connections and of their connections’ own connections. A difficult and time-consuming task. And one that Google may have solved.
The latest edition of Business Week reports that Google has a patent pending on technology that measures influence in social networks. It apparently measures both the direct influence people have in terms of volume of connections, but also how successful your posts and feeds depending on how many people open, read and forward them.
The new technology could track not just how many friends you have on Facebook but how many friends your friends have. Well-connected chums make you particularly influential. The tracking system also would follow how frequently people post things on each other’s sites. It could even rate how successful somebody is in getting friends to read a news story or watch a video clip, according to people familiar with the patent filing.
It will be intriguing to see how this technology develops and what Google use it for. The measurement of influence online is of critical importance to brands, marketers and advertisers alike. Brands want to know how influential people who talk about their brand are, or how influential the people in their online community are. Marketers want to find these influential people and focus on what they are saying and what brands are saying to them. Advertisers can use this information to help target ads across social networks.
Of course, there must also be a benefit for Google. Given that their attempts at running their own social networks have not had the same success in sheer numbers as the likes of Facebook, MySpace and Hi5, Google is looking for other opportunities to capitalise upon this growing trend. They’re doing what they’ve done to the web – they don’t provide all the content they just offer a great way to search and prioritise it. So Google could become the Google of social media.
The latest edition of the McKinsey Quarterly includes the results from their global survey of Web 2.0 in firms. The survey documents the developments that we see at FreshNetworks – more firms are using more Web 2.0 tools for more complex business purposes. McKinsey go even further, noting that a significant finding from this year’s survey is that:
Companies that are deriving business value from these tools are now shifting from using them experimentally to adopting them as part of a broader business practice.
Web 2.0 tools are starting to enter the mainstream in business, those who trial them find them beneficial and want to look at ways they can use these tools across their business, helping them meet multiple aims.
If you’re interested in some of the detail of the McKinsey study, I’d suggest you go to the article on their site here (you will need to sign-up, although it is free). However, for me the most interesting findings are:
More community-based tools are growing in their use. In 2008, 34% of businesses studies used blogs (compared with 21% in 2007); 32% used wikis (compared with 24% in 2007)
Web 2.0 tools are popular both for internal purposes (94% of firms studied) and for interfacing with customers (87% of firms studied). When they are being used for the latter purpose, this is primarily to improve service to existing customers and then as an acquisition tool
Blogs were more popular in Asia-Pacific and India; social networking particularly popular in North America and China; and, mash-ups and rating more popular in Europe
The biggest barriers to using Web 2.0 tools are a lack of understanding of the financial benefits (28% of respondents), internal cultural barriers (22%) and lack of skills (17%)
This last point, the barriers to adoption, show the areas where we as an industry need to focus our efforts to help clients. We have written before on this blog about measurement and ROI in online communities and in social media (see posts here, here and here) and it seems that this is the biggest barrier that firms need support with. Perhaps as these firms move from trialling the use of new tools, to using them for specific business purposes, the measurement of how they contribute to these will be easier.
I’m returning to the subject of measuring the ROI of online communities after an all day session with a client today. We spoke a lot about how they are to measure the benefit of the online community they are launching and about the metrics they are going to measure and the KPIs they will report on.
I’ve worked with a lot of clients and read a lot of things on metrics and KPIs and have noticed that many people confuse these two terms. Really they’re very different. The best description I have seen of how and why they are different is a presentation from Dennis Mortensen, that I’ve copied below. It’s worth a look as it quickly and clearly shows how and why metrics are different from KPIs.
This ties in to the debate on measurement in online communities. There are many things that you can measure (metrics) but only some things that show the impact your community is having vis-à-vis the business or other objectives of the site. These are KPIs and are what you probably will be using to show the success of your online community.
So work out all the metrics that you can track with your online community, and work out what KPIs you are going to report on to show the success of your community. Both are measurements but only the latter is really a measurement of success of ROI.
The answer to this question is simple and complex. The simple answer is that you make sure you establish a set of criteria when you launch the community and then judge success against these. The complex answer come with some of the ways in which you may judge success.
The workshops that I’ve run with clients at FreshNetworks only prove that every online community has a different set of aims and so needs to be measured in different ways. However, a few basic principals exist that show the possible ways in which you can measure ROI.
First of all it’s useful to distinguish between actual measures of ROI and measures of the health of the community. As people who build and manage online communities, at FreshNetworks we are interested in metrics such as number of people who log-on each day, proportion who make contributions, average amount of time spent on the community and other such measures. These help us to benchmark the community against others that we have run and understand how each community is developing. It’s also useful for us to be able to build a set of metrics across community types so that we can better understand consumer behaviour in them. In most cases however, such measures are not (just) what we use to measure ROI.
A client may want to own the debate in a certain area. It may want to work with its customers to improve or develop its product in an online research community. It may be looking to engage a new audience demographic. It may want to amplify the word of mouth for its services. There are many reasons why people may set up online communities and understanding these on a very granular level is the first stage to effective measurement of ROI.
Then it’s just a matter of work out which factors will help you to understand how you are performing against these aims. There are really two types of data you can measure in a community
Qualitative data – this could mean measuring the quality of conversations about the brand, the response to the brand and competitors, the increase in actual conversations and dialogue about the brand, the extent to which these conversations are positive, the quality of ideas generated. Whatever you’re trying to achieve there will be qualitative measures that help you to measure this. Of course the problem with qualitative data is exactly that – it’s qualitative. Although this is the nature of online communities – they’re a space for conversations and discussions. So we often find that establishing detailed qualitative measures of success can be the most revealing and the most informative.
Quantitative data – to some extent this is much easier to measure, the trick is measuring the right thing and not measuring too much. From Google Analytics to more bespoke packages there are ways of quantifying a member’s interactions with the site and building a set of measures from this. It may be about reaching more people (possibly of a certain type), or about engaging the same people for longer. Quantitative measures can be good for understanding how people interact with the site.
This is really the fundamental difference between these two types of data and these two ways of measuring. Qualitative data helps us to understand the quality and usefulness of the contributions to the online community. Quantitative data, on the other hand, shows us how people have interacted with the site.
Most are important to measure and different clients have different aims from their online community – this means that different baskets of measures are needed. Sometimes you’re more interested in the quality of debate and so want to include a greater number of qualitative measures; sometimes you’re more interested in the reach and penetration of the community and so want to include more quantitative measures.
Whatever you measure it is that first question that’s important: what are you trying to acheive with this online community; how will it help your business? Answering that is where it all begins and one of the most critical stages in planning a successful online community strategy.
Over the weekend I wrote about Ian Schafer, who sold advertising space on his Twitter feed for a month on eBay; the item closed at $1,082.01 (see post here). This experiment interests me for a couple of reasons:
Ian’s Twitter feed is followed by 520 people and so this works out at about $2 per person the advertising will reach over the month
The advertising space was sold by auction and there was considerable competition for it. The final price appears to be one that the bidders felt appropriate and as such represents the market value of this advertising space
On one hand I was surprised by the relatively low fee paid for the advertising – with Twitter, people with similar interests and background tend to follow each other. So I’d expect Ian’s followers to be a fairly discrete group and one that could suit really targeted advertising. But the auction on eBay was open and included a number of bidder, so it may be that this is a fair price.
So, an open auction sold off advertising space that would effectively reach an individual’s social network. If we believe that this is a fair indicator of value of this space then it could be that this value is in fact the current true market value of advertising to Ian’s social network on Twitter. In other words this value is a reflection of the influence Ian has.
I’d love to repeat this experiment with a range of other users, or indeed on other social networks. Flooding the market in this way would, of course, have an impact on the prices paid, but it would be the relative prices that interested me most. There is much talk about measuring influence in social networks, and to me allowing the market to decide a fair price for advertising on somebody’s page would be an interesting proxy for such measurement.
The more influential somebody is, the more people that will follow them, the more those that follow them will return and the more trust they will put in what they see there. The price that somebody is willing to pay to be associated with such a person, and with their social media presence, could be an interesting commentary on the influence people believe they have. Letting the market decide the (relative?) value of somebody’s presence and contacts in social media could allow us to start to understand influence.
Next week I’m speaking at the SocialMediaInfluence conference in London on Measuring Influence and Audience online. It’s a tricky subject and looking around today I have been unable to find any examples of an approach which has been successfully and repeatedly applied.
The problem appears to be that whilst there are a whole range of metrics that we can measure in social media (see The Social Organisation blog for a fairly comprehensive list) but none of these truly gets to the crux of the problem. What we want to do is know is to measure the influence that a single blogger, commenter or video upload has. What is the value of a blog post praising Coca-Cola in terms that Coca-Cola could understand and measure. As many of our clients ask us, what’s the ROI of encouraging this kind of activity.
The answer is that it’s difficult to measure, not because we don’t have a range of metrics (we do) but because at the moment our understanding of what causes a particular post or a particular individual to be influential is limited. We can measure proxies, such as trackbacks, links to the site from other sites (and the number of links to the sites that link there). But these really only reflect an inherent influence that we still haven’t measured.
What we really want to know is how influential is everybody that is exposed to an piece of content, and how influential are all the people they influence. Of course calculating this number would be difficult if not impossible. And the information you need to gather would be huge. It really wouldn’t be worthwhile.
Which is why some more basic measure is needed. Take the sites like Dell’s Ideastorm and MyStarbucksIdea. These get peers to vote posts up or down depending how relevant they think they are. You can then migrate only the more popular posts to the front page or the top of the list. This kind of rough approach might be a crowd-sourced way of measuring influence. We know that the most popular posts are those that people in the community think the brand needs to listen to most. Perhaps this is the only measure of influence we need.
People talk about your brand everywhere online. A search in Google would find hundreds if not thousands of places where people are talking about or commenting on your brand. You could be mistaken for thinking that the rise of social media has led to people being exposed to more discussions about your brand that you don’t control. And you might think this is dangerous and get concerned about each of those hundreds or thousands of discussions.
But the mistake in this analysis is to assume that every discussion is equal. They’re not. A blog that criticises your brand but is read by hardly anybody is of very little importance to you. In fact the majority of discussions about your brand will each only have a very small audience. You don’t need to occupy yourself with them all, but rather with those which are repeatedly discussing you, or those that have significant influence.
This raises the question of how you measure influence. Aside from a very small number of well read bloggers, it is difficult at first glance to identify how influential somebody is online. You could look at how many posts they’ve made on a forum – but if nobody is reading them then that’s not a great measure.
Of more use is to measure the network effect that an individual has. How many forums or blogs are they active on and how many people read their posts on these sites? How many social networks are they a member of and how many friends do they have there?
These measures are much more useful when it comes to measuring influence – if somebody posts something about your brand how many different people will see it and how much will they trust the poster?
To some extent these measures can be made into a simple formula. Take each social network that somebody is a member of and give it a weighting for importance – so one that is more specific to your brand or product might be weighted more than a generic social network. So, taking a brand that sells baby products, a social network specifically for mums might be weighted twice as heavily as a Facebook. You can then multiply this number by the number of contacts you have on each site to get a measure of influence by site. And sum them to get a measure for the poster overall.
For example, Poster A has 100 friends on a Mum’s site (weighted 2) and 350 friends on Facebook (weighted 1) would have a total influence measure of 550. Compare this with Poster B, with 150 friends on a Mum’s site but only 70 friends on Facebook. Their influence measure would be 370.
So Poster A is about half as influential again as Poster B.
This is a good basis for a measure of influence and a way of finding the people you need to track and engage online. The problem at the moment is that the data needed to make this formula work isn’t readily available. Back to the drawing board maybe.
Mark Beth Kemp from Forrester thinks that the lifetime value of a customer is no longer enough. Both this and the ROI model are deficient because they don’t take account of customer dynamics.
One interesting example she gave was about the launch of the Wii. They wanted to launch a games console that appealed to an expanded market and to sell the benefits of gaming. One critical barrier they saw was the difficulty of persuading parents of children (specifically mums) that this was a new games console that kids should enjoy using. They turned this barrier into an opportunity and launched the alpha mums programme. Finding the most active mums online and seeding them with the product. These mums were encouraged to host parties where other mums could try the Wii. The result was incredible and shows the power of harnessing customer dynamics. One alpha mum alone sold 200 consoles to her email contacts.
For Mary Beth, your new high value customers are not those who spend the most with you over their lifetime. It’s more complex. The highest value comes from your Ambassadors – those who spend a relatively large amount personally and also have a lot of social value – they have a lot of contacts and have influence over these people. Finding exactly who these people are can be complex although really it just relies on finding out where they socialise (online or offline) and how many people they are in contact with there. This gives you their reach. You can then calculate how valuable their recommendations are likely to be and that gives you a crude measure of social value. Perhaps a developing but more useful measure for marketeers.
Image via Wikipedia
The latest edition of the 
The answer to this question is simple and complex. The simple answer is that you make sure you establish a set of criteria when you launch the community and then judge success against these. The complex answer come with some of the ways in which you may judge success.
