Tag Archives: metrics

Open Source Community Metrics

Today at Open Source Bridge, I’ll be leading a session about Open Source Community Metrics: Tips and Techniques for Measuring Participation at 3:45pm in B302.

Do you know what people are really doing in your open source project? The best thing about open source projects is that you have all of your community data in the public at your fingertips. You just need to know how to gather the data about your open source community so that you can hack it all together to get something interesting that you can really use. Having good community data and metrics for your open source project is a great way to understand what works and what needs improvement over time, and metrics can also be a nice way to highlight contributions from key project members. This session will focus on tips and techniques for collecting and analyzing metrics from tools commonly used by open source projects using examples from what I’ve learned doing MeeGo metrics.

A few topics:

  • General guidance for coming up with a set of metrics that makes sense for your project.
  • Tips and techniques for collecting metrics from tools commonly used by open source projects: Bugzilla, MediaWiki, Mailman, IRC and more.
  • General approaches and technical details about using various data collection tools, like mlstats.
  • Techniques for sharing this data with your community and highlighting contributions from key community members.

For anyone who loves playing with data as much as I do, metrics can be a fun way to see what your community members are really doing in your open source project. It’s like people watching, but with data.

MeeGo Community and Metrics Presentation Videos

I just realized that while I was frantically catching up from my much needed Thanksgiving vacation right after the MeeGo Conference that I completely forgot to post the videos and presentation materials from my sessions at the conference. I presented two sessions:

State of the Community


Presentation Materials (PDF) and More Info

An Inside Look into the MeeGo Metrics


Presentation Materials (PDF) and More Info

You can also watch videos of all of the other sessions at the MeeGo Conference. For those of you interested in online communities, you should definitely watch Dave Neary’s Community Anti-Patterns presentation.

Community Manager Tip: Automate Tasks

This is one of those things that can be hard to make the time for, and it is more technically challenging than many other community management tasks, but over the longer term it can really pay off in time savings. Think about those repetitive tasks that you do over and over – monthly community metrics come to mind as one of the most common examples. I started really looking at the time it was taking me to compile my monthly metrics, and I vowed to start automating as much of it as I can. Right now, I’m getting close to having most things at least partially automated.

Here are a few ways to automate your community tasks:

  • APIs: Many APIs are easier to use than you might think and can be a great way to suck data out of commonly used services like wikis. You can often format a URL and get a file without any programming required.
  • Database queries: Yes, I’m picky, but I’m almost never happy with the reporting tools in community software, and I always end up needing a few database queries. If you don’t have the technical skills, find a geek to help you write a few queries that can be set up to automatically run every month and email you the results.
  • Scripts: I have one gigantic shell script and a couple of smaller ones where I dump a bunch of commands that run other stats gathering programs, do database queries, download data into files (parsing if needed), etc. This requires a little programming knowledge, but it’s easier than it sounds.
  • Schedule: Many web hosts make it easy to schedule your scripts to run every hour, day, week, month or on some custom schedule with a nice, user-friendly interface into cron (which can be incomprehensible to some people in its native form).

Additional Reading

Part of a series of community manager tips blog posts.

Image by hobvias sudoneighm used under the Creative Commons Attribution 2.0 Generic license.

Community Manager Tip: Have Great Metrics

For community managers, having excellent metrics is one of the best ways to show your progress and help justify your efforts to management when talking about budgets and staffing for the community. It provides an early warning system and diagnostics for potential community issues, which gives you time to make corrections before things get too bad.

Here are a few tips for having great metrics:

  • Measure many details to help you diagnose issues, but focus on a smaller subset that are used to determine success / failure.
  • The smaller subset should map to your goals and strategies for your community as a whole to show that you are meeting your objectives.
  • Share your metrics with your community. I have a public report with the data and a second internal report with more detailed analysis and suggestions for where the team can improve.
  • Measure across a few categories. I use awareness, membership and participation / engagement.

Additional Reading

Note: I hope to make this into a series of short posts (approximately weekly) to share quick tips for community managers.

Image by Flickr user Kevinzhengli used under the Creative Commons Attribution 2.0 Generic License.

Online Community Metrics

This morning, I ran across a list of social media metrics on the Page One PR site, and I realized that I spend quite a bit of time talking to clients about success metrics, but I haven’t spent much time writing about metrics on this blog. In a previous post, I talked about my general guidelines for online community success metrics:

The metrics that you select will depend on your specific goals, but common community metrics include page views or visits, new member sign ups, and participation (new posts or replies). It is easy to go overboard and measure everything; however, I recommend that you pick a couple (no more than 4 or 5) of the most important measurements to use to report to management on your success. You should have an analytics package or reporting tools that allow you to drill down for more details that you can use to help troubleshoot issues and understand the data, but use these as background materials for your team.

In other words, your success metrics are a small number of items that you use to determine success or failure over a period of time. You should measure many other items that you can use as indicators for what works or what doesn’t work, but make sure you separate what you are measuring because it helps you do your job vs. what metrics you are using to determine success.

Now, let’s get more specific. Online community efforts, including social media, can be very difficult to measure. I try to focus success metrics across three areas: awareness, membership, and engagement.

Awareness

Awareness is focused on getting people to notice your community and visit it to learn more. I typically use a general purpose website analytics package, like Google Analytics, to track visits to the community or page views as my primary measure of awareness.

Membership

Membership looks at the people who are paying attention to your community on an ongoing basis. These are the people who take the time to sign up and join the community as members. I usually use the number of new members or the total number of members of the community as the success metric for membership.

Engagement

Engagement is all about the conversations that people are having in your community and their interactions with other community members. The number of discussions, replies, or comments are typical ways to measure engagement in a community.

While awareness, membership, and engagement are long-term metrics, you should also have success metrics for shorter-term programs that also tie back into your overall community metrics. For example, you will want to keep track of when you do any kind of outreach (online or offline) and watch relevant success metrics for your community both before and after the outreach activity. This additional measurement will help you determine what methods of outreach work best for your community and will determine the success or failure of a shorter-term program.

While the success metrics mentioned above are geared toward online communities, you can use a similar approach for social media efforts. Awareness could include mentions of your company or product name across various social media sites (Twitter, blogs, video, etc.) Membership might include Twitter followers or RSS feed subscribers for your blog. Engagement could include comments on your blog posts and Twitter @replies.

Exactly how you will measure these success metrics depends entirely on your community dynamics, the community platform capabilities, and what activities in the community are the most important for your members. Don’t get too caught up in the examples I’ve listed here. The important part is finding a way to measure all three of these areas: awareness, membership, and engagement. Exactly how you will determine success will probably be a little different for each community.

Community ROI, Metrics, and Events

Just a quick post to point people to the list of ForumOne Online Community Events in 2008. For anyone wanting to learn more about online communities or network with other community managers, these might be good events to attend.

ForumOne has also released a couple of interesting reports:

Related Fast Wonder Posts: