Tag Archive for 'metrics'

Open Source Community Metrics and State of the Puppet Community

Many of you probably know that I’ve spent the past week in Belgium for Puppet Camp Ghent and FOSDEM. I’ll be writing a blog post on the Puppet Labs blog later this week to talk about Puppet Camp Ghent, but I wanted to at least get my presentations out here while I finished writing the longer post.

Puppet Camp Ghent was amazing. I saw a few old friends and connected in person with quite a few community members that I had not yet met in person. Overall, I was very happy with the event, and the people at HoGent were great hosts. There were so many amazing presentations, and we’re getting them uploaded to the Puppet Camp page as soon as we get the slides from the speakers. Here is the presentation that I delivered on the State of the Puppet Community.

state-of-puppet-community

I had an amazing time at FOSDEM, too. I helped facilitate the Configuration / Systems Management DevRoom on Saturday along with a DevRoom dinner that evening. I love working in such a collaborative industry. The DevRoom and the dinner were organized collaboratively with our primary competitors, but we all worked together to pull it off in a way that benefited the industry. Aside from the DevRoom, I got to see a lot of old friends and had a great time!

At FOSDEM, I also gave a short version of my Open Source Community Metrics talk. If you are interested in open source metrics, you might rather look at the longer version that I presented at LinuxCon Barcelona in November. I also had a great conversation from Jesus at Bitgeria, and they are doing some awesome stuff with open source community metrics that you should look at if you are interested in metrics.

Next on my agenda are trips to Stockholm, Sweden and Oslo, Norway for two more Puppet Camps in the next two weeks before heading back home to Portland.

Blogging Elsewhere

I’ve been neglecting my blog, but it isn’t because I haven’t been blogging. I just haven’t been blogging here :)

Here are a few posts that I’ve written over the past couple of months for work as part of my Community Manager gig at Puppet Labs:

I also wrote a few posts on my cookbook site, What Dawn Eats. Don’t forget that you can buy the cookbook (available in paperback, kindle edition or PDF)!

Open Source Community Metrics: LinuxCon Barcelona

I wanted to share the presentation that I will be giving today at LinuxCon Barcelona at 1:20pm, Open Source Community Metrics: Tips and Techniques for Measuring Participation. This is similar to the presentation that I gave a few weeks ago at the LibreOffice Conference in Berlin, but I have added some new data and included different examples. You might also be interested in seeing the Puppet Community Metrics that I recently started posting on the Puppet Labs website.

You can download the presentation from SlideShare.

Talk Abstract:

Do you know what people are really doing in your open source project? Having good community data and metrics for your open source project is a great way to understand what works and what needs improvement over time, and metrics can also be a nice way to highlight contributions from key project members. This session will focus on tips and techniques for collecting and analyzing metrics from tools commonly used by open source projects. It’s like people watching, but with data.

The best thing about open source projects is that you have all of your community data in the public at your fingertips. You just need to know how to gather the data about your open source community so that you can hack it all together to get something interesting that you can really use. This session will be useful for anyone wanting to learn more about the communities they manage or participate in.

LibreOffice Conference: Open Source Metrics

Today at the LibreOffice Conference in Berlin, I will be presenting a session titled, “Open Source Community Metrics: Tips and Techniques for Measuring Participation.” It has tools, techniques and examples of metrics from the LibreOffice project, Puppet and MeeGo to illustrate several ways to gather and interpret the metrics for your open source project.

If you are interested in watching the presentation, it will be on the LibreOffice Conference live stream starting at 18:00 CEST in Berlin or 9am Pacific time.

You can also download a copy of the presentation from SlideShare.

Talk Abstract

Do you know what people are really doing in your open source project? Having good community data and metrics for your open source project is a great way to understand what works and what needs improvement over time, and metrics can also be a nice way to highlight contributions from key project members. This session will focus on tips and techniques for collecting and analyzing metrics from tools commonly used by open source projects. It’s like people watching, but with data.

The best thing about open source projects is that you have all of your community data in the public at your fingertips. You just need to know how to gather the data about your open source community so that you can hack it all together to get something interesting that you can really use. We’ll start with some general guidance for coming up with a set of metrics that makes sense for your project and talk about the LibreOffice community metrics. The focus of the session will be on tips and techniques for collecting metrics from tools commonly used by open source projects: Bugzilla, MediaWiki, Mailman, IRC and more. It will include both general approaches and technical details about using various data collection tools, like mlstats. The final section of the presentation will talk about techniques for sharing this data with your community and highlighting contributions from key community members. For anyone who loves playing with data as much as I do, metrics can be a fun way to see what your community members are really doing in your open source project.

Crunching the numbers: Open Source Community Metrics at OSCON

Dave Neary and I co-presented a session about metrics at OSCON on Wednesday based on what we have learned so far from doing the MeeGo metrics.

Description

Every community manager knows that community metrics are important, but how do you come up with a plan and figure out what you want to measure? Most community managers have their own set of hacky scripts for extracting data from various sources after they decide what metrics to track. There is no standardized Community Software Dashboard you can use to generate near-real-time stats on your community growth.

Like most open source projects, we have diverse community infrastructure for MeeGo, including Mailman, Drupal, Mediawiki, IRC, git, OpenSuse Build Service, Transifex and vBulletin. We wanted to unify these sources together, extract meaningful statistics from the data we had available to us, and present it to the user in a way that made it easy to see if the community was developing nicely or not.

Building on the work of Pentaho, Talend, MLStats, gitdm and a host of others, we built a generic and open source community dashboard for the MeeGo project, and integrated it into the website. The project was run in the open at on the MeeGo wiki and all products of the project are available for reuse.

This presentation covered the various metrics we wanted to measure, how we extracted the data from a diverse set of services to do it, and more importantly, how you can do it too.

Open Source Community Metrics

Today at Open Source Bridge, I’ll be leading a session about Open Source Community Metrics: Tips and Techniques for Measuring Participation at 3:45pm in B302.

Do you know what people are really doing in your open source project? The best thing about open source projects is that you have all of your community data in the public at your fingertips. You just need to know how to gather the data about your open source community so that you can hack it all together to get something interesting that you can really use. Having good community data and metrics for your open source project is a great way to understand what works and what needs improvement over time, and metrics can also be a nice way to highlight contributions from key project members. This session will focus on tips and techniques for collecting and analyzing metrics from tools commonly used by open source projects using examples from what I’ve learned doing MeeGo metrics.

A few topics:

  • General guidance for coming up with a set of metrics that makes sense for your project.
  • Tips and techniques for collecting metrics from tools commonly used by open source projects: Bugzilla, MediaWiki, Mailman, IRC and more.
  • General approaches and technical details about using various data collection tools, like mlstats.
  • Techniques for sharing this data with your community and highlighting contributions from key community members.

For anyone who loves playing with data as much as I do, metrics can be a fun way to see what your community members are really doing in your open source project. It’s like people watching, but with data.

MeeGo Community and Metrics Presentation Videos

I just realized that while I was frantically catching up from my much needed Thanksgiving vacation right after the MeeGo Conference that I completely forgot to post the videos and presentation materials from my sessions at the conference. I presented two sessions:

State of the Community


Presentation Materials (PDF) and More Info

An Inside Look into the MeeGo Metrics


Presentation Materials (PDF) and More Info

You can also watch videos of all of the other sessions at the MeeGo Conference. For those of you interested in online communities, you should definitely watch Dave Neary’s Community Anti-Patterns presentation.

Community Manager Tip: Automate Tasks

This is one of those things that can be hard to make the time for, and it is more technically challenging than many other community management tasks, but over the longer term it can really pay off in time savings. Think about those repetitive tasks that you do over and over – monthly community metrics come to mind as one of the most common examples. I started really looking at the time it was taking me to compile my monthly metrics, and I vowed to start automating as much of it as I can. Right now, I’m getting close to having most things at least partially automated.

Here are a few ways to automate your community tasks:

  • APIs: Many APIs are easier to use than you might think and can be a great way to suck data out of commonly used services like wikis. You can often format a URL and get a file without any programming required.
  • Database queries: Yes, I’m picky, but I’m almost never happy with the reporting tools in community software, and I always end up needing a few database queries. If you don’t have the technical skills, find a geek to help you write a few queries that can be set up to automatically run every month and email you the results.
  • Scripts: I have one gigantic shell script and a couple of smaller ones where I dump a bunch of commands that run other stats gathering programs, do database queries, download data into files (parsing if needed), etc. This requires a little programming knowledge, but it’s easier than it sounds.
  • Schedule: Many web hosts make it easy to schedule your scripts to run every hour, day, week, month or on some custom schedule with a nice, user-friendly interface into cron (which can be incomprehensible to some people in its native form).

Additional Reading

Part of a series of community manager tips blog posts.

Image by hobvias sudoneighm used under the Creative Commons Attribution 2.0 Generic license.

Community Manager Tip: Have Great Metrics

For community managers, having excellent metrics is one of the best ways to show your progress and help justify your efforts to management when talking about budgets and staffing for the community. It provides an early warning system and diagnostics for potential community issues, which gives you time to make corrections before things get too bad.

Here are a few tips for having great metrics:

  • Measure many details to help you diagnose issues, but focus on a smaller subset that are used to determine success / failure.
  • The smaller subset should map to your goals and strategies for your community as a whole to show that you are meeting your objectives.
  • Share your metrics with your community. I have a public report with the data and a second internal report with more detailed analysis and suggestions for where the team can improve.
  • Measure across a few categories. I use awareness, membership and participation / engagement.

Additional Reading

Note: I hope to make this into a series of short posts (approximately weekly) to share quick tips for community managers.

Image by Flickr user Kevinzhengli used under the Creative Commons Attribution 2.0 Generic License.

Online Community Metrics

This morning, I ran across a list of social media metrics on the Page One PR site, and I realized that I spend quite a bit of time talking to clients about success metrics, but I haven’t spent much time writing about metrics on this blog. In a previous post, I talked about my general guidelines for online community success metrics:

The metrics that you select will depend on your specific goals, but common community metrics include page views or visits, new member sign ups, and participation (new posts or replies). It is easy to go overboard and measure everything; however, I recommend that you pick a couple (no more than 4 or 5) of the most important measurements to use to report to management on your success. You should have an analytics package or reporting tools that allow you to drill down for more details that you can use to help troubleshoot issues and understand the data, but use these as background materials for your team.

In other words, your success metrics are a small number of items that you use to determine success or failure over a period of time. You should measure many other items that you can use as indicators for what works or what doesn’t work, but make sure you separate what you are measuring because it helps you do your job vs. what metrics you are using to determine success.

Now, let’s get more specific. Online community efforts, including social media, can be very difficult to measure. I try to focus success metrics across three areas: awareness, membership, and engagement.

Awareness

Awareness is focused on getting people to notice your community and visit it to learn more. I typically use a general purpose website analytics package, like Google Analytics, to track visits to the community or page views as my primary measure of awareness.

Membership

Membership looks at the people who are paying attention to your community on an ongoing basis. These are the people who take the time to sign up and join the community as members. I usually use the number of new members or the total number of members of the community as the success metric for membership.

Engagement

Engagement is all about the conversations that people are having in your community and their interactions with other community members. The number of discussions, replies, or comments are typical ways to measure engagement in a community.

While awareness, membership, and engagement are long-term metrics, you should also have success metrics for shorter-term programs that also tie back into your overall community metrics. For example, you will want to keep track of when you do any kind of outreach (online or offline) and watch relevant success metrics for your community both before and after the outreach activity. This additional measurement will help you determine what methods of outreach work best for your community and will determine the success or failure of a shorter-term program.

While the success metrics mentioned above are geared toward online communities, you can use a similar approach for social media efforts. Awareness could include mentions of your company or product name across various social media sites (Twitter, blogs, video, etc.) Membership might include Twitter followers or RSS feed subscribers for your blog. Engagement could include comments on your blog posts and Twitter @replies.

Exactly how you will measure these success metrics depends entirely on your community dynamics, the community platform capabilities, and what activities in the community are the most important for your members. Don’t get too caught up in the examples I’ve listed here. The important part is finding a way to measure all three of these areas: awareness, membership, and engagement. Exactly how you will determine success will probably be a little different for each community.