We just launched the latest in our Practitioner Guide series with a guide on Demonstrating Organizational Value. As you can imagine, this is a big and difficult topic, so this is our most ambitious guide so far, and it’s our first guide outside of the “Getting Started” series of guides.
The idea for this guide came about as a result of several conversations with Bob Killen around the time of his KubeCon talk: Why is this so hard? Conveying the Business Value of Open Source (slides and video) along with the White Whale talk from the Linux Foundation Member Summit. After seeing these talks, I knew that I wanted to work with Bob to turn his ideas into a practitioner guide.
I’m co-chair for the CHAOSS OSPO (Open Source Program Office) Working Group, and the topic of how to demonstrate the value of our work in open source has been a popular topic of conversation since we started the group. However, given the current financial climate and the number of OSPOs that have been the targets of recent cutbacks and layoffs, this feels like a particularly important topic right now.
Creating an open source contribution strategy can help organizations frame their discussions with leadership to demonstrate the value of their open source efforts in ways that resonate with leadership. At a minimum, the open source strategy should contain details in the following areas, which are each addressed in the “How to Take Action” sections contained in the guide:
Supporting your organization’s goals
Determining which open source projects are the most critical for your organization
Assessing open source project health risk
Prioritizing within your organization’s limited resources
Measuring & framing value
I hope you find this guide useful as you think about how to demonstrate the value of your open source initiatives within your organization!
I spend a lot of time thinking about the sustainability of open source projects and the role that contributor sustainability plays in overall project sustainability. This involves more than just attracting contributors, but in ensuring they stay, grow, and thrive within our open source projects, and that can be a lot easier if we think outside of our traditional boxes and get creative about recruiting and retaining contributors.
Within the CHAOSS project, some of our most successful initiatives are driven mostly or partly by folks from our CHAOSS Africa chapter. We also have a CHAOSS Asia chapter that has recently re-started and is also driving some interesting work within the project. As a result, I’ve seen first hand some of the challenges, but also the rewards, that come from making it easier for people to contribute from countries that haven’t always been well-represented in many open source projects.
On a similar note, until recently, I was co-chair of the CNCF Contributor Strategy Technical Advisory Group where we were working to help CNCF projects grow their contributor bases. The CNCF mentoring programs were managed out of this group, and I was struck by how many mentees stick around after their time in the program with some mentees even becoming mentors! Another initiative within this group was the Deaf and Hard-of-hearing Working Group where I was inspired by all of the amazing work to help folks more actively participate in CNCF events and projects! This work was so successful that a second group was created by people from the blind and visually impaired community with similar goals of making it easier for people to participate.
All of this work got me thinking about how traditional pathways into open source don’t work for everyone and that we need to expand our contributor pipelines beyond the usual audiences if we want to have more sustainable open source projects where everyone can feel comfortable participating. This was the idea behind bringing together a panel of experts at the recent Open Source Summit in Denver where I led a panel discussion about this topic with Shuah Khan, Ruth Ikegah, and Matt Denny. I hope you enjoy this video of our discussion!
Note: A huge thank you to Sandeep Kanabar who helped put this proposal together, but was unable to attend at the last minute, and thanks again to Matt for stepping in on short notice!
I’ve been involved with open source project metrics for a very long time, and people often ask me which metrics they should use, but this isn’t really a question that I can answer. What you measure and how you interpret those metrics depend on your goals and what your organization is trying to accomplish. For this blog post, I’m using “organization” loosely. It could mean your company, university, or non-profit, but it might also mean aligning with funding organizations if your OSPO or other open source efforts were funded by another organization.
“one of the best places to start isn’t actually with the metrics, but by spending some time understanding the overall goals for the project. If the project is primarily driven by one organization or owned by an organization, you should also consider the goals for that organization. By thinking strategically about the overall goals, you’ll be in a better place to decide what you need to measure to determine whether the project is achieving its goals. Open source projects generate a tsunami of data that can be overwhelming, but by focusing on the goals, you can develop a metrics strategy that helps you focus on the metrics that matter most for a particular project.”
It can help to ask yourself, “what is important for my organization or the project?” This often means starting with your team’s open source strategy and aligning it with your organization’s goals. The most important part of putting together strategies and plans for your open source efforts is aligning them with the overall goals for your organization. By taking some time and effort to make sure you support the overall strategy for your organization, then it will be much easier to justify continuing these efforts during the next planning or funding cycle. This will also help you make the case to senior management, executives, funders, and others on the leadership team who aren’t likely to be involved in the low level details. Explaining how your open source contributions support the goals of your organization can help the executive or leadership team understand the strategic importance of this work so that you can continue your work in open source.
Once you have a strategy defined that aligns with the strategy of your organization, then you can figure out what you need to measure to show whether you are achieving your goals. There are a couple of reasons that starting with the goals is important, since metrics can go awry if you aren’t focused on the right things.
You won’t know if you are successful if you aren’t measuring the right things. If you aren’t measuring the things that are important for your project or organization, you won’t know if you are making progress in the areas that you care the most about. For example, if you want to improve the performance of a particular piece of open source software, you’ll want to have success criteria and measurement based on specific types of performance. If you want to gain influence within an open source project, maybe you measure increases in contributions or the number of employees moving into positions of influence.
Measurement impacts behavior, and people do different things depending on what you measure. For example, if you publish metrics that focus on the number of comments on issues, you are likely to start getting more comments on issues. If what you are really trying to do is get people to help review and approve contributions, then additional comments on issues might not help as much as looking at reviews on change requests (pull requests / merge requests).
Once you decide on your success criteria, you need to make sure that you can get the data required to measure it and start measuring it now to get a good baseline. There are plenty of tools available to gather contribution data about open source projects. Some of the commonly used tools can be found in the CHAOSS project, but you can also likely get a pretty good sense for the data by looking at your code repositories and other communication channels. GitHub, for example, has some pretty good data under the insights tab.
After you have your metrics, you need to actually do something with them to show that you are making progress toward accomplishing your goals. Think about which metrics you should be showing to your leadership and which ones should be shared with your team and the broader community. But communicating metrics is much more than just showing some charts or graphs, you also need to interpret those metrics and tell the story about what they mean. The CHAOSS Practitioner Guides can help you think about the interpretation and how you might tell the story about what your metrics mean. Without interpretation and explanation, all you have are numbers, which are way less powerful than the story about what the data means in the context of how it helps your organization achieve their goals.
Here are a few additional links and resources to help you think about building your metrics strategy and telling the story about what the metrics mean:
When I started in the role of Director of Data Science for CHAOSS, one of the first things I did was start the Data Science Working Group (WG) as a way to build community around the data science work that many of us were already doing within the CHAOSS project. I am incredibly proud of what we’ve accomplished in less than 2 years.
Yesterday, we published a CHAOSS blog post about what we’ve been working on lately, but here are a few highlights.
We are also driving several research projects out of the working group. I’ve already blogged about the Relicensing and Forks research that I’ve been working on, but we also have research looking into projects that move from private ownership into a foundation, archived projects, and a collection of research taxonomies.
I also wanted to remind people that like all of the CHAOSS working groups, the Data Science WG is open to everyone! All you need to join the Data Science WG is an interest in using data to understand the open source world around us. Most of our work is analysis of data, writing guides, and discussions about using metrics. You don’t need any special skills, and you don’t need to know any advanced statistics, machine learning, or AI. We’re even planning a CHAOSS Data Science Hackathon, which will be co-located with Open Source Summit North America and CHAOSScon in Denver, CO on June 26, 2025. To learn more, visit our repository, join our meetings, or reach out to us in the #wg-data-science channel in CHAOSS Slack. We hope you’ll join us!
In the Computer magazine article, I talked about how the CHAOSS project is providing advice and resources for proactively using metrics to improve open source project health and sustainability before a crisis occurs to make software more sustainable and reliable for everyone. Here’s a short quote from the Computer magazine article:
“Building sustainable open source projects over the long term can be a challenge. Project leaders, maintainers, and contributors are busy people who don’t always have the time to focus on growing a community along with maintaining their software. Using metrics is one way to help identify potential issues and areas where a project can be improved to make it more sustainable over the long term. Metrics are best used if they aren’t used once and never again. By monitoring the data over time, projects can understand trends that might indicate areas for improvement as well as see if those improvements are having the desired effect. Being proactive about improving sustainability before it becomes a crisis can help make open source software more sustainable and reliable for everyone” – Read the rest of the IEEE Computer magazine article for more.
The newest guide in the series, Practitioner Guide: Getting Started with Building Diverse Leadership, was written by Peculiar C. Umeh. It expands on the theme of improving health and sustainability of open source projects by creating a welcoming and inclusive environment that encourages contributions from a wide variety of people. Here’s a quote from the guide:
“A community or project with diverse leadership offers significant advantages because diverse leadership leverages diverse perspectives to build an innovative community, create a welcoming and inclusive environment, and empower individuals from all backgrounds to contribute their unique talents. New and existing contributors feel more included when they can see other people in leadership positions who are like them (Linux Foundation, 2021). When diverse leaders collaborate, their intersection sparks innovation and creates a more harmonious global leadership system. It represents a global and diverse user base, which improves the usability of the project because more users’ voices are represented in decision-making about the project’s design and functionality. It enhances decision-making processes by incorporating various viewpoints and experiences, leading to better problem-solving and more effective strategies. It promotes a culture of inclusion and respect, improving morale and engagement among community members and ultimately contributing to projects’ long-term success and sustainability.” – Read the Practitioner Guide: Getting Started with Building Diverse Leadership for more.
The other new guide in the series, Practitioner Guide: Getting Started with Sunsetting an Open Source Project, is also about making open source more sustainable by being clear about the future of an open source project so that users can make responsible decisions and avoid using open source technologies that are no longer being maintained or updated with security fixes. Here’s a quote from the guide:
“Many open source projects, even widely used ones, become abandoned for a variety of reasons (e.g., evolving interests, family situations, employment changes), but abandonment can be done in a responsible way by proactively sunsetting the project (Miller et al. 2025). Sunsetting is an important consideration for corporate environments where it can be easy to lose track of projects that were created by employees who later walked away from the project and left if abandoned. You don’t want abandoned open source projects with security vulnerabilities sitting in your organization’s source code repositories where someone might trust that project simply because they trust your organization. Finding inactive projects and responsibly sunsetting them is a good business decision and something that many open source teams / Open Source Program Offices (OSPOs) do on a regular basis. It’s important to remember that not every open source project can or should exist forever: technologies evolve, corporate priorities change, and people’s interests change. Part of the beauty of open source is that we work in the open as we innovate, and some of those innovative projects will stand the test of time, while others should be responsibly deprecated via a sunset process. Sunsetting an open source project should take your user’s needs into account, and where possible, offer users time to migrate to a replacement technology. At a minimum, it’s important to signal that the project will no longer be maintained, updated, or have security patches so that users know that they should no longer be using the project.” – Read the Practitioner Guide: Getting Started with Sunsetting an Open Source Project for more.
As always, these CHAOSS guides are under an open source license, so you’re free to use and modify them to meet your needs.
Within the CHAOSS project, we know that people often struggle to make productive use of the tsunami of data about open source projects. One of my focus areas over the past 2 years within the CHAOSS project has been to develop a series of Practitioner Guides designed to help develop insights that can be used to improve the project health of an open source project. So far, we have 5 guides: Introduction, Contributor Sustainability, Responsiveness, Organizational Participation, and Security with more guides coming soon.
I’ve written about these guides in an OpenSource.net blog post and recorded a CHAOSScast podcast about each guide. I’ve also done quite a few talks related to the topics in these guides, which can be found on my Speaking page. The most recent one was a joint talk with Peculiar C. Umeh at FOSS Backstage with a video that is available to watch.
I won’t go into more detail here, since I’ve already linked to other blog posts, podcasts, and talks on the topic, but I encourage you to have a look at the Practitioner Guides to find ways to make your open source projects healthier and more sustainable!
For my regularly scheduled (once every year and a half) blog post, I wanted to announce that July 3rd is my last day at VMware, and I will be joining the CHAOSS project as their new Director of Data Science.
It was really hard to leave VMware after almost 5 years (including my time at Pivotal). The work was fun, and I worked with so many amazing people that I will miss dearly! But as many of you know, I have a deep passion for data, and in particular open source community metrics, so the opportunity to work full time on the CHAOSS project is the dream job that I just couldn’t turn down. I’ve been working in this space for 10+ years with the CHAOSS project, and before CHAOSS, I was working with Bitergia and a variety of open source tools that later evolved into the software that is now part of the CHAOSS project. I’ll be taking July off and then will be starting my role with CHAOSS in August. A big thank you to the Alfred P. Sloan Foundation for making this possible through the grant that is funding the Director of Data Science position and other CHAOSS project initiatives.
I will be continuing my work on the OpenUK Board and as co-chair for the CNCF Contributor Strategy Technical Advisory Group, which have kept me very busy in addition to my work at VMware and in my role on the CHAOSS Governing Board.
Over the past year and a half, I’ve done quite a few presentations on topics ranging from how companies can work in open source communities to open source health / metrics to leading in open source, which can be found on my Speaking page. The highlight was giving a keynote about growing your contributor base at KubeCon EU in front of an audience of 10,000+, which was amazing and terrifying at the same time!
In addition to my world tour of conference presentations, I was quoted in a Linux Foundation Diversity Report, won a few awards for my UK work in open source as part of the OpenUK Honours list in 2021 and 2023, and I’ve written a few blog posts since my last post here on my own blog:
On the personal side, Paul and I bought a new house in November, and we have become the people who sit in their back garden and talk about how adorable the squirrels and birds are. Since we live in an area near quite a bit of green space, we have regular visits from foxes and even spotted one badger on our backyard wildlife camera!
Since I don’t post here often, if you want to keep up with what I’ve been doing, I post occasionally on Mastodon and Instagram.
It’s time again for my regularly scheduled (once every year and a half) blog post to avoid completely neglecting my personal blog. While I don’t blog often, I do still update my Speaking page on a regular basis, and conferences have really ramped up over the past couple of months! I’ll admit to being really tired of attending boring virtual events, so when the in-person events started back up, I went to all of them! In my rush of excitement about traveling and seeing people again, I agreed to do way too many talks – 10 talks in two months. Here are a few of the topics I’ve been talking about over the past year and a half, and you can visit my Speaking page to get links to slides and videos where available:
Navigating and mitigating open source project risk
Good governance practices for open source projects
Metrics and measuring project health
Becoming a speaker and getting talks accepted at conferences
Being a good corporate citizen in open source
I’ve also written quite a few blog posts on the VMware Open Source Blog and elsewhere on similar topics:
As part of my work on the OpenUK board, I was interviewed for a featured section about Open Source Program Offices in the report, State of Open: The UK in 2021 Phase Two: UK Adoption where I talked about VMware’s OSPO.
On a more personal note, we’ve been doing really well throughout the pandemic. We finally had our first real vacation in Malta, where we relaxed while eating and drinking our way through Malta along with swimming, snorkeling, reading, and enjoying the sunshine. I still keep an updated list of every book I read here on my blog if you’d like to know what I’ve been reading.
Since I don’t post here often, if you want to keep up with what I’ve been doing, I post more frequently on Twitter.
I realized that I haven’t posted anything in over a year and a half here, but I’ve definitely been busy! The biggest change is that Pivotal was acquired by VMware a few months ago, and I have moved into the Open Source Program Office as Director of Open Source Community Strategy where I continue to work remotely from my flat in the UK. I love my new job, and I get to work with a bunch of really amazing people! While I haven’t been blogging here, I have written several blog posts on the VMware Open Source Blog about building community and strategy.
I’ve been doing quite a few talks at conferences and other events, including some virtual ones, on a wide variety of topics including community building, open source metrics, Kubernetes, and more. Links to presentations and videos where available can be found on the speaking page.
I’m one of the rotating hosts for the new CHAOSScast podcast where we chat about a wide variety of open source metrics topics. I also wrote a post on the CHAOSS blog with a video that talks about how I’m using metrics at VMware to learn more about the health of our open source projects. If you’re as passionate about data and metrics as I am, CHAOSS is an open source community that welcomes contributors of all types, and it’s a fun group of people, so you should join us!
I’ve joined the OpenUK Board of Directors to help promote collaboration around open technologies (open source, open hardware, and open data) throughout the UK. We have weekly presentations that are free for anyone to attend every Friday, and we’re always looking for volunteers who want to help out on a wide variety of committees.
There are also a few other miscellaneous things that I’ve done recently:
