On July 11, 2024 in New York City, open source enthusiasts and strategists gathered for an insightful workshop led by two CHAOTICS: Georg Link, Open Source Strategist at Bitergia, and Stephanie Lieggi, Executive Director at UCSC OSPO and CROSS, UC Santa Cruz. This event was held the day after the United Nations’s OSPOs for Good event, so a theme throughout the workshop was how to drive value for OSPOs across governments, agencies, non-profits, universities, and industry. The event focused on addressing a common challenge of OSPOs: demonstrating the value of working in the open and justifying resources for open source projects. Here are the key themes, topic points, and takeaways from the workshop.
Introduction: The Value of Working in the Open
The workshop opened with a compelling question: Have you ever struggled to show the value of working in the open? This question set the stage for exploring effective strategies to justify resources for open source initiatives. Georg introduced the Goal-Question-Metrics (GQM) approach, which is a recommended approach from the CHAOSS project. Because the workshop revolved around the GQM approach, we will explain it here.
The Goal Question Metric (GQM) Approach
At CHAOSS, we have advocated for using the GQM because it is a structured method for deriving metrics that align with organizational goals. It involves three key steps:
- Goals: Identify and understand your organizational goals. These can vary significantly but typically include objectives like recruiting talent or enhancing community engagement.
- Questions: Break down these goals into specific, actionable questions. For example, to assess recruitment efforts, one might ask, “Who are important contributors?” or “How many did we help hire?”
- Metrics: Develop metrics to answer these questions. Metrics should be operational and data-driven, such as the number of contributions by name, hiring successes, or project activity levels. Some good data points, like the number of commits, may not be relevant to the question you need to answer.
Selecting the right metrics is crucial. They should align with organizational goals and also be mindful of data sources and potential unintended consequences.
After identifying the most effective metrics, tell a story with them to highlight how your OSPO contributed to the success of the organization.
Sharing Organizational Goals
In an interactive discussion, workshop participants shared diverse organizational goals, emphasizing the importance of aligning open source efforts with the broader mission statements of their organizations. We collected some bullet points on a screen and participants discussed many of them.
The most common goals included:
- Being a thought leader: Working in the open to demonstrate the skill and knowledge of the organization to lead conversations.
- Helping the community: Giving back to the community by working transparently and in the open.
- Diversity: Including many voices, especially those who would not get a seat at the table if it weren’t for working in the open.
- Tech literacy: Upskilling employees by allowing them to work in the open on new technologies.
- Civic utility: Working in the open can bridge the gap between the government agency or organization and the constituents who are being served.
- Interoperability:Working in the open allows for alignment of technological advances within the organization and in the larger open source community.
Case Studies
Finding ways to demonstrate the value of working in the open is not a new challenge, although the creation of new OSPOs is now elevating the stakes in sustaining these efforts. To speed up the process of finding meaningful metrics and stories, the workshop participants were introduced to people who could share their past experiences.
CHAOSS Project: Community Health Analytics
Sophia Vargas from Google’s OSPO presented the CHAOSS Project, which focuses on community health analytics. Their key questions addressed workload distribution, project slowdown, and new contributor onboarding. Their metrics used included the percentage of work done by top contributors and response times.
Open Source Foundations
Arun Gupta from Intel discussed the benefits of open source foundations like the Linux Foundation, CNCF, Eclipse Foundation, and Apache Foundation. Metrics for evaluating these foundations included project counts, contributor numbers, code contributions, and financial investments.
Metrics for OSS Sustainability
Vladimir Filkov from UC Davis and Charlie Schweik from UMass Amherst highlighted the importance of process-driven sustainability metrics. Their approach involved comprehensive metrics and analytics using AI, and they link the metrics to governance and policy actions.
Global Community Technology Challenge
Wilfred Pinfold from OpenCommons discussed efforts to get cities collaborating on consistent services through the Global Community Technology Challenge.
GitHub and WHO OSPO
Cynthia Lo from GitHub shared the development of an open source metrics dashboard for the social sector, focusing on repository health metrics that the World Health Organization found valuable.
https://github.com/WorldHealthOrganization/world-health-org-metrics
OSPO Levels and Responsibilities
Ana J. Santamaria from TODO, Linux Foundation, discussed the various responsibilities of OSPOs and the importance of demonstrating ROI. She emphasized enabling non-technical managers and shared resources for further learning.
Small-Group Discussion
The second half of the workshop engaged participants in small-group discussions. In groups of 3, they discussed their goals and the challenges of measuring success. Each member had the chance to share their own perspective while the others practiced active listening and then gave feedback. After the small-group discussion, the groups could share with the other groups what they had discussed.
Get Involved and Drive Open Source Success
This workshop provided a primer to strategies to measure the value of working in the open. By leveraging the GQM approach, sharing organizational goals, and learning from real-world case studies, participants left with actionable insights to enhance their open-source initiatives.
To continue your journey in open source success, we encourage you to explore the following resources:
- Join the CHAOSS Community: Dive into the world of community health analytics and learn how to apply metrics to your projects. Visit CHAOSS’s Quick Start for New Contributors to get started and join CHAOSS on Slack.
- Participate in OSPOs4Good: Join the Slack community for ongoing discussions, support, and collaboration.
- Explore the OSPO Landscape: Learn more about OSPO responsibilities and best practices through the Linux Foundation’s TODO Group.
Your involvement is crucial to advancing the open source movement. Whether you are just starting or are an experienced strategist, there are always new ways to contribute and innovate. Let’s work together to demonstrate and maximize the value of working in the open!