matt

By Matt Germonprez and Georg Link

CHAOSScon 2023 Europe is now complete! Thanks to everyone who helped put on another wonderful event and thanks to everyone who took the time to attend and participate! It was really great to connect with everyone in the beautiful city of Brussels. 

This CHAOSScon, we led several discussions centering on two key questions. We asked participants to reflect on the questions in small groups and report back. This post will highlight a few of the key takeaways — and also provide captured comments that maybe didn’t make it into this post. 

What challenges exist for using metrics within your OSPO or Community?

The Consistent Use of Metrics within an Organization or Community

Many of the responses centered on the difficulties associated with the consistent use of metrics within an organization or community. For example, with the sheer size of many organizations and communities, the deployment and interpretation of metrics can vary a lot between people. This leads us, in the CHAOSS Community, to a 2023 goal of developing ways to communicate simple metric strategies and results that can be easily shared between people. 

The full set of recorded comments for the first question with general categorizations is here: 

• Organization/Community: 
  • There isn’t a central place for metrics discussions within an organization.
  • There isn’t a common taxonomy for how to speak about metrics.
  • Fragmentation of metrics within an organization can make consistent use difficult.
  • The size of an organization can make consistent use of metrics difficult.
  • The size of a community can make consistent use of metrics difficult.
  • Different stages of project maturity can make consistent use of metrics difficult.
  • There is minimal guidance on replicating CHAOSS structures locally.
• Metrics/Metrics Model: 
  • How to determine business value from a project?
  • How to measure the value of participation in a project?
  • How to determine business risk from a project?
  • How to determine company impact on a project?
  • How to measure the cost of non-participation?
  • Metrics themselves can be quite complex

What should the CHAOSS project be working on in the future?

Building Collaborative Communities and Helping People Communicate Better

There are many, many things that we could work on within the CHAOSS project in 2023. Recurring themes from CHAOSScon include (1) helping people connect with others in similar contexts to discuss health-related concerns (i.e., corporate open source program offices) and (2) developing ways to help people communicate about metrics within their respective organization or community. 

The full set of recorded comments for the second question with general categorizations is here: 

• CHAOSS Software: 
  • Enable all contributions to be seen in a dashboard.
  • Add social, design, and educational material use (like a social calendar) into a dashboard of contributor metrics to cross-visualize data about the ecosystem beyond code.
• CHAOSS Operations: 
  • Continue to support the newcomer experience.
  • Assist others with the interpretation of results.
  • Have mechanisms to help interpret the data in non-biased ways / manipulating it towards certain predetermined hypotheses.
  • Provide badging to validate business metrics strategies.
  • Provide metrics and metrics model validation.
  • Support user groups that need specific metrics and metrics models to help in a variety of contexts.
  • Develop personas for groups of people with similar interests.
  • Build better starting points for people who may not want to build metrics.
  • Have social justice built into the metric development process, not as charity, but to address systemic issues continuously.
  • Provide Pathways into the metrics, more context on where they have been used in the past (e.g. to answer the question what do people like me use?).
  • Think about people who work in different languages – is CHAOSS accessible for them?
• CHAOSS Communication: 
  • Talk with other communities to understand different views of metrics.
  • Tell user stories of how others are using the metrics and metrics models in practice.
  • Provide a framework for how to apply metrics.
  • Provide ways to represent and talk about metrics in business meetings.
  • Push a goal-oriented approach.
  • Articulating why metrics are important to help the business case.
  • Provide a set of five metric models to use that do five different things.
• Metrics/Metrics Model: 
  • Software license compliance between repos
  • Test coverage in repos.
  • Code quality
  • Pipeline status
  • Ensuring a broad community of voices.
  • Build productivity and efficiency metrics and metrics models.
  • Need to understand the signal-to-noise ratio.
  • More human-centric metrics, the kind that help to signal/prevent burnout.
  • An estimate of how difficult it might be to implement. A signal of whether tech experience is required to implement a particular metric, or if it can be done by a quick survey, or if it can be identified automatically, or if it requires big program management lift. 

Thanks again to everyone for an amazing CHAOSScon 2023 in Brussels! We hope to see you soon!!

Photo Credit: Sean Goggins