CHAOSS Metrics

Release 202001

Released metrics are only a subset of metric ideas that are being developed. If you would like to learn more and discuss different metrics please visit the working group repositories. The metrics are sorted into Focus Areas. CHAOSS uses a Goal-Question-Metric format to present metrics. Individual metrics are released based on identified goals and questions. The metrics include a detail page with definitions, objectives, and examples.

Focus Areas by Working Group

Common Metrics WG

Diversity and Inclusion WG

Evolution WG

Risk WG

Value WG

Important Dates for Release 202001

Release Freeze: January 1st, 2020
Candidate Release: January 24th, 2020
Public Comment Period: January 1st, 2020 to January 24th, 2020
Metrics Release Date: January 31st, 2020

Download a PDF of the 202001 Release

Common Metrics

Common Metrics Repository: https://github.com/chaoss/wg-common

Focus Area - What

Goal:
Understand what contributions from organizations and people are being made.

Metric/Details Question
Types of Contributions What types of contributions are being made?

Focus Area - When

Goal:
Understand when contributions from organizations and people are happening.

Metric/Details Question
Activity Dates and Times What are the dates and timestamps of when contributor activities occur?
Time to First Response How much time passes between when an activity requiring attention is created and the first response?

Focus Area - Who

Goal:
Understand organizational and personal engagement with open source projects.

Metric/Details Question
Contributors Who are the contributors to a project?
Organizational Diversity What is the organizational diversity of contributions?

Diversity and Inclusion

D&I Repository: https://github.com/chaoss/wg-diversity-inclusion

Focus Area - Event Diversity

Goal:
Identify the diversity and inclusion at events.

Metric/Details Question
Attendee Demographics How diverse and inclusive are the attendees?
Code of Conduct at Event How does the Code of Conduct for events support diversity and inclusion?
Diversity Access Tickets How are Diversity Access Tickets used to support diversity and inclusion for an event?
Family Friendliness How does enabling families to attend together support diversity and inclusion of the event?
Speaker Demographics How well does the speaker lineup for the event represent a diverse set of demographics and can be improved in the future?

Focus Area - Governance

Goal:
Identify how diverse and inclusive project governance is.

Metric/Details Question
Board/Council Diversity What is the diversity within our governing board or council?
Code of Conduct for Project How does the Code of Conduct for the project support diversity and inclusion?

Focus Area - Leadership

Goal:
Identify how healthy community leadership is.

Metric/Details Question
Mentorship How effective are our mentorship programs at supporting diversity and inclusion in our project?
Sponsorship How effective are long-time members who sponsor people in supporting diversity and inclusion in a community?

Evolution

Evolution Repository: https://github.com/chaoss/wg-evolution

Scope: Aspects related to how the source code changes over time, and the mechanisms that the project has to perform and control those changes.

Focus Area - Code Development Activity

Goal:
Learn about the types and frequency of activities involved in developing code.

Metric/Details Question
Code Changes What changes were made to the source code during a specified period?
Code Changes Lines What is the sum of the number of lines touched (lines added plus lines removed) in all changes to the source code during a certain period?

Focus Area - Code Development Efficiency

Goal:
Learn how efficiently activities around code development get resolved.

Metric/Details Question
Reviews Accepted How many accepted reviews are present in a code change?
Reviews Declined What reviews of code changes ended up declining the change during a certain period?
Reviews Duration What is the duration of time between the moment a code review starts and moment it is accepted?

Focus Area - Code Development Process Quality

Goal:
Learn about the processes to improve/review quality that are used (for example: testing, code review, tagging issues, tagging a release, time to response, CII Badging).

Metric/Details Question
Reviews What new review requests for changes to the source code occurred during a certain period?

Focus Area - Issue Resolution

Goal:
Identify how effective the community is at addressing issues identified by community participants.

Metric/Details Question
Issues New What are the number of new issues created during a certain period?
Issues Active What is the count of issues that showed activity during a certain period?
Issues Closed What is the count of issues that were closed during a certain period?
Issue Age What is the average time that open issues have been open?
Issue Response Time How much time passes between the opening of an issue and a response in the issue thread from another contributor?
Issue Resolution Duration How long does it take for an issue to be closed?

Focus Area - Community Growth

Goal:
Identify the size of the project community and whether it is growing, shrinking, or staying the same.

Metric/Details Question
New Contributors How many contributors are closing issues for the first time?
New Contributors Closing Issues How many contributors are making their first contribution to a given project and who are they?

Risk

Risk Repository: https://github.com/chaoss/wg-risk

Focus Area - Business Risk

Goal:
Understand how active a community exists around/to support a given software package.

Metric/Details Question
Committers How robust and diverse are the contributors to a community?
Elephant Factor What is the distribution of work in the community?

Focus Area - Code Quality

Goal:
Understand the quality of a given software package.

Metric/Details Question
Test Coverage How well is the code tested?

Focus Area - Licensing

Goal:
Understand the potential intellectual property(IP) issues associated with a given software package’s use.

Metric/Details Question
License Coverage How much of the code base has declared licenses?
License Declared What are the declared software package licenses?
OSI Approved Licenses What percentage of a project’s licenses are OSI approved open source licenses?

Focus Area - Security

Goal:
Understand how transparent a given software package is with respect to dependencies, licensing (?), security processes, etc.

Metric/Details Question
CII Best Practices badge What is the current CII Best Practices status for the project?

Value

Value Repository: https://github.com/chaoss/wg-value

Focus Area - Ecosystem Value

Goal:
Estimate the value of an open source project’s ecosystem.

Metric/Details Question
Social Currency Metric System (SCMS) How does one measure the value of community interactions and accurately gauge “trust” within a community as evident from qualitative sentiment?

Focus Area - Labor Investment

Goal:
Estimate the labor investment in open source projects.

Metric/Details Question
Labor Investment What was the cost of an organization for its employees to create the counted contributions (e.g., commits, issues, and pull requests)?
Project Velocity What is the development speed for an organization?

Focus Area - Living Wage

Goal:
Expanding opportunities for people to make a living wage in open source.

Metric/Details Question
Organizational Project Skill Demand How many organizations are using this project and could hire me if I become proficient?
Job Opportunities How many job postings request skills with technologies from a project?

CHAOSS contributors include:

Ahmed Zerouali, Akshita Gupta, Amanda Brindle, Alberto Martín, Alberto Pérez García-Plaza, Alexander Serebrenik, Alexandre Courouble, Alolita Sharma, Alvaro del Castillo, Ahmed Zerouali, Ana Jimenez Santamaria, Andre Klapper, Andrea Gallo, Andy Grunwald, Andy Leak, Aniruddha Karajgi, Anita Sarma, Ankit Lohani, Ankur Sonawane, Anna Buhman, Armstrong Foundjem, Atharva Sharma, Ben Lloyd Pearson, Benjamin Copeland, Bingwen Ma, Boris Baldassari, Bram Adams, Brian Proffitt, Camilo Velazquez Rodriguez, Carol Chen, Carter Landis, Chris Clark, Christian Cmehil-Warn, Damien Legay, Dani Gellis, Daniel German, Daniel Izquierdo Cortazar, David A. Wheeler, David Moreno, David Pose, Dawn Foster, Derek Howard, Don Marti, Drashti, Dylan Marcy, Eleni Constantinou, Emma Irwin, Fil Maj, Gabe Heim, Georg J.P. Link, Gil Yehuda, Harish Pillay, Harshal Mittal, Henri Yandell, Henrik Mitsch, Ildiko Vancsa, Jacob Green, Jaice Singer Du Mars, Jason Clark, Javier Luis Cánovas Izquierdo, Jeff McAffer, Jeremiah Foster, Jessica Wilkerson, Jesus M. Gonzalez-Barahona, Jocelyn Matthews, Johan, Johan Linåker, John Mertic, Jon Lawrence, Jonathan Lipps, Jono Bacon, Jordi Cabot, Jose Manrique Lopez de la Fuente, Joshua R. Simmons, Josianne Marsan, Kate Stewart, Keanu Nichols, Kevin Lumbard, Kristof Van Tomme, Lars, Laura Gaetano, Lawrence Hecht, Leslie Hawthorne, Luis Cañas-Díaz, Luis Villa, Lukasz Gryglicki, Mark Matyas, Martin Coulombe, Matthew Broberg, Matt Germonprez, Matt Snell, Michael Downey, Miguel Ángel Fernández, Mike Wu, Neil Chue Hong, Nick Vidal, Nicole Huesman, Nishchith K Shetty, Nithya Ruff, Parth Sharma, Patrick Masson, Peter Monks, Pranjal Aswani, Prodromos Polychroniadis, Quan Zhou, Ray Paik, Remy DeCausemaker, Robert Lincoln Truesdale III, Robert Sanchez, Rupa Dachere, Saloni Garg, Saleh Motaal, Samantha Logan, Santiago Dueñas, Sarvesh Mehta, Sarah Conway, Sean P. Goggins, Shane Curcuru, Sharan Foga, Shreyas, Stefano Zacchiroli, Thom DeCarlo, Tobie Langel, Tom Mens, UTpH, Valerio Cosentino, Venu Vardhan Reddy Tekula, Vicky Janicki, Victor Coisne, Vinod Ahuja, Vipul Gupta, Will Norris, Xavier Bol, Zibby Keaton

Are you eligible to be on this list? You are if you helped in any capacity, for example: Filed an issue. Created a Pull Request. Gave feedback on our work. Please open an issue or post on the mailing list if we've missed anyone.

CHAOSS Governing Board members at time of release:

  • Andrea Gallo, Linaro
  • Ben Lloyd Pearson, Nylas
  • Brian Proffitt, Red Hat
  • Daniel Izquierdo, Bitergia
  • Daniel M. German, University of Victoria
  • Dawn Foster, Pivotal
  • Don Marti, Mozilla
  • Georg Link, Bitergia
  • Ildikó Vancsa, OpenStack
  • Kate Stewart, Linux Foundation
  • Matt Germonprez, University of Nebraska at Omaha
  • Nicole Huesman, Intel
  • Ray Paik, GitLab
  • Sean Goggins, University of Missouri
  • Wayne Beaton, Eclipse Foundation

Copyright © 2020 CHAOSS a Linux Foundation® project. All rights reserved. The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our Trademark Usage page. Linux is a registered trademark of Linus Torvalds. Privacy Policy and Terms of Use.