CHAOSS metrics are identified and defined using a continuous contribution process. The metrics are officially released biannually following a 30 day comment period. To contribute to the release or comment on metrics under review, please follow the links provided in the working group tables located below.
To obtain a pdf copy of previous releases or see what is new in this release please visit the Release History.
To make suggestions or edits to this webpage please visit the website repo and open an issue or create a pull request.
Metrics on this page were debated in working groups and undergo a 30 day comment period to ensure validity. Released metrics are only a subset of many possible metrics. CHAOSS acknowledges that more metrics exist and is working to identify and release new metrics in the future. If you would like to learn more about metrics, suggest new metrics, and or help define metrics please visit our participate page.
The CHAOSS project recognizes that there are ethical and legal challenges when using the metrics and software provided by the CHAOSS community. Ethical challenges exist around protecting community members and empowering them with their personal information. Legal challenges exist around GDPR and similar laws or regulations that protect personal information of community members. Particular challenges may arise in the use that is specific to your context.
Focus Areas by Working Group
CHAOSS metrics are sorted into Focus Areas. CHAOSS uses a Goal-Question-Metric format to present metrics. Individual metrics are released based on identified goals and questions. The metrics include a detail page with definitions, objectives, and examples.
Common Metrics Focus Areas
- What - contributions are being made
- When - are contributions occurring
- Who - is making contributions
Diversity and Inclusion Areas
- Code Development Activity
- Code Development Efficiency
- Code Development Process Quality
- Issue Resolution
- Community Growth
Tentative Dates for Next Release 2021-02
Release Freeze: January 1st, 2021
Public Comment Period: January 1st, 2021 to January 31st, 2021
Official Metrics Release Date: First week of February 2021
Common Metrics Repository: https://github.com/chaoss/wg-common/
Focus Area - What
Understand what contributions from organizations and people are being made.
|Types of Contributions||What types of contributions are being made?|
Focus Area - When
Understand when contributions from organizations and people are happening.
|Activity Dates and Times||What are the dates and timestamps of when contributor activities occur?|
|Time to First Response||How much time passes between when an activity requiring attention is created and the first response?|
|Time to Close||How much time passes between creating and closing an operation such as an issue, review, or support ticket?|
Focus Area - Who
Understand organizational and personal engagement with open source projects.
Diversity and Inclusion
D&I Repository: https://github.com/chaoss/wg-diversity-inclusion/
Focus Area - Event Diversity
Identify the diversity and inclusion at events.
|Attendee Demographics||How diverse and inclusive are the attendees?|
|Code of Conduct at Event||How does the Code of Conduct for events support diversity and inclusion?|
|Diversity Access Tickets||How are Diversity Access Tickets used to support diversity and inclusion for an event?|
|Family Friendliness||How does enabling families to attend together support diversity and inclusion of the event?|
|Speaker Demographics||How well does the speaker lineup for the event represent a diverse set of demographics and can be improved in the future?|
Focus Area - Governance
Identify how diverse and inclusive project governance is.
|Board/Council Diversity||What is the diversity within our governing board or council?|
|Code of Conduct for Project||How does the Code of Conduct for the project support diversity and inclusion?|
Focus Area - Leadership
Identify how healthy community leadership is.
|Inclusive Leadership||How well is a project setup for diverse leadership?|
|Mentorship||How effective are our mentorship programs at supporting diversity and inclusion in our project?|
|Sponsorship||How effective are long-time members who sponsor people in supporting diversity and inclusion in a community?|
Focus Area - Project and Community
Identify how diverse and inclusive our project places, where community engagement occurs, are.
||How does the documentation accommodate different users?||Issue 321|
|Documentation Usability||What is the usability of documentation from content and structure perspectives?|
|Issue Label Inclusivity||How well are project issues labeled to invite new contributors, skilled contributors, non-code contributors, and other types of contributors?|
||How is project burnout identified and managed within an open source project?||Issue 319|
Evolution Repository: https://github.com/chaoss/wg-evolution
Scope: Aspects related to how the source code changes over time, and the mechanisms that the project has to perform and control those changes.
Focus Area - Code Development Activity
Learn about the types and frequency of activities involved in developing code.
|Code Changes||What changes were made to the source code during a specified period?|
|Code Changes Lines||What is the sum of the number of lines touched (lines added plus lines removed) in all changes to the source code during a certain period?|
Focus Area - Code Development Efficiency
Learn how efficiently activities around code development get resolved.
|Change Requests Accepted
||How many accepted change requests are present in a code change?||Issue 385|
|Change Requests Declined
||What reviews of change requests ended up declining the change during a certain period?||Issue 384|
|Change Requests Duration
||What is the duration of time between the moment a change request starts and moment it is accepted?||Issue 383|
Focus Area - Code Development Process Quality
Learn about the processes to improve/review quality that are used (for example: testing, code review, tagging issues, tagging a release, time to response, CII Badging).
||What new requests for changes to the source code occurred during a certain period?||Issue 386|
Focus Area - Issue Resolution
Identify how effective the community is at addressing issues identified by community participants.
|Issues New||What are the number of new issues created during a certain period?|
|Issues Active||What is the count of issues that showed activity during a certain period?|
|Issues Closed||What is the count of issues that were closed during a certain period?|
|Issue Age||What is the average time that open issues have been open?|
|Issue Response Time||How much time passes between the opening of an issue and a response in the issue thread from another contributor?|
|Issue Resolution Duration||How long does it take for an issue to be closed?|
Focus Area - Community Growth
Identify the size of the project community and whether it is growing, shrinking, or staying the same.
|Inactive Contributors||How many Contributors have gone inactive over a specific period of time?|
|New Contributors||How many contributors are closing issues for the first time?|
|New Contributors Closing Issues||How many contributors are making their first contribution to a given project and who are they?|
Risk Repository: https://github.com/chaoss/wg-risk
Focus Area - Business Risk
Understand how active a community exists around/to support a given software package.
|Committers||How robust and diverse are the contributors to a community?|
|Elephant Factor||What is the distribution of work in the community?|
Focus Area - Code Quality
Understand the quality of a given software package.
|Test Coverage||How well is the code tested?|
Focus Area - Licensing
Understand the potential intellectual property (IP) issues associated with a given software package’s use.
|License Coverage||How much of the code base has declared licenses?|
|License Declared||What are the declared software package licenses?|
|OSI Approved Licenses||What percentage of a project’s licenses are OSI approved open source licenses?|
||Does the software package have an associated SPDX document as a standard expression of dependencies, licensing, and security-related issues?||Issue #106|
Focus Area - Security
Understand security processes and procedures associated with the software’s development.
|CII Best Practices badge||What is the current CII Best Practices status for the project?|
Value Repository: https://github.com/chaoss/wg-value
Focus Area - Communal Value
Identify the degree to which a project is valuable to its community of users (including downstream projects) or contributors
|Project Velocity||What is the development speed for an organization?|
||How does one measure the value of community interactions and accurately gauge “trust” within a community as evident from qualitative sentiment?||Issue 121|
Focus Area - Individual Value
Identify the degree to which a project is valuable to me as an individual user or contributor
|Organizational Project Skill Demand||How many organizations are using this project and could hire me if I become proficient?|
|Job Opportunities||How many job postings request skills with technologies from a project?|
Focus Area - Organizational Value
Identify the degree to which a project is monetarily valuable from an organization's perspective
|Labor Investment||What was the cost of an organization for its employees to create the counted contributions (e.g., commits, issues, and pull requests)?|
CHAOSS contributors include:
Aastha Bist, Abhinav Bajpai, Ahmed Zerouali, Akshara P, Akshita Gupta, Amanda Brindle, Alberto Martín, Alberto Pérez García-Plaza, Alexander Serebrenik, Alexandre Courouble, Alolita Sharma, Alvaro del Castillo, Ahmed Zerouali, Amy Marrich, Ana Jimenez Santamaria, Andre Klapper, Andrea Gallo, Andy Grunwald, Andy Leak, Aniruddha Karajgi, Anita Sarma, Ankit Lohani, Ankur Sonawane, Anna Buhman, Armstrong Foundjem, Atharva Sharma, Ben Lloyd Pearson, Benjamin Copeland, Bingwen Ma, Boris Baldassari, Bram Adams, Brian Proffitt, Camilo Velazquez Rodriguez, Carol Chen, Carter Landis, Chris Clark, Christian Cmehil-Warn, Damien Legay, Dani Gellis, Daniel German, Daniel Izquierdo Cortazar, David A. Wheeler, David Moreno, David Pose, Dawn Foster, Derek Howard, Don Marti, Drashti, Dylan Marcy, Eleni Constantinou, Elizabeth Barron, Emma Irwin, Eriol Fox, Fil Maj, Gabe Heim, Georg J.P. Link, Gil Yehuda, Harish Pillay, Harshal Mittal, Henri Yandell, Henrik Mitsch, Igor Steinmacher, Ildiko Vancsa, Jacob Green, Jaice Singer Du Mars, Jason Clark, Javier Luis Cánovas Izquierdo, Jeff McAffer, Jeremiah Foster, Jessica Wilkerson, Jesus M. Gonzalez-Barahona, Jilayne Lovejoy, Jocelyn Matthews, Johan Linåker, John Mertic, Jon Lawrence, Jonathan Lipps, Jono Bacon, Jordi Cabot, Jose Manrique Lopez de la Fuente, Joshua Hickman, Joshua R. Simmons, Josianne Marsan, Justin W. Flory, Kate Stewart, Keanu Nichols, Kevin Lumbard, King Gao, Kristof Van Tomme, Lars, Laura Dabbish, Laura Gaetano, Lawrence Hecht, Leslie Hawthorne, Luis Cañas-Díaz, Luis Villa, Lukasz Gryglicki, Mariam Guizani, Mark Matyas, Martin Coulombe, Matthew Broberg, Matt Germonprez, Matt Snell, Michael Downey, Miguel Ángel Fernández, Mike Wu, Neil Chue Hong, Neofytos Kolokotronis, Nick Vidal, Nicole Huesman, Nishchith K Shetty, Nithya Ruff, Nuritzi Sanchez, Parth Sharma, Patrick Masson, Peter Monks, Pranjal Aswani, Pratik Mishra, Prodromos Polychroniadis, Quan Zhou, Ray Paik, Remy DeCausemaker, Ria Gupta, Robert Lincoln Truesdale III, Robert Sanchez, Rupa Dachere, Saicharan Reddy, Saloni Garg, Saleh Abdel Motaal, Samantha Lee, Samantha Venia Logan, Samson Goddy, Santiago Dueñas, Sarit Adhikari, Sarvesh Mehta, Sarah Conway, Sean P. Goggins, Shane Curcuru, Sharan Foga, Shaun McCance, Shreyas, Silona Bonewald, Sophia Vargas, Sri Ramkrishna, Stefano Zacchiroli, Stefka Dimitrova, Tharun Ravuri, Thom DeCarlo, Tianyi Zhou, Tobie Langel, Saleh Abdel Motaal, Tom Mens, UTpH, Valerio Cosentino, Venu Vardhan Reddy Tekula, Vicky Janicki, Victor Coisne, Vinod Ahuja, Vipul Gupta, Will Norris, Xavier Bol, Xiaoya, Zibby Keaton
Are you eligible to be on this list? You are if you helped in any capacity, for example: Filed an issue. Created a Pull Request. Gave feedback on our work. Please open an issue or post on the mailing list if we've missed anyone.
CHAOSS Governing Board members at time of release:
- Andrea Gallo, Linaro
- Ben Lloyd Pearson, Nylas
- Brian Proffitt, Red Hat
- Daniel Izquierdo, Bitergia
- Daniel M. German, University of Victoria
- Dawn Foster, VMware
- Don Marti, Consumer Reports
- Georg Link, Bitergia
- Ildikó Vancsa, OpenStack
- Kate Stewart, Linux Foundation
- Matt Germonprez, University of Nebraska at Omaha
- Nicole Huesman, Intel
- Ray Paik, GitLab
- Sean Goggins, University of Missouri
- Wayne Beaton, Eclipse Foundation