Practitioner Guide: Contributor Sustainability

Primary metrics:

If you haven’t already read the Practitioner Guide: Introduction - Things to Think about When Interpreting Metrics, please pause now and read that guide.

Contributor sustainability is an important part of assessing whether an open source project and community have enough contributors for the project to be sustained over the long-term, so contributor sustainability has a large impact on overall project sustainability. There are a lot of projects with a single maintainer (see xkcd Dependency), and many projects struggle to find enough people to actively participate in their projects and continue to maintain them over the long term (Egbahl 2016). Avelino et al. (2019) found that even popular projects may become permanently abandoned and fail to recover from losing too many key contributors.

The reality is that there are a lot of open source projects and not enough contributors, so maintainers are in desperate need for help across the various types of contributions needed to have a successful and sustainable open source project. However, it's important to think about how each new contribution will create additional work for maintainers to review and accept it, which can be mitigated by having appropriate automation, processes, and documentation to help contributors become successful, but eventually, some of these contributors will need to become maintainers to handle the additional workload from new contributors (Eghbal 2020).

If there are not enough contributors and maintainers to sustain a project, this increases the risks that the project will fail, which creates a variety of often significant challenges for the users and other projects that depend on it.

Step 1: Identify Trends

There are a number of things that can impact contributor sustainability. By starting with Contributor Absence Factor, you can assess the risk to the project if key contributors / maintainers decided to leave, but it’s also important to look at other trends related to Contributors along with the Types of Contributions that people are making to identify potential contributor risks that might impact the overall sustainability of your project.

Contributor Absence Factor

Contributor Absence Factor is important for understanding contributor sustainability because it visualizes the question "how many contributors can we lose before a project stalls?" It helps to identify how widely the work in a project is distributed across contributors and identify the key people in a project that are doing the majority of the work. If the majority of work is being performed by a single person or small number of people, it increases the risk that a project could become unsustainable if that person or people were no longer working on the project (Avelino et al. 2019).

Contributor Absence Factor bar chart with balanced contributions ranging from 16% of commits to 6% of commits

Contributors

The contributors metric looks broadly at who contributes to a project and can be visualized in many different ways. While you can look at this as a single number of contributors over time, it can help to break this down to help you understand how many contributors are active along with how many have increasing or decreasing activity over time. While the contributor absence factor metric is good at identifying key contributors, you should also try to understand contributor sustainability more broadly to look at overall trends across all contributors to see if you are experiencing contributor growth or decline.

Active Contributors over time and growth analysis showing positive and negative contributor balances

Types of Contributions

Multiple, varied contributions impact open source project health and sustainability, and contributions may include writing code, managing the community, triaging bugs, evangelizing the project, supporting users, or helping in other ways. A variety of contribution types can demonstrate that a project is mature and well-rounded with sufficient activity to sustain all aspects of the project, and enable paths to leadership that are supportive of a variety of contribution types and people with varying expertise beyond coding.

Contributions by Data Source for various sources over time including GitHub, Twitter, pipermail, Slack

Step 2: Diagnosis

As mentioned in the Practitioner Guide Introduction, you should start by talking to a few people who are intimately involved in the project so that you can look together at the trends and interpret them in a way that makes sense for the project. For example, the image below shows that over 60% of the commits in this project in the past year were made by a single individual, which in most cases increases the risk that the project could become unsustainable if that person or people were no longer working on the project. However, it’s also important to think about this in light of other aspects of the project. For example, if this project is mostly documentation that many people have the knowledge to update or if it’s code that is fairly simple and easy for others to understand or if the project is mostly used by a single individual, then the risk is relatively low. On the other hand, if the project is complex, large, and / or widely used, then the risk is high and the project should focus on recruiting more contributors and maintainers to increase contributor sustainability.

Bus Factor pie chart showing that one contributor has made 61.8% of the commits with next largest contributors making 10.8% and 9.77%

It’s also important to look at other trends related to contributors. For example, the graph below shows that active contributors peaked and then declined for this project, so it would be important to diagnose why that might be the case. If the project was something bounded in time (e.g., related to a conference or workshop) that was completed or a project where most of the work has been completed and is not being lightly maintained, this would be a normal pattern to see. However, if this is a typical open source project, this could indicate that there are serious issues impacting the retention of contributors that should be diagnosed and addressed.

Contributor Growth by Engagement showing that the active contributors peaked in April 2023 and dwindled to nothing by October 2023

Step 3: Gather Additional Data if Needed

CHAOSS has other metrics related to contributor sustainability that can help diagnose specific problems within your community.

Additional Metrics:

Step 4: Make Improvements

Recruiting new contributors who can become maintainers is crucial for the success of projects where contributor sustainability is a concern, and it is even possible for some projects to recover from being abandoned by their primary developers by recruiting new maintainers (Avelino et al. 2019). By focusing on recruiting new contributors before you lose your key developer(s), you can have time to onboard contributors and proactively prevent a crisis later.

There are a couple of things that the Contributor Absence Factor can tell you that can be used to help drive contributor improvements that make your project more sustainable. First, it helps you decide how sustainable your current contributor situation is. If one person is making most of the contributions and the project is large or complex, you should focus on ways to distribute the load and have more people involved in the project. Second, the Contributor Absence Factor metric can help you find people who might be contributing more than you realized, which can help you think about who you can encourage to contribute more or maybe find someone who could move into a leadership role (e.g., reviewer or maintainer). With respect to leadership roles, you might look at someone making ten percent of the contributions and decide that they are ready to become a maintainer. One way to do this is to reduce the scope for new maintainers. If they aren’t ready to be a maintainer for the entire project, maybe they can maintain a sub-project or a specific section of the project while they build their expertise in other areas. One way to do this is by using OWNERS or CODEOWNERS files to give people responsibility for different areas within your repository, including docs or community sections. People who are making fewer, but regular contributions might be good candidates for mentorship or becoming reviewers with an eye toward making them maintainers after they gain a bit more experience.

It’s also important to look at your contribution guides or other onboarding documentation to make sure that new contributors can easily get started within your project (see Additional Reading section for links). Good documentation is how we scale the things that take up precious time for already overworked maintainers and free up their time to work on other things. At a minimum, a new contributor needs to understand how to spin up an environment where they can do their development, the expectations for testing and how to run tests, any processes or expectations that you have for pull requests, and instructions for other requirements. If this is well documented, new contributors can get started with a minimal amount of help from existing maintainers, which can save you a lot of time in the long run (Eghbal 2020). When a project doesn’t have good onboarding docs, maintainers can get frustrated by the amount of time they spend on new contributor questions, which can make it hard for new contributors to feel welcome and take a long time for them to become productive. This is how people get discouraged and drift away from your project. This doesn’t mean that you need to spend days and weeks writing perfect onboarding documents. Anything is better than nothing, and if you start with a few things that will help people get started quickly, new contributors can actually help make the onboarding documents better by adding more details and additional instructions for things that they found confusing or that they struggled with.

While onboarding documentation is a great way to get people started at scale, mentoring has been shown to be an effective and efficient way onboard new contributors to help them be productive in your project more quickly (Fagerholm et al. 2014). However, mentoring takes time away from a maintainer in the short-term in the hope of making the project more sustainable over the long-term, so it's important to think about which repeat contributors might be good candidates for mentorship (Egbahl 2020).

Having good first issues or help wanted labels are excellent places to start because these help people find something that they can work on while they learn more about the project, but you need to put some thought into crafting these issues into something with enough information for new contributors to use. Good first issues should be targeted as something simple that a brand new contributor could pick up and complete in a short amount of time to help them learn more about your contribution process. Help wanted labels can be for issues that are a little more involved so that people who have already started to contribute can find something else to work on. Good first issues and help wanted labels along with good contribution guides helps you build a sustainable pipeline of contributors for your project.

However, good first issues and help wanted labels are passive requests for help, so I also encourage maintainers to also be proactive and specific about ways that people can help. Asking someone specific to review a PR or answer a question from a user demonstrates that you recognize their unique expertise and want their help. Knowing that we’re wanted and appreciated makes us feel good, which can be a strong motivator to contribute to an open source project or to continue contributing. Reaching out to someone and acknowledging their work while encouraging them to do more can help you recruit people who could take on increasing responsibilities (e.g., reviewer or maintainer) within your project. Don’t be afraid to reach out to your power users in addition to your contributors, and if you work at a company, you can use your corporate relationships with other organizations to find people who might be interested in contributing.

Defining the roles and responsibilities for contributors, reviewers, and maintainers can help with recruiting new people into these roles. It can help to think about this as a contributor ladder where contributors can climb up to become reviewers and those reviewers can become maintainers. What’s important is to document this and make sure that people understand how they can climb the ladder and gain more responsibilities within the project. Moving more people into leadership roles, like reviewer and maintainer, can help reduce the bus factor and make your project more sustainable over time.

The catch here and with many metrics is that we don’t want to just think about people who are making code contributions. This is a good start, but you should also be thinking about how you can move people into leadership positions to be responsible for things that might not show up in GitHub or GitLab, like documentation, community management, marketing and other important roles. This is why it is so important to also look at the Types of Contributions metric. It’s common to see maintainers underestimating the amount of time spent on some of these tasks, and recruiting more people into other roles can further distribute the workload and increase contributor sustainability.

Maintainers for a project with a low Contributor Absence Factor, especially if they’re the only maintainer, should have some type of succession plan in place. At a minimum someone else should have admin access to everything needed to update the software and publish releases (including publishing to relevant package managers) along with documentation for how to do this. Ideally, there is another maintainer on the project who can share the responsibilities equally and already has the access, skills, and knowledge to perform all project tasks, which is one reason that the above discussions about building your contributor base and recruiting maintainers is so important. While not ideal, for single maintainer projects succession planning could include providing access and documentation to a trusted colleague or friend. The key to succession planning is to put it in place now, before you think you need it.

Step 5: Monitor Results

How you monitor the results will depend on what improvements you decided to make. Continuing to monitor these 3 metrics are a good start. If you used other data from Step 3, you should also monitor those metrics.

If you are recruiting new contributors, approvers, and maintainers, it might take a while to see the results of those efforts, so you might not see significant improvements for 3 to 6 months, so you’ll need to monitor this over a longer period of time.

Cautions and Considerations

  • It’s critical to think about the human dynamics that can influence contributor sustainability and to treat people with kindness and respect as you work to make improvements.

Additional Reading

Feedback

We would love to have feedback to learn more about how people are using the CHAOSS Practitioner Guides and how we can improve them over time. Please complete this short survey to provide your feedback.

Contributors

The following people contributed to this guide:

  • Dawn Foster
  • Chan Voong
  • Jeffrey Osier-Mixon
  • Luis Cañas Díaz

References

CHAOSS Practitioner Guides are MIT licensed, living documents, and we welcome your feedback and input. You can suggest edits to this document at https://github.com/chaoss/wg-data-science/blob/main/practitioner-guides/contributor-sustainability.md