Newcomer Experience
Question: How well does an open source community attend to welcoming newcomers?
Description
Projects want to provide an environment in which new community members can actively listen, engage, and contribute to an open source community. Newcomer experience (e.g., onboarding and orientation) is a way of measuring and understanding how open source communities welcome and support new community members.
Objectives
- Determine how your community is designed to improve the experience of new community members
- Determine how effective the documentation is in helping new community members take the first steps to become independent contributors
- Determine how your community recognizes and supports new community members
- Determine how your community provides resources for new community members
- Determine potential areas for improvement in diversity, equity, and incluion issues for new community members in particular
Implementation
Filters
The newcomer experience can be filtered on:
- Member Demographics
- Types of Contributions
- Amount of a person’s open source experience
- Time between a person’s first and subsequent engagements within a project
- New contributors
- Occasional contributors
- Inactive contributors
Tools Providing the Metric
Data Collection Strategies
Quantitative
- Observe second, third, or fourth contributions from first-time contributors
- Observe New Contributors Closing Issues to determine if new contributors are able to close issues
Qualitative
- Survey: This could include newcomer experience questions such as:
- Survey Likert item (1-x): I feel welcome in the community
- Survey Likert item (1-x): I feel like I am making a contribution to the community
- Survey Likert item (1-x): I am satisfied with the current community structure
- Survey Likert item (1-x): I would like to help more in the community
- Survey Likert item (1-x): I have the ability to have an impact in the community
- Survey Likert item (1-x): The community treats new community members well
- Open ended question: What's something that the community lacks and you wish it didn't (e.g., more in-person events, more research on DEI, more meetings)?
- Open ended question: When you first started working with the community, what were some obstacles you encountered (e.g., couldn't find information on getting started, couldn't find meeting times, wasn't sure where to start)?
- Effectiveness of Mentorship programs specifically aimed at supporting newcomers.
- Effectiveness of Newcomer onboarding training
- Effectiveness of Peer-to-peer mentorship programs internal to the community
- Effectiveness of Project Documentation through related metrics:
References
- Blincoe, K., Sheoran, J., Goggins, S., Petakovic, E., & Damian, D. (2016). Understanding the popular users: Following, affiliation influence and leadership on GitHub. Information and Software Technology, 70, 30–39.
- Canfora, G., Di Penta, M., Oliveto, R., & Panichella, S. (2012). Who is Going to Mentor Newcomers in Open Source Projects? Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering, 44:1-44:11. https://doi.org/10.1145/2393596.2393647
- Padala, H. S., Mendez, C., Fronchetti, F., Steinmacher, I., Steine-Hanson, Z., Hilderbrand, C., Horvath, A., Hill, C., Simpson, L., Burnett, M., Gerosa, M., & Sarma, A. (2022). How Gender-Biased Tools Shape Newcomer Experiences in OSS Projects. IEEE Transactions on Software Engineering, 48(1), 241–259. https://doi.org/10.1109/TSE.2020.2984173
- Steinmacher, I., Graciotto Silva, M. A., Gerosa, M. A., & Redmiles, D. F. (2015). A systematic literature review on the barriers faced by newcomers to open source software projects. Information and Software Technology, 59, 67–85. https://doi.org/10.1016/j.infsof.2014.11.001
- Steinmacher, I., Wiese, I. S., Conte, T., Gerosa, M. A., & Redmiles, D. (2014). The Hard Life of Open Source Software Project Newcomers. Proceedings of the 7th International Workshop on Cooperative and Human Aspects of Software Engineering, 72–78. https://doi.org/10.1145/2593702.2593704
Known Contributors
- Anita Ihuman
- Matt Germonprez
- Sean Goggins
- Ruth Ikegah
- Katie Schueths
- Amy Marrich
- Kristi Progri
- Elizabeth Barron
- Dawn Foster
- Kevin Lumbard
To edit this metric please submit a Change Request here: https://github.com/chaoss/wg-dei/blob/main/focus-areas/project-and-community/newcomer-experience.md
To reference this metric in software or publications please use this stable URL: https://chaoss.community/?p=4891
The usage and dissemination of health metrics may lead to privacy violations. Organizations may be exposed to risks. These risks may flow from compliance with the GDPR in the EU, with state law in the US, or with other laws. There may also be contractual risks flowing from terms of service for data providers such as GitHub and GitLab. The usage of metrics must be examined for risk and potential data ethics problems. Please see CHAOSS Data Ethics document for additional guidance.