Several months ago, we ran a survey to better understand the CHAOSS community. Thank you so much to those who participated in that survey! We really appreciate your valuable input. From the survey, we were able to identify five themes that you told us about. We’ll describe those five themes here and discuss what we are doing in the CHAOSS project to help address these themes.
| Theme 1: Clear Communication | |
| Comments | What We’ve Done in CHAOSS |
| Processes about how to check-in with the wider community would be helpful. | Each regional chapter hosts their own Community Meetings and we post those meeting summaries and links to the meeting recordings in Slack and on Discourse, so anyone in the community can quickly stay up to date on what’s going on in other parts of CHAOSS. We will continue to improve cross-chapter and cross-group communication among the entire CHAOSS community through our newsletters and weekly CHAOSS Community meetings. |
| It is hard to navigate the detail of the focus areas | Sifting through the ever growing list of CHAOSS metrics has proven to be challenging for our users and those new to the community. Previously, focus areas were used to categorize metrics into contextual buckets. The user would have to be familiar with which working group developed the metric, and which focus area we decided provided the best fit. This required a level of knowledge of CHAOSS and the way we work that was a barrier for those new to the community. We have since moved away from displaying metrics in this way, and have moved toward a more intuitive model of categorization. First we assign 1-2 high-level topics to each metric, then we assign a number of keywords, which provide more context. This gives the user the ability to search for more specific criteria. Also, we improved the UX by no longer displaying metrics in one single document. Now there are visually clear “tiles” on our website that allow users to easily explore our metrics based on the high-level topic they’re interested in. With the addition of our metrics models, we also help the user identify sets of metrics that naturally come together. |
| Theme 2: Working Groups | |
| Comments | What We’ve Done in CHAOSS |
| Sunsetting working groups may be a good thing. | The structure of CHAOSS has evolved over the past year to shift away from the concept of “working groups that develop metrics through a particular lens” and more toward the context in which metrics can be used to help answer questions. Because some metric development benefits from specific areas of expertise, we chose to retain our DEI and Risk Working Groups, but have effectively folded the rest into other groups. We now have 3 working groups focused on metric development: Risk, DEI, and Common. |
| Preference for the real use cases in metrics models | With the hiring of our Director of Data Science and the launch of our Context Working Groups, bridging the gap between theoretical and practical use of our metrics and metrics models will be a primary focus of CHAOSS moving forward. |
| Contextual work can be distinct from metrics working groups, and this is good. | As mentioned previously, the evolution of CHAOSS means a shift toward a separation of Context Working Groups and the working groups responsible for metrics development. In this way, we are able to collaborate with our user community to answer the questions that arise in their individual contexts. We now have 3 Context Working Groups where these high-level discussions occur: Corporate OSPOs, University OSPOs, and Open Source Scientific Software. An outgrowth of our continued discussions with CHAOSS’s expanding community may be additional Context Working Groups. |
| The Common WG seems to be where “things that don’t clearly go anywhere fit”. | The Common Working Group has indeed been through a few iterations! It does seem to act as a catch-all, especially with regard to metric development. But as we have pared down the number of Working Groups, it makes more sense that this development would land in Common. CHAOSS will continue to communicate that message and clarify the mission of the Common Working Group. |
| Theme 3: Software | |
| Comments | What We’ve Done in CHAOSS |
| Build community around software. | This has always been challenging for us, and it’s something we continue to strive for. With the hiring of our Director of Data Science, and conversations with potential partners, we aim to allocate and acquire additional dedicated resources for this effort. This will likely be a focus for us in 2024, as it is critical to the sustainability of CHAOSS software. |
| Theme 4: Newcomer Experiences | |
| Comments | What We’ve Done in CHAOSS |
| Improve processes of where to begin, especially when a person comes to CHAOSS with an interest where work is being done in the many places. | Newcomer experience has been a primary focus for the past 12-18 months. With the creation of a newcomers Quickstart, we have a central document with a few action items for newcomers to complete. This allows them to learn about CHAOSS and be more prepared to fully join our community. We are improving the long term onboarding experience by moving away from a monthly onboarding call (which was overwhelming and not as inclusive as it could be because of time zones) toward building out a set of learning modules for newcomers to work through at their leisure. This will occur in a learning management system and will allow newcomers to move at their own pace, and learn about CHAOSS in digestible chunks. We also launched a Tour Guide program whereby newcomers can receive individual attention and guidance to find the spaces within CHAOSS where they might have an interest in participating. |
| Promote new opportunities for people to engage with the community as they come up. | As mentioned previously, our Tour Guides act as a link between opportunities and new contributors, but we are also organizing the many sub projects within CHAOSS through a more structured project management framework. By bringing together contributors with project management experience, we can allow them to contribute by keeping these smaller projects on track, clarifying engagement opportunities, and communicating them regularly to our community. We also promote the use of the “First Timers Only” labels on issues in our many repositories. |
| Make “where to begin” clearer in documentation. | With the development of our Quickstart, and prominent links to this on the CHAOSS website homepage, newcomer Slack channel, and in the newcomer Slack Bot, we now have one central place where we can point newcomers. |
| Theme 5: Community Experiences | |
| Comments | What We’ve Done in CHAOSS |
| As the community has grown, it is beginning to feel more fragmented. | This is a challenge for any global community with a lot of moving parts. We have many Slack channels and many sub-projects and it’s difficult to keep track of what is going on with everyone. We implemented a Discourse Forum with the idea that it would be a way to bring the whole community together in a more substantial way than many fragmented Slack channels, but this has not been as well received as we hoped. The CHAOSS community will continue to iterate on and think about ways to combat this issue. |
| A learning pathways framework would be helpful. | As mentioned, CHAOSS is currently building out such a framework using a learning management system. We think this will bring clarity to the onboarding process and help educate potential contributors so they can actively engage with the community in the way that means the most to them. |
| There are times that focus on tools becomes too central. | While tools are a big part of CHAOSS (even appearing in our name), they are definitely not the only thing CHAOSS works on. We hope that with clearer communication on all the sub-projects happening in CHAOSS, it will not feel like our tools take center stage. |
| There may be too many working groups. | As mentioned, we have effectively cut our metrics development working groups down to 3. |
| Provide more workshops, peer mentoring, and programs on software and metrics (perhaps also newcomer experience) | We plan to include some of this type of education in our learning management system modules. Currently, we don’t have the individual resources to consistently host other types of workshops (aside from the occasional CHAOSScon). By making it easier to learn about CHAOSS, we hope to grow our community of leaders who can facilitate these types of workshops. It would be outstanding to be able to provide these experiences for our community! |
