Academic Open Source Project Impact

Question: What is the impact of open source projects that an academician or a team of academicians creates as an important part of a university reappointment, tenure, and promotion process?

Description

Academics need to show evidence of their scholarly output in tenure and promotion cases. Creating an open source project may be an impactful contribution and this metric helps to show this. This metric is for a new project that was created as part of an academic job and released as open source. This metric is not related to open source contributions made to an existing open source project.

Objectives

The goal is to support RPT (Reappointment, Tenure, and Promotion) by drawing forward key open source impact measures, such as:

  • Understanding the scope of the community around the created open source project
  • Understanding the growth, maturity, or decline of the open source project
  • Identifying other projects that depend on your project
  • Identifying journal articles that reference your project

Implementation

The usage and dissemination of health metrics may lead to privacy violations. Organizations may be exposed to risks. These risks may flow from compliance with the GDPR in the EU, with state law in the US, or with other law. There may also be contractual risks flowing from terms of service for data providers such as GitHub and GitLab. The usage of metrics must be examined for risk and potential data ethics problems. Please see CHAOSS Data Ethics document for additional guidance.

Some ideas for how to measure Academic Open Source Project Impact:

  • Impact measured through publication in Journal of Open Source Software
  • Number of downstream-dependencies of software in consideration
  • Something similar to H-Index (doesn’t currently exist)
  • CiteAs API provides standardized citations for open source software
  • Number of downloads
  • Number of contributors
  • Number of software/library citations
  • Number of stars (GitHub)
  • Number of published articles that cite the software or project
  • Downloads of pre-prints that cite the software or project
  • Regularity of updates
  • Lines of Code
  • Number of community contributions, not from the research team
  • Downstream dependencies

Filters

Visualizations

Tools Providing the Metric

Data Collection Strategies

References

Contributors

  • Stephen Jacobs
  • Vinod Ahuja
  • Elizabeth Barron
  • Matt Germonprez
  • Kevin Lumbard
  • Georg Link
  • Sean P Goggins
  • Johan Linaker