Code sharing in clinical prediction model research remains limited and inconsistent, according to an AI-assisted scoping review of open-access published literature. The analysis, published in Nature Medicine, examined code availability across studies that develop or evaluate clinical prediction models and found that a substantial share of publications do not make their code publicly accessible.
The review also revealed significant variability in sharing practices and in the quality of documentation that accompanies shared code. Even when code is released, the level of detail, usability, and supporting information varies widely, making it difficult for other researchers to reproduce results or build on existing models.
Clinical prediction models are widely used to estimate the risk of disease, guide treatment decisions, and support clinical workflows. Their reliability depends on transparency, because other researchers and clinicians need to verify how a model was built, which variables it uses, and how it performs in different populations. Without access to the underlying code, independent validation becomes far more difficult.
The scoping review used artificial intelligence to help screen and analyze the open-access literature, a method that allowed researchers to assess a large body of publications more efficiently than traditional manual review. The approach focused on code availability statements and the actual presence of shared code, rather than relying solely on authors’ declarations.
The findings point to a persistent gap between the principles of open science and actual practice in clinical prediction modeling. Journals and funders increasingly encourage or require data and code sharing, but the review suggests that policies have not yet translated into routine, high-quality code release across the field.
Poor documentation compounds the problem. When code is shared without clear instructions, dependency information, or example data, it can be nearly impossible for others to run it. That limits reproducibility and slows the translation of prediction models from research into clinical tools.
The review’s authors note that variability in practices makes it hard to compare studies or assess the overall state of code sharing. Standardized reporting and clearer expectations for what constitutes usable code could help close the gap.
For clinicians and health systems, the stakes are practical. Prediction models increasingly inform decisions about diagnosis, treatment, and resource allocation. If the code behind those models cannot be inspected or tested, institutions have fewer ways to confirm that a model is appropriate for their patients.
The analysis adds to a growing body of evidence that open science practices in medicine are uneven. While data sharing has received considerable attention, code sharing has lagged, even though it is often essential for reproducing computational research.
The review calls attention to the need for better incentives, clearer journal requirements, and training for researchers on how to share code effectively. Without such changes, the field risks continuing to produce prediction models that cannot be fully verified or reused.
The findings are based on an analysis of open-access published literature and do not assess code sharing in paywalled or non-public studies. Still, the results offer a broad picture of current practices and highlight where improvement is most needed.
16
