Jenkins 2023 Recap

Contributed by: Wadeck Follonier

The Jenkins Security team has multiple missions, with the most visible to users being the publication of advisories.

In 2023, the team published 17 advisories: 4 included Jenkins core, and 13 were solely about plugins. In total, 211 vulnerabilities were announced.

In terms of reporting trends, we have seen an increase in people reporting CVEs originating from dependencies. Our position on this issue is that when analyzing CVEs, if they are impacting we correct them and publish an advisory; otherwise, we publish a blog post only for the most popular ones, such as Spring4Shell or Log4Shell. With dozens of CVEs published every month, we prefer not to publish information for each one unless it is relevant.

To reduce the occurrence of those issues and provide reporters with a point of reference, we have updated the security policy in Docker.

We would also like to extend our gratitude to the GitHub Security Lab for reporting 27 vulnerabilities through their CodeQL rules and for their collaborative efforts in both the analysis and the enhancement of our tools.

The team does not rely solely on “external” security researchers to find vulnerabilities; a considerable amount of time during the year is spent on proactively auditing the code. This year was marked by some specific research areas: OAuth flow and disabled SSL validation, in addition to the usual CSRF, XSS, or XXE vulnerabilities.

Additionally, we have expanded the scope of our security reviews for Pull Requests, particularly in Jenkins core. Approximately 70 PRs were labeled as “security-approved”. This proactive measure allowed the team to prevent the introduction of some vulnerabilities, which is significantly less costly than addressing them after they have been incorporated into the production code.

As part of our mission, raising community awareness about security is also a key factor in reducing the introduction of vulnerabilities at an early stage. In this regard, we welcomed Andrea to our security team for a summer internship from June to August. By the end of it, he had successfully reported 28 vulnerabilities. For more information about his internship, you can read his blog post.

Education and tooling are closely related for us. We are investing time to propose new security automation and to improve our existing tooling. This year, we introduced the existing CodeQL ruleset for hosting requests. This provided future maintainers with detailed explanations of the findings. Along with this effort, we also enhanced our security scan by supporting warning suppression through code annotation.

Continuing with tooling improvements, we added a beta feature to the bot present in SECURITY tickets, allowing maintainers to block or unblock releases of their plugins. The intent of this feature is to reduce the likelihood of an involuntary release when one is staged in private for a future advisory.

We also introduced the possibility of having an exclusive CD mode. This option allows maintainers to force all releases of their plugins to be done through the official CI, reducing the risk related to their laptops being compromised.

Nowadays, AI is everywhere. For security, it’s often a bit more complicated due to the aspect of confidentiality. To circumvent this while still delving into the topic, we have been using OpenAI API since October to triage the new JENKINS tickets. It’s the public tracker for Jenkins, but sometimes, vulnerability reports are wrongly submitted there. AI has helped us detect whether something is security-related and thus expands our scope without needing to monitor everything. This approach has proven to be quite successful after three months.

Along with other small projects, we continue to work on more long-term/background projects, such as the introduction of the Content-Security-Policy header. Throughout the year, we made significant progress in covering the core usage of inline JavaScript.

Contributed by: Mark Waite

Jenkins is a project of the Continuous Delivery Foundation. We’re grateful for the many different ways that the Continuous Delivery Foundation supports the Jenkins project. They provide financial support for Jenkins infrastructure, promotional support for Jenkins initiatives, infrastructure support for the Jenkins issue tracker, and consultation on many other topics. The Jenkins project was well represented at cdCon 2023 and at cdCon Japan.

Contributed by: Damien Duportal

Infrastructure costs decreased by 21% in 2023 while usage increased. Jenkins use is increasing worldwide and Jenkins infrastructure continues to meet the growing needs.

Jenkins infrastructure broadened its operating environments in 2023 by expanding its use of ARM64 processors to reduce costs. Additional software improvements were implemented in the Jenkins Pipeline to reduce costs while maintaining reliability.

Costs were further reduced by the Artifactory bandwidth reduction project. We’ve gone from 50 TB per month to 15 TB per month of artifact repository bandwidth use, thanks to help from JFrog and implementations of artifact caching proxies by the infrastructure team.

We’re grateful for donations from our organizational sponsors like CloudBees, the Continuous Delivery Foundation, and the Linux Foundation and for our infrastructure sponsors including GitHub, JFrog, Atlassian, Microsoft, DigitalOcean, Netlify, PagerDuty, Discourse, Datadog, and Cloudflare. We deeply appreciate the mirror providers around the world like Oregon State University Open Source Lab, Tsinghua University, XMission, Yamagata University, Servana, Belnet, and RWTH Aachen University.

Contributed by: Mark Waite

The badly outdated and unmaintained Prototype.js JavaScript library has been removed from Jenkins core and from over 50 Jenkins plugins. Special thanks to Basil Crow and Tim Jacomb for their work to identify and replace Prototype.js references in Jenkins JavaScript.

User interface look and feel improvements continued throughout 2023. Improvements were delivered in forms, menus, and pages. Menus are easier to navigate and easier to understand.

Multiple improvements were released for the Jenkins management pages, including the new “Appearance” page that makes it easier to use Jenkins themes. Forms, pages, and menus have all been enhanced in 2023.

Navigation of cloud management pages is simpler and clearer thanks to layout and navigation enhancements.

Hundreds of user interface improvement pull requests have been merged this year. Thanks to all those contributors!

Contributed by: Bruno Verachten

The year 2023 has been a significant period for the Jenkins project, marked by notable advancements and achievements. The community as a whole has played a crucial role in shaping the project’s trajectory.