Dependencies and Class Loading

Jenkins has a complex, modular architecture. To enable plugins to use the rich array of libraries available in the Java ecosystem, and build on one another using plugin-to-plugin APIs, the Jenkins plugin extension mechanism goes beyond simple plugin manifests.

The de facto build tool for Jenkins plugins is Apache Maven. All examples here will use Maven. Building plugins using Gradle is also possible but particular capabilities may lag behind.

Jenkins class loading

Before diving into how to refer to libraries and other plugins from your plugin, it is useful to understand the elements of Java class loading and how that applies to Jenkins. If you are not a Java developer, you can get away with ignoring all this for simple plugins, but will need to understand class loading to troubleshoot strange errors or perform advanced packaging.

The plugin class loader hierarchy

□ Java Platform
 ↖
  □ “application classpath” (servlet container): java -jar jenkins.war
   ↖
    □ Jenkins core: jenkins.war!/WEB-INF/lib/*.jar
     ↖
      □ plugin A: $JENKINS_HOME/plugins/a.jpi!/WEB-INF/lib/*.jar
       ↖                                                             ↖
        □ plugin C: $JENKINS_HOME/plugins/c.jpi!/WEB-INF/lib/*.jar  ← □ UberClassLoader
     ↖ ↙                                                             ↙
      □ plugin B: $JENKINS_HOME/plugins/b.jpi!/WEB-INF/lib/*.jar
      ⋮

Java class loaders have parents to which they delegate. For purposes of Jenkins, what matters most is that “Jenkins core” delegates to the Java Platform, and Jenkins plugins all delegate to Jenkins core.

Plugins may also delegate to one another. In the above example, C declares dependencies on A and B. In its c.jpi!/META-INF/MANIFEST.MF this would look something like:

Plugin-Dependencies: a:1.13,b:1.6

Thus C can directly refer to classes defined in either A or B, as well as to Jenkins and the Java Platform:

package org.jenkinsci.plugins.c;
import java.util.Date;
import jenkins.model.Jenkins;
import org.jenkinsci.plugins.a.Alpha;
import org.jenkinsci.plugins.b.Beta;
public class Charlie {/* … */}

Each (enabled) plugin gets its own ClassLoader. It is possible for two distinct plugins to define a class of the same name, so long as neither depends on the other.

There is also a single special UberClassLoader which delegates to all enabled plugins. This never loads any additional resources but it can be used to look up any plugin class or resource.

Initiating vs. defining loaders

When you initiate loading of a class

Class<?> alpha = Charlie.class.getClassLoader().loadClass("org.jenkinsci.plugins.a.Alpha");

you are asking a particular ClassLoader (here, C’s) to look up a class by name. (Normally this will start by asking all of its parents to find that class, and only look in c.jpi!/WEB-INF/lib/*.jar as a last resort.)

What happens if Alpha.class refers to various other classes, for example by using them in import statements? Java considers the defining loader of Alpha.class (A’s), and only asks that class loader. The fact that you started loading in C is irrelevant. This means that C cannot provide some class A might need to link against. A needs to be able to “see” any such class on its own. Otherwise you will get a NoClassDefFoundError at runtime.

Context class loaders

Java defines a Thread.getContextClassLoader(). Jenkins does not use this much; it will normally be set by the servlet container to the Jenkins core loader.

Some Java libraries have a fundamental design flaw, originating in premodular systems with a “flat classpath”, whereby they expect classes defined by clients of the library to be available by name without any qualification:

package org.somelib;
public class LibClass {
    public static Object use(String name) throws Exception {
        return getUserClass(name).newInstance();
    }
    private Class<?> getUserClass(String name) throws ClassNotFoundException {
        return Class.forName(name);
    }
}

This does not work properly in Jenkins because the library org.somelib might be defined in plugin A, while the user class is defined in plugin B depending on A. Class.forName(String) uses the calling class (LibClass) to determine the initiating ClassLoader; since plugin A cannot load classes from plugin B, this will result in a ClassNotFoundException.

Other libraries try to work around the issue as follows:

package org.somelib;
public class LibClass {
    public static Object use(String name) throws Exception {
        return getUserClass(name).newInstance();
    }
    private Class<?> getUserClass(String name) throws ClassNotFoundException {
        return Class.forName(name, true, Thread.currentThread().getContextClassLoader());
    }
}

This also fails by default in Jenkins, since the contextClassLoader can see only Jenkins core (not even plugin A, much less B). Jenkins plugin code can work around the issue in some cases:

package org.jenkinsci.plugins.b;
import org.somelib.LibClass;
class UsesThatLib {
    Object someMethod() {
        Thread t = Thread.currentThread();
        ClassLoader orig = t.getContextClassLoader();
        t.setContextClassLoader(UsesThatLib.class.getClassLoader());
        try {
            return LibClass.use(UsesThatLib.class.getName());
        } finally {
            t.setContextClassLoader(orig);
        }
    }
}

(When the particular class loader needed is unclear, UberClassLoader can be used as a fallback, though this is not as safe since lookups could be ambiguous in case two unrelated plugins both bundled the same library.)

Ultimately however it is a design flaw in the library if it fails to allow clients to directly specify a ClassLoader to use for lookups (or preregister Class instances for particular names). Consider patching the library or looking harder for appropriate APIs that already exist. As an example, java.io.ObjectInputStream (used for deserializing Java objects) by default uses a complicated algorithm to guess at a ClassLoader, but you can override resolveClass to remove the need for guessing (as hudson.remoting.ObjectInputStreamEx in fact does).

Views and other resources

Jenkins plugins normally contain “Stapler views” like config.jelly as well as other resources in src/main/resources/. While you can explicitly load these from Java code:

package org.jenkinsci.plugins.a;
public class Alpha {
    /** loads {@code /org/jenkinsci/plugins/a/config.txt} from {@code a.jpi!/WEB-INF/lib/a.jar} */
    static URL config() throws IOException {
        return Alpha.class.getResource("config.txt");
    }
}

normally such resources would be loaded on your behalf, for example by the convention of Jenkins looking for a view. In such cases the lookup passes through UberClassLoader, so your resource path (/org/jenkinsci/plugins/a/config.txt) had better be globally unique.

Messages.properties used for localization is a little different, since this is actually compiled to Messages.class during the build, and thus behaves like any other Java class referred to statically from your plugin code:

package org.jenkinsci.plugins.a;
public class Alpha {
    /** compiled from {@code /org/jenkinsci/plugins/a/Messages.properties#Alpha.message} */
    static String message() throws IOException {
        return Messages.Alpha_message();
    }
}

Depending on other plugins

Making your plugin depend on other plugins is easy: just declare dependencies in your POM, by hand or using your favorite IDE.

<dependencies>
    <dependency>
        <groupId>org.jenkins-ci.plugins</groupId>
        <artifactId>a</artifactId>
        <version>1.13</version>
    </dependency>
    <dependency>
        <groupId>org.jenkins-ci.plugins</groupId>
        <artifactId>b</artifactId>
        <version>1.6</version>
    </dependency>
</dependencies>

The Maven packaging type for Jenkins plugins understands to translate this to the Plugin-Dependencies manifest header, which will be understood by the Jenkins plugin manager, as well as the update center and other tools.

The Maven compiler plugin similarly understands that a-1.13.jar and b-1.6.jar should be added to your classpath when building your plugin.

Extensions and inversion of control

A “service locator” pattern is used throughout Jenkins for modularity and extensibility. For example, if a plugin (or core) defines an API

package org.jenkinsci.plugins.someapi;
import hudson.ExtensionPoint;
public interface Checker extends ExtensionPoint {
    boolean doesThisSeemOK(String input);
}

then another plugin may declare a dependency on that API

<dependency>
    <groupId>org.jenkins-ci.plugins</groupId>
    <artifactId>someapi</artifactId>
    <version>1.0</version>
</dependency>

and add an extension:

package org.jenkinsci.plugins.somethingelse;
import hudson.Extension;
import org.jenkinsci.plugins.someapi.Checker;
@Extension
public class MyChecker implements Checker {
    @Override
    public boolean doesThisSeemOK(String input) {
        return !input.contains("/");
    }
}

Now any code able to link against someapi can use those implementations; most commonly this is done inside the same API plugin:

package org.jenkinsci.plugins.someapi;
import hudson.ExtensionList;
class RunsChecks {
    static boolean allFine(String input) {
        for (Checker c : ExtensionList.lookup(Checker.class)) {
            if (!c.doesThisSeemOK(input)) {
                return false;
            }
        }
        return true;
    }
}

It is important to understand that while MyChecker needs to link against Checker, mandating that dependency, RunsChecks does not need to be able to link against MyChecker (or any of the other implementations). While the local variable c’s implementation class might be in the somethingelse plugin, it need only care about the declared type Checker.

Bundling third-party libraries

Sometimes plugins need to use Java libraries beyond what is available in the Java Platform and Jenkins itself. For example, a plugin connecting to a particular service might use a Java SDK provided by the vendor.

Doing this is very easy—in principle. Simply declare a Maven dependency on that library:

<dependency>
    <groupId>com.yoyodyne.cloud</groupId>
    <artifactId>cloud-access-sdk</artifactId>
    <version>1.0</version>
</dependency>

(This assumes that the library is available in Maven Central. If not, it is possible to upload artifacts to the Jenkins Artifactory repository for use from plugins. Ask on the developer list for help. Do not attempt to keep such binaries in source control.)

Besides making SDK classes (say, com.yoyodyne.cloud.*) available during compilation, the maven-hpi-plugin used to create Jenkins plugins will notice that this dependency is not itself a Jenkins plugin, and instead bundle it inside yourplugin.hpi as WEB-INF/lib/cloud-access-sdk-1.0.jar.

At runtime, the plugin class loader will load classes from WEB-INF/lib/cloud-access-sdk-1.0.jar, just as it would from WEB-INF/lib/yourplugin.jar (your plugin’s own code, from src/main/java/ and src/main/resources/). Thus your plugin’s classes can refer to classes in that library. Other plugins depending on your plugin can, too.

Checking WEB-INF/lib/*.jar for junk

Beware that Maven dependencies include all transitive dependencies. This can lead to unexpected results when bundling libraries. For example, the POM for com.yoyodyne.cloud:cloud-access-sdk might declare that it needs commons-net:commons-net:3.5. Your plugin will thus wind up bundling commons-net-3.5.jar as well. If you are not careful, WEB-INF/lib/ may fill up with megabytes of stuff which is not actually used.

Using library wrapper plugins

In practice it is undesirable for Jenkins feature plugins to bundle assorted third-party libraries. Typically other plugins will need some of the same libraries, so multiple plugins will wind up bundling copies. Even if all of these copies happen to be the exact same version, “downstream” plugins can wind up getting linkage errors. And users will wind up downloading multiple copies of the same code, increasing product and $JENKINS_HOME/plugins/ sizes, update center load, HotSpot compiler delays, etc.

Another problem is that updating library versions becomes harder to centralize when they are bundled independently in multiple plugins. While having each plugin define an exact version of a library does reduce the risk of API compatibility errors, this is outweighed by the need to update a library with assurance when new security vulnerabilities (or other critical fixes) are announced.

To centralize library management, you can instead define a wrapper plugin, or find one someone else has already defined. This is a Jenkins plugin which contains no sources of its own and simply includes a dependency on some library. After being published on the update center, other “feature” plugins can declare plain plugin-to-plugin dependencies on it and thus use the library.

Occasionally it can make sense to include a few API classes in the wrapper plugin itself, where any Jenkins code using the library would reasonably need some boilerplate adapters to standard Jenkins facilities.

To mitigate the risk of library plugin updates with incompatible changes breaking plugin functionality, it is suggested to include the major version of the library in the wrapper plugin’s artifactId: for example, commons-lang3. (This assumes that the library follows something like Semantic Versioning.) Thus there could be multiple major releases loaded simultaneously. Of course this means that critical fixes must be issued as updates to all release lines, so only do this for supported versions of the library.

pluginFirstClassLoader and its discontents

The default behavior of Java class loaders, and of Jenkins plugin class loaders as well, is to service loadClass requests first by asking the parent loader(s) for the class of that name, and only if that fails to check whether the class could be defined among JARs present in the initiating loader. This is generally sensible since it ensures that types mentioned in bytecode from different plugins are resolved at runtime to the same Class objects and thus allowing APIs to work using shared type signatures.

Sometimes, however, it is not wanted for the parent to be searched first. For example, Jenkins core currently bundles a plethora of third-party libraries and exposes all of their packages to plugins. For that matter, some plugins bundle third-party libraries that are then incidentally exposed to other plugins needing a dependency on APIs defined in the bundling plugins. If a plugin also needs some variant of the same library for its own use, it will surprisingly not be able to load it, because the parent class loader will find it first.

To avoid that behavior, it is possible to set a flag pluginFirstClassLoader in the plugin’s Maven POM. (This simply produces a JAR manifest entry that is interpreted by the Jenkins plugin system at runtime.) This flag instructs the plugin class loader to check its own JARs first for any mentioned class names. You must be very careful when using this mode, since any type normally defined in core (or an “upstream” plugin) which is mentioned in any part of the Java signature of a method you are calling must not be duplicated in your JARs, or linkage errors will result. Thus it is best reserved to libraries used purely by internal implementation.

A more limited flag is maskClasses, which blocks only selected classes or packages from the parent loader, rather than everything. You must manually verify that the masked classes are complete under the transitive closure of the Java linker: for example, masking one package but not another from a library bundled in core could make classes in the masked package unresolvable.

If you did not understand any part of this section, do not use these options. Even if you did, think twice.

Shading

Another approach to taming the complexity of library dependencies and class loading is loosely referred to as shading, exemplified by the Maven Shade plugin (though multiple techniques are possible). In this case, rather than trying to control where a given classLoader.loadClass("org.apache.commons.lang.StringUtils") call finds the defining loader, the idea is to bundle the entire library under a distinct package prefix, rewriting all static and reflective class (or resource) loads, both within the library and from code defined in the plugin doing the bundling. Thus your-plugin.hpi!/WEB-INF/lib/commons-lang-shaded.jar might contain entries like org/jenkinsci/plugins/yourplugin/commonslang/StringUtils.class; and plugin source code like

import org.apache.commons.lang.StringUtils;
// …
if (StringUtils.isEmpty(arg)) {/* … */}

would result in bytecode in your-plugin.hpi!/WEB-INF/lib/classes.jar referring in its constant pool to the type org.jenkinsci.plugins.yourplugin.commonslang.StringUtils. Since that type name is unique in the whole Jenkins JVM, there is no risk of it being loaded from the wrong place; from the perspective of Jenkins, it is as if you copied and pasted the whole Commons Lang library into some sources in your plugin.

While this system does address the risk of linkage errors, it does nothing to reduce the profusion of library versions in Jenkins, as described in the section on library wrappers. In some cases, however, library wrappers themselves will use this same trick for official purposes, so that the wrapped library will be present under a package name indicating the major release version. Plugins using the wrapped library therefore would refer to the repackaged name:

import org.jenkinsci.commons_lang2.StringUtils;

@Restricted annotations

The public modifier in Java allows types, methods, constructors, and fields to be accessed from any other class in the system capable of linking against the defining type. Thus it forms part of the API, as do protected members.

However in some cases use of these access modifiers are forced for technical reasons rather than out of an intent to define an API: a method might need to be accessed from a class in another package in the same plugin, preventing use of default package access; an @Extension needs to be public to allow the Jenkins service loader to instantiate it; a FormValidation doCheckName(@QueryParameter String value) method must be public to expose it as a Stapler web method and thus to JavaScript on a form.

In such cases you should block the member from being used by outside code: nonessential API additions are at risk of being used in unintended ways and forcing your plugin to maintain the member more or less forever lest backward compatibility be broken. This can be accomplished by use of a special annotation available in Jenkins code:

@Restricted(NoExternalUse.class)
@Extension
public class MyListener extends ItemListener {/* … */}

Other plugins attempting to refer to MyListener will receive a build-time error. Therefore you are free to rename, move, delete, or otherwise modify MyListener at any time.

Several kinds of restriction are available; consult Javadoc for details.

This system has no effect on accesses using java.lang.reflect.

JenkinsRule vs. acceptance-test-harness class loading

There are three main categories of automated test used in Jenkins:

  • Unit tests, including mocking frameworks such as PowerMock.

  • Functional tests based on the JenkinsRule API defined in jenkins-test-harness.

  • Acceptance tests located in the acceptance-test-harness repository.

These have different levels of fidelity to the class loading behavior of plugin code running in production Jenkins servers. Unit tests simply pick up the Java classpath (java.class.path) defined by Maven’s test scope.

Acceptance tests run a full Jenkins server and install plugins (including the plugin(s) being tested) using Jenkins’ normal mechanisms. Since the test does not compile or link against any types defined in the Jenkins runtime (only against the Selenium web driver), and does not even run in the same JVM as Jenkins, it has no interaction with the class loading of Jenkins. Thus the class loading behavior of plugins running in an acceptance test can be assumed to be the same as in production.

Class loading in functional tests is intermediate in behavior, but closer to that of unit tests. Test code does link against Jenkins core and (test-scoped) plugin types, and everything in the Java classpath is in fact loaded in Java’s application class loader—including plugins in test scope. This means that certain mistakes in plugin metadata (for example, misuse of pluginFirstClassLoader) may go unnoticed.

(JenkinsRule does start a real Jenkins service, and in some cases other plugins can get installed which were not in the Java classpath. These get their own class loaders.)

Whenever there is any doubt about whether class loading behavior could affect plugin or core code, use RealJenkinsRule which launches Jenkins in a separate JVM with the regular class loader hierarchy.

Jenkins modules

Historical versions of jenkins.war included a few components called modules which are built and packaged just like plugins, and can refer to types defined in Jenkins core, but which are bundled alongside core and cannot be managed by users in the plugin manager. The usual reason for this design is to include features which either cannot be disabled, or must be present early in the Jenkins startup sequence, before plugins have been fully initialized.

For purposes of class loading, these components behave like anything else bundled inside core: modules do not get their own class loaders. For Maven dependency management purposes, plugins can declare provided-scope dependencies on these modules if they wish to use their APIs, taking care to select the same version as is bundled in the baseline version of core.

JEP-230 removed this system at least from the default Jenkins distribution.