Manage Jenkins agents with Kubernetes native PodTemplate
Jenkins has great integration with Kubernetes. No matter the controller or agents, they all work well in Kubernetes Pods. Configuration as Code is an awesome idea. People don’t need to open the UI page of Jenkins again and again. We can put all the system configuration into a git repository. You might already feel tired to edit a super huge Kubernetes YAML file if there are many many Jenkins agents that need to be maintained. It’s pretty likely to make a mistake in a big YAML file.
I’m going to provide a solution to make our life easier maintaining Jenkins . You only need to update a ConfigMap if you want to make any changes to the Jenkins system configuration. And you only need to create a Kubernetes native PodTemplate resource if you want to add a new Jenkins agent. To achieve this goal, you only need to install an extra Kubernetes deployment.
Please allow me to show you how to do it. Before going forward, let’s assume you already have a Kubernetes cluster, and you have enough permission to access the relevant resources.
Firstly, put all the Jenkins system config into a ConfigMap, such as:
apiVersion: v1
data:
jenkins_user.yaml: |
jenkins:
mode: EXCLUSIVE
numExecutors: 0
scmCheckoutRetryCount: 2
disableRememberMe: true
clouds:
- kubernetes:
name: "kubernetes"
serverUrl: "https://kubernetes.default"
skipTlsVerify: true
kind: ConfigMap
metadata:
name: jenkins-casc-config
namespace: kubesphere-devops-systemSecondly, let your Jenkins load its configuration from a ConfigMap. Please see also the following important parts:
spec:
template:
spec:
containers:
- image: ghcr.io/linuxsuren/jenkins:lts
env:
- name: CASC_JENKINS_CONFIG
value: "/var/jenkins_home/casc_configs/" # loading config file from a directory that was mount from a ConfigMap
volumeMounts:
- mountPath: /var/jenkins_home/casc_configs
name: casc-config # mount from a volume
volumes:
- configMap:
defaultMode: 420
name: jenkins-casc-config # clamin a ConfigMap volume, all the CasC YAML content will be here
name: casc-configthen, the core part is the Kubernetes controller. Please see the following code snippet:
apiVersion: apps/v1
kind: Deployment
metadata:
name: jenkins-agent
namespace: kubesphere-devops-system
spec:
template:
spec:
containers:
- image: kubespheredev/devops-controller:dev-v3.2.1-rc.3-6726130
name: controller
args:
- --enabled-controllers
- all=false,jenkinsagent=true,jenkinsconfig=true # only enable the necessary features of this controllerThis controller will watch all the PodTemplates (containing the label jenkins.agent.pod), then convert it to Jenkins-style PodTemplate, merge it into
the CasC YAML file finally.
When everything is setup, you can try to add a Kubernetes native PodTemplate. A few seconds later, you can use it in your Pipeline. Considering there are many potential technical details, I already put all the necessary files to this git repository. Please feel free to mainain this project if it is helpful in your work.
About the author
