Intro to Jenkins with Docker

Jenkins-Docker integration

In the dynamic landscape of software development, the need for efficient and scalable Continuous Integration and Continuous Deployment (CI/CD) processes has become paramount. Jenkins, an open-source automation server, and Docker, a powerful containerization platform, have emerged as dynamic allies in achieving seamless CI/CD scaling. This article will delve into the integration of Jenkins and Docker, exploring how this combination can significantly enhance scalability in your development workflows.

Table of contents

• Introduction to Docker
  • What is Docker?
  • Key concepts of Docker
  • Advantages of Docker
• How Jenkins and Docker work together
• Installing Docker on Windows
  • Hardware prerequisites
  • Specifications
  • Download and install Docker
  • Testing Docker
• How to configure Docker with the configuration file
• How to configure Docker in Jenkins
  • Install Jenkins Docker plugin
  • Configure Docker build agent in Jenkins
  • Test Docker containers in Jenkins
• Troubleshooting common Jenkins-Docker errors
• Conclusion

Introduction to Docker

What is Docker?

Docker is a powerful platform that simplifies application development, deployment, and management by leveraging containerization technology. Containers are lightweight, standalone, and executable units that encapsulate an application and its dependencies. Docker allows developers to create, share, and run applications consistently across different environments, from development to production.

Key concepts of Docker

Containers

Docker containers are encapsulated environments that package an application and its dependencies, ensuring consistency and reproducibility across various stages of the software development lifecycle. They operate as isolated entities, sharing the host operating system’s kernel but maintaining their own file system, processes, and network space.

Images

Docker images operate as the blueprint for containers. They are lightweight, read-only snapshots that include the application code, runtime, libraries, and system tools needed for the application to run. Images are versioned and can be stored in Docker registries, facilitating easy sharing and distribution.

Dockerfile

The Dockerfile is a script used to create Docker images. It contains instructions on assembling the image, including base image selection, application installation, and configuration. Dockerfiles allow for the automation of image creation, ensuring reproducibility and reducing the chances of errors.

Advantages of Docker

Consistency

Docker ensures consistency in the development and deployment process by encapsulating applications and their container dependencies. This consistency reduces the possibility of issues arising from differences between development, testing, and production environments.

Scalability

Containers can be easily imitated and scaled horizontally to accommodate changes in demand. Docker’s lightweight nature and quick startup times make it ideal for dynamically scaling applications based on workload.

Resource Efficiency

Docker containers share the host operating system’s resources, minimizing overhead and allowing for efficient use of system resources. This results in higher density and cost-effectiveness compared to traditional virtualization.

Rapid Deployment

Docker facilitates rapid application deployment by eliminating the need for complex setup processes. Containers can be created quickly, streamlining the development and release cycles.

DevOps Integration

Docker plays a crucial role in DevOps practices by providing a standardized platform for collaboration between development and operations teams. It simplifies the implementation of continuous integration and continuous deployment (CI/CD) pipelines.

How Jenkins and Docker work together

People often ask: Can I use Docker with Jenkins? The answer is a resounding “yes”!

Docker and Jenkins are powerful tools that synergize well to enhance Continuous Integration and Continuous Delivery (CI/CD) pipelines. Docker and Jenkins, a popular automation server, complement each other in various ways within CI/CD workflows:

Consistent Environments

Docker allows you to encapsulate an application and its dependencies into a container, ensuring consistency across different environments. This helps in avoiding the “it works on my machine” issue. Jenkins can leverage Docker containers to create and run builds, ensuring the build environment is identical throughout the development lifecycle.

Isolation and Reproducibility

Docker containers provide isolation for applications and their dependencies. This isolation ensures that each CI/CD pipeline stage runs in a controlled and reproducible environment. Jenkins can utilize Docker to spawn isolated build environments for each job, making it easier to reproduce builds and ensuring that builds are not affected by changes in the underlying infrastructure.

Resource Efficiency

Docker containers are lightweight and share the host OS kernel, making them efficient in terms of resource utilization. This allows for quick and scalable provisioning of build agents. Jenkins can dynamically spin up Docker containers as build agents, optimizing resource usage and enabling parallel execution of jobs.

Artifact Management

Docker images can serve as deployable artifacts containing the application and its dependencies. These images are versioned and can be stored in Docker registries. Jenkins can integrate with Docker registries to push and pull images, facilitating the deployment of consistent and versioned artifacts across different environments.

Scalability and Parallelization

Docker enables parallel execution of containers, allowing for scalable and efficient resource utilization during the testing and deployment stages. Jenkins can orchestrate the parallel execution of Docker containers, enabling the parallelization of tasks within the CI/CD pipeline for faster feedback and reduced build times.

Installing Docker on Windows

With Docker becoming an integral part of daily development, it must be installed on your Windows machine. Here is a step-by-step guide to help you set up Docker on your Windows operating system.

Hardware prerequisites

Ensuring your computer meets three key prerequisites is essential to installing Docker on your system. Firstly, your system should have a 64-bit processor featuring Second Level Address Translation (SLAT), a crucial architectural feature for efficient virtualization. Secondly, 4GB of system RAM is required to support Docker’s operations effectively.

Lastly, enabling hardware virtualization in your computer’s BIOS settings is imperative, ensuring that Docker can leverage virtualization technology for optimal performance and functionality. Meeting these requirements will lead to a smooth and successful Docker installation on your system.

Specifications

For a successful installation of Docker on Windows, ensuring that your operating system meets the specified requirements is imperative. For Windows 11 64-bit, the Home or Pro version 21H2 or higher or the Enterprise or Education version 21H2 or higher is recommended.

Likewise, for Windows 10 64-bit, we advise using the Home or Pro 22H2 (build 19045) or higher or Enterprise or Education 22H2 (build 19045) or higher for optimal performance. The minimum requirements for Windows 10 include the Home or Pro version 21H2 (build 19044) or higher or the Enterprise or Education version 21H2 (build 19044) or higher.

Note
The official Docker page mentions all the required hardware prerequisites and specifications details.

Download and install Docker

Visit the official Docker website to download Docker Desktop, ensuring it’s compatible with Windows 10 or 11.

Note
Refer to the official documentation if you need help installing Docker on Mac or Linux.

Open the downloaded installer to initiate the installation process.

Docker requires a virtualization engine to run Linux on Windows. It is recommended to use WSL 2 (Windows Subsystem for Linux).

Configure Docker to use WSL 2 for optimal performance. Click Ok to start the Docker installation process. And wait for the installation to complete.

For proper registration with Windows, restart your computer after the Docker installation.

After the restart, click on the Docker shortcut on your desktop. Check the license agreement and ensure compliance, especially for large organizations.

Accept the license terms by clicking on the Accept button.

After that, complete your Sign-up process by entering your email address, ID, and password.

The Docker application has been successfully installed, and you can now access it without any issues.

Note
Windows Subsystem for Linux (WSL) 2 is a complete Linux kernel built by Microsoft, which lets Linux distributions run without managing virtual machines. With Docker Desktop running on WSL 2, users can leverage Linux workspaces and avoid maintaining both Linux and Windows build scripts.

Testing Docker

Open your command line tool and run the below command code:

docker run hello-world

This command will run and download the hello world Docker image as a quick test.

If you frequently work with Docker, configure Docker Desktop to start automatically with Windows. For that, enable this option from the settings menu.

By following these guides, you can successfully install and configure Docker on your Windows machine, allowing you to integrate it into your development workflow seamlessly.

How to configure Docker with the configuration file

The best way to configure the Docker engine on Windows is using a configuration file. That configuration file can be found at the path C:\Program Files\Docker\Docker\resources\daemon.

You can also create this file if it doesn’t already exist with the help of the below syntax.

{
    "allow-nondistributable-artifacts": ["host_port_or_cidr|redacted"],
    "authorization-plugins": [],
    "bridge": "",
    "cluster-advertise": "",
    "cluster-store": "",
    "data-root": "path|redacted",
    "debug": true,
    "default-ulimits": {},
    "disable-legacy-registry": false,
    "dns": ["ip|redacted"],
    "dns-opts": [],
    "dns-search": ["domain|redacted"],
    "exec-opts": [],
    "experimental": true,
    "fixed-cidr": "redacted",
    "graph": "deprecated|redacted",
    "group": "",
    "hosts": ["endpoint|redacted"],
    "insecure-registries": ["host_port_or_cidr|redacted"],
    "labels": [],
    "live-restore": true,
    "log-driver": "",
    "log-level": "",
    "max-concurrent-downloads": 3,
    "max-concurrent-uploads": 5,
    "metrics-addr":"endpoint|redacted",
    "mtu": 0,
    "pidfile": "redacted",
    "raw-logs": false,
    "registry-mirrors": ["registry|redacted"],
    "shutdown-timeout": 15,
    "storage-driver": "",
    "storage-opts": [],
    "swarm-default-advertise-addr": "endpoint|redacted",
    "tlscacert": "redacted",
    "tlscert": "redacted",
    "tlskey": "redacted",
    "tlsverify": true
}

You must add the desired configuration changes to the configuration file. For example, this sample configures the Docker Engine to accept incoming connections on port 2375. All other configuration options will use the same default values.

{

    "hosts": ["tcp://0.0.0.0:2375"]

}Code language: JSON / JSON with Comments (json)

Likewise, the following sample configures the Docker daemon file to keep images and containers in another alternate path. If not specified, the default path is c:\programdata\docker.

{   

    "data-root": "d:\\docker"

}Code language: JSON / JSON with Comments (json)

The following sample configures the Docker daemon only to accept secured connections over port 2375.

{

    "hosts": ["tcp://0.0.0.0:2376", "npipe://"],

    "tlsverify": true,

    "tlscacert": "C:\\ProgramData\\docker\\certs.d\\ca.pem",

    "tlscert": "C:\\ProgramData\\docker\\certs.d\\server-cert.pem",

    "tlskey": "C:\\ProgramData\\docker\\certs.d\\server-key.pem",

}Code language: JSON / JSON with Comments (json)

How to configure Docker in Jenkins

Install Jenkins Docker plugin

Which Docker plugin should you use for Jenkins? You can install the official Docker plugin to integrate Docker functionality into Jenkins. Follow these steps to communicate with Docker hosts seamlessly:

Navigate to Manage Jenkins on the left side of the Jenkins dashboard.

In the Manage Jenkins window, click Plugins.

To install the Docker plugin, go to the Available tab in the plugin manager window. Use the search field to search for Docker. Once you find it, check the box next to the Docker plugin in the search results. Finally, click the Install button to start the plugin installation process.

Once all the necessary components of the plugin are downloaded, select the checkbox at the bottom of the screen to prompt Jenkins to restart.

Jenkins will restart automatically upon completing the installation, ensuring the seamless integration of the Docker plugin into the Jenkins environment.

How to use Docker as a build agent in Jenkins

Configuring a Docker build agent in Jenkins is crucial to optimizing your Continuous Integration (CI) process. Follow the steps below in the Manage Jenkins window of your Jenkins dashboard to set up the Docker build agent:

Navigate to the Manage Jenkins > Clouds option in the System Configuration tab.

Expand the New cloud tab.

Put a name under Cloud name and opt for Docker. Then click the Create button.

Fill in the relevant URI for the Docker host. Ensure to enable the host by selecting the corresponding checkbox.

Check the Expose DOCKER_HOST box to make the Docker host accessible.

Select the Docker Agent templates button to reveal additional configuration options. And then click on Add Docker Template.

Specify a label for identifying the host and activate the agent by selecting the Enabled option. Then, name the Docker template and provide the valid path to the image.

Once you configure the Docker build agent, click Save to save your settings.

Test Docker containers in Jenkins

To verify the successful configuration of the Docker host in Jenkins, conduct a test by creating a Freestyle project. Follow these steps:

Navigate to the Jenkins dashboard and select New Item on the left side.

Provide a name for the job, choose the Freestyle project option, and click OK to create the project.

In the project configuration, choose Restrict where this project can be run and specify the Docker host label in the provided field.

Add a straightforward build step, such as the echo command, to display an active message in the shell. After configuring the job, click Save to finalize the settings.

Jenkins will create the job and display the project window. Initiate the build process by selecting Build Now from the left side menu within the project window.

Once the job is complete, click on the corresponding entry in the Build History section to open the Build window. And ensure that the executed command aligns with your expectations.

Troubleshooting common errors

Invalid agent type error

Causes

This error commonly arises when referencing a Docker container in a Jenkinsfile. Still, the required plugins, i.e., the Jenkins Docker plugin and the Jenkins Docker Pipeline plugin, are missing from the Jenkins environment.

Invalid agent type "docker" specified. Must be one of [any, label, none].Code language: JavaScript (javascript)

Solution

Installing the Jenkins Docker plugin and the Jenkins Docker Pipeline plugin is essential to resolve this error. These plugins enable Jenkins to seamlessly interact with Docker, facilitating the integration of Docker containers within Jenkins pipelines. Once these plugins are in place, the Invalid agent type error should be resolved, allowing for the proper utilization of Docker containers in Jenkins pipelines.

Docker not found in Jenkins pipeline

When I am running Jenkins docker container using this below command:

docker run -p 8080:8080 -p 50000:50000 -v /var/run/docker.sock:/var/run/docker.sock --name jenkins jenkinsCode language: JavaScript (javascript)

I am getting this error message:

[app] Running shell script

+ docker ps

/var/jenkins_home/workspace/app@tmp/durable-e839f204/script.sh: 2: 

/var/jenkins_home/workspace/app@tmp/durable-e839f204/script.sh: docker: not foundCode language: JavaScript (javascript)

Causes

It indicates that the Jenkins agent executing the pipeline does not have Docker installed or accessible. Jenkins pipelines that involve Docker commands require the Docker binary to be available on the Jenkins agent to interact with the Docker daemon. In this scenario, the Docker executable is missing from the Jenkins agent’s environment, resulting in the reported error when attempting to execute Docker commands within the pipeline.

Solution

To resolve this error in a Jenkins pipeline, ensure that the Jenkins agent executing the job has Docker installed and available. In running Jenkins within a Docker container, as mentioned in your reference, it’s necessary to set up the Jenkins Docker container in a way that allows it to communicate with the Docker daemon on the host machine.

One solution is to modify the Docker run command for the Jenkins container to include the Docker executable and socket from the host machine. Update the Docker run command as follows:

docker run -p 8080:8080 -p 50000:50000 -v /var/run/docker.sock:/var/run/docker.sock -v $(which docker):/usr/bin/docker --name jenkins jenkins
Code language: JavaScript (javascript)

Final thoughts on scaling CI/CD with Jenkins and Docker

In conclusion, the integration of Jenkins with Docker presents a powerful synergy that enhances the efficiency and agility of continuous integration and delivery processes. This collaboration streamlines application development, deployment, and version control, offering a robust foundation for modern software practices.

By combining Jenkins’ automation capabilities with Docker’s containerization advantages, teams can achieve greater consistency, scalability, and speed in their workflows. This dynamic duo simplifies the management of dependencies and promotes a more seamless and reliable software delivery pipeline. Embracing Jenkins and Docker together marks a significant stride towards achieving a more resilient and adaptable CI/CD ecosystem in the ever-evolving software development landscape.

Related articles

• How to create Jenkins pipelines
• Declarative vs scripted pipelines in Jenkins
• Jenkins for test automation
• Jenkins-JMeter integration
• Jenkins-Git-SVN integration
• How to integrate Jenkins with Kubernetes

Jenkins with Docker tutorial