Date Published: April 7, 2019

    Automated CI-CD with Docker and Jenkins

    This repository is a tutorial it tries to exemplify how to automatically manage the process of building, testing the highest coverage and deployment phases.

    GITHUB URL- Click on GitHub URL to Download the Java Project.

    https://github.com/AVM-Consulting/CI-CD-with-Dockers-and-Jenkins

    Our goal is to ensure our pipeline works well after each code being pushed. The processes we want to auto-manage:

    Just One commit and the application is deployed in Docker Container

    1. Code checkout
    2. Run tests
    3. Compile the code
    4. Run Sonarqube analysis on the code
    5. Create Docker image
    6. Push the image to Docker Hub
    7. Pull and run the image

    First step, running up the services

    Since one of the goals is to obtain the Sonarqube report of our project, we should be able to access sonarqube from the jenkins service. Docker compose is a best choice to run services working together. We configure our application services in a yaml file as below.

    docker-compose.yml

    services:
      sonarqube:
        build:
          context: sonarqube/
        ports:
          - 9000:9000
          - 9092:9092
        container_name: sonarqube
      jenkins:
        build:
          context: jenkins/
        privileged: true
        user: root
        ports:
          - 8080:8080
          - 50000:50000
        container_name: jenkins
        volumes:
          - /tmp/jenkins:/var/jenkins_home #Remember that, the tmp directory is designed to be wiped on system reboot.
          - /var/run/docker.sock:/var/run/docker.sock
        depends_on:
          - sonarqube
    

    Paths of docker files of the containers are specified at context attribute in the docker-compose.yml file. Content of these files as follows.

    sonarqube/Dockerfile
      
          FROM sonarqube:6.7-alpine
     
    jenkins/Dockerfile
      
         FROM jenkins:2.60.3
    

    If we run the following command in the same directory as the docker-compose.yml file,The Sonarqube and Jenkins containers will up and run.

    docker-compose -f docker-compose.yml up --build
      
    docker ps 
    
    CONTAINER ID        IMAGE                COMMAND                  CREATED              STATUS              PORTS                                              NAMES
    
    06bf0758596ft55        pipeline_jenkins     "/bin/tini -- /usr..."   About a minute ago   Up About a minute   0.0.0.0:8080->8080/tcp, 0.0.0.0:50000->50000/tcp   jenkins
    
    a6dh78h9fhfjk88        pipeline_sonarqube   "./bin/run.sh"           About a minute ago   Up About a minute   0.0.0.0:9000->9000/tcp, 0.0.0.0:9092->9092/tcp     sonarqube
    

    GitHub configuration

    We’ll define a service on Github to call the Jenkins Github webhook because we want to trigger the pipeline.Go to Settings – Integrations & services.

    The Jenkins Github plugin should be shown on the list of available services as below.

    After this, we should add a new service by typing the URL of the dockerized Jenkins container along with the /github-webhook/ path.

    The next step is that create an SSH key for a Jenkins user and define it as Deploy keys on our GitHub repository.

    If everything goes well, the following connection request should return with a success.

    PTY allocation request failed on channel 0
    Hi <your github username>/<repository name>! You've successfully authenticated, but GitHub does not provide shell access.
    Connection to github.com closed.
    

    Jenkins configuration

    We have configured Jenkins in the docker compose file to run on port 8080 therefore if we visit http://localhost:8080 we will be greeted with a screen like this.

    We need the admin password to proceed to installation. It’s stored in the /var/jenkins_home/secrets/initialAdminPassword directory and also It’s written as output on the console when Jenkins starts.

    jenkins      | *************************************************************
    
    jenkins      |
    
    jenkins      | Jenkins initial setup is required. An admin user has been created and a password generated.
    
    jenkins      | Please use the following password to proceed to installation:
    
    jenkins      |
    
    jenkins      | 45638c79cecd4f43962da2933980197e
    
    jenkins      |
    
    jenkins      | This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
    
    jenkins      |
    
    jenkins      | ************************************************************
    

    To access the password from the container.

    docker exec -it jenkins sh

    $cat /var/jenkins_home/secrets/initialAdminPassword

    After entering the password, we will download recommended plugins and define an admin user.

    After clicking Save and Finish and Start using Jenkins buttons, we should be seeing the Jenkins homepage. One of the seven goals listed above is that we must have the ability to build an image in the Jenkins being dockerized. Take a look at the volume definitions of the Jenkins service in the compose file.

    -/var/run/docker.sock:/var/run/docker.sock

    The purpose is to communicate between the Docker Daemon and the Docker Client(we will install it on Jenkins) over the socket. Like the docker client, we also need Maven to compile the application. For the installation of these tools, we need to perform the Maven and Docker Client configurations under:-

    Manage Jenkins – Global Tool Configuration Menu.

    We added the Maven and Docker installers and have checked the Install automatically checkbox.

    Now we will perform checkout codebaseand pushing an image to Docker Hub,we need to define the Docker Hub Credentials.Keep in mind that if we are using a private repo,we must define Github credentials.These definitions are performed under:-

    Jenkins Home Page – Credentials – Global credentials – Add Credentials menu.

    Go to Jenkins – Credentials – System – mark env variables.

    Create a repository in docker hub so the built image is pushed to docker registry with its credentials (need to be filled in jenkinsFile)

    We use the value we entered in the ID field to Docker Login in the script file. Now, we define pipeline under:-

    Jenkins home page – New item menu.

    In this step, we select GitHub hook trigger for GITScm pooling options for automatic run of the pipeline by Github hook call.

    Also in the Pipeline section, we select the Pipeline script from SCM as Definition, define the GitHub repository and the branch name, and specify the script location.

    After that, when a push is done to the remote repository or when you manually trigger the pipeline by Build Now option, the steps described in Jenkins file will be executed.

    After that, when a Deployment is done to the remote repository and automatic trigger to pipeline by Build Now You can see the Workflow step by step .Go to Docker Login check the Images and then check the container in which the application is running.

    After Deployment is done and Container is running , You can check it by URL and Port number on which the service is running.

    http://ec2-50-16-10-93.compute-1.amazonaws.com:8090/

    Review important points of the Jenkins file

    stage('Initialize'){
    
        def dockerHome = tool 'myDocker'
    
        def mavenHome  = tool 'myMaven'
    
        env.PATH = "${dockerHome}/bin:${mavenHome}/bin:${env.PATH}"
    
    }
    

    The Maven and Docker client tools we have defined in Jenkins under Global Tool Configuration menu are added to the PATH environment variable for using these tools with sh command.

    stage('Push to Docker Registry'){
    
        withCredentials([usernamePassword(credentialsId: 'dockerHubAccount', usernameVariable: 'USERNAME', passwordVariable: 'PASSWORD')]) {
    
            pushToImage(CONTAINER_NAME, CONTAINER_TAG, USERNAME, PASSWORD)
    
        }
    
    }
    

    withCredentials provided by Jenkins Credentials Binding Plugin and bind credentials to variables. We passed dockerHubAccount value with credentialsId parameter. Remember that, dockerHubAccount value is Docker Hub credentials ID we have defined it under:-

    Jenkins Home Page – Credentials – Global credentials – Add Credentials menu.

    In this way, we access to the username and password information of the account for login.

    Sonarqube configuration

    For Sonarqube we have made the following definitions in the pom.xml file of the project.

    ...
    <dependencies>
    ...
        <dependency>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>sonar-maven-plugin</artifactId>
            <version>2.7.1</version>
            <type>maven-plugin</type>
        </dependency>
    ...
    </dependencies>
    

    In the docker compose file, we gave the name of the Sonarqube service which is Sonarqube, this is why in the pom.xml file, the sonar URL was defined as http://sonarqube:9000.

    Things To Remember:

    1 Do not forget to add environment variables in jenkins WRT to Jenkinsfile

    2 Docker Hub Repository should be created and also the repository should be marked in the Variable in JenkinsFile

    JenkinsFile Configuration

    def CONTAINER_NAME="jenkins-pipeline"
    
    def CONTAINER_TAG="latest"
    
    def DOCKER_HUB_USER="XYZ"
    
    def HTTP_PORT="8090"
    

    def pushToImage(containerName, tag, dockerUser, dockerPassword){

     sh "docker login -u $dockerUser -p $dockerPassword"
    
     sh "docker tag $containerName:$tag $dockerUser/$containerName:$tag"
    
     sh "docker push $dockerUser/$containerName:$tag"
    
     echo "Image push complete"
    
    }
    

    def runApp(containerName, tag, dockerHubUser, httpPort){

     sh "docker pull $dockerHubUser/$containerName"
    
     sh "docker run -d - rm -p $httpPort:$httpPort - name   $containerName $dockerHubUser/$containerName:$tag"
    
     echo "Application started on port: ${httpPort} (http)"
    
    }
    

    By the way, check out our best AWS deal: https://www.avmconsulting.net/well-architected-review

    Comments

    1. college :

      I’m extremely imρrеssed along with your wгitіng skills as neatly as with the strᥙcture on your blog.
      Is this a paid suЬject mаtter or did you customize it yourself?
      Anyway stay up the nice qᥙaⅼity writing, it’s rare to look a nice weblog lіke this one theѕe days..

    2. corporate :

      obviouѕly ⅼike your web site but you have tߋ take
      a look at the sреlling on ѕeveral of your ⲣosts. Several of them are rifе with spelling problems and I to find it very troublesome to inform the truth however I’ll definitely come again again.

    Write a Reply or Comment

    Your email address will not be published. Required fields are marked *