Complete Automation of Deploying and Managing Web Application on Kubernetes using Jenkins.

10 min readJul 19, 2020

**Complete Automation of Deploying and Managing Web Application on Kubernetes using Jenkins.**

Agenda:

Perform second task of DevOps on top of Kubernetes where we use Kubernetes resources like Pods, ReplicaSet, Deployment, PVC and Service.(You can go through with my task-2 which I have posted on 7/july/2020)

Create container image that’s has Jenkins installed using Dockerfile.
When we launch this image, it should automatically starts Jenkins service in the container.
Create a job chain of job1, job2, job3 and job4 using build pipeline plugin in Jenkins
Job1 : Pull the Github repo automatically when some developers push repo to Github.
Job2 :

1.Now this job will check the code of developer, accordingly Jenkins will launch the pod for the language in which the code is written by developer.

2. Expose your pod so that testing team could perform the testing on the pod

3. Make the data to remain persistent ( If server collects some data like logs, other user information )

6. Job3 : Deploy developer code in directory /var/www/html of respective language interpreter PODS.

7. Job4 : Test your app if it is working or not. if app is not working , then send email to developer with error messages and redeploy the application after code is being edited by the developer.

Pre-requisites:

Must have Jenkins installed
Must have Docker installed
Must have Minikube installed
Must have python installed (Opt, I use to mail error message to developer).
Must have Kubectl configured

Task Workflow:

Step 1

→ I created one container image for Jenkins using Dockerfile. Here I also install epel-release and ssh-pass package which we gonna need for SSH connection between Jenkins container and Base OS RHEL-8 and also as soon as this container launch it will automatically start Jenkins service in the container.

FROM centos:7RUN yum install wget -y
RUN yum install git -y
RUN yum install sudo -y
RUN yum install net-tools -yRUN wget https://pkg.jenkins.io/redhat-stable/jenkins.repo -O /etc/yum.repos.d/jenkins.repoRUN rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
RUN  yum makecache -y
RUN yum install jenkins -y
RUN yum install java-11-openjdk.x86_64 -y
RUN echo -e "jenkins ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoersRUN yum install /sbin/service -y
CMD sudo service jenkins start -DEFOREGROUND && /bin/bash
EXPOSE 8080#For SSH Connection between Container and Base OS RHEL 8.
RUN yum install epel-release -y
RUN yum install sshpass -y

This is the docker command I use to build the container image.

docker build -t jenkins /root/devt2/jenkinsDoc

This is the docker command to launch the Jenkins Container, I had done the patting (Port forwarding) to have the outside connectivity with this container and also I mount the RHEL-8 directory dev_task3 with container dev_task3 directory. When this container launch on the fly it will create a directory dev_task3 in this Jenkins container.

docker run -it -p 9999:8080 -v /root/dev_task3:/dev_task3 --name jenkinsos jenkins

As we know docker container is running in isolation world, so when we launch a container using this image, container where Jenkins is installed is isolated from base OS RHEL 8. Now if we want to launch an launch application server PODS in kuberentes cluster using Jenkins so for that we have to do a SSH connection between Jenkins container and Base OS RHEL 8. Through SSH Jenkins will establish a connection, go inside the base OS RHEL-8 and launch the application server PODS in kuberentes cluster.

Watch this video to know how to do a SSH connection between docker container and Redhat (RHEL-8):

We have to run this command in Jenkins container to establish a connection between Jenkins container and base OS RHEL-8. First command will generate a private key and public key of container which is full encrypted. Second command will establish a connection between Jenkins container and RHEL-8. Using ssh root@192.168.99.103 we can go inside RHEL-8 and launch Pods or run any command in RHEL-8.

ssh-keygen -t rsa -b 4096
sshpass -p "prison@123" ssh-copy-id -o StrictHostKeyChecking=no root@192.168.99.103

→ In second dockerfile I configure httpd server and php interpreter. We will use this dockerfile to create a image and run the developer code in it on httpd server.

FROM centos:latest
RUN yum install httpd -y
RUN yum install php -y
CMD /usr/sbin/httpd -DFOREGROUND
EXPOSE 80
COPY index.html /var/www/html

Step2

Job1:

When developer push the code this Job will pull the GitHub repository automatically through poll scm. This repository is store in Jenkins container and also in base OS RHEL-8 because we had done the patting between Jenkins container and base OS RHEL-8.

Job1 Output:

In GitHub repository we have 4-files name index.html, index.php, Mail.py, php_Mail.py.

If web application server is not running properly because of some error in code, we will use this Mail.py file to send an alert message and log report to developer.

Index.php and index.html is code which we gonna deploy on application server pods. It is very simple code.

Step 3

Job2:

Now this job will check the code of developer, accordingly Jenkins will launch the pod for the language in which the code is written by developer. For launching the PODS I have used Manifest file, one for html language and second for php kanguage.

In Manifest I have used kubernetes resources like Pods, ReplicaSet, Deployment, PVC and Service.

Pods: Pod is nothing but one box where container’s reside (such as Docker containers).

Replica Set:- Replication Set have 3 duty i.e.

1) To monitor Pods, launch or terminate the Pod according to the requirements (Scale out and Scale in)

2) To create replica of Pod.

3) To set Desire for Pod. Replica set useSet/Rule based Selector/filter. Replica Set don’t use Podname for monitoring because it change every time when it get terminated/fail by any reason. So RS use labels/Tag for monitoring the pods.

Deployment : Deployment use rolling strategy for website to get zero downtime for clients. If developer had done some changes in code of website then he upload new version code on SCM (source code management) tool GitHub. From GitHub integration tool like Jenkins/Openshift we can use to push that code in registry. Now kubernetes download the image from registry and ask the docker to launch the container in pod. Now deployment come in picture it automatically behind the scene terminate the container which is not in use by any client and launch the container with new image version. If client is using older container then deployment will wait till all client goes then behind the scene it terminate this older container and launch the new one. Deployment behind the scene using rolling strategy for downtime and also it use replica set for managing the pods.

PVC (Persistence Volume Claim) :

Kubernetes have very beautiful resource which can store the data permanently. Kubernetes use PVC (persistence volume claim) for storing data permanently, where In docker,podman, cri-o by default storage is empheral in nature (means as soon as container fail all data you lose) . For using pvc we have to mount the pvc with pod. Now if any container/pod fails when they launch new container, they can retrieve all the older data from pvc folder. In this task I used PVC to persistant log file of web application and /var/www/html directory where web application code stored.

Service : In kubernetes there are 3 types of service i.e.

Cluster_Ip : To have local connectivity between pods we use Cluster_Ip
Nodeport : To have outside connectivity with client we use Nodeport.
LoadBalancer : To have outside connectivity with client we use Nodeport.

Created Manifest file for PHP web Application server Deployment.

→ Pvc:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: php-pvc
  labels:
    env: php-testing
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: php-log-pvc
  labels:
    env: php-testing
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

→ Service, Pods, Deployemet, Replica Set :

apiVersion: v1
kind: Service
metadata:
  name: php-svc
  labels:
    name: php-svc
spec:
  selector:
    name: php-pod
  ports:
  - port: 80
    targetPort: 80
    nodePort: 30001
  type: NodePort---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: php-app
  labels:
    name: php-app
    env: php-testing
spec:
  replicas: 1
  selector:
    matchExpressions:
    - { key: name, operator: In , values: [ php-pod ] }
  template:
    metadata:
      name: php-pod
      labels:
        name: php-pod
    spec:
      containers:
      - name: php-container
        image: vimal13/apache-webserver-php
        volumeMounts:
        - name: phplogs
          mountPath: /var/log/httpd/
        - name: phpstorage
          mountPath: /var/www/php/
      volumes:
      - name: phplogs
        persistentVolumeClaim:
          claimName: php-log-pvc
      - name: phpstorage
        persistentVolumeClaim:
          claimName: php-pvc

Created Manifest file for HTML web Application server Deployment.

→ PVC:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: html-pvc
  labels:
    env: html-testing
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: html-log-pvc
  labels:
    env: html-testing
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

→ Service, Pods, Deployemet, Replica Set :

apiVersion: v1
kind: Service
metadata:
  name: html-svc
  labels:
    name: html-svc
spec:
  selector:
    name: html-pod
  ports:
  - port: 80
    targetPort: 80
    nodePort: 30000
  type: NodePort---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: html-app
  labels:
    name: html-app
    env: html-testing
spec:
  replicas: 1
  selector:
    matchExpressions:
    - { key: name, operator: In , values: [ html-pod ] }
  template:
    metadata:
      name: html-pod
      labels:
        name: html-pod
    spec:
      containers:
      - name: html-container
        image: vimal13/apache-webserver-php
        volumeMounts:
        - name: htmllogs
          mountPath: /var/log/httpd/
        - name: htmlstorage
          mountPath: /var/www/html/
      volumes:
      - name: htmllogs
        persistentVolumeClaim:
          claimName: html-log-pvc
      - name: htmlstorage
        persistentVolumeClaim:
          claimName: html-pvc

Job2 Output:

Step 4

Job3:

Developer is code is store in jenkins container /dev_task3. This directory is being already mounted with Redhat Base OS. So now I will use kubectl cp command to copy developer code in directory /var/www/html of respective language interpreter PODS. As given below for html and php language.

#HTML:
html=$(sudo ssh root@192.168.99.103 kubectl get pod -o jsonpath="{.items[0].metadata.name}")sudo ssh root@192.168.99.103 sleep 5sudo ssh root@192.168.99.103 kubectl cp /root/dev_task3/*.html $html:/var/www/html/#PHP:
php=$(sudo ssh root@192.168.99.103 kubectl get pod -o jsonpath=”{.items[1].metadata.name}”)sudo ssh root@192.168.99.103 sleep 5sudo ssh root@192.168.99.103 kubectl cp /root/dev_task3/*.php $php:/var/www/html/

Job3 Output:

Step 5

Job4:

This job will test/check your app if it is working or not. if app is not working , then it will send email to developer with error messages and redeploy the application after code is being edited by the developer.

Monitoring of pods are already taken care by kubernetes and also we are using poll scm in job1 so as soon as edited code push on GitHub, Jenkins will automatically pull the code and run all the job in series. All 4 job has chaining with other jobs.

Here I intentionally push error php code. Let see what happen.

This is the job4 shell code where it check server is running properly or not, if not then it will send mail to developer. For mail I created 2 python program file 1 for php code and other one for html. You can also use jenkins mail plugins.

Python program file for I used for application which is build on php:

import smtplib
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
from email.mime.base import MIMEBase
from email import encoders
import os.pathemail = 'sender mail id'
password = 'password'
send_to_email = 'receiver mail id'
subject = '!!! Testing Server Report !!!'
message = "Web App Server-2 is not running, debug the code and please find the log file in attachment."
file_location = "/root/dev_task3/php_log/log.txt"msg = MIMEMultipart()
msg['From'] = email
msg['To'] = send_to_email
msg['Subject'] = subjectmsg.attach(MIMEText(message, 'plain'))# Setup the attachment
filename = os.path.basename(file_location)
attachment = open(file_location, "rb")
part = MIMEBase('application', 'octet-stream')
part.set_payload(attachment.read())
encoders.encode_base64(part)
part.add_header('Content-Disposition', "attachment; filename= %s" % filename)# Attach the attachment to the MIMEMultipart object
msg.attach(part)server = smtplib.SMTP('smtp.gmail.com', 587)
server.starttls()
server.login(email, password)
text = msg.as_string()
server.sendmail(email, send_to_email, text)
server.quit()