DevSecLab Series: Pipeline Time! (PT.1)
Unlike the other post that I have made, this blog post will cover a variety of things related to the construction of the Jenkins pipeline. I will give an overview of Jenkins, explain terms and basic pipeline concepts, and construct our own Jenkins pipeline from scratch! Let's get started.
Jenkins Overview
Jenkins is an open-source automation server that automates the repetitive technical tasks involved in the continuous integration and delivery of software. This application is written in Java and contains plugins that are built to enable the ease of continuous deployment and integration purposes. Jenkins allows developers to do the following:
Build, test, and deploy code continuously
Continuously deliver software by integrating with testing/deployment frameworks
In comparison to other CI/CD tools, Jenkins has several advantages that make it an excellent tool to use for enterprise and home-lab environments:
It is portable and can easily be installed across multiple operating systems
It is widely supported by a vast number of Fortune 500 companies and developers
Over 1000+ plugins to enhance pipeline development and automation.
So, are you convinced that Jenkins is actually awesome?
Alright! Just in case you need a little more persuasion, here is a video that I would recommend: https://www.youtube.com/watch?v=p7-U1_E_j3w
These guys will go into much more detail than I have about Jenkins, and they'll also include a little demonstration in the end.
Jenkins Pipeline Concepts
In this section, we will cover some of the pipeline concepts for constructing your pipeline.
Pipeline
A user-defined block that'll contain all the processes such as build, test, and deploy either in a Jenkinsfile, or text file with pipeline code, or in the pipeline script tab on the UI.
Node
A machine that will execute an entire workflow. Necessary for the pipeline to function.
Agent
An agent is a directive that can run multiple builds with only one instance of Jenkins. Label parameters include:
Any: Runs the pipeline/stage on any available agent.
None: Applied at the root of the pipeline and indicates that there isn't a global agent for the pipeline. Have to specify an agent for each stage.
Stages
This block contains all the word that needs to be carried out. The work is specified in the form of stages.
Steps
A machine that will execute an entire workflow. Necessary for the pipeline to function.
Pipeline Syntax Overview
One thing to note is that there are two types of Groovy-DSL pipeline syntax that Jenkins supports. These two types are Declarative and Scripting pipeline syntax.
Declarative pipeline syntax supports the pipeline as code concept and is written into a Jenkinsfile. The Jenkinsfile is a text file that stores the entire workflow that you've constructed as code that can be checked into an SCM system such as GitHub or GitLab. Within this Jenkinsfile, a sample declarative pipeline contains the pipeline block, which defines all the work that is done throughout your pipeline:
Jenkinsfile (Declarative Syntax)
pipeline {
stage('Checkout') {
// more work to be done
}
stage('Build') {
// more work to be done
}
stage('Deploy') {
// more work to be done
}
}This gives developers the flexibility needed to be able to add or remove functionality from the pipeline, push it to the SCM repository, and integrate that with Jenkins to build upon commit.
Scripted pipeline syntax, on the other hand, provides more flexibility for the users when developing a pipeline. Unlike the declarative pipeline syntax, the scripted pipeline syntax allows developers to add exception handling through the use of try/catch/finally blocks in your code. I'll show you exactly what this is in the next section when we construct a pipeline from scratch!!
Building Jenkins Pipeline
Alright, so we've covered some of the pipeline basics. Now, we'll get into the fun. The first thing that you'll want to do is to log in to your Jenkins server. Once you've done that, on the right side, click the option "Create a new item".
Inside of that, you'll want to entire a pipeline name and specify the project type. The project type should be "Pipeline", which is highlighted in the image above. Afterward, press OK.
It should redirect you to the project dashboard, which is shown in the image above. This dashboard will contain a visual of what's happening when you build your pipeline. As you start to customize your pipeline build by adding in additional plugins, this dashboard will change.
For now, you'll want to hit the Configure button on the left-hand side of your screen and scroll all the way to the bottom. You should see the Pipeline window there:
This is where we'll store our pipeline code. For the pipeline code that will follow this paragraph, I will explain each and every stage and process in great detail before we execute the build. Copy and paste this block of code into the script text-box:
pipeline {
// Declaring any agent that we have
agent any
stages {
stage('Checkout'){
steps{
script{
try{
// Get some code from a GitHub repository: sample repository
git 'https://github.com/jglick/simple-maven-project-with-tests.git'
} catch (Exception e){
echo 'Exception has occured... please help:' + e.toString()
sh 'DO SOMETHING!!!!!'
}
}
}
}
stage('Build') {
steps {
script{
try{
// Build the project
sh "mvn clean install -DskipTests"
} catch (Exception e){
echo 'Exception has occured... please help:' + e.toString()
sh 'DO SOMETHING!!!!!'
}
}
}
}
stage('Test'){
steps{
script{
try{
// Run the test cases for this project
sh 'mvn -Dmaven.test.failure.ignore=true test'
} catch (Exception e){
echo 'Exception has occured... please help:' + e.toString()
sh 'DO SOMETHING!!!!!'
}
}
}
post {
// If Maven was able to run the tests, even if some of the test
// failed, record the test results and archive the jar file.
success {
junit '**/target/surefire-reports/TEST-*.xml'
archiveArtifacts 'target/*.jar'
}
}
}
}
}Pipeline Breakdown
Before you save your changes, let me explain what this pipeline is going to do stage by stage.
So we've declared that this is a pipeline and that we will use any agent that is available at the moment we execute this pipeline. By default, we'll end up using the master because we have no other agents or nodes configured for this machine.
In the "Checkout" stage, we will simply grab a sample Java project from Github that will do the work for us. This project will have some test cases that we can build as well.
In the "Build" stage, we are going to use Maven, which we have installed on our master build server already, to build the Java project.
In the "Test" stage, we are going to use Maven to execute the test cases that are written for this project. Appended to the end of this stage is a post-stage, which will show the test case results on the dashboard and archive the jars that build as a result of the code compilation.
Also, take note that each of these stages is surrounded by a try/catch block. If the commands that are inside of the try block somehow throws an error, that error will be caught and displayed for the user to troubleshoot and debug in the console logs.
I've added some comments in the code as well to explain what each command will do, so please feel free to read those as well.
Building the Pipeline
So now that you've copied and pasted the code into the text-block in the configure panel, please save and exit the pipeline.
You'll be redirected to the project dashboard. On the left-hand side of the screen, click the "Build Now" button to execute the pipeline build.
Once you've executed the pipeline build, you'll start to see the Stage View being populated with information for each stage of your pipeline. Whether it succeeds or fails, you will see the stages being executed. If you want to see more about what happens in real time, click on the build number. On the left hand side, you'll see "Console Output".
That is all for this post. If you have any issues or questions, please feel free to email me or post a message in the chat. I will do my best to answer any questions you have.
Have fun!!
Cheers!