DevSecOps with GitHub Action and SaaS Tools
CICD - DevSecOps using GitHub Action
DevSecOps automatically bakes in security at every phase of the software development lifecycle, enabling development of secure software at the speed of Agile and DevOps. It integrates application and infrastructure security seamlessly into Agile and DevOps processes and tools. It addresses security issues as they emerge, when they’re easier, faster, and less expensive to fix and also makes application and infrastructure security a shared responsibility of development, security, and IT operations teams.
Implementing “shift left” to a DevOps team guarantees application security at the earliest stages in the development lifecycle, as part of DevSecOps (collaboration between development, security, and operations). To shift left means to move a process to the left on the traditional linear depiction of the software development lifecycle (SDLC). There are two common subjects of shift left initiatives in DevOps: security and testing.
This is a sample project to demonstrate the E2E Github Action release workflow with all Security Controls gates integrated with different Cloud SaaS CI/CD tools offering.
Tools used are listed below
- Requirements & Planning
- Language used
- SCM
- Build
- Continuous Integration
- Code Quality
- SAST (Static Analysis Security Testing)
- SCA (Software Composition Analysis)
- DAST (Dynamic Application Security Testing)
- OpenAPI Spec Validator
- Infra as Code Scan
- Secret Scan
- Maven Repository
- Helm Repository
- Container Scan - Docker Image
- Container Repository
- Deploying to Azure - AKS
- Deploying to GCP - GKE
- Provisioning(IaC) AWS - EC2
- Unit Testing
- Functional Testing
- Test Management
- Performance Testing
- Create Release Note & ag
CI/CD E2E Release Workflow
This Project implements the below CI/CD E2E release workflow integrated with CI, CD, Security Scans, IaC and Test automation using Cloud SaaS tool sets.
Sample release workflow implementation using Github Action is present here
Sample release workflow execution is present here
Jira and Github Integration - (Planning,Requirement & Test Execution)
Jira and Github Integration will provide better traceability between Planning, Requirement, Code, Build and Test Execution. Linking your GitHub to Jira gives team the ability to see their branches, commit messages, build, test cases & results, pull requests and release information in the context of Jira tickets. More information can find here.
When a developer makes a commit, they should add a Jira issue key to the commit message as shown below:
git commit -m "JUDE-42 add a README file to the project."
git push origin <branchname>
Then GitHub Commit information would appear in Jira as shown below.
Once GitHub Action build is completed, build information get reflected in Jira as shown below.
Using Cucumber & Gherkin, we can add Test Plan & Test Cases to a specific story in Jira.
Feature: Test Health Check Rest API
Scenario: Health Check Api call is successful
Given user wants to call health check api
When user call health check api with the url "http://35.194.14.180/api/v1"
Then return result
Test results will be uploaded back to Jira to provide Requirement Traceability Matrix
Integrating Cloud SaaS DevSecOps tools using GitHub Action
Integration & Implementation of E2E CI/CD release workflow using Github Action, this has been achieved using different Cloud SaaS tools listed below.
1) Maven - Build and Unit Test
Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project’s build, reporting and documentation from a central piece of information.
Code is build using Maven and unit test cases are executed using JUnit The test coverage result, which is aggregated by Jacoco would be uploaded to Github Action as artifacts. Please check pom.xml as well.
test:
name: Build and Unit Test
runs-on: ubuntu-latest
steps:
- name: Check out the code
uses: actions/checkout@v1
with:
fetch-depth: 0
- name: Set up JDK
uses: actions/setup-java@v1
with:
java-version: 1.8
- name: Cache Maven packages
uses: actions/cache@v1
with:
path: ~/.m2
key: $-m2-$
restore-keys: $-m2
- name: Build
run: mvn -B clean package -DskipTests
- name: Run UnitTest and Verify
run: mvn -B verify -DexcludedGroups="Smoke | Staging | BrowserStack | LamdaTest"
- name: Generate JaCoCo Badge
id: jacoco
uses: cicirello/jacoco-badge-generator@v2
- name: Log code coverage percentage
run: |
echo "coverage = $"
echo "branch coverage = $"
- name: Upload code coverage report
uses: actions/upload-artifact@v2
with:
name: jacoco-report
path: target/site/jacoco/
- name: Adding Junit Report
uses: ashley-taylor/junit-report-annotations-action@master
if: always()
with:
access-token: $
- name: Publish Unit Test Results
uses: EnricoMi/publish-unit-test-result-action/composite@v1
with:
files: target/surefire-reports/*.xml
In pom.xml, we need to add the jacoco plugin.
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.7</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>generate-code-coverage-report</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
Sample test result:-
2) SonarQube Cloud - Code Quality
SonarQube is an open-source platform developed by SonarSource for continuous inspection of code quality to perform automatic reviews with static analysis of code to detect bugs, code smells, and security vulnerabilities on 20+ programming languages.
Inspect the code using SonarQube and enable the Quality Gate Check in CI/CD workflow. Results are uploaded to SonarQube Cloud SaaS offering. GitHub integration with SonarQube can find here.
sonar:
name: Inspect - Using Sonar
runs-on: ubuntu-latest
needs: [test]
steps:
- name: Check out the code
uses: actions/checkout@v1
with:
fetch-depth: 0
- name: Set up JDK 11
uses: actions/setup-java@v1
with:
java-version: 11
- name: Cache SonarCloud packages
uses: actions/cache@v1
with:
path: ~/.sonar/cache
key: $-sonar
restore-keys: $-sonar
- name: Cache Maven packages
uses: actions/cache@v1
with:
path: ~/.m2
key: $-m2-$
restore-keys: $-m2
- name: Build and analyze
env:
GITHUB_TOKEN: $
SONAR_TOKEN: $
run: mvn -B verify -DexcludedGroups="Smoke | Staging | LamdaTest | BrowserStack" org.sonarsource.scanner.maven:sonar-maven-plugin:sonar -Dsonar.issuesReport.html.enable=true -Dsonar.projectKey=judebantony_cicd-github-action-example
- name: SonarQube Quality Gate check
uses: sonarsource/sonarqube-quality-gate-action@master
env:
GITHUB_TOKEN: $
SONAR_TOKEN: $
timeout-minutes: 5
with:
scanMetadataReportFile: target/sonar/report-task.txt
In pom.xml, we need to add the below configuration.
<sonar.organization>judebantony</sonar.organization>
<sonar.projectKey>cicd-github-action-example</sonar.projectKey>
<sonar.host.url>https://sonarcloud.io</sonar.host.url>
SonarQube dashbord:-
Quality Gate Check:-
3) Codecov - Code Coverage
Codecov improve code quality and workflow with highly integrated tools to group, merge, archive, and compare coverage reports.
Uploaded the code coverage result to Codecov Cloud SaaS offering. GitHub integration with Codecov can find here.
codecov:
name: Inspect - Using Codecov
runs-on: ubuntu-latest
needs: [test]
steps:
- name: Check out the code
uses: actions/checkout@v1
with:
fetch-depth: 0
- name: Set up JDK 8
uses: actions/setup-java@v1
with:
java-version: 1.8
- name: Cache Maven packages
uses: actions/cache@v1
with:
path: ~/.m2
key: $-m2-$
restore-keys: $-m2
- name: Build and analyze
env:
GITHUB_TOKEN: $
SONAR_TOKEN: $
run: mvn -B verify -DexcludedGroups="Smoke | Staging | LamdaTest | BrowserStack" cobertura:cobertura
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2
with:
token: $
flags: unittests
name: codecov-umbrella
fail_ci_if_error: true
verbose: true
In pom.xml, we need to add the below plugin.
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cobertura-maven-plugin</artifactId>
<version>2.7</version>
<configuration>
<formats>
<format>html</format>
<format>xml</format>
</formats>
<check />
</configuration>
</plugin>
CodeCov dashboard:-
4) CodeQL - SAST
CodeQL is the code analysis engine developed by GitHub to automate security checks. You can analyze your code using CodeQL and display the results as code scanning alerts.
Use the native Github Action CodeQL for SAST scan and upload the result to GitHub Advanced Security tab. Configuring CodeQL code scanning can find here.
codeqlScan:
name: SAST Scan using CodeQL
runs-on: ubuntu-latest
needs: [sonar, codecov]
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: [ 'java' ]
steps:
- name: Checkout repository
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
with:
languages: $
- name: Autobuild
uses: github/codeql-action/autobuild@v1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
GitHub Advanced Security Tab:-
5) AppScan CodeSweep - SAST
HCL AppScan CodeSweep is a free to use security tool, designed for beginners and professionals alike, who need a quick, simple, and platform friendly program.
Use the AppScan CodeSweep for SAST scan and upload the result to GitHub Advanced Security tab using sairf OWASP format.
appScan:
name: SAST Scan using AppScan CodeSweep
runs-on: ubuntu-latest
needs: [sonar, codecov]
steps:
- name: Checkout repository
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Run AppScan CodeSweep
uses: HCL-TECH-SOFTWARE/appscan-codesweep-action@v1
with:
asoc_key: $
asoc_secret: $
env:
GITHUB_TOKEN: $
HCL AppScan CodeSweep dashboard:-
6) Codacy - SAST
Codacy automatically analyzes your source code and identifies issues as you go, helping you develop software more efficiently with fewer issues down the line. Through static code review analysis, Codacy notifies you of security issues, code coverage, code duplication, and code complexity in every commit and pull request.
Use the Codacy for SAST scan and upload the result to GitHub Advanced Security tab using sairf OWASP format.
codacyScan:
name: SAST Scan using Codacy
runs-on: ubuntu-latest
needs: [sonar, codecov]
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Run Codacy Analysis CLI
uses: codacy/codacy-analysis-cli-action@master
with:
output: codacy.sarif
format: sarif
gh-code-scanning-compat: true
max-allowed-issues: 2147483647
- name: Upload SARIF results file
uses: github/codeql-action/upload-sarif@main
with:
sarif_file: codacy.sarif
Codacy dashboard.
7) Snyk - SCA
Snyk (pronounced sneak) is a developer security platform for securing code, dependencies, containers, and infrastructure as code.
Use the Snyk for SCA scan and upload the result to GitHub Advanced Security tab using sairf OWASP format. GitHub integration with Snyk can find here.
snykScan:
name: SCA Scan using Snyk
runs-on: ubuntu-latest
needs: [codeqlScan, codacyScan, appScan]
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up Maven
run: mvn -N io.takari:maven:wrapper -Dmaven=3.8.2
- name: Run Snyk to check for vulnerabilities
continue-on-error: true
uses: snyk/actions/maven-3-jdk-11@master
env:
SNYK_TOKEN: $
with:
args: --sarif-file-output=snyk.sarif
- name: Upload result to GitHub Code Scanning
uses: github/codeql-action/upload-sarif@v1
with:
sarif_file: snyk.sarif
Snyk dashboard:-
8) Dependabot - SCA
Use the github native Dependabot for SCA scan and upload the result to GitHub Advanced Security tab. Enabling the Dependabot in GitHub can find here.
version: 2
updates:
- package-ecosystem: "maven"
directory: "/"
schedule:
interval: "daily"
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "weekly"
Dependabot dashboard:-
9) BlackDuck CoPilot - SCA
Black Duck is a complete open source management solution, which fully discovers all open source in your code. It can map components to known vulnerabilities, and identify license and component quality risks.
Use the Black Duck for SCA scan and upload the result to GitHub Advanced Security tab using sairf OWASP format.
blackduck:
name: SCA Scan using BlackDuck
runs-on: ubuntu-latest
needs: [codeqlScan, codacyScan, appScan]
steps:
- name: Check out the code
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up JDK
uses: actions/setup-java@v1
with:
java-version: 1.8
- name: Set up Maven
run: mvn -N io.takari:maven:wrapper -Dmaven=3.8.2
- name: Build with Maven
run: |
mvn install -DskipTests=true -Dmaven.javadoc.skip=true -B -V
mvn test -B
- name: Upload to CoPilot
run: bash <(curl -s https://copilot.blackducksoftware.com/ci/githubactions/scripts/upload)
BlackDuck CoPilot dashboard-
10) FOSSA - SCA
FOSSA accelerate open source adoption with scalable, end-to-end management for third-party code, license compliance and vulnerabilities.
Use the FOSSA for SCA scan and upload the result to GitHub Advanced Security tab using sairf OWASP format. GitHub integration with FOSSA can find here.
fossaScan:
name: SCA Scan using FOSSA
runs-on: ubuntu-latest
needs: [codeqlScan, codacyScan, appScan]
steps:
- name: Check out the code
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Run FOSSA Scan
uses: fossas/fossa-action@v1
with:
api-key: $
FOSSA dashboard:-
11) ShiftLeft - SCA & SAST
ShiftLeft provides an early, accurate picture of risk so devs can quickly secure code and stay focused on the transformation.
Use the ShiftLeft for SCA & SAST scan and upload the result to GitHub Advanced Security tab using sairf OWASP format. GitHub integration with ShiftLeft can find here.
shitLeftScan:
name: SAST and SCA Scan using ShiftLeft
runs-on: ubuntu-latest
needs: [codeqlScan, codacyScan, appScan, rest-api-static-security-testing]
steps:
- uses: actions/checkout@v2
- name: Perform Scan
uses: ShiftLeftSecurity/scan-action@39af9e54bc599c8077e710291d790175c9231f64
env:
WORKSPACE: ""
GITHUB_TOKEN: $
SCAN_AUTO_BUILD: true
with:
output: reports
- name: Upload report
uses: github/codeql-action/upload-sarif@v1
with:
sarif_file: reports
ShiftLeft dashboard:-
12) Snyk - Infrastructure as Code Scan
Snyk (pronounced sneak) is a developer security platform for securing code, dependencies, containers, and infrastructure as code.
Use Snyk to secure the Infrastructure as Code Files. GitHub integration with Snyk can find here.
snykIaSScan:
name: SAST - Scan IaC Configs using Snyk
runs-on: ubuntu-latest
needs: [dependabot, snykScan, blackduck, fossaScan]
steps:
- name: Check out the code
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Run Snyk to check configuration files for security issues
continue-on-error: true
uses: snyk/actions/iac@master
env:
SNYK_TOKEN: $
with:
file: |
manifests/deployment.yml
manifests/service.yml
- name: Upload result to GitHub Code Scanning
uses: github/codeql-action/upload-sarif@v1
with:
sarif_file: snyk.sarif
Snyk dashboard:-
13) OPA - Infrastructure as Code Scan
The Open Policy Agent (OPA, pronounced “oh-pa”) is an open source, general-purpose policy engine that unifies policy enforcement across the stack. OPA provides a high-level declarative language that lets you specify policy as code and simple APIs to offload policy decision-making from your software. You can use OPA to enforce policies in microservices, Kubernetes, CI/CD pipelines, API gateways, and more.
Use OPA to scan the Infrastructure as Code Files using OPA Policies.
opa-test:
name: SAST - Scan IaC Configs Using OPA
runs-on: ubuntu-latest
needs: [dependabot, snykScan, blackduck, fossaScan, shitLeftScan]
steps:
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: OPA-Test manifests standards.
uses: ameydev/ci-cd-standards-using-opa@master
env:
MANIFESTS_PATH_PATTERN: manifests/
LIBRARY_PATH: opa-policies/
DATA: data.kubernetes.admission.deny
OPA result:-
14) Trufflehog - Secret Scan
Trufflehog runs behind the scenes to scan your environment for secrets like private keys and credentials, so you can protect your data before a breach occurs.
Use Trufflehog to find any secrets present in the source code.
trufflehogScan:
name: Secret Scan Using Trufflehog
runs-on: ubuntu-latest
needs: [dependabot, snykScan, blackduck, fossaScan]
steps:
- name: Check out the code
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: trufflehog-actions-scan
uses: edplato/trufflehog-actions-scan@master
with:
scanArguments: "--regex --entropy=False --max_depth=5"
15) GitGuardian - Secret Scan
GitGuardian is a developers-first solution scanning GitHub activity in real-time for API secret tokens, database credentials, certificates.
Use GitGuardian to find any secrets present in the source code. Github integration with GitGuardian is present here.
gitguardian:
name: Secret Scan Using GitGuardian
runs-on: ubuntu-latest
needs: [dependabot, snykScan, blackduck, fossaScan, shitLeftScan]
steps:
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: GitGuardian scan
uses: GitGuardian/ggshield-action@master
with:
args: -v --all-policies
env:
GITHUB_PUSH_BEFORE_SHA: $
GITHUB_PUSH_BASE_SHA: $
GITHUB_PULL_BASE_SHA: $
GITHUB_DEFAULT_BRANCH: $
GITGUARDIAN_API_KEY: $
GitGuardian dashboard:-
16) Snyk - Container Image Scan
Snyk (pronounced sneak) is a developer security platform for securing code, dependencies, containers, and infrastructure as code.
Use Snyk to perform container image scan. GitHub integration with Snyk can find here.
snykImageScan:
name: Image Scan using Snyk
runs-on: ubuntu-latest
needs: [jfrogArtifactPush, gitHubPakageArtifactPush]
steps:
- name: Check out the code
uses: actions/checkout@v1
with:
fetch-depth: 0
- name: Set up JDK 8
uses: actions/setup-java@v1
with:
java-version: 1.8
- name: Cache Maven packages
uses: actions/cache@v1
with:
path: ~/.m2
key: $-m2-$
restore-keys: $-m2
- name: Package
run: mvn -B clean package -DskipTests
- name: Build a Docker image
run: docker build -t your/image-to-test .
- name: Run Snyk to check Docker image for vulnerabilities
continue-on-error: true
uses: snyk/actions/docker@master
env:
SNYK_TOKEN: $
with:
image: your/image-to-test
args: --sarif-file-output=snyk.sarif --file=Dockerfile
- name: Upload result to GitHub Code Scanning
uses: github/codeql-action/upload-sarif@v1
with:
sarif_file: snyk.sarif
Snyk dashboard:-
17) Jfrog Artifactory - Publish Artifact(jar)
Jfrog Artifactory is the only Universal Repository Manager supporting all major packaging formats, build tools and CI servers.
Publish the Jar to Jfrog Artifactory.
jfrogArtifactPush:
name: Publish Artifact to Jfrog Artifactory
runs-on: ubuntu-latest
needs: [snykIaSScan, trufflehogScan]
steps:
- name: Check out the code
uses: actions/checkout@v1
with:
fetch-depth: 0
- name: Set up JDK 8
uses: actions/setup-java@v1
with:
java-version: 1.8
- name: Cache Maven packages
uses: actions/cache@v1
with:
path: ~/.m2
key: $-m2-$
restore-keys: $-m2
- name: Package
run: mvn -B clean package -DskipTests
- name: Publish JFrog Artifact
uses: advancedcsg-open/action-jfrog-cli@master
with:
url: 'https://judebantony.jfrog.io/artifactory'
credentials type: 'username'
user: $
password: $
args: u "target/*.jar" "/libs-snapshot-repo-libs-release-local" --recursive=true
Jfrog Artifactory UI:-
18) GitHub Package - Publish Artifact(jar)
Github Packages is a software package hosting service that allows you to host your software packages privately or publicly and use packages as dependencies in your projects.
Publish the Jar to Github Packages.
gitHubPakageArtifactPush:
name: Publish Artifact to GitHub Package
runs-on: ubuntu-latest
needs: [snyIaSScan, trufflehogScan]
permissions:
contents: read
packages: write
steps:
- name: Check out the code
uses: actions/checkout@v1
with:
fetch-depth: 0
- name: Set up JDK 8
uses: actions/setup-java@v1
with:
java-version: 1.8
- name: Cache Maven packages
uses: actions/cache@v1
with:
path: ~/.m2
key: $-m2-$
restore-keys: $-m2
- name: Publish package
run: mvn --batch-mode deploy -DskipTests
env:
GITHUB_TOKEN: $
In pom.xml, we need to add the below configuration.
<distributionManagement>
<repository>
<id>github</id>
<name>GitHub Packages</name>
<url>https://maven.pkg.github.com/judebantony/cicd-github-action-example</url>
</repository>
</distributionManagement>
GitHub Package UI:-
19) JFrog Artifactory - Build Docker Image and Publish
Jfrog Artifactory is the only Universal Repository Manager supporting all major packaging formats, build tools and CI servers.
Publish the Container Image to Jfrog Artifactory. Dockerfile is present here.
jfrogImageBuild:
name: Build Docker Image and Publish to JFrog Artifactory
runs-on: ubuntu-latest
needs: [snykImageScan, jfrogXrayImageScan]
steps:
- name: Check out the code
uses: actions/checkout@v1
with:
fetch-depth: 0
- name: Set up JDK 8
uses: actions/setup-java@v1
with:
java-version: 1.8
- name: Cache Maven packages
uses: actions/cache@v1
with:
path: ~/.m2
key: $-m2-$
restore-keys: $-m2
- name: Package
run: mvn -B clean package -DskipTests
- name: Set up QEMU️
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Login to JFrog
uses: docker/login-action@v1
with:
registry: 'https://judebantony.jfrog.io'
username: $
password: $
- name: Build and Publish Image
id: docker_build
uses: docker/build-push-action@v2
with:
context: .
push: true
tags: judebantony.jfrog.io/default-docker-local/cigithubaction:latest
Jfrog Artifactory Image Registry UI:-
20) GitHub Package - Build Docker Image and Publish
Github Packages is a software package hosting service that allows you to host your software packages privately or publicly and use packages as dependencies in your projects.
Publish the Container Image to Github Package. Dockerfile is present here.
gitHubPakageImageBuild:
name: Push Docker image to GitHub Package
runs-on: ubuntu-latest
needs: [snykImageScan, jfrogXrayImageScan]
permissions:
contents: read
packages: write
steps:
- name: Check out the repo
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up JDK 8
uses: actions/setup-java@v1
with:
java-version: 1.8
- name: Cache Maven packages
uses: actions/cache@v1
with:
path: ~/.m2
key: $-m2-$
restore-keys: $-m2
- name: Package
run: mvn -B clean package -DskipTests
- name: Set up QEMU️
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Log in to the Container registry
uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
with:
registry: https://ghcr.io
username: $
password: $
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
with:
images: ghcr.io/judebantony/cigithubaction:latest
- name: ghcr.io
uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
with:
context: .
push: true
tags: ghcr.io/judebantony/cigithubaction:latest
In pom.xml, we need to add the below configuration.
<distributionManagement>
<repository>
<id>github</id>
<name>GitHub Packages</name>
<url>https://maven.pkg.github.com/judebantony/cicd-github-action-example</url>
</repository>
</distributionManagement>
GitHub Package Image Registry UI:-
21) Docker Hub - Build Docker Image and Publish
Docker hub is the world’s easiest way to create, manage, and deliver your teams’ container applications.
Publish the Container Image to Docker hub. Dockerfile is present here.
dockerHubImageBuild:
name: Push Docker image to Docker Hub
runs-on: ubuntu-latest
needs: [snykImageScan, jfrogXrayImageScan]
steps:
- name: Check out the repo
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up JDK 8
uses: actions/setup-java@v1
with:
java-version: 1.8
- name: Cache Maven packages
uses: actions/cache@v1
with:
path: ~/.m2
key: $-m2-$
restore-keys: $-m2
- name: Package
run: mvn -B clean package -DskipTests
- name: Log in to Docker Hub
uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
with:
username: $
password: $
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
with:
images: docker.io/judebantony/cigithubaction
- name: Build and push Docker image
uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
with:
context: .
push: true
tags: $
labels: $
Docker Hub Image Registry UI:-
22) CD - Deploy to Azure AKS
AKS allows you to quickly deploy a production ready Kubernetes cluster in Azure.
Deploy the Container image to Azure AKS using kubectl and GitHub Action, kubeconfig manifest files are available here.
qadeploy:
name: QA Deployment to AKS
runs-on: ubuntu-latest
needs: [jfrogImageBuild, dockerHubImageBuild, gitHubPakageImageBuild]
steps:
- name: Check out the code
uses: actions/checkout@v1
with:
fetch-depth: 0
- name: Login to JFrog
uses: docker/login-action@v1
with:
registry: 'https://judebantony.jfrog.io'
username: $
password: $
- name: Setting AKS Context
uses: azure/aks-set-context@v1
with:
creds: $
cluster-name: 'JudeAKSCluster'
resource-group: 'DefaultResourceGroup-EUS'
- name: Create AKS Namespace
run: |
kubectl create namespace cigithubactionqa --dry-run -o json | kubectl apply -f -
- name: Create Secret
uses: azure/k8s-create-secret@v1
with:
container-registry-url: 'https://judebantony.jfrog.io'
container-registry-username: $
container-registry-password: $
namespace: 'cigithubactionqa'
secret-name: 'cigithubactionqa-crd'
- name: Deploy app to AKS
uses: azure/k8s-deploy@v1
with:
manifests: |
manifests/deployment.yml
manifests/service.yml
images: |
judebantony.jfrog.io/default-docker-local/cigithubaction:latest
imagepullsecrets: |
cigithubactionqa-crd
namespace: 'cigithubactionqa'
AKS Console:-
23) Functional Test - Using Cucumber.
Cucumber is the world’s leading tool for BDD.
Run the Cucumber Test Cases and upload the result to cucumber.io and GitHub Action. The feature files are present here.
qatest:
name: QA Validation and Report
runs-on: ubuntu-latest
needs: [qadeploy]
steps:
- name: Check out the code
uses: actions/checkout@v1
with:
fetch-depth: 0
- name: Set up JDK
uses: actions/setup-java@v1
with:
java-version: 1.8
- name: Cache Maven packages
uses: actions/cache@v1
with:
path: ~/.m2
key: $-m2-$
restore-keys: $-m2
- name: Run Test Automation
run: mvn -B verify -DexcludedGroups="Staging | LamdaTest | BrowserStack" -Dgroups="Smoke"
env:
CUCUMBER_PUBLISH_TOKEN: $
- name: Upload Test Automation Report
uses: deblockt/cucumber-report-annotations-action@v1.7
with:
access-token: $
path: "target/cucumber-reports/cucumber.json"
check-status-on-error: 'neutral'
annotation-status-on-error: 'warning'
- name: Upload Cucumber report
uses: actions/upload-artifact@v2
with:
name: cucumber-report
path: target/cucumber-reports/cucumber.html
Cucumber Report:-
Cucumber.io Report:-
24) Functional UI Test - Using BrowserStack.
BrowserStack is an Indian cloud web and mobile testing platform that provides developers with the ability to test their websites and mobile applications across on-demand browsers, operating systems and real mobile devices.
Selenium is an open-source umbrella project for a range of tools and libraries aimed at supporting web browser automation. Selenium provides a playback tool for authoring functional tests without the need to learn a test scripting language.
Run the Selenium Cucumber based UI Test Cases using BrowserStack Selenium Grid and capture the result in their Cloud SaaS offering. The test case feature files are present here.
browserStackTest:
name: 'BrowserStack QA Test Validation'
runs-on: ubuntu-latest
needs: [qadeploy]
steps:
- name: 'BrowserStack Env Setup'
uses: browserstack/github-actions/setup-env@master
with:
username: $
access-key: $
- name: 'BrowserStack Local Tunnel Setup'
uses: browserstack/github-actions/setup-local@master
with:
local-testing: start
local-identifier: random
- name: Check out the code
uses: actions/checkout@v1
with:
fetch-depth: 0
- name: Set up JDK
uses: actions/setup-java@v1
with:
java-version: 1.8
- name: Cache Maven packages
uses: actions/cache@v1
with:
path: ~/.m2
key: $-m2-$
restore-keys: $-m2
- name: Run BrowserStack Test Automation
run: mvn -B verify -DexcludedGroups="Staging | Smoke | LamdaTest" -Dgroups="BrowserStack"
env:
CUCUMBER_PUBLISH_TOKEN: $
- name: 'BrowserStackLocal Stop'
uses: browserstack/github-actions/setup-local@master
with:
local-testing: stop
BrowserStack dashboard:-
25) Functional UI Test - Using LamdaTest.
Run your Selenium test automation scripts across online selenium grid of desktop, Android and iOS mobile browsers. Develop, test, and deliver faster every time with automated cross browser testing using LambdaTest online Automation Browser Testing Grid.
Execute the Selenium Cucumber based UI Test Cases using LamdaTest and capture the result. The feature files are present here.
lamdaTest:
name: 'LamdaTest QA Test Validation'
runs-on: ubuntu-latest
needs: [qadeploy]
steps:
- name: Start Tunnel
id: tunnel
uses: LambdaTest/LambdaTest-tunnel-action@v1
with:
user: $
accessKey: $
tunnelName: "testTunnel"
- name: Check out the code
uses: actions/checkout@v1
with:
fetch-depth: 0
- name: Set up JDK
uses: actions/setup-java@v1
with:
java-version: 1.8
- name: Cache Maven packages
uses: actions/cache@v1
with:
path: ~/.m2
key: $-m2-$
restore-keys: $-m2
- name: Run LamdaTest Automation
run: mvn -B verify -DexcludedGroups="Staging | Smoke | BrowserStack" -Dgroups="LamdaTest"
env:
CUCUMBER_PUBLISH_TOKEN: $
LT_EMAIL: $
LT_ACCESS_KEY: $
- name: Export Tunnel Logs for debugging
uses: actions/upload-artifact@v2
with:
name: tunnel_logs
path: $
LambdaTest dashboard:-
26) DAST Scan - Using StackHawk.
StackHawk makes it simple for developers to find, triage, and fix application security bugs. Scan your application for AppSec bugs in the code your team wrote, triage and fix with provided documentation, and automate in your pipeline to prevent future bugs from hitting prod.
Use StackHawk for DAST scan for all the OpenAPI (REST APIs). StackHawk config file is present here.
stackhawkScan:
name: DAST Scan using StackHawk
runs-on: ubuntu-20.04
needs: [qatest, browserStackTest, lamdaTest]
steps:
- name: Checkout code
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Download OpenAPI Spec
run: |
curl http://20.81.93.165/api-docs > openapi.json
- name: Run HawkScan
uses: stackhawk/hawkscan-action@v1.3.2
continue-on-error: true
with:
apiKey: $
configurationFiles: stackhawk.yml
codeScanningAlerts: true
githubToken: $
environmentVariables: |
APP_HOST
APP_ENV
APP_ID
env:
APP_HOST: http://20.81.93.165
APP_ENV: Development
APP_ID: ea0079f1-648e-4bdb-aa2d-233696082b4e
StackHawk dashboard:-
27) DAST Scan - Using OWASP ZAP.
OWASP ZAP is an open-source web application security scanner. It is intended to be used by both those new to application security as well as professional penetration testers. It is one of the most active Open Web Application Security Project projects and has been given Flagship status.
Use OWASP ZAP for DAST scan for all the OpenAPI (REST APIs). OWASP ZAP rule file is present here.
zap_scan:
runs-on: ubuntu-latest
name: DAST Scan using OWASP ZAP
needs: [qatest, browserStackTest, lamdaTest]
steps:
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: ZAP Scan
uses: zaproxy/action-api-scan@v0.1.0
with:
token: $
docker_name: 'owasp/zap2docker-stable'
format: openapi
target: 'http://35.194.14.180/api-docs'
rules_file_name: 'zap/rules.tsv'
cmd_options: '-a'
ZAP Scan Report:-
Result is uploaded in Github Action.
Issue created on GitHub
28) API Validator - Using Swagger Editor Validator.
Swagger Editor lets you edit Swagger API specifications in YAML inside your browser and to preview documentations in real time. Valid Swagger JSON descriptions can then be generated and used with the full Swagger tooling (code generation, documentation, etc).
Use Swagger Editor Validator for validating the OpenAPI Specification of REST APIs .
test_swagger_editor_validator:
runs-on: ubuntu-latest
name: Swagger Editor Validator Remote
needs: [stackhawkScan, zap_scan]
steps:
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Download OpenAPI Spec
run: |
curl http://35.194.14.180/api-docs > openapi.json
- name: Validate OpenAPI definition
uses: char0n/swagger-editor-validate@v1.2.1
with:
definition-file: openapi.json
Build output:-
29) Setting up Approval Gates and Email.
Approval gates for deployment using Github Environment feature.
stagingdeployapproval:
name: Waiting for Staging Deployment Approval
runs-on: ubuntu-latest
needs: [stackhawkScan]
environment: staging
steps:
- name: Email Status
uses: dawidd6/action-send-mail@v3
with:
server_address: smtp.gmail.com
server_port: 465
username: $
password: $
subject: Stage Deployment $ waiting for your approval.
to: judebantony@gmail.com
from: judebantonyofficial@gmail.com
body: Please review the test result and approve it.
reply_to: judebantonyofficial@gmail.com
in_reply_to: judebantonyofficial@gmail.com
ignore_cert: true
convert_markdown: true
priority: low
30) CD - Deploy to Azure AKS using Helm.
Helm helps you manage Kubernetes applications — Helm Charts help you define, install, and upgrade even the most complex Kubernetes application.
Deploy the Container image to Azure AKS using Helm, manifest files are available here.
stagingdeploy:
name: Staging Deployment Using Helm To AKS
runs-on: ubuntu-latest
needs: [stagingdeployapproval]
steps:
- name: Check out the code
uses: actions/checkout@v1
with:
fetch-depth: 0
- name: Install Helm
uses: Azure/setup-helm@v1
with:
version: v3.7.1
- name: Login to JFrog
uses: docker/login-action@v1
with:
registry: 'https://judebantony.jfrog.io'
username: $
password: $
- name: Setting AKS Context
uses: azure/aks-set-context@v1
with:
creds: $
cluster-name: 'JudeAKSCluster'
resource-group: 'DefaultResourceGroup-EUS'
- name: Create AKS Namespace
run: |
kubectl create namespace cigithubactionstaging --dry-run -o json | kubectl apply -f -
- name: Create Secret
uses: azure/k8s-create-secret@v1
with:
container-registry-url: 'https://judebantony.jfrog.io'
container-registry-username: $
container-registry-password: $
namespace: 'cigithubactionstaging'
secret-name: 'cigithubactionstaging-crd'
- name: Run Helm Deploy
run: |
helm upgrade \
--install \
--create-namespace \
--atomic \
--wait \
--namespace cigithubactionstaging \
cigithubaction \
./helm/aks \
AKS Console:-
31) CD - Deploy to Google GKE using Harness.
The Harness Software Delivery Platform includes Continuous Integration, Continuous Delivery, Continuous Efficiency, Continuous Verification, and Continuous Features. The platform is designed to help companies accelerate their cloud initiatives as well as their adoption of containers and orchestration tools like Kubernetes and Amazon ECS.
Deploy the Container image to Google GKE using Harness.
uatdeploy:
name: UAT Deployment using Harness
runs-on: ubuntu-latest
needs: [uateployapproval]
steps:
- name: Run Harness UAT Deployment
run: |
curl -X POST -H 'content-type: application/json' --url https://app.harness.io/gateway/api/webhooks/Tlugr1ZdISx44rvm4flAiXHMb3uKG3ikyiHSbOks?accountId=aGS5Pi_WSPa9IsdlTlJc7g -d '{"application":"1FYrnQdZROqjpAQdCBIMbw"}'
GKE console:-
Harness console:-
32) Load/Performance Testing - K6.
K6 is an open-source load testing tool and cloud service providing the best developer experience for API performance testing.
Use K6 to do the Performance Test. Load Test file is present here.
k6_cloud_test:
name: Perf Testing - k6 cloud test run
runs-on: ubuntu-latest
needs: [terraform]
steps:
- name: Checkout
uses: actions/checkout@v1
with:
fetch-depth: 0
- name: Run k6 cloud test
uses: k6io/action@v0.1
with:
filename: k6-test.ts
cloud: true
token: $
K6 dashboard:-
33) Functional Test using Xray and Jira.
XRay is a Test Management tool integrated with Jira. Built for every member of your team to plan, test, track and release great software · Manage all your tests as Jira issues.
Xray Jira plugin is used to create the test case in Gherkin and Cucumber for each Jira story and executed as part of CI/CD pipleline(GitHub Action). The result is uploaded back to Jira. XRay Test Execution config file is present here.
jiraXrayTest:
name: Functional Test using Xray and Jira
runs-on: ubuntu-latest
needs: [uatdeploy]
steps:
- name: Check out the code
uses: actions/checkout@v1
with:
fetch-depth: 0
- name: Set up JDK
uses: actions/setup-java@v1
with:
java-version: 1.8
- name: Cache Maven packages
uses: actions/cache@v1
with:
path: ~/.m2
key: $-m2-$
restore-keys: $-m2
- name: Get Xray Cloud API token
env:
CLIENT_ID: $
CLIENT_SECRET: $
id: xray-token
run: |
echo ::set-output name=XRAY_TOKEN::$(curl -H "Content-Type: application/json" -X POST --data "{ \"client_id\": \"$CLIENT_ID\",\"client_secret\": \"$CLIENT_SECRET\" }" https://xray.cloud.xpand-it.com/api/v1/authenticate| tr -d '"')
- name: Get Features from XRAY
shell: bash
run: |
curl -H "Content-Type: application/json" --output ./features.zip -X GET -H "Authorization: Bearer $" "https://xray.cloud.xpand-it.com/api/v1/export/cucumber?keys=JUDE-43"
unzip -o features.zip -d ./src/test/resources/com/jba/ci/bdd/
- name: Build
run: mvn -B clean package -DskipTests
- name: Run UnitTest and Verify
run: mvn -B verify -DexcludedGroups="Smoke | Staging | BrowserStack | LamdaTest"
- name: Generate JaCoCo Badge
id: jacoco
uses: cicirello/jacoco-badge-generator@v2
- name: Log code coverage percentage
run: |
echo "coverage = $"
echo "branch coverage = $"
- name: Upload to XRAY
shell: bash
run: |
curl -X POST -H "Authorization: Bearer $" -F info=@testexec_cloud_template.json -F results=@"target/cucumber-reports/cucumber.json" "https://xray.cloud.xpand-it.com/api/v1/import/execution/cucumber/multipart"
Jira Xray test execution dashboard:-
34) Test Management using Katalon.
Katalon is a Test Management tool integrated with Jira.
- name: Katalon Report Uploader
uses: katalon-studio/report-uploader@v0.0.7.11
env:
EMAIL: $
PASSWORD: $
PROJECT_ID: 494065
TYPE: junit
REPORT_PATH: target/surefire-reports/
Katalon test execution dashboard:-
35) Release Tag & Note Creation.
Create a release tag for the branch.
releaseTag:
name: Release Tag & Note Creation
runs-on: ubuntu-latest
needs: [k6_cloud_test]
steps:
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Bump version and push tag
id: tag_version
uses: anothrNick/github-tag-action@1.26.0
env:
GITHUB_TOKEN: $
WITH_V: true
- name: Build Changelog
id: github_release
uses: mikepenz/release-changelog-builder-action@v1
env:
GITHUB_TOKEN: $
- name: Create Release
uses: actions/create-release@v1
with:
tag_name: $
release_name: Release $
body: $
env:
GITHUB_TOKEN: $
Release Tag:-
36) IaC - using Terraform - Create AWS EC2.
Terraform is an open-source infrastructure as code software tool created by HashiCorp. Users define and provide data center infrastructure using a declarative configuration language known as HashiCorp Configuration Language, or optionally JSON.
Set up the AWS EC2 instances using Terraform , terraform manifest file is available here.
terraform:
name: "Terraform - Create AWS EC2"
runs-on: ubuntu-latest
needs: [uatdeploy]
steps:
- name: Checkout
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Setup Terraform
uses: hashicorp/setup-terraform@v1
with:
cli_config_credentials_token: $
- name: Terraform Format
id: fmt
run: |
cd terraform/
terraform fmt -check
- name: Terraform Init
id: init
run: |
cd terraform/
terraform init
- name: Terraform Validate
id: validate
run: |
cd terraform/
terraform validate -no-color
- name: Terraform Plan
id: plan
run: |
cd terraform/
terraform plan -no-color
continue-on-error: true
- uses: actions/github-script@0.9.0
id: return_plan_outpot
env:
PLAN: "terraform\n$"
with:
github-token: $
script: |
const output = `#### Terraform Format and Style \`$\`
#### Terraform Initialization ️\`$\`
#### Terraform Validation \`$\`
#### Terraform Plan \`$\`
<details><summary>Show Plan</summary>
\`\`\`\n
${process.env.PLAN}
\`\`\`
</details>
*Pusher: @$, Action: \`$\`*`;
return output;
- name: Create plan result
uses: "finnp/create-file-action@master"
env:
FILE_NAME: "plan.html"
FILE_DATA: "$"
- name: Upload Terraform Plan result
uses: actions/upload-artifact@v2
with:
name: terrform-plan-result
path: plan.html
- name: Terraform Plan Status
if: steps.plan.outcome == 'failure'
run: exit 1
- name: Terraform Apply
run: |
cd terraform/
terraform apply -auto-approve
Terraform dashbord:-
Author
LinkedIn
GitHubLicense
This project is licensed under the Apache License - see the LICENSE file for details