Overview

This tutorial will guide you through the step by step procedure to set up Jenkins on a ubuntu machine for automation purposes.

Connect to the remote machine using SSH and follow the steps mentioned below.

Install Java 8

• sudo apt update

• sudo apt-get install openjdk-8-jdk

• java -version

Install Jenkins

• sudo wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -

• sudo sh -c 'echo deb Debian Jenkins Packages binary/ > /etc/apt/sources.list.d/jenkins.list'

• sudo apt update

• sudo apt install jenkins

• sudo systemctl start jenkins

• sudo systemctl status jenkins (should be active)

Install and setup apache2 for Jenkins

• sudo apt install apache2

• sudo a2enmod proxy

• sudo a2enmod proxy_http

• sudo a2enmod headers

• sudo vi /etc/apache2/sites-available/000-default.conf and add following code inside VirtualHost xml tag:

##### ############################################################ ####
##### ============================================================ ####
##### The Build Business Websites Jenkins and Apache Configuration ####
##### ============================================================ ####
##### ############################################################ ####

ProxyPass         /jenkins  http://localhost:8080/jenkins nocanon
ProxyPassReverse  /jenkins  http://localhost:8080/jenkins

ProxyRequests     Off
AllowEncodedSlashes NoDecode

<Proxy http://localhost:8080/jenkins*>
  Order deny,allow
  Allow from all
</Proxy>

##### ============================================================ ####
##### ############################################################ ####

• sudo vi /etc/default/jenkins and make sure

JENKINS_ARGS="--webroot=/var/cache/$NAME/war --httpPort=$HTTP_PORT --prefix=/jenkins"

• sudo systemctl restart jenkins

• sudo systemctl restart apache2

• Open port 80 of the remote machine to make it accessible from the local machine

Setup Jenkins

• sudo vi /var/lib/jenkins/secrets/initialAdminPassword

• copy the initial password inside the file

• now open remote-host-ip/jenkins in local browser

• paste the password, select the required plugin to install and start the installation

Executed test results and reports of all tests can be viewed in the target folder.

Pre-requisite: karate Runner plugin installation in VS code editor as specified in Readme.md

If not done during setup, then please install karate runner plugin as shown below.

Target folder generates test results folders as per time stamp of test suite runs.

When we run TestRunner file (ex: @regression), all the features under classpath would run and feature wise reports would be generated and placed under target/surefire-reports.

Post execution of test runs, the framework will automatically generate: surefire-reports

• karate-summary.html

• cucumber-html-reports folder

A folder will automatically create under ./target/ folder along with timestamp which will contain cucumber-html-reports.

Select the karate runner tab and click on overview-features.html to view the reports as shown in image.

Summary of service features and test status can be seen in details with Pass / Fail / run time stamp

****

Now, observe below Feature Report, you can see lots of verbose logging in the console and the actual output whether the features/scenarios are passed/ failed are skipped.

• karate-summary.html:

Overview

This page offers step-wise details to test the implementation of state-level UI customization.

Steps

Create a new maven project for State-level UI automation for the respective state.

Create a new repository in GitHub for the project and push the code to GitHub.

Keep the following settings.xml file under the .m2 folder in your system. Update the username and personal access token in the file accordingly.

<profiles>
<profile>
<id>github</id>
<repositories>
<repository>
<id>central</id>
<url>https://repo1.maven.org/maven2</url>
</repository>
<repository>
<id>github</id>
<url>https://maven.pkg.github.com/egovernments/ui-statelevel-automation</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
</profile>
</profiles>
<servers>
<server>
<id>github</id>
<username>USERNAME</username>
<password>PERSONAL ACCESS TOKEN</password>
</server>

Add the dependency of the core UI automation repo in the pom.xml of the newly created maven project. This will download the core UI automation jar from the central maven repository.

<dependency>
<groupId>stateeGovernanceUI</groupId>
<artifactId>satateegovui</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>

Add the state-level tests in this new maven project and use the already existing step definitions and utilities from the core UI automation repo.

• Add feature files under the package: src/test/java/

• Add step definitions under the package: src/main/java/stepDefinition

• Add test data in the following folder: src/test/java/TestData

• Add testing.xml with the test which is needed to run

• To run the tests, execute the following maven command:

$ mvn test -DdataPath=pathToDataPropertiesFile -DcredsPath=pathToCredentialsPropertiesFile

Note:: Find below the data.properties and credentials.properties file below for your reference. In the files, replace your applicable creds and data in the placeholder. For example: Replace “password = {Enter password}” with “password = pass@178"

data.properties

credentials.properties

Sample state-wise demo project is in the following Github location for your reference - https://github.com/egovernments/ui-statelevel-implementation

Overview

DIGIT Test Automation framework majorly deals with various API services validation. It enables to test and validate back end services across all active environments and as well as tenants.

Scope

• Framework is robust enough to function with minimal changes

• Framework handles multi tenant with only properties update

• Framework can switch environment with no manual update in automation code

• Framework handles Kafka related services

• Framework supports browser integration for making citizen payment via payment gateway

List of Services Completed

• Core Services

  • egov-accesscontrol, egov-enc-service, egov-filestore, egov-idgen, egov-indexer, egov-localization, egov-location, egov-mdms-service, egov-notification-mail, egov-notification-sms, egov-otp, egov-persister, egov-pg-service, egov-searcher, egov-url-shortening, egov-user, egov-workflow-v2, pdf-service, report, telemetry, user-otp, zuul

• Business Services

  • billing-service, collection-services, dashboard-analytics, dashboard-ingest, egf-instrument, egf-master, egov-apportion-service, egov-hrms

• Municipal Services

  • bpa-calculator, bpa-services, firenoc-calculator, firenoc-services, land-services, property-services, pt-calculator-v2, rainmaker-pgr, sw-calculator, sw-services, tl-calculator, tl-services, ws-calculator, ws-services

• E2E - Integration of services

E2E pt services, E2E ws services, E2E pgr services, E2E tl services, mCollect

Approach

Automation is accomplished by building a custom framework using Karate Framework. Karate Framework is a BDD framework which is written on top of cucumber. Unlike cucumber which requires us to write step definitions for each and every scenarios, Karate supports writing scenarios directly in the feature files.

• Automation framework fetches data for respective modules wherever predefined values are required while creating a payload

• And uses generic functions to generate random mobile number, name, etc to pass values for respective fields in payload

• Framework allows to test kafka related services by using third party tool as a proxy to access kafka clusters via REST API called kafka Rest Proxy

Tools Used

• Java

• JavaScript

• Karate

• Cucumber

• JUnit

• Kafka Rest Proxy

• Maven

Environments currently working on

• https://qa.digit.org/

• https://uat.digit.org/

Observations

• Currently, there are failures in the automation scripts due to existing bugs

• Since there are some limitations with Kafka Rest Proxy, Kafka related test cases will not run. Awaiting the required fixes in its upcoming release

Reference Docs

Doc Links

Problem Statement

• Provide an approach on how Consumers and Producer based APIs will be tested

• POC on below core services to showcase all automation principles and guide

• Localisation- Upsert and search call

• FileStore- Search File

• Property Tax- Create Property

The Solution

• Design & Implement Robust Automation Framework

• Build a modular Framework so any future changes don’t have an impact on the entire framework but limited to a specific area

The Goal

• Implement a framework that is robust enough to function with minimal changes

• Framework handles multi tenant with only properties update

• Framework can switch environment with no manual update

• A new environment can be integrated with update to framework properties

Plan and Outcomes

Automation Framework: High-Level Architecture

Description

Global Properties:

1. Configuration The only rule is that on start-up Karate expects a file called karate-config.js to exist on the classpath and contain a JavaScript function. The function is expected to return a JSON object and all keys and values in that JSON object will be made available as script variables. And that's all there is to Karate configuration! You can easily get the value of the current 'environment' or 'profile', and then set up 'global' variables using some simple JavaScript.

2. Global Variables: Non-JSON values such as Java object references or JS functions are supported only if they are at the "root" of the JSON returned from karate-config.js.

3. Multiple Environment: Karate is flexible, you can easily overwrite config variables within each individual test-script - which is very convenient when in dev-mode or rapid-prototyping and also environment specific Config will be taken care of here. external config file having host name, credentials, stateCode, cityCode, etc.

4. Test Properties: Contains all properties related to Tests cases like Feature file, Http requests, responses, test data etc.

5. Framework Properties: Contains All properties related to Karate framework like pom.xml, YAML files, .md files etc.

6. Reports: We can generate default BDD cucumber reports and we can integrate reporting plugins.

7. Tests: Contains test scenarios feature files with execution tags, Assertions etc.

8. MDMS call: Fetching the test data from MDMS and using in tests.

Technologies Used

• Java

• JavaScript

• Gherkin

• BDD framework - Behavior Driven Development is a software development approach

Workflow Introduction

This document is to provide a high-level abstraction of Kafka workflow. Same documentation would be scaled to granular technical components as and when a grouping of services can be categorised from the ecosystem.

It’s important to note that we need to have closure engagements between development, solution engineering and QA automation teams to accommodate data streaming, staging and ecosystem workflows.

Kafka Abstraction Work Flow

Kafka SMS Consumers

egov.core.notification.sms : egov-notification-sms listens to this topic to get the data.

Kafka Email Consumers

egov.core.notification.email : egov-notification-mail listens to this topic to listen for the updates on emails and then to send notifications to user.

Kafka Producers

Following are the Producer topic:

egov.core.notification.sms.otp :- This topic is used to send OTP to user mobile number.

org.egov.core.notification.email :- This topic is used to send OTP to user email id.

Approach 1: User OTP

The user-otp service send the OTP to user on login request, on password change request and during new user registration.

1. /userOtp API - Invoke (PRODUCER) this API using Karate (user-otp).

2. Connect to the Kafka Server running on QA using karate with respective TOPIC.

3. Define a Consumer class for the OTP producer Topic for reading producer data. Consumers will listen to the topic and verify for a given mobile number for which OTP has been generated or not.

4. Persist OTP produced data across the test flow of User OTP API.

Actions to Explore and Data Collection

1. Full set of Producer and Consumer list required from End to End Eco-system

2. Filter grouping, Replication, High Availability, Broker Config Elements

Log Aggregation Solution

1. Log Aggregation Solution: Collect logs from a number of eGov services and then make them available to consumers in a standard format.