1 of 7

QA Automation

DIGIT Test Automation Framework details

Overview

DIGIT Test Automation framework majorly deals with various API services validation. It enables to test and validate back end services across all active environments and as well as tenants.

Scope

Framework is robust enough to function with minimal changes
Framework handles multi tenant with only properties update
Framework can switch environment with no manual update in automation code
Framework handles Kafka related services
Framework supports browser integration for making citizen payment via payment gateway

List of Services Completed

Core Services
- egov-accesscontrol, egov-enc-service, egov-filestore, egov-idgen, egov-indexer, egov-localization, egov-location, egov-mdms-service, egov-notification-mail, egov-notification-sms, egov-otp, egov-persister, egov-pg-service, egov-searcher, egov-url-shortening, egov-user, egov-workflow-v2, pdf-service, report, telemetry, user-otp, zuul
Business Services
- billing-service, collection-services, dashboard-analytics, dashboard-ingest, egf-instrument, egf-master, egov-apportion-service, egov-hrms
Municipal Services
- bpa-calculator, bpa-services, firenoc-calculator, firenoc-services, land-services, property-services, pt-calculator-v2, rainmaker-pgr, sw-calculator, sw-services, tl-calculator, tl-services, ws-calculator, ws-services
E2E - Integration of services

E2E pt services, E2E ws services, E2E pgr services, E2E tl services, mCollect

Approach

Automation is accomplished by building a custom framework using Karate Framework. Karate Framework is a BDD framework which is written on top of cucumber. Unlike cucumber which requires us to write step definitions for each and every scenarios, Karate supports writing scenarios directly in the feature files.

Automation framework fetches data for respective modules wherever predefined values are required while creating a payload
And uses generic functions to generate random mobile number, name, etc to pass values for respective fields in payload
Framework allows to test kafka related services by using third party tool as a proxy to access kafka clusters via REST API called kafka Rest Proxy

Tools Used

Environments currently working on

Observations

Currently, there are failures in the automation scripts due to existing bugs
Since there are some limitations with Kafka Rest Proxy, Kafka related test cases will not run. Awaiting the required fixes in its upcoming release

Reference Docs

Doc Links

Kafka Consumer Producer Approach

Workflow Introduction

This document is to provide a high-level abstraction of Kafka workflow. Same documentation would be scaled to granular technical components as and when a grouping of services can be categorised from the ecosystem.

It’s important to note that we need to have closure engagements between development, solution engineering and QA automation teams to accommodate data streaming, staging and ecosystem workflows.

Kafka Abstraction Work Flow

Kafka SMS Consumers

egov.core.notification.sms : egov-notification-sms listens to this topic to get the data.

Kafka Email Consumers

egov.core.notification.email : egov-notification-mail listens to this topic to listen for the updates on emails and then to send notifications to user.

Kafka Producers

Following are the Producer topic:

egov.core.notification.sms.otp :- This topic is used to send OTP to user mobile number.

org.egov.core.notification.email :- This topic is used to send OTP to user email id.

Approach 1: User OTP

The user-otp service send the OTP to user on login request, on password change request and during new user registration.

/userOtp API - Invoke (PRODUCER) this API using Karate (user-otp).
Connect to the Kafka Server running on QA using karate with respective TOPIC.
Define a Consumer class for the OTP producer Topic for reading producer data. Consumers will listen to the topic and verify for a given mobile number for which OTP has been generated or not.
Persist OTP produced data across the test flow of User OTP API.

Actions to Explore and Data Collection

Full set of Producer and Consumer list required from End to End Eco-system
Filter grouping, Replication, High Availability, Broker Config Elements

Log Aggregation Solution

Log Aggregation Solution: Collect logs from a number of eGov services and then make them available to consumers in a standard format.

Jenkins Setup for Automation

Overview

This tutorial will guide you through the step by step procedure to set up Jenkins on a ubuntu machine for automation purposes.

Connect to the remote machine using SSH and follow the steps mentioned below.

Install Java 8

sudo apt update
sudo apt-get install openjdk-8-jdk
java -version

Install Jenkins

sudo wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb Debian Jenkins Packages binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt update
sudo apt install jenkins
sudo systemctl start jenkins
sudo systemctl status jenkins (should be active)

Install and setup apache2 for Jenkins

sudo apt install apache2
sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod headers
sudo vi /etc/apache2/sites-available/000-default.conf and add following code inside VirtualHost xml tag:

##### ############################################################ ####
##### ============================================================ ####
##### The Build Business Websites Jenkins and Apache Configuration ####
##### ============================================================ ####
##### ############################################################ ####

ProxyPass         /jenkins  http://localhost:8080/jenkins nocanon
ProxyPassReverse  /jenkins  http://localhost:8080/jenkins

ProxyRequests     Off
AllowEncodedSlashes NoDecode

<Proxy http://localhost:8080/jenkins*>
  Order deny,allow
  Allow from all
</Proxy>

##### ============================================================ ####
##### ############################################################ ####

sudo vi /etc/default/jenkins and make sure

JENKINS_ARGS="--webroot=/var/cache/$NAME/war --httpPort=$HTTP_PORT --prefix=/jenkins"

sudo systemctl restart jenkins
sudo systemctl restart apache2
Open port 80 of the remote machine to make it accessible from the local machine

Setup Jenkins

sudo vi /var/lib/jenkins/secrets/initialAdminPassword
copy the initial password inside the file
now open remote-host-ip/jenkins in local browser
paste the password, select the required plugin to install and start the installation

Automation Framework Knowledge Base

Problem Statement

Provide an approach on how Consumers and Producer based APIs will be tested
POC on below core services to showcase all automation principles and guide
Localisation- Upsert and search call
FileStore- Search File
Property Tax- Create Property

The Solution

Design & Implement Robust Automation Framework
Build a modular Framework so any future changes don’t have an impact on the entire framework but limited to a specific area

The Goal

Implement a framework that is robust enough to function with minimal changes
Framework handles multi tenant with only properties update
Framework can switch environment with no manual update
A new environment can be integrated with update to framework properties

Plan and Outcomes

Automation Framework: High-Level Architecture

Description

Global Properties:

Configuration The only rule is that on start-up Karate expects a file called karate-config.js to exist on the classpath and contain a JavaScript function. The function is expected to return a JSON object and all keys and values in that JSON object will be made available as script variables. And that's all there is to Karate configuration! You can easily get the value of the current 'environment' or 'profile', and then set up 'global' variables using some simple JavaScript.
Global Variables: Non-JSON values such as Java object references or JS functions are supported only if they are at the "root" of the JSON returned from karate-config.js.
Multiple Environment: Karate is flexible, you can easily overwrite config variables within each individual test-script - which is very convenient when in dev-mode or rapid-prototyping and also environment specific Config will be taken care of here. external config file having host name, credentials, stateCode, cityCode, etc.
Test Properties: Contains all properties related to Tests cases like Feature file, Http requests, responses, test data etc.
Framework Properties: Contains All properties related to Karate framework like pom.xml, YAML files, .md files etc.
Reports: We can generate default BDD cucumber reports and we can integrate reporting plugins.
Tests: Contains test scenarios feature files with execution tags, Assertions etc.
MDMS call: Fetching the test data from MDMS and using in tests.

Technologies Used

Java
JavaScript
Gherkin
BDD framework - Behavior Driven Development is a software development approach

Automation Test Tags

Automation Test Reporting

Executed test results and reports of all tests can be viewed in the target folder.

Pre-requisite: karate Runner plugin installation in VS code editor as specified in Readme.md

If not done during setup, then please install karate runner plugin as shown below.

Target folder generates test results folders as per time stamp of test suite runs.

When we run TestRunner file (ex: @regression), all the features under classpath would run and feature wise reports would be generated and placed under target/surefire-reports.

Post execution of test runs, the framework will automatically generate: surefire-reports

karate-summary.html

As we are using the JUnit runner for the execution of different scenarios that have been mentioned in the different *.feature files, it automatically creates a report for each of the feature files which are stored in the path target/surefire-reports.

cucumber-html-reports folder

Pretty Format generates the Cucumber test report in the HTML format, i.e. an HTML file. It is the most readable report format. It generates the report in the same way as it is a feature file, so tracing is also made easy. Also, you can specify the location where you want this report to be placed after the test execution.

A folder will automatically create under ./target/ folder along with timestamp which will contain cucumber-html-reports.

<To Integrate cucumber reporting, first we need to add cucumber maven dependency in pom.xml file and save. dependency> <groupId>net.masterthought</groupId> <artifactId>cucumber-reporting</artifactId> <version>3.8.0</version> </dependency>
Test Results target folder: eGov Test Automation Results are composed at cucumber-html-reports with timestamp of test execution reports.

Select the karate runner tab and click on overview-features.html to view the reports as shown in image.

Summary of service features and test status can be seen in details with Pass / Fail / run time stamp

****

Now, observe below Feature Report, you can see lots of verbose logging in the console and the actual output whether the features/scenarios are passed/ failed are skipped.

karate-summary.html:

State-Level UI Customization

Overview

This page offers step-wise details to test the implementation of state-level UI customization.

Steps

Create a new maven project for State-level UI automation for the respective state.

Create a new repository in GitHub for the project and push the code to GitHub.

Keep the following settings.xml file under the .m2 folder in your system. Update the username and personal access token in the file accordingly.

<profiles>
<profile>
<id>github</id>
<repositories>
<repository>
<id>central</id>
<url>https://repo1.maven.org/maven2</url>
</repository>
<repository>
<id>github</id>
<url>https://maven.pkg.github.com/egovernments/ui-statelevel-automation</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
</profile>
</profiles>
<servers>
<server>
<id>github</id>
<username>USERNAME</username>
<password>PERSONAL ACCESS TOKEN</password>
</server>

Add the dependency of the core UI automation repo in the pom.xml of the newly created maven project. This will download the core UI automation jar from the central maven repository.

<dependency>
<groupId>stateeGovernanceUI</groupId>
<artifactId>satateegovui</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>

Note: Version may differ based on the latest version.

Add the state-level tests in this new maven project and use the already existing step definitions and utilities from the core UI automation repo.

Add feature files under the package: src/test/java/

Add step definitions under the package: src/main/java/stepDefinition

Add test data in the following folder: src/test/java/TestData

Add testing.xml with the test which is needed to run

To run the tests, execute the following maven command:

$ mvn test -DdataPath=pathToDataPropertiesFile -DcredsPath=pathToCredentialsPropertiesFile

Note:: Find below the data.properties and credentials.properties file below for your reference. In the files, replace your applicable creds and data in the placeholder. For example: Replace “password = {Enter password}” with “password = pass@178"

data.properties

baseUrl={Enter Base Url ex:https://digit.org/}empUrl={Enter employee ex:employee/language-selection}citizenUrl={ Enter Citizen Url ex:digit-ui/citizen}mobile={Enter Mobile Number ex:9869313101}languageEng={Enter langauage ex:ENGLISH}State={Enter State ex:Punjab}location={Enter location ex:Amritsar}PT=Property TaxMy_Application=My ApplicationsmColl=mCollectPT_Property={Enter PT_Sub property ex:Vacant Land}wrongpin={Enter Wrong Pincode ex:400013 }pincode={Enter correct pincode ex:143001}pincode_2={Enter other pincode ex:143003}city={Enter city name ex:Amritsar}locality={Enter Localtiy ex: Ajit Nagar - Area1}googleMap={Enter google map location ex:Ajit Nagar Road, Sultanwind Gate}file={enter file name ex:blankpage.jpeg}ValidMobile={Enter valid number ex:9999999999}InvalidNumber={Enter Invalid number Application number eg:PB-PGR-2021-08-20-002211}InvalidMobile={Enter Invalid mobile number ex:9869313102}CompaintType={Enter complaint type ex:Streetlights}streetlight_subtype={Enter Streetlight Subtype ex:No streetlight}Mohalla={Enter Mohalla name related pin ex:Main Road Abadpura}Mohalla_2={Enter Mohlla name related pin ex:Preet Nagar}Property_Residential=ResidentialResidentialSubType={Enter Residential subtype ex:Independent house}Property_Commercial=CommercialCommercialSubType={Enter commercial subtype ex:Hotel}Property_Institutional=InstitutionalInstitutionalSubType={Enter Institutional Subtype ex:Temple}Slum_Area={Enter Slum Area related to pincode ex:K.K sahi}Slum_Area2={Enter Slum Area related to pincode ex:Gangadhar Sahi}Sanitation_Type1={Sanitattion Type ex:Conventional septic tank}Sanitation_Type2={{Sanitattion Type ex:Septic tank with soak pit}Vehicle_Type={Vehicle Type ex:Mahindra - Bolero Pickup - 1000 Ltrs}Vehicle_Type2={Vehicle Type ex:Tata - 407 - 3000 Ltrs}cardnumber={Enter Credit card number}cardmonth={Enter Credit card month ex:12}cardyear={Enter Credit card year ex:25}dsonumber={Enter DSO Number ex:8919146216}reason1={Enter DSO reason 1 ex: Vehicle is under repair}reason2={Enter DSO reason 2 ex:Inaccessible road}adminReason1={Enter Admin reason 1 ex:Not able to contact applicant}adminReason2={Enter DSO reason 2 ex:Others}adminReason3={Enter DSO reason 3 ex:Duplicate Application }status1={Enter DSO Status 1 ex:Pending for DSO Approval }status2={Enter DSO Status 2 ex:Pending for DSO Assignment}status3={Enter DSO Status 3 ex:DSO Rejected }status4={Enter DSO Status 4 ex:DSO InProgress }serviceCategory={Enter Service Category ex:Advertisement Fee (OMD)}serviceType={Enter Service Type ex:Hoardings}mcollectStatus1={Enter mCollect Status 1 ex:Paid}mcollectStatus2={Enter mCollect Status 2 ex:Active}mcollectStatus3={Enter mCollect Status 3 ex:Cancelled}ownership={Enter Ownershipe Type ex:Single Owner}specialOwner={Enter Specail Owner ex:General ID}Field_Inspectors={Enter Field Inspectors ex:test}Approver={Enter Field Approver ex:test}Financial_Years={Enter Financial Years ex:2020-21}

credentials.properties

GRO={EnterGRO USER NAME }LME={EnterLME USER NAME }CSR={EnterCSR USER NAME }CE={EnterCounter Emp USER NAME }COLL={EnterCollector USER NAME } EE={EnterEmp Editor USER NAME }FSTPO={EnterFSTPO USER NAME }CC={EnterCreator & Collector USER NAME }CEC={EnterCreator,Editor & Collector USER NAME }ADMIN={EnterAdmin USER NAME }S1={EnterMcollect CE USER NAME }DV={EnterDocument Verifier USER NAME }FI={Enter Field Inspector USER NAME }AP={Enter Application Approver USER NAME}PTCE={Enter PT Counter EMP USER NAME}pin={Enter Pin ex:12345}password={Enter Employee Password ex:12345}

Sample state-wise demo project is in the following Github location for your reference - https://github.com/egovernments/ui-statelevel-implementation

State-Level UI Customization

Overview

This page offers step-wise details to test the implementation of state-level UI customization.

Steps

Create a new maven project for State-level UI automation for the respective state.

Create a new repository in GitHub for the project and push the code to GitHub.

Keep the following settings.xml file under the .m2 folder in your system. Update the username and personal access token in the file accordingly.

<profiles>
<profile>
<id>github</id>
<repositories>
<repository>
<id>central</id>
<url>https://repo1.maven.org/maven2</url>
</repository>
<repository>
<id>github</id>
<url>https://maven.pkg.github.com/egovernments/ui-statelevel-automation</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
</profile>
</profiles>
<servers>
<server>
<id>github</id>
<username>USERNAME</username>
<password>PERSONAL ACCESS TOKEN</password>
</server>

Add the dependency of the core UI automation repo in the pom.xml of the newly created maven project. This will download the core UI automation jar from the central maven repository.

<dependency>
<groupId>stateeGovernanceUI</groupId>
<artifactId>satateegovui</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>

Note: Version may differ based on the latest version.

Add the state-level tests in this new maven project and use the already existing step definitions and utilities from the core UI automation repo.

Add feature files under the package: src/test/java/

Add step definitions under the package: src/main/java/stepDefinition

Add test data in the following folder: src/test/java/TestData

Add testing.xml with the test which is needed to run

To run the tests, execute the following maven command:

$ mvn test -DdataPath=pathToDataPropertiesFile -DcredsPath=pathToCredentialsPropertiesFile

data.properties

credentials.properties

Sample state-wise demo project is in the following Github location for your reference - https://github.com/egovernments/ui-statelevel-implementation