We are using re-indexing to get all the data to the respective indexer. We have 2 steps for this. The first is to run the connector from the playground. This is followed by a legacy indexer service call from the indexer service, which internally calls the respective plain search service to get the data and send it to the respective indexer.
Access to kubectl of the environment targetted
Postman scripts
Plain search APIs in the respective services
We have two services for water and sewerage
water-services
sewerage-services
Connect to playground pod.
Delete the Kafka connector if already exists with the Kafka connection, using the below command through playground pod.
Run below Kafka connector curl from playground pod:
port forward to egov-indexer pod and run below curl throw postman.
Delete the Kafka connection after all the data has been re-indexed by following the below command through the playground pod.
Alias water-services-enriched as water-services through the Kibana server.
Connect to playground pod.
Delete the Kafka connector if already exists with the Kafka connection, using the below command through playground pod.
Run below Kafka connector curl from playground pod:
port forward to egov-indexer pod and run below curl throw postman.
Delete the Kafka connection after all the data has been re-indexed by following the below command through the playground pod.
Alias water-services-enriched as water-services through the Kibana server.
Water service is the eGov application, which helps and gives flexibility to municipalities and citizens to handle the water service like applying for the water connection and searching for water connections. The application goes through different stages that the states identify for given roles. If that role takes the action then the application goes to the next state. Based on the state, a citizen gets a notification (SMS and in-app ). Citizens also can pay for application fees or employees can collect the fee for the application.
Before you proceed with the documentation, make sure the following pre-requisites are met -
Java 8
The Kafka server is up and running
egov-persister service is running and has a water service persister config path added to it
PSQL server is running and a database is created to store water connection/application data
knowledge of eGov-mdms service, eGov-persister, eGov-idgen, eGov-sms, eGov-email,eGov-user, eGov-localization, eGov-workflow-service will be helpful.
Apply for a water connection.
Searching for water connections.
Apply for property creation.
Can take different actions based on state (Workflow)
Notification based on the application state.
egov.waterservice.createwaterconnection
This variable contains the kafka topic name which is used to create new water connection application in the system.
egov.waterservice.updatewaterconnection
This variable contains the kafka topic name which is used to update the existing water connection application in the system.
egov.waterservice.updatewaterconnection.workflow.topic
This variable contains the kafka topic name which is used to update the process instance of the water connection application.
egov.idgen.wcapid.name
This variable contain the idgen format name for water application
egov.idgen.wcapid.format
This variable contain the idgen format for water application ex:- WS/[CITY.CODE]/[fy:yyyy-yy]/[SEQ_EGOV_COMMON]
egov.idgen.wcid.name
This variable contain the idgen format name for water connection
egov.idgen.wcid.format
This variable contain the idgen format for water connection ex:- WS_AP/[CITY.CODE]/[fy:yyyy-yy]/[SEQ_EGOV_COMMON]
To Do
Table UML diagram
master-config.json for water service
Property creation through WNS module https://github.com/egovernments/egov-mdms-data/blob/DEV/data/pb/PropertyTax/PTWorkflow.json
https://github.com/egovernments/configs/blob/DEV/egov-persister/water-persist.yml https://github.com/egovernments/configs/blob/master/egov-persister/water-meter.yml
Actions
Role Action Mapping
Create businessService (workflow configuration) using the __/businessservice/_create. Following is the product configuration for the water service.
Workflow for property creation through Water and Sewerage module
The indexer provides the facility for indexing the data to elastic search. https://github.com/egovernments/configs/blob/DEV/egov-indexer/water-service.ymlhttps://github.com/egovernments/configs/blob/DEV/egov-indexer/water-services-meter.yml
Setup
Write the configuration for the water service. https://github.com/egovernments/configs/blob/DEV/egov-indexer/water-service.ymlhttps://github.com/egovernments/configs/blob/DEV/egov-indexer/water-services-meter.yml
Provide the absolute path of the checked-in file to DevOps, to add it to the file-read path of egov-indexer. The file will be added to the egov-indexer's environment manifest file for it to be read at the start-up of the application.
Run the egov-indexer app, Since it is a consumer, it starts listening to the configured topics and indexes the data.
Modify connection
After connection activation or legacy connection, we can edit the connection. This process is based on a defined workflow. Any action is based on defined roles on the action level. For the edit connection, we need to upload some supporting documents and mandatory info.
Workflow config for edit connection
Create businessService (workflow configuration) using the __/businessservice/_create. Following is the product configuration for editing the water connection.
Notification
Notification will be sent to the property owners and connection holders based on different application states.
Capturing connection holders
We can add connection holders to the water connection who will be the owner of the connection. We can fill in the connection holders' details or we can just make the property owner to the connector holder.
The connection holder will get a notification based on a different state of the application. We are pushing the data of the connection holders in the user service too.
Multiple Road Type Support We can add road-cutting details of multiple roads to the water connection. For each road that goes undercutting process, we have to fill in their road type details and road cutting area. Based on this information, the application one-time fee estimate is calculated.
Add MDMS configs required for water connection registration and restart MDMS service.
Deploy the latest version of ws-services service.
Add water-service and water-services-meter persister yaml path in persister configuration and restart persister service.
Add role-action mapping for APIs.
Create businessService (workflow configuration) accordingly for trade water connection, modify water connection
Add ws-service indexer yaml path in indexer service configuration and restart indexer service.
This ws-service module is used to manage water service connections against a property in the system.
Provide backend support for the different water connection registration processes.
Mseva and SMS notifications on application status changes.
Elastic search index for creating visualizations and Dashboards.
Supports workflow which is configurable
To integrate, a host of ws-service modules should be overwritten in the helm chart.
/ws-services/wc/_create should be added as the create endpoint for creating water application/connection in the system
/ws-services/wc/_search should be added as the search endpoint. This method handles all requests to search existing records depending on different search criteria
/ws-services/wc/_update should be added as the update endpoint. This method is used to update fields in existing records or to update the status of the application based on workflow.
API Swagger Documentation
Water Calculator Service
/ws-services/wc/_create
/ws-services/wc/_update
/ws-services/wc/_search
(Note: All the APIs are in the same postman collection therefore the same link is added in each row)
Sewerage Calculator Service is one of the major business logic services which is used for calculation of sewerage charge, generating demand, update existing demand, SMS & email notification to the ULB officials on-demand generation and also triggering demands(job scheduler) at some intervals and estimation of sewerage charge(one-time cost) which involves cost like road-cutting charge, form fee, scrutiny fee etc.
Before you proceed with the documentation, make sure the following pre-requisites are met -
Java 8
Kafka server is up and running
egov-persister service is running and has sewerage service persister configs path added in it
PSQL server is running and a database is created to store sewerage connection / application data
Following services should be up and running:
egov-perister
egov-mdms
sw-services
billing-service
Sewerage calculator services present in municipal services provide multiple functionalities like calculating sewerage charges, generating demands for a particular sewerage connection, updating demands, SMS & email notification to the ULB officials on-demand generation and also triggering demands(job scheduler) at some intervals and estimation of sewerage charge(one-time cost) which involves cost like road-cutting charge, form fee, scrutiny fee etc. The different functionalities provided by sewerage calculator services are:
Sewerage charge calculation
Demand generation(here as its always non-metered demand will be generated based on time period)
Sewerage charge estimation (one-time cost which involves cost like road-cutting charge, form fee, scrutiny fee etc.)
Deploy the latest version of sw-service and sw-calculator
Add sewerage-persist.yml file in config folder in git and add that path in persister . (The file path is to be added in environment yaml file in param called persist-yml-path )
Criteria :
connection type
building type
calculation attribute
property usage type
If all criteria will match for that sewerage connection this slab will use for calculation.
For application one-time fee, the estimation will return all the related tax head based on criteria. For estimation, all configuration is present in sw-services-calculation.
All the above master configuration is used for estimation.
Following are the exemptions and taxes that are calculated:
Form fee
Scrutiny fee
Other charges
Road cutting charges
One time fee
Security charges
Tax and cess
Sewerage charge is based on billing slab, for sewerage application charge will be based on slab and tax based on master configuration.
Interest
Below is a sample of master data JSON for interest :
Penalty
Below is a sample of master data JSON for penalty**:**
Round Off
If the fraction is greater than equal to 0.5 the number is round up else it’s round down. eg: 100.4 will be rounded to 100 while 100.6 will be rounded to 101.
Role Action Mapping
Once sewerage is sent to the calculator, its tax estimates are calculated. Using the tax head estimates demand details are created. For every tax head, the estimated demand generates function will create a corresponding demand detail.
Whenever _calculate API is called demand is first searched based on the connection no or application no and the demand from and to period. If demand already exists the same demand is updated else new demand is generated with consumer code as connection no or application no and demand from and to a period equal to financial year start and end period.
In the case of an update, if the tax head estimates change, the difference in amount for that tax head is added as new demand detail. For example, if the initial demand has one demand detail with SEWERAGE_CHARGE equal to 120
After updating if the SEWERAGE_CHARGE increases to 150 we add one more demand detail to account for the increased amount. The demand detail will be updated to:
RoundOff is bill based i.e every time bill is generated round off is adjusted so that the payable amount is the whole number. Individual SW_ROUNDOFF in demand detail can be greater than 0.5 but the sum of all SW_ROUNDOFF will always be less than 0.5.
For generating the demand for non metered connection we have a feature for generating the demand in batch. The scheduler is responsible for generating the demand based on the tenant.
The scheduler can be hit by scheduler API or we can schedule cron job or we can put config to kubectl which will hit scheduler based on config.
After the scheduler been hit we will search the list of the tenant (city) present in the database.
After getting the tenants we will pick up tenant one by one and generate the demand for that tenant.
We will load the consumer codes for the tenant and push the calculation criteria to Kafka. Calculation criteria contain minimal information (We are not pushing large data to Kafka), calculation criteria contain consumer code and one boolean variable.
After pushing the data into Kafka we are consuming the records based on the batch configuration. Ex:-> if the batch configuration is 50 so we will consume the 50 calculation criteria at a time.
After consuming the record(Calculation criteria) we will process the batch for generating the demand. If the batch is successful so will log the consumer codes which have been processed.
If some records failed in batch so we will push the batch into dead letter batch topic. From the dead letter batch topic, we will process the batch one by one.
If the record is successful we will log the consumer code, If the record is failed so we will push the data into a dead letter single topic.
Dead letter single topic contains information about failure records in Kafka.
Use cases
If the same job trigger multiple time what will happen?
If the same job triggers multiple times we will process again as mentioned above but at the demand level we will check the demand based on consumer code and billing period, If demand already exists then we will update the demand otherwise we will create the demand.
Are we maintaining success or failure status anywhere?
Currently, we are maintaining the status of failed records in Kafka.
Configuration
We need to configure the batch size for Kafka consumer. This configuration is for how much data will be processed at a time.1sw.demand.based.batch.size=10
sw-calculator will be integrated with sw-service. sw-services internally invoke the sw-calculator service to calculate and generate demand for the charges.
SW calculator application is used to calculate the sewerage application one time Fees and connection charges based on the different billing slabs that's why the calculation and demand generation logic will be separated out from SW service. So in future, if calculation logic needs to modify then changes can be carried out for each implementation without modifying the SW service.
To Activate the Sewerage Service application, the user needs to pay the ONE_TIME_FEE for the connection. To calculate the ONE_TIME_FEE sw-calculator/sewerageCalculator/_estimate API is use.
To generate the demand for non-metered sewerage connection /sw-calculator/sewerageCalculator/_calculate API is use.
User can pay partial / full / advance amount for the Non-Metered connection bill. In these cases, Billing service would call back /sw-calculator/sewerageCalculator/_updateDemandAPI to update the details of the demand generated.
/sw-calculator/sewerageCalculator/_jobscheduler API is use to generate demand for Non-metered connections. This API can be called periodically.
/sw-calculator/sewerageCalculator/_applyAdhocTax API is used to add Rebate or Penalty on any bill and based on that the bill amount will be adjusted.
(Note: All the API’s are in the same postman collection therefore same link is added in each row)
__
__
Put the indexer config file into the config repo under egov-indexer folder. (egovernments/configs )
__All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.
The combination of the above can be used to define the billing slab. Billing Slab is defined in MDMS under sw-services-calculation folder with the . The following is the sample slab.
All content on this page by is licensed under a .
Title
Link
/sw-calculator/sewerageCalculatorr/_estimate
/sw-calculator/sewerageCalculator/_calculate
/sw-calculator/sewerageCalculator/_updateDemand
/sw-calculator/sewerageCalculator/_jobscheduler
/sw-calculator/sewerageCalculator/_applyAdhocTax
Title
Link
API Swagger Contract
Sewerage Service Document
Water Calculator Service is used for creating meter reading, searching meter reading, updating existing meter reading, calculation of water charge, demand generation, SMS & email notification to ULB officials on-demand generation and estimation of water charge(one-time cost) which involves cost like road-cutting charge, form fee, scrutiny fee, etc.
Before you proceed with the documentation, make sure the following pre-requisites are met -
Java 8
Kafka server is up and running
egov-persister service is running and has water service persister configs path added in it
PSQL server is running and a database is created to store water connection / application data
Following services should be up and running:
egov-perister
egov-mdms
ws-services
billing-service
Calculate water charge and taxes based on billing slab.
Calculate meter reading charge for water connection
Generate demand
Scheduler for generating the demand(for non metered connection)
Deploy the latest version of ws-service and ws-calculator
Add water-persist.yml & water-meter.yml file in config folder in git and add that path in persister . (The file path is to be added in environment yaml file in param called persist-yml-path )
Criteria :
connection type
building type
calculation attribute
property usage type
The combination of the above can be used to define the billing slab. Billing Slab is defined in MDMS under ws-services-calculation folder with the WCBillingSlab. The following is the sample slab.
If all criteria will match for that water connection this slab will use for calculation.
For application one-time fee, the estimation will return all the related tax head based on criteria. For estimation, all configuration is present in ws-services-calculation.
All the above master configuration is used for estimation.
Following are the exemptions and taxes that are calculated:
Form fee
Scrutiny fee
Meter charge (For metered connection)
Other charges
Road cutting charges
One time fee
Security charges
Tax and cess
Water charge is based on billing slab, for water application charge will be based on slab and tax based on master configuration.
Interest
Below is a sample of master data JSON for interest :
Penalty
Below is a sample of master data JSON for penalty :
Round Off
If the fraction is greater than equal to 0.5 the number is round up else it’s round down. eg: 100.4 will be rounded to 100 while 100.6 will be rounded to 101.
Actions
Role Action Mapping
Once water is sent to the calculator, its tax estimates are calculated. Using these tax head estimates demand details are created. For every tax head, the estimated demand generates function will create a corresponding demand detail.
Whenever _calculate API is called demand is first searched based on the connection no or application no and the demand from and to period. If demand already exists the same demand is updated else new demand is generated with consumer code as connection no or application no and demand from and to a period equal to financial year start and end period.
In the case of an update, if the tax head estimates change, the difference in amount for that tax head is added as new demand detail. For example, if the initial demand has one demand detail with WATER_CHARGE equal to 120
After updating if the WATER_CHARGE increases to 150 we add one more demand detail to account for the increased amount. The demand detail will be updated to:
RoundOff is bill based i.e every time bill is generated round off is adjusted so that the payable amount is the whole number. Individual WS_ROUNDOFF in demand detail can be greater than 0.5 but the sum of all WS_ROUNDOFF will always be less than 0.5.
Description :
For generating the demand for non metered connection we have a feature for generating the demand in batch. The scheduler is responsible for generating the demand based on the tenant.
The scheduler can be hit by scheduler API or we can schedule cron job or we can put config to kubectl which will hit scheduler based on config.
After the scheduler been hit we will search the list of the tenant (city) present in the database.
After getting the tenants we will pick up tenant one by one and generate the demand for that tenant.
We will load the consumer codes for the tenant and push the calculation criteria to Kafka. Calculation criteria contain minimal information (We are not pushing large data to Kafka), calculation criteria contain consumer code and one boolean variable.
After pushing the data into Kafka we are consuming the records based on the batch configuration. Ex:-> if the batch configuration is 50 so we will consume the 50 calculation criteria at a time.
After consuming the record(Calculation criteria) we will process the batch for generating the demand. If the batch is successful so will log the consumer codes which have been processed.
If some records failed in batch so we will push the batch into dead letter batch topic. From the dead letter batch topic, we will process the batch one by one.
If the record is successful we will log the consumer code, If the record is failed so we will push the data into a dead letter single topic.
Dead letter single topic contains information about failure records in Kafka.
Use cases:
If the same job trigger multiple time what will happen?
If the same job triggers multiple times we will process again as mentioned above but at the demand level we will check the demand based on consumer code and billing period, If demand already exists then we will update the demand otherwise we will create the demand.
Are we maintaining success or failure status anywhere?
Currently, we are maintaining the status of failed records in Kafka.
Configuration :
We need to configure the batch size for Kafka consumer. This configuration is for how much data will be processed at a time.1ws.demand.based.batch.size=10
ws-calculator will be integrated with ws-service. ws-services internally invoke the ws-calculator service to calculate and generate demand for the charges.
WS calculator application is used to calculate the water application one time Fees and meter reading charges based on the different billing slabs that's why the calculation and demand generation logic will be separated out from the WS service. So in future, if calculation logic needs to modify then changes can be carried out for each implementation without modifying the WS service.
Once the water connection is activated for metered-connection, employee can add meter reading details using this API - /ws-calculator/meterConnection/_createwhich in-turn will generate the demand. For the Non-Metered connections, the scheduler APIs need to be called periodically to generate the demand.
For the Metered Connection service, to get the previous meter reading /meterConnection/_search API is used.
To Activate the Water Service application, the user needs to pay the ONE_TIME_FEE for the connection. To calculate the ONE_TIME_FEE /waterCalculator/_estimate API is used.
To generate the demand for metered or non-metered water connection /waterCalculator/_calculate API is used.
User can pay partial / full / advance amount for the Metered or Non-Metered connection bill. In these cases, Billing service would call back /waterCalculator/_updateDemandAPI to update the details of the demand generated.
/waterCalculator/_jobscheduler API is use to generate demand for Non-metered connections. This API can be called periodically.
/waterCalculator/_applyAdhocTax API is used to add Rebate or Penalty on any bill and based on that the bill amount will be adjusted.
Title
Link
API Swagger Contract
Water Service Document
Title
Link
/ws-calculator/meterConnection/_create
/ws-calculator/meterConnection/_search
/waterCalculator/_estimate
/waterCalculator/_calculate
/waterCalculator/_updateDemand
/waterCalculator/_jobscheduler
/waterCalculator/_applyAdhocTax
(Note: All the API’s are in the same postman collection therefore same link is added in each row)
__
__
This is one of the major application of the egov stack which helps municipal and citizens to handle sewerage services like creating sewerage connection, searching sewerage connection, sewerage connection, also creating property if it doesn't exist and creating and updating workflow were depending on different roles of the logged-in user he/she can perform various actions like editing or perform document verification and many more specified in workflow documentation and SMS & email notification sent to the owner for various actions performed in the workflow.
Before you proceed with the documentation, make sure the following pre-requisites are met -
Java 8
Kafka server is up and running
egov-persister service is running and has sewerage service persister config path added in it
PSQL server is running and a database is created to store sewerage connection / application data
knowledge of eGov-mdms service, eGov-persister, eGov-idgen, eGov-sms, eGov-email,eGov-user, eGov-localization, eGov-workflow-service will be helpful.
The sewerage service provides multiple functionalities starting from serving as a central repository where one can create sewerage connection, update sewerage connection, search for a particular sewerage connection based on different criteria like mobile number, sewerage connection nos etc. and also creating property if it doesn't exist, creating and updating workflow were depending on different roles of the logged-in user he/she can perform various actions like editing or perform document verification and many more specified in workflow documentation and SMS & email notification sent to the owner for various actions performed in the workflow. The different services provided by the sewerage services are
Creating sewerage connection
Updating sewerage connection
Apply for Property creation.
Searching sewerage connection based on a few criteria mentioned in swagger documentation later
Creating property if it doesn't exist
Creating and updating workflow.
To Do
Table UML Diagram
****
MDMS configuration
master-config.json for sewerage service
Actions & Role Action Mapping
Actions
Role Action Mapping
Workflow business service config
Create businessService (workflow configuration) using the __/businessservice/_create. Following is the product configuration for sewerage service
Workflow for property creation through Water and Sewerage Module
Indexer config for sewerage-service
Setup
Provide the absolute path of the checked-in file to DevOps, to add it to the file-read path of egov-indexer. The file will be added to the egov-indexer's environment manifest file for it to be read at the start-up of the application.
Run the egov-indexer app, Since it is a consumer, it starts listening to the configured topics and indexes the data.
Modify connection
After connection activation or legacy connection, we can edit the connection. This process based on a defined workflow. Any action is based on defined roles on the action level. For edit connection, we need to upload some supporting documents and mandatory info.
Workflow config for edit connection
Create businessService (workflow configuration) using the __/businessservice/_create. Following is the product configuration for editing sewerage connection.
1{ 2 "RequestInfo": { 3 "apiId": "Rainmaker", 4 "action": "", 5 "did": 1, 6 "key": "", 7 "msgId": "20170310130900|en_IN", 8 "requesterId": "", 9 "ts": 1513579888683, 10 "ver": ".01", 11 "authToken": "{{Auth_Token}}" 12 }, 13 "BusinessServices": [ 14 { 15 "tenantId": "pb", 16 "businessService": "ModifySWConnection", 17 "business": "sw-services", 18 "businessServiceSla": 259200000, 19 "states": [ 20 { 21 "sla": null, 22 "state": null, 23 "applicationStatus": null, 24 "docUploadRequired": false, 25 "isStartState": true, 26 "isTerminateState": false, 27 "isStateUpdatable": false, 28 "actions": [ 29 { 30 "action": "INITIATE", 31 "nextState": "INITIATED", 32 "roles": [ 33 "SW_CEMP" 34 ] 35 } 36 ] 37 }, 38 { 39 "sla": null, 40 "state": "INITIATED", 41 "applicationStatus": "INITIATED", 42 "docUploadRequired": false, 43 "isStartState": false, 44 "isTerminateState": false, 45 "isStateUpdatable": true, 46 "actions": [ 47 { 48 "action": "SUBMIT_APPLICATION", 49 "nextState": "PENDING_FOR_APPROVAL", 50 "roles": [ 51 "SW_CEMP" 52 ] 53 } 54 ] 55 }, 56 { 57 "sla": 86400000, 58 "state": "PENDING_FOR_APPROVAL", 59 "applicationStatus": "PENDING_FOR_APPROVAL", 60 "docUploadRequired": false, 61 "isStartState": false, 62 "isStateUpdatable": true, 63 "isTerminateState": false, 64 "actions": [ 65 { 66 "action": "APPROVE_CONNECTION", 67 "nextState": "APPROVED", 68 "roles": [ 69 "SW_APPROVER" 70 ] 71 }, 72 { 73 "action": "REJECT", 74 "nextState": "REJECTED", 75 "roles": [ 76 "SW_APPROVER" 77 ] 78 }, 79 { 80 "action": "SEND_BACK", 81 "nextState": "INITIATED", 82 "roles": [ 83 "SW_APPROVER" 84 ] 85 } 86 ] 87 }, 88 { 89 "sla": null, 90 "state": "REJECTED", 91 "applicationStatus": "REJECTED", 92 "isStateUpdatable": false, 93 "docUploadRequired": false, 94 "isStartState": false, 95 "isTerminateState": true 96 }, 97 { 98 "sla": null, 99 "state": "APPROVED", 100 "applicationStatus": "APPROVED", 101 "isStateUpdatable": false, 102 "docUploadRequired": false, 103 "isStartState": false, 104 "isTerminateState": true 105 } 106 ] 107 } 108 ] 109} 110
Notification
Notification will be sent to the property owners and connection holders based on different application states.
Capturing connection holders
We can add connection holders to the sewerage connection which will be the owner of the connection. We can fill in the connection holders' details or we can just make the property owner to the connection holder.
The connection holder will get a notification based on a different state of the application. We are pushing the data of the connection holders in the user service too.
Multiple Road Type Support We can add road cutting details of multiple roads to the sewerage connection. For each road which goes under cutting process we have to fill their road type details and road cutting area. Based on this information, application one time fee estimate is calculated.
This sw-service module is use to manage sewerage service connections against a property in the system.
Provide backend support for the different sewerage connection registration process.
Mseva and SMS notifications on application status changes.
Elastic search index for creating visualisations and Dashboards.
Supports workflow which is configurable
To integrate, the host of sw-service module should be overwritten in helm chart.
/sw-services/swc/_create should be added as the create endpoint for creating sewerage application/connection in the system
/sw-services/swc/_search should be added as the search endpoint .This method handles all requests to search existing records depending on different search criteria
/sw-services/swc/_update should be added as the update endpoint. This method is used to update fields in existing records or to update the status of the application based on workflow.
(Note: All the API’s are in the same postman collection therefore same link is added in each row)
__
__
Technical Doc
DSS has two sides to it. One is the process in which the data is pooled into ElasticSearch and the other is the way it is fetched, aggregated, computed, transformed and sent across.
As this revolves around a variety of data sets, there is a need for making this configurable. So that, tomorrow, given a new scenario is introduced, then it is just a configuration away from getting the newly introduced scenario involved in this process flow.
This document explains the steps on how to define the configurations for the analytics side of DSS for W&S.
What is analytics?
Analytics: Micro Service which is responsible for building, fetching, aggregating and computing the data on ElasticSearch to a consumable data response. This is later used for visualizations and graphical representations.
Analytics Configurations: Analytics contains multiple configurations. We need to add the changes related to Water & Sewerage(W&S) in this dashboard analytics. Here is the location: configs/egov-dss-dashboards/dashboard-analytics at qa · egovernments/configs Below is a list of configurations that need to be changed to run W&S successfully.
Chart API Configuration
Master Dashboard Configuration
Role Dashboard Mappings Configuration
Each visualization has its own properties. Each visualization comes from different data sources (Sometimes it is a combination of different data sources). In order to configure each visualization and its properties, we have a chart API configuration document.
In this, visualization code, which happens to be the key, will be having its properties configured as a part of the configuration and are easily changeable.
Here is the sample ChartApiConfiguration.json data for the W&S.
Click here to check the complete configuration
Master dashboard configuration is the main configuration which defines the dashboards to be painted on the screen.
It includes all the visualizations, their groups, the charts which come within them and even their dimensions as what should be their height and width.
Click here for the complete configuration
Master Dashboard Configuration which was explained earlier contains the list of dashboards which are available. Given the instance where Role Action Mapping is not maintained in the Application Service, this configuration will act as Role - Dashboard Mapping Configuration.
In this, each role is mapped against the Dashboard which they are authorised to see. This was used earlier when the Role Action Mapping of eGov was not integrated.
Later, when the Role Action Mapping started controlling the dashboards to be seen on the client side, this configuration was just used to enable the dashboards for viewing.
Click here to check the configuration
common-masters/uiCommonConstants.json
Click here to check the complete configuration
roleaction.json
Click here to check the complete configuration
Action test.json
Click here to check the complete configuration
W&S-DSS consists of multiple graphs which represent the data of W&S. Each graph has its own configuration which will describe the chart and its type.
DSS consists of the following charts in W&S:
Revenue Tab
Overview
Total Cumulative Collection
Top ULB By Performance
Bottom ULB by Performance
W&S Collection by Usage Type
W&S Collection by Channel Type
W&S Key Financial Indicators
W&S Tax Heads
Service Tab
Overview
Total Cumulative Connection
Water Connection By Usage
Sewerage Connections By Usage
Connection By Channel
W&S Connection Ageing
Overview
The overview graph contains multiple data information as below in the selected time period.
Today’s Collection: This represents today’s collection amount for Water and Sewerage Application Fee + Consumption Charges.
Total Collection: This represents the total collection amount for Water and Sewerage Application Fee + Consumption Charges.
Target Collection: This represents the target collection amount for the Water and Sewerage connections.
Target Achievement: This represents the target achieved in percentage. This is calculated by the formula- (Total Collection/Target Collection)*100%
This Graph contains the Water and Sewerage collection amount information on a monthly basis as a cumulative line graph for each water and sewerage collection separately. This will change as per the denomination amount filter selection.
line - this graph/chart is data representation on date histograms or date groupings.
This graph represents the ULBs based on the Target achieved in bar chart representation with the % of target achieved in descending order.
This graph represents the ULBs based on the target achieved in bar chart representation with the % of target achieved in ascending order.
W&S Collection by Usage Type
This graph shows the collection amount based on the usage/property type and this amount will change as per the denomination filter change.
This graph shows the collection amount based on the collection channel type(eg-online, counter, etc) and this amount will change as per the denomination filter change.
This tabular chart representation graph shows multiple W&S information like Target Collection, Total Collection, Total Transactions, Total Connections and Target Achievement (in %). And this table shows the data at the district level and also has the drill-down chart for each district to ulb and from ulb to ward level data for the same.
xtable type allows to the addition of multiple computed fields with the aggregated fields dynamically added.
To add multiple computed columns, computedFields [] where actionName (IComputedField<T> interface), fields [] names as in exist in query key, newField as name to appear for computation must be defined.
On click of any district name will enter into drill-down charts, which will represent that specific District data.
On click of the ULB will navigate to wards under that specific ULB and each ward shows the specific data regarding that ward.
This tabular chart representation graph shows multiple W&S information like Total Collection, Penalty, Interest and Target Collection. And this table shows the data at the district level and also has the drill-down chart for each district to ulb and from ulb to ward level data for the same.
table type allows to addition of multiple aggregated fields.
On click of any district name will enter into drill-down charts, which will represent that specific District data.
On click of the ULB will navigate to wards under that specific ULB and each ward shows the specific data regarding that ward.
Overview
The overview graph contains multiple data information as below in the selected time period.
Total Applications: This represents the total number of applications for new, modify, and disconnection requests.
SLA Compliance: This represents the total SLA achieved in percentage.
Total Active Connections: This represents the total active water and sewerage connections.
Water-Metered Connections: This represents the total active metered water connections.
Water-Non Metered Connections: This represents the total active non-metered water connections.
Sewerage Connections: This represents the total active sewerage connections.
Total Cumulative Connections
This Graph contains the Water and Sewerage connections information on a monthly basis as a cumulative line graph for each water and sewerage connection separately.
line - this graph/chart is data representation on date histograms or date groupings.
This graph shows the active Water connections based on the usage/property type.
Sewerage Connections by Usage Type
This graph shows the active Sewerage connections based on the usage/property type.
This graph shows the active water and sewerage connections by channel type(eg-online, counter, etc).
W&S Connection Ageing
This tabular chart representation graph shows W&S applications that have been submitted and have been pending to be worked on. Information like Pending from 0 to 3 days, Pending from 3 to 7 days, Pending from 7 to 15 days, Pending for more than 15 days, and Total Pending Applications is shown in the chart. And this table shows the data at the district level and also has the drill-down chart for each district to ulb and from ulb to ward level data for the same.
xtable type allows the addition of multiple computed fields with the aggregated fields dynamically added.
To add multiple computed columns, computedFields [] where actionName (IComputedField<T> interface), fields [] names as in exist in query key, newField as name to appear for computation must be defined.
On click of any district name will enter into drill-down charts, which will represent that specific District data.
On click of the ULB will navigate to wards under that specific ULB and each ward shows the specific data regarding that ward.
isRoundOff: This property is introduced to round off the decimal values. Ex: if the value is 25.43 by using isRoundOff property in the configuration we will get it as 25. if the value is 22.56 round of value will be 23. This can be used for insights configuration as well for overview graphs.
Key(eg: fsmTotalrequest): This is the Visualization Code. This key will be referred to in further visualization configurations.
This is the key which will be used by the client application to indicate which visualization is needed for display.
chartName: The name of the Chart which has to be used as a label on the Dashboard. The name of the Chart will be a detailed name.
In this configuration, the Name of the Chart will be the code of Localization which will be used by the Client Side.
queries: Some visualizations are derived from a specific data source. While some others are derived from different data sources and are combined together to get a meaningful representation.
The queries of aggregation which are to be used to fetch the right data in the right aggregated format are configured here.
queries.module: The module/domain level, on which the query should be applied on. Water and sewerage are W&S.
queries.indexName: The name of the index upon which the query has to be executed is configured here.
queries.aggrQuery : The aggregation query in itself is added here. Based on the Module and the Index name specified, this query is attached to the filter part of the complete search request and then executed against that index
queries.requestQueryMap: Client Request would carry certain fields which are to be filtered. The parameters specified in the Client Request are different from the parameters in each of these indexed documents.
In order to map the parameters of the request to the parameters of the ElasticSearch Document, this mapping is maintained.
queries.dateRefField: Each of these modules have separate indexes. And all of them have their own date fields.
When there is a date filter applied against these visualizations, each of them has to apply it against their own date reference fields.
In order to maintain what is the date field in which index, we have this configured in this configuration parameter.
chartType: As there are different types of visualizations, this field defines what is the type of chart/visualization that this data should be used to represent.
Chart types available are:
metric - this represents the aggregated amount/value for records filtered by the aggregate es query
pie - this represents the aggregated data on grouping. This is can be used to represent any line graph, bar graph, pie chart or donut
line - this graph/chart is data representation on date histograms or date groupings
perform - this chart represents groping data as performance-wise.
table - represents a form of plots and values with headers as grouped on and a list of its key, values pairs.
xtable - represents an advanced feature of the table, it has additional capabilities for dynamic adding header values.
valueType: In any case of data, the values which are sent to the plot, might be a percentage, sometimes an amount and sometimes it is just a count.
In order to represent them and differentiate the numbers from the amount from the percentage, this field is used to indicate the type of value that this Visualization will be sending.
action: Some of the visualizations are not just aggregations of data sources. There might be some cases where we have to do a post-aggregation computation.
For Example, in the case of the top 3 Performing ULBs, the Target and Total Collection are obtained and then the percentage is calculated. In these kinds of cases, the action that has to be performed on the data obtained is defined in this parameter.
documentType: The type of document upon which the query has to be executed is defined here.
drillChart: If there is a drill down on the visualization, then the code of the Drill Down Visualization is added here. This will be used by Client Service to manage drill-downs.
aggregationPaths: All the queries will be having Aggregation names in them. In order to fetch the value out of each Aggregation Response, the name of the aggregation in the query will be an easy bet. These aggregation paths will have the names of Aggregation in it.
insights: It is to show the data with the comparison of last year with arrow symbols, it will show the data in how much % is increased or decreased.
_comment: In order to display information on the “i” symbol of each visualization, Visualization Information is maintained in this field.
Postman collection for W&S-dss: https://www.getpostman.com/collections/f346eea97268ac94888b,
__All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.
``
Property creation through WNS module
Persister configuration:
The indexer provides the facility for indexing the data to elastic search.
Write the configuration for sewerage service.
Put indexer config file to the config repo under egov-indexer folder. ( )
All content on this page by is licensed under a .
Environment Variables
Description
egov.sewarageservice.createconnection
This variable contains the kafka topic name which is used to create new sewerage connection application in the system.
egov.sewarageservice.updateconnection
This variable contains the kafka topic name which is used to update the existing v connection application in the system.
egov.sewerageservice.updatesewerageconnection.workflow.topic
This variable contains the kafka topic name which is used to update the process instance of the sewerage connection application.
egov.idgen.scapid.name
This variable contain the idgen format name for sewerage application
egov.idgen.scapid.format
This variable contain the idgen format for sewerage application ex:- WS/[CITY.CODE]/[fy:yyyy-yy]/[SEQ_EGOV_COMMON]
egov.idgen.scid.name
This variable contain the idgen format name for sewerage connection
egov.idgen.scid.format
This variable contain the idgen format for sewerage connection ex:- WS_AP/[CITY.CODE]/[fy:yyyy-yy]/[SEQ_EGOV_COMMON]
Title
Link
API Swagger Documentation
Sewerage Calculator Service
Title
Link
/sw-services/swc/_create
/sw-services/swc/_update
/sw-services/swc/_search
