Before you proceed with the DIGIT Deployment on AWS

Prerequisites

• Install AWS CLI on your local machine so that you can use aws cli commands to provision and manage the cloud resources on your account.

• Install AWS IAM Authenticator that helps you authenticate your connection from your local machine so that you should be able to deploy DIGIT services.

Get AWS access:

When you have the command-line-access configured, everything is set for you to proceed with the terraform to provision the DIGIT Infra-as-code.

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

The Amazon Elastic Kubernetes Service (EKS) is one of the AWS services for deploying, managing and scaling any distributed and containerized workloads, here we can provision the EKS cluster on AWS from the ground up using terraform (infra-as-code) and then deploy the DIGIT platform services as config-as-code using Helm.

Pre-reads

• Know about EKS: https://www.youtube.com/watch?v=SsUnPWp5ilc

• Know what is terraform: https://youtu.be/h970ZBgKINg

Prerequisites

1. AWS account with the admin access to provision EKS Service, you can always subscribe to free AWS account to learn the basics and try, but there is a limit to what is offered as free, for this demo you need to have a commercial subscription to the EKS service, if you want to try out for a day or two, it might cost you about Rs 500 - 1000.

   1. Note: Post the Demo, for the eGov internal folks, you can request for the AWS access for 4 hrs time bound access to eGov's training AWS account upon on the request and available number of slots per day)

2. Install kubectl on your local machine that helps you interact with the kubernetes cluster

3. Install Helm that helps you package the services along with the configurations, envs, secrets, etc into a kubernetes manifests

4. Install terraform version (0.14.10) for the Infra-as-code (IaC) to provision cloud resources as code and with desired resource graph and also it helps to destroy the cluster at one go.

   1. If you already have a different version of the terraform version running install tfswitch that would allow you to have multiple terraform version in the same machine and toggle between the desired versions.

5. Setup AWS Account

​All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

The Amazon Elastic Kubernetes Service (EKS) is one of the AWS services for deploying, managing, and scaling any distributed and containerized workloads, here we can provision the EKS cluster on AWS from ground up and using an automated way (infra-as-code) using terraform and then deploy the DIGIT Services config-as-code using Helm.

Pre-reads

• Know about EKS: https://www.youtube.com/watch?v=SsUnPWp5ilc

• Know what is terraform: https://youtu.be/h970ZBgKINg

Installation Steps

While would have helped you to get your hands dirty and build the Kubernetes cluster on a local/single VM instance, which you can consider for either local development, or to understand the details involved in infra and deployment.

DIGIT is a and open source eGov stack, depending on the scale and performance running DIGIT on production requires advanced capabilities like HA, DRS, autoscaling, resiliency, etc.. most of these capabilities are provided out of the box by the commercial clouds like AWS, Google, Azure, VMware, OpenStack, etc.. and also the private clouds like NIC and few SDCs implemented clouds, all these cloud providers provide the kubernetes-as-a-managed-service that makes the entire infra setup and management seamless and automated, like infra-as-code, config-as-code.

Before we jump into the supported cloud providers, it is important to know DIGIT is completely cloud agnostic, be it a commercial clouds or on premise, the differentiator is just that when the cloud provider provides kubernetes as a managed service or not. In case of managed services like , etc.. we do not need to provision and manage the kubernetes cluster components from the ground up, just the working knowledge of the kubernetes is enough. In the absence of managed kubernetes service, we need to first create the kubernetes cluster itself out of available/required no of VMs and then ensure we manage the cluster apart from running the actual workloads. To get more understanding on kubernetes and managed kubernetes services, please go through the following pre-read.

Pre-reads

• Know the basics of Kubernetes:

• Know the commands

• Know kubernetes manifests:

• Know how to manage env values, secrets of any service deployed in kubernetes

• Know how to port forward to a pod running inside k8s cluster and work locally

• Know sops to secure your keys/creds:

Prerequisites

• Unlike quickstart, full installation requires state/user-specific configurations ready before proceeding with the deployment.

• You need to have the fully qualified DNS (URL) (Should not be dummy)

• Persistent storage depending on the cloud you are using for the Kafka, ES, etc.

• Either a standalone or a hosted PostGres DB above v11.x

• MDMS with the master data like Roles, Access, Actions, tenants, etc. Sample is

• Gov services specific Configs like persister, searcher configs etc. Sample is

• GeoLocation provider configs (Google Location API), SMS Gateway, Payment Gateway, etc.

1. Choose the Cloud

Choose your cloud and follow the Instruction to set up a Kubernetes cluster before moving on to the Deployment.

2. Deploy DIGIT

Post infra setup (Kubernetes Cluster), the deployment involves 2 stages and 2 modes. Check out the stages first and then the modes. As part of a sample exercise, we will deploy the PGR module. However, deployment steps are similar. The prerequisites have to be configured accordingly.

The 2 Stages

• each service global, local env variables

• Number of replicas/scale of individual services (Depending on whether dev or prod)

• mdms, config repos (Master Data, ULB, Tenant details, Users, etc)

• sms g/w, email g/w, payment g/w

• GMap key (In case you are using Google Map services in your PGR, PT, TL, etc)

• S3 Bucket for Filestore

• URL/DNS on which the DIGIT will be exposed

• SSL Certificate for the above URL

• End-points configs (Internal/external)

Stage 2: Run the digit_setup deployment script and simply answer the questions that it asks.

All Done, wait and watch for 10 min, you'll have the DIGIT setup completed and the application will be running on the given URL.

The 2 Modes of Deployment

Essentially, DIGIT deployment means that we need to generate Kubernetes manifests for each individual service. We use the tool called Helm, which is an easy, effective and customizable packaging and deployment solution. So depending on where and which env you initiate the deployment there are 2 modes that you can deploy.

1. From local machine - whatever we are trying in this sample exercise so far.

3. Post Deployment Steps

Post-deployment - the application is now accessible from the configured domain.

To try out PGR employee login - Create a sample tenant, city, user to login and assign LME employee role using the seed script.

• Seed the sample data.

• Import the following postman collection into the postman and run it. This collection has the seed data that enable sample test users and localisation data.

4. Assessment of the DIGIT Deployment

By now we have successfully completed the DIGIT setup on the cloud. Use the URL that you mentioned in your env.yaml Eg: https://mysetup.digit.org and create a grievance to ensure the PGR module deployed is working fine. Refer to the product documentation below for the steps.

Credentials:

1. Citizen: You can use your default mobile number (9999999999) to sign in using the default Mobile OTP 123456.

2. Employee: Username: GRO and password: eGov@4321

Post grievance creation and assignment of the same to LME, capture the screenshot of the same and share it to ensure your setup is working fine.

5. Destroy the Cluster

Post validating the PGR functionality share the API response of the following request to assess the correctness of successful DIGIT PGR Deployment.

Finally, clean up the DIGIT Setup if you wish, using the following command. This will delete the entire cluster and other cloud resources that were provisioned for the DIGIT Setup.

Conclusion

All Done, we have successfully created infra on the cloud, deployed DIGIT, bootstrapped DIGIT, performed a transaction on PGR and finally destroyed the cluster.

Annexures:

• s

All content on this page by is licensed under a .

If you have any question please write to us

Kindly use the appropriate discussion category and labels to address the issues better.

Discussion Board

Terraform is an open source infrastructure as code (IaC) software tool that allows DevOps engineers to programmatically provision the physical resources an application requires to run.

Infrastructure as code is an IT practice that manages an application's underlying IT infrastructure through programming. This approach to resource allocation allows developers to logically manage, monitor and provision resources -- as opposed to requiring that an operations team manually configure each required resource.

Terraform users define and enforce infrastructure configurations by using a JSON-like configuration language called HCL (HashiCorp Configuration Language). HCL's simple syntax makes it easy for DevOps teams to provision and re-provision infrastructure across multiple cloud and on-premises data centers.

Cloud Resources Required For DIGIT

Before we provision the cloud resources, we need to understand and be sure about what resources need to be provisioned by terraform to deploy DIGIT. The following picture shows the various key components. (EKS, Worker Nodes, PostGres DB, EBS Volumes, Load Balancer)

Considering the above deployment architecture, the following is the resource graph that we are going to provision using terraform in a standard way so that every time and for every env, it'll have the same infra.

• EKS Control Plane (Kubernetes Master)

• Work node group (VMs with the estimated number of vCPUs, Memory)

• EBS Volumes (Persistent Volumes)

• RDS (PostGres)

• VPCs (Private network)

• Users to access, deploy and read-only

Prerequisites

1. AWS Account

2. Terraform

3. (Optional) Create your own keybase key before you run the terraform

   • Use this URL https://keybase.io/ to create your own PGP key, this will create both public and private key in your machine, upload the public key into the keybase account that you have just created, and give a name to it and ensure that you mention that in your terraform. This allows to encrypt all the sensitive information.

   • Example user keybase user in eGov case is "egovterraform" needs to be created and has to uploaded his public key here - https://keybase.io/egovterraform/pgp_keys.asc

   • you can use this portal to Decrypt your secret key. To decrypt PGP Message, Upload the PGP Message, PGP Private Key and Passphrase.

4. Clone the DIGIT-DevOps GitHub repo where the terraform sample DIGIT template scripts to provision EKS cluster is available and below is the structure of the files.

Understand Terraform Script

• Ideally, one would write the terraform script from the scratch using this doc.

• Here we have already written the terraform script that one can reuse/leverage that provisions the production-grade DIGIT Infra and can be customized with the user specific configuration.

• Clone the following DIGIT-DevOps where we have all the sample terraform scripts available for you to leverage.

git clone --branch release https://github.com/egovernments/DIGIT-DevOps.git

cd DIGIT-DevOps/infra-as-code/terraform

### You'll see the following file structure 


└── modules
    ├── db
    │   └── aws
    │       ├── main.tf
    │       ├── outputs.tf
    │       └── variables.tf
    ├── kubernetes
    │   └── aws
    │       ├── eks-cluster
    │       │   ├── main.tf
    │       │   ├── outputs.tf
    │       │   └── variables.tf
    │       ├── network
    │       │   ├── main.tf
    │       │   ├── outputs.tf
    │       │   └── variables.tf
    │       └── workers
    │           ├── main.tf
    │           ├── outputs.tf
    │           └── variables.tf
    └── storage
        └── aws
            ├── main.tf
            ├── outputs.tf
            └── variables.tf

In here, you will find the main.tf under each of the modules that has the provisioning definition for DIGIT resources like EKS cluster, Network, RDS, and Storage, etc. All these are modularized and reacts as per the customized options provided. Follow the below steps to configure your terraform and run

1. Terraform State Setup

Create Terraform backend to specify the location of the backend Terraform state file on S3 and the DynamoDB table used for the state file locking. This step is optional.

Remote state is simply storing that state file remotely, rather than on your local filesystem. In a enterprise project and/or if Terraform is used by a team, it is recommended to setup and use remote state.

The following main.tf will create s3 bucket to store all the state of the execution to keep track.

cd DIGIT-DevOps/Infra-as-code/terraform/sample-aws/remote-state

provider "aws" {
  region = "ap-south-1"
}

#This is a bucket name that you can name as you wish
resource "aws_s3_bucket" "terraform_state" {
  bucket = "try-digit-eks-yourname" 

  versioning {
    enabled = true
  }

  lifecycle {
    prevent_destroy = true
  }
}

#This is a bucket name that you can name as you wish
resource "aws_dynamodb_table" "terraform_state_lock" {
  name           = "try-digit-eks-yourname"
  read_capacity  = 1
  write_capacity = 1
  hash_key       = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }

}

2. Network Infrastructure Setup

Setting up the VPC, Subnets, Security Groups, etc.

Amazon EKS requires subnets must be in at least two different availability zones.

1. Create AWS VPC (Virtual Private Cloud).

2. Create two public and two private Subnets in different availability zones.

3. Create Internet Gateway to provide internet access for services within VPC.

4. Create NAT Gateway in public subnets. It is used in private subnets to allow services to connect to the internet.

5. Create Routing Tables and associate subnets with them. Add required routing rules.

6. Create Security Groups and associate subnets with them. Add required routing rules.

3. EKS Cluster Setup

Create EKS cluster. Kubernetes clusters managed by Amazon EKS make calls to other AWS services on your behalf to manage the resources that you use with the service. For example, EKS will create an Auto Scaling Groups for each instance group if you use managed nodes.

Setting up the IAM Roles and Policies for EKS: EKS requires a few IAM Roles with relevant Policies to be pre-defined to operate correctly.

IAM Role: Create Role with the needed permissions that Amazon EKS will use to create AWS resources for Kubernetes clusters and interact with AWS APIs.

IAM Policy: Attach the trusted Policy (AmazonEKSClusterPolicy) which will allow Amazon EKS to assume and use this role.

The following main.tf contains the detailed resource definitions that need to be provisioned, please have a look at it.

Navigate to the directory: DIGIT-DevOps/Infra-as-code/terraform/sample-aws

# master configs, terraform state helps to maintain the flow of the execution
terraform {
  backend "s3" {
    bucket = "try-digit-eks-yourname"
    key = "terraform"
    region = "ap-south-1"
  }
}

# Network Module
module "network" {
  source             = "../modules/kubernetes/aws/network"
  vpc_cidr_block     = "${var.vpc_cidr_block}"
  cluster_name       = "${var.cluster_name}"
  availability_zones = "${var.network_availability_zones}"
}

# PostGres DB
module "db" {
  source                        = "../modules/db/aws"
  subnet_ids                    = "${module.network.private_subnets}"
  vpc_security_group_ids        = ["${module.network.rds_db_sg_id}"]
  availability_zone             = "${element(var.availability_zones, 0)}"
  instance_class                = "db.t3.medium"
  engine_version                = "11.5"
  storage_type                  = "gp2"
  storage_gb                    = "100"
  backup_retention_days         = "7"
  administrator_login           = "egovdev"
  administrator_login_password  = "${var.db_password}"
  db_name                       = "${var.cluster_name}-db"
  environment                   = "${var.cluster_name}"
}

# ********** EKS Cluster (Control Plane) **********
data "aws_eks_cluster" "cluster" {
  name = "${module.eks.cluster_id}"
}

data "aws_eks_cluster_auth" "cluster" {
  name = "${module.eks.cluster_id}"
}

provider "kubernetes" {
  host                   = "${data.aws_eks_cluster.cluster.endpoint}"
  cluster_ca_certificate = "${base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data)}"
  token                  = "${data.aws_eks_cluster_auth.cluster.token}"
  #load_config_file       = false
}

module "eks" {
  source          = "terraform-aws-modules/eks/aws"
  version         = "17.24.0"
  cluster_name    = "${var.cluster_name}"
  vpc_id          = "${module.network.vpc_id}"
  cluster_version = "${var.kubernetes_version}"
  subnets         = "${concat(module.network.private_subnets, module.network.public_subnets)}"

  worker_groups = [
    {
      name                          = "spot"
      subnets                       = "${concat(slice(module.network.private_subnets, 0, length(var.availability_zones)))}"
      override_instance_types       = "${var.override_instance_types}"
      kubelet_extra_args            = "--node-labels=node.kubernetes.io/lifecycle=spot"
      additional_security_group_ids = ["${module.network.worker_nodes_sg_id}"]
      asg_max_size                  = 4
      asg_desired_capacity          = 4
      spot_allocation_strategy      = "capacity-optimized"
      spot_instance_pools           = null
    }
  ]
  tags = "${
    tomap({
      "kubernetes.io/cluster/${var.cluster_name}" = "owned",
      "KubernetesCluster" = "${var.cluster_name}"
    })
  }"
 
}

# ********** EBS Volumes for the statefulsets (PVCs)
module "es-master" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "es-master"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "2"
  
}
module "es-data-v1" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "es-data-v1"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "25"
  
}

module "zookeeper" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "zookeeper"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "2"
  
}

module "kafka" {

  source = "../modules/storage/aws"
  storage_count = 3
  environment = "${var.cluster_name}"
  disk_prefix = "kafka"
  availability_zones = "${var.availability_zones}"
  storage_sku = "gp2"
  disk_size_gb = "50"
  
}

4. Custom variables/configurations:

You can define your configurations in variables.tf and provide the env specific cloud requirements so that using the same terraform template you can customize the configurations.

├── sample-aws
│   ├── main.tf 
│   ├── outputs.tf
│   ├── providers.tf
│   ├── remote-state
│   │   └── main.tf
│   └── variables.tf

Following are the values that you need to replace in the following files, the blank ones will be prompted for inputs while execution.

variables.tf

## Add Cluster Name
variable "cluster_name" {
  default = "<Desired Cluster name>"  #eg: my-digit-eks
}

## Add vpc_cidr_block
variable "vpc_cidr_block" {
  default = "CIDR" 
}

# If you want prod grade N/W, you can define HA, DRS with multi zone
variable "network_availability_zones" {  
  default = ["ap-south-1b", "ap-south-1a"]
}

# Which zone, it matters
variable "availability_zones" {
  default = ["ap-south-1b"]
}

variable "kubernetes_version" {
  default = "1.20"
}

# instance type for your worker nodes like r5a.large is 8 vCPU and 16GB RAM
variable "instance_type" {
  default = "r5a.large"
}

# spot instance configuration
variable "override_instance_types" {
  default = ["r5a.large", "r5ad.large", "r5d.large", "t3a.xlarge"] 
}

# number of machines as per estimate
variable "number_of_worker_nodes" {
  default = "3"
}

##Add ssh key in case you want to ssh to nodes
variable "ssh_key_name" {
  default = "ssh key name"
}

# terraform users ssh public key, you need to one for you, refer below to create yours
variable "iam_keybase_user" {
  default = "keybase:egovterraform"  #replace with your keybase 
}

# will be prompted to provide during the execution
variable "db_password" {}

5. Terraform Execution: Infrastructure Resources Provisioning

Once you have finished declaring the resources, you can deploy all resources.

1. terraform init: command is used to initialize a working directory containing Terraform configuration files.

2. terraform plan: command creates an execution plan, which lets you preview the changes that Terraform plans to make to your infrastructure.

3. terraform apply: command executes the actions proposed in a Terraform plan to create or update infrastructure.

After the complete creation, you can see resources in your AWS account.

Now that we know what the terraform script does, the resources graph that it provisions and what custom values should be given with respect to your env.

Let's begin to run the terraform scripts to provision infra required to Deploy DIGIT on AWS.

1. First CD into the following directory and run the following command 1-by-1 and watch the output closely.

### Create the remote-state first, remember that the state name should be unique
cd DIGIT-DevOps/infra-as-code/terraform/sample-aws/remote-state
terraform init

terraform plan

terraform apply

### Once the remote state is created, you can create the DIGIT Infra
cd DIGIT-DevOps/infra-as-code/terraform/sample-aws
terraform init

terraform plan

terraform apply

Upon Successful execution following resources gets created which can be verified by the command "terraform output"

• s3 bucket: to store terraform state.

• Network: VPC, security groups.

2. Use this link to get the kubeconfig from EKS to get the kubeconfig file and being able to connect to the cluster from your local machine so that you should be able to deploy DIGIT services to the cluster.

aws sts get-caller-identity

# Run the below command and give the respective region-code and the cluster name
aws eks --region <region-code> update-kubeconfig --name <cluster_name>

3. Finally, Verify that you are able to connect to the cluster by running the following command

kubectl config use-context <cluster_name>

kubectl get nodes

NAME                                             STATUS AGE   VERSION               OS-Image           
ip-192-168-xx-1.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2   
ip-192-168-xx-2.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2   
ip-192-168-xx-3.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2   
ip-192-168-xx-4.ap-south-1.compute.internal   Ready  45d   v1.15.10-eks-bac369   Amazon Linux 2 

Cleanup

To destroy previously-created infrastructure with Terraform, run below command: But you can do this step after deployed DIGIT

terraform destroy: command is a convenient way to destroy all remote objects managed by a particular Terraform configuration.

Now we have all the deployments configs ready, you can now run the following command and provide the necessary details asked and this interactive installer, it will take care of the rest.

What you can try deploy is:

• DIGIT's core platform services

• PGR

• TL (Trade License)

• PT (Property Tax)

• WS (Water & Sewerage)

• etc

1. Prerequisites

1. DIGIT uses golang (required v1.13.3) automated scripts to deploy the builds on to Kubernetes - Linux or Windows or Mac

2. All DIGIT services are packaged using helm charts Installing Helm

3. kubectl is a CLI to connect to the kubernetes cluster from your machine

4. Install CURL for making api calls

5. Install Visualstudio IDE Code for better code/configuration editing capabilities

6. Install Postman to run digit bootstrap scripts

Run Deployer

1. Run the egov-deployer golang script from the DIGIT-Devops repo

root@ip:# cd DIGIT-DevOps/deploy-as-code/deployer

root@ip:# go run standalone_installer.go

#Be prepared for the following questions
1. Do you have the Kubernetes Setup?
2. Provide the path of the intented env kubeconfig file
3. Which version of the DIGIT that you want to install
4. What DIGIT Modules that you choose to install (Choose PGR)
5. All, done, Now do you want to preview the deployment manifests 
6. Are you good to proceed with the DIGIT Installation

All Done.

All Done, wait and watch for 10 min, you'll have the DIGIT setup completed and the application will be running on the given URL.

Note:

• If you do not have your domain yet, you can edit the host file entries and map the nginx-ingress-service loadbalancer id like the below

  • When you find it, add following lines to the hosts file, save and close the file.

  • aws-load-balancer-id digit.try.com

• If you have the GoDaddy like account and a DNS records edit access you can map the load balancer id to desired DNS Create cname record with load balancer id and domain.

You can now test the Digit application status in command prompt/terminal by using the below command.

curl -Is http://digit.try.com/employee/login |  head -n 1

OutPut:
HTTP/2 200

Note: Initially pgr-services would be in crashloopbackoff state, but after performing below Post Deployment Steps pgr-services will start running.

Post-deployment, now the application is accessible from the configured domain.

To try out employee login, Lets create a sample tenant, city, user to login and assign LME employee role through the seed script

1. We have to do the kubectl port-forwarding of the egov-user service running from Kubernetes cluster to your localhost, this will now give you access to egov-user service directly and interact with the API directly.

kubectl port-forward svc/egov-user 8080:8080 -n egov
Output:
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

2. Seed the sample data

• Ensure you have the postman to run the following seed data API, if not Install postman on your local

• Import the following postman collection into the postman and run it, this will have the seed data that enable sample test users and localization data.

  • DIGIT Bootstrap

To test the Kubernetes operations through kubectl from your local machine, please execute the below commands.

#get the pods running in the cluster
kubectl get pods -n egov

Output:

NAME                                                    READY   STATUS    RESTARTS   AGE
citizen-79cf89659c-8glf4                                1/1     Running   0          14d
egov-accesscontrol-78b78dddb9-tfbkf                     1/1     Running   0          21d
egov-enc-service-574cd7b5b5-hmhh4                       1/1     Running   0          36d
egov-idgen-84954b565b-xsnqj                             1/1     Running   0          45d
egov-indexer-5f5fbb6f4b-58rtm                           1/1     Running   0          43d
egov-localization-6cc5977bb9-gm7f9                      1/1     Running   0          27d
egov-mdms-service-65d6d65d8c-t85d9                      1/1     Running   0          21d
egov-user-6676968d76-8n6t6                              1/1     Running   0          29d
egov-workflow-v2-5cdb96bcf5-dcgmf                       1/1     Running   0          36d
employee-749464fbfb-tptlh                               1/1     Running   0          14d
nginx-ingress-controller-b9678869c-mkslb                1/1     Running   0          49d
pgr-services-b9f4ffdbf-5h5kd                            1/1     Running   0          38d
zuul-788bf8cd8b-9nxfl                                   1/1     Running   0          41d


#Delete the pods so that it gets restarted automatically

kubectl delete pods zuul-788bf8cd8b-9nxfl egov-workflow-v2-5cdb96bcf5-dcgmf pgr-services-b9f4ffdbf-5h5kd -n egov

Output:

pod "zuul-788bf8cd8b-9nxfl" deleted
pod "egov-workflow-v2-5cdb96bcf5-dcgmf" deleted
pod "pgr-services-b9f4ffdbf-5h5kd" deleted

You have successfully completed the DIGIT Infra, Deployment setup and Installed a DIGIT - PGR module.

Use the below link in browser

digit.try.com/employee

Use the below credentials to login into the complaint section

Username: GRO

Password: eGov@4321

City: CITYA

DIGIT Deployment - Assessment

By now we have successfully completed the digit setup on cloud, use the URL that you mentioned in your env.yaml Eg: https://mysetup.digit.org and create a grievance to ensure the PGR module deployed is working fine. Refer the below product documentation for the steps.

Credentials:

1. Citizen: You can use your default mobile number (9999999999) to sign in using the default Mobile OTP 123456.

2. Employee: Username: GRO and password: eGov@4321

Post grievance creation and assignment of the same to LME, capture the screenshot of the same and share it to ensure your setup is working fine.

Destroy the Cluster

Post validating the PGR functionality share the API response of the following request to assess the correctness of successful DIGIT PGR Deployment.

Finally, cleanup the DIGIT Setup if you wish, using the following command. This will delete the entire cluster and other cloud resources that were provisioned for the DIGIT Setup.

cd DIGIT-DevOps/infra-as-code/terraform/my-digit-eks
terraform destroy

Conclusion

All Done, we have successfully Created infra on Cloud, Deployed Digit, Bootstrapped DIGIT, Performed a Transaction on PGR and finally destroyed the cluster.

The Azure Kubernetes Service (AKS) is one of the Azure services for deploying, managing and scaling any distributed and containerized workloads, here we can provision the AKS cluster on Azure from the ground up using (infra-as-code) and then deploy the DIGIT platform services as config-as-code using .

Pre-reads

• Know about AKS:

• Know what is terraform:

Prerequisites

1. Azure subscription: If you don't have an Azure subscription, create a before you begin.

2. Install Azure

3. Configure Terraform: Follow the directions in the article,

4. Azure service principal: Follow the directions in the Create the service principal section in the article, . Take note of the values for the appId, displayName, password, and tenant.

5. Install on your local machine that helps you interact with the kubernetes cluster

6. Install that helps you package the services along with the configurations, envs, secrets, etc into a

All content on this page by is licensed under a .

Pre-reads

• Know the basics of Kubernetes:

• Know the commands

• Know kubernetes manifests:

• Know how to manage env values, secrets of any service deployed in kubernetes

• Know how to port forward to a pod running inside k8s cluster and work locally

• Know sops to secure your keys/creds:

Post Kubernetes Cluster setup, the deployment has got 2 stages. As part of this sample exercise we can deploy PGR and show what are the various configuration required, however deployment steps are similar for all other modules too, just that the prerequisites differ depending on the feature like SMS Gateway, Payment Gateway, etc

1. Prepare Deployment Configuration File

It's important to prepare a your global deployment configuration yaml file that will contain all the necessary user specific custom values like URL, gateways, persistent storage ids, DB details etc.

Navigate to the following file in your local machine from previously cloned DevOps git repository

2. After cloning the repo CD into the folder DIGIT-DevOps and type the "code ." command that will open the visual editor and opens all the files from the repo DIGIT-DevOps
3. • Here you need to replace the following as per your values

     • URL that you want to access DIGIT

     • SMS gateway to receive otp, transaction mobile notification, etc

     • MDMS, Config repo url, here is where you provide master data, tenants and various user/role access details.

     • GMap key for the location service

     • Payment gateway, in case you use PT, TL, etc
4. Update your credentials and sensitive data in the secret file as per your details.

   • SOPS expects an encryption key to use it to encrypt/decrypt a specified plain text and keep the details secured, there are couple of option which you can use it to generate the encryption key
5. You have to fork the following repos that contains the master data and default configs which you would customize as per your specific implementation later point in time. Like (Master Data, ULB, Tenant details, Users, etc) to your respective github account.

   • New github user should be enabled to access the earlier forked repo's
6. Update the deployment configs for the below as per your specification

   • Number of replicas/scale of each individual services (Depending on whether dev or prod load)

   • You must update sms g/w, email g/w, payment g/w details for the notification and payment gateway services, etc.

   • Update the config, mdmd github repos wherever marked

   • Update GMap key (In case you are using Google Map services in your PGR, PT, TL, etc)

   • URL/DNS on which the DIGIT will be exposed

   • SSL Certificate for the above URL

   • Any specific endpoints configs (Internal/external)

Annexures:

• s

All content on this page by is licensed under a .

manages your hosted Kubernetes environment. AKS allows you to deploy and manage containerized applications without container orchestration expertise. AKS also enables you to do many common maintenance operations without taking your app offline. These operations include provisioning, upgrading, and scaling resources on demand.

Installation Steps

All content on this page by is licensed under a .

Terraform is an open source infrastructure as code () software tool that allows DevOps engineers to programmatically provision the physical resources an application requires to run.

Infrastructure as code is an IT practice that manages an application's underlying IT infrastructure through programming. This approach to resource allocation allows developers to logically manage, monitor and provision resources -- as opposed to requiring that an operations team manually configure each required resource.

Terraform users define and by using a JSON-like configuration language called HCL (HashiCorp Configuration Language). HCL's simple syntax makes it easy for DevOps teams to provision and re-provision infrastructure across multiple cloud and on-premises data centers.

Cloud resources required for DIGIT

Before we provision the cloud resources, we need to understand and be sure about what resources need to be provisioned by terraform to deploy DIGIT. The following picture shows the various key components. (AKS, Node Pools, Postgres DB, Volumes, Load Balancer)

Understand Terraform Script

• Here we have already written the terraform script that one can reuse/leverage that provisions the production-grade DIGIT Infra and can be customized with the user specific configuration.

Save the file and exit the editor

5. Terraform Execution: Infrastructure Resources Provisioning

Once you have finished declaring the resources, you can deploy all resources.

1. terraform init: command is used to initialize a working directory containing Terraform configuration files.

2. terraform plan: command creates an execution plan, which lets you preview the changes that Terraform plans to make to your infrastructure.

3. terraform apply: command executes the actions proposed in a Terraform plan to create or update infrastructure.

After the complete creation, you can see resources in your Azure account.

Now that we know what the terraform script does, the resources graph that it provisions and what custom values should be given with respect to your env.

Let's begin to run the terraform scripts to provision infra required to Deploy DIGIT on Azure.

1. First CD into the following directory and run the following command 1-by-1 and watch the output closely.

6. Test the Kubernetes cluster

The Kubernetes tools can be used to verify the newly created cluster.

1. Once terraform apply execution is done it will generate the Kubernetes configuration file or you can get it from Terraform state.

2. Set an environment variable so that kubectl picks up the correct config.

3. Verify the health of the cluster.

You should see the details of your worker nodes, and they should all have a status Ready, as shown in the following image:

Coming soon ...

On National Informatica Cloud

Coming soon .....

Running Kubernetes on-premise gives a cloud-native experience on SDC when it comes to Deploying DIGIT.

Whether States have their own on-premise data centre or have decided to forego the various managed cloud solutions, there are a few things one should know when getting started with on-premise K8s.

One should be familiar with Kubernetes and the control plane consists of the Kube-apiserver, Kube-scheduler, Kube-controller-manager and an ETCD datastore. For managed cloud solutions like Google’s Kubernetes Engine (GKE) or Azure’s Kubernetes Service (AKS) it also includes the cloud-controller-manager. This is the component that connects the cluster to the external cloud services to provide networking, storage, authentication, and other feature support.

To successfully deploy a bespoke Kubernetes cluster and achieve a cloud-like experience on SDC, one need to replicate all the same features you get with a managed solution. At a high-level this means that we probably want to:

• Automate the deployment process

• Choose a networking solution

• Choose the right storage solution

• Handle security and authentication

Let us look at each of these challenges individually, and we’ll try to provide enough of an overview to aid you in getting started.

Automating the deployment process

Using a tool like an ansible can make deploying Kubernetes clusters on-premise trivial.

When deciding to manage your own Kubernetes clusters, we need to set up a few proof-of-concept (PoC) clusters to learn how everything works, perform performance and conformance tests, and try out different configuration options.

After this phase, automating the deployment process is an important if not necessary step to ensure consistency across any clusters you build. For this, you have a few options, but the most popular are:

• kubeadm: a low-level tool that helps you bootstrap a minimum viable Kubernetes cluster that conforms to best practices

• kubespray: an ansible playbook that helps deploy production-ready clusters

If you already using ansible, kubespray is a great option otherwise we recommend writing automation around kubeadm using your preferred playbook tool after using it a few times. This will also increase your confidence and knowledge in the tooling surrounding Kubernetes.