On AWS
Provision infra for DIGIT on AWS using Terraform
Pre-read:
Prerequisites
You'll get a Secret Access Key and Access Key ID. Save them safely.
Open the terminal and Run the following command you have already installed the AWS CLI and you have the credentials saved. (Provide the credentials and you can leave the region and output format as blank)
The above will create the following file In your machine as /Users/<your username>/.aws/credentials
DIGIT - Infra Resource Graph on AWS
Before we provision the cloud resources, we need to understand and be sure about what resources need to be provisioned by terraform to deploy DIGIT. The following picture shows the various key components. (EKS, Worker Nodes, PostGres DB, EBS Volumes, Load Balancer)
Considering the above deployment architecture, the following is the resource graph that we are going to provision using terraform in a standard way so that every time and for every env, it'll have the same infra.
EKS Control Plane (Kubernetes Master)
Work node group (VMs with the estimated number of vCPUs, Memory)
EBS Volumes (Persistent Volumes)
RDS (PostGres)
VPCs (Private network)
Users to access, deploy and read-only
Understand the Resource Graph in Terraform script:
Here we have already written the terraform script that provisions the production-grade DIGIT Infra and can be customized with the specified configuration.
In here, you will find the main.tf under each of the modules that has the provisioning definition for DIGIT resources like EKS cluster, RDS, and Storage, etc. All these are modularized and reacts as per the customized options provided.
Example:
VPC Resources:
VPC
Subnets
Internet Gateway
Route Table
EKS Cluster Resources:
IAM Role to allow EKS service to manage other AWS services
EC2 Security Group to allow networking traffic with EKS cluster
EKS Cluster
EKS Worker Nodes Resources:
IAM role allowing Kubernetes actions to access other AWS services
EC2 Security Group to allow networking traffic
Data source to fetch latest EKS worker AMI
AutoScaling Launch Configuration to configure worker instances
AutoScaling Group to launch worker instances
Database
Configuration in this directory creates set of RDS resources including DB instance, DB subnet group, and DB parameter group.
Storage Module
Configuration in this directory creates EBS volume and attaches it together.
The following main.tf with create s3 bucket to store all the state of the execution to keep track.
The following main.tf contains the detailed resource definitions that need to be provisioned, please have a look at it.
Dir: DIGIT-DevOps/Infra-as-code/terraform/sample-aws
Custom variables/configurations:
You can define your configurations in variables.tf and provide the env specific cloud requirements so that using the same terraform template you can customize the configurations.
Following are the values that you need to mention in the following files, the blank ones will be prompted for inputs while execution.
Important: Create your own keybase key before you run the terraform
Run terraform
Now that we know what the terraform script does, the resources graph that it provisions and what custom values should be given with respect to your env.
Let's begin to run the terraform scripts to provision infra required to Deploy DIGIT on AWS.
First CD into the following directory and run the following command 1-by-1 and watch the output closely.
Upon Successful execution following resources gets created which can be verified by the command "terraform output"
s3 bucket: to store terraform state.
Network: VPC, security groups.
EKS cluster: with master(s) & worker node(s).
Storage(s): for es-master, es-data-v1, es-master-infra, es-data-infra-v1, zookeeper, kafka, kafka-infra.
Finally, Verify that you are able to connect to the cluster by running the following command
Last updated