EKS Getting Started

Introduction

  • Purpose of the Guide: This guide is designed to assist users in comprehending the necessary prerequisites and the steps involved to successfully execute the project.
  • Definitions:
    • Environment: The project utilized terraform workspace for switch between environments.
  • Project's folder structure:
    • docs: Project's documentation
    • env: Folder contains environment specific backend and modules configuration
    • modules: Project's modules
    • plans: Folder for storing generated plans
    • scripts: terraform init, terraform plan and terraform apply, with workspace (environment) configuration applied, implemented in both bash and powershell scripts.
  • Backend: aws backend is configured by <ENV>.backend.tfvars file.
  • Customization:
    • Modules can be customized by editing their respective <ENV>.tfvars files, where ENV is selected workspace.
    • Additional resources and configurations can be added as per project requirements.

Getting Started

Please note that this project is designed to provision new AWS resources, including the setup of a new Elastic Kubernetes Service (EKS) cluster, among other components. It has not been tested with existing EKS clusters (terraform import) or pre-configured AWS resources. Users planning to deploy this project should be aware that it assumes a fresh environment for optimal compatibility and performance. Existing AWS resource configurations may require adjustments or could lead to unpredictable outcomes if used in conjunction with this project.

  • Prerequisites:

    • Amazon account with AdminstratorAccess permission policy.
    • Terraform installed on your local machine.
    • AWS CLI installed on your local machine
    • S3 for storing Terraform backend.
    • You will need a domain under your control with the ability to manage its DNS settings for Kiali and Grafana dashboards. After deployment, Kiali and Grafana dashboards will be up and running.
    • Some knowledge of Terraform, Kubernetes, Helm and Istio.

  • Initial Setup:

    • Rename default.backend.tfvars to dev.backend.tfvars and define a proper backend configuration.

    • Run terraform init to initialize the backend:

      .\scripts\init.ps1 -ENV "dev"

    • Create a new workspace dev:

      terraform workspace new dev

      Empty dev workspace will be created and selected

    • Rename default.tfvars to dev.tfvars and configure project's modules.

    • Adjust parameters as per your infrastructure needs

    • Make sure that optional modules are disabled on this stage:

      project_modules         = {
    kiali           = false
    prometheus      = false
    grafana         = false
    letsencrypt     = false
}

Deployment with Terraform

The deployment can be divided in 3 phases:

  • Deploy resources
  • Create an A records for your domains
  • Deploy services

There is couple of reasons for such dividing:

  1. kubernetes_manifest terraform statements will try to connect cluster first to check if resource is already created on a plan stage, and will fail because cluster is not created yet.
  2. Let's encrypt will perform ACME challenge for your domains as a part of certification issuing process, which requires your domain to point to Load Balancer address. DNS record update is async and might take 5-60 minutes depending on provider.

  • Phase I - Deploy resources (~10 minutes): Considering everything is set in Initial setup, you can start building the infrastructure

    • Plan terraform changes:

      .\scripts\plan.ps1

    • Review the output

    • Apply the changes:

      .\scripts\apply.ps1

  • Phase II - Create CNAME records for domains (~ 5-60 minutes):

    • Obtain the load balaner DNS address via Portal/AWS CLI or by running:

      kubectl get svc/istio-ingress -n istio-system

      EXTERNAL-IP will be the load blancer address

    • Create CNAME record for kiali

    • Create CNAME record for grafana

    • Wait up to 60 minutes (depending on your doamin registrar) and check that domain is resolved to cluster's load balancer

  • Phase III - Build the services (~5-10 minutes): Assuming that above is implemented

    • Turn optional modules on:

      project_modules         = {
    kiali           = true
    prometheus      = true
    grafana         = true
    letsencrypt     = true
}

    • Plan terraform changes:

      .\scripts\plan.ps1

    • Review the output

    • Apply the changes:

      .\scripts\apply.ps1

    Enjoy!

Usage

  • Connecting to the cluster: Update kubeconfig with new context:

    aws eks update-kubeconfig --region <your region> --name <cluster name>