AWS Configuration - mihai-dobre/autoscaling GitHub Wiki

AWS Configuration

To make sure that all resources communicate with each other, we need to follow a simple rule: all resources need to be in the same VPC(virtual private cloud). That being said, we start by creating a Fargate powered cluster with a new VPC(Virtual Private Cloud) attached to it.

Steps followed

  1. Create with ECS Cluster for Web application(api) along with a new VPC containing:
  • default security group
  • 2 public subnets
  • internet gateway
  • routing table
  • availability zones
  1. Create an Application Load Balancer(ALB) with the following key attibutes:
  • using the VPC and subnets created above
  • internet-facing
  • accepting incoming requests on port 80
  1. Create a new security group for ALB
  • Rule: accept incoming traffic on LB port 80 from internet(0.0.0.0/0).
  1. Create a target group for ALB
  • target port 6000 as the container running the web applicaiton
  • select target type as 'IP'. This will target all the IPs in the VPC
  • register targets: select the VPC where targets should live (10.0.0.0/16). Keep same port 6000.
  1. Create Elastic Cache Redis
  • use same VPC.
  • create new subnet group for redis in the same VPC.

  1. Edit Redis security group to allow traffic form VPC on port 6379

  2. Create a RDS Postgres database

  • use the same master VPC
  1. Create security group for Postgres
  • add inbound rule to accept incoming traffic on port 5432 from VPC
  1. Create ECS cluster for celery workers
  • simple Fargate cluster
  • no new VPC
  1. Create ECR repositories for web/api and celery worker.
  • Build and upload corresponding docker images to the repositories.

  1. Create a new role for Elastic Container Service -> Service Task execution.

  2. Create a new task definition for web/api.

  • compatibility: Fargate
  • choose appropriate Task size(CPU and memory)
  • network mode: awsvpc
  • map port 6000
  • choose web/api repository
  • set environment variables
  • set entrypoint and command
  • set CloudWatch log configuration
  1. Create a new task for the celery worker.
  • compatibility: Fargate
  • choose appropriate Task size(CPU and memory)
  • network mode: awsvpc
  • choose web/api repository
  • set environment variables
  • set entrypoint and command
  • set CloudWatch log configuration
  1. Create an ECS service for web cluster
  • use Fargate compatibility
  • attach existing ALB
  • incoming requests on port 80
  • auto-assign public IP enabled
  • enable service discovery
  • set desired count
  • use same master VPC
  1. Create an ECS service for celery worker
  • use Fargate compatibility
  • auto-assign public IP enabled
  • use auto scaling
  • define auto scaling policy as avg memory based, 70%, maximum tasks 10
  • use same master VPC
  1. Keep in mind to set rules for incoming traffic on all new/existing security groups according to the resources attached to them.
  • 80 for incoming requests on ALB from internet
  • 6000 for incoming requests from ALB to web/api Fargate instances(within VPC)
  • 6379 for Redis requests coming from within VPC
  • 5432 for Postgres db requests coming from within VPC
  1. Get DNS name of the ALB and check for health endpoint(/health).

Disclaimer

This is solution should be used in a test environment and never in production.