AWS Notes 2 - keshavbaweja-git/guides GitHub Wiki

AWS Cloud -

  • comprehensive and widely adopted cloud platform.
  • most extensive global cloud infrastructure based in
  • 25 geographic regions around the world.
  • multiple isolated and physically separate data center locations known as Availability Zones.
  • These AZs are connected by low latency, high throughput, and highly redundant networking and provide a mechanism to build a high available and fault-tolerant architecture.
  • If an application is architected to be hosted across multiple AZs, it offers resilience against outage of one or more AZs.

Amazon Virtual Private Cloud (Amazon VPC)

  • provision a logically isolated section of the AWS Cloud
  • launch AWS resources in a virtual network.
  • Within a VPC, different sub-networks or subnets can be defined to manage routing of network traffic.
  • A VPC spans across all AZs in a region while subnets are located in a single AZ.
  • A Public subnet has connectivity to Internet and allows inbound and outbound Internet traffic.
  • A Private subnet does not have network route to Internet Gateway and no connectivity to Internet.

Customer portal is a web application that is deployed on Websphere application servers. As I noted in my proposal document, Websphere is a commercial, licensed product typically with restrictions on number of servers/CPUS cores that can be used. Also provisioning new servers in a corporate data center can take weeks, due to these limitations it is not possible to scale out the application servers easily and quickly in response to increased workload. Let's build out the architecture on AWS and see how it addresses scalability and availability concerns -

Elastic Compute Cloud (Amazon EC2)

  • secure, scalable compute capacity in the cloud.
  • offers the broadest and deepest compute platform with choice of processor, storage, networking, operating system, and purchase model.
  • you can provision hundreds of servers in minutes.
  • number of instance types to choose from, there are instances that are optimized for computation or memory. There are also general purpose instances.
  • The flexibility that EC2 offers to you is that you can easily change instance types and sizes. You can choose from three purchase options
    • On Demand pay an hourly rate
    • Spot (spare capacity offered at discounted prices, can be reclaimed by AWS on a two minutes notice)
    • Reserved (significant saving with a commitment period of one to three years)

Amazon EC2 Auto Scaling

  • automatically scale your Amazon EC2 capacity up or down according to conditions you define.
  • you can ensure that the number of Amazon EC2 instances you’re using scales up seamlessly during demand spikes to maintain performance, and scales down automatically during demand lulls to minimize costs.
  • particularly well suited for applications that experience hourly, daily, or weekly variability in usage. EC2 Auto Scaling is enabled by Amazon CloudWatch and available at no additional charge beyond Amazon CloudWatch fees.

Elastic Load Balancing

  • Automatically distributes your incoming application traffic across all the EC2 instances that you are running.

  • Manages incoming requests by optimally routing traffic so that no one instance is overwhelmed.

  • To use Elastic Load Balancing with your Auto Scaling group, attach the load balancer to your Auto Scaling group. This registers the group with the load balancer, which acts as a single point of contact for all incoming web traffic to your Auto Scaling group. After attaching a load balancer to your Auto Scaling group, you can configure your Auto Scaling group to use Elastic Load Balancing metrics (such as the Application Load Balancer request count per target) to scale the number of instances in the group as demand fluctuates. Optionally, you can add Elastic Load Balancing health checks to your Auto Scaling group so that Amazon EC2 Auto Scaling can identify and replace unhealthy instances based on these additional health checks.

  • As you can see this is a highly scalable and flexible infrastructure,

  • but the development teams need to learn and configure all of these technologies which does require some time and effort.

  • To ease this effort, I propose that PA use Elastic Beanstalk service, this service automates the provisioning and configuration of the compute infrastructure that we have just discussed.

  • Additionally it provides several curated runtimes will all dependencies preconfigured.

  • For customer portal, we will use Apache Tomcat which is Java based application server.

Database

  • AWS offers a managed Relational Database Service known as RDS
  • RDS supports multiple database engines including Oracle and popular open source engines like MySQL and PostgreSQL
  • Aurora is another relational database engine that has been developed by AWS
  • AWS has been leading innovation in cloud architecture and RDS Aurora is a prime example of cloud native, distributed, purpose build solution for modern applications
  • Fully managed RDS engines - hardware provisioning, database setup, patching, and backups are automated
  • Distributed, fault-tolerant, self-healing storage system that auto-scales up to 128TB per database cluster.
  • high performance and availability with up to 15 low-latency read replicas
  • replication across three Availability Zones (AZs) with six copies of data
  • Failover to an replica in another AZ takes less than 60 secs.
  • Aurora multi master (upto 4 masters, Oracle RAC instances need to run in the same data centre, it is not performant across data centres)

Database migrations

  • time consuming and complex.
  • minimum downtime for customer portal during data migration and the migration process should be automated with robust rollback/fall-forward plans.
  • AWS Database Migration Service (DMS) and AWS Schema Conversion Tool (SCT).

Schema conversion tool

  • analyzes source database schema and automatically converts the schema and a majority of the database code objects to a format compatible with the target database.
  • Any objects that cannot be automatically converted are clearly marked so that they can be manually converted to complete the migration.
  • SCT can also scan application source code for embedded SQL statements and convert them as part of a database schema conversion project.

Database Migration Service

  • helps to migrate databases to AWS quickly and securely.
  • The source database remains fully operational during the migration, minimising downtime to applications that rely on the database. - - AWS DMS works by launching a Replication server that manages data flow between source and target database instances.
  • DMS and SCT are offered as free services when migrating to AWS Aurora. Free use of DMS and SCT is available for 6 months (183 days) from the time you start an eligible DMS instance.

Route 53

  • highly available, scalable DNS service
  • Domain Name Registration
  • Domain Name Resolution
  • Health Checks

Extremely reliable and cost effective DNS service that connects Internet users to applications and resources hosted on AWS cloud. It supports various routing policies like -

  • Latency based
  • Geolocation based
  • Weighted round robin

CloudFront

  • Content Delivery Network from AWS that provides a fast and consistent experience for global user base.
  • CloudFront is made up more than 220 edge locations and regional caches that are connected to AWS network backbone.
  • AWS network backbone is a global, fully redundant fibre optic network.
  • With CloudFront user request is routed to nearest edge location and from there on AWS network backbone.
  • CloudFront can not only speed up delivery of static content and media streaming, it can can also be used for Dynamic content served by web applications and APIs.
  • This is due to the intelligent network routing and network optimization techniques like TLS handshake termination.

WAF protects web applications and APIs against common web exploits and bots that may

  • affect availability
  • compromise security
  • consume excessive resources

WAF allows various security rules to be configured that allow or block traffic based on defined criteria. Preconfigured rules managed by AWS and 3rd party providers are available in AWS Marketplace for a quick start. There are various collection of security rules that are curated for

  • OWASP Top 10,
  • CVE,
  • IP reputation lists &
  • vulnerabilities in products like Apache & Nginx.

CloudFront is integrated with CloudWatch Metrics and Logs. You can save detailed logs and analyse the behaviour of security rules configured. WAF can be integrated with CloudFront, ALB and other AWS services.

AWS Shield

  • Managed DDoS protection service.
  • Safeguards applications hosted on AWS cloud
  • It employs network traffic flow monitoring and anomaly detection algorithms for always on detection and automated mitigations

AWS Shield is offered in two tiers

  • Standard (no additional charge)
  • Advanced
    • 24*7 access to DDoS Response Team
    • Protection against DDoS related spikes in infrastructure costs
    • Layer 7 attacks
    • Custom protection
    • WAF is free

Elastic Beanstalk

  • Greatly simplifies running web applications on AWS infrastructure
  • Deliver web applications
    • Short, quick development cycles
    • Low cost
    • Minimum downtime
    • Adherence to IT standards
  • Accelerate innovation
  • Freedom to use different technology platform
  • Systematic approach to CI/CD

Provision -> Deploy -> Monitor/Manage

What does it do?

  • Infrastructure provisioning and configuration
  • Deployment strategies
  • Load balancing
  • Auto scaling
  • Health monitoring
  • Analysis & debuggin
  • Logging