Web Hosting - fcrimins/fcrimins.github.io GitHub Wiki

See also: Docker Notes

AWS Service Load Balancing (6/1/17)

A Comprehensive Guide to Building a Scalable Web App on Amazon Web Services - Part 1 (5/31/17)

  • "Among those services, the bulk of your learning will be in EC2, VPC, S3, and one or more of the persistence services including RDS or DynamoDB."
  • "The best practice here is to make your App Tier instances stateless and store things like session state in a different tier. We discuss this more more in 4.3 Architecting for Scalability"
  • "The role of the Cache Tier is to store ephemeral data like user session information, or the results of commonly requested queries."
    • We don't seem to have a cache tier.
    • "You can setup any cache server of your choice on an EC2 instance, or AWS provides the ElastiCache service as a managed Cache Tier and lets you choose whether it should use Memcached or Redis as the underlying software. Redis is generally the newer and more popular solution on newer projects."
    • "When your Cache Tier holds all ephemeral state like session values, your App Tier no longer has to maintain this state itself. This means you can launch additional EC2 instances, and as long as they are configured to look to the Cache Tier for ephemeral data and the Database Tier for persistent data, they will "just work." For this reason, a Cache Tier is usually an essential part of auto scaling."
  • "If you decide to use AWS DynamoDB you are entering a world that comes as close to zero administration as I have seen."
    • "If you're using DynamoDB you should be paranoid about things going wrong in your Database Tier. If you're not using DynamoDB, you should be very, very paranoid about things going wrong."
  • "Hard-coding an IP address in your Web Tier configuration is the most basic form of Service Discovery and suffers from the problems of not being dynamically updated when new servers get added, and not auto-removing unhealthy instances. The classic way to handle Service Discovery for ASGs is using an Elastic Load Balancer as described in the official documentation."
  • "So in November 2014, AWS released AWS Key Management Service. This service is a perfect complement to the OWASP recommendations and provides a centralized place to manage all of your keys. Of course, you'll still need to implement best practices in your app itself."

Flask Deployment Options (5/18/17)

  • Handling multiple requests in Flask
    • "Is there any way that I can make my Flask application accept requests from multiple users?"
    • "Yes, deploy your application on a different WSGI server, see the Flask deployment options documentation."
  • Flask Deployment Options
    • "While lightweight and easy to use, Flask's built-in server is not suitable for production as it doesn’t scale well and by default serves only one request at a time."
    • This said, when you go to the AWS EB link on that page, the example EB deployment still runs using "Flask's build-in server," just with debug set to False.
  • Deploying a Flask Application to AWS Elastic Beanstalk
  • Dockerizing a Python web app (and deploying it to EB)
    • This approach uses a Dockerrun.aws.json file in place of docker-compose.yml
  • Configuring Docker Environments (for deployment to EB)
    • "Specify images by name in Dockerrun.aws.json. Note these conventions:"
      • "Images in official repositories on Docker Hub use a single name (for example, ubuntu or mongo)."
      • "Images in other repositories on Docker Hub are qualified with an organization name (for example, amazon/amazon-ecs-agent)."
      • "Images in other online repositories are qualified further by a domain name (for example, quay.io/assemblyline/ubuntu or account-id.dkr.ecr.us-east-1.amazonaws.com/ubuntu:trusty)."
    • Includes instructions on "Using Images from an Amazon ECR Repository" on EB
  • Difference between Amazon ec2 and AWS Elastic Beanstalk
    • "Elastic Beanstalk is one layer of abstraction away from the EC2 layer. Elastic Beanstalk will setup an 'environment' for you that can contain a number of EC2 instances, an optional database, as well as a few other AWS components such as a Elastic Load Balancer, Auto-Scaling Group, Security Group. Then Elastic Beanstalk will manage these items for you whenever you want to update your software running in AWS."
    • "EC2 Container Service is Amazon's managed replacement for running your own Mesos cluster. It's a good solution if you're running multiple applications, and simplifies management and billing. If you're running a single application, unless you just like the Dockerized model, Beanstalk is a better option."
  • CloudAcademy: Amazon EC2 Container Service and Elastic Beanstalk: Docker on AWS
    • "three ways to run Docker containers on AWS"
      1. "Deploying Docker containers directly to an Ec2 instance."
      2. "Using Docker containers on Elastic Beanstalk."
      3. "Docker cluster management using the AWS EC2 Container Service."
    • "Even a slight difference between your development, test, and production environments may completely break your application. Traditional development models follow a change management process to solve these kind of the problems. But this process won’t fit in today’s rapid build and deploy cycles."
  • Amazon ECS FAQs
    • "Q: How is Amazon ECS different from AWS Elastic Beanstalk?"
    • "Q: I want to launch containers. Why do I have to launch Tasks?"
      • "Tasks allow you to define a set of containers that you would like to be placed together, their properties, and how they may be linked. Tasks include all the information that EC2 Container Service needs to make the placement decision. To launch a single container, your Task Definition should only include one container definition."
  • EC2 Container Service Load Balancing
    • This is what I've been looking for.
    • Application Load Balancer
      • "This dynamic mapping allows you to have multiple tasks from a single service on the same container instance."

Getting Started with AWS

  • Set up multi factor identification (MFA) on root account and all IAM accounts
  • Common user access policies (distinct from security groups)
    • AdministratorAccess - almost everything root can do
    • PowerUserAccess - everything except IAM (i.e. except create new users & groups)
  • Security groups are firewalls for EC2 instances
    • Choose who the ports are open to using CIDR notation (IP/#, e.g. IP/32 is a single IP)

SSL certificate signing request generation

AWS TODO

  • AWS Certificate Manager has free certs
    • Supposedly they only work though with Elastic Load Balancing (and Amazon CloudFront)
    • We aren't using this because Play Framework is handling our encryption, not ELB. We'll still probably need to use ELB though. Here's the communication chain: user -> ELB -> ECS/EC2 -> Docker container -> Play
  • EC2 Guide: Hosting a website on Amazon EC2

Amazon Web Services

  • EC2 (Docker) Container Registry (ECR) - Alternative to using DockerHub as a container registry.
    • Store container images securely with Amazon ECR - Create and manage a new private image repository and use the Docker CLI to push and pull images. Access to the repository is managed through AWS Identity and Access Management.
  • EC2 Container Service (ECS) - Before you can run tasks in Amazon ECS, you need to launch container instances into your cluster. For more information about how to set up and launch container instances, see Setting Up with Amazon ECS and Getting Started with Amazon ECS.
  • AWS Docker Basics
  • The Amazon ECS instance and service roles are automatically created for you in the console first run experience, so if you intend to use the Amazon ECS console, you can move ahead to Create a Key Pair. If you do not intend to use the Amazon ECS console, and instead plan to use the AWS CLI, complete the procedures in Amazon ECS Container Instance IAM Role and Amazon ECS Service Scheduler IAM Role before launching container instances or using Elastic Load Balancing load balancers with services.
    • Note that if you plan to launch instances in multiple regions, you'll need to create a key pair in each region.
    • To connect to your Linux instance from a computer running Mac or Linux, specify the .pem (in ~/keys) file to your SSH client with the -i option and the path to your private key.
  • Amazon EC2 console
  • The Amazon ECS CLI supports Docker Compose [FWC - this is the ecs-cli command which is different from the aws command], a popular open-source tool for defining and running multi-container applications. For more information about installing and using the Amazon ECS CLI, see Using the Amazon ECS Command Line Interface.
    • ecs-cli configuration: $ ecs-cli configure --region us-east-1 --cluster hamstoo
  • Docker on Amazon Lightsail
  • Storing configuration information in a private bucket in Amazon S3 and granting read-only access to your container instance IAM role is a secure and convenient way to allow container instance configuration at launch time.
  • ecs-cli Command Reference
  • Allowing inbound SSH traffic via here ("If you try to connect to your instance and get an error message Network error: Connection timed out or Error connecting to [instance], reason: -> Connection timed out: connect"
  • Using Data Volumes in Tasks

Moving hosting from GoDaddy to AWS

  • "In order to move from GoDaddy to AWS, you can a) just move your code to AWS (if you have a static website, move it to S3 instead of EC2), and point your GoDaddy DNS records at your new host (e.g. your EC2 instance's IP address). In EC2, your instance's IP address will change when the instance reboots, etc. As such it is a dynamic IP address, not well suited for hosting a website. Instead, you need to allocate a static IP address, once that can be assigned to an instance - AWS call this an 'Elastic IP'. This is what you will use for your A record. (The same holds true whether you use GoDaddy's DNS or Route53 - you need an A record that points to the IP address of your server - but there is no requirement to use Route53 just because you are using AWS to host your site - there are some exceptions - e.g. using an elastic load balancer)."
  • You don't get to choose your elastic IP. You allocate one, and then associate it with the instance you want to use it with (you can move it between instances if needed). It remains the same until you release the IP address (even it is not associated with an instance). Just ensure you allocate an IP address in the same region (e.g. US East) and scope (i.e. EC2 or VPC) as the instance you want to associate it with.