SES SQS SNS RDS EC2 S3 Cloudfront Lambda - sgml/signature GitHub Wiki

          +----------------------+     +----------------------+
          |   Client Request     |     |     API Gateway      |
          |  (HTTP POST/GET)     |     |  (REST or HTTP API)  |
          +----------+-----------+     +----------+-----------+
                     |                         |
                     +-----------+-------------+
                                 |
                                 v
                        +----------------------+
                        |      Lambda          |
                        |  (Ephemeral Runtime) |
                        +----------+-----------+
                                   |
                +------------------+------------------+
                |                                     |
                v                                     v
      +------------------+               +------------------------+
      | Check /tmp/sqlite|               |  Recreate SQLite DB    |
      |   exists?        |               |  (if missing or stale) |
      +--------+---------+               +-----------+------------+
               |                                     |
               v                                     v
      +------------------------+         +------------------------+
      |  Write/Read to SQLite  |<--------|  Init schema + tables  |
      |  in /tmp/sqlite.db     |         |  (CREATE TABLE etc.)   |
      +------------------------+         +------------------------+
               |
               v
      +------------------------+
      |  Return response to    |
      |  API Gateway           |
      +------------------------+
               |
               v
      +------------------------+
      |  Response to Client    |
      +------------------------+

| AWS Service            | Spring MVC Equivalent              |
|------------------------|------------------------------------|
| EC2 Instance           | Application Host / Tomcat Server   |
| RDS (e.g., MySQL)      | Spring Data JPA + MySQL            |
| EFS                    | Shared File Storage (e.g., NFS)    |
| VPC                    | Network Boundary / DMZ             |
| Subnets                | Deployment Zones / Availability    |
| ELB (Load Balancer)    | Reverse Proxy / DispatcherServlet  |
| Security Groups        | Firewall Rules / Access Filters    |
| IAM Roles              | Spring Security Roles              |
| CloudWatch             | Actuator + Logging Frameworks      |
| Elastic Beanstalk      | Spring Boot Deployment Automation  |

testing_serverless_code:
  options:
    - name: OpenFaaS
      url: https://www.openfaas.com
      description: This open-source platform allows you to build, deploy, and test serverless functions using Docker containers. It works seamlessly on Windows.
    - name: VirtualBox
      url: https://www.virtualbox.org
      description: Create a Linux virtual machine to emulate your production environment and test your serverless code.
    - name: VMware
      url: https://www.vmware.com
      description: Create a Linux virtual machine to emulate your production environment and test your serverless code.
    - name: LocalStack
      url: https://localstack.cloud
      description: Emulates a complete cloud stack locally, allowing you to run and test various cloud services on your Windows machine.
    - name: Fn Project
      url: https://fnproject.io
      description: An open-source container-native serverless platform that can be run locally using Docker. It supports multiple languages and offers a flexible runtime.

(lambda-invoke-workflow
  ;; name: Secure Lambda Invocation
  ;; on:
  ;;   workflow_dispatch:

  (workflow
    (name "Secure Lambda Invocation")
    (trigger workflow_dispatch))

  ;; jobs:
  ;;   invoke-lambda:
  ;;     runs-on: ubuntu-latest
  ;;     permissions:
  ;;       id-token: write
  ;;       contents: read

  (job invoke-lambda
    (runner ubuntu-latest)
    (permissions
      (id-token write)
      (contents read))

    ;; steps:
    ;;   - name: Configure AWS credentials via OIDC
    ;;     uses: aws-actions/configure-aws-credentials@v2
    ;;     with:
    ;;       role-to-assume: arn:aws:iam::123456789012:role/GitHubOIDCRole
    ;;       aws-region: us-west-2

    (step configure-aws-credentials
      (uses aws-actions/configure-aws-credentials@v2)
      (with
        (role-to-assume "arn:aws:iam::123456789012:role/GitHubOIDCRole")
        (aws-region "us-west-2")))

    ;;   - name: Invoke Lambda
    ;;     run: |
    ;;       aws lambda invoke \
    ;;         --function-name my-lambda-function \
    ;;         --payload '{"key": "value"}' \
    ;;         response.json
    ;;       cat response.json

    (step invoke-lambda
      (run
        (aws lambda invoke
          (function-name "my-lambda-function")
          (payload "{\"key\": \"value\"}")
          (output "response.json"))
        (cat "response.json")))))

;; IAM Role Setup (for reference)

(iam-role GitHubOIDCRole
  ;; Trust Policy
  ;; {
  ;;   "Version": "2012-10-17",
  ;;   "Statement": [
  ;;     {
  ;;       "Effect": "Allow",
  ;;       "Principal": {
  ;;         "Federated": "arn:aws:iam::123456789012:oidc-provider/token.actions.githubusercontent.com"
  ;;       },
  ;;       "Action": "sts:AssumeRoleWithWebIdentity",
  ;;       "Condition": {
  ;;         "StringEquals": {
  ;;           "token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
  ;;           "token.actions.githubusercontent.com:sub": "repo:your-org/your-repo:ref:refs/heads/main"
  ;;         }
  ;;       }
  ;;     }
  ;;   ]
  ;; }

  (trust-policy
    (principal
      (federated "arn:aws:iam::123456789012:oidc-provider/token.actions.githubusercontent.com"))
    (action sts:AssumeRoleWithWebIdentity)
    (condition
      (StringEquals
        ("aud" "sts.amazonaws.com")
        ("sub" "repo:your-org/your-repo:ref:refs/heads/main"))))

  ;; Permissions Policy
  ;; {
  ;;   "Version": "2012-10-17",
  ;;   "Statement": [
  ;;     {
  ;;       "Effect": "Allow",
  ;;       "Action": "lambda:InvokeFunction",
  ;;       "Resource": "arn:aws:lambda:us-west-2:123456789012:function:my-lambda-function"
  ;;     }
  ;;   ]
  ;; }

  (permissions-policy
    (action lambda:InvokeFunction)
    (resource "arn:aws:lambda:us-west-2:123456789012:function:my-lambda-function")))

+-------------------------------------------------------------+
|                           VPC                               |
|                                                             |
|  +-------------------+     +-------------------+            |
|  |   Public Subnet   |     |   Private Subnet  |            |
|  |  (ELB, NAT GW)    |     | (EC2, RDS, EFS)   |            |
|  +-------------------+     +-------------------+            |
|         |                         |                         |
|         |                         |                         |
|   +-------------+         +------------------+              |
|   | Elastic     |         | Elastic Beanstalk|              |
|   | Load Balancer|        | EC2 Instances     |             |
|   +-------------+         +------------------+              |
|         |                         |                         |
|         |                         |                         |
|   +-------------------------------+                         |
|   |       Security Group(s)       |                         |
|   |  - ELB SG (ingress: 80/443)   |                         |
|   |  - EC2 SG (ingress: from ELB) |                         |
|   |  - EFS SG (ingress: 2049)     |                         |
|   +-------------------------------+                         |
|                                                             |
+-------------------------------------------------------------+

Relationship Breakdown:
-----------------------
VPC:
  - Encloses all networking components and resources.

Subnets:
  - Public Subnet: Hosts ELB and NAT Gateway.
  - Private Subnet: Hosts EC2 (Beanstalk), RDS, and EFS.

Security Groups:
  - ELB SG: Allows inbound HTTP/HTTPS traffic from the internet.
  - EC2 SG: Allows inbound traffic from ELB SG.
  - EFS SG: Allows NFS traffic (port 2049) from EC2 SG.

Elastic Beanstalk:
  - Deploys EC2 instances into private subnets.
  - Attaches EC2 SG and optionally mounts EFS volumes.
  - Can configure ELB in public subnet to route traffic to EC2.

Elastic Load Balancer:
  - Public-facing; forwards traffic to EC2 instances in private subnet.
  - Associated with ELB SG.

EFS (optional):
  - Mount targets created in private subnets.
  - Secured via EFS SG allowing access from EC2 SG.

<!-- Step 1: Clone Blue Database to Create Green Database -->
<xsl:template name="clone-database">
    <xsl:param name="blueDatabase" />
    <xsl:variable name="greenDatabase">
        <xsl:copy-of select="$blueDatabase" />
    </xsl:variable>
</xsl:template>

<!-- Step 2: Apply Schema Changes to Green Database -->
<xsl:template name="apply-schema-changes">
    <xsl:param name="greenDatabase" />
    <xsl:variable name="updatedGreenDatabase">
        <!-- Define schema changes -->
        <xsl:copy>
            <!-- Add or modify tables, fields, etc. -->
            <table name="NewTable">
                <field name="NewField" type="varchar" />
            </table>
        </xsl:copy>
    </xsl:variable>
</xsl:template>

<!-- Step 3: Validate and Test Green Database -->
<xsl:template name="validate-database">
    <xsl:param name="greenDatabase" />
    <xsl:choose>
        <xsl:when test="validation($greenDatabase)">
            <xsl:text>Validation Successful</xsl:text>
        </xsl:when>
        <xsl:otherwise>
            <xsl:text>Validation Failed</xsl:text>
        </xsl:otherwise>
    </xsl:choose>
</xsl:template>

<!-- Step 4: Promote Green Database to Production -->
<xsl:template name="promote-to-production">
    <xsl:param name="greenDatabase" />
    <xsl:variable name="productionDatabase" select="$greenDatabase" />
</xsl:template>

<!-- Step 5: Monitor Green Database and Roll Back if Needed -->
<xsl:template name="monitor-and-rollback">
    <xsl:param name="greenDatabase" />
    <xsl:param name="blueDatabase" />
    <xsl:choose>
        <xsl:when test="monitor($greenDatabase) = 'issues_detected'">
            <xsl:variable name="activeDatabase" select="$blueDatabase" />
            <xsl:text>Rollback Completed</xsl:text>
        </xsl:when>
        <xsl:otherwise>
            <xsl:text>Migration Successful</xsl:text>
        </xsl:otherwise>
    </xsl:choose>
</xsl:template>

stages:
  - stage: "Upgrade from PostgreSQL 11 to PostgreSQL 12"
    steps:
      - step: "Preparation and Backup"
        description: >
          Identify the current production RDS instance (blue environment) running PostgreSQL 11 and take a snapshot for a reliable rollback option.
        url: "https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_CommonTasks.BackupRestore.html"
      - step: "Compatibility Review"
        description: >
          Review AWS documentation and PostgreSQL 12 release notes to confirm that your application, extensions, and configurations are supported.
        url: "https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_UpgradeDBInstance.Upgrading.html"
      - step: "Initiate Blue/Green Deployment"
        description: >
          Use the RDS Blue/Green Deployments feature to create a green environment that mirrors your current production (blue) instance.
        url: "https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/blue-green-deployments-overview.html"
      - step: "Apply Upgrade in Green Environment"
        description: >
          In the green environment, trigger the major engine upgrade from PostgreSQL 11 to PostgreSQL 12 and adjust parameter groups if necessary.
        url: "https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_UpgradeDBInstance.Upgrading.html"
      - step: "Testing and Validation"
        description: >
          Run functional, integration, and performance tests using AWS monitoring tools to validate that PostgreSQL 12 is operating as expected.
        url: "https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_Monitoring.html"
      - step: "Switchover to Production"
        description: >
          Execute the blue/green switchover to route production traffic to the upgraded PostgreSQL 12 (green) environment with minimal downtime.
        url: "https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/blue-green-deployments-overview.html"

  - stage: "Upgrade from PostgreSQL 12 to Latest Version"
    steps:
      - step: "Preparation and Snapshot"
        description: >
          With PostgreSQL 12 now in production, create a fresh snapshot to serve as a rollback baseline before initiating the next upgrade.
        url: "https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_CommonTasks.BackupRestore.html"
      - step: "Review New Version Compatibility"
        description: >
          Verify that your application and extensions are compatible with the latest PostgreSQL version by reviewing release notes and AWS documents.
        url: "https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_UpgradeDBInstance.Upgrading.html"
      - step: "Initiate Second Blue/Green Deployment"
        description: >
          Trigger a new blue/green deployment from the PostgreSQL 12 environment to create a fresh green environment for the upgrade.
        url: "https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/blue-green-deployments-overview.html"
      - step: "Apply Upgrade in the Second Green Environment"
        description: >
          Upgrade the green environment from PostgreSQL 12 to the latest PostgreSQL version available, updating parameter groups as required.
        url: "https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_UpgradeDBInstance.Upgrading.html"
      - step: "Comprehensive Testing"
        description: >
          Conduct comprehensive integration, functional, and performance tests to ensure the upgraded environment meets all operational requirements.
        url: "https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PerfInsights.html"
      - step: "Final Switchover"
        description: >
          Complete the final switchover by routing production traffic to the green environment running the latest PostgreSQL version.
        url: "https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/blue-green-deployments-overview.html"

def create_instance_from_ami_vpc(self, ami_id, key_name, instance_type='t2.micro'):
    """
    Finds an existing instance launched from the given AMI ID,
    retrieves its VPC ID, and launches a new instance in the same VPC.
    
    :param ami_id: AMI ID to base the new instance on
    :param key_name: Key pair name
    :param instance_type: EC2 instance type
    :return: New instance ID or None
    """
    try:
        # Step 1: Find an instance using the AMI
        response = self.ec2_client.describe_instances(
            Filters=[{'Name': 'image-id', 'Values': [ami_id]}]
        )
        reservations = response.get('Reservations', [])
        if not reservations:
            print(f"No instances found using AMI {ami_id}")
            return None

        instance = reservations[0]['Instances'][0]
        vpc_id = instance['VpcId']
        print(f"Found VPC ID {vpc_id} from instance {instance['InstanceId']}")

        # Step 2: Find a subnet in that VPC
        subnets = self.ec2_client.describe_subnets(
            Filters=[{'Name': 'vpc-id', 'Values': [vpc_id]}]
        )
        subnet_id = subnets['Subnets'][0]['SubnetId']

        # Step 3: Launch new instance
        new_instance = self.ec2_client.run_instances(
            ImageId=ami_id,
            InstanceType=instance_type,
            KeyName=key_name,
            SubnetId=subnet_id,
            MinCount=1,
            MaxCount=1
        )
        new_instance_id = new_instance['Instances'][0]['InstanceId']
        print(f"Launched new instance {new_instance_id} in VPC {vpc_id}")
        return new_instance_id

    except Exception as e:
        print(f"Error creating instance from AMI VPC: {e}")
        return None

import boto3
import subprocess
import os
from crontab import CronTab

class WordPressProvisioner:
    def __init__(self, region="us-west-2", key_name="wordpress-key", secret_name="wordpress-ec2-key", notification_email="[email protected]"):
        self.region = region
        self.key_name = key_name
        self.secret_name = secret_name
        self.notification_email = notification_email
        self.pem_content = None
        self.instance_id = None
        self.ec2 = boto3.client("ec2", region_name=self.region)
        self.secrets = boto3.client("secretsmanager", region_name=self.region)
        self.cloudwatch = boto3.client("cloudwatch", region_name=self.region)
        self.sns = boto3.client("sns", region_name=self.region)
        self.topic_arn = None

    # Step 0: Create EC2 instance and store PEM + instance ID
    def create_ec2_instance(self):
        key_pair = self.ec2.create_key_pair(KeyName=self.key_name)
        self.pem_content = key_pair["KeyMaterial"]

        instance = self.ec2.run_instances(
            ImageId="ami-0c55b159cbfafe1f0",
            InstanceType="t2.micro",
            KeyName=self.key_name,
            MinCount=1,
            MaxCount=1,
            SecurityGroups=["default"]
        )
        self.instance_id = instance["Instances"][0]["InstanceId"]
        print("EC2 instance created:", self.instance_id)

    # Step 1: Store PEM in Secrets Manager
    def store_pem_in_secrets_manager(self):
        if not self.pem_content:
            raise ValueError("PEM key not found in memory. Run create_ec2_instance() first.")
        try:
            self.secrets.create_secret(Name=self.secret_name, SecretString=self.pem_content)
        except self.secrets.exceptions.ResourceExistsException:
            self.secrets.put_secret_value(SecretId=self.secret_name, SecretString=self.pem_content)
        print("PEM key stored in Secrets Manager as:", self.secret_name)

    # Step 2: Instructions to SSH and install WordPress
    def install_wordpress(self):
        """
        Manual Instructions:
        1. Retrieve PEM from Secrets Manager:
           aws secretsmanager get-secret-value --secret-id wordpress-ec2-key --query 'SecretString' --output text > temp.pem
           chmod 400 temp.pem

        2. Get EC2 public IP:
           aws ec2 describe-instances --instance-ids <instance_id> --query 'Reservations[0].Instances[0].PublicIpAddress' --output text

        3. SSH into EC2:
           ssh -i temp.pem ec2-user@<public_ip>

        4. Install WordPress manually on EC2.
        """

    # Step 3: Instructions to update WordPress using plugin
    def generate_update_script(self):
        """
        Manual Instructions:
        1. Log into WordPress admin.
        2. Install "Easy Updates Manager" plugin.
        3. Enable automatic updates for core, plugins, and themes.
        """

    # Step 4: Create weekly cron job (optional)
    def create_cron_job(self):
        cron = CronTab(user=True)
        job = cron.new(command=f"bash {os.path.abspath('update_wordpress.sh')}", comment="Weekly WordPress Update")
        job.setall("0 3 * * 1")
        cron.write()
        print("Cron job scheduled for weekly WordPress update")

    # Step 5: Terminate EC2 instance
    def terminate_ec2_instance(self):
        if not self.instance_id:
            raise ValueError("Instance ID not set. Run create_ec2_instance() first.")
        self.ec2.terminate_instances(InstanceIds=[self.instance_id])
        print("EC2 instance termination initiated:", self.instance_id)

    # Step 6: Create CloudWatch alarm for EC2 CPU usage
    def create_cloudwatch_alarm(self):
        alarm_name = f"HighCPU-{self.instance_id}"
        self.cloudwatch.put_metric_alarm(
            AlarmName=alarm_name,
            ComparisonOperator="GreaterThanThreshold",
            EvaluationPeriods=1,
            MetricName="CPUUtilization",
            Namespace="AWS/EC2",
            Period=300,
            Statistic="Average",
            Threshold=70.0,
            ActionsEnabled=True,
            AlarmActions=[self.topic_arn] if self.topic_arn else [],
            AlarmDescription="Alarm when CPU exceeds 70 percent",
            Dimensions=[{"Name": "InstanceId", "Value": self.instance_id}]
        )
        print("CloudWatch alarm created:", alarm_name)

    # Step 7: Create SNS topic and subscribe email
    def setup_sns_notifications(self):
        topic = self.sns.create_topic(Name="WordPressAlerts")
        self.topic_arn = topic["TopicArn"]

        self.sns.subscribe(
            TopicArn=self.topic_arn,
            Protocol="email",
            Endpoint=self.notification_email
        )
        print("SNS topic created and email subscription sent to:", self.notification_email)
        print("Note: Confirm the subscription via email before alarms will trigger notifications.")

# Usage
if __name__ == "__main__":
    wp = WordPressProvisioner()
    wp.create_ec2_instance()
    wp.store_pem_in_secrets_manager()
    wp.install_wordpress()
    wp.generate_update_script()
    wp.create_cron_job()
    wp.setup_sns_notifications()
    wp.create_cloudwatch_alarm()
    # wp.terminate_ec2_instance()

+-------------------------------------------------------------+
| Elastic Beanstalk Recovery & Debug Checklist                |
+-------------------------------------------------------------+

[ ] Review logs from the failing instance
    - Elastic Beanstalk Console > Logs > Request Logs > Instance Logs
    - Look for:
        • App crash traces
        • Port binding errors
        • Missing environment variables
        • Composer or Docker errors

[ ] Check .ebextensions and container commands
    - Ensure all commands are idempotent
    - Avoid assumptions:
        • Pre-existing directories
        • Mounted volumes
    - Use `|| true` to prevent non-critical failures:
        container_commands:
          01_setup:
            command: "mkdir -p /var/app/data || true"

[ ] Rebuild environment (no database binding)
    - Pros:
        • Clears corrupted deployments
        • Resets configuration drift
        • Reprovisions EC2 instances
        • Tests idempotency
    - Cons:
        • Causes temporary downtime
        • Ephemeral data is lost
        • Requires redeployment if version is not pinned
        • May mask deeper configuration issues

[ ] Terminate EC2 instance manually
    - Pros:
        • Triggers Elastic Beanstalk to replace the instance
        • Faster than full rebuild
        • Preserves environment config
    - Cons:
        • Doesn’t fix root cause
        • May cause load imbalance
        • Risk of repeated failures

[ ] Identify EC2 instance with incorrect application version
    - EC2 Console > Filter by Elastic Beanstalk tags
    - Compare deployed version in Elastic Beanstalk console
    - Terminate mismatched instance via EC2 Console
    - Elastic Beanstalk will auto-replace it

[ ] Fix 502 Bad Gateway after severe state
    - Elastic Beanstalk Console > Check environment health
    - Look for “Severe” or “Degraded” status
    - View logs to confirm proxy or app failure
    - If isolated: terminate instance via EC2 Console
    - If systemic: use “Rebuild Environment” from Actions menu
    - Redeploy correct application version
    - Confirm health returns to “Green”