Compute services

Deep dive serverless

• EC2
  • AMI
  • EC2 Image Builder
  • Placement groups
  • Pricing models
  • Spot instance
  • ASG
  • CLI
• Elastic Beanstalk
• ECS
• Fargate
• EKS
• ECR
• Lambda
  • Network settings
  • Lambda Storage Options
  • Limits
  • Versions
  • Environment variables
  • Invocations
    • Asynchronous invocation
    • S3 Event Notification
    • Event Source Mapping
  • Monitoring and Logging
  • Edge
• Other services

Virtual machine manager.

Fixed performance offers guaranteed number of vCPUs at all times (general purpose m5 family). Burstable performance (f.e. t3) provides baseline level of vCPUs with the ability to burst above the baseline - use 20% of available power and bank the rest for future use. Burstable has standard (once bank tokens are used, falls back to baseline level) and unlimited (if no tokens are available, charges on-demand pricing) modes of operation.

Instance types (comparison tool):

• general purpose
• compute optimized - batch processing, media transcoding, HPC, scientific modeling, machine learning, gaming servers
• memory optimized - databases, distributed web scale cache stores, real-time processing of big unstructured data
• accelerated computing (f.e. GPU, good for machine learning)
• storage optimized - high frequency OLTP (online transaction processing) systems, databases, distributed file systems, data warehousing apps

Instance type naming convention (f.e. m5.2xlarge):

• m - instance class
• 5 - generation (AWS improves it over time)
• 2xlarge - size

Root device (storage) types:

• Instance store - hard disk on host; all data is lost, when VM is stopped or terminated (ephemeral)
• EBS - network drive

Tenancy option:

• dedicated
  • dedicated instance - VM stays on host that is only used by you (physical host may change)
  • dedicated host - physical host is reserved
• multi-tenant

EC2 User Data - bootstrap script (runs as root).

EC2 Hibernate - stop option, which preserves RAM (so that app on VM can start up faster). Requires root volume to be EBS. Dumps RAM to root volume, which, therefore, must be encrypted. Use cases are long-running processing or services that take time to initialize. Not supported on all instance families.

vCPU configuration represents a thread that is running on CPU, therefore, multiple threads can be on a single CPU, providing more vCPUs. Number of cores and threads per core can be changed in launch configuration (see available options).

Capacity reservation can be used to ensure availability. Billed at the time of planned date. Requests are made per AZ. Can be combined with Reserved Instances and Savings Plan.

EC2 Enhanced Networking (SR-IOV) can be used to get higher bandwidth, higher PPS, and lower latency. Best current option is to use Elastic Network Adapter (up to 100Gpbs). Elastic Fiber Adapter gives even better performance, but works only with Linux (actually bypasses OS for connections) - good use for tightly coupled workloads.

Batch service runs and manages multi-node parallel jobs.

Compute Optimizer analyzes (machine learning) historical utilization data and provides recommendations on performance improvement and cost reductions.

Instance Metadata (IMDS) allows EC2 instance to learn about itself without using an IAM Role at http://169.254.169.254/latest/meta-data. Both metadata and userdata (launch script) are available. IMDSv1 is accessing the endpoint directly, while IMDSv2 (default in 2023) requires additional steps:

• Get session token

  TOKEN=`curl -X PUT "http://169.254.269.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"

• User requested token in IMDS requests:

  curl -X PUT "http://169.254.269.254/latest/metadata/profile" -H "X-aws-ec2-metadata-token: $TOKEN"

Customizable OS image for VM instances. Region level resource, however, can be copied over regions. To encrypt an existing unencrypted AMI follow copying process and specify encryption setting. Encryption can only be applied, but not removed.

The basic and free of charge is Amazon Linux 2 AMI. Can be downloaded as a VM (iso format) to run on VM software, such as VMWare, Hyper-v, etc, on-prem. Custom VM can also be imported to run on EC2 or be saved for DR.

Creates and tests AMI and container images. Can also create a new image automatically when software updates (base image) are available, and distribute images to other regions and or accounts as well.

Image Pipeline defines end-to-end configuration of building an image. This includes image recipe, infrastructure (EC2, VPC and other settings), distribution (regions and/or accounts to deliver the image) and testing.

Image Recipe defines source image and build components on top of it. Can be version controlled and shared. AWS also owns and manages set of build and test components that can be used, such as update-linux and simple-boot-test-linux.

For testing stage Image Builder launches an EC2 instance and performs configured components (scripts). At a minimum the following managed policies are required: EC2InstanceProfileForImageBuilder, AmazonSSMManagedInstanceCore. It also installs SSM agent, which can be set to be deleted after testing.

Specify how instances should be launched.

Available strategies are:

• cluster - low-latency group in a single AZ (same hardware); best network performance, but high risk of simultaneous failure; can be used for BigData job
• spread - each instance is placed on distinct rack; can span across multiple AZs within a region; max 7 instances per AZ per group; used for critical applications to avoid simultaneous failure
• partition - instances are grouped as partitions that are placed on distinct racks; can span across multiple AZs within a region; max 7 instances per AZ per group; partition information is available as VM metadata; use cases are Kafka, Cassandra, and so on

Reserved instances vs savings plan comparison:

• cloudhealthtech
• gorillastack

Instance size flexibility (applies only to Linux VMs) allows applying large VMs reservation to 2 medium sized VMs or about 50% of extra large VM.

If EBS is attached, and VM is stopped or terminated, AWS still charges for it. Charges also apply for any data transfer out of region.

Spot instance is launched via Spot request:

• maximum price
• desired number of instances
• launch specifications
• request type
  • one-time
  • persistent
• validity (from, until)

Canceling a Spot request does not terminate instances. First, cancel request, then terminate associated instances.

Spot fleet - a set of Spot and, optionally On-Demand instances. Specify possible launch pools: number of instances, type, OS, AZ, etc. AWS chooses best fitting pool and launches it. Spot block - specify the minimum duration (1 to 6 hours). AWS launches instances, only if it is guarantied to run for specified amount of time (in rare cases can still be interrupted).

Spot instance allocation strategies:

• lowest price - generally short workload, cheapest pool
• diversified - distributed across all pools, best for long workload, great for availability
• capacity optimized - pool with the optimal capacity

Defines min, max, and desired number of instances. Performs health checks on each instance. Exists within one or more AZs in a single region. Works with on-demand and spot instances. Launch template defines instance configuration. Free of charge, pay only for underlying resources being launched.

Attributes:

• Launch configuration
  • AMI and instance type
  • EC2 User Data
  • EBS volumes
  • Security Groups
  • SSH key pair
• min/desired/max size
• network and subnets information
• load balancer information
• scaling policies

Scaling can be triggered by CloudWatch alarms. Alarm can monitor any metrics, f.e. average CPU (metrics are computed as average across all instances). It is also possible to add custom metrics that application exposes, such as number of connected users.

ASG rules are various metrics that can be set directly in a group, such as average CPU, number of requests, etc.

Scaling policies:

• dynamic
  • target tracking (easiest for setup) - set selected metric to desired value, e.g. average CPU to 40%; automatically creates CloudWatch alarms
  • simple - execute scaling action based on custom CloudWatch alarm. Specify action and capacity units or percent of the group
  • step - similar to simple, but perform scaling action in steps
• predictive - scaling based on AWS continuous forecast, which is based on historical metrics data. Good metric to scale on are CPUUtilization, RequestCountPerTarget, Average Network In/Out.
• scheduled actions - change size settings on schedule

Scaling cooldown is a time period after scaling event, when ASG will not add or terminate additional instances to allow metrics to stabilize. Default is 300 seconds.

Termination policies:

• default - find AZ with most number of instances and delete the one with oldest launch configuration

Lifecycle hooks allow additional steps to be performed, while instance is in pending or terminating state.

Launch configuration is now legacy and is getting replaced by launch template. Configuration has to be recreated on any change, while template can have versions, create parameter subsets to be inherited and re-used, and allows provisioning of both On-Demand and Spot instances.

Instance Refresh replaces running instances with new ones (e.g. new version of launch template). Minimum healthy percentage setting specifies how many instances must be running at any point in time, warm-up time setting specifies how long to wait before assuming new instance is in ready state.

# Launch ec2
# Optional SSH key name
$ aws ec2 run-instances --subnet-id <subnet_id> --image-id <ami> --instance-type <instance_type> --key-name <key_name>

# Terminate instance
$ aws ec2 terminate-instances --instance-ids <instance_id>

Metadata is accessible inside an instance in the following URL. Append path to see metadata contents. Append slash, if path is a placeholder for other paths and values.

$ curl http://169.254.169.254/latest/meta-data/

Managed service that provides a platform (single interface) for deploying and scaling web apps and services. Free of charge - pay only for underlying resources. Supports applications written in Java, PHP, Python, Ruby, Go, .NET, and Node.js. Also supports Packer Builder, single, multi-container or preconfigured Docker; even custom platform (advanced).

Client is responsible for code and configuration settings, while Beanstalk manages necessary resources for deploying and running the application: capacity adjustment, load balancing, automatic scaling, app health monitoring (free), S3 buckets, RDS, etc.

Application is a collection of Beanstalk components with configurations. Application version is iteration of application. Environment is a collection of components running a particular application version (only one at a time is possible).

Environment tiers:

• web server environment - traditional web application with load balancer exposed to clients
• worker environment - exposed SQS queue receives tasks, while workers pull them

Worker environment can also run jobs on schedule. Requires cron.yaml file in the root directory to define schedule. More on worker environment.

RDS can be deployed as part of Beanstalk or separately. First option is useful for testing and quick deployment, because database will be deleted as part of Beanstalk environment. Production setup mostly deploys database separately. For external RDS deployment additional configuration must be supplied/specified, e.g. security group rules and connection string information (environment properties).

Docker integration allows deploying Docker container either to a single EC2 instances or ECS cluster.

Elastic Beanstalk Cloning feature allows quickly creating a duplicate environment (primary use is testing).

Documentation summary.

Deployment modes:

• single instance - includes a single EC2 instance with Elastic IP and optional RDS database
• HA with load balancer - includes all components for production deployment - load balancer, HA RDS, ASG across multiple AZs

Use Buildfile, Procfile and/or platform hooks to configure and run custom code during provisioning.

Buildfile is used for commands that run for short period and exit upon completion. Should be placed in the root directory and follow the format <process_name>:<command>, e.g. make: ./build.sh.

Procfile is used for long-running processes, e.g. your application. Location and format is same as for Buildfile. Specified commands are run continuously, are being monitored and restarted if process terminates.

Platform hooks are used for running custom scripts during specified stage of EC2 provisioning process. Depending on the stage, files must be placed in different directory:

• .platform/hooks/prebuild - before configuring and setting up application and web server
• .platform/hooks/predeploy - after configuring and setting up application and web server, but before their deployment
• .platform/hooks/postdeploy - after application has been deployed

Beanstalk can store at most 1000 application versions per account. Lifecycle policy is used for removing old versions - based on time or space (too many versions). A separate option can still keep the bundle in S3 for recovery.

All parameters set in UI can be configured via code - .ebextensions directory. Config file supports JSON and YAML, file extension must be .config. Can also contain CloudFormation resources to provision any additional AWS resources.

Container management service.

Use cases:

• long running apps
• batch processing
• microservices

Containers are defined in task definition. Tasks can be run individually or as part of a service, which can manage multiple tasks simultaneously. Tasks can be invoked by EventBridge.

Can be integrated with EFS for persistent multi-AZ shared storage.

ECS launch type - client manages and provisions the infrastructure (EC2 instances). ECS agent is launched on agents (uses preconfigured AMI) and gets registered with ECS cluster.

If task is launched without a specific port, it is assigned a random port on EC2 instance. ALB can still forward traffic to tasks in a group by registering with a service and using dynamic port mapping. That means any port must be allowed on EC2 instance for incoming traffic from ALB.

Fargate launch type - containers are launched on the platform. Each task gets its own unique ENI, thus, unique IP. For load balancing ENI security group must allow traffic on exposed port for ALB.

Up to 10 containers can be defined in a single task definition.

Metadata in JSON format that describes how to run a Docker container.

• image name
• port binding (container and optionally host binding for EC2 launch type)
• memory, CPU requirements
• environment variables
• network info
• IAM role
• logging configuration

If host port mapping is skipped, Dynamic Host Port mapping is assigned. ALB still finds the correct port due to integration.

Data Volumes (Bind Volumes) is a way to share storage between multiple containers. EC2 Tasks use EC2 storage, and, therefore, tied to EC2 lifecycle. Fargate Tasks use ephemeral storage, which is tied to containers using them.

Defines where to place new tasks based on available CPU, memory and ports. Works only with EC2 launch configuration.

Task scheduling follows the process below (in the same order):

1. Identify instances that satisfy CPU, memory and ports requirements
2. Identify instances that satisfy task placement constraints
3. Identify instances that satisfy _task placement strategy

Task placement strategies:

• Binpack - least available CPU and memory; helps minimizing number of instances in use
• Random
• Spread - place tasks evenly based on specific value - instanceId, AZ, etc

Task placement constraints:

• distinctInstance - place each task on different container instance
• memberOf - place task on instances that satisfy an expression (Cluster Query Language)

Instance profile (role) is used by ECS agent to make calls to ECS, send container logs to CloudWatch, pull images from ECR, fetch secrets from Secrets Manager or SSM Parameter Store.

ECS Task Role is attached directly to the task providing specific permissions required for application. Best practice is to create a separate role for each service. Role is defined in task definition.

Automatically increase/decrease the desired number of ECS tasks. Service scaling changes number of tasks in the cluster. Service CPU Usage CloudWatch metric can be used to trigger scaling - to add or remove task.

ECS Auto Scaling uses AWS Application Auto Scaling, which exposes 3 metrics:

• ECS Service average CPU
• ECS Service average memory utilization
• ALB request count per target

Scaling strategies:

• Target Tracking - based on specific CloudWatch metric
• Step Scaling - based on CloudWatch Alarm
• Scheduled Scaling - based on specified date/time

Scale infrastructure to accommodate ECS Service Auto Scaling.

• ASG scaling
• ECS Cluster Capacity Provider - automatically provision and scale infrastructure based ECS Tasks.

ECS version update can control how many tasks can be started and stopped, and in which order. Represented with 2 settings (actual running capacity is 100%):

• minimum healthy percent - if less than 100%, allows ECS terminate some portion of current version
• maximum percent - indicates how many tasks with the new version can be launched

Serverless platform for running containers. AWS runs containers based on CPU/RAM requirements.

Manages server infrastructure (cluster) for ECS or EKS.

Manager Kubernetes cluster.

Supported node types:

• Managed Node Group - EKS manages ASG
• Self-managed Node Group - create, manage and register nodes in the cluster yourself; can still leverage ASG and use prebuilt AMIs as well
• Fargate

Data volumes are exposed via Kubernetes StorageClass, supported services are:

• EBS
• EFS (works also with Fargate)
• FSx for Lustre
• FSx for NetApp ONTAP

Provides private and public repositories (http://gallery.ecr.aws). Images are backed by S3.

# Acquire login credentials
$ aws ecr get-login-password --region <region> | docker login --username AWS
--password-stdin <aws_account_id>.dkr.ecr.<region>.amazonaws.com

Serverless event-driven compute service. Runs virtual functions on demand. Pricing is based on # of requests, compute time and memory amount used.

Use cases:

• backend processing
• event processing
• stream processing
• data processing

Supported languages:

• Node.js
• Python
• Java
• C# (.NET or Powershell)
• Go
• Ruby
• Custom Runtime API (open source)

Lambda Container Image can be used to run code in other languages (base image must implement Lambda Runtime API). Maximum image size is 10GB, ECR is used to host the image. Base images already exist for common languages, e.g. Python Go, Ruby, custom one also can be created. Containers can be tested locally using Lambda Runtime Interface Emulator.

Handler (setting) - <filename_without extention>.<function_name>.

Event object is a JSON document containing all the details on the original event that Lambda function will process, e.g. source name, parameters. Lambda runtime converts this document to corresponding object depending on runtime, e.g. dict in Python. Context object is a JSON document describing Lambda function itself, e.g. request ID, memory limit, log group name. It is also passed to function at runtime.

Function URL is a unique URL endpoint that doesn't change. Helps to expose Lambda function without API Gateway or ALB. This URL can only be accessed through public internet. Can be applied to $LATEST or aliases, but not to specific versions. Resource-based policy can configure which AWS accounts, CIDR block or IAM entities can trigger this function. Available security settings (AuthType):

• None - allow public and unauthenticated access, only resource based policy is in effect
• AWS_IAM - IAM is used to authenticate and authorize requests, principal must have lambda:InvokeFunctionURL permission

By default, executes in AWS owned VPC, which grants it access to public internet, and, therefore, all public endpoints and services of AWS. Optionally enable VPC access, if Lambda needs to interact with private resources - subnet ID and security group ID are needed (ENI is set up in private VPC to enable connection to resources within this VPC). This also makes the function a subject to routing rules.

Deploying Lambda function in public subnet does not automatically assign public IP and internet access to it. Internet access can be configured by routing outbound traffic to NATGW.

Long-term storage options include S3, EFS, and AWS database options. For EFS both function and EFS must reside in the same VPC.

External libraries and dependencies can also be packaged together with the source code, and, therefore, be included in a deployment package, though layers are preferred. Deployment package size (total) is applied both to all files that are uploaded the same way (either in layer or zip package form). For deployment package greater than 50MB, store it in S3 first, then reference in Lambda function service.

Lambda Execution Context is a temporary runtime environment that consists of all dependencies, such as HTTP or SDKs clients, database connections and so on. Execution Context is maintained after Lambda function has finished in anticipation of another Lambda function execution.

Static initialization is part of Execution Context and is the 3rd phase (after downloading code, configuring runtime and other configurations), where external libraries and dependencies are imported, and external connections (clients) are set. To reduce initialization time it is recommended to initialize clients and importing libraries outside of handler function, so it can be re-used across invocations. Also import only necessary dependencies from SDKs.

Execution context also includes /tmp directory - can be used to download a big file or share files across invocations that share execution environment (cache). Has it's own lifetime, but data stored there should be assumed not to be available after a single function execution.

Use cases:

• Custom runtimes (C++, Rust, etc).
• Externalize dependencies to re-use them - enabled faster update of core logic and sharing common libraries across multiple functions

Maximum 5 layers per function, maximum 250MB total (maximum Lambda package size).

Code + dependencies (uncompressed) - max 250MB.

There is also a concurrent execution limit across all functions in a region, which acts as a safety feature. Default is 1,000 per region; if exceeded TooManyRequestsException error is shown with status code 429 for synchronous invocation, for async see below. Can be increased through request to AWS support. Reserved concurrency guarantees that a set number of executions is always available, while restricting other concurrent requests, but also acts as a limit. Once one function reaches the limit of concurrent executions, it also throttles other functions in the same region. Currently unreserved capacity (shared across all functions in a given region) automatically gets increased based on usage, but could also be raised through AWS support request.

$LATEST marks the latest code (version) uploaded to Lambda. Each alteration (code or attributes) create a new version. Version is immutable - code, environment variables and other attributes can not be changed.

Aliases can be used to reference different versions of code, e.g. development/prod environments. First, a new version has to be published (AWS also exposes it via unique ARN, which include version number at the end). Then an alias can be created, which points to specific version. AWS also creates another ARN for alias. Alias can only reference a version, not another alias.

Weighted alias defines percentage-wise how much traffic will be served between specified versions to be able to test changes without full roll out.

Environment variables are a key/value pairs in a string form. Lambda service adds system environment variables as well. Secrets can also be stored as environment variables; can be encrypted by the Lambda service key or CMK.

Synchronous invocation waits for the returned result. Error handling must be performed on client side (retries, etc). Triggered by user (CLI, SDK, API Gateway, ALB, CloudFront, etc), service (AWS Cognito, AWS Step Functions) or other services (Amazon Alexa, Kinesis Data Firehose).

Lambda can be exposed as an HTTP(S) endpoint via ALB or API Gateway. For ALB it must be registered in a target group; HTTP payload will be transformed into JSON document. Lambda should return a JSON, which ALB will convert to HTTP (so initial JSON should follow some rules). ALB multi-header values feature can be enabled to allow passing multiple values for HTTP headers or query strings, e.g. example.com/path?key=value1&key=value2. This will be converted to an array - {"key": ["value1", "value2"]}.

Destinations can be configured to report successful or failed events for asynchronous invocations. Available services are SNS, SQS, Lambda, and EventBridge (recommended over DLQ, as it offers more services). For Event Source Mapping it can be set only for discarded event batches; supported services are SNS and SQS.

Asynchronous invocation works with number of AWS services, such as S3, SNS, CloudWatch Events, etc. Events are registered in the internal Event Queue, which Lambda is reading from.

On error events Lambda performs maximum (can be configured anywhere from 0 to 2) 2 more attempts (total of 3 attempts): 1 minute after first attempt, 2 minutes after 2 attempt. This means that single event may be processed 3 times, thus, Lambda function should be idempotent. DLQ can be additionally set up (SNS or SQS) to process failed attempts later. Lambda function will need additional IAM permission to be able to perform SendMessage call on SQS. DLQ is associated with a particular version of a function, and can be used as event source for another function (to be re-processed later).

Destinations (recommended) allows configuring destinations for successful and failed events, which can be SQS, SNS, another Lambda, or EventBridge. Also contains invocation record, which includes response payload as well, as opposed to DLQ, which doesn't contain any information at all (just notifies about failed event).

Typical patterns include delivering directly to Lambda function asynchronously or to SQS queue, which is later read by Lambda function. Events are Typically delivered in seconds, but could take a minute or longer as well. For non-versioned objects this could result in a single event for 2 writes that are made at the same time. To ensure each action results in an event, enable bucket versioning.

Supported on:

• Kinesis Data Streams
• SQS and SQS FIFO queue
• DynamoDB Streams

Lambda pulls (synchronously) data from source. Internally Event Source Mapping within Lambda services polls the source; upon receiving a batch it invokes Lambda function with event batch.

By default, if function returns an error, the entire batch is reprocessed until function succeeds or items in the batch expire. To ensure in-order processing for shards, processing for the affected shared is paused until error is resolved; one of the following resolutions can be performed:

• discard old events (can go to destination) - SQS or SNS
• restrict number of retries
• split the batch

For queues a long polling is used, where batch size can also be specified. Recommended queue visibility timeout is 6x the Lambda function timeout. DLQ can be used only on the queue, since synchronous Lambda function doesn't have such a feature.

Execution logs are stored in CloudWatch Logs; make sure Lambda function has IAM permissions to write to CloudWatch Logs.

Metrics (invocations, duration, concurrent executions, error counts, success rate throttles, async delivery failures, iterator age for Kinesis and DynamoDB Streams) are displayed in CloudWatch metrics.

X-Ray tracing can also be enabled, which is called Active Tracing, which runs X-Ray daemon for you. Ensure Lambda function has correct permissions, namely AWSXRAYDaemonWriteAccess managed policy. The following environment variables allow communication with the daemon:

• _X_AMZN_TRACE_ID
• AWS_XRAY_CONTEXT_MISSING - default is LOG_ERROR
• AWS_XRAY_DAEMON_ADDRESS - in the form of <ip_address>:<port>

Deploy function alongside CloudFront instead of a specific region. Lambda can be used to change requests and responses of CloudFront:

• viewer request - modify client's request
• origin request - modify CloudFront's request
• origin response - modify origin's response to CloudFront
• viewer response - modify CloudFront response to client

Use cases:

• web site security and privacy
• dynamic web app
• search engine optimization (SEO)
• intelligent routing across origins and data centers
• real-time image transformation
• A/B testing
• authentication and authorization
• user tracking and analytics

• LightSail

Preconfigured OSs, web apps, and development stacks. Popular choices are Wordpress, Windows OS, Ubuntu, Node.js.