DevOps - GregLinthicum/From-Logistic-Regression-to-Long-short-term-memory-RNN GitHub Wiki
AWS CLI, Describe instance, Add Internet Gateway
aws ec2 describe-instances --instance-ids i-1234567890abcdef0
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb sudo apt install ./google-chrome-stable_current_amd64.deb
CD/CI
CDK (1.90.1) with Typescript, CDK guide, API, API 1.129.0. There is no charge for using the AWS CDK. CDK Lambda
AWS Copilot for AWS App Runner, Amazon ECS, and AWS Fargate AWS Copilot provides a simple declarative set of commands to quickly launch and easily manage containerized applications on AWS. AWS Copilot pipeline.yml
AWS Proton AWS Proton is a fully managed delivery service for deploying container and serverless applications. Proton is not like Beanstalk or App Runner with which one can start with code or an image and then get a running service. Proton solves a very different problem. It helps to set, update, and monitor all of that infra, deployments, and automation and allow others to self service. Proton sits on top of Fargate (or other AWS services like Lambda) and offers a broader scope. It simplifies the deployment and management of containerized and serverless applications by providing infrastructure provisioning, automated deployments, and centralized control.
ROSA, Red Hat OpenShift Service on AWS (ROSA) is a managed Kubernetes service that combines the power of OpenShift with the scalability and reliability of Amazon Web Services (AWS). ROSA supports cloud-native development with tools like Quarkus and Buildah (OCI image builder). It simplifies application deployment using its internal registry. Quinoa is a Quarkus extension that allows you to develop, build, and serve single-page apps or web components (built with Node.js frameworks like React, Angular, Vue, Lit, Svelte, Astro, SolidJS) alongside Quarkus.
- Spring Boot dominates the market due to its maturity and widespread use. (~60%)
- Quarkus is gaining ground, especially for Kubernetes-based microservices. (~10%)
- Micronaut appeals to developers seeking lightweight alternatives with fast startup times.(~5%)
- Helidon is emerging and shows promise for microservices development.(~1%)
- Knative is gaining ground in the serverless space.
App Runner runs on top of AWS ECS and Fargate. Compared to them, App Runner is a lot easier to get into, but you lose some of the more fine-grained configuration options. Also, cost estimation for App Runner is far simpler — AWS changes a fixed CPU and Memory fee per second (Jun. 2, 2021). App Runner doesn’t lock you in the Amazon ecosystem, as generated images will work with all cloud providers.
AWS Marketplace for well-engineered CI/CD practice
A pipeline that builds and tests your Android app with AWS Device Farm .
AWS Cloud Development Kit v2 (April 2021), Typescript For Kubernetes users, the cdk8s project enables you to use CDK constructs for defining Kubernetes configuration in TypeScript, Python, and Java.
ADF - AWS Deployment Framework ADF allows for staged, parallel, multi-account, cross-region deployments of applications or resources via the structure defined in AWS Organizations while taking advantage of services such as AWS CodePipeline, AWS CodeBuild and AWS CodeCommit to alleviate the heavy lifting and management compared to a traditional CI/CD setup.
Pipeline with canary deployments for Amazon ECS using AWS App Mesh
DORA State of DevOps research program represents six years of research and data from over 31,000 professionals worldwide. It is the longest running academically rigorous research investigation of its kind, providing an independent view into the practices and capabilities that drive high performance in technology delivery and ultimately organizational outcomes. Our research uses behavioral science to identify the most effective and efficient ways to develop and deliver software.
2019 Accelerate State of DevOps Report This year’s report confirmed analyst reports that DevOps is crossing the chasm, with the proportion of elite performers almost tripling compared to last year. This is just one of the insights provided in the new 2019 Accelerate State of DevOps Report.
policy-as-code (CloudFormation)
AWS Automation samples: Continuous Delivery With CodePipeline and AWS Step Functions, AWS DevOps CI/CD | CI CD Setup Using CodeBuild CodePipeline ECS | ECS Fargate | CI CD Setup, CodeDeploy to EC2/on-premises, AWS CodePipeline With Jenkins
Faster CI Builds with Docker Layer Caching and BuildKit
Infrastructure as a code
Each layer in OpsWorks has a set of built-in recipes assigned to each lifecycle event, although some layers lack Undeploy recipes. When a lifecycle event occurs on an instance, AWS OpsWorks Stacks runs the appropriate set of recipes for the associated layer. If you have installed custom cookbooks, you can have AWS OpsWorks Stacks run some or all of the recipes automatically by assigning each recipe to a layer's lifecycle event. After an event occurs, AWS OpsWorks Stacks runs the specified custom recipes after the layer's built-in recipes. Custom recipes execute in the order that you enter them in the console. An alternative way to control execution order is to implement a meta recipe that executes the recipes in the correct order.
AWS CloudFormatiom: Mapping AMI to regions
AWS ECS using Docker Compose, CloudFormation + Docker Compose
When to use CloudFormation with OpsWorks
IT teams can integrate CloudFormation with OpsWorks to configure newly deployed EC2 instances with Chef or Puppet, rather than simple shell scripting. In this deployment model, use CloudFormation to deploy all of the AWS resources for your application, and use OpsWorks to do the detailed configuration of the operating systems and infrastructure. Changes are deployed in the same manner -- CloudFormation for AWS resource changes, and OpsWorks for the operating system and application changes.
Ohai, Chef Infra, Chef InSpec and AWS Security Hub
When to use CloudFormation with Elastic Beanstalk
Use CloudFormation to deploy Elastic Beanstalk along with the other resources if other resources include Database or other Storage Services.
Use CodePipeline to deploy to CloudFormatiom Chandra Shettigar.. Fargate Using CloudFormation
AutoScalingReplacingUpdate - Cette stratégie vous permet de spécifier si AWS CloudFormation remplace un groupe Auto Scaling par un nouveau groupe, ou s'il remplace uniquement les instances du groupe Auto Scaling.
Elastic Beanstalk
The two settings choices are complementary, although there is some overlap between env.yaml and .ebextentions.
For example, things like CName, EnvironmentName or SolutionStack can only be set in env.yaml. So instead of setting up their values in EB console or CLI, you could define them in env.yaml to ensure reproducible deployments of your EB. Thus, env.yaml can be considered are more specific to EB environment itself, not to your application.
In contrast, .ebextentions contains configuration scripts that are more coupled with details of your application. What commands should your run to bootstrap it, what config files should you setup or modify it.
Generally I see .ebextentions used much more often than env.yaml. Thus, to begin your EB journey I would recommend just focusing on .ebextentions.
There are many more possibilities of setting up EB than only .ebextentions or env.yaml. There are also .platform, deployment hooks, Buildfile, Procfile, saved configurations and probably some other things I can't remember.
Create a saved configuration to save settings that you have applied to an existing environment during or after environment creation.
Saved Configurations override .ebextensions (where they overlap)
Load Balancers
Application Load Balancer supports up to 5 target groups per listener's rules, each having their weight. You can register an instance with multiple target groups. You then use rules to configure which path will use which target group, which means that a given path will be routed to a particular port. You configure health checks on a per target group basis. Finally, you can configure more than one load balancer with an auto-scaling group.
Step Functions
AWS Step Functions got better price
EC2
Lifecycle Hooks Moving AMI to another region
OldestLaunchConfiguration (Jayendra Patil) https://jayendrapatil.com/wp-content/uploads/2016/03/AWS-Auto-Scaling-v2-2048x1521.png
Policies, Permissions
Policy restructuring When you create a policy, AWS validates, processes, and transforms the policy before storing it. When AWS returns the policy in response to a user query or displays it in the console, AWS transforms the policy back into a human-readable format without changing the permissions granted by the policy. This can result in differences in what you see in the policy visual editor or JSON tab: Visual editor permission blocks can be added, removed, or reordered, and content within a block can be optimized. In the JSON tab, insignificant white space can be removed, and elements within JSON maps can be reordered. In addition, AWS account IDs within the principal elements can be replaced by the ARN of the AWS account root user. Because of these possible changes, you should not compare JSON policy documents as strings. (Free) Access Analyzer 2019. Resource Policies only. AWS Resource Access Manager
Google Cloud
Run Selenium Tests With Google Cloud CI
Other
Microservices in IBM, Quarkus on IBM
The Top 5,899 Microservice Open Source Projects on Github
Eclipse Vert.x vs Spring Boot, Quarkus
Sinatra(2007) alternative to Ruby on Rails, Merb(2008), Nitro, and Camping
Udemy AWS Systems Manager - free
Dockers
[Desktop Docker replacement] (https://rancher.com/why-rancher) Rancher lets you streamline cluster deployment on bare metal, private clouds, public clouds or vSphere and secure them using global security policies. Use Helm or our App Catalog to deploy and manage applications across any or all these environments, ensuring multi-cluster consistency with a single deployment. Use nerdctl commit to create a new container image & tag. Nerdctl.
AWS offers three containerization tools: ECS, EKS, and Fargate. EKS and Fargate were launched at Re:Invent 2017, while ECS predates the field — EKS, Fargate, and even Kubenertes. ECS is first and foremost a stable container orchestration platform, built on EC2, Elastic Compute Cloud. Although the trio of AWS services covers many use cases, there’s always the option of running your own orchestrator such as Kubernetes, Mesos, or Docker Swarm on AWS. This approach requires more knowledge, but provides more flexibility and control. It’s better to use a tool that automates installation and common tasks. Kops, Kubespray & kubeadm are among the better options for self-managed Kubernetes clusters on AWS. ECS also integrates with X-Ray for request tracing—useful for distributed systems.
gossip
Salesforce officially owns Slack (July 2021). Microsoft Teams was born as a competitor to Slack.
Catalyst - Docker Registry System
Docker Registry System, Catalyst is an open-source project that allows you to store and distribute container images. The Catalyst Docker Registry can run in different flavors, such as development, production, or custom modes. You can configure it using a YAML file, and it supports various storage options, including local storage and integration with Amazon S3. < Here are some key details:
- Configuration: The Docker Registry comes with a sample configuration file called config_sample.yml. You can copy this to config.yml and customize it according to your needs. The configuration file defines settings like log levels, storage options, and authentication methods.
- Flavors: The registry can run in different flavors, including: o Common: Used by all other flavors as base settings. o Dev: For development purposes (debug mode). o Prod: For production deployments. o Test: Used by unit tests. o OpenStack: To integrate with OpenStack.
- Authentication Options: o Standalone: Run the server in stand-alone mode, without using the Index service on index.docker.io. o Index Endpoint: Configure the hostname of the Index endpoint (defaults to index.docker.io). o Disable Token Auth: Disable checking of tokens with the Docker index (you can provide your own authentication method).
- S3 Options: If you choose S3 storage, you can configure the following options: o s3_access_key: S3 access key. o s3_secret_key: S3 secret key. To set up your own Docker Registry using Catalyst, follow the provided configuration guidelines and choose the appropriate flavor for your use case. You can find the full source code for Catalyst on GitHub/catalyst-zero