3. Lambda Requirements for Automatic Artifacts Build - epam/aws-syndicate GitHub Wiki

Below you can find the requirements Lambda functions should meet so that they can be used for automatic artifacts build.

CLI command to create a Python Lambda:

syndicate generate lambda
    --name $lambda_name
    --runtime python
    --project_path $project_path

The structure of the $project_path folder:

    .
    ├── $project_path
    │   └── src
    │       ├── commons
    │       │   ├── __init__.py
    │       │   ├── abstract_lambda.py
    │       │   ├── exception.py
    │       │   └── log_helper.py
    │       └── lambdas
    │           ├── $lambda_name
    │           │   ├── __init__.py
    │           │   ├── deployment_resources.json
    │           │   ├── handler.py
    │           │   ├── lambda_config.json
    │           │   ├── local_requirements.txt
    │           │   └── requirements.txt
    │           ├── __init__.py
    │           └── ...
    └── ...

Structure's requirements:

1. Lambda file should include a function with any name <lambda_func>(event, context). The function will be an entry point to the Lambda.
2. The root folder where the Lambda is stored should have a file named lambda_config.json which describes Lambda configuration in a specific format and lists all its dependencies and triggers.
3. Lambda module can include the requirements.txt file which lists external libraries, on which the Lambda code depends. If the file is added, all listed libraries will be added to the Lambda .zip. Otherwise, it is considered that the Lambda function does not have any external dependencies.
4. Lambda module can include the local_requirements.txt file which lists all modules from the project repository (paths to the modules), on which the Lambda code depends. If the file is there, all the listed modules will be added to the Lambda .zip. Otherwise, it is considered that the Lambda does not need any neighbor modules.

CLI command to create a Java Lambda:

syndicate generate lambda
    --name $lambda_name
    --runtime java
    --project_path $project_path

The structure of the $project_path folder:

 .
    ├── $project_path
    │   └── jsrc
    │       └── main
    │           └── java
    │               └── com
    │                   └── $projectpath
    │                       └── $lambda_name.java
    └── ...

To enable convenient development of java-based Lambda functions, an additional Maven plugin was created. It generates a Lambda meta description file during the java application build. The plugin includes annotations, which are used as a basis for creation of Lambda meta descriptions.

To enable automatic meta description generation, the Lambdas comprising the Java application should meet the following requirements:

1. Lambda handler must be marked with the @LambdaHandler annotation
2. The pom.xml Lambda module must include the following plugin:

<plugin>
    <groupId>com.aws.syndicate</groupId>
    <artifactId>deployment-configuration-maven-plugin</artifactId>
    <version>1.04</version>
    <configuration>
        <packages>
            <package>com.aws.syndicate.demo</package>
        </packages>
    </configuration>
    <executions>
        <execution>
            <id>generate-config</id>
            <phase>compile</phase>
            <inherited>false</inherited>
            <goals>
                <goal>gen-deployment-config</goal>
            </goals>
        </execution>
    </executions>
</plugin>

The Dependencies section should also include the annotations module:

<dependency>
    <groupId>com.aws.syndicate</groupId>
    <artifactId>deployment-configuration-annotations</artifactId>
    <version>1.02</version>
</dependency>

The plugin configuration should include packages in which the plugin should look for Lambdas (in the example above, this is the com.aws.syndicate.demo package).

On project build, the meta description file named deployment_resources.json is generated in the target folder. It is further processed and used by the framework to deploy the Lambda resources to the account.

Annotations also allow to reference other resources mentioned in meta descriptions. There are different annotations that allow to specify the configuration of Lambda functions.

The following annotations can be used (the required parameters are marked with an asterisk *):

1. @LambdaHandler – the annotation which marks the class as Lambda handler. The parameters are:

   • lambdaName* – the name that is used on Lambda deployment.

   • roleName* - the name of the role under which the Lambda is executed.

   • aliasName - the name of the lambda alias added to lambda. Lambda aliases will not be used if annotation is missing.

   • isPublishVersion - turns on versioning for the Lambda. Lambda versioning will not be used if annotation is missing.

   • methodName - the name of the method within the current class that is used as a handler.

   • timeout - the lambda function timeout (default value – 300 seconds)

   • memory - the memory allocated to Lambda (default value – 1024 MB)

   • regionScope - the region/regions to where the Lambda is deployed (the default value – RegionScope.DEFAULT means that the Lambda is deployed only to the region specified in the sdct.conf file during the deploy)

   • subnets – the list of IDs of subnets where the Lambda is be placed to (default value – an empty list.)

   • SecurityGroupsIds – the list of the IDs of security groups to which the Lambda belongs (default value – an empty list)

   • tracingMode – turns on the X-Ray for the Lambda.

   • runtime – chose Java version for lambda runtime. (default value – DeploymentRuntime.JAVA11)

   • architecture – chose an architecture for lambda. There are two options: Architecture.X86_64 (used by default if not specified) and Architecture.ARM64 (in scope of Java works with runtime = DeploymentRuntime.JAVA11)

   • layers - list of layer names, that should be linked to the lambda

2. @EnvironmentVariable - the annotation that allows to set environment variables in the Lambda configuration:

   • key* - the environment variable key value.
   • value* - the environment variable value.
   • valueTransformer - the transformer that allows you to change the source value to the desired one. There are two types: ValueTransformer.USER_POOL_NAME_TO_USER_POOL_ID and ValueTransformer.USER_POOL_NAME_TO_CLIENT_ID. Not required, by default is absent.

   @EnvironmentVariables - the repeatable annotation

3. @DynamoDbTriggerEventSource - the annotation which allows to subscribe a Lambda to a DynamoDB stream:

   • TargetTable* - the name of the table to which the Lambda subscribed as a trigger.
   • batchSize* - the number of records passed to the Lambda as input.

   @DynamoDbTriggerEventSource - the repeatable annotation

4. @RuleEventSource - the annotation which allows to subscribe a Lambda to a CloudWatch rule.

   • TargetRule* – the name of the rule to which the Lambda is subscribed.

   @RuleEvents - a repeatable annotation

5. @S3EventSource - the annotation that allows to subscribe a Lambda to events in an S3 bucket

   • targetBucket* - the name of the bucket to which the Lambda is subscribed.
   • events* - the array of line-events that trigger the Lambda (for example,“ s3:ObjectRemoved”*)

   @S3EventSource - the repeatable annotation

6. @SnsEventSource - the annotation which allows to subscribe a Lambda to an SNS topic:

   • targetTopic* - the name of an SNS topic to which the Lambda function is subscribed.
   • regionScope – the region or regions in which the SNS topic is located. The default value
     • RegionScope.DEFAULT means that the Lambda is deployed only to the SNS topic in the regions specified in the config during deploy.

   @SnsEvents - the repeatable annotation.

7. @SqsTriggerEventSource - the annotation that allows to subscribe the Lambda to an SQS queue:

   • targetQueue* – the name of the SQS queue to which the Lambda is subscribed.
   • batchSize* – the number of records passed used at the Lambda event.
   • functionResponseTypes* - a list of current response type enums applied to the event source mapping. The default value is an empty list. Available response types: FunctionResponseType.REPORT_BATCH_ITEM_FAILURES.

   @SqsEvents - the repeatable annotation.

8. @LambdaConsurrency - the annotation that limits the maximum number of Lambda executions within a specified time unit.

   • executions* – the maximum number of Lambda function executions within the specified time unit.

9. @DeadLetterConfiguration - the annotation that allows to configure DLQ for the Lambda.

   • resourceType* – the DLQ type, SQS queue or an SNS topic
   • resourceName* – the DLQ resource name.

10. @DependsOn - the annotation that allows to establish dependencies between resources that are to be deployed (for example, if a Lambda depends on a table, to make it easier for developers to orient in the dependencies, we can specify this table in the @DependsOn annotation):

    • name* – the name of the resource in the meta description
    • resourceType* - the type of the resource.

    @Dependencies - the repeatable annotation.

11. @LambdaLayer - the annotation that allows to set up lambda layer

    • layerName - the name for layer
    • libraries - list of libraries, that would be packed into the layer
    • runtime - Java version tag for lambda layer runtime. (default value – DeploymentRuntime.JAVA11)
    • architectures - list of architecture tags for lambda layer
    • artifactExtension - artifact extension (default value – ArtifactExtension.ZIP)

12. @LambdaUrlConfig - the annotation that enables Lambda URL.

    • authType - default value AuthType.NONE
    • invokeMode - default value InvokeMode.BUFFERED

13. @Tag - the annotation that allows to set tag to the Lambda

    • key* - the tag key.
    • value* - the tag value.

    @Tags - the repeatable annotation

After the plugin is run within the specified phase (for example – compile), the deployment_resources.json file with Lambda functions meta descriptions is generated in the target folder.

First off, make sure you have npm package manager installed on your local machine if you want to deploy NodeJS lambdas:

$ npm -v
6.14.4

CLI command to create a NodeJS Lambda:

syndicate generate lambda
    --name $lambda_name
    --runtime nodejs
    --project_path $project_path

The structure of the $project_path folder:

   .
    ├── $project_path
    │   └── app
    │       └── lambdas
    │           └── $lambda_name
    │               ├── deployment_resources.json
    │               ├── lambda_config.json
    │               ├── index.js
    │               ├── package.json
    │               └── package-lock.json
    └── ...

The process of using NodeJS lambdas is pretty simple and doesn't require any additional knowledge except those is configuring npm.

Root folder requirements:

1. Lambda's root folder must also contain deployment_resources.json file to declare necessary AWS services tightly connected to the lambda. After generating it contains lambdas basic execution role.
2. Must have a file with actual lambdas logic written in JavaScript - index.js. Basic handler is generated by the Syndicate. You should extend with your business-logic.
3. Must have a file named lambda_config.json which describes Lambda configuration in terms of the Syndicate. It's generated automatically with default options and you ought to change them if you desire.
4. Must have package.json npm file with it's configuration. Most settings are set, you just need to put third party dependencies. It'll be downloaded and packaged with syndicate assemble
5. Can also have package-lock.json file which has nothing to do with the Syndicate but is useful for NodeJS package manager. If it's deleted it'll be created again.

Once the lambda is created and its dependencies are declared just assemble your bundle. Use syndicate assemble command (syndicate build actually invokes assemble inside):

$ syndicate assemble
Building artifacts ...
Command assemble node: project_path: app 
6.14.4
...

First off, make sure you have .NET SDK installed on your local machine if you want to deploy .NET lambdas:

$ dotnet --info
.NET SDK:
Version:           8.0.402
...

CLI command to create a .NET Lambda:

syndicate generate lambda
    --name $lambda_name
    --runtime dotnet
    --project_path $project_path

The structure of the $project_path folder:

   .
    ├── $project_path
    │   └── dnapp
    │       └── lambdas
    │           └── $lambda_name
    │               ├── deployment_resources.json
    │               ├── lambda_config.json
    │               ├── Function.cs
    │               └── Function.csproj
    └── ...

Root folder requirements:

1. Lambda's root folder may contain deployment_resources.json file to declare necessary AWS services tightly connected to the lambda. After generating it contains lambdas basic execution role. Resources defined in this file may be moved to any other project deployment_resources.json file.
2. Must have a file with actual lambdas logic written in .NET - Function.cs. Basic handler is generated by the Syndicate. You should extend with your business-logic.
3. Must have a file named lambda_config.json which describes Lambda configuration in terms of the Syndicate. It's generated automatically with default options and you may change them if you desire.
4. Must have an XML document with extension .csproj that contains all the information and instructions that MSBuild needs to build your project.

Once the lambda is created and its dependencies are declared just assemble your bundle. Use syndicate assemble command (syndicate build actually invokes assemble inside):

$ syndicate assemble
Building artifacts ...
Command assemble dotnet: project_path: dnapp 
...