Dynamic Configuration Update for AWS Lamba Functions - saayam-for-all/docs GitHub Wiki

All our backend implementations are deployed as Lambda functions in our 1.0 MVP. Later on, if needed, we can deploy them as Docker containers in EKS(K8S). We store all our config parameters and secrets in AWS Parameter Store. All our Lambda functions register listeners for listening to any change in these config parameters or secrets. When any config parameter or secret is changed, we follow this process:

Event-Driven Configuration Refresh (Advanced)

You can combine services to actively notify running Lambda instances of configuration changes without waiting for a cold start. EventBridge + SNS + SSM:

  • An EventBridge rule detects a change in a Parameter Store parameter.
  • This rule triggers an SNS topic.
  • The SNS topic is subscribed to by the Lambda function(s), which then receive a notification event.
  • The function's handler logic is designed to use this event to reload the configuration from the SSM Parameter Store, ensuring all warm instances have the latest data.

There are other ways to update these Lambda configurations which we are NOT following in our implementation:

  • Using the AWS SDKs or AWS CLI The primary way to programmatically update a function's settings is by calling the UpdateFunctionConfiguration API operation via an AWS SDK (available for Python, Node.js, Java, etc.) or the AWS CLI. Functionality: This method allows you to change settings such as memory size, timeout, environment variables, execution role, and layers. Permissions: The entity performing the update (e.g., another Lambda function, a CI/CD pipeline, an EC2 instance) must have the necessary IAM permissions to call UpdateFunctionConfiguration. Considerations: Updates apply only to the unpublished $LATEST function version. Changes can take a short time to propagate. You may need to publish a new version after the update and shift traffic using aliases if you use versioning in your production environment.

E.g. CLI: aws lambda update-function-configuration --function-name "my-function" --environment '{"Variables":{"KEY":"new_value"}}' --memory-size 256

  • Using External Configuration Stores For configuration values that need to change frequently at runtime without redeployment or updating the function's configuration directly, use external data stores. The Lambda function retrieves the current values during or just before invocation. AWS Systems Manager Parameter Store or AWS Secrets Manager: Store configuration data or sensitive information here. Your Lambda function can use the AWS SDK to retrieve these parameters when it starts (during the initialization phase for better performance) or on every invocation. Amazon S3: Store a configuration file (e.g., a JSON file) in an S3 bucket and have the Lambda function read it at runtime. Amazon DynamoDB: Use a DynamoDB table to store dynamic key-value configuration pairs, allowing for rapid, low-latency lookups.