Configuration settings - department-of-veterans-affairs/abd-vro GitHub Wiki

Configuration settings are needed for different use cases (unit tests) and environments (dev, qa, prod, ...).

VRO is run the following ways, where environment variables are set for the containers:

• locally via Docker (and docker-compose), usually as a result of running ./gradlew app:dockerComposeUp
• deployed to LHDI's EKS

• If the setting is a secret, it must be put in the Secrets Vault. The secrets will be made available as environment variables.
• Prefer container-scoped settings or configuration files rather than exposing those settings outside the container in Helm configurations.
  • Prefer to add them to application*.yml (for Java) or settings*.py (for Python). Those files allow different setting values per deployment environment. For example, the application-dev.yml is used for deployments to LHDI dev.
  • If the setting needs to be overridden, allow it to be overridden by an environment variable. This should rarely be needed.
• Minimize changes to Helm configurations (under the helm folder). This facilitates diagnosing issues in LHDI that originate from LHDI changes outside our control by allowing us to say: "VRO hasn't change any Helm configurations but things stopped working today". Settings that must be in Helm configurations:
  • VRO_SECRETS_* environment variables -- see Secrets Vault
  • Environment variables that refer to a Helm value or variable, e.g., pv.tracking.mountPath
  • Settings that are shared across containers, i.e., when the setting value must be the same for those containers. For example, connection URLs to VRO's DB, MQ, and Redis.

Also check out Secrets-Vault#configuration-setting-vs-environment-variable and comments in scripts/setenv.sh.

Before using an environment variable, use a configuration setting that is scoped to only the relevant container -- read Secrets-Vault#configuration-setting-vs-environment-variable and see section "Spring's application.yml" below.

Use environment variables in code so that they can be overridden for deployment. For example:

• build.gradle: in dockerRun settings block, set the default values so that it works in your local development environment -- example
• docker-compose.yml: in environment settings block -- example
• Dockerfile: using the ENV command -- example

In addition to the above, environment variables are initialized by:

• setenv.sh (in the abd-vro-dev-secrets repo) - sets environment variables for local builds
• helmchart/values.yaml (for LHDI's EKS deployments) -- Helm chart configurations

Environment variables are used:

• in app/src/main/resources/application.yml (Java Spring Framework) - default values are set for environment variables that don't exist
  • app/src/test/resources/application.yml (for unit tests)
  • app/src/main/resources/application-compose.yml (used when running docker-compose)
  • app/src/main/resources/application-docker.yml (used when running docker)
• in settings.py (for Python code)
• by VRO containers (when run by Docker or EKS) - the variables are referenced by application.yml (for Java containers) and settings.py (for Python containers)

Java code in the VRO application should prefer to use configurations set in application.yml via Spring's @Value annotation -- environment variables should not be referenced directly. This encourages consistency in how VRO is configured.

For Python code, environment variables should be referenced in the settings.py file only (and not in the other files). The settings.py file functions like Java/Spring's application.yml.

The ENV environment variable is used to set spring.profiles.active.

Similarly, environment variable SPRING_PROFILES_ACTIVE (Spring Profile doc) determines which other application-<PROFILE>.yml is loaded, in addition to application.yml -- see this StackOverflow answer. Several application-<PROFILE>.yml files can be loaded by delimiting the profile names with a comma, where properties in the later profiles will override those in earlier profiles if there is overlapping settings. -- see Profiles.

Spring property spring.profiles.include is used to load additional application-<PROFILE>.yml files regardless of the active profile. However, consider these guidelines when using this property.

Associated with profiles, Spring's @Profile(<PROFILE>) can be used to enable certain Spring Beans, Components, Services, etc. -- however, use it sparingly and responsibly.

@ActiveProfiles can be used for a @SpringBootTest that requires special profile(s), e.g., with settings that are different from application-test.yml.

The value of ENV can be any of LHDI's EKS environments (dev, qa, sandbox, prodtest, prod) or local (your local docker-compose environment). spring.profiles.active is set to the value of ENV, which will cause Spring to load application-$ENV.yml, in addition to the default application.yml.

Following this suggestion, VRO has:

• application.yml - shared properties across all environments; always loaded by Spring
• application-prodtest.yml - non-shared properties for the prod-test environment
• application-prod.yml - non-shared properties for the prod environment. Secrets should reference environment variables, which will be set in Kubernetes.
• application-nonprod.yml - shared properties for non-prod environments (local, dev, qa, sandbox). Spring Profile Groups are set up (in application.yml under spring.profiles.group) so that application-nonprod.yml will be loaded, followed by one of the following (if they exist):
  • application-local.yml - for your local docker-compose environment
  • (Optionally) application-dev.yml, application-qa.yml, application-sandbox.yml - for the corresponding LHDI EKS environment. These don't need to exist, esp. if their configuration is no different than application-nonprod.yml

When running VRO, the loaded Spring profile names are shown in one the first few lines of the app's log output, e.g., gov.va.vro.VroApplication: The following 2 profiles are active: "nonprod", "dev".

To override the active profile (spring.profiles.active), set SPRING_PROFILES_ACTIVE. This may be useful to set properties for mock services for example.

Feature flags help developers gate which code should be enabled or not. The microservice for feature flags exists inside of service-python/featuretoggle and [java location TBD].

Feature flag status is stored in Redis and checked by the main app, which attaches the data to routes as needed. The app queries the microservice which updates the timestamp of the last check in order to ensure flag data is up-to-date.

Update the features.yml file in the featuretoggle service with the feature flag name and status.