Exygy Admin Info - civiform/civiform GitHub Wiki
- Add user to CloudFlare
- Setting up roles and resources needed for AWS Resource Check
- Setting up role for civiform-staging-deploy automatic deployment
- Setting up new demo site
- Redirect one domain to another
- Add Auth0 to demo site
- Add new community contributor
- Prevent new-contributor auto-replies from Github
- Adding CloudFlare authentication to a dev site
- Add WAF to AWS instance
Setting up roles and resources needed for AWS Resource Check
- Log in to the given account you want to be able to check.
- IAM -> Identity providers
- Add provider
- Choose
OpenID Connect
. For provider URL, enterhttps://token.actions.githubusercontent.com
. For Audience, entersts.amazonaws.com
. - Click
Add provider
- IAM -> Policies
Create policy
- Select
RDS
for the service. - Under
Actions allowed
, expandList
and chooseDescribeDBInstances
. - Under
Resources
, selectAll
. - Click
Next
- Under
Policy name
enterciviform-resource-check-action
- click
Create policy
- IAM -> Roles
Create role
- Choose
Web identity
and then pick thetoken.actions.githubusercontent.com
identity provider withsts.amazonaws.com
for the audience. Enterciviform
for the GitHub organization. - Click
Next
- Choose the
civiform-resource-check-action
permission. ClickNext
. - Enter
civiform-resource-check-action
for the role name. ClickCreate role
.
Same as above, but skip step 3 and in step 4, select AdministratorAccess
for the policy and civiform-staging-deploy-action
as the role name. The environment secret ROLE_ARN
in civiform-staging-deploy
will be the ARN for this particular role you create.
- Create new account in AWS Organizations. Set the email address to
civiform-instances+<instance-name>@googlegroups.com
. - Once the account is created, move it into the
civiform-dev
folder in the organization. - Switch to the OrganizationAccountAccessRole for the new account.
- To make this easier, you can create a bookmark that contains the following for the URL field. Replace
accountId
anddisplayName
appropriately.javascript:(function(){ var accountId = '149536465499'; var displayName = 'allegheny-demo'; var currentUrl = encodeURIComponent(window.location.href); var switchAccountUrl = 'https://signin.aws.amazon.com/switchrole?account=' + encodeURIComponent(accountId) + '&roleName=OrganizationAccountAccessRole&displayName=' + encodeURIComponent(displayName) + '&redirect_uri=' + currentUrl; window.location.href = switchAccountUrl; })();
- Request a new certificate. FQDN should be the full URL for the new site. Leave the rest as default.
- In Cloudflare, add a new CNAME record with the name and value given for the certificate in the last step. After adding, ensure the AWS certificate page shows Status = Issued rather than Pending. Note the certificate ARN.
- Copy the most recently created config file in https://github.com/civiform/civiform-staging-deploy for the new account. Modify the following values appropriately:
- Comment at the top noting what the instance is, owner, purpose.
ACCOUNT_ID
SENDER_EMAIL_ADDRESS
STAGING_PROGRAM_ADMIN_NOTIFICATION_MAILING_LIST
STAGING_TI_NOTIFICATION_MAILING_LIST
STAGING_APPLICANT_NOTIFICATION_MAILING_LIST
BASE_URL
STAGING_HOSTNAME
APP_PREFIX
SSL_CERTIFICATE_ARN
- Modify your AWS CLI to use the new account.
-
The way Nick has it set up is to contain the sensitive credentials in
~/.aws/credentials
, and each profile in~/.aws/config
.~/.aws/credentials
[exygy-root] aws_access_key_id = <ask Nick to help with this> aws_secret_access_key = <ask Nick to help with this>
~/.aws/config
[profile nickb-dev] region = us-east-1 role_arn = arn:aws:iam::<account ID>:role/OrganizationAccountAccessRole source_profile = exygy-root [profile demo] region = us-east-1 role_arn = arn:aws:iam::<account ID>:role/OrganizationAccountAccessRole source_profile = exygy-root
Then, set the
AWS_PROFILE
environment variable to choose which profile the AWS CLI should use.
- From your clone of
civiform-staging-deploy
runbin/setup-instance
and get the initial deploy complete. When asked for secrets, you can putnone
for ADFS and ESRI secrets. For OIDC, use the client ID and secret in theCiviForm demo site
Auth0 application for both admin and applicant. - Add the DNS record for the https://.civiform.dev pointing to the load balancer given at the end of the deployment.
- Do the Add Auth0 to demo site instructions found below.
- Within the new demo instance, turn on feature flags (bias towards turning flags on, but don't turn on incomplete things) and set other variables as desired. In particular:
-
SHOW_NOT_PRODUCTION_BANNER_ENABLED
=true
-
STAGING_DISABLE_DEMO_MODE_LOGINS
=false
WHITELABEL_CIVIC_ENTITY_FULL_NAME
WHITELABEL_CIVIC_ENTITY_SHORT_NAME
- Seed programs and questions, and publish them.
- Do the steps at Setting up role for civiform-staging-deploy automatic deployment
- Modify the
aws_deploy_all_demos.yaml
action to include the new demo site. Make sure to also change theenvironment
in the job to a new name. - In the civiform-staging-deploy environments, create a new environment.
- Create a
CONFIG_FILE
environment variable that is the name of the new config file. - Create a
DEMO_NAME
environment variable that describes the site. This is only used for Slack notifications. - Create a
ROLE_ARN
environment secret, set to the ARN for theciviform-staging-deploy-action
role. - Open a PR to civiform-staging-deploy with the new changes.
Below as an example, we will redirect miami-demo.civiform.dev
to civiform-demo.civiform.dev
. The "redirected site" is miami-demo
and the "target site" is civiform-demo
.
It seems this part actutally isn't necessary. When CloudFlare redirects the request, it seems the load balancer sees the request as civiform-demo.civiform.dev already, so no need to mess with the certificate. I'm leaving it here just in case we need to do this in the future.
First, you will need to regenerate the certificate used for the target site (in this case civiform-demo.civiform.dev
) to include the redirected site as a valid domain for the certificate.
- Go to AWS Certificate Manager for the target site's account.
- Create a new certificate (see https://github.com/civiform/civiform/wiki/Deploy-System), adding the both the target and redirected sites (along with any others existing in the current certificate).
- Add the CNAME/Value for the domain validation for the certificate as described in the previous link. Each domain needs to have a separate entry. Wait for each domain to be validated.
- Redeploy the target site with the updated certificate set in the config for
SSL_CERTIFICATE_ARN
.
This part we have to do.
Next, we will set up the redirect in CloudFlare.
- Log in to CloudFlare -> [email protected]'s account
- Click
civiform.dev
- On the left nav, go to
Rules -> Redirect Rules
- Create a new rule.
- Custom filter expression
-
Hostname
equals
miami-demo.civiform.dev
- URL redirect type
Dynamic
, expressionconcat("https://civiform-demo.civiform.dev", http.request.uri.path)
, Status code301
. EnsurePreserve query string
is checked. - Order doesn't matter (where the rule fires after)
- On the left nav, go to
Rules -> Redirect Rules
- Find and edit the entry for the one you want to change, or create a new one if it doesn't exist yet.
-
Name
should be the pre-redirected name (miami-demo
) -
Target
should be the load balancer address of the site you are redirecting to (civiform-demo-civiform-lb-1366112934.us-east-1.elb.amazonaws.com
) - Select
Proxy status
so the redirect rule can take place.
All of our demo sites (except staging currently) are hooked up to the same Auth0 application. To add a new site:
- Log into Auth0 and edit the
CiviForm demo site
application. - In
Allowed Callback URLs
, addhttps://<sitename>.civiform.dev/callback/OidcClient
andhttps://<sitename>.civiform.dev/callback/OidcAdmin
. This is assuming you've setAPPLICANT_OIDC_PROVIDER_NAME
toOidcClient
andADMIN_OIDC_PROVIDER_NAME
toOidcAdmin
. - Add
https://<sitename>.civiform.dev
toAllowed Logout URLs
,Allowed Web Origins
, andAllowed Origins (CORS)
. - Click
Save Changes
. - Redeploy the application if you made these changes after deployment.
In GitHub, go to the "civiform/civiform" project -> Settings -> Collaborators and teams -> Add people. Give them the Triage role. That'll send the person an invite. Once they accept the invite, you can assign them to issues.
If you run into an issue where there are no more seats to add, ping Nick Burgan or another Exygy engineer to see if we can clear up a seat or need to add more.
We have a script that runs when a new contributor comments on a issue that comments on the same issue and directs them to our "How to get started" guide. This will respond to anyone who has never merged a commit, has not already commented on this issue, and is not on our "Ignore List". New full-time contributors should be added to the "Ignore List" to prevent these auto-replies. To add them to the Ignore List:
- Go to the "Settings" tab in the
civiform
repo. - In the lefthand sidebar, expand the "Secrets and variables" drop down under the "Security" section
- Click on "Actions"
- On the actions page, click on the "Variables" tab
- Click the "edit" icon next on the
CONTRIBUTOR_COMMENT_AUTO_REPLY_IGNORE_LIST
variable - Add the new contributor's Github username to the list. Names should be separated by commas with no spaces in between.
- For the domain (civiform.dev), go to
SSL/TLS
->Overview
and clickConfigure
. Ensure the mode is set toFull (Strict)
rather than the defaultFlexible
. When it'sFlexible
and you have a CNAME proxied through CloudFlare, it will try to connect to the server via HTTP instead of HTTPS, which will cause an infinite redirect. If it's set to anything else, chooseCustom SSL/TLS
and chooseFull (Strict)
. - In DNS settings, make sure the CNAME for the site has
Proxied
enabled (orange). - In the top level [email protected] account, go to
Zero Trust
. - Go to
Access
->Applications
. - Click
civiform demo sites
and clickConfigure
. - Click
Add public hostname
and add the subdomain you want to protect, withciviform.dev
as the domain. - Click
Save application
.
After a minute or two, you should get a login prompt when you try to access the site. Log in with your @exygy.com
or @google.com
email address.
How does this work? Under Access -> Policies
you'll find a policy called civiform demo site
. In here, you'll see the policy is Emails ending in
with the two domains. You can also add specific Gmail addresses in here if you need to. Back in Access -> Applications
, you can see the policy is applied in the Policies
tab. In the Login methods
tab, you can see we've got a Google
method.
In Settings
-> Authentication, you can see the
Google` login method. If you edit this, you'll see the App ID and a hidden secret.
In the Exygy GCP org, there is a project called civiform-demo-idp
. If you go to https://console.cloud.google.com/apis/credentials?project=civiform-demo-idp&supportedpurview=project which is APIs & Services
-> Credentials
, you'll see the OAuth 2.0 client ID.
A Web Application Firewall can cut down on the malicious traffic reaching our demo/staging instances. This should be used carefully, though, as it can cause problems during probers.
- While logged in under the AWS account you want to set up the WAF for, go to https://us-east-1.console.aws.amazon.com/wafv2/homev2/start?region=us-east-1.
- Click
Create web ACL
. - Give it any name. Leave
Regional resources
selected. ClickAdd AWS Resources
, chooseApplication Load Balancer
, then choose the LB for your instance and clickAdd
. - Click
Next
. - Click
Add Rules
->Add managed rule groups
. - Expand
AWS managed rule groups
, scroll down toFree rule groups
, and turn them all on except forAdmin protection
(don't really need this). We maybe don't need some of the others (e.g. Windows, PHP), but since we get malicious traffic probing for those things, this keeps that noise down. - Click
Edit
underCore rule set
, then change theSizeRestrictions_BODY
toOverride to Allow
. This is needed in order not to block program exports. - Click
Save rule
and thenProceed
. - Scroll back down to the
Windows operating system
rule set andEdit
it. ChangeWindowsShellCommands_BODY
toOverride to Allow
. For some reason, exporting the default seeded minimal program triggers this rule. - Click
Save rule
and thenProceed
. - Click
Add rules
. - Ignore the warning that says
Total WCUs exceed 1500 and are subject to additional costs.
. It does cost a little, but not much. - Click
Next
3 times, leaving all the other defaults, thenCreate web ACL
.
On some sites the WAF may intermittently block calls that we want to allow. AWS is currently changing UI so just providing a high level overview of the change. The following rules built-in rules were modified:
- RuleSet
AWS-AWSManagedRulesSQLiRuleSet
edited to have ruleSQLi_BODY
action becomeOverride to Count
to allow for this traffic.
If you ever need to disable the WAF, go to the AWS WAF
-> Web ACLs
-> your ACL. Go to the Associated AWS Resources
tab, choose the load balancer, and click Disassociate
. You can then re-enable it similarly.