messaging deployed using gitOps into the vega infrastructures - WideChat/Rocket.Chat GitHub Wiki

• Create a namespace in the target cluster by making a PR to the vap-infrastructure git repo.
• Preposition/deploy 2 rocketchat k8s secrets into the target namespace, found in lastpass.
• Preposition/deploy a tls secret for the hostname being deployed. Example:
  • kubectl create secret tls messaging-tls --cert=./secrets/messaging-vega-dev-viasat-io-bundle.crt --key=./secrets/messaging-vega-dev-key.key --namespace messaging
• Make a PR against the vap-infrastructure git repo which describes your application to be deployed. Place the configuration files in the correct directories, under the target cluster.
• Sync/deploy your application to the target cluster using ArgoCd
• To update the app version/tag, edit the tag info in the values.yaml file and make a PR against the vap-infrastructure git repo Once merged, Sync/deploy with ArgoCd.

NOTE: Since the 5.1 upstream catchup merge with Rocketchat, mongodb can use changeStreams instead of oplog. In order to do that the RC db user must have admin permissions:

• To do this, after a successful deployment, you need to manually change the mongodb secret for mongo-uri in Rancher to use the same user and authSource as the current mongo-oplog-uri - and then restart the RC server.
• This is not optimal gitOps best practice, but since it is a one-time configuration, and RC will shortly be retired in our tech stack, it will suffice.

• Rocketchat instances in support of Chatbot and Engineering Chat (more use-cases coming) are deployed into the VAP infrastructures using a GitOps model which leverages the ArgoCD tool.
  • VAP documentation
  • ArgoCd
  • There are three VAP environments: dev, preprod, prod
• Monitoring our instances in the VAP can be done through an instance of Grafana managed by the VAP team
  • Viarise Grafana Instance
  • Cluster wide metrics can be viewed in Viarise, including node, pod and namespace metrics, which are very useful for understanding the state of the cluster and for debugging performance issues if they arise.
  • Rocketchat app metrics can be viewed in Viarise in the dashboard named Rocket.Chat Metrics. This dashboard includes app metrics like REST API times, rooms, chats, callbacks, meteor facts, etc... also used for debugging and health monitoring.

• A namespace needs to be created in the target cluster.
• Make a PR against the vap-infrastructure repo which defines a new namespace. For example in the preprod environment HERE - The VAP support team will need to sync and deploy this to the target cluster.

Rocketchat and two mongodb secrets need to be deployed manually and in advance, into the target cluster. This is a one time operation when you first deploy the namespace.

• There are two kubernetes secrets found in lastpass named RC-gitOps deployment secrets
• You also need to manually deploy the tls secret which includes the certs for the hostname being deployed.

• Deploying into the VAP using the GitOps model requires making PRs to the vap-infrastructure git repo
  • The contents of the git repo will represent exactly what we are deploying to the cluster.
  • The ArgoCD tool monitors the git repo, looks for diffs between what is described in the repo and what is currently deployed in the cluster, and it offers the option to Sync the two. We can also configure Argo to AutoSync whenever it detects a diff.
• There are two git directories per environment where we represent/describe our application in the repo. Below using our Brazil instance in Preprod as an example:
  • Argo initially looks at our Helm Tennant Config found here
    • This yaml file describes our application, gives it a name and a namespace; it tells Argo which cluster to deploy into and where to look for the application configuration files.
    • NOTE: We create and commit this file just once when we are first setting up our application with Argo.
  • The application configuration files are maintained by us and found here. Editing these files will notify ArgoCd that there are changes to be applied to the cluster.
    • values.yaml - Used to configure the rocketchat deployment with this Rocketchat Helm Chart. This chart is officially supported and recommended by the upstream Rocket.Chat team. It creates a Rocketchat deployment, a Mongodb statefulset, and an ingress object into kubernetes.
    • mongo_backup_all_manifests.yaml - Describes our mongodb backups using kubernetes cronjobs.
    • custom_resource_defs.yaml - Describes our network policies (most of which are standardized by VAP) and configures a servicemonitor to allow for prometheus scraping.
    • kustomization.yaml - Describes all of the config files we want to include in our deployment. It gives Argo the complete list of resources to be rendered in the cluster.

• Updating an app is done by making a PR to the vap-infrastructure repo with the new tag to be deployed.
  • The rocketchat tag to be deployed is configured in the image block in the values.yaml file.
  • After committing to git, go to ArgoCd, search for your application, and quickly Argo will notice the diff and show Out of Sync, at which point hitting the Sync button will deploy the new version to the cluster. NOTE: Argo can be configured to auto Sync, and likely we will eventually use that to support a fully automated CI/CD.

• We can standup new deployments (a dedicated instance in the EU region for example) easily.
  • Copy one of our helm tenant config files found here and rename it appropriately using the template <cluster>-<unique-app-name>
  • Copy one of our directories from an existing deployment, give it a unique name and place it under the cluster folder where you want the instance to be deployed. Example Directory Here

IMPORTANT: Edit the namespace attribute in EVERY file to match the new namespace.

- `metadata.name`
- `spec.destination.namespace`
- `spec.source.path`

- Edit the `args` blocks to reflect the correct namespace and s3 bucket where the backup should reside.

- `nameOverride`
- `image.repository`
- `image.tag`
- `mongodb.global.namespaceOverride` 
- `ingress.tls.secretName`
- `ingress.tls.hosts`
- `host`

- `spec.selector.matchLabels.instance`
- `spec.selector.matchLabels.name`

- mongodb-secret.yaml (needs editing)
- rocketchat-rocketchat-secret.yaml (needs editing)
- tls secret (the hostname cert)