Deploying SMILE - mskcc/smile-server GitHub Wiki

CMO MetaDB deployment

Contents:

The CMO MetaDB deployment repository houses the docker-compose file for deploying the CMO MetaDB system and related components.

A. Requirements and Pre-build Steps

SMILE Repositories

This section details the environment variables that need to be set up before running SMILE microservices and its dependencies (nats-jetstream, neo4j).

Additional resources for running nats-jetstream and neo4j with docker:

1. NATS JetStream with TLS Encryption

Note: Skip this step if only running the SMILE web service.

SMILE requires additional configuration for running nats-jetstream to support encryption.

The environment variables expected by docker-compose.yaml if running all of SMILE.

Env Variable Description Example
JS_KEY some key for tls somesecretkey
JS_CERT_FILE cert file for tls path/to/nats-cert.pem
JS_KEY_FILE key file for tls path/to/nats-key.pem
JS_CA_FILE path to rootCA file path/to/nats/rootCA.crt
JS_EXTRA_ARGS optional, for debugging "-DV"

2. Neo4j

Neo4j with docker requires bind mounting a volume. Make and set a directory to set NEO4J_DATA_HOME to.

Env Variable Description Example
NEO4J_DATA_HOME local path for neo4j data home ${METADB_DATA_DIR}/neo4j
NEO4J_USERNAME admin username must be neo4j
NEO4J_PASSWORD admin password somepassword

Make sub-directories for Neo4j if they don't already exist:

mkdir -p ${NEO4J_DATA_HOME}/conf ${NEO4J_DATA_HOME}/data ${NEO4J_DATA_HOME}/import ${NEO4J_DATA_HOME}/logs ${NEO4J_DATA_HOME}/plugins

3. SMILE Services

a. Define the following environment variables

Env Variable Description Example
METADB_DATA_DIR local data dir on host machine & parent dir of nats/neo4j data dirs /path/to/smile/data/dir
METADB_CONFIG_HOME local dir on host machine to clone of cmo-metadb-configuration repo /path/to/cmo-metadb-configuration
METADB_DEPLOYMENT_HOME local dir on host machine to clone of cmo-metadb-deployment repo /path/to/cmo-metadb-deployment

b. Make sub-directories for SMILE services in ${METADB_DATA_DIR} if they don't already exist:

mkdir -p ${METADB_DATA_DIR}/logs ${METADB_DATA_DIR}/logs/cmo-metadb ${METADB_DATA_DIR}/logs/cpt-gateway ${METADB_DATA_DIR}/logs/sample-publisher ${METADB_DATA_DIR}/logs/consistency-checker ${METADB_DATA_DIR}/logs/irt-publisher ${METADB_DATA_DIR}/logs/label-generator ${METADB_DATA_DIR}/logs/request-filter

c. Prep environment for setting up NATS Jetstream consumers and streams with docker-compose

Env Variable Description Example
NATS_DATA_HOME local path for nats/js data home ${METADB_DATA_DIR}/nats
NATS_ADMIN_RESOURCES path to admin.properties for SMILE, this file is ${METADB_CONFIG_HOME}/resources/admin.properties

Example admin.properties:

# nats admin properties
nats.consumer_name=[admin username]
nats.consumer_password=[admin password]
nats.url=nats://[domain]:4222

# spring data neo4j properties (admin username must be neo4j)
spring.data.neo4j.username=neo4j
spring.data.neo4j.password=[admin password]
spring.data.neo4j.uri=bolt://[domain]:7687

B. Deploying SMILE with TLS Encryption

Follow these steps before proceeding:

1. Configuring NATS for docker

2. Configuring Neo4j for docker before proceeding.

3. Configuring environment variables for SMILE services

On local machine:

Run the following commands:

  1. Replace the host name for NATS URL in all application.properties files in ${METADB_CONFIG_HOME}:
find ${METADB_CONFIG_HOME}/* -type f -name "*.properties" -exec sed -i '' 's/plvicmometadb1/jetstream/g' {} +
cd ${METADB_DEPLOYMENT_HOME}/metadb
  1. Stop and start up all SMILE services
docker-compose down
docker-compose up -d

Note: See Other Deployments section for what commands to run specifically on the SMILE production and development machines.

Other Deployments

On production machine:

cd ${METADB_DEPLOYMENT_HOME}/metadb
docker-compose down
docker-compose up -d

On dev machine:

cd ${METADB_DEPLOYMENT_HOME}/scripts
bash deploy-metadb-dev-server.sh

C. Deploying the SMILE Server Web App Only

Follow step #2 for configuring Neo4j for docker before proceeding and skip to step #4 below.

  1. Set your SPRING_CONFIG_LOCATION for running the SMILE web sersvice.

An example application.properties can be found here

Set ${SPRING_CONFIG_LOCATION} as the path to the example application.properties from cmo-metadb-configuration or your own version of this file.

export SPRING_CONFIG_LOCATION=/path/to/application.properties
  1. Start the SMILE web service with neo4j:
cd ${METADB_DEPLOYMENT_HOME}/webapp
docker-compose down
docker-compose up -d

The Swagger page should be available at: http://localhost:3000/swagger-ui.html

D. Deploying the GraphQL client only

Follow these steps to deploy the GraphQL client locally and connect it to any instance of Neo4j.

docker run -d --name graphql-client -e NEO4J_USERNAME=${NEO4J_USERNAME} -e NEO4J_PASSWORD=${NEO4J_PASSWORD} -e NEO4J_GRAPHQL_URI=bolt://${NEO4J_HOSTNAME}:7687 -p 4000:4000 cmometadb/graphql-client:1.0.0.RELEASE

Simply replace ${NEO4J_USERNAME}, ${NEO4J_PASSWORD}, and ${NEO4J_HOSTNAME} with the appropriate connection properties.

The -d argument allows you to run the docker image in the "detached" mode.

To view the logs, run:

docker logs -f graphql-client