Test Your Container With JupyterHub! - SeaDataCloud/Documentation GitHub Wiki

If you have a service that runs as one container per user, this is how you can test its integration with JupyterHub.

This is a simple test JupyterHub image where any user can log in with the same password.

The Hub then runs a container from your image, creates the user's directory (unless it exists) with uid=1000 and gid=100, and passes the following environment variables into your container for your usage:

• VRE_USERNAME (the user's username, a unique string, e.g. "alice")

• VRE_DISPLAYNAME (the user's display name, e.g. "Alice Doe", as obtained from Marine ID service).

• FILESELECTION_PATH (the path to the file(s), selected by the user in the fileselector prior to starting the service, relative to the user's NextCloud directory, .e.g /Photos/Hummingbird.jpg for the path /.../nextclouddata/<username>/files/Photos/Hummingbird.jpg).

• NB_UID and NB_GID: These are the uid and gid that own the user data. Their values are 1000 and 100, so your service should run as these.

  • Note 1: If you have not done this yet, you can change them using RUN_AS_USER and RUN_AS_GROUP in the docker-compose.yml, see below.
  • Note 2: They are passed as environment variable because in a perfect world, your service would be able to change its own uid/gid based on these. JupyterNotebooks can. If you cannot change them at runtime, make sure your image does run as uid 1000, and that the uid is easily changeable in your Dockerfile.

Please note that JupyterHub will forward the user to the url /user/<VRE_USERNAME> on port 8888, so your service should listen there! (If this is new to you, make sure you have read https://github.com/SeaDataCloud/Documentation/wiki/Integrating-a-service-into-the-SeaDataCloud-Virtual-Research-Environment). If port 8888 is not convenient for you, you can change it by adding c.DockerSpawner.port=80 and c.DockerSpawner.container_port=80 to the jupyterhub_config.py.

Run these steps on a computer that has docker and docker-compose installed:

Get the image:

# Pull image:
docker login registry-sdc.argo.grnet.gr # for creds ask Themis
docker pull registry-sdc.argo.grnet.gr/jupyterhub_vretest:20200120
docker tag registry-sdc.argo.grnet.gr/jupyterhub_vretest:20200120 jupyterhub_vretest:20200120

# OR: Build image:
# Get the material here:
# https://github.com/SeaDataCloud/vre-config/tree/master/for_testing_services/BUILD_IMAGE
cd BUILD_IMAGE
docker build -t jupyterhub_vretest:20200130 .

Create test network:

docker network create vre-test

Now, make a location for fake user directories. (In reality, this is where the user's NextCloud data would be, either mounted, or as a synchronized copy.)

mkdir /WHEREVER/jupyterhub_test
cd /WHEREVER/jupyterhub_test
mkdir ./fake_userdirectories

This is where directories for users will be created on login.

Or you can create them manually before login, so that the data will be visible inside your container (if the permissions are correct):

# Create fake data for user "alice"
# Readable if your service runs as uid 1000

cd /WHEREVER/jupyterhub_test
mkdir ./fake_userdirectories/alice
echo "test123" >> ./fake_userdirectories/alice/test.txt
chown 1000:100 ./fake_userdirectories/alice/test.txt

Download the config file (can stay unchanged):

cd /WHEREVER/jupyterhub_test
wget https://raw.githubusercontent.com/SeaDataCloud/vre-config/master/for_testing_services/test_jupyterhub/jupyterhub_config.py

Now get the docker-compose file, and adapt some settings:

cd /WHEREVER/jupyterhub_test
wget https://raw.githubusercontent.com/SeaDataCloud/vre-config/master/for_testing_services/test_jupyterhub/docker-compose_template.yml
cp docker-compose_template.yml docker-compose.yml
vi docker-compose.yml

What to change in docker-compose.yml:

• The test password to be used for login - choose any value you want: TEST_PW: 'changeme!XYZXYZ'
• The absolute path to your fake user directories: HOST_LOCATION_USERDIRS: '/XYZXYZ/fake_userdirectories'
• Add the name of your image to be tested in DOCKER_JUPYTER_IMAGE: XYZXYZ (there is two images that you can use for trying, a plain notebook jupyterhub/singleuser:0.8 and Uni Liège's DIVA notebook abarth/divand-jupyterhub:2020-01-27T1800.
• If your services does not run as uid 1000, you may have to change RUN_AS_USER/RUN_AS_GROUP to make sure the user's data is readable and writable for your service's process. (These settings, in case of Juypter Notebooks, change also under which uid / gid the process is run - with some restrictions, e.g. it is not easy to run them as 33 - in that case ask Merret for further instructions).

Optionally...

• Change the port where the service listens on, by changing port 443 to your desired port in this section: ports: - 443:8000. (8000 is where JupyterHub listens inside the container, and should not be changed).
• Change the location where your user's data will be mounted, which is by default into /home/jovyan/work/nextcloud/: Change USERDIR_INSIDE_CONTAINER: '/home/jovyan/work'to the desired path. The "/nextcloud" part will be added.
• Switch on SSL, by changing USE_SSL: 'false' to USE_SSL: 'true'. Note that in this case you must also add valid paths to SSL certificate and private key: /XYZXYZ/ssl/certs/_cert_and_chain.pem and /XYZXYZ/ssl/private/server.key, which are now commented out.

Finally, run it:

cd /WHEREVER/jupyterhub_test
docker-compose up

This will start the hub, and show you the hub's logs. You can stop it by Ctrl-C.

Remove everything after testing by:

cd /WHEREVER/jupyterhub_test
docker-compose down
docker network rm vre-test

# Remove every container separately:
docker rm vretest-<username>

Now, the service should be available on port 443 (or another one, if you have changed it): <yourhost:443. (Note that if you have not enabled SSL, it may be dangerous if port 443 is open in your firewall! Remember to close your service after testing!)

If you go there with your browser, you should see a simple login form where you can provide the test username of your choice (any string withoutt special characters will do, e.g. alice), the password (what ou configured) and a display name (e.g. Alice Doe Senior).

• logging in using any username and the configured test-password, and see whether you get fowarded to your GUI.
• whether you can see the user data in /home/jovyan/work/nextcloud (or another place, if you have changed it in docker-compose!).
• and anything else you'd like to function!

Note that your containers will be named vretest-<username>. You can see their logs, and also get into their command line using these commands:

# logs:
docker logs vretest-<username>

# get into them:
docker exec -it vretest-<username> /bin/bash