AWS Setup

Following The Littlest JupyterHub setup documentation available here. While it is a bit outdated in terms of e.g. screenshots, it covers all the necessary instance creation. Some notes:

• Use the ssh-and-web security group.
• Ensure that the drive space is above about 15GB, with 20GB being the current target.
• Use the CosmicDS ssh keys.

Setting Up CosmicDS

SSH access

The documentation above suggests to start installing packages through the terminal interface on the JupyterHub platform. However, we're going to be circumventing the normal notebook/hub interface, so it is recommended that you begin installing packages through your preferred terminal.

To login to the JupyterHub instance, ensure you have a copy of the CosmicDS ssh keys locally, then login using

$ ssh -i CosmicDS.pem ubuntu@<public-ip-address>

where the <public-ip-address> can be found on the instances page of the AWS console for JupyterHub instance.

Python environments

JupyterHub has several environments used for different purposes.

• Hub environment: JupyterHub, authenticators, spawners, TLJH plugins and the TLJH configuration management code is installed into this environment.
• User environment: Jupyter Notebook, JupyterLab, nteract, kernels, and packages the users wanna use (such as numpy, scipy, etc) are installed here.

We'll only be focused on the User environment for our case.

Installing the CosmicDS package

To setup the CosmicDS webapp, we need to install it into the user's python environment.

1. SSH into the JupyterHub instance using the method above.
2. NodeJS is required to install the CosmicDS webapp. Due to some incompatibility with the latest NodeJS releases (specifically, hashing functionality changes), we need to use the NodeJS 16 LTS version:

$ curl -fsSL https://deb.nodesource.com/setup_16.x | sudo -E bash -
$ sudo apt-get install -y nodejs

4. Now, we need to tell NodeJS to use the legacy openssl provider:

$ export NODE_OPTIONS=--openssl-legacy-provider

5. Now we can install CosmicDS. Activate the User environment, which controls the packages that the spawned user kernels have access to:

$ source /opt/tljh/user/bin/activate

6. Install the CosmicDS package

$ sudo $(which pip) install git+https://github.com/cosmicds/cosmicds.git

Setting up a custom spawner

We're not using JupyterHub as a platform for user coding, so we don't want to give users the ability to create notebooks or terminal sessions -- instead, we're going to override the spawner to just directly provide the CosmicDS web app.

We first need to install the jhsingle-native-proxy, which allows for wrapping an arbitrary webapp so it can be used in place of jupyter-singleuser in a JupyterHub setting.

1. SSH into the JupyterHub instance using the method above.
2. Once in the instance, activate the User environment:

$ source /opt/tljh/user/bin/activate

3. Install the proxy using sudo $(which pip) install jhsingle-native-proxy.
4. We'll now create a new reference script for the spawner. We'll do so by creating a new bash file

$ sudo vi /usr/local/bin/voila-wrapper

and fill it with the following:

#!/bin/bash
export LC_ALL=C.UTF-8
export LANG=C.UTF-8

port="8888"
delim='='
presentation_path=/opt/tljh/user/lib/python3.9/site-packages/cosmicds/stories/hubbles_law/CosmicDS.ipynb

for var in "$@"
  do
    echo "$var"
    splitarg=${var%%$delim*}

    if [ "$splitarg" == "--port" ]; then
      # Yes, port was given in one arg e.g. --port=8888
      port=${var#*$delim}
      echo "Setting external port $port"
    fi

done

jhsingle-native-proxy --destport 0 --port $port voila $presentation_path {--}port={port} {--}no-browser {--}debug {--}Voila.base_url={base_url}/ {--}Voila.server_url=/ {--}Voila.tornado_settings allow_origin={origin_host} {--}Voila.ip=0.0.0.0 {--}VoilaConfiguration.preheat_kernel=True {--}VoilaConfiguration.http_keep_alive_timeout=30 {--}VoilaExecutor.iopub_timeout=100 {--}VoilaConfiguration.enable_nbextensions=True {--}VoilaConfiguration.template='cosmicds-default' --progressive

Note: the presentation_path is explicitly set, this may be a reference to a pre-set environment variable in the future. 5. To override the default spawner settings, we'll create a new file that'll be read by the spawner daemon:

sudo vi /opt/tljh/config/jupyterhub_config.d/spawner.py

and fill it with the following:

c.Spawner.cmd = '/usr/local/bin/voila-wrapper'
c.Spawner.environment = {'JUPYTERHUB_ANYONE': '1'}
c.Spawner.http_timeout = 120

At this point, our spawner and webapp should be fully installed. We'll reload the jupyterhub using sudo tljh-config reload. Then, visit the AWS instance's public address. You should see the CosmicDS webapp, however some functionality will not work (mainly, the pywwt widgets).

Fixing PyWWT

Currently, the pywwt relay functionality doesn't work well with the url redirecting that happens on JupyterHub. To fix this requires editing the jupyter_relay.py file in the installed pywwt library. We do this on the installed version, since installing from source circumvents some of the necessary repo actions. Further investigation will address this. For now, simply edit the /opt/tljh/user/lib/python3.9/site-packages/pywwt/jupyter_relay.py file and replace the get_notebook_server_base_url with

def get_notebook_server_base_url():
    """
    Get the "base_url" of the current Jupyter notebook server.
    """
    global _server_base_url

    if _server_base_url is None:
        _server_base_url = _compute_notebook_server_base_url()

        base_url = os.environ.get("JUPYTERHUB_BASE_URL") #API_URL")
        service_prefix = os.environ.get("JUPYTERHUB_SERVICE_PREFIX")

        if base_url is not None and service_prefix is not None:
            #base_url = base_url.replace("/hub/api", "")

            if service_prefix.endswith('/') and service_prefix.startswith('/'):
                service_prefix = service_prefix[1:-1]

            _server_base_url = f"{base_url}{service_prefix}"

        if not _server_base_url.endswith("/"):
            _server_base_url = _server_base_url + "/"

    return _server_base_url

Fin