Start Docker - TheLadders/pipeline GitHub Wiki

  • You need to be able to run Docker containers, which probably means a virtual machine (unless you are running Linux) with at least 5GB, preferably 8GB of RAM in it, and need to be configured for multiple cores.
  • Choose either Option 1 or Option 2, then proceed to "Start the container"

Option 1. USB Install: Install using Docker Toolbox

OSX: Double click on this pkg

osx/DockerToolbox-1.8.2a.pkg

Windows: Run this exe

windows/DockerToolbox-1.8.0a.exe

Next, start the "Docker Quickstart Terminal" app.

Go back to the USB stick and run this script:

./install-pipeline.sh

After the script creates a docker-machine called pipelinebythebay with 5gb of ram allocated you will need to setup your shell environment by running

eval "$(docker-machine env pipelinebythebay)"

If you need the external IP address of your docker-machine to access services on it you can use this command.

docker-machine ip pipelinebythebay

Another example of using it in a script might be. Since I have a bash alias to open chrome. This opens the spark notebook in crhome.

alias openchrome='/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --kiosk'
openchrome http://$(docker-machine ip pipelinebythebay):39000

Option 2: SKIP THIS STEP IF YOU DID OPTION 1 ABOVE!!!!!

You already have Docker, or are running Linux, do the following

  • Linux: install docker: curl -sSL https://get.docker.com/ | sh

  • From the USB stick (to avoid a lengthy download): docker load < <usbpath>/usbstick/pipelinebythebay.tar

  • Verify image is downloaded

docker images

Start the Docker Container:

  • THIS A VERY LONG COMMAND. BE SURE TO COPY/PASTE ALL OF IT!
  • Ignore the following if you see it after logging in to Docker...
  • Instead of ~/pipeline/notebooks below, use the path to where you copied the files
  • The notebooks dir will be created for you if it's not already there
WARNING: Your kernel does not support swap limit capabilities, memory limited without swap.

docker run -it -m 8g -v ~/pipeline/notebooks:/root/pipeline/notebooks -p 30080:80 -p 34040-34045:4040-4045 -p 39160:9160 -p 39042:9042 -p 39200:9200 -p 37077:7077 -p 36060:6060 -p 36061:6061 -p 32181:2181 -p 38090:8090 -p 38099:8099 -p 30000:10000 -p 30070:50070 -p 30090:50090 -p 39092:9092 -p 36066:6066 -p 39000:9000 -p 39999:19999 -p 36081:6081 -p 35601:5601 -p 37979:7979 -p 38989:8989 bythebay/pipeline bash

Follow the Wiki to continue exploring -->

⚠️ **GitHub.com Fallback** ⚠️