Docker Algorithms - PARC-projects/video-query-home GitHub Wiki

For the Algorithms pipeline, we use three Docker containers:

Container to compute deep neural net features for a video collection,
Load_DB Container to load the video data into the Agile Video Query database, and
Broker Container for running broker.py (for updating queries).

Before using the containers, make sure that the trained models and prototxt files being used by src/features_GPU_compute/calcSig_wOF_ensemble.sh are in the src/features_GPU_compute/models folder.

1. Container to compute deep neural net features

Build the image for this container from the main video-query-algorithms/ folder:
docker build -t avq_compute_dnn_features .
Start up a new container in interactive mode:
```
docker run -it --name <name, optional> --runtime=nvidia -v <folder with video collections>:/video_data avq_compute_dnn_features
```
where <folder with video collections> is the path of the folder containing your video collections. It should have subdirectories that contain the videos.
```
<folder with video collections>
 |--video_collection_1
   |--video1.mp4
   |--video2.mp4
 |--video_collection_2
   |--video3.mp4
   |--video4.mp4
   etc.
```
File and subdirectory names cannot contain spaces. If necessary, execute find . -type f -name "* *.xml" -exec rename "s/\s/_/g" {} \; to remove spaces from names.
From within the container, follow the documentation at Compute Video Features for steps 1. build_wof_clips.py and 2. calcSig_wOF_ensemble.sh.
Exit the container.
For subsequent sessions restart the container using docker start -it <container name>

2. Load_DB container

Build the image for this container from the main video-query-algorithms/ folder:
docker build --file Load_DB_Dockerfile -t avq_load_db .
Verify the user-defined docker network video-query-api_default exists:
docker network ls
Start up a new container in interactive mode:
```
docker run -it --rm --name avq_load_db -v <folder with video features>:/video_data --network video-query-api_default avq_load_db
```
where <folder with video features> is the path of the folder containing the deep net features computed in step 1 above. It should have subdirectories for videos and separate subdirectories for each set of video features. (Video features computed by means other than step 1 can also be loaded using the Load_DB container.)

From within the container, execute

cd /code/src
export API_CLIENT_USERNAME=<api username>
export API_CLIENT_PASSWORD=<api password>

Then follow the documentation at Compute Video Features for step
3. load_db.py, using http://video-query-api:8001 for the api url.
Exit the container.

3. Broker container

In broker.py, set BASE_URL to http://video-query-api:8001
The broker does not use GPU computing (yet!), so it can be run from a server with only cpu resources.
Build the image for this container from the main video-query-algorithms/ folder:
docker build --file Load_DB_Dockerfile -t avq_broker .

Start up a new container in interactive mode:

docker run -it --name avq_broker --network video-query-api_default avq_broker

From within the container, execute

cd /code/src
export API_CLIENT_USERNAME=<api username>
export API_CLIENT_PASSWORD=<api password>
export BROKER_THREADING='True'
export COMPUTE_EPS=.000003
export RANDOM_SEED=None

Start the broker in background mode:
```
nohup python broker.py &
```
Exit the container without stopping it:
```
CTRL^P CTRL^Q
```