Workflows - multiparty/cardinal GitHub Wiki

Once you have set up your clusters in whichever cloud environment you have chosen, you can send workflow requests to them via the following steps. The examples in the code blocks assume three parties (so three clusters) but you may have more than that.

NOTE: Although there are sample payloads below, most of these fields are ignored for now, while the repo is running hard-coded protocol files. The most important fields for now are "PID", "other_cardinals", and "jiff_server" (where applicable).

1. Make sure your input files are at the S3 buckets and keys that you specified in your Cardinal Dockerfile.

2. Note down the external IP addresses of your Cardinal services. These will be the server IPs that you send your requests to.

3. Send a POST request with the following JSON payload to whichever cluster is representing party one:

   http://{IP:PORT}/api/start_jiff_server

   {    
       "workflow_name": {{NAME}},
       "dataset_id": {{ID}},
       "operation": {{OP}},
       "PID": 1,
       "other_cardinals": [2, "http://{{P2_IP}}:{{P2_PORT}}"], [3, "http://{{P3_IP}}:{{P3_PORT}}"](/multiparty/cardinal/wiki/2,-"http://{{P2_IP}}:{{P2_PORT}}"],-[3,-"http://{{P3_IP}}:{{P3_PORT}}"),
   }
   

4. If the previous request was successful, you will receive a JSON response with the key "JIFF_SERVER_IP". Use this to make POST requests to all the parties' cardinal servers to submit a workflow:

   http://{IP:PORT}/api/submit

   {    
       "workflow_name": {{NAME}},
       "dataset_id": {{ID}},
       "operation": {{OP}},
       "PID": 1,
       "other_cardinals": [2, "http://{{P2_IP}}:{{P2_PORT}}"], [3, "http://{{P3_IP}}:{{P3_PORT}}"](/multiparty/cardinal/wiki/2,-"http://{{P2_IP}}:{{P2_PORT}}"],-[3,-"http://{{P3_IP}}:{{P3_PORT}}"),
       "jiff_server": {{JIFF_SERVER_IP}}
   }
   

   Make sure to change the "PID" and "other_cardinals" fields depending on which party you are sending the request to. "PID" should always be the party ID of the server you are sending the request to, while "other_cardinals" should contain the information for the other servers.

5. This may take a few minutes, but if successful, you will receive a JSON response with the key "ID" whose value will be the name of your workflow. You should also see the Kubernetes pods, services, and config maps that were created in your cloud provider's Kubernetes console.

6. Once the pods have finished running, you should see an output file in the S3 bucket you specified in your Cardinal Dockerfile.

7. To stop a workflow, send the following POST request to all the servers:

   http://{IP:PORT}/api/workflow_complete

   {    
       "workflow_name": {{NAME}},
       "dataset_id": {{ID}},
       "operation": {{OP}},
       "PID": 1,
       "other_cardinals": [2, "http://{{P2_IP}}:{{P2_PORT}}"], [3, "http://{{P3_IP}}:{{P3_PORT}}"](/multiparty/cardinal/wiki/2,-"http://{{P2_IP}}:{{P2_PORT}}"],-[3,-"http://{{P3_IP}}:{{P3_PORT}}"),
       "jiff_server": {{JIFF_SERVER_IP}}
   }
   

   (Keep in mind the same note about "PID" and "other_cardinals" as with the submit endpoint)