Run JupyterHub Notebook's Programatically - isgaur/AWS-BigData-Solutions GitHub Wiki

==============================================================================================

Option 1: Run programmatically using SageMaker Notebook terminal :

Steps to execute the notebook programmatically:I did test these all steps and was able to execute my existing Jupyter notebook inside a sagemaker notebook Successfully.

1. Open your notebook JupyterHub and click on New -> Terminal.

2. Once Terminal is opened , the default location will be /home/ec2-user . Then do cd SageMaker/ . This will take you to local sagemaker notebook directory which has all the user notebooks (*.ipynb) stored by deafult. 

3. Perform ls -lrt to find the notebook (*.ipynb) you intend to run programmatically.

4.Use the following command to execute the notebook programmatically - This will execute the selected notebook.These notebooks will specify the respective kernel in which the notebook was saved last time.

    Command - jupyter nbconvert --to notebook --execute ishan.ipynb 

This will open the notebook, execute it, capture new output, and save the result in ..nbconvert.ipynb. Specifying --inplace will overwrite the input file instead of writing a new file. By default, nbconvert will abort conversion if any exceptions occur during execution of a cell. If you specify --allow-errors (in addition to the –execute` flag) then conversion will continue and the output from any exception will be included in the cell output.

In case you want more options to use python API to execute the notebooks programmatically. Please check the documentation here.[1]

=============================================================================================

Option 2:Run using AWS API gateway and AWS Lambda :

In case you are looking to invoke deployed SageMaker model using serverless architecture , there is an option to do so as well.i.e. once the SageMaker endpoint is deployed , you can invoke it using AWS API Gateway and AWS Lambda service.I did find very informative AWS article and tested it my AWS environment and it did work Successfully. Here is the article reference. [2]. This documentation provided detailed step by step instruction to do so.

==============================================================================================

Reference documentation:

[1] https://nbconvert.readthedocs.io/en/latest/usage.html#convert-notebook [2] https://aws.amazon.com/blogs/machine-learning/call-an-amazon-sagemaker-model-endpoint-using-amazon-api-gateway-and-aws-lambda/