Serverless functions on Google Cloud using distpy

For this example we will trigger when a file appears in a Google Storage bucket.

Hello World

Create a new Cloud Function triggered on a Google Storage bucket (so that it becomes active when new data appears), with the target language set to Python. We will use the in-line editor.

In the requirements.txt add distpy:

# Function dependencies, for example:
# package>=version
distpy

In the main.py, comment out the response to the trigger and replace it with print(distpy.__version__) which will verify that distpy is installed and available to your function.

import distpy

def hello_gcs(event, context):
    """Triggered by a change to a Cloud Storage bucket.
    Args:
         event (dict): Event payload.
         context (google.cloud.functions.Context): Metadata for the event.
    """
    print(distpy.__version__)
    #file = event
    #print(f"Processing file: {file['name']}.")

Create the function and test it. The output logs should include the printed version number.

Building a serverless processing pipeline

When setting up distpy as a full processing pipeline we breakdown the steps and put the workers inside the serverless functions, triggered by storage buckets.