gcs Cloud Function - ghdrako/doc_snipets GitHub Wiki

Cloud function - regional

Cloud Functions uses containers to set an isolated environment for your function. These are called Cloud Functions instances. If multiple functions are executed in parallel, multiple instances are created.

These events can come from one of the following providers: *• HTTP

  • Cloud Storage
  • Cloud Pub/Sub
  • Cloud Logging via Pub/Sub sink
  • Cloud Firestore
  • Firebase Realtime Database, Storage, and Authentication

If you create a sink to forward the logs to Pub/Sub, then you can trigger Cloud Functions

Cloud Functions characteristics

  • Serverless - underlying infrastructure is abstracted from the end user.
  • Event-driven
  • Stateless - do not store state nor data
  • Autoscaling

Stateless

Cloud SQL connectivity

Cloud Functions is stateless and the state needs to be saved on external storage or in a database. This can be done with external storage such as Cloud Storage or a database such as Cloud SQL. In general, any external storage can be used. With Cloud Functions, you can connect to Cloud SQL using a local socket interface that's provided in the Cloud Functions execution environment. It eliminates the need to expose your database to a public network.

Connecting to internal resources in a VPC network

If your function needs to access services within a VPC, you can connect to it directly by passing a public network. To do this, you need to create a serverless VPC access connector from the network menu and refer to the connector when you deploy the function. Note that this does not work with Shared VPC and legacy networks.

Environmental variables

Cloud Functions allows you to set environment variables that are available during the runtime of the function. These variables are stored in the function's backend and follow the same life cycle as the function itself. These variables are set using the --set-env- vars flag; for example:

gcloud functions deploy env_vars --runtime python37 --set-env-
vars FOO=bar --trigger-http

Cold start

new instances are created in the following cases:

  • When the function is deployed
  • Scaling up is required to handle the load
  • When replacing an existing instance is triggered

Cold starts can impact the performance of your application.

Runtime environments

  • Node.js 10, 12, and 14
  • Python 3.7, 3.8, and 3.9
  • Go 1.11 and 1.13
  • Java 11
  • .NET Core 3.1
  • Ruby 2.6 and 2.7
  • PHP 7.4

Each function is single-threaded and is run in an isolated environment with the intended context.

When you define a function, you can also define the requirements or dependencies file in which you state which modules or libraries your function is dependent on. Libraries will be loaded when your function is executed. This causes delays in terms of execution.

There are two types of Cloud Functions:

  • HTTP functions - invoked by HTTP(S) requests. The POST, PUT, GET, DELETE, and OPTIONS HTTP methods are accepted. Arguments can be provided to the function using the request body. The invocation can be defined as synchronous as it can return a response that's been constructed within the function. Cloud Functions handles HTTP requests using common frameworks. For Node.js, this is Express 4.16.3, for Python, this is Flask 1.0.2, and for Go, this is the standard http.HadlerFunc interface.
  • background functions - are invoked by events such as changes in the Cloud Storage bucket, messages in the Cloud Pub/Sup topic, or one of the supported Firebase events

Events

Events can be defined as things happening in or outside the GCP environment. When they occur, you might want certain actions to be triggered.

Use Case Description
Data Processing / ETL Listen and respond to Cloud Storage events such as when a file is created, changed, or removed. Process images, perform video transcoding, validate and transform data, and invoke any service on the Internet from your Cloud Function.
Webhooks Via a simple HTTP trigger, respond to events originating from 3rd party systems like GitHub, Slack, Stripe, or from anywhere that can send HTTP requests.
Lightweight APIs Compose applications from lightweight, loosely coupled bits of logic that are quick to build and that scale instantly. Your functions can be event-driven or invoked directly over HTTP/S.
Mobile Backend Use Google's mobile platform for app developers, Firebase, and write your mobile backend in Cloud Functions. Listen and respond to events from Firebase Analytics, Realtime Database, Authentication, and Storage.
IoT Imagine tens or hundreds of thousands of devices streaming data into Cloud Pub/Sub, thereby launching Cloud Functions to process, transform and store data. Cloud Functions lets you do in a way that's completely serverless. 
index.js:
/**
* Background Cloud Function to be triggered by Pub/Sub.
* This function is exported by index.js, and executed when
* the trigger topic receives a message.
*
* @param {object} data The event payload.
* @param {object} context The event metadata.
*/
exports.helloWorld = (data, context) => {
const pubSubMessage = data;
const name = pubSubMessage.data
    ? Buffer.from(pubSubMessage.data, 'base64').toString() : "Hello World";
console.log(`My Cloud Function: ${name}`);
};


gcloud functions deploy helloWorld \
  --stage-bucket [BUCKET_NAME] \
  --trigger-topic hello_world \
  --runtime nodejs8

gcloud functions describe helloWorld

Test function

DATA=$(printf 'Hello World!'|base64) && gcloud functions call helloWorld --data '{"data":"'$DATA'"}'

The cloud tool returns the execution ID for the function, which means a message has been written in the log.

View logs or go Logging > Logs Explorer

gcloud functions logs read helloWorld

Deploying function

gcloud deploy cloud functions $FUNCTION_NAME \
--region=$REGION \
--entry-point=$ENTRY_POINT \
--memory=$MEMORY \
--runtime=$RUNTIME \
--service-account=$SERVICE_ACCOUNT\
--source=$SOURCE \
--stage-bucket=$STAGE_BUCKET \
--timeout=$TIMEOUT \
--retry
  • $REGION: The region of the function.
  • $ENTRY_POINT: The name of the function (as defined in the source code).
  • $MEMORY: The limit on the amount of memory that can be used. The allowed value are 128 MB, 256 MB, 512 MB, 1,024 MB, and 2,048 MB.
  • $SERVICE_ACCOUNT: The IAM service account associated with the function.
  • $SOURCE: The source code's location. Can be either Cloud Storage, a source repository, or the local filesystem.
  • $STAGE_BUCKET: If a function is deployed from a local directory, it defines the name of the Cloud Storage bucket that the source code will be stored in.
  • $TIMEOUT: The function's execution timeout.
  • --retry: Applies only to background functions. If present, it defines that the function should retry running if it's not executed successfully.

Triggers

When deploying a new function, you must specify

  • --trigger-topic,
  • --trigger-bucket,
  • --trigger-http.

When deploying an update to an existing function, the function keeps the existing trigger unless otherwise specified

To define an HTTP trigger, use the following command:

--trigger-http

An endpoint will be assigned to the function.

To trigger a function on changes to a Cloud Storage bucket, use the following command:

--trigger-bucket=$TRIGGER_BUCKET

$TRIGGER_BUCKET: The Google Cloud Storage bucket name. Every change that's made to the files in this bucket will trigger function execution.

For Cloud Storage, Cloud Functions supports the following trigger types:

  • google.storage.object.finalize
  • google.storage.object.delete
  • google.storage.object.archive
  • google.storage.object.metadataUpdate

To trigger a function on messages that are arriving in a Pub/Sub queue, use the following command:

--trigger-topic=$TRIGGER_TOPIC

$TRIGGER_TOPIC: The name of the Pub/Sub topic. Messages arriving in the queue will trigger this function. The message's content will be passed to the function.

For other sources, such as Firebase, use the following command:

--trigger-event=$EVENT_TYPE
--trigger-resource=$RESOURCE

Here, we have the following options: • $EVENT_TYPE: The action that should trigger the function • $RESOURCE: A resource from which the event occurs

example of configuring a trigger from Pub/Sub:

gcloud functions deploy hello_pubsub --runtime python37
--trigger-topic mytopic

Some more advanced triggers, such as using Firebase authentication. Check out https://cloud.google.com/functions/docs/calling/ for examples for every possible trigger.

IAM

  • Cloud Function Admin: Has the right to create, update, and delete functions. Can set IAM policies and view source code.
  • Cloud Functions Developer: Has the right to create, update, and delete functions, as well as view source code. Cannot set IAM policies. • Cloud Functions Viewer: Has the right to view functions. Cannot get IAM policies, nor view the source code. • Cloud Function Invoker: Has the right to invoke an HTTP function using its public URL.

Note that for the Cloud Functions Developer role to work, you must also assign the user the IAM Service Account User role on the Cloud Functions runtime service account.

Limits

Three types of quotas:

  • Resource limits: Defines the total amount of resources your functions can consume
  • Time limits: Defines how long things can run for
  • Rate limits: Defines the rate at which you can call the Cloud Functions API

Free tier: 2 million invocations, 1 million seconds of compute time, and 5 GB of egress network traffic.

Resource