PubSub - bobbae/gcp GitHub Wiki

Cloud Pub/Sub is a fully-managed real-time messaging service that allows you to send and receive messages between independent applications.

https://www.youtube.com/watch?v=pU1zA-DMlWk

PubSub IoT Example

Cloud IoT Core to push airline data to PubSub, Dataflow, and BigQuery.

https://www.youtube.com/watch?v=-y3s6XY70Os

A list of task queues

https://taskqueues.com/

Kafka

Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

https://kafka.apache.org/

Kafka is open source. You can see its implementation:

https://github.com/apache/kafka

Review some examples:

https://github.com/apache/kafka/tree/trunk/examples/src/main/java/kafka/examples

Using Kafka and Confluent allows customers to integrate legacy systems and Google services like BigQuery and Dataflow in real time.

https://www.youtube.com/watch?v=LjF8IqKKbWc

MQTT

https://www.integrasources.com/blog/mqtt-protocol-iot-devices/

AWS SQS, SNS, Kinesis

https://medium.com/@campfirecode/sqs-vs-sns-vs-kinesis-which-aws-messaging-service-to-use-1fa3aa6be97d

Mocking Events in Pub/Sub

https://medium.com/@mattbancroft03/mocking-events-in-pub-sub-triggered-cloud-functions-gcp-python-5cccc726b6ad

Secure pub sub flow with terraform

https://medium.com/google-cloud/setting-up-secure-pub-sub-flow-with-go-and-terraform-6b779026a3e1

Push messages to Cloud Function endpoint in a different GCP project

https://blog.searce.com/pub-sub-push-messages-to-cloud-function-endpoint-in-a-different-gcp-project-d8939434503

Qwiklabs

Google Cloud Pub/Sub: Qwik Start - Python

Getting Started with Apache Kafka and Confluent Platform on Google Cloud

PubSub - bobbae/gcp GitHub Wiki

PubSub Lite

Reservations

Choosing PubSub or PubSub lite

Using Terraform to create BigQuery subscriptions to read from PubSub

Cloud Tasks