Setting up the Cloudfront collector - OXYGEN-MARKET/oxygen-market.github.io GitHub Wiki

HOME » SNOWPLOW SETUP GUIDE » Step 1: setup a Collector » Setting up the Cloudfront collector

The Cloudfront collector is the most common collector employed by Snowplow uses.

A tracking pixel (called i) is uploaded to Amazon Cloudfront CDN. The Snowplow Tracker sends data to the collector by making a GET request for the pixel, and appending the data to be passed to the pixel query string. The Cloudfront Collector uses Cloudfront logging to record the request (including the query string) to S3.

1. Simple and robust (no moving parts). All the collector does is faithfully log GET requests from trackers. Because logging is done using the standard Amazon Cloudfront logging, it is incredibly reliable.
2. Scalable. The Cloudfront collector is powered by Amazon's cloud infrastructure: specifically its content delivery network, which is built to billions of requests per day.

This guide assumes you have:

• An account with Amazon Web Services
• S3 and Cloudfront enabled on your AWS account

Setting up the Cloudfront Collector is a five stage process:

1. Setup a bucket on Amazon S3 for the 1x1 tracking pixel i. This is the pixel that will be requested by every GET made by the Snowplow tracker.
2. Upload the tracking pixel to the bucket.
3. Create a bucket on S3 for the Snowplow logs, generated by the Cloudfront collector.
4. Create a Cloudfront distribution for serving the tracking pixel that is now stored in S3. This will ensure that the pixel is fetched very quickly (using Cloudfront's CDN) and crucially we will use Cloudfront logging to record every request made of the tracking pixel. These requests will contain all the data passed to the collector from the tracker, appended to the GET request in the form of a query string.
5. Test your tracking pixel on Cloudfront.

Note: We recommend running all Snowplow AWS operations through an IAM user with the bare minimum permissions required to run Snowplow. Please see our IAM user setup page for more information on doing this.

You could also send your payload to Iglu webhook adapter. For that, the tracking pixel would also have to be uploaded to com.snowplowanalytics.iglu folder as v1 image (the instructions are similar to those outlined in Pre-requisites):

• Create folder com.snowplowanalytics.iglu in the S3 bucket you set up earlier
• Rename i tracking pixel into v1 and upload it to that folder

Return to the setup guide.