SnowPlow Tracker Protocol - OXYGEN-MARKET/oxygen-market.github.io GitHub Wiki

HOME > SNOWPLOW TECHNICAL DOCUMENTATION > Trackers

Snowplow trackers fire events, which are either GET or POST requests of a Snowplow collector, whenever an event on a website or application takes place. By appending events' parameters and values to the end of those GET requests or adding them into the body of POST messages, trackers can pass data into the collectors, for processing by Snowplow.

The Snowplow Tracker Protocol is the list of all the parameters that Snowplow trackers use when firing events to push data into the Snowplow collectors. Each parameter maps onto one or more fields in the Snowplow events table employed in storage. Here we document which field in the Snowplow events table each parameter added to the query string maps onto.

Snowplow has been architected to be as easy as possible for developers to create their own alternative subsystems. This documentation should be used by anyone who would like to build their own tracker: by utilising the parameters documented here, the author of a new tracker can be confident that his / her tracker will work with the rest of the Snowplow stack, and be confident where the values associated with each parameter on every call will be available to query in Snowplow, whether that's in Hive or Infobright or another database.

Please note that the end point where the GET/POST request should be made depends on which collector is used. Refer to the collectors documentation for more information. Also take into consideration that Clodfront collector accepts only GET requests.

In the first part of this guide, we cover the parameters in the Snowplow tracker protocol that are common across different event types. In the second part, we document the parameters that are relevant for specific events that are recognised in the Snowplow event model. Please note: this model is evolving over time as we incorporate more events and grow the set of fields associated with each of the standard events. In all cases, we do our best to ensure that any changes are backwards compatible. (So we are happy adding new parameters, but do not remove parameters once they have been incorporated.)

Back to top

• 1 Common parameters (platform and event independent)
  • 1.1 Application parameters
  • 1.2 Date / time parameters
  • 1.3 Event / transaction parameters
  • 1.4 Snowplow tracker version
  • 1.5 User parameters
  • 1.6 Device parameters
• 2 Platform-specific parameters
  • 2.1 Web-specific parameters
  • 2.2 Internet of Things-specific parameters
• 3 Snowplow events
  • 3.1 Pageview events
  • 3.2 Page pings
  • 3.3 Link clicks
  • 3.4 Ad impressions
  • 3.5 Ecommerce transactions
  • 3.6 Social events
  • 3.7 Item views
  • 3.8 Error tracking
  • 3.9 Custom structured events
  • 3.10 Custom unstructured events
• 4 Custom contexts
• 5 Reserved parameters

The tracker namespace parameter is used to distinguish between different trackers. The name can be any string that does not contain a colon or semi-colon character. Tracker namespacing allows you to run multiple trackers, pinging to different collectors.

The event vendor parameter makes it possible to distinguish between events defined by different companies. Note: the event vendor parameter is deprecated.

The application ID parameter is used to distinguish data from different websites and applications.

As a Snowplow user, you can define application IDs for each of your different digital products and track behaviour of your users across all of them using the same Snowplow instance by setting the app_id in your tracker of choice.

Potential platform values: (to finalise and complete...)

Back to common field types.

It is possible to record the time that an event occurs on the clients-side (i.e. in the tracker), or server side (i.e. by the collector). When using the JavaScript tracker to track web events, it makes sense to rely on the collector logs to identify the time that events occured, as Snowplow tracking tags are fired as events happen, and so the time they are received server-side should be an accurate representation of the time the event being tracked occured. In other situations (e.g. when using mobile trackers), the time the collector receives the data may be sometime after an event occurred, and so it makes sense to record the timestamp on the client-side, in which case this is handled by the tracker.

The ttm field is used for a timestamp set on the client which should be taken as accurate. For example, it should be used when using a tracker to process a batch of existing events with known timestamps independent of the device clock for the tracker.

The tracker can pass a client-side timestamp to the collector using the above parameters.

Back to common field types.

Every line of data passed from the tracker should contain an event field (e) to denote the type of event being tracked. For details about the potential values that e can take, and the corresponding event types that they refer to, see the section detailing Snowplow events.

The transaction ID (tid) can be used in situations where there is a risk of duplicate records for the same event. In this case, the transaction ID can be used to aid deduping of records. This field has been deprecated in favour of the event ID.

The event ID (eid) is the unique identifier (UUID) for this row. Historically we generated this in the Enrichment process but where possible we are migrating this to being generated in the trackers.

Back to common field types.

For deployments where multiple trackers are used (e.g. for businesses that use the JavaScript tracker to track events on their domains alongside the Pixel tracker to track events on 3rd party domains), it is useful to be able to distinguish data generated from each tracker. It can also be useful when tracker versions are updated, so that it is easier to see if an update in tracker accounts for a feature of the data at analysis time.

Back to common field types.

We recommend always setting the uid / user_id parameter: as this is the cornerstone of all customer-centric analytics.

In contrast, setting vid / visit_id is optional. It is possible to define when sessions begin and end client-side, in the tracker. But it is equally possible to define session start and stop times at the ETL or analytics phase, in which case it need not be set in the tracker at all. Note: Google Analytics defines sessions server side.

Setting ip is optional: Snowplow collectors log IP address as standard. However, you can override the value derived from the collector by populating this value in the tracker. This is useful, if traffic is being proxied to a Snowplow collector, for example.

If you are tracking events both server-side and client-side and you want the network_userid to be set in your server-side events, use the tnuid parameter.

Back to common field types.

We intend to build out the list of device related properties over time.

Back to common field types.

In addition, there is a set of browser-specific parameters that only makes sense to record for events that happen on web platforms (p=web). These parameters are relevant across all web events, regardless of the event type. (E.g. if it is a pageview, pageping, transaction, media play etc...)

Back to common field types.

In addition, there is a set of device-specific parameters that only makes sense to record for events that happen on the Internet of Things (p=iot). These parameters are relevant across all Internet of Things events, regardless of the event type:

Back to common field types.

At its heart, Snowplow is a platform for granular tracking of events. Currently, Snowplow understands the following events. In the tracker protocol, each event is denoted by an e=... parameter.

We are working to make the data model for each of the above events richer, and expand the 'Snowplow event library' to support a wider selection of events that businesses running Snowplow wish to track.

In each case, we use the &e parameter to indicate the type of event that is being tracked by Snowplow to the value indicated in the above table

Pageview tracking is used to record views of web pages.

Currently, recording a pageview involves recording an event where e=pv. All the fields associated with web events can be tracked. There are no other pageview specific fields:

// Key common parameters
duid=aeb1691c5a0ee5a6    // Domain user ID
&vid=2                  // Domain session index
&tid=508780             // Transaction ID
&aid=pbzsite            // App ID
&p=web 				// Platform ID

// Key data points related to page view
&e=pv                   // event = page view
&url=http%3A%2F%2Ftest.psybazaar.com%2F2-tarot-cards    // Page URL
&page=Tarot%20cards                                     // Page title
&refr=http%3A%2F%2Ftest.psybazaar.com%2F                // Referrer URL

// Other browser-specific parameters
&lang=en-US
&fp=3511643688
&f_pdf=1
&f_qt=0
&cd=32
&cookie=1
&tz=Europe%2FLondon

e=pv           // page view
&aid=pbzsite   // app_id
&p=web         // platform
&tid=580794    // transaction ID
&dtm=1361555202287  // client timestamp

&page=Psychic Bazaar                  // Page Title
&url=http=//www.psychicbazaar.com/    // Page URL
&ds=1120x1848                         // Document dimensions
&cs=UTF-8                             // Document character set

&res=1920x976                         // Device monitor dimensions
&vp=873x390                           // Viewport dimensions
&duid=91a88a7ec90ebbb1                // Domain user ID
&fp=3324966434                        // User fingerprint
&vid=3                                // Domain session ID

&tv=js-0.13.1                         // Tracker version
&lang=en-GB                           // Browser language
&tz=Europe/London                     // Client time zone

Back to event tracking.

Page pings are used to record users engaging with content on a web page after it has originally loaded. It can be used to track e.g. how far down an article a user scrolls.

If enabled, the page ping function checks for engagement with a page after load. (E.g. mousemovement, scrolling etc...) If there is some sort of engagement in a specified time interval, a page ping is sent.

Page pings are identified by e=pp. As well as all the standard web fields, there are four additional fields that pp includes, which are used to identify how users are scrolling over web pages:

Example:

e=pp        // Page ping
// Max and min x and y offsets
pp_mix=0    
pp_max=7
pp_miy=0
pp_may=746

// Other relevant fields
duid=91a88a7ec90ebbb1 // Domain user id
vid=1                 // Domain session index
page=Tarot cards - Psychic Bazaar    // Page title
refr=http=//www.psychicbazaar.com/   // Page referrer
url=http://www.psychicbazaar.com/2-tarot-cards // Page URL

tid=344664            // Transaction ID
dtm=1361534887845     // Client timestamp
vp=1105x390           // Viewport dimensions
ds=1097x1413          // Document dimensions
aid=pbzsite           // App ID
lang=en-GB            // Browser language
cs=UTF-8              // Docuemnt characterset
res=1920x976          // Monitor resolution / size

Back to event tracking.

Link clicks are treated as unstructured events. The schema for a link click event can be found here.

Back to event tracking.

Deprecation warning: the special ad impression querystring parameters described in this section are deprecated. Instead, ad impressions, ad clicks, and ad conversions are all treated as unstructured events. There schemas are available at the following locations:

Ad impression schema

Ad click schema

Ad conversion schema

As well as setting e=ad, there are four specific parameters that can be set when an ad impression is tracked:

Note: if possible, it often makes sense to pass in the user_id generated by your ad server into the ad_uid field, so that this can be matched with the user_id generated by Snowplow at analysis time.

Example:

duid=aeb1691c5a0ee5a6   // Domain user ID  
&vid=2                  // Domain session ID
&tid=508780             // Transaction ID  
&aid=1                  // App ID

&e=ad                    // event = ad impression
&ad_ba=126422315640      // banner ID
&ad_ca=d-546135          // campaign ID
&ad_ad=diageo            // advertiser ID
&ad_uid=0cbffbf8-a9c5-426f-9369-6e53f1677efc      // user ID

Back to event tracking.

To track an ecommerce transaction, fire a transaction event (e=tr) to register the transaction, and then fire item events (e=ti) to log specific data about the items that were part of that transaction. The order_id, (captured using the ti parameter) is used to link the transaction-level and item-level data at analysis time.

Transaction event example:

duid=aeb1691c5a0ee5a6   // Domain user ID  
&vid=2                  // Domain session index
&aid=1                  // App ID

&e=tr            	// Transacton event type.
&tr_id=12345        // Order ID
&tr_af=westernWear 	// Affiliation
&tr_tt=19.99 		// Transaction total value
&tr_tx=4.99 		// Transaction tax value
&tr_sh=2.99 		// Transaction shipping price
&tr_ci=london		// City on customer address
&tr_st=london 		// State on customer address
&tr_co=united kingdom	// Country on customer address
&tr_cu= GBP    // Transaction currency

Item hit example:

uid=aeb1691c5a0ee5a6    // User ID  
&vid=2                  // Visit ID (i.e. session number for this user_id)  
&tid=508780             // Transaction ID  
&aid=1                  // App ID
&tv=js-0.5.2            // Tracker version

&e=ti 				// Transaction item event type
&ti_id=12345 		// Order ID
&ti_sk=pbz0025 		// Item SKU
&ti_na=black-tarot 	// Item name
&ti_ca=tarot 		// Item category
&ti_pr=7.99 		// Item price
&ti_qu=1 			// Item quantity
&ti_cu=USD      // Currency

Back to event tracking.

Note! This has not been implemented yet.

uid=aeb1691c5a0ee5a6    // User ID  
&vid=2                  // Visit ID (i.e. session number for this user_id)  
&tid=508780             // Transaction ID  
&aid=1                  // App ID
&tv=js-0.5.2            // Tracker version

&e=s            // Social event type
&sa=like        // Social Action
&sn=facebook    // Social Network
&st=/home       // Social Target

Back to event tracking.

Pageviews track page load events. Itemviews track views of specific items e.g. articles on a content site, videos on a video site, or products on an online retail site.

This functionality has not been developed yet. When it is, it will be documented here.

Back to event tracking.

This functionality has not been developed yet. When it is, it will be documented here.

Back to event tracking.

Back to the top.

Custom event tracking is used to track events that are not natively supported by Snowplow. (Like ad impressions, page views, ecomm transactions.)

As well as setting e=ue, there are five custom event specific parameters that can be set:

Add-to-basket example:

uid=aeb1691c5a0ee5a6    // User ID  
&vid=2                  // Visit ID (i.e. session number for this user_id)  
&tid=508780             // Transaction ID  
&aid=1                  // App ID
&tv=js-0.5.2            // Tracker version

&e=se                    // event = custom  
&se_ca=ecomm            // event_category = ecomm  
&se_ac=add-to-basket    // event_action = add-to-basket  
&se_la=178              // event_label = 178 (product_id of item added to basket)  
&se_pr=1                // event_property = 1 (quantity of item added to basket)  
&se_va=14.99            // event_value = 14.99 (price of item added to basket)  

Watch-video-clip example:

uid=aeb1691c5a0ee5a6    // User ID  
&vid=2                  // Visit ID (i.e. session number for this user_id)  
&tid=508780             // Transaction ID  
&aid=1                  // App ID
&tv=js-0.5.2            // Tracker version

&e=se                    // event = custom  
&se_ca=video            // event_category = video  
&se_ac=play             // event_action = play  
&se_la=291              // event_label = 291 (video_id of video played)  
&se_pr=13.2             // event_property = 13.2 (number of seconds into video that clip starts playing)  
&se_va=0.0025           // event_value = 0.0025 (ad revenue associated with view)  

Back to event tracking.

Custom unstructured event tracking is used to track events that are not natively supported by Snowplow and allow arbitrary name: value pairs associated with the event.

An example of an unstructured event for a product view event:

{
  schema: 'iglu:com.my_company/viewed_product/jsonschema/1-0-0',
  data: {
    "product_id": "ASO01043",
    "price": 49.95
  }
}

The tracker will wrap this self-describing JSON in an outer self-describing JSON, which is what gets sent:

{

  // Tells Snowplow this is an unstructured event
  schema: 'iglu:com.my_company/unstruct_event/jsonschema/1-0-0',
  data: {

    // Tells Snowplow this is a viewed_product event
    schema: 'iglu:com.my_company/viewed_product/jsonschema/1-0-0',
    data: {

      // The event data itself
      "product_id": "ASO01043",
      "price": 49.95
    }
  }
}

As well as setting e=ue, there are two custom event specific parameters that can be populated with the outer self-describing JSON:

The tracker can decide to pass the ue_pr or the ue_px parameter depending on configuration. Encoding properties into URL-safe Base64 allows for more data while sacrificing readability.

viewed_product example (using percent encoding and the key ue_pr):

uid=aeb1691c5a0ee5a6   // User ID  
&vid=2                 // Visit ID (i.e. session number for this user_id)  
&tid=508780            // Transaction ID  
&aid=1                 // App ID
&tv=js-0.13.1          // Tracker version

&e=ue                  // event = unstructured  
&ue_pr="%7B%22schema%22:%22iglu:com.my_company/unstruct_event/jsonschema/1-0-0%22,%22data%22:%7B%22schema%22:%22iglu:com.my_company/viewed_product/jsonschema/1-0-0%22,%22data%22:%7B%22product_id%22:%22ASO01043%22,%22price%22:49.95%7D%7D%7D"
                       // event_properties =  {
                                                schema: 'iglu:com.my_company/unstruct_event/jsonschema/1-0-0',
                                                data: {
                                                  schema: 'iglu:com.my_company/viewed_product/jsonschema/1-0-0',
                                                  data: {
                                                    "product_id": "ASO01043",
                                                    "price": 49.95
                                                  }
                                                }
                                              }

viewed_product example (using base 64 encoding and the key ue_px):

uid=aeb1691c5a0ee5a6   // User ID  
&vid=2                 // Visit ID (i.e. session number for this user_id)  
&tid=508780            // Transaction ID  
&aid=1                 // App ID
&tv=js-0.13.1          // Tracker version

&e=ue                  // event = unstructured  
&ue_px=ew0KICBzY2hlbWE6ICdpZ2x1OmNvbS5teV9jb21wYW55L3Vuc3RydWN0X2V2ZW50L2pzb25zY2hlbWEvMS0wLTAnLCANCiAgZGF0YTogew0KICAgIHNjaGVtYTogJ2lnbHU6Y29tLm15X2NvbXBhbnkvdmlld2VkX3Byb2R1Y3QvanNvbnNjaGVtYS8xLTAtMCcsDQogICAgZGF0YTogew0KICAgICAgInByb2R1Y3RfaWQiOiAiQVNPMDEwNDMiLCANCiAgICAgICJwcmljZSI6IDQ5Ljk1DQogICAgfQ0KICB9DQp9
                       // event_properties = {
                                                schema: 'iglu:com.my_company/unstruct_event/jsonschema/1-0-0',
                                                data: {
                                                  schema: 'iglu:com.my_company/viewed_product/jsonschema/1-0-0',
                                                  data: {
                                                    "product_id": "ASO01043",
                                                    "price": 49.95
                                                  }
                                                }
                                              }

Back to event tracking.

Custom contexts can be used to attach additional data in the form of a JSON to any Snowplow event.

Each individual custom context is a self-describing JSON such as:

{
  schema: 'iglu:com.my_company/user/jsonschema/1-0-0'
  data: {
    fb_uid: '9999xyz'
  }
}

All custom contexts to be attached to an event will be wrapped in an array by the user and passed to the tracker, which will wrap them in a self-describing JSON:

{

  // Tells Snowplow this is an array of custom contexts
  schema: 'iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-0'
  data: [
    {

      // Tells Snowplow that this is a "user" context
      schema: 'iglu:com.my_company/user/jsonschema/1-0-0',
      data: {

        // The context data itself
        fb_uid: '9999xyz'
      }
    }
  ]
}

The tracker can be configured to encode the context into URL-safe Base64 to ensure that no data is lost or corrupted. The downside is that the data will be bigger and less readable. Otherwise the data will be percent-encoded.

Example of a custom context attached to the watch-video-clip structured event from above using percent encoding and the key co:

uid=aeb1691c5a0ee5a6    // User ID  
&vid=2                  // Visit ID (i.e. session number for this user_id)  
&tid=508780             // Transaction ID  
&aid=1                  // App ID
&tv=js-0.5.2            // Tracker version

&e=se                   // event = custom  
&se_ca=ecomm            // event_category = ecomm  
&se_ac=add-to-basket    // event_action = add-to-basket  
&se_la=178              // event_label = 178 (product_id of item added to basket)  
&se_pr=1                // event_property = 1 (quantity of item added to basket)  
&se_va=14.99            // event_value = 14.99 (price of item added to basket)  

&co=%7B%22schema%22:%22iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-0%22,%22data%22:%5B%7B%22schema%22:%22iglu:com.my_company/user/jsonschema/1-0-0%22,%22data%22:%7B%22fb_uid%22:%229999xyz%22%7D%7D%5D%7D
                        // context =  {
                                        schema: 'iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-0'
                                        data: [
                                          {
                                            schema: 'iglu:com.my_company/user/jsonschema/1-0-0',
                                            data: {
                                              fb_uid: '9999xyz'
                                            }
                                          }
                                        ]}
                                      }

Example of a custom context attached to the watch-video-clip structured event from above using URL-safe Base64 encoding and the key cx:

uid=aeb1691c5a0ee5a6    // User ID  
&vid=2                  // Visit ID (i.e. session number for this user_id)  
&tid=508780             // Transaction ID  
&aid=1                  // App ID
&tv=js-0.5.2            // Tracker version

&e=se                   // event = custom  
&se_ca=ecomm            // event_category = ecomm  
&se_ac=add-to-basket    // event_action = add-to-basket  
&se_la=178              // event_label = 178 (product_id of item added to basket)  
&se_pr=1                // event_property = 1 (quantity of item added to basket)  
&se_va=14.99            // event_value = 14.99 (price of item added to basket)  

&cx=ew0KICBzY2hlbWE6ICdpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wJyANCiAgZGF0YToge1sNCiAgICB7DQogICAgICBzY2hlbWE6ICdpZ2x1OmNvbS5teV9jb21wYW55L3VzZXIvanNvbnNjaGVtYS8xLTAtMCcgDQogICAgICBkYXRhOiB7DQogICAgICAgIGZiX3VpZDogJzk5OTl4eXonDQogICAgICB9DQogICAgfQ0KICBdfQ0KfQ==
                        // context =  {
                                        schema: 'iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-0'
                                        data: [
                                          {
                                            schema: 'iglu:com.my_company/user/jsonschema/1-0-0',
                                            data: {
                                              fb_uid: '9999xyz'
                                            }
                                          }
                                        ]}
                                      }

Back to top.

u is a reserved parameter because it is used for click tracking / URI redirects in the Pixel Tracker.

Back to top.