Weather enrichment - OXYGEN-MARKET/oxygen-market.github.io GitHub Wiki

HOME > SNOWPLOW SETUP GUIDE > Step 3: Setting up Enrich > Configurable enrichments > Weather enrichment

JSON Schema iglu:com.snowplowanalytics.snowplow.enrichments/weather_enrichment_config/jsonschema/1-0-0 Compatibility R74+ Data provider OpenWeatherMap

This enrichment uses OpenWeatherMap service to look up weather conditions in which event has been occurred.

There are five possible fields you can add to the "parameters" section of the enrichment configuration JSON: "apiKey", "cacheSize", "geoPrecision", "apiHost", and "timeout".

• apiKey is your key you need to obtain from OpenWeatherMap. Notice that free key coulnd't be used for weather enrichment, you need to subscribe paid plan.
• cacheSize is amount of requests underlying client need to store. Usually it's amount requests for your plan, plus 1% for errors
• timeout is a time in seconds after which request should be considered failed. Notice that failed weather enrichment will filter out whole your event, whether this failure be timeout or invalid API key
• apiHost is one several (history.openweathermap.org, api.openweathermap.org, pro.openweathermap.org) API hosts. For most cases history.openweathermap.org should be fine
• geoPrecision is fraction of one to which geo coordinates will be rounded for storing in cache. Less precise value is 1 assume ~60km infelicity in worst cast, most precise value is 10 assume ~6km infelicity.

Here's pretty common configuration suited for starter plan:

{
    "schema": "iglu:com.snowplowanalytics.snowplow.enrichments/weather_enrichment_config/jsonschema/1-0-0",

    "data": {
        "enabled": true,
        "vendor": "com.snowplowanalytics.snowplow.enrichments",
        "name": "weather_enrichment_config",
        "parameters": {
            "apiKey": "{{KEY}}",
            "cacheSize": 5100,
            "geoPrecision": 1,
            "apiHost": "history.openweathermap.org",
            "timeout": 5
        }
    }
}

To produce the enriched weather-related data, the values of latitude, longitude, and time are passed with API request to OpenWeatherMap.

Ultimately ip parameter (mapped to user_ipaddress) is used to obtain latitude and longitude geographical coordinates. It is achieved by yet another enrichment, namely IP lookups enrichment, which produces geo_latitude and geo_longitude.

The time parameter is the derived_tstamp value calculated during the common enrichment process. To find out more about derived_tstamp, please, follow this link.

This enrichment uses 3rd party, OpenWeatherMap, service to look up the weather conditions in which the event occurred.

As a reminder, bear in mind how geoPrecision works. Its value ranges from 1 to 10. Less precise value is 1 and it assumes ~60km infelicity in worst cast. Most precise value is 10 which assumes ~6km infelicity.

The data generated by this enrichment conforms to the org.openweathermap/weather JSON schema.

An example of the returned data follows. This is the kind of stringified JSON value which populates derived_contexts parameter/field in enriched/good data (post enrichment).

{
    "main": {
        "temp": 301.308,
        "temp_min": 301.308,
        "temp_max": 301.308,
        "pressure": 1021.91,
        "sea_level": 1024.77,
        "grnd_level": 1021.91,
        "humidity": 90
    },
    "wind": {
        "speed": 4.39,
        "deg": 190.002
    },
    "clouds": {
        "all": 0
    },
    "weather": [
        {
            "id": 800,
            "main": "Clear",
            "description": "Sky is Clear",
            "icon": "01d"
        }
    ],
    "dt": "1997-07-16T19:20+01:00"
}

Below is the summary of the output fields for the dedicated org_openweathermap_weather_1 table (post shredding and storage processes).