Overview - sedgewickmm18/mmfunctions GitHub Wiki
The mmfunctions repo serves as example how to build custom functions for the Analytics Service (AS) pipeline.
The two major steps required are
- building a python package and publishing it to a public repository (such as github.com)
- registering the new package with AS
To install this package clone the repository locally and run
pip install https://github.com/ibm-watson-iot/functions
and
pip install <mmfunctions>
Caveat: For Linux replace pip with pip3
The repository contains a few functions in the mmfunctions subdirectory along with the python init.py file to mark it as module.
What you need in a module
As usual start with the dependencies to be imported. Analytics Service functions are built around pandas dataframes along with typical python modules to work with dates, numbers, matrices etc. Furthermore AS makes use of SQL databases to get date from and store results. Last but not least the AS' data types and classes are available as the iotfunctions module.
import datetime as dt
import logging
import json
import pandas as pd
import numpy as np
from scipy import stats
from sqlalchemy import Column, Integer, String, Float, DateTime, Boolean, func
from iotfunctions.base import BaseTransformer,BaseSimpleAggregator,BaseComplexAggregator
from iotfunctions.metadata import EntityType
from iotfunctions.db import Database
from iotfunctions import bif
from iotfunctions import ui
from iotfunctions.ui import UIMultiItem, UISingle ,UISingleItem, UIFunctionOutSingle, UIFunctionOutMulti
from iotfunctions.enginelog import EngineLogging
Right after that you'll find global logging setting and the package URL assignment
PACKAGE_URL= "git+https://github.com/sedgewickmm18/mmfunctions@"
When registering a function or module, this variable tells the job where to install the code from - it runs pip install from the kubernetes cluster.
Caveat: There won't be any warning if it cannot install the code due to a typo ! Your job simply won't produce any data.
The rest are the custom classes that actually make up the payload of mmfunctions module.
The repository also comes with two helper commands, register.py and 'unregister.py` to test and register the module with AS and to unregister the functions in the module one by one.
Running register.py test with the additional parameter bypasses actual registration. Insteand it initializes an instance per class in the mmfunction module and runs the execute() method once.