Overview

This module enables users to build a simple Storm topology (either "in-line enrichment" or "aggregation/alerting based on sliding windows") using Javascript snippets. The resulting topology can be inserted into streaming enrichments or analytics with only JSON configuration needed.

Security considerations

Currently the JS storm topology engine is not safe for non-admin users on secure clusters. Full JVM based sandboxing and RBAC is on the roadmap, watch this space. In the meantime access to this enrichment engine should be restricted to admin users (by restricting the read rights of the uploaded JAR).

Logging

The Storm Script Streaming Toplogy currently has no Aleph2 logging.

How to use aleph2_storm_script_topo

JavaScriptTopology

This is an example of using the JavaScriptBolt to do some custom string processing in javascript. An IP-number is split into network and subnet part. The script functions check() and splitI{ are used to process the data. The ReducerCounterBolt uses the results from the JavaScriptBolt to count occurrences of IP nuumber networks and emits the data.

JavaScriptTopology2

This is a more elaborate example of using a more generic JavaScriptMapperBolt and JavaScriptFolderBolt. The topology also uses a TimerSpout, additionally to the TopologyEntryPointSpout in order to enable periodic processing of IP number/subnet occurrences.

JavaScriptMapperBolt - Javascript functions

This bolt uses the function map(jsonIn) to process JSON data received from the TopologyEntryPoint. The function returns a java HashMap containing a "mapkey" and json string of "mapValueJson" , which contains a custom json object holding the mapped data.

JavaScriptFolderBolt - Javascript functions

The internal function fold(mapKey,mapValueJson) is not necessarily intended to be modified by user's java script. It extracts mapped data, retrieves the state and calls the subsequent update and store functions.

The user specific ** function update(mapKey,state)** updates the state and returns a new state object.

The user specific function checkEmit(mapKey,state) checks the state and emits an object as a JSON string if user specific criteria are met. Here a counter is checked bases on a threshold.

Since the internal state is modified based on a timestamp the function reset(mapKey) can be used to reset the state after emitting.

The store() function stores the folded data in an internal map.

Every time the timer spouts send a timer message to the JavaScriptFolderBolt the function allEntries() is used to retrieve the internal _map objects containing the current state. The Bolt then iterates over the map entries and calls checkEmit() and/or reset() to check if data is ready to be emitted.

Additionally the function emit() can be used directly inside Javascript to emit objects.