mxp.plugins.GeoBatchFlows - geosolutions-it/MapStore GitHub Wiki

In order to use this plugin you need to include this files in the build.cfg:

• src/mxp/plugins/GeoBatchFlows.js
• src/mxp/widgets/grid/GeoBatchFlowsGrid.js
• src/mxp/widgets/grid/GeoBatchConsumerGrid.js
• src/mxp/widgets/form/GeoBatchRunLocal.js
• src/mxp/widgets/form/GeoBatchRunLocalForm.js

and their dependencies

NOTE: In the default configuration these files are present.

This plugin adds a new admin panel to start and manage GeoBatch ingestion flows.

GeoStore is used to archive completed flows.

The panel contains a list of available flows on the left, with the ability to start a new run for one of them.

On the right, a grid shows the list of runs for the currently selected flow, with the ability to choose between Active and Archived runs.

The plugin is quite complex and can require a detailed configuration of the flows.

You can configure it adding this plugin with this configuration:

    {
         "ptype": "mxp_geobatch_flows",
         "geoBatchRestURL":"http://localhost:9000/geobatch/rest/",
         "geoStoreRestURL":"http://localhost:8082/geostore/rest/",
         "skipFlowsNotInRunConfigs": true,
         "showConsumersDetails": true,
         "forceOrder": true,
         "consumersPlugins": [
            ...
         ],
         "autoOpen": true,
         "runConfigs": {
            ...
         },
         "actionTarget":{
           "target": "north.tbar",
           "index": 2
         }
    }

The basic parameters are:

• geoBatchRestURL: The URL of the GeoBatch rest service
• geoStoreRestURL: The URL of the GeoStore rest service (used for runs archival)
• skipFlowsNotInRunConfigs if true, only flows configured in the runConfigs section will be shown on the flows (left) panel, if false all the flows available in GeoBatch will be shown.
• showConsumersDetails: if true enables a detail collapsable panel on each run row (in the right panel); this detail panel will show progress and info for each flow action, in addition to global progress and info
• forceOrder: if true forces the order of flows in the left panel to be sorted according to order attributes in flows configuration in the runConfigs section
• consumersPlugins: custom plugins list for the runs grid
• autoOpen if true the panel will be shown on startup.
• runConfigs list of configuration objects for the single flows (explained in detail in the next section).

Each flow needs to be configured in the runConfigs section of the plugin configuration.

The exact configuration parameters depend on the flow type.

Each flow type is implemented by a different ExtJS plugin (xtype).

Some default plugins are present in standard MapStore, others can be implemented for specific usage.

The list of standard flow types is:

• geobatch_run_local: this flow type doesn't require any input from the user, the flow is started using an empty geobatch.run file
• geobatch_run_local_form: this flow type allows the user to choose a file from a preconfigured folder (using a file manager widget) or upload a new one to be submitted to GeoBatch as the flow input

Each flow needs a configuration block inside the runConfigs section.

The block needs to have the GeoBatch flow id as a key (the name of the flow configuration XML file), so that the configuration is automatically matched with the corresponding GeoBatch flow.

Some configuration properties are mandatory for any flow type.

    "runConfigs": {
        "flow_id":{
            "xtype": "...",
            "order": 1
        }
    }

• xtype: widget implementing the flow input form and starting it
• order: optional number indicating the order of the flow in the list (if the forceOrder option is true)

    "runConfigs": {
        "flow_id":{
            "xtype": "geobatch_run_local",
            "order": 1
        }
    }

PS. MapStore and GeoBatch application needs to share a folder for file uploading and ingestion

geobatch_run_local_form uses a simplified file manager plugin internally, to show files to be ingested and allow the upload of new files. Each flow can use a different folder so that files are logically organized.

    "runConfigs": {
        "flow_id":{
            "xtype": "geobatch_run_local_form",
            "baseDir": "/geobatch/upload/folder",
            "fileRegex": "\\.zip$",
            "path":"/folder",
            "order": 1
        }
    }

These additional configuration properties are needed:

• baseDir: absolute path of a server folder where the uploaded file will be copied for ingestion (needs to be shared between the MapStore application and GeoBatch application)
• fileRegex: Regular Expression filter for files to be shown for ingestion by the file manager
• path: sub path of the file manager root from which the user will choose a file to ingest

To enable a correct progress / task visualizzation for flow instances, GeoBatch flows need to be configured accordingly.

Moreover a recent version of GeoBatch (from the master branch) has to be used.

These are the requirements for flows configuration:

• the EventConsumerConfiguration section needs to contain a listenerId for a LoggingProgressListener; the LoggingProgressListener has to be referenced only by EventConsumerConfiguration so that it can track global progress / task
• each ActionConfiguration section needs to contain a listenerId for a LoggingProgressListener; the LoggingProgressListener has to be referenced only by that ActionConfiguration so that it can track the action progress / task
• the EventConsumerConfiguration and each ActionConfiguration needs to contain a listenerId for a CumulatingProgressListener; the same CumulatingProgressListener has to be shared by all the sections (global and actions)
• the CumulatingProgressListener needs to have the appendToListenerForwarder property set to true

<FlowConfiguration>

    <id>flow_id</id>
    ...

    <EventConsumerConfiguration>

        <listenerId>ConsumerLogger0</listenerId>
        <listenerId>Cumulator</listenerId>

        <Action1Configuration>
            <listenerConfigurations/>
            
            <!-- Listeners -->
            <listenerId>ConsumerLogger1</listenerId>
            <listenerId>Cumulator</listenerId>
        </Action1Configuration>
 
        <Action2Configuration>
            <listenerConfigurations/>
            
            <!-- Listeners -->
            <listenerId>ConsumerLogger2</listenerId>
            <listenerId>Cumulator</listenerId>
        </Action2Configuration>

    </EventConsumerConfiguration>


    <ListenerConfigurations>
        <LoggingProgressListener>
            <serviceID>loggingListenerService</serviceID>
            <id>ConsumerLogger0</id>
            <loggerName>it.geosolutions.ConsLogger</loggerName>
        </LoggingProgressListener>
        <LoggingProgressListener>
            <serviceID>loggingListenerService</serviceID>
            <id>ConsumerLogger1</id>
            <loggerName>it.geosolutions.ConsLogger</loggerName>
        </LoggingProgressListener>
        <LoggingProgressListener>
            <serviceID>loggingListenerService</serviceID>
            <id>ConsumerLogger2</id>
            <loggerName>it.geosolutions.ConsLogger</loggerName>
        </LoggingProgressListener>
        
        <CumulatingProgressListener>
            <serviceID>cumulatingListenerService</serviceID>
            <id>Cumulator</id>
	    <appendToListenerForwarder>true</appendToListenerForwarder>
        </CumulatingProgressListener>
    </ListenerConfigurations>

</FlowConfiguration>