Miner Tools - minersoft/miner GitHub Wiki

Introduction

Miner tools are extensions that provides connectors to new data sources or add other functionality to miner. In this case they are python modules that implement simple api. Tools are also a convenient way to share miner related development .

Installation of Miner Tools

There are different tools that extends miner functionality. They can provide connectors for additional data sources or data output. Miner Tools get installed in the following way:

First to get list of known tools, type

>>> INSTALL

Then to install specific tool, type

>>> INSTALL http

To update list of known tools

UPDATE

To update specific tool to the latest version

UPDATE http

To update miner

UPDATE miner

To uninstall tool

UNINSTALL http

To start using tool functionality, type

USE http

You can put this line in ~/.minerrc file, then it will be automatically executed at miner start-up.

Tool management requires internet connection Http proxy settings if any are taken from Internet Explorer settings on Windows or http_proxy environment variable on Linux.

Alternatively tools may be installed from the manually downloaded tarball: INSTALL http http.tar.gz

Writing your code/script

If you want to develop some python modules or miner scripts for use within miner just create tool that points to folder containing your work.

INSTALL mytool path/to/my/work
USE mytool
IMPORT mymodule
PRINT mymodule.func()

Following tools are currently available:

  • http - parse urls and http headers
  • httpd_logs - reads ncsa (common log format) and customized apache logs
  • mysql - connector to mysql database
  • excel_SX - Allows to generate excel reports (but has many dependencies)
  • pcap - enables miner to read/write network captures