Mal4s Configuration Files - secure411dotorg/mal4s GitHub Wiki

Mal4s uses two config files, one which has the settings specific to your own computer and the other which will change for various types of *.mal4s data files.

Mal4s will try to find default config files and will try to guess the name of the config files based on the name of the data file if possible. A data input file named anything--httpm.mal4s indicates for Mal4s to look for a text configuration file named httpm.conf, for example.

mal4s --text-config-dir ~/Downloads will help Mal4s to find your text config files automatically, for example if you are downloading them along with a data file into your Downloads dir. Another common place to keep text config files is /opt/mal4s.

You can optionally specify one or both config file names on the command line when you start Mal4s, such as:

mal4s --load-config /opt/mal4s/mal4s.conf --load-text-config /opt/mal4s/httpm.conf 2014-01-01--httpm.conf

Automatic selection of configuration files

mal4s.conf is automatically read from /usr/local/share/mal4s/ (Linux/Mac) or the under the INSTALLDIR\data\ directory (Windows) if the file exists, unless --load-config PATH2CONFIG is specified at the command line.

Mal4s will attempt to locate a matching text config file based on the input file name unless reading from STDIN. Text config files are searched for in the working directory first, then the text-config-dir, and then /usr/local/share/mal4s/ (Linux/Mac) or INSTALLDIR\data\ directory (Windows). To take advantage of text config matching, name your file following this format:

somename--textconfname.mal4s
sample--newns.mal4s

The textconfname portion of the file name will match textconfname.conf. sample--newns.mal4s will load newns.conf which is included in the mal4s share directory.

The text-config-dir may be specified in the main config with text-config-dir=/path2/, or at the command line with --text-config-dir /path2/.

--load-config mal4s.conf

The main config file could be named anything - for ease of referring to it we'll call it mal4s.conf. The first thing you may wish to customize is the viewport resolution for the size of your screen:

[display]
#viewport=1024x768
#viewport=1200x960
viewport=1920x1280

# Or for full screen:
#fullscreen=true

I use # at the start of lines that I want to comment out for no effect. By leaving several common screen resolutions in place, I can quickly switch for situations where I attach to an external monitor such as a projector.

The rest of the options which must be in the main config file are in a section headed [gource]

[gource]
# Expect data on STDIN if no file is specified on command line
path=-
# Path to images of any kind whose name plus ".png" will exactly match the "n1" field
# For example put a set of png files named for two letter country codes in the dir
plotter-image-dir=/opt/mal4s/flags
# To prevent custom commands from executing external programs set to true
disable-exec=false
# To prevent opening a browser from within Mal4s set to true
disable-browser=false
# To take a screenshot and exit after awhile at end of data, uncomment
#screenshot-at-end=true
# You will want stop-at-end=true - otherwise the display shrinks and folds up to nothing
stop-at-end=true
# Controls the amount of white fog by the lines and balls
bloom-intensity=.025
# Display and timing controls - probably already optimal for Mal4s
disable-auto-rotate=false
camera-mode=overview
file-idle-time=0
auto-skip-seconds=1
key=true
background-image=/opt/mal4s/dissectcyber.png
title=Get tools and data at https://DissectCyber.com
#caption-size SIZE - Font size of captions.
#caption-colour FFFFFF - Caption colours in hex (eg FF0000).
#caption-duration SECONDS - How long each captions appears for in seconds.
caption-duration 5
#caption-offset PIXELS - Horizontal offset of captions (0=centered)

Some additional esoteric settings probably not needed by most Mal4s users can be found by reading about "gource", the program for data visualization of version control logs.

--load-text-config newns.conf

--load-text-config PATH2CONFIG bypasses the search paths and attempts to load the configuration only from the path specified. This is required when using STDIN for the input data as the file name can not read from STDIN.

Included in the data/ dir of mal4s is a sample text config file named newns.conf as well as more sample text config files such as httpm.conf and dnsm.conf.

The main purpose of a text config file is to control how imported data fields display on screen when hovering the mouse over a particular ball.

Data fields and formatting

Summary of fields:

Description of mal4s file fields

######Non-displayable Fields:

A few of the fields are required by the original gource platform that Mal4s is based on, such as:

  • first field: epoch time stamp which controls display sequence / timing
  • third field: A(dd), M(odify) or D(elete)
  • fifth field: which is an RGB color instruction for the individual ball

The fields mentioned above are not addressable or displayable.

Displayable Field Identifiers

Pipe symbols | delimit all fields except the "branching" field which is delimited by forward slash /. Branching fields are what creates a new branching off of the diagram.

Let's say your main category is continents, roughly ARIN, RIPE, APNIC, LACNIC, AFRINIC for Internet hosts. All data branches off initially from a central point on one of those 5 branches. If your next branching category is the country, a new branch is created for each country represented in the data.

Each additional / separating data elements will cause another category level of branching. The end of branching activity is reached when you hit the next pipe symbol |.

Branching control fields are prefaced by a b; non-branching control fields are prefaced by an n.

A few fields have names, such as plotter, host, and tld. We tried to keep the names generic since Mal4s could be used to display things other than Internet hosts - however given how handy it is for Internet hosts, here we are with features and names related to Internet domain name syntax.

Example line: 1388892986|BIGROCK SOLUTIONS LIMITED|A|arin/US/198/56/238/61/NS1.SOMENSHOST.COM~NS2.SOMENSHOST.COM/Enzu Inc/NS2.COOLRINGSONLINE.COM|C638EE|us|18978|subpoena.coolringsonline.com|BIGROCK SOLUTIONS LIMITED|2006-07-18|OK

  • ${plotter} - The plotter name. This is the image that shoots the green lasers as it is plotting new round balls. In the example line, this is the first BIGROCK SOLUTIONS LIMITED.
  • ${host} - The host name. NS2.COOLRINGSONLINE.COM
  • ${tld} - The Top Level Domain of the host, in this case COM
  • ${bNUM} - The branch names. In the branching field, arin would be ${b1} and Enzu Inc would be ${b8}
  • ${nNUM} - Non-branching fields: ${n1} would display 18978, ${n2} is subpoena.coolringsonline.com, etc.

hover-box-format-sample

Formatting the lines

Under the [text] section of the config file, hover-line-NUM=FORMAT will define the hover text format.

[text]
# What to show by default if a field has no data:
hover-unset-field=(No Data)
# Up to 16 lines to display when hovering over a ball:
hover-line-1=${plotter}
hover-line-2=========================================
hover-line-3=${host}
hover-line-4=${n4} (registration date)
hover-line-5=
hover-line-6=${n3} (registrar)
hover-line-7=${n5} (status)
hover-line-8=
hover-line-9=${b7} (nameserver)
hover-line-10=========================================
hover-line-11=${b1} ${b2} ${n6} (RIR & country of IP)
hover-line-12=AS${n1}
hover-line-13=${b8} (organization of IP)
hover-line-14=
hover-line-15=${b3}.${b4}.${b5}.${b6}
hover-line-16=${n2} (ptr)

# When designing what to put in the display, I find it useful to make myself a guide to the fields in my data:
# GUIDE TO FIELD POSITIONS
# plotter	Email address of domain registrant if available otherwise registrar
# host		FQDN such as my.example.com
# tld		All characters to the right of last dot in host
#
# FIELDS STARTING WITH b CAUSE BRANCHING OF LINES
# b1		RIR such as arin, ripe, et al from IP whois
# b2		Two letter country code from IP whois
# b3		ipv4 1st octet
# b4		ipv4 2nd octet
# b5		ipv4 3rd octet
# b6		ipv4 4th octet
# b7		Authoritative name server host for my.example.com
# b8		Company name that IP address netblock is allocated to
#
# FIELDS STARTING WITH n HAVE NO BRANCHING EFFECT
# n1		Name of png to display for the plotter, such as a two letter country code
#		n1 works if you have a /flags/ dir with us.png, fr.png etc for every country code
# n2		Reverse DNS of the IP address aka PTR
# n3		Registrar
# n4		Registration date
# n5		Status of domain at registrar
# n6		Full name of country from IP whois

With the above config, the example line would display this on hover:

hover-box-format-sample

Mal4s supports a maximum of 16 hover lines. You must specify each line from 1 to the desired number of lines. No line number can be skipped. hover-unset-field=TEXT will replace any field that is out of the range of the data supplied to it with whatever you put as the TEXT, basically a default value.

Legend aka Key Field Counts:

By default a legend with counts displays on the left side of the screen. The field(s) to aggregate in the count are set using the key-format under the [text] section, normally located in the text config file.

[text]
key-format=${b2}${n1} ${tld}
key-width=130

With the above settings the legend will display the 2nd branching field (such as the two letter country code), next to the 1st non-branching field (such as the AS number), a space, and the TLD of the host.

You can adjust the width of the key with key-width. Key width should be an integer representing the desired width in pixels for the left hand legend display.

mal4s-sample

Timeline Captions

Mal4s can display captions along the timeline by specifying a caption file (using --caption-file) in the pipe ('|') delimited format below:

timestamp - A unix timestamp of when to display the caption.

caption - The caption to appear.

Example:

1275543595|GODADDY
1327553595|TUCOWS

Main configuration file options:

caption-size SIZE - Font size of captions.
caption-colour FFFFFF - Caption colours in hex (eg FF0000).
caption-duration SECONDS - How long each captions appears for in seconds.
caption-offset PIXELS - Horizontal offset of captions (0=centered).

Action configuration settings

The button actions for mouse click, f5, f7, f9, and O, for open, may be configured to execute any command the user is able to run at the command prompt. For this reason, command execution and open browser are disabled by default. You must add enable-exec and/or enable-browser to the main config or --enable-exec/--enable-browser on the command line to enable this feature.

Main config

[gource]
enable-exec=true
enable-browser=true
disable-exec=true
disable-browser=true
# Mac browsers
browser-command=open -a Safari
#browser-command=open -a Firefox
#browser-command=open -a "Google Chrome"

Text config

[text]
browser-url=http://example.com/?d=${n1}
mouseclick-action=#!${browser} 'http://www.google.com/search?client=ubuntu&channel=fs&q=${plotter}&ie=utf-8&oe=utf-8'
f5-action=#>mal4s-out.html <a href="example.com/?d=${host}">${plotter}</a>
f7-action=#>/home/ubuntu/hostlist.txt ${host}
f9-action=#!echo 'The possibilities are endless!'

NOTE: The command in f9-action, "echo", will be executed, but will not display the output, "The possibilities are endless!", to the screen as it logically should. The output is displayed in a subshell that is not attached to your terminal. This can be fixed by redirecting the output to file or another program. If you are using MacOSX, try #!echo 'The possibilities are endless!' | open -fFn The standard output of any command must be redirected to see it.

Action: Command execution

To execute a command, begin the action with #!COMMAND where COMMAND is the command you wish to execute. Using a complete path to the executable is the best (safest) practice. Mal4s will perform field substitution using the field identifiers described in displayable fields identifiers above. The #!${browser} command can be used to reference the browser set in main config file.

If browser-command is not set or disable-browser is set, the action will not execute. Commands other than #!${browser} will not execute if disable-exec is set. disable-exec does not disable the browser command. The default settings for Mal4s are disable-exec and disable-browser.

Append text to file

To append text to file, begin the action with #>pathto/file.ext. If the file does not exist, it will be created, if it does, it will be appended to. Spaces are not permitted in file names or path.

Mal4s stops reading the file name at the first space after #>, so failure to follow the no spaces in path or filename rule will result in the file name being wrong. #>C:\Program Files\test.out Text would result in a file being created (or appended to) name C:\Program with Files\test.out Text in it. Note, even if you properly quoted the path and file name, it would still not have the intended result.

Image on hover is still experimental, so the options may change. The feature and related configuration options are described in Image-on-hover

Built in defaults

Text defaults

hover-replace-unset= defaults to blank.

key-format=${n1}

key-width=90.0

key-off=false

hover-line-1=hover-line-1 not configured

[gource] defaults (main section)

wrap-hover-lines=true

truncate-hover-lines=false

wrap-max-lines=3

hover-line-length=50

path=-

browser-command= default empty

ffp=false

hide=hosts,branchnames,progress

enable-exec=false Version 1.2.2+

enable-browser=false Version 1.2.2+

disable-exec=true Version 1.2.2+

disable-browser=true Version 1.2.2+

stop-on-idle=false

screenshot-at-end=false (true also sets stop-at-end=true)

stop-at-end=false

dont-stop=false

disable-auto-rotate=false

auto_skip_seconds=1.0

host-idle-time=0.0

time-scale=1.0

loop=false

logo= empty by default

logo-offset=20.0x20.0

colour-images=false

default-plotter-image= empty by default plotter-image-dir= empty by default

camera-mode=overview

padding=1.1

bloom-multiplier=1.0

bloom-intensity=0.025

background_colour=000000

background-image=PATH_TO/dissectcyber.png

title= empty by default.

font-size=16

branch-colour=FFFFFF

font-colour=FFFFFF

highlight-colour=FFFFFF

selection-colour=FFFFFF default is vec3(1.0, 1.0, 0.3f) unknown HEX translation.

elasticity=0.0

log-format=mal4s

date-format= default is blank. Using hide=date instead changes animation.

max-hosts=0 no max by default.

max-plotter-speed=500.0

max-host-lag=5.0

plotter-idle-time=3.0

plotter-friction=1.0

plotter-scale=1.0

highlight-all-plotters=false

highlight-branches=false

caption-file= blank by default

caption-duration=10.0

caption-size=16

caption-offset=0

caption-colour=FFFFFF

hash-seed=31

[display] defaults

Gource Controls Documentation

FIXME make sure we are documenting every possible option including those mentioned in the gource docs:

https://code.google.com/p/gource/wiki/Controls