wopr - morinim/ultra GitHub Wiki
Analysing the large volume of data produced by evolutionary algorithms is a complex and critical task.
WOPR is a visualisation tool designed to simplify this process by displaying data at various levels, enabling a deeper understanding of the emergent evolutionary processes. It also allows testing on multiple datasets with real-time comparisons against reference results.
The name WOPR pays homage to the iconic "War Operation Plan Response" computer from the film WarGames. The main interface evokes the aesthetic of the fictional system, with its blinking panels and multiple screens.
ULTRA can be configured to produce several types of search logs, each offering unique insights into the evolutionary process:
- the dynamic file contains summary information recorded per generation, including changes in fitness (both of the best individual found so far and of the entire population);
- the layers file provides a snapshot of each layer in the population structure, detailing data such as ages and fitness levels;
- the population file offers a detailed representation of the clustering of individuals during evolution.
As the evolution process progresses, ULTRA continuously updates these files generation by generation. Simultaneously, WOPR polls the files and generates graphical representations of the system's current state and its trajectory over time.
WOPR employs a variety of visualisation methods to present data effectively.
- An error bar plot displaying the mean and standard deviation of the population fitness, alongside the best fitness achieved in each generation.
- An error bar plot showing the mean, maximum and standard deviation of the population age per generation.
- A combo box presenting a sequential list of the best individuals found during the search.
- A histogram showing the distribution of individuals across fitness bins.
- A shaded plot illustrating the trend in entropy.
- A heat map where each row represents a layer and each column corresponds to an individual's fitness.
- An error bar diagram illustrating the range of ages within each layer, with vertical lines marking the suprema (upper bounds) of ages in each layer.
By turning complex numerical data into clear, visually intuitive formats, WOPR provides a powerful environment for exploring and understanding the dynamics of evolutionary algorithms.
wopr monitor irisDisplays evolution information for the Iris dataset (requires at least one of the files iris.dynamic.txt, iris.layers.txt, iris.population.txt).
The user can also specify a directory:
wopr monitor iris_folder/
This works only if iris_folder/ contains exactly one dataset file (.csv).
Performs one or more evolutionary searches on one or multiple datasets, optionally comparing the results with reference data. Produces one XML output file per input dataset.
wopr run --runs 4 --generations 100 iris.csvRuns 4 evolutions of 100 generations each using the Iris dataset. The results are saved to iris.summary.xml.
~/wopr/$ ls
wopr* sonar.csv sonar.xml
~/wopr/$ ./wopr run sonar.csvHere wopr starts a test on sonar.csv implicitly using the search parameter contained in sonar.xml:
<ultra>
<search>
<generations>200</generations>
<runs>100</runs>
<threshold>84%</threshold>
</search>
<dataset>
<output_index>last</output_index>
</dataset>
</ultra>You can execute multiple tests at the same time:
~/wopr/$ ls
wopr* sonar.csv sonar.xml petalrose.csv petalrose.xml iris.csv iris.xml
~/wopr/$ ./wopr run --noguiThis performs parallel tests on sonar.csv, petalrose.csv and iris.csv. The --nogui switch launches wopr in text-only mode.
- Number of runs. Shows, for each dataset, how many runs have been executed relative to the configured limits. Each dataset is represented by a group of vertical bars on the x-axis. The bars indicate the number of runs: the current runs performed for the dataset and, optionally, the reference runs used for comparison. A horizontal line is drawn for each dataset at the level of the configured maximum number of runs, providing a clear visual indication of the allowed capacity.
- Success rate. Shows the success rate achieved on each dataset. Each dataset is represented by a group of vertical bars on the x-axis. The bars indicate: the current success rate, expressed as a percentage and, optionally, the reference success rate used for comparison.
- Fitness across datasets. Provides a compact, side-by-side comparison of fitness statistics across all datasets. Each dataset is displayed in its own subplot, arranged in a square grid for efficient use of space. The x-axis represents the number of runs executed, while the y-axis shows fitness values. For each dataset, the plot displays: the mean fitness, shown as a point with vertical error bars representing the standard deviation and the best fitness, shown as a separate marker at the same run count. If both current and reference best fitness values are finite and the current best is worse than the reference, the subplot background is highlighted in red. This provides an immediate visual cue for underperforming datasets.
wopr summary new/If you want to compare the overview with another batch of tests, you can specify a second directory:
wopr summary new/ reference/
_ ___ ___ ___
\ \ // / \ | |_) | |_)
\_\/\/ \_\_/ |_| |_| \
GREETINGS PROFESSOR FALKEN.
Please enter your selection:
> wopr monitor [path]
OBSERVE A RUNNING TEST IN REAL-TIME
The path must point to a directory containing a search log produced by
ULTRA. If omitted, the current working directory is used.
Omit the file extension when specifying a test path (e.g. use
"/path/test" instead of "/path/test.csv").
Available switches:
--dynamic <filepath>
--layers <filepath>
--population <filepath>
Monitor files with non-default names.
--refresh <seconds>
Set the refresh rate for updating plots.
--window <nr>
Restrict the monitoring window to the last `nr` generations.
-------------------------------------------------------------------
> wopr run [path]
EXECUTE TESTS ON THE SPECIFIED DATASET(s)
The argument must be a folder containing at least one .csv dataset
(and optionally a config file), or a specific dataset file. If omitted,
the current directory is used.
Available switches:
--generations <nr>
Set the maximum number of generations in a run.
--nogui
Disable the graphical user interface (headless mode).
--reference <directory>
Specify a directory containing reference results.
--runs <nr>
Perform the specified number of evolutionary runs.
--threshold <val>
Set the success threshold. Values ending in '%' are treated as
accuracy; otherwise, as fitness value.
-------------------------------------------------------------------
> wopr summary <directory> [directory]
DISPLAY OR COMPARE STATS FOR COMPLETED TESTS
Displays a high-level overview of the first directory. If a second
directory is provided, a comparison is performed.
--help
Display this help screen.
--imguidemo
Enable ImGUI demo panel.
SHALL WE PLAY A GAME?
- merge_summary.py. A small command-line utility designed to merge two ULTRA experiment summaries into a single consolidated XML report.