Handling Results with `ResultTable`

In JISA, data can be recorded in structures each known as a ResultTable. In essence, a ResultTable object represents a "table of results". They are created with a fixed number of columns, each with its own title and unit to which data are added, row by row. They can then be used to quickly output your recorded data in CSV format or be given to various GUI elements to graphically live-display your data.

The name ResultTable is an umbrella term that covers all the different types of objects that behave in this way. Current in JISA there are two types of ResultTable that you can create and use:

ResultList - Holds data in memory
ResultStream - Writes data directly to a CSV file without holding in memory

Since they both ResultTable objects, they seem to work in exactly the same way from an "outside" perspective, despite that internally the way they store the data is completely different.

Creating a ResultTable
Adding Rows
Accessing Data
Attributes
Displaying and Saving Data
Loading from File
Finding Columns
Extracting Lists
Extracting Matrices
Filtering
Sorting
Splitting

Creating a `ResultTable`

Top ↑

Regardless of which implementation of ResultTable we are using, we first must start by defining the columns of the table. This is done by creating Column<T> objects, where T is the data type of the column. JISA has the following pre-defined Column<T> types:

// Floating-point number column
Column<Double>  numerical = Column.ofDoubles("Name", "Units");
                          = Column.ofDecimals("Name", "Units"); // Alias for ofDoubles()

// Integer number column
Column<Integer> integer   = Column.ofIntegers("Name", "Units");

// Long integer number column
Column<Long>    longInt   = Column.ofLongs("Name", "Units");

// Boolean (true/false) column
Column<Boolean> bool      = Column.ofBooleans("Name", "Units");

// Text column
Column<String>  string    = Column.ofStrings("Name", "Units");
                          = Column.ofText("Name", "Units");    // Alias for ofStrings()

For instance, if we wanted three numerical columns to hold "Voltage", "Current" and "Frequency" values respectively we could write:

Java

Column<Double> voltage   = Column.ofDecimals("Voltage", "V");
Column<Double> current   = Column.ofDecimals("Current", "A");
Column<Double> frequency = Column.ofDecimals("Frequency", "Hz");

Kotlin

val voltage   = Column.ofDecimals("Voltage", "V")
val current   = Column.ofDecimals("Current", "A")
val frequency = Column.ofDecimals("Frequency", "Hz")

Python

voltage   = Column.ofDecimals("Voltage", "V")
current   = Column.ofDecimals("Current", "A")
frequency = Column.ofDecimals("Frequency", "Hz")

We can then create a ResultList by passing it these columns as arguments:

Java

ResultTable rTable = new ResultList(voltage, current, frequeny);

Kotlin

val rTable = ResultList(voltage, current, frequency)

Python

rTable = ResultList(voltage, current, frequency)

In effect, we have just created the following blank table (in memory), ready for adding data to:

Voltage [V]	Current [A]	Frequency [Hz]
...	...	...

However, if we are expecting to be dealing with a large amount of data (ie so much that we might run out of memory), or if we want every data-point to be committed to permanent storage as soon as it is taken, then we will want to create a ResultStream instead. This is done almost the same way except that you need to specify the name of the file you want to write to:

Java

ResultTable rStream = new ResultStream("path/to/file.csv", voltage, current, frequency);

Kotlin

val rStream = ResultStream("path/to/file.csv", voltage, current, frequency)

Python

rStream = ResultStream("path/to/file.csv", voltage, current, frequency)

To support legacy code which uses the old version of ResultTable, you can create tables of entirely numerical columns by just specifying the column names:

ResultTable table = new RestultList("Voltage", "Current", "Frequency");

Adding Rows

Top ↑

After creating a ResultTable, you can add data to it, row by row, in one of three ways. Let's take the following example:

val time      = Column.ofLongs("Time", "ms")
val voltage   = Column.ofDecimals("Voltage", "V")
val current   = Column.ofDecimals("Current", "A")
val frequency = Column.ofDecimals("Frequency", "Hz")

val rList = ResultList(time, voltage, current, frequency)

We can add data to this by:

Specifying values in column order, using addData(...)
Assigning values to columns by use of a lambda with addRow(...)
Specifying a column-to-value mapping by use of mapRow(...)

The specifics of each are detailed in the sections below, using the example table defined above.

Simple Column Order (`addData`)

The first, and most simplistic, way of adding data is to call addData(...) and specify each value in column order:

row.addData(100, 1.0, 50e-3, 5.5);

This will add a new row like so:

Time [ms]	Voltage [V]	Current [A]	Frequency [Hz]
100	1.0	0.05	5.5

This method works the same in Java, Kotlin, and Python, and thus is equally suitable for use in all three.

Lambda Expression (`addRow`)

However, it can often improve the readability of your code to define new rows by specifying column-value pairings. This can be done with the addRow(...) method, which works particularly well in both Java and Kotlin, but not so in Python due to lambda functions in Python being limited to a single line. This method expects to be given a lambda that takes the new row as an argument, and then sets the value of each column in the row one by one, like so:

Java

rList.addRow(row -> {
    row.set(time, 100);
    row.set(frequency, 5.5);
    row.set(current, 50e-3);
    row.set(voltage, 1.0);
});

Kotlin

rList.addRow {
    it[time]      = 100
    it[frequency] = 5.5
    it[current]   = 50e-3
    it[voltage]   = 1.0
}

The order in which you assign values to columns in unimportant when using this method. Any columns you don't set a value for will be automatically set to null (None in Python-speak).

Map/Dictionary (`mapRow`)

One can instead supply a map of columns to values, by using mapRow(...). This method works particularly well in Kotlin and Python. It also works in Java, but doesn't offer anything over the addRow(..) method while being slightly more cumbersome. In Kotlin, you can define maps by using the to infix function to associate each key (i.e., column) to its value. In Python, you define a dictionary using the curly brackets {...} and the key: value syntax. Taking the same example as before:

Kotlin

rList.mapRow(
    time      to 100,
    frequency to 5.5,
    current   to 50e-3,
    voltage   to 1.0
)

Python

rList.mapRow({
    time:      100,
    frequency: 5.5,
    current:   50e-3,
    voltage:   1.0
})

The order in which you assign values to columns in unimportant when using this method. Any columns you don't set a value for will be automatically set to null (None in Python-speak).

Example

As an example let's say we're sweeping voltage on an SMU:

Java

Column<Double> voltage = Column.ofDecimals("Voltage", "V");
Column<Double> current = Column.ofDecimals("Current", "A");
ResultTable    rList   = new ResultList(voltage, current);

...

for (double v : voltages) {

    smu.setVoltage(v);

    rList.addRow(row -> {
        row.set(voltage, smu.getVoltage());
        row.set(current, smu.getCurrent());
    });

}

Kotlin

val voltage = Column.ofDecimals("Voltage", "V")
val current = Column.ofDecimals("Current", "A")
val rList   = ResultList(voltage, current)

...

for (v in voltages) {

    smu.voltage = v

    rList.mapRow(
        voltage to smu.voltage,
        current to smu.current
    )

}

Python

voltage = Column.ofDecimals("Voltage", "V")
current = Column.ofDecimals("Current", "A")
rList   = ResultList(voltage, current)

...

for v in voltages:

    smu.voltage = v
    
    rList.mapRow({
        voltage: smu.getVoltage(),
        current: smu.getCurrent()
    })

In the above example, for each voltage value that we set the SMU to source, we record a voltage and current measurement in rList. Thus, afterwards our rList object will contain something like:

Voltage [V]	Current [A]
0.0	0.0
10.0	0.03
20.0	0.06
30.0	0.09
40.0	0.12

Accessing Data

Top ↑

All ResultTable objects are iterable, which means you can loop over them. On each iteration, they will yield a Row object that represents a single row of data. You can get the value of each column from a Row by use of get(...) (or [...] in kotlin) and passing it the Column<> object of the column you want the value of:

Java

final Column<Double> V = Column.ofDecimals("Voltage", "V");
final Column<Double> I = Column.ofDecimals("Current", "A");
final Column<Double> T = Column.ofDecimals("Temperature", "K");

ResultTable results = new ResultList(V, I, T);

...

for (Row row : results) {

    double voltage     = row.get(V);
    double current     = row.get(I);
    double temperature = row.get(T);

}

Kotlin

val V = Column.ofDecimals("Voltage", "V")
val I = Column.ofDecimals("Current", "A")
val T = Column.ofDecimals("Temperature", "K")

val results = ResultList(V, I, T)

...

for (row in results) {

    val voltage     = row[V]
    val current     = row[I]
    val temperature = row[T]

}

Python

V = Column.ofDecimals("Voltage", "V")
I = Column.ofDecimals("Current", "A")
T = Column.ofDecimals("Temperature", "K")

results = ResultList(V, I, T)

...

for row in results:

    voltage     = row.get(V)
    current     = row.get(I)
    temperature = row.get(T)

Individual rows can be access in random order by use of get(..) (or [...] in Kotlin) on the ResultTable itself:

Java

final Column<Double> V = Column.ofDecimals("Voltage", "V");
final Column<Double> I = Column.ofDecimals("Current", "A");
final Column<Double> T = Column.ofDecimals("Temperature", "K");

ResultTable results = new ResultList(V, I, T);

...

// Get individual rows
Row row5 = results.get(5);
Row row0 = results.get(0);
Row row3 = results.get(3);

// Extract column values from each row
double v0 = row0.get(V);
double t5 = row5.get(T);
double i3 = row3.get(I);

// All in one go
double v0 = results.get(0, V);
double t5 = results.get(5, T);
double i3 = results.get(3, I);

Kotlin

val V = Column.ofDecimals("Voltage", "V")
val I = Column.ofDecimals("Current", "A")
val T = Column.ofDecimals("Temperature", "K")

val results = ResultList(V, I, T)

...

// Get individual rows
val row5 = results[5]
val row0 = results[0]
val row3 = results[3]

// Extract column values from each row
val v0 = row0[V]
val t5 = row5[T]
val i3 = row3[I]

// All in one go
val v0 = results[0, V]
val t5 = results[5, T]
val i3 = results[3, I]

Python

V = Column.ofDecimals("Voltage", "V")
I = Column.ofDecimals("Current", "A")
T = Column.ofDecimals("Temperature", "K")

results = ResultList(V, I, T)

...

# Get individual rows
row5 = results.get(5)
row0 = results.get(0)
row3 = results.get(3)

# Extract column values from each row
v0 = row0.get(V)
t5 = row5.get(T)
i3 = row3.get(I)

# All in one go
v0 = results.get(0, V)
t5 = results.get(5, T)
i3 = results.get(3, I)

Attributes

Any ResultTable can have values stored along-side it to provide supplementary information. For instance, it's quite common that one would want to save experimental parameters along with the result data from a measurement. To facilitate this, there are two methods: setAttribute(key, value) and getAttribute(key).

Each attribute must have its own unique (string) key to identify it. To add an attribute to a ResultTable use setAttribute(). For instance, if we want to save the dimensions of a device we're measuring the conductivity of, we could do:

resultTable.setAttribute("Length", "400 um");   // L = 400 um
resultTable.setAttribute("Width", "200 um");    // W = 200 um
resultTable.setAttribute("Thickness", "40 nm"); // D = 40 nm

Then, when dealing with resultList later, these values can be recalled by use of getAttribute(...):

Java

String length    = resultTable.getAttribute("Length");
String width     = resultTable.getAttribute("Width");
String thickness = resultTable.getAttribute("Thickness");

Kotlin

val length    = resultTable.getAttribute("Length")
val width     = resultTable.getAttribute("Width")
val thickness = resultTable.getAttribute("Thickness")

Python

length    = resultTable.getAttribute("Length")
width     = resultTable.getAttribute("Width")
thickness = resultTable.getAttribute("Thickness")

As we will see in the next section, these attributes will be saved with the data automatically when output as a CSV file.

Displaying and Saving Data

You can output the data in a ResultTable to both the terminal, visually (by use of a Table GUI element) or to a file.

For the purposes of example in this section, let's say we have a table called resultTable with three columns: two Double columns holding voltage and current values respectively and a String column holding notes about the measurement.

ASCII Tables

To display the ResultTable in the terminal as an ASCII table, use the outputTable() method like so:

resultTable.outputTable();

it will result in something looking like this being printed to the terminal (standard out stream):

========================================
| Voltage [V] | Current [A] | Notes    |
========================================
| 1.0         | 0.001       | None     |
+-------------+-------------+----------+
| 2.0         | 0.002       | None     |
+-------------+-------------+----------+
| 3.0         | 0.003       | None     |
+-------------+-------------+----------+
| 4.0         | 0.004       | None     |
+-------------+-------------+----------+
| 5.0         | 0.005       | None     |
+-------------+-------------+----------+
| 6.0         | 0.006       | None     |
+-------------+-------------+----------+

To output it to a file instead, just specify the path as an argument:

// Linux/Mac
resultTable.outputTable("/path/to/file.txt");

// Windows
resultTable.outputTable("C:\\path\\to\\file.txt");

`Table` GUI Element

You can create a Table GUI element to show the contents of a ResultTable in real time. This is explained further on the Tables page. In short:

Java

Table table = new Table("Results", resultTable);
table.show();

Kotlin

val table = Table("Results", resultTable)
table.show()

Python

table = Table("Results", resultTable)
table.show()

This will result in a window opening like so:

As is also explained in the GUI pages, this element can also be incorporated into larger GUI structures, such as Grid elements instead of just being opened as its own stand-alone window.

CSV Format

To output the data in CSV format we use the outputCSV() method like so:

resultTable.outputCSV();

this will result in the following being printed to the terminal:

% ATTRIBUTES: {}
"Voltage [V]" {Double}, "Current [A]" {Double}, "Notes" {String}
1.0, 0.001, "None"
2.0, 0.002, "None"
3.0, 0.003, "None",
4.0, 0.004, "None",
5.0, 0.005, "None",
6.0, 0.006, "None"

To output this to a file instead, just specify the path as an argument:

// Linux/Mac
resultTable.outputCSV("/path/to/file.csv");

// Windows
resultTable.outputCSV("C:\\path\\to\\file.csv");

You may have noticed the "% Attributes: {}" line. This is where any attributes set on the ResultTable are written when output as CSV. For instance, if before calling output() we have done the following:

resultTable.setAttribute("Length", "400 um");   // L = 400 um
resultTable.setAttribute("Width", "200 um");    // W = 200 um
resultTable.setAttribute("Thickness", "40 nm"); // D = 40 nm

then the CSV output would instead look like this:

% ATTRIBUTES: {"Length": "400 um", "Width": "200 um", "Thickness": "40 nm"}
"Voltage [V]" {Double}, "Current [A]" {Double}, "Notes" {String}
1.0, 0.001, "None"
2.0, 0.002, "None"
3.0, 0.003, "None",
4.0, 0.004, "None",
5.0, 0.005, "None",
6.0, 0.006, "None"

Compressed Binary Format

As well as outputting a ResultTable in a CSV format, JISA lets you write it to a compressed binary format by use of the outputBinary(...) method.

// Unix/Linux/Mac
resultTable.outputBinary("path/to/file.jdf");

// Windows
resultTable.outputBinary("C:\\path\\to\\file.jdf");

This will output the data (including attributes) to a file in an exact binary representation (using a format called JDF or "JISA Data Format") that JISA can read back in later. For large datasets, this is a significantly more space-efficient way of storing data.

The downside to this is that the data is no-longer human readable in the file, and the compression/decompression required when saving/loading can slow things down.

Loading from File

You can load data (and attributes) previously written to a CSV or binary file back into a a ResultList or ResultStream object by using ResultList.loadFile(...). Alternatively, a CSV file can be opened as a ResultStream by using ResultStream.loadFile(...).

Loading as a ResultList will load the data back into memory whereas loading as a ResultStream will simply create a ResultStream object that uses the specified file as its backing file -- meaning any changes made to the ResultStream will be written directly to the file.

Java

ResultTable list1  = ResultList.loadFile("/path/to/file.csv");
ResultTable list2  = ResultList.loadFile("/path/to/file.jdf");
ResultTable stream = ResultStream.loadFile("/path/to/file.csv");

Kotlin

val list1  = ResultList.loadFile("/path/to/file.csv")
val list2  = ResultList.loadFile("/path/to/file.jdf")
val stream = ResultStream.loadFile("/path/to/file.csv")

Python

lst1 = ResultList.loadFile("/path/to/file.csv")
lst2 = ResultList.loadFile("/path/to/file.jdf")
strm = ResultStream.loadFile("/path/to/file.csv")

As mentioned, this will also load in any attributes that were saved with the files, allowing you to retrieve them with getAttribute(...) calls.

Finding Columns

Often you may find yourself in a situation where you have a ResultTable object but not the Column<> objects that were used to create it. For instance, you may have loaded in some data from a CSV file using ResultList.loadFile(...). For these scenarios, ResultTable object provide the find...Column(...) method which lets you retrieve the desired Column<Type> object by specifying its name.

For instance, if you know the name and type of the column you can use it like so:

Java

Column<Double>  D = resultTable.findDoubleColumn("Name");
Column<Integer> I = resultTable.findIntegerColumn("Name");
Column<Long>    L = resultTable.findLongColumn("Name");
Column<Boolean> B = resultTable.findBooleanColumn("Name");
Column<String>  S = resultTable.findStringColumn("Name");

Kotlin

val D = resultTable.findDoubleColumn("Name")
val I = resultTable.findIntegerColumn("Name")
val L = resultTable.findLongColumn("Name")
val B = resultTable.findBooleanColumn("Name")
val S = resultTable.findStringColumn("Name")

Python

D = resultTable.findDoubleColumn("Name")
I = resultTable.findIntegerColumn("Name")
L = resultTable.findLongColumn("Name")
B = resultTable.findBooleanColumn("Name")
S = resultTable.findStringColumn("Name")

Alternatively, if you have a Column<Type> object to hand with the same name, units and type you can simply use that - ResultTable objects and their Row objects are smart enough to match them.

However, to save it from having to search each time, you can use findColumn(...) to find the matching column in the ResultTable like so:

Column<Type> other  = ...;
Column<Type> column = resultTable.findColumn(otherColumn);

Extracting Lists

You can extract any column, or any value determined by an operation performed on each row, as a List<> object by use of toList(...). For instance, let's say we have a ResultTable and we want to extract one of its columns as a list of doubles:

Java

List<Double> values = resultTable.toList(column);

Kotlin

val values = resultTable.toList(column)

If, for example, we instead wanted to return a list that is the product of two columns we could supply a (Row) → Double lambda expression instead like so:

Java

List<Double> values = resultTable.toList(row -> row.get(column1) * row.get(column2));

Kotlin

val values = resultTable.toList { it[column1] * it[column2] }

In fact, you can think about our first example of retrieving the values of a single column as just being shorthand for:

Java

List<Double> values = resultList.toList(row -> row.get(column));

Kotlin

val values = resultTable.toList { it[column] }

Extracting Matrices

You can extract any combination of numerical columns as a RealMatrix object by using toMatrix(...) and specifying what you want to be in each column of the matrix. For instance, if we had the following ResultTable:

Column<String> timestamp = Column.ofText("Timestamp");
Column<Double> voltage   = Column.ofDecimals("Voltage", "V");
Column<Double> current   = Column.ofDecimals("Current", "A");
ResultTable    table     = new ResultList(timestamp, voltage, current);

for (...) {

  table.addRow(row -> {
       row.set(timestamp, Util.getCurrentTimeString());
       row.set(voltage, smu.getVoltage());
       row.set(current, smu.getCurrent());
  });

}

Then we can extract both the voltage and current columns (since they're both numbers) as a RealMatrix like so:

Java

RealMatrix matrix = table.toMatrix(voltage, current);

Kotlin

val matrix = table.toMatrix(voltage, current)

The result is a matrix with two columns, the first being the values of voltage and the second being those of current. Just as before, we can instead supply expressions for each matrix column. For instance, we could achieve the same result by writing:

Java

RealMatrix matrix = table.toMatrix(r -> r.get(voltage), r -> r.get(current));

Kotlin

val matrix = table.toMatrix({ r -> r[voltage] }, { r -> r[current] })

// == or using "it" ==

val matrix = table.toMatrix({ it[voltage] }, { it[current] })

This lets us retrieve values that are derived from those in the table. For instance, if we wanted a single column matrix of power (i.e. voltage * current):

Java

RealMatrix matrix = table.toMatrix(r -> r.get(voltage) * r.get(current));

Kotlin

val matrix = table.toMatrix { it[voltage] * it[current] }

Filtering

You can create a new coy of a ResultTable that only contains rows matching a given criterion by using the filter(...) method. This takes a lambda expression (specifically a Predicate<Row> object) that should return true or false to determine whether a given row should be retained or not.

For instance, if we have a table with three columns: "Time [s]", "Voltage [V]" and "Current [A]", we could filter these data to give us only the rows where time > 50 seconds like so:

Java

ResultTable after50 = table.filter(r -> r.get(time) > 50)

Kotlin

val after50 = table.filter { r -> r[time] > 50 }

// == or, by using "it" ==

val after50 = table.filter { it[time] > 50 }

Sorting

You can get a ResultTable to return a sorted copy of itself by using the sorted(...) method. This method requires you to either specify an expression as a lambda function to return a sortable valuable based on values in each row, or a Column<> of some sortable type:

Java

Column<Double> number = Column.ofDecimals(...);
Column<String> string = Column.ofText(...);

// By specifying column
ResultTable sortedByNumber = table.sorted(number);
ResultTable sortedByString = table.sorted(string);

// By specifying exression
ResultTable sortedByAbs = table.sorted(r -> Math.abs(r.get(number)));

Kotlin

val number = Column.ofDecimals(...)
val string = Column.ofText(...)

// By specifying column
val sortedByNumber = table.sorted(number)
val sortedByString = table.sorted(string)

// By specifying exression
val sortedByAbs = table.sorted { abs(it[number]) }

For example, if we had some temperature-dependent data but it was not in any particular order:

Column<Double> temperature = Column.ofDecimals("Temperature", "K");
Column<Double> resistance  = Column.ofDecimals("Resistance", "Ohm");
ResultTable    table       = new ResultList(temperature, resistance);

// Data gets added to table here but at random temperatures

then we can get a copy that is in ascending order of temperature like so:

ResultTable sortedByTemperature = table.sorted(temperature);

Splitting

Often times, when analysing data, you will find it useful to be able to split a table of data into multiple tables based on the value in a given column (or an expression). ResultTable objects let you do this by use of split(...). Just as with other examples, you either supply it with a Column<> or a lambda expression. It will then return a map of each unique value in that column/expression to filtered copies of the ResultTable containing only rows with that value.

Map<Type, ResultTable> split = table.split(Column<Type> column);
Map<Type, ResultTable> split = table.split(((Row) -> Type) expression);

For instance, if we had output curve data (SD Voltage, SG Voltage and SD Current) in the columns drain, gate and current respectively, then it's likely we will want to split the data by gate like so:

Java

Map<Double, ResultTable> split = table.split(gate);

split.forEach((gate, data) -> {
  // gate is the gate voltage and data all the rows with that gate voltage
});

Kotlin

val split = table.split(gate)

for ((gate, data) in split) {
  // gate is the gate voltage and data all the rows with that gate voltage
}

You can also split a ResultTable in smaller ones based on the direction of change of a given numerical column/expression by use of directionalSplit(...). For instance, if drain is swept up then down, this will split the data into two sets, one for when drain is increasing and one of when drain is decreasing:

List<ResultTable> split = table.directionalSplit(drain);

if drain sweeps up and down multiple times, the list will contain a ResultTable for each up-sweep and each down-sweep in alternating order.

Result Handling - OE-FET/JISA GitHub Wiki

Handling Results with `ResultTable`

Contents

Creating a `ResultTable`

Adding Rows

Simple Column Order (`addData`)

Lambda Expression (`addRow`)

Map/Dictionary (`mapRow`)

Example

Accessing Data

Attributes

Displaying and Saving Data

ASCII Tables

`Table` GUI Element

CSV Format

Compressed Binary Format

Loading from File

Finding Columns

Extracting Lists

Extracting Matrices

Filtering

Sorting

Splitting

⚠️ GitHub.com Fallback ⚠️

Result Handling - OE-FET/JISA GitHub Wiki

Handling Results with ResultTable

Contents

Creating a ResultTable

Adding Rows

Simple Column Order (addData)

Lambda Expression (addRow)

Map/Dictionary (mapRow)

Example

Accessing Data

Attributes

Displaying and Saving Data

ASCII Tables

Table GUI Element

CSV Format

Compressed Binary Format

Loading from File

Finding Columns

Extracting Lists

Extracting Matrices

Filtering

Sorting

Splitting

⚠️ **GitHub.com Fallback** ⚠️

Handling Results with `ResultTable`

Creating a `ResultTable`

Simple Column Order (`addData`)

Lambda Expression (`addRow`)

Map/Dictionary (`mapRow`)

`Table` GUI Element

⚠️ GitHub.com Fallback ⚠️