Result Handling - OE-FET/JISA GitHub Wiki
In JISA, data can be recorded in structures each known as a ResultTable. In
essence, a ResultTable object represents a "table of results". They are
created with a fixed number of columns, each with its own title and unit to
which data are added, row by row. They can then be used to quickly output your
recorded data in CSV format, compressed binary files, or be given to various GUI
elements to graphically live-display your data.
The name ResultTable is an umbrella term that covers all the different types
of objects that behave in this way. Current in JISA there are two types of
ResultTable that you can create and use:
-
ResultList- Holds data in memory -
ResultStream- Writes data directly to a CSV file without holding in memory
Since they both ResultTable objects, they seem to work in exactly the same way
from an "outside" perspective, despite that internally the way they store the
data is completely different.
- Creating a
ResultTable - Adding Rows
- Accessing Data
- Attributes
- Displaying and Saving Data
- Loading from File
- Finding Columns
- Grouping Tables into a Single File (
ResultGroup) - Extracting Lists
- Extracting Matrices
- Filtering
- Sorting
- Splitting
Regardless of which implementation of ResultTable we are using, we first must
start by defining the columns of the table. This is done by creating Column<T>
objects, where T is the data type of the column. JISA has the following
pre-defined Column<T> types:
// Floating-point number column
Column<Double> numerical = Column.ofDoubles("Name", "Units");
= Column.ofDecimals("Name", "Units"); // Alias for ofDoubles()
// Integer number column
Column<Integer> integer = Column.ofIntegers("Name", "Units");
// Long integer number column
Column<Long> longInt = Column.ofLongs("Name", "Units");
// Boolean (true/false) column
Column<Boolean> bool = Column.ofBooleans("Name", "Units");
// Text column
Column<String> string = Column.ofStrings("Name", "Units");
= Column.ofText("Name", "Units"); // Alias for ofStrings()
For instance, if we wanted three numerical columns to hold "Voltage", "Current" and "Frequency" values respectively we could write:
Java
Column<Double> voltage = Column.ofDecimals("Voltage", "V");
Column<Double> current = Column.ofDecimals("Current", "A");
Column<Double> frequency = Column.ofDecimals("Frequency", "Hz");Kotlin
val voltage = Column.ofDecimals("Voltage", "V")
val current = Column.ofDecimals("Current", "A")
val frequency = Column.ofDecimals("Frequency", "Hz")Python
voltage = Column.ofDecimals("Voltage", "V")
current = Column.ofDecimals("Current", "A")
frequency = Column.ofDecimals("Frequency", "Hz")We can then create a ResultList by passing it these columns as arguments:
Java
ResultTable rTable = new ResultList(voltage, current, frequeny);Kotlin
val rTable = ResultList(voltage, current, frequency)Python
rTable = ResultList(voltage, current, frequency)In effect, we have just created the following blank table (in memory), ready for adding data to:
| Voltage [V] | Current [A] | Frequency [Hz] |
|---|---|---|
| ... | ... | ... |
However, if we are expecting to be dealing with a large amount of data (ie so
much that we might run out of memory), or if we want every data-point to be
committed to permanent storage as soon as it is taken, then we will want to
create a ResultStream instead. This is done almost the same way except that
you need to specify the name of the file you want to write to:
Java
ResultTable rStream = new ResultStream("path/to/file.csv", voltage, current, frequency);Kotlin
val rStream = ResultStream("path/to/file.csv", voltage, current, frequency)Python
rStream = ResultStream("path/to/file.csv", voltage, current, frequency)To support legacy code which uses the old version of ResultTable, you can
create tables of entirely numerical columns by just specifying the column names:
ResultTable table = new RestultList("Voltage", "Current", "Frequency");After creating a ResultTable, you can add data to it, row by row, in one of
three ways. Let's take the following example:
val time = Column.ofLongs("Time", "ms")
val voltage = Column.ofDecimals("Voltage", "V")
val current = Column.ofDecimals("Current", "A")
val frequency = Column.ofDecimals("Frequency", "Hz")
val rList = ResultList(time, voltage, current, frequency)We can add data to this by:
- Specifying values in column order, using
addData(...) - Assigning values to columns by use of a lambda with
addRow(...) - Specifying a column-to-value mapping by use of
mapRow(...)
The specifics of each are detailed in the sections below, using the example table defined above.
The first, and most simplistic, way of adding data is to call addData(...) and
specify each value in column order:
row.addData(100, 1.0, 50e-3, 5.5);This will add a new row like so:
| Time [ms] | Voltage [V] | Current [A] | Frequency [Hz] |
|---|---|---|---|
| 100 | 1.0 | 0.05 | 5.5 |
This method works the same in Java, Kotlin, and Python, and thus is equally suitable for use in all three.
However, it can often improve the readability of your code to define new rows by
specifying column-value pairings. This can be done with the addRow(...)
method, which works particularly well in both Java and Kotlin, but not so in
Python due to lambda functions in Python being limited to a single line. This
method expects to be given a lambda that takes the new row as an argument, and
then sets the value of each column in the row one by one, like so:
Java
rList.addRow(row -> {
row.set(time, 100);
row.set(frequency, 5.5);
row.set(current, 50e-3);
row.set(voltage, 1.0);
});Kotlin
rList.addRow {
it[time] = 100
it[frequency] = 5.5
it[current] = 50e-3
it[voltage] = 1.0
}The order in which you assign values to columns in unimportant when using this
method. Any columns you don't set a value for will be automatically set to
null (None in Python-speak).
One can instead supply a map of columns to values, by using mapRow(...). This
method works particularly well in Kotlin and Python. It also works in Java, but
doesn't offer anything over the addRow(..) method while being slightly more
cumbersome. In Kotlin, you can define maps by using the to infix function to
associate each key (i.e., column) to its value. In Python, you define a
dictionary using the curly brackets {...} and the key: value syntax. Taking
the same example as before:
Kotlin
rList.mapRow(
time to 100,
frequency to 5.5,
current to 50e-3,
voltage to 1.0
)Python
rList.mapRow({
time: 100,
frequency: 5.5,
current: 50e-3,
voltage: 1.0
})The order in which you assign values to columns in unimportant when using this
method. Any columns you don't set a value for will be automatically set to
null (None in Python-speak).
As an example let's say we're sweeping voltage on an SMU:
Java
Column<Double> voltage = Column.ofDecimals("Voltage", "V");
Column<Double> current = Column.ofDecimals("Current", "A");
ResultTable rList = new ResultList(voltage, current);
...
for (double v : voltages) {
smu.setVoltage(v);
rList.addRow(row -> {
row.set(voltage, smu.getVoltage());
row.set(current, smu.getCurrent());
});
}Kotlin
val voltage = Column.ofDecimals("Voltage", "V")
val current = Column.ofDecimals("Current", "A")
val rList = ResultList(voltage, current)
...
for (v in voltages) {
smu.voltage = v
rList.mapRow(
voltage to smu.voltage,
current to smu.current
)
}Python
voltage = Column.ofDecimals("Voltage", "V")
current = Column.ofDecimals("Current", "A")
rList = ResultList(voltage, current)
...
for v in voltages:
smu.voltage = v
rList.mapRow({
voltage: smu.getVoltage(),
current: smu.getCurrent()
})In the above example, for each voltage value that we set the SMU to source, we
record a voltage and current measurement in rList. Thus, afterwards our
rList object will contain something like:
| Voltage [V] | Current [A] |
|---|---|
| 0.0 | 0.0 |
| 10.0 | 0.03 |
| 20.0 | 0.06 |
| 30.0 | 0.09 |
| 40.0 | 0.12 |
All ResultTable objects are iterable, which means you can loop over them. On
each iteration, they will yield a Row object that represents a single row of
data. You can get the value of each column from a Row by use of get(...) (or
[...] in kotlin) and passing it the Column<> object of the column you want
the value of:
Java
final Column<Double> V = Column.ofDecimals("Voltage", "V");
final Column<Double> I = Column.ofDecimals("Current", "A");
final Column<Double> T = Column.ofDecimals("Temperature", "K");
ResultTable results = new ResultList(V, I, T);
...
for (Row row : results) {
double voltage = row.get(V);
double current = row.get(I);
double temperature = row.get(T);
}Kotlin
val V = Column.ofDecimals("Voltage", "V")
val I = Column.ofDecimals("Current", "A")
val T = Column.ofDecimals("Temperature", "K")
val results = ResultList(V, I, T)
...
for (row in results) {
val voltage = row[V]
val current = row[I]
val temperature = row[T]
}Python
V = Column.ofDecimals("Voltage", "V")
I = Column.ofDecimals("Current", "A")
T = Column.ofDecimals("Temperature", "K")
results = ResultList(V, I, T)
...
for row in results:
voltage = row.get(V)
current = row.get(I)
temperature = row.get(T)Individual rows can be access in random order by use of get(..) (or [...] in
Kotlin) on the ResultTable itself:
Java
final Column<Double> V = Column.ofDecimals("Voltage", "V");
final Column<Double> I = Column.ofDecimals("Current", "A");
final Column<Double> T = Column.ofDecimals("Temperature", "K");
ResultTable results = new ResultList(V, I, T);
...
// Get individual rows
Row row5 = results.get(5);
Row row0 = results.get(0);
Row row3 = results.get(3);
// Extract column values from each row
double v0 = row0.get(V);
double t5 = row5.get(T);
double i3 = row3.get(I);
// All in one go
double v0 = results.get(0, V);
double t5 = results.get(5, T);
double i3 = results.get(3, I);Kotlin
val V = Column.ofDecimals("Voltage", "V")
val I = Column.ofDecimals("Current", "A")
val T = Column.ofDecimals("Temperature", "K")
val results = ResultList(V, I, T)
...
// Get individual rows
val row5 = results[5]
val row0 = results[0]
val row3 = results[3]
// Extract column values from each row
val v0 = row0[V]
val t5 = row5[T]
val i3 = row3[I]
// All in one go
val v0 = results[0, V]
val t5 = results[5, T]
val i3 = results[3, I]Python
V = Column.ofDecimals("Voltage", "V")
I = Column.ofDecimals("Current", "A")
T = Column.ofDecimals("Temperature", "K")
results = ResultList(V, I, T)
...
# Get individual rows
row5 = results.get(5)
row0 = results.get(0)
row3 = results.get(3)
# Extract column values from each row
v0 = row0.get(V)
t5 = row5.get(T)
i3 = row3.get(I)
# All in one go
v0 = results.get(0, V)
t5 = results.get(5, T)
i3 = results.get(3, I)Any ResultTable can have values stored along-side it to provide supplementary
information. For instance, it's quite common that one would want to save
experimental parameters along with the result data from a measurement. To
facilitate this, there are two methods: setAttribute(key, value) and
getAttribute(key).
Each attribute must have its own unique (string) key to identify it. To add an
attribute to a ResultTable use setAttribute(). For instance, if we want to
save the dimensions of a device we're measuring the conductivity of, we could
do:
resultTable.setAttribute("Length", "400 um"); // L = 400 um
resultTable.setAttribute("Width", "200 um"); // W = 200 um
resultTable.setAttribute("Thickness", "40 nm"); // D = 40 nmThen, when dealing with resultList later, these values can be recalled by use
of getAttribute(...):
Java
String length = resultTable.getAttribute("Length");
String width = resultTable.getAttribute("Width");
String thickness = resultTable.getAttribute("Thickness");Kotlin
val length = resultTable.getAttribute("Length")
val width = resultTable.getAttribute("Width")
val thickness = resultTable.getAttribute("Thickness")Python
length = resultTable.getAttribute("Length")
width = resultTable.getAttribute("Width")
thickness = resultTable.getAttribute("Thickness")As we will see in the next section, these attributes will be saved with the data automatically when output as a CSV file.
You can output the data in a ResultTable to both the terminal, visually (by
use of a Table GUI element) or to a file.
For the purposes of example in this section, let's say we have a table called
resultTable with three columns: two Double columns holding voltage and
current values respectively and a String column holding notes about the
measurement.
To display the ResultTable in the terminal as an ASCII table, use the
outputTable() method like so:
resultTable.outputTable();it will result in something looking like this being printed to the terminal (standard out stream):
========================================
| Voltage [V] | Current [A] | Notes |
========================================
| 1.0 | 0.001 | None |
+-------------+-------------+----------+
| 2.0 | 0.002 | None |
+-------------+-------------+----------+
| 3.0 | 0.003 | None |
+-------------+-------------+----------+
| 4.0 | 0.004 | None |
+-------------+-------------+----------+
| 5.0 | 0.005 | None |
+-------------+-------------+----------+
| 6.0 | 0.006 | None |
+-------------+-------------+----------+
To output it to a file instead, just specify the path as an argument:
// Linux/Mac
resultTable.outputTable("/path/to/file.txt");
// Windows
resultTable.outputTable("C:\\path\\to\\file.txt");You can create a Table GUI element to show the contents of a ResultTable in
real time. This is explained further on the Tables page. In short:
Java
Table table = new Table("Results", resultTable);
table.show();Kotlin
val table = Table("Results", resultTable)
table.show()Python
table = Table("Results", resultTable)
table.show()This will result in a window opening like so:
As is also explained in the GUI pages, this element can also be incorporated
into larger GUI structures, such as Grid elements instead of just being opened
as its own stand-alone window.
To output the data in CSV format we use the outputCSV() method like so:
resultTable.outputCSV();this will result in the following being printed to the terminal:
% ATTRIBUTES: {}
"Voltage [V]" {Double}, "Current [A]" {Double}, "Notes" {String}
1.0, 0.001, "None"
2.0, 0.002, "None"
3.0, 0.003, "None",
4.0, 0.004, "None",
5.0, 0.005, "None",
6.0, 0.006, "None"
To output this to a file instead, just specify the path as an argument:
// Linux/Mac
resultTable.outputCSV("/path/to/file.csv");
// Windows
resultTable.outputCSV("C:\\path\\to\\file.csv");You may have noticed the "% Attributes: {}" line. This is where any attributes
set on the ResultTable are written when output as CSV. For instance, if before
calling output() we have done the following:
resultTable.setAttribute("Length", "400 um"); // L = 400 um
resultTable.setAttribute("Width", "200 um"); // W = 200 um
resultTable.setAttribute("Thickness", "40 nm"); // D = 40 nmthen the CSV output would instead look like this:
% ATTRIBUTES: {"Length": "400 um", "Width": "200 um", "Thickness": "40 nm"}
"Voltage [V]" {Double}, "Current [A]" {Double}, "Notes" {String}
1.0, 0.001, "None"
2.0, 0.002, "None"
3.0, 0.003, "None",
4.0, 0.004, "None",
5.0, 0.005, "None",
6.0, 0.006, "None"
As well as outputting a ResultTable in a CSV format, JISA lets you write it to
a compressed binary format by use of the outputBinary(...) method.
// Unix/Linux/Mac
resultTable.outputBinary("path/to/file.jdf");
// Windows
resultTable.outputBinary("C:\\path\\to\\file.jdf");This will output the data (including attributes) to a file in an exact binary representation (using a format called JDF or "JISA Data Format") that JISA can read back in later. For large datasets, this is a significantly more space-efficient way of storing data.
The downside to this is that the data is no-longer human readable in the file, and the compression/decompression required when saving/loading can slow things down.
You can load data (and attributes) previously written to a CSV or JDF file back
into a ResultList object by using ResultList.loadFile(...). Alternatively,
CSV files can be opened as a ResultStream by using
ResultStream.loadFile(...).
Loading into a ResultList will load the data into memory whereas opening as a
ResultStream will simply create a ResultStream object that uses the
specified file as its backing file — meaning any changes made to the
ResultStream will be written directly to the file.
Java
ResultTable list1 = ResultList.loadFile("/path/to/file.csv");
ResultTable list2 = ResultList.loadFile("/path/to/file.jdf");
ResultTable stream = ResultStream.loadFile("/path/to/file.csv");Kotlin
val list1 = ResultList.loadFile("/path/to/file.csv")
val list2 = ResultList.loadFile("/path/to/file.jdf")
val stream = ResultStream.loadFile("/path/to/file.csv")Python
lst1 = ResultList.loadFile("/path/to/file.csv")
lst2 = ResultList.loadFile("/path/to/file.jdf")
strm = ResultStream.loadFile("/path/to/file.csv")The ResultList.loadFile(...) method will automatically detect if the file is a
CSV or JDF file based on its contents. As mentioned, these methods will also load in any
attributes that were saved with the files, allowing you to retrieve them with
getAttribute(...) calls.
Often you may find yourself in a situation where you have a ResultTable object
but not the Column<> objects that were used to create it. For instance, you
may have loaded in some data from a CSV file using ResultList.loadFile(...).
For these scenarios, ResultTable object provide the find...Column(...)
method which lets you retrieve the desired Column<Type> object by specifying
its name.
For instance, if you know the name and type of the column you can use it like so:
Java
Column<Double> D = resultTable.findDoubleColumn("Name");
Column<Integer> I = resultTable.findIntegerColumn("Name");
Column<Long> L = resultTable.findLongColumn("Name");
Column<Boolean> B = resultTable.findBooleanColumn("Name");
Column<String> S = resultTable.findStringColumn("Name");Kotlin
val D = resultTable.findDoubleColumn("Name")
val I = resultTable.findIntegerColumn("Name")
val L = resultTable.findLongColumn("Name")
val B = resultTable.findBooleanColumn("Name")
val S = resultTable.findStringColumn("Name")Python
D = resultTable.findDoubleColumn("Name")
I = resultTable.findIntegerColumn("Name")
L = resultTable.findLongColumn("Name")
B = resultTable.findBooleanColumn("Name")
S = resultTable.findStringColumn("Name")Alternatively, if you have a Column<Type> object to hand with the same name, units
and type you can simply use that - ResultTable objects and their Row objects
are smart enough to match them.
However, to save it from having to search each time, you can use
findColumn(...) to find the matching column in the ResultTable like so:
Column<Type> other = ...;
Column<Type> column = resultTable.findColumn(otherColumn);To combine multiple tables into a single data structure, one can employ the
ResultGroup class. These can contain multiple tables, as well as other
ResultGroup objects, allowing one to create a directory-like structure.
ResultGroup root = new ResultGroup();
ResultTable tableA = ...;
ResultTable tableB = ...;
ResultGroup inner = new ResultGroup();
ResultTable tableC = ...;
root.addTable("Table A", tableA);
root.addTable("Table B", tableB);
root.addGroup("Inner", inner);
inner.addTable("Table C", tableC);
root.outputCSV("dataCSV.tar.gz");
root.outputBinary("dataJDF.tar.gz");The output format for these group objects is a compressed tarball (i.e.,
gzipped) within which each table is stored as either a CSV or JDF file
(depending on which output... method was called).
You can extract any column, or any value determined by an operation performed on
each row, as a List<> object by use of get(...). For instance, let's say
we have a ResultTable and we want to extract one of its columns as a list of
doubles:
Java
List<Double> values = resultTable.toList(column);Kotlin
val values = resultTable.toList(column)Python
values = resultTable.toList(column)If, for example, we instead wanted to return a list that is the product of two
columns we could supply a (Row) → Double lambda expression instead like so:
Java
List<Double> values = resultTable.toList(r -> r.get(column1) * r.get(column2));Kotlin
val values = resultTable.toList { it[column1] * it[column2] }Python
values = resultTable.toList(lambda r: r.get(column1) * r.get(column2))In fact, you can think about our first example of retrieving the values of a single column as just being shorthand for:
Java
List<Double> values = resultList.get(r -> r.get(column));Kotlin
val values = resultTable.toList { it[column] }Python
values = resultTable.toList(lambda r: r.get(column))You can extract any combination of numerical columns as a RealMatrix object by
using toMatrix(...) and specifying what you want to be in each column of the
matrix. For instance, if we had the following ResultTable:
Column<String> timestamp = Column.ofText("Timestamp");
Column<Double> voltage = Column.ofDecimals("Voltage", "V");
Column<Double> current = Column.ofDecimals("Current", "A");
ResultTable table = new ResultList(timestamp, voltage, current);
for (...) {
table.addRow(row -> {
row.set(timestamp, Util.getCurrentTimeString());
row.set(voltage, smu.getVoltage());
row.set(current, smu.getCurrent());
});
}Then we can extract both the voltage and current columns (since they're both
numbers) as a RealMatrix like so:
Java
RealMatrix matrix = table.toMatrix(voltage, current);Kotlin
val matrix = table.toMatrix(voltage, current)Python
matrix = table.toMatrix(voltage, current)The result is a matrix with two columns, the first being the values of voltage
and the second being those of current. Just as before, we can instead supply
expressions for each matrix column. For instance, we could achieve the same
result by writing:
Java
RealMatrix matrix = table.toMatrix(r -> r.get(voltage), r -> r.get(current));Kotlin
val matrix = table.toMatrix({ r -> r[voltage] }, { r -> r[current] })
// == or using "it" ==
val matrix = table.toMatrix({ it[voltage] }, { it[current] })Python
matrix = table.toMatrix(lambda r: r.get(voltage), lambda r: r.get(current))This lets us retrieve values that are derived from those in the table. For
instance, if we wanted a single column matrix of power (i.e. voltage * current):
Java
RealMatrix matrix = table.toMatrix(r -> r.get(voltage) * r.get(current));Kotlin
val matrix = table.toMatrix { it[voltage] * it[current] }Python
matrix = table.toMatrix(lambda r: r.get(voltage) * r.get(current))You can create a new coy of a ResultTable that only contains rows matching a
given criterion by using the filter(...) method. This takes a lambda
expression (specifically a Predicate<Row> object) that should return true or
false to determine whether a given row should be retained or not.
For instance, if we have a table with three columns: "Time [s]", "Voltage [V]"
and "Current [A]", we could filter these data to give us only the rows where
time > 50 seconds like so:
Java
ResultTable after50 = table.filter(r -> r.get(time) > 50)Kotlin
val after50 = table.filter { r -> r[time] > 50 }
// == or, by using "it" ==
val after50 = table.filter { it[time] > 50 }You can get a ResultTable to return a sorted copy of itself by using the
sorted(...) method. This method requires you to either specify an expression
as a lambda function to return a sortable valuable based on values in each row,
or a Column<> of some sortable type:
Java
Column<Double> number = Column.ofDecimals(...);
Column<String> string = Column.ofText(...);
// By specifying column
ResultTable sortedByNumber = table.sorted(number);
ResultTable sortedByString = table.sorted(string);
// By specifying exression
ResultTable sortedByAbs = table.sorted(r -> Math.abs(r.get(number)));Kotlin
val number = Column.ofDecimals(...)
val string = Column.ofText(...)
// By specifying column
val sortedByNumber = table.sorted(number)
val sortedByString = table.sorted(string)
// By specifying exression
val sortedByAbs = table.sorted { abs(it[number]) }For example, if we had some temperature-dependent data but it was not in any particular order:
Column<Double> temperature = Column.ofDecimals("Temperature", "K");
Column<Double> resistance = Column.ofDecimals("Resistance", "Ohm");
ResultTable table = new ResultList(temperature, resistance);
// Data gets added to table here but at random temperaturesthen we can get a copy that is in ascending order of temperature like so:
ResultTable sortedByTemperature = table.sorted(temperature);Often times, when analysing data, you will find it useful to be able to split a
table of data into multiple tables based on the value in a given column (or an
expression). ResultTable objects let you do this by use of split(...). Just
as with other examples, you either supply it with a Column<> or a lambda
expression. It will then return a map of each unique value in that
column/expression to filtered copies of the ResultTable containing only rows
with that value.
Map<Type, ResultTable> split = table.split(Column<Type> column);
Map<Type, ResultTable> split = table.split(((Row) -> Type) expression);For instance, if we had output curve data (SD Voltage, SG Voltage and SD
Current) in the columns drain, gate and current respectively, then it's
likely we will want to split the data by gate like so:
Java
Map<Double, ResultTable> split = table.split(gate);
split.forEach((gate, data) -> {
// gate is the gate voltage and data all the rows with that gate voltage
});Kotlin
val split = table.split(gate)
for ((gate, data) in split) {
// gate is the gate voltage and data all the rows with that gate voltage
}You can also split a ResultTable in smaller ones based on the direction of
change of a given numerical column/expression by use of directionalSplit(...).
For instance, if drain is swept up then down, this will split the data into
two sets, one for when drain is increasing and one of when drain is
decreasing:
List<ResultTable> split = table.directionalSplit(drain);if drain sweeps up and down multiple times, the list will contain a
ResultTable for each up-sweep and each down-sweep in alternating order.

