IO with the File Device (Java) - Gleethos/neureka GitHub Wiki
The abstraction provided by the Device
interface
does not necessitate that concrete implementations
represent accelerator hardware.
Generally speaking a device is a thing that
stores tensors and optionally also handles ExecutionCall
instances.
Therefore, an implementation might also represent a simple
storage device like your local SSD or HDD...
Consequently there is a useful Device implementation which
will fulfill this requirement.
The FileDevice
class can be instantiated via a
factory method by passing a String expression containing a path :
Device device = FileDevice.at("my/relative/path/to/a/directory");
File devices manage directories, meaning that one FileDevice
instance uniquely handles
tensors stored in this directory (excluding subdirectories).
The following code shows how to store tensors on a file device :
// Creating a device which manages the directory targeted by a given path :
Device<?> device = FileDevice.at( "my/relative/path/to/directory" );
// ... and a tensor which we want to persist :
Tsr<Double> a = Tsr.of([2, 4, 3], 4);
// Initially the device does not know about the tensor "a" :
assert !device.contains(a);
// But when we store it on the file device...
device.store( a, "tensor-name.idx" ); // 'device.store( a )' if the name should be generated...
// ... then the tensor exists as a "idx" file :
assert new File( "my/relative/path/to/directory/" + "tensor-name.idx" ).exists();
// ... and the RAM will be freed ...
assert a.getUnsafe().getData() == null;
// If we want to delete the tensor from the device then we call the following :
device.free( a );
// ... and the file is gone :
assert !new File( "my/relative/path/to/directory/" + "tensor-name.idx" ).exists();
Tensors stored on a file device or any other device for that matter can be restored by calling the restore
method.
On the other hand the free
method will delete the underlying file of a given tensor.
device.restore( myTensor ); // Loads the tensor but does not delete the file.
device.free( myOtherTensor ); // Deletes the provided tensor (if stored on the file device).
Contrary to other device implementations however, the FileDevice
instance will not delete the underlying data source (in this case a file) which stores the recently restored tensor.
This is because contrary to let's say an OpenCLDevice
the whole purpose of the FileDevice