Deletion of least recently accessed study from retrieve cache - dcm4che/dcm4chee-arc-light GitHub Wiki

Description

The test is to verify that object(s) of least recently accessed studies are deleted from Retrieve Cache if the deleter threshold is reached on the retrieve cache storage system.

Configuration

In addition to basic configuration done for retrieve cache, below configuration is needed to enable deletion of least recently accessed studies from retrieve cache if the deleter threshold is reached.

Using Archive UI

Go to Menu->Configuration, then on Devices page, Edit the dcm4chee-arc device. Go to Extensions and
Edit extension of Device Extension. Again go to its Extensions and Edit extension of Archive Device.
Enter values in fields Purge Storage Polling Interval, Purge Storage Fetch Size and Delete Study Batch Size as required by one's application needs and Save.
Next go to the archive device extension's Child Objects and select a retrieve-cache storage from Storage Descriptor. Enter value for field Deleter Threshold and Export Storage ID. One may add more than one Deleter Threshold and Save.

Using LDAP

One may either

create a LDIF file (e.g.):

  version: 1		
  dn: dcmStorageID=retrieve-cache,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
  changetype: modify
  add: dcmDeleterThreshold
  dcmDeleterThreshold: 200MB
  -
  
  dn: dcmStorageID=retrieve-cache,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
  changetype: modify
  add: dcmExportStorageID
  dcmExportStorageID: fs1
  -
  
  dn: dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
  changetype: modify
  add: dcmPurgeStoragePollingInterval
  dcmPurgeStoragePollingInterval: PT5M
  -
  
  dn: dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
  changetype: modify
  add: dcmPurgeStorageFetchSize
  dcmPurgeStorageFetchSize: 100
  -
  
  dn: dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
  changetype: modify
  add: dcmDeleteStudyBatchSize
  dcmDeleteStudyBatchSize: 10
  -

and import it to the LDAP Server by using the ldapmodify command line utility.

or use the Add Attribute... and Add Value... function of Apache Directory Studio to add attributes on Storage Level (e.g.: dcmStorageID=dcm4chee_store) in the Archive Configuration.

One may refer to Storage Descriptor and Archive Device to understand the description of attributes.

One may also add a deleter threshold as 1[hour=10-11 dayOfWeek=*]100MB, which means

priority = 1
size = 100MB
schedule = timings between 10a.m. to 11a.m. any day of the week

Refer ISO-8601 Duration format for more know-how on Durations.

Go to the Control tab on Configuration page in archive UI and reload the configuration.

Verification

Ensure that you have studies both on fs1 and retrieve-cache storage systems.

Once the polling interval has been reached, and the space on retrieve-cache storage system is below the configured threshold limit, the object(s) of studies which are least recently accessed start getting deleted. Below is an example of how it is seen in Wildfly server log

      2018-05-29 15:09:24,849 INFO  [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedScheduledExecutorService-default-Thread-8) 
      Usable Space on Storage[id=retrieve-cache, uri=file:/home/vrinda/work/wildfly-11.0.0.Final/standalone/data/retrieve-cache/] below 2GB - 
      start deleting 123.749376MB
      
      2018-05-29 15:10:24,939 INFO  [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedScheduledExecutorService-default-Thread-13) 
      Successfully delete objects of Study[pk=746, uid=1.3.12.2.1107.5.8.1.12345678.199508041416590859569, id=000001] on Storage[id=retrieve-cache, 
      uri=file:/home/vrinda/work/wildfly-11.0.0.Final/standalone/data/retrieve-cache/]
      
      2018-05-29 15:10:24,959 DEBUG [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedScheduledExecutorService-default-Thread-13) 
      Successfully delete Location[pk=9772, systemID=retrieve-cache, path=2018/05/29/5E9204E8/5E9C00D5/6459578D, tsuid=1.2.840.10008.1.2, 
      size=529440, status=TO_DELETE, objectType=DICOM_FILE] from Storage[id=retrieve-cache, 
      uri=file:/home/vrinda/work/wildfly-11.0.0.Final/standalone/data/retrieve-cache/]
      
      2018-05-29 15:10:24,968 DEBUG [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedScheduledExecutorService-default-Thread-13) 
      Successfully delete Location[pk=9773, systemID=retrieve-cache, path=2018/05/29/5E9204E8/5E9C00D5/5E9C75E8, tsuid=1.2.840.10008.1.2, 
      size=529440, status=TO_DELETE, objectType=DICOM_FILE] from Storage[id=retrieve-cache, 
      uri=file:/home/vrinda/work/wildfly-11.0.0.Final/standalone/data/retrieve-cache/]

Note that the least recently accessed studies are deleted only if they are also available on fs1 storage system. Hence dcmExportStorageID need to be configured as shown above in configuration

Repeat the test by ensuring that complete study is present on fs1, but only some object(s) of study are available on retrieve-cache. Once polling interval is reached, object(s) of study get deleted from retrieve-cache as can be seen in below Wildfly server log.

      2018-05-29 15:22:30,778 INFO  [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedScheduledExecutorService-default-Thread-4) 
      Usable Space on Storage[id=retrieve-cache, uri=file:/home/vrinda/work/wildfly-11.0.0.Final/standalone/data/retrieve-cache/] below 2GB - 
      start deleting 124.38016MB
      
      15:22:30,869 INFO  [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedScheduledExecutorService-default-Thread-4) 
      Successfully delete objects of Study[pk=748, uid=1.3.12.2.1107.5.8.1.12345678.199508041416590859569, id=000001] on 
      Storage[id=retrieve-cache, uri=file:/home/vrinda/work/wildfly-11.0.0.Final/standalone/data/retrieve-cache/]
          
      15:22:30,901 DEBUG [org.dcm4chee.arc.delete.impl.PurgeStorageScheduler] (EE-ManagedScheduledExecutorService-default-Thread-4) 
      Successfully delete Location[pk=9780, systemID=retrieve-cache, path=2018/05/29/5E9204E8/5E9C00D5/6459578D, 
      tsuid=1.2.840.10008.1.2, size=529440, status=TO_DELETE, objectType=DICOM_FILE] from Storage[id=retrieve-cache, 
      uri=file:/home/vrinda/work/wildfly-11.0.0.Final/standalone/data/retrieve-cache/]