Backup of Nearline Storage System - dcm4che/dcm4chee-arc-light GitHub Wiki

Overview

  • Nearline storage is configured to have backup of objects on online cache storage system. The objects shall get deleted from the online cache storage system, once objects have been exported to nearline storage system.
  • Now assume that the nearline storage (from a different provider, eg. S3) system is also getting full day by day, which calls for backup of objects on this nearline storage system to an external archive, so that the online cache storage system can continue to backup the objects without any disruptions.

Note : The use case here is very similar to Replacement of Nearline Storage System. The only difference is that, instead of replacing the old nearline storage system altogether, the objects are sent to external archive (which acts as long term archive) from where they can be retrieved later.

Configuration

  • Assume you already have an online cache storage and nearline storage configuration as shown below.

  • Note here that online cache storage is acting as retrieve cache for nearline storage system (dcmRetrieveCacheStorageID: cache1).

  • Nearline storage acts as export storage for online cache storage (dcmExportStorageID: nearline1).

  • Also, cache1 is referred as Object Storage ID for the DCM4CHEE Application Entity.

      version: 1
  dn: dcmStorageID=cache,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
  dcmInstanceAvailability: ONLINE
  dcmProperty: pathFormat={now,date,yyyy/MM/dd}/{0020000D,hash}/{0020000E,hash}/{00080018,hash}
  dcmProperty: checkMountFile=NO_MOUNT
  objectClass: dcmStorage
  dcmStorageID: cache
  dcmStorageDuration: CACHE
  dcmExportStorageID: nearline
  dcmURI: ${jboss.server.data.url}/cache/
  
  dn: dcmStorageID=nearline,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
  dcmInstanceAvailability: NEARLINE
  dcmProperty: pathFormat={now,date,yyyy/MM/dd}/{0020000D,hash}/{0020000E,hash}/{00080018,hash}
  dcmProperty: checkMountFile=NO_MOUNT
  objectClass: dcmStorage
  dcmStorageID: nearline
  dcmStorageDuration: PERMANENT
  dcmRetrieveCacheStorageID: cache
  dcmURI: ${jboss.server.data.url}/nearline/

  dn: dicomAETitle=DCM4CHEE,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
  objectClass: dcmArchiveNetworkAE
  objectClass: dcmNetworkAE
  objectClass: dicomNetworkAE
  dicomAETitle: DCM4CHEE
  dicomAssociationAcceptor: TRUE
  dicomAssociationInitiator: TRUE
  dicomNetworkConnectionReference: cn=dicom,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
  dicomNetworkConnectionReference: cn=dicom-tls,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
  dcmAcceptedUserRole: user
  dcmAcceptedUserRole: admin
  dcmMetadataStorageID: metadata
  dcmObjectStorageID: cache
  dcmQueryRetrieveViewID: hideRejected
  dcmUpdateLocationStatusOnRetrieve: FALSE
  dicomDescription: Hide instances rejected for Quality Reasons

Backing up of objects present on old nearline storage system is explained below :

  • To do this, add a Remote Application Entity for the external archive.

  • Add a separate retrieve cache storage system. Ensure it has a deleter threshold set to it and also it is cache storage.

  • Newly created Retrieve cache storage is acting as retrieve cache for the nearline storage system (dcmRetrieveCacheStorageID: rc).

  • Nearline storage system shall start acting as cache for the external archive (dcmStorageDuration: CACHE).

  • External Archive AET shall be configured as External Retrieve AET for the nearline storage (dcmExternalRetrieveAET: TEST).

  • Configure also an exporter (with storage commitment) to export objects from nearline storage to external archive, and once exported shall send storage commitment request to external archive. This exporter shall also delete objects from rc (dcmDeleteStudyFromStorageID: rc). Ensure that external archive configuration has your AE (which acts as Storage Commitment SCU), since it is needed for storage commitment to work (in this case it is dicomAETitle: DCM4CHEE).

      version: 1        
  dn: dcmStorageID=rc,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
  dcmInstanceAvailability: ONLINE
  dcmProperty: pathFormat={now,date,yyyy/MM/dd}/{0020000D,hash}/{0020000E,hash}/{00080018,hash}
  dcmProperty: checkMountFile=NO_MOUNT
  objectClass: dcmStorage
  dcmStorageID: rc
  dcmStorageDuration: CACHE
  dcmDeleterThreshold: 500MB
  dcmURI: ${jboss.server.data.url}/rc/        
  
  dn: dcmStorageID=nearline,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
  dcmInstanceAvailability: NEARLINE
  dcmProperty: pathFormat={now,date,yyyy/MM/dd}/{0020000D,hash}/{0020000E,hash}/{00080018,hash}
  dcmProperty: checkMountFile=NO_MOUNT
  objectClass: dcmStorage
  dcmStorageID: nearline
  dcmStorageDuration: CACHE
  dcmRetrieveCacheStorageID: rc
  dcmExternalRetrieveAET: TEST
  dcmURI: ${jboss.server.data.url}/nearline/
  
  dn: dcmExporterID=ExportFromNearlineToExternalArchive,dicomDeviceName=dcm4chee-arc,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
  objectClass: dcmExporter
  dcmExporterID: ExportFromNearlineToExternalArchive
  dcmQueueName: Export1
  dcmURI: dicom:TEST
  dcmStgCmtSCP: TEST
  dcmDeleteStudyFromStorageID: rc
  dicomAETitle: DCM4CHEE
  dicomDescription: Export from nearline to external archive

External Archive Configuration :

    version: 1
    
    dn: dicomDeviceName=test-ng,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    objectClass: dcmDevice
    objectClass: dicomDevice
    dicomDeviceName: test-ng
    dicomInstalled: TRUE
    
    dn: cn=dicom,dicomDeviceName=test-ng,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    objectClass: dcmNetworkConnection
    objectClass: dicomNetworkConnection
    dicomHostname: test-ng
    cn: dicom
    dicomPort: 11112
    
    dn: dicomAETitle=TEST,dicomDeviceName=test-ng,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org
    objectClass: dcmNetworkAE
    objectClass: dicomNetworkAE
    dicomAETitle: TEST
    dicomAssociationAcceptor: TRUE
    dicomAssociationInitiator: TRUE
    dicomNetworkConnectionReference: cn=dicom,dicomDeviceName=test-ng,cn=Devices,cn=DICOM Configuration,dc=dcm4che,dc=org

  • Export the objects from nearline1 to external archive using RESTful service Export Matching Studies and passing the query parameter storageID=nearline1

  • One may also do a C-Move from external archive to rc storage system to retrieve the objects when needed. To do this, configure a new AE Title(s) of the Archive and select only rc as Object Storage ID in its Archive Network AE Extension. Use this newly configured AE as destination in the C-Move. Ensure that external archive configuration has your newly configured AE which is acting as the destination.

Add a custom footer
⚠️ **GitHub.com Fallback** ⚠️