Verify - Nanook/NKit GitHub Wiki

Verifying with NKit is a bit more involved than might first be expected. Different verification methods are performed depending on the Task and if:

  • Checksums are contained in the image header
  • A Dat file is available
  • An NKit Scan is available

When verifying, each of these may or may not be present. NKit will use the most reliable methods for the Task being performed.

NKit's Verify mode is configured by setting the v param to:

  • y: Auto Verify mode
  • n: Off
  • datLookup: Always gather checksum information from the image being processed and look it up in a dat

It is also enabled by using the Verify Task. The Verify Task uses the value of the v param, however if it's n it is overridden with y.

The Verify Task is basically the Scan Task without the scan data being saved to disk.

This documentation attempts to explain the Verify Methods NKit uses and when. It uses the term full image to mean the image as if expanded.

Verify Methods

NKit is able to verify using the following methods.

Verify Method Requires Description
InScanCompare - Compares in-memory scans from 2 steps e.g. Convert step and Verify step
InChecksum Header Checksums Input image header checksums are compared to the full image
DatMatch Dat File Checksums of a dat file entry matched by input filename are compared to the full image
ScanCompare NKit Scan File An NKit scan matched by input filename is compared to the expanded input image
DatLookup Dat File The full image is checksumed and looked up by MD5 or SHA1 along with the CRC32 for a match
No Verify - There are no methods to verify the image

If DatMatch fails to match the dat entry it will attempt a DatLookup.

The following sections explain how they are applied per task type.

Convert Task Verify

Lossless Convert - Method: InScanCompare

When converting to a lossless file format (e.g. rvz or lossless wbfs/ciso) NKit will take 2 steps, one to convert the input image while creating an in-memory scan of the full image and another to verify the conversion to compare the input scan with the verify scan.

Lossy Convert - Method: DatLookup

Lossy conversions (e.g. regular wbfs/ciso) require a dat file. The input image will be converted then verified. The MD5 or SHA1 plus the CRC32 will be looked up in the dat for a match.

Lossy conversions will not match regular preservation dats. Custom dats would need to be provided. If you do not have a specific dat to support lossy images then ensure verify is turned off (set to n).

DatLookup Verify - Method: DatLookup

The verify param (v) is set to datLookup. The input image will be converted then verified. The MD5 or SHA1 plus the CRC32 will be looked up in the dat for a match.

Dedupe Task Verify

Default - Method: InScanCompare

Dedupe verify will dedupe the input image while creating a scan of the full image. The scan is written to disk and opened. The verify step then scans the deduped image as a full image and compares the verify scan against the input scan.

DatLookup Verify - Method: DatLookup

The verify param (v) is set to datLookup. The input image will be deduped then verified. The MD5 or SHA1 plus the CRC32 will be looked up in the dat for a match.

Fix Task Verify

Fix requires a dat file to validate that the fixed file is correct and matches a dat entry. Without a dat it cannot be confirmed that it has been successful.

Default - Method: DatLookup

The input image will be modified then verified. The MD5 or SHA1 plus the CRC32 will be looked up in the dat for a match.

Verify Task + Expand and Scan Task Verify

The default verify. NKit employs a fallback approach to attempt to find the most reliable verify from available data.

Method Description
InChecksum Header checksums in the input image are used to verify the full image
DatMatch No header checksums, if there's a dat then lookup the input filename for a match to verify against
ScanCompare No header checksums or datMatch. Load an input filename matched NKit Scan to verify against
DatLookup The last resort, read the input image calculating MD5 or SHA1 plus CRC32 to lookup a dat entry

Summary

Once understanding the many verify modes above, this table should be fairly self explanatory. It concisely displays the logic

Task Lossless Checksums DatMatch NKitScan Dat Verify Method
Convert ✅ - - - - InScanCompare
Convert ❌ - - - ✅ DatLookup
Convert ❌ - - - ❌ No Verify
Dedupe - - - - - InScanCompare
Extract - - - - - No Verify
Fix - - - - ✅ DatLookup
Fix - - - - ❌ No Verify
Other - ✅ - - - InChecksum
Other - ❌ ✅ - - DatMatch
Other - ❌ ❌ ✅ - ScanCompare
Other - ❌ ❌ ❌ ✅ DatLookup
Other - ❌ ❌ ❌ ❌ No Verify

Other is Verify, Scan, Expand

When Verify mode (v) is set to 'datLookup' there must be a dat set. This mode restricts the Verifying to the rows that have the Dat column set to ✅.