2. System design - radhikahorti/Embedding-Synthetic-Pattern GitHub Wiki

This chapter’s goal is to provide a detailed description of the process of solving the given problem statement using a functional block diagram and architecture. Implementation for three different tone-curves is also discussed.

2.1 Functional block diagram

ARGUS is an API for acquiring images and associated metadata from cameras. The fundamental ARGUS operation is capturing that is acquiring an image from a sensor and processing it into a final output image. For input, first the RAW image frames are extracted from the live capture.Other inputs include the option selected by the user and the type of of the synthetic pattern to be embedded. The options available to the user are as follows: • Simple user input (defines the position where the synthetic pattern needs to be embedded). • Based on matching the contrast of the synthetic pattern to the best suited position in the RAW image(defines only the size of the pattern that needs to be embedded). • Based on matching the contrast of the synthetic pattern to the desired location so as to not disturb the input RAW image (defines the position where the synthetic pattern needs to be embedded). • Based on matching the mean intensity of the synthetic pattern to the desired location so as to not disturb the input RAW image (defines the position where the synthetic pattern needs to be embedded).

Figure 2.1: Functional block diagram to embed synthetic patterns into raw images while preserving natural image statistics.

The synthetic pattern array that is selected is then embedded onto the input image array and the modified RAW image array is generated. The input RAW image and the modified RAW image is then fed into the local statistics measurement block that compares: • Local histogram • Contrast • Mean luminance • Mean intensity • Standard Deviation

Heatmaps for the same are generated for better visual comparison Both the RAW images are enocded into JPEG format by making use of ARGUS APIs. The SSIM score for both the images is generated for performing the global statistics comparison.

2.2 Generating the Synthetic Patterns

In order to add unique visual features to a picture for uses such as data concealing, watermarking, or image enhancement, one must generate synthetic patterns for embedding onto images.In computer vision and image processing, the Image-to-Image (img2img) diffusion process is a method used for image creation, image-to-image translation, and image editing. We make use of this method to generate synthetic patterns.

2.2.1 Dot Pattern

TensorFlow is used in Python to build an img2img diffusion setup. In order to optimize the generation of a dot pattern over a gray backdrop, it defines a generator and discriminator network. The generator first generates random noise, which it then shapes to fit the form of the dot pattern and combines it with the backdrop and dot pattern to create a picture. After that, the discriminator evaluates these created pictures using both fictitious and authentic examples. The networks’ parameters are adjusted during training to reduce the disparity between produced and real pictures over a period of epochs. Ultimately, a picture with a dot pattern is produced by the trained generator and stored as an png image

Figure 2.2: Functional block diagram for generating the Dot Pattern.

2.2.2 Uniform Color Patch

This setup uses an img2img diffusion technique to create a homogeneous color patch. It starts with a picture of random noise and, over a predetermined number of steps, diffuses it towards a desired color. The picture converges to the desired color by the repetitive adjustment of pixel values by the diffusion process. Once the picture has undergone 100 diffusion stages, it is transformed into a PIL image using the uint8 format and saved as an png image. To manage the diffusion process and the final look of the picture, the user may choose the target color (RGB values between 0 and 1), the number of diffusion stages, and the image size.

Figure 2.3: Functional block diagram for generating the Uniform Color Patch.

2.2.3 QR Code

This setup generates a QR code using the URL "NVIDIA" using the ‘qrcode‘ library. It configures the version, error correction level, box size, and border of QR codes. The QR picture is created by the code with a white backdrop and a black foreground, and it is saved as ”qr.jpg”. It further uses the built-in image viewer to display the QR code. This script is handy for creating QR codes with text, contact information, or website URLs on them. It also lets you customize the characteristics of the QR code and the data that it encodes.

2.3 Framework for Embedding the Synthetic Patterns

2.3.1 Simple User Input

Capture RAW image and convert the raw file into array.Obtain synthetic pattern from diffusion model and convert the 8 bit image into an array. Obtain position and size of the pattern through the user.Check is the position defined by the user is within the image1.Compare the pixel values at the corresponding spots in picture 1 with those in image 2 multiplied by 256.Update the value of the pixel in image1 at that position to that the value of image2 multiplied by 256 if the pixel value in image1 is greater than the pixel value in image2 multiplied by 256.Then the pattern is embedded.

2.3.2 Matching the Contrast of the Synthetic Pattern to the

best suited location in the input image

Capture RAW image and convert the raw file into array.Obtain synthetic pattern from diffusion model and convert the 8 bit image into an array. Obtain position and size of the pattern through the user.Calculate the standard deviation & determine the contrast of the pattern & Retrieves the dimensions (height and width) .Extract a block of the same size as the pattern from the image data array and then compare the contrast of the block with the pattern. If the contrast of the current block is smaller than the previous one, update the coordinates to embed the pattern to the current coordinates.

2.3.3 Matching the Contrast of the Selected block in the input

image to the Synthetic Pattern

Capture RAW image and convert the raw file into array.Obtain synthetic pattern from diffusion model and convert the 8 bit image into an array. Obtain position and size of the pattern through the user.The function calculates the contrast of the selected pattern and the contrast of the specified block from the image.It adjusts the contrast of the pattern iteratively until it aligns within a threshold (in this case, within 5 units).Once the contrast is adjusted, it embeds the modified pattern into the specified block in the image.

2.3.4 Matching the Mean Intensity of the Selected block in the

input image to the Synthetic Pattern Capture RAW image and convert the raw file into array.Obtain synthetic pattern from diffusion model and convert the 8 bit image into an array. Obtain position and size of the pattern through the user.The function calculates the mean intensity of the selected pattern and the mean intensity of the specified block from the image.It computes the intensity difference between the block and pattern, then normalizes the pattern array.The function then adjusts the intensity of the pattern by applying the intensity difference and normalization. Then the pattern is embedded.

We have chosen to implement four different variations to embed synthetic pattern into the input RAW image as mentioned above to be able to compare the local and global statistics. The local statistics that are being compared are local histogram, contrast, mean luminance, mean intensity and standard deviation. The statistics are displayed in the form of a heat map for better visual representation.Then the RAW images are encoded into JPEG format to be able to compare the global statistic in the terms of SSIM (Structural Similarity Index).The methods are then compared on the basis of these statistics to find the most optimized method to embed synthetic pattern onto the RAW image