In this section, we overview the project's model & system architecture and introduce our solutions to several security concerns. To find out more about our security design, you can read this Research Memo.

The charts above demonstrate the story map of the mobile app. From the user's perspective, the app design can be divided into four stages. For each stage, we list our goals for a skeletal product and a minimum viable product (MVP), as well as other stretch goals. Brief implementation tips are also included.

The system contains an Android frontend application and a Python backend server, communicating via HTTPS.

To generate a new digital watermark, a user must register and log into an Author account through the authentication handler. The app will then request a unique tag from the backend main handler and invoke the local steganography SDK to embed the tag in the image provided by the user. Meanwhile, the user may add custom messages to the watermark and encrypt them with a passcode. Before exporting the "stamped" image, the app will upload all the messages along with the image checksum, which will be paired with the tag in the SQL database through the main handler.

Whoever receives the image, known as the Viewer, can retrieve the digital watermark without login. The app will extract the tag, query the main handler, and verify the image checksum. Aside from public messages, the viewer may also enter the passcode to access the encrypted messages.

The front-end for the author contains the following components.

The module allows the author to register an account with a unique username of at most 20 characters long and a password of at least 8 characters long. It runs a checker on the local device to reject too simple passwords. The username is cached on the local device, but the password is not. The module would ask users to retry if the username is not unique. The first time the user logs in to the account, the user's front end will receive a Cookie from the authentication handler as the certificate for all future communication with the server. If the Cookie is lost or expired, the user needs to log in again and obtain a new Cookie.

When the author wants to create a new watermark, the front end generates a new random 56-bit tag. The uniqueness of the tag will be verified at the backend.

The module allows the user to input a base image for the watermark as well as the messages to include. The message contains some text and optionally a file < 16MB. The module calls the Crypto SDK if the user would like to encrypt the text or file using a passcode.

The module first calls the tag extractor service of the viewer's front end and aborts if there is already a watermark in the image. Then, the embedder calls the Steganography SDK to embed the tag in the image. It also calculates a 256-bit SHA256 hash of the output image by calling the crypto SDK.

Once the input and embed processes finish, the app will upload all the data to the server by calling the Main Handler. If the upload fails (e.g., the tag is not unique, which is not likely to happen unless the user is malicious), the module would let the user retry.

After the server acknowledges that the data is correctly uploaded, the module will let the user export the new image, and optionally share it in other apps.

The module allows the author to grant or reject the usage permission requests from viewers.

The front end for the viewer is in the same application as the front end for the server. However, it contains different modules, so we separate them in the architecture.

The module allows the user to import an image.

The module reverses the process of the Tag Embedder. It calls the Steganography SDK to extract the tag embedded in the picture.

The module reverses the process of the Data Uploader. It sends the tag to the main handler to retrieve the corresponding watermark data.

The module calls the Crypto SDK to check that the image is not modified.

The module displays the data to the user and allows the user to enter the passcode to decrypt the encrypted messages, as well as downloading the accompanied files.

The module allows the viewer to request usage permission directly through the app, without exposing the contact information of the author.

The authentication handler manages user authentication.

When a user sends a register request, the module first checks whether the username is duplicate. If not, it appends the hashed password with a random salt value, and hash again. Then, it encrypts the username to generate a Cookie, sends it back to the user, and stores the username, salt, and double-hashed password in the database.

The module verifies if the username & hashed password appear in the database. If so, it regenerates a Cookie and sends it to the user. The module also allows the user to retrieve the salt value.

The main handler manages data related to the images.

The module saves the watermark messages of the user. The module first verifies the username and cookie it receives and checks again to ensure that the tag is unique. Then, it stores the watermark record, along with a timestamp. Files are stored separately in the server and represented as URIs in the SQL database.

The module allows the viewer front-end to retrieve the messages using the tag. Note that the username and timestamp fields will not be sent unless the author has set them as public.

• uid, INTEGER type, automatically incremented with AUTOINCREMENT, primary key.

• username, VARCHAR(20) type, unique.

• fullname, VARCHAR(40) type.

• email, VARCHAR(40) type.

• phone_number, VARCHAR(40) type.

• password, VARCHAR(256) type, in the form of <hash algorithm>$<salt>$<salted and hashed password>.

• created, DATETIME type, default to be CURRENT_TIMESTAMP.

• last_login, DATETIME type, default to be CURRENT_TIMESTAMP.

• is_deleted, BIT type, 1 if the row has been deleted else 0.

• tag, VARCHAR(64) type, primary key, the unique tag that will be embedded in each image.

• imgname, VARCHAR(64) type, the file name of the image embedded.

• owner, VARCHAR(20) type, owner's username of the image, foreign key to users.

• checksum, VARCHAR(256) type, checksum of the image for checking the integrity of the image later.

• created, DATETIME type, default to be CURRENT_TIMESTAMP.

• fullname_public, BIT type, set to 1 if the user wants to link its fullname info to the image, else set to 0.

• email_public, BIT type, set to 1 if the user wants to link its email info to the image, else set to 0.

• phone_public, BIT type, set to 1 if the user wants to link its phone info to the image, else set to 0.

• time_public, BIT type, set to 1 if the user wants to share the time when this image was processed, else set to 0.

• message, TEXT type, the message enclosed for this image. Set to empty string if message not enclosed.

• message_encrypted, BIT type, set to 1 if the enclosed message is a cyphertext, else set to 0.

• file_path, VARCHAR(64) type, the name of the folder that stores the enclosed file of the image.

• is_deleted, BIT type, 1 if the row has been deleted else 0

• id, INTEGER type, automatically incremented with AUTOINCREMENT, primary key.

• imgtag, VARCHAR(64) type, foreign key to images.

• username, VARCHAR(20) type, owner of this authorization relation, foreign key to users

• message, TEXT type, the message sent along with the request.

• status, VARCHAR(20) type, current status of the permission request. "PENDING", "REJECTED" or "GRANTED".

• created, DATETIME type, default to be CURRENT_TIMESTAMP.

• is_deleted, BIT type, 1 if the row has been deleted else 0.

The username column is indexed to prevent a side-channel attack based on response time.

As there is no Steganographic SDK off the shelf, we will implement one on our own based on existing algorithms. For the first LSB algorithm listed below, we plan to directly code in Kotlin to provide the steganography interface. And if we apply neural network algorithms, we may run the Python script in Kotlin to load the trained model and generate the encoded image.

LSB method is what we apply in the final version of our app. It's the easiest way to embed secret information. By replacing the least significant bit of the images with binary bits of secret information data, the secret information can be hidden in the speech. During the decoding process, the only thing to do is to extract secret information bits from the corresponding locations.

• Relevant introduction: https://www.geeksforgeeks.org/image-based-steganography-using-python, https://www.boiteaklou.fr/Steganography-Least-Significant-Bit.html.

It's a learned steganographic algorithm that enables robust encoding and decoding of arbitrary strings into images, which approaches perceptual invisibility. It comprises a deep neural network that learns an encoding/decoding process. Due to the training model structure, it's resistant to the variation in lighting, shadows, perspective, occlusion and viewing distance of the images. The requirement of the input image size is 400*400 and the size limit of the secret message is 56bits.

• Relevant code and algorithms can be found in StegaStamp.

This paper tries to place a full size color image within another image of the same size. Deep neural networks are simultaneously trained to create the hiding and revealing processes and are designed to specifically work as a pair. The system is trained on images drawn randomly from the ImageNet database, and works well on natural images from a wide variety of sources. Unlike many popular steganographic methods that encode the secret message within the least significant bits of the carrier image, our approach compresses and distributes the secret image's representation across all of the available bits.

• Relevant algorithms can be found with Pytorch Version and Tensorflow Version.

The following Q&A section helps further illustrate how each module works and, more importantly, why it is designed this way. Note that some security measures are not guaranteed in the minimal viable product.

• Can attackers invalidate my watermark by modifying some pixels / taking a screenshot / adding noise / reformatting / compressing / cropping the image?

It depends on the steganography algorithm. For the minimum viable product, we intend to use the Least Significant Bit (LSB) method. By embedding redundant information, the watermark can resist cropping attacks. For the stretch goal, we intend to use a more robust model based on machine learning to resist most modifications. For example, StegaStamp can retrieve the embedded information from an image that is printed out and rescanned. To destroy the watermark, the attacker has to modify the image thoroughly, creating identifiable artifacts. More discussion about the steganography algorithms can be found here.

• Can attackers forge my watermark?

No, as long as the attacker doesn't know your account password, s/he cannot steal your identity.

• Can attackers maliciously modify my image and let others believe that it's still a work of mine?

No, the checksum guarantees that the image cannot be modified without being noticed.

• Is the digital watermark still secure as everyone can use the app to extract the fingerprint?

Yes. Compared to traditional watermarks, hidden watermarks increase the barrier for various attacks. More importantly, the system does not count on steganography for security; rather, it relies on cryptography techniques, such as encryption, randomness, and secure hashing.

• Will my private data be uploaded to the server?

We attach great importance to your privacy and hence our backend only asks for minimal information. Your account password will be hashed before being transmitted to the server, and it will be hashed again with a salt value before getting stored in the database. Your IP address is not recorded. The base image for digital watermarking also stays locally - only a secure hash is uploaded. The message part is a bit tricky. Since the image can only hold limited information, we have to store your watermark message on the Cloud. In our stretch goals, we intend to solve this privacy issue by encrypting all the information with a random key, which is generated locally and embedded in the image together with the tag. As the image is not uploaded, the server cannot decrypt the message.

• Will my Internet Providers know that I'm using the app?

Unfortunately, yes if they want. Nevertheless, all the data is transmitted securely based on the TLS 1.3 protocol. For extreme confidentiality, consider connecting to a VPN or a TOR network.

• If I embed the same message twice with the same passcode, will the attacker know they are the same without knowing the passcode?

No, random bits will be padded before encryption to prevent this problem. Nevertheless, encryption does not prevent the attacker from estimating the length of the message.

• Will the image checksum reveal any information about the image I'm using?

No, we will use SHA256 as the hashing function to generate a 256-bit checksum for the whole image, so you don't need to worry about the vulnerability of ECB mode. Also, the checksum is calculated for the image with the watermark embedded, and we will ensure that there is some randomness in the embedding process. Therefore, the attacker cannot use the checksum to verify if you are using a certain base image.

• Can attackers guess my tag without access to the picture?

Theoretically yes, but practically NO. Each tag is a 56-bit long random number. Even if the app has a million users, only 1/70000000000 of the tags are valid.

• Can attackers determine the timestamp of my watermark by comparing tags?

No, because tags are not generated in increasing/decreasing order.

• What types of messages can I embed?

You can upload any type of data. For the viewer, though, the minimum viable product can only display text and images inside the app.

• Can I authorize another user to reproduce the image?

Yes, you may customize an authorization message.

• Can I choose not to reveal my username in the watermark?

Yes, you may choose to keep your username and the timestamp information private when generating the watermark. You can later publish them when there is a copyright conflict.

• Can I use different usernames for different images?

Yes, you may register multiple accounts to achieve this purpose.

• Can I use the app as a standard steganography tool?

Yes, you can choose not to reveal the username/timestamp and set a secure passcode for the message you intend to hide. Then, the app is effectively a standard steganography tool.