Robust Steganography without Embedding Based on Secure Container Synthesis and Iterative Message Recovery

1 Introduction

Steganography refers to the hiding of a secret message into a media carrier [Bandyopadhyay et al., 2008], such as an image [Cheddad et al., 2010], so as to conduct concealed communication by transmitting the carrier in a public channel [Marvel et al., 1999], such as the Internet or social software. Traditional steganography methods directly embed the secret message into the carrier image by modulating some unnoticeable components, such as the least significant bits of pixel values [Van Schyndel et al., 1994] or texturerich areas [Holub and Fridrich, 2012]. To further improve hidden capacity and message recovery robustness, various deep learning-based steganography methods have been proposed [Baluja, 2017; Zhu et al., 2018]. However, the embedding processing inevitably leads to abnormalities of statistical characteristics of the carrier image and, in consequence, to an inherent risk of detection by machine learning-based steganalysis tools.

∗Corresponding authors

To overcome this issue, steganography without embedding (SWE) methods can be designed to implement a mapping relationship between secret messages and hash sequences to select a natural image from a prepared database [Zhou et al., 2015]. However, the required image quantity of such mapping-based SWE methods is exponential to their hidden capacity, which limits their efficacy. To address this problem, synthesised-based SWE methods [Hu et al., 2018] implicitly represent a secret message by synthesising a container image using deep generative networks, such as GANs [Goodfellow et al., 2020]. These methods can hide more than 100 message bits in one image, which is several times the hidden capacity of mapping-based methods. Due to this relatively high hidden capacity and the modification-free hiding process, which is inherently immune to typical steganalysis tools, synthesisbased SWE has gained wide interest.

However, there are still a number of drawbacks that hinder real-world applications based on existing synthesis-based SWE methods: (1) most of them are weak in terms of robustness because they fail to ensure accurate message recovery under various image attacks; (2) existing SWE methods designed to enhance robustness train their generator using adversarial training, which leads to significant degradation of image synthesis fidelity; and (3) they suffer from the risk of message leakage, since, due to the mapping between secret messages and synthesised containers, if someone else who owns the extractor they can also recover hidden messages.

To tackle the above drawbacks, in this paper, we propose Secure container synthesis and Iterative message recoverybased Steganography Without Embedding (SI-SWE) as a novel robust SWE method. SI-SWE consists of a secure weight modulation-based generator, which constructs a transformation from secret messages to synthesised images, and a robust difference predictor, which predicts the difference between two unknown messages by contrasting their transformed images. Our proposed secure container synthesis mechanism introduces a secure factor to modulate the synthesis of the secure generator, while the iterative message recovery updates an estimate of the secret message using the predicted difference for a pair of container images: the original and one that is reconstructed from the estimate by the generator. Furthermore, we design a novel adversarial training strategy that uses various image attacks to only train the difference predictor. Separating the training of the generator and the difference predictor has two benefits. On one hand, using an adversarial training strategy to train the difference predictor improves the recovery robustness, since the difference predictor can accurately predict the differences from tampered images and the iterative mechanism also reduces the impact of attacks through the reconstruction. On the other hand, the generator is trained without considering attacks to reduce degradation caused by adversarial training, which guarantees high synthesis fidelity. In addition, secure factors are introduced to modulate the transformation of secret messages by the generator, which means that different secure factors will lead to different synthesised containers for hiding the same message. In this manner, using fake factors will mislead the recovery of secret messages since the reconstructed images cannot accurately represent their message differences against the container images, which in turn effectively prevents message leakage.

Our main contributions in this paper can be summarised as follows:

• Robust message recovery: we design a difference predictor via iterative reconstructions of container images together with an adversarial training strategy for message recovery, leading to strong robustness to accurately recover hidden secret messages even from seriously tampered container images;
• Enhanced undetectability: we design different strategies for the training of the weight modulation-based generator and the difference predictor, with the former trained without considering attacks and the latter using adversarial training. As a result, SI-SWE reduces degradation caused by adversarial training and guarantees high synthesis fidelity to minimise the risk of container images being suspected, while ensuring high recovery robustness;
• Avoiding message leakage: we introduce secure factors to modulate the transformation from secret messages to synthesised container images so that the differences between different messages can only be accurately predicted from their synthesising images modulated by the same secure factor. This ensures that hidden messages are inaccessible without knowing the correct factor, significantly reducing the risk of message leakage.

Our code is made available at https://github.com/Lemok00/ SI-SWE.

2 RelatedWork

2.1 Steganography Based on Embedding

Traditional steganography embeds a secret message into a carrier image through (minimal) modifications. [Van Schyndel et al., 1994] first proposes to replace the least significant bits (LSBs) of pixel values in images with secret message bits, while [Sharp, 2001] extends this method by adjusting the whole pixel values to avoid "pairs of values" [Westfeld and Pfitzmann, 1999]. For resistance against steganalysis, [Pevn`y et al., 2010] introduces high-dimensional models to minimise the weighted difference of feature vectors, while [Holub and Fridrich, 2012] embeds secret bits into texture-rich and noisy image regions, and [Holub et al., 2014] extends this method to arbitrary domains.

Deep learning-based steganography approaches yield improvements in terms of undetectability, capacity and robustness. For improved undetectability, [Tang et al., 2017; Tang et al., 2020] propose the use of GANs or a reinforcement learning framework to predict probability maps for modification, which minimises pixel-level embedding costs. To enlarge the hidden capacity, [Baluja, 2017; Baluja, 2019] train deep networks to hide full-size images into another image. [Shaoping et al., 2021] employs invertible neural networks to improve the recovery quality with a large capacity. For improved robustness, [Zhu et al., 2018; Wengrowski and Dana, 2019] train DL-based models against perturbation pipelines or specialised datasets to ensure accurate message recovery from tampered or camera-captured images. Furthermore, [Xu et al., 2022] proposes a normalising flow which models the distribution of redundant components to recover high-quality images from tampered images.

However, embedding-based steganography methods carry the inherent risk that any modification on natural images leaves traces in statistical characteristics, resulting in their lack of resistance against steganalysis tools.

2.2 SteganographyWithout Embedding

To overcome this issue, [Zhou et al., 2015] proposes to hide messages by choosing unmodified natural images whose hash sequences are the same as message segments. To improve both hidden capacity and recovery robustness, image hashes can be generated based on the scale invariant feature transform (SIFT) [Zheng et al., 2017], discrete wavelet transform coefficients [Liu et al., 2020], or objects recognised by a Faster R-CNN [Luo et al., 2020]. Moreover, [Zou et al., 2022] uses an unsupervised clustering algorithm for the image database to improve efficiency and robustness. However, such mapping-based SWE methods require a number of database images exponential to the hidden capacity, limiting the hidden capacity.

Significantly higher hidden capacities can be achieved through image synthesis-based methods. [Hu et al., 2018] first proposes to train GANs [Goodfellow et al., 2020] with an extractor to achieve concealed message transmission through synthesised container images. To improve recovery reliability and synthesise diversity, [Liu et al., 2022] disentangles images into structure and texture representations, and synthesise a container image by a randomly sampled texture and a structure representing a message. For application in online social networks, [You et al., 2022] trains both the message hiding and recovery modules against a differential approximation of JPEG compression to synthesise container images which can defend against all possible attacks in transmission update 𝑻 times channels.

3 ProposedMethod

3.1 Overview

An overview of our proposed SI-SWE method is given in Figure 1. Our approach consists of a secure container synthesis mechanism for message hiding, and an iterative message recovery mechanism for message recovery. For message hiding, we first map a secret message m to a secret vector z and then input z together with a secure factor f to a generator G to synthesise a container image x which can then be transmitted through noisy channels with various image attacks. For message recovery, the hidden secret message is recovered using a difference predictor P together with the same factor f and the generator G to iteratively update an estimate vector ˆ zt and reconstruct the attacked container image ˆx. In the following, we explain the key modules of SI-SWE in detail.

3.2 Mapping Processes

Instead of directly inputting a message m to the generator, we take a vector z to achieve flexible hidden capacity and construct a mapping relationship between z and m. For this, we employ themapping processM and its inverse process ˜M proposed in [Liu et al., 2022], where M maps m to z through mapping its every σ-bit segment si to a float value zi in range

where hi is the decimal value of si. ˜M performs the inverse mapping

and recovers the segment si from hi.

3.3 Generator

In SI-SWE, the key module for message hiding is the generator G, which takes a secret vector z and a secure factor f as input to synthesise a container image x as

where Gtrans(·) transforms z to a high-dimension feature map, Gmap(·) maps f to a latent vector, and Gsyn(·) adopts StyleGAN2 [Karras et al., 2020] as its backbone architecture to synthesise high fidelity images.

The training procedure of G is illustrated in Figure 2, where three images are firstly synthesised using different combinations of secret vectors and secure factors as

where, based on to the mapping interval of M, z1 and z2 each consist of nz elements sampled from a uniform distribution U(-1, 1), while f 1 and f 2 are sampled from a Bernoulli distribution B(nf , 0.5) with nf the length of f and 0.5 the probability.

To train the generator to synthesise containers modulated by secure factors for message hiding, we use the difference predictor P to calculate a prediction loss Lpred and an inverse loss Linv. Since x21 and x11 are synthesised using the same factor but with different vectors, Lpred is used to ensure their difference vector d = (z1 - z2 ) can be accurately predicted, and we thus define the loss as

where ∥ · ∥1 denotes the L1 loss. On the other hand, x12 is synthesised using the same vector as x11 but with a different factor. The inverse loss Linv is thus designed to prevent message recovery using fake factors by maximising the predicted difference and is obtained as

where τinv refers to a margin greater than 0 and indicates that the difference vector whose norm exceeds the margin will not contribute to the loss.

To ensure realistic image synthesis, a generative loss term Lgen is introduced to make all synthesised images indiscriminative from real images and is defined as

where X = {x11 , x12 , x21 } and D denotes a discriminator trained against the generator.

In addition, since using only the generative loss results in the generator synthesising images of a limited number of contents and styles [Kazemi et al., 2019], we introduce content and style diversity losses, Lcd and Lsd, to enhance synthesis diversity. For this, we train the generator to synthesise images with content diversity while using different secret vectors, and with style diversity while using different secure factors and define the loss terms as

where Lcc and Lsc refer to the content and style consistent losses proposed in [Kazemi et al., 2019], and τc and τs are margins. We thus associate the secret vectors with the synthesised contents since the locality of contents makes the recovery of vectors more reliable, and the secure factors with the synthesised styles since the globality of styles will lead to a significant recovery error when there is a minor difference between the input factors.

The over loss of the generator is then obtained as

where the hyper-parameters λpred, λinv, λgen, λcd, λsd control the impact of each loss term. It should be noted that, to reduce the degradation of synthesis performance, our generator is trained without considering any image attacks.

3.4 Difference Predictor

As shown in Figure 1(c), we design a difference predictor P to predict the difference between a pair of images ˆx and ˆ yt-1 asˆ

where Pfeat refers to a feature encoder, Pref to a reference attention module [Yu et al., 2021b], and Phead to a convolutional prediction head.

Pref fuses encoded features ψx and ψy as

with

where γ is a learnable parameter, and k(·), q(·), v(·) are three functions implemented by 1 × 1 convolutions.

The training procedure of the difference predictor is illustrate in Figure 3 and consists of three parts:

(1) Attack images. To train the difference predictor for robust message recovery, we first tamper a container image x to obtain an attacked container ˆx as

where A denotes an attack randomly selected from Table 1, and the gradient back-propagation is stopped at ˆx to avoid the effect of non-differential operations in attacks.

(2) Initialise estimates. We initialise an estimate vector ˆ z0 by randomly sampling it from the uniform distribution U(-1, 1) and synthesise a reconstructed container ˆ y0 through the generator G as

where f is the secure factor used to hide the secret vector into the container image x.

(3) Update estimates. We update the estimate vector ˆ z and the reconstructed container ˆ y iteratively T times, where T is uniformly sampled from {1, ..., Tmax}. In particular, in the t-th iteration (1 ≤ t ≤ T), the difference predictor firstly contrasts ˆx and ˆ yt-1 to predict a difference vector ˆ dt, and the estimate vector ˆ zt-1 is then updated to ˆ zt as

where ⊕ denotes an element-wise addition operation. Finally, the reconstructed container is updated, using ˆ zt and f, as

We calculate a vector recovery loss

and an image reconstruction loss

of the t-th iteration, where z is the hidden secret vector. The total loss of the difference predictor is then obtained as

where λvec and λimg balance vector recovery and image reconstruction.

4 Experimental Results

4.1 Experimental Setup

To demonstrate the superiority of SI-SWE, we compare it with seven state-of-the-art synthesis-based SWE methods, namely DCGAN-Steg [Hu et al., 2018], SAGAN-Steg [Yu et al., 2021a], SSteGAN [Wang et al., 2018], WGAN-Steg [Li et al., 2020], GDA-Steg [Peng et al., 2022], IDEAS [Liu et al., 2022], and CIS-Net [You et al., 2022]. We use three publicly available datasets: LSUN [Yu et al., 2015] Bedrooms, LSUN Churches, and FFHQ [Karras et al., 2019] to train the models. All images in the datasets are resized to a resolution of 256×256 pixels. We train SI-SWE with two different input dimensions, namely SI-SWE-64 with nz = nf = 64, and SI-SWE-256 with nz = nf = 256. The training hyperparameters are set to λpred = λvec = 10, λinv = 5, and τinv = 0.75 to ensure both robust message recovery and preventing message leakage, while we set λcd=4 to enhance the diversity of the synthesised images' contents. We further set, empirically, Tmax = 10, λgen = 2 and, λsd = λimg = τc = τs = 1, while σ in the mapping process is set to 1.

4.2 Message Recovery Robustness

To compare the robustness of SI-SWE with other SWE methods, we evaluate their recovery accuracies under the attacks listed in Table 1, where each attack (except image quantisation) adopts 6 different parameters with only 3 of the 6 having been used during the adversarial training. The average accuracy results for all methods and all attacks are reported in Table 2.

From there, it is evident that our SI-SWE method achieves superior message recovery robustness, with SI-SWE-64 yielding 100% under all attacks, and SI-SWE-256 giving the second best average accuracy of 98.92%. Although CIS-Net32 obtains slightly higher accuracies for some attacks compared to SI-SWE-256, it supports only a significantly lower hidden capacity (32 vs. 256 bits), which is also difficult to expand as demonstrated by the severely reduced accuracy results for CIS-Net-64 (73.62% vs. 94.12%).

4.3 Container Undetectability

To evaluate the undetectability of the SWE methods, we test their synthesised container images in terms of resistance against steganalysis tools and synthesis fidelity.

For resistance evaluation, we select three well-known steganalysis tools, namely StegExpose [Boehm, 2014], XuNet [Xu et al., 2016], and YeNet [Ye et al., 2017]. We obtain the receiver operating characteristic (ROC) curves of detection results and report the area under the curve (AUC) values in Table 3. As we can see from thee, all SWE methods achieve AUC values close to 0.5, which means that detection by the steganalysis tools is close to random guessing. This confirms that SWE methods are fundamentally immune to the detection by current steganalysis tools, as they synthesise realistic images to represent the secret message without modifications.

The synthesis fidelity of synthesised images determines their imperceptibility to visual inspection, and is determined by the authenticity and diversity of the generated images. We use the Fr´echet inception distance (FID), a widely employed synthesis fidelity metric, to evaluate the synthesis fidelity of all SWE methods, and report the obtained results in Table 4. As is apparent from there, SI-SWE outperforms all othe SWE methods on all three datasets, with SI-SWE-64 yielding an average FID of 19.93, and SI-SWE-256 an even better average FID of 17.92 as it takes higher-dimensional secret vectors as input which leads to more diverse characteristics. Container images generated by SI-SWE are thus less likely to be suspected compared to images synthesised by the other approaches.

Examples of images synthesised by all SWE methods are shown in Figure 4 to also allow a more subjective evaluation. Looking at the container images generated by SI-SWE, it is evident that they are of high authenticity and diversity, and that it would be very hard to identify them as synthesised images.

4.4 AvoidingMessage Leakage

To evaluate SI-SWE's effectiveness in avoiding message leakage, we evaluate the recovery accuracies to recover messages from quantised (QT) images using different secure factors: the real factor and other randomly sampled fake factors. Table 5 lists the obtained results and shows significantly decreased recovery accuracies when using fake factors compared to using the real secure factor for SI-SWE-64 and SISWE-256. This confirms that our proposed method ensures that hidden messages are inaccessible without knowing the real factor, thus significantly reducing the risk of message leakage.

4.5 Ablation Studies

Adversarial Training Strategies

We train SI-SWE using the following adversarial training strategies and compare the resulting performance in terms of recovery and synthesis: without attacks as a baseline, using the strategy of CIS-Net, and using our proposed strategy. Table 6 shows the obtained average recovery accuracies and FID scores, and demonstrates that our proposed strategy significantly enhances the robustness of message recovery, since more attacks are used, and effectively reduces degradation of image synthesis fidelity due to the the separated training of generator and difference predictor.

τinv Settings

We perform experiments to investigate how different settings of τinv impact the performance of SI-SWE. For this, we evaluate the FID scores together with the recovery accuracies from quantised (QT) images for different factor bit error rates (f-BERs) between the real and fake factors. The factor-BER is defined as

and we use three f-BERs in our experiments, f-BER1 = 12.5%, f-BER2 = 25%, and f-BER3 = 50%. We evaluate for τinv = {0.25, 0.5, 0.75, 1}, and report the results in Table 7. From there, we can observe that there is a trade-off between synthesis fidelity performance of and the ability to avoid message leakage. When τinv increases, recovery accuracies decrease to prevent access to hidden messages, while FID scores increase indicating worse synthesis fidelity. Compared to τinv = {0.25, 0.5}, setting τinv = 0.75 much better avoids message leakage while FID scores increase only slightly, whereas τinv = 1 significantly compromises the synthesis fidelity so that container images are more likely to arouse suspicion. We consequently recommend τinv = 0.75 as the best balance to avoid message leakage while guaranteeing high synthesis fidelity to enhance undetectability.

Diversity Loss Settings

To investigate the impact of the diversity loss terms Lcd and Lsd, and the hyper-parameter λcd, on the synthesis fidelity of SI-SWE, we evaluate the FID scores when removing the loss items and varying the weight parameter. As shown in Table 8, removing the diversity loss term or reducing λcd leads to degraded fidelity, demonstrating the effectiveness and necessity of this component, while we recommend setting λcd to 4.

5 Conclusions

In this paper, we have proposed SI-SWE, a synthesised-based SWE method which is fundamentally immune to detection by steganalysis tools. SI-SWE not only guarantees high image synthesis fidelity, but also achieves reliable message recovery under a variety of image attacks, while the messages hidden in container images are inaccessible without knowing the right secure factor to reduce the risk of message leakage. Compared to other state-of-the-art synthesis-based SWE methods, SI-SWE is shown to effectively prevent message leakage while achieving superior performance in terms of recovery robustness and synthesis fidelity. In future work, we plan to further enlarge the hidden capacity of our approach, which, in comparison to some other techniques such as fullimage hiding methods, is still relatively small.

Acknowledgements

This work was supported in part by the National Innovation 2030 Major S&T Project of China (2020AAA0104203), the National Natural Science Foundation of China (62006007, 61602527), the Natural Science Foundation of Hunan Province, China (2022GK5002, 2020JJ4746), the Special Foundation for Distinguished Young Scientists of Changsha (kq2209003), and in part by 111 Project (No. D23006).