Generative Co-Design of Antibody Sequences and Structures via Black-Box Guidance in a Shared Latent Space

1 Introduction

Antibodies are glycoproteins produced by the immune system that specifically bind to antigens-foreign molecules that trigger immune responses. Antigen recognition and binding arise primarily from the complementarity-determining regions (CDRs) of antibodies [Janeway et al., 2001; Presta, 1992]. As such, the design of CDRs is a pivotal step in developing effective therapeutic antibodies and plays a key role in advancing drug discovery.

The space of possible antibodies is vast and discrete, and experimental validation is slow and costly [Larman et al., 2012]. This drives a significant development of in silico antibody design approaches to help build a small, enriched library of candidates to identify viable options with minimal experimental measurements. Traditional computational methods sample antibody sequences/structures based on handcrafted and statistical energy functions [Lapidoth et al., 2015; Adolf-Bryfogle et al., 2018; Warszawski et al., 2019], which are inefficient and easily trapped in local optima. Recent advancements in diffusion-based deep generative models enable the simultaneous distribution modeling of both the sequence and structure of CDRs [Jin et al., 2022; Luo et al., 2022; Kong et al., 2023], thereby improving design efficiency. These models have the advantage of incorporating both antigen epitopes and antibody framework structures during the design process, which is essential to enhance binding affinity.

∗Corresponding author

The clinical effectiveness of antibody-based therapeutics relies not only on their biological performance-such as target specificity and potency-but also on their biophysical and chemical characteristics, collectively referred to as "developability". Critical aspects of developability, including solubility, thermal stability, aggregation tendency, and immunogenicity, must be carefully optimized to guarantee both manufacturability and safety in clinical applications [VillegasMorcillo et al., 2024; Sumida et al., 2024]. While most efforts primarily focus on enhancing antigen-binding properties, some recent works [Villegas-Morcillo et al., 2024; Kulyt˙e et al., 2024; Zhou et al., 2024] incorporate developability into antibody design by guiding either the sequence or structure in the raw data space, or fine-tuning the model, which usually requires extensive property evaluations.

In this work, we employ deep generative models that learn a joint antibody sequence and structure distribution to design antibody CDRs de novo, optimizing not only for antigen affinity but also for enhanced developability. Following the principle that sequence determines structure, and structure determines function [Branden and Tooze, 2012], we propose a novel LatEnt blAck-box Design (LEAD) framework (Fig. 1), which guides property optimization through the shared latent space of sequences and structures. Specifically, to address real-world scenarios where many property evaluators are not differentiable, we design our guidance strategy based on black-box techniques, allowing flexible and generalpurpose optimization across diverse antibody design tasks. The contributions are summarized as follows:

• We propose LatEnt blAck-box Design (LEAD), a

method that jointly optimizes sequence and structure within the shared latent space of a diffusion model for antigen-specific antibody design. LEAD establishes a general framework, and different variants of LEAD are proposed for single and multiple property optimization.

• LEAD enables property guidance by incorporating black-box optimization into the denoising process, without any additional training of diffusion models.
• Experimental results demonstrate that LEAD achieves superior optimization efficiency compared to existing approaches that perform guidance in the raw antibody space. Specifically, it requires fewer property evaluations while maintaining or exceeding performance in both single and multiple-property optimization.

2 RelatedWork

Antibody Design Deep learning methods have achieved great success in antibody design. Large Language Model (LLM)-based approaches are efficient in generating protein sequences [Saka et al., 2021; Shuai et al., 2023], but they struggle to design antibodies for specific antigens since protein function is usually shaped by a complex interplay between sequence and structural features. The first CDR sequence-structure co-design model is introduced for designing antibodies against SARS-CoV-2 [Jin et al., 2022], but its reliance on an antigen-specific predictor limits its generalizability to other antigens. Recently, diffusion models have shown significant potential in both continuous and categorical data domains and have been successfully applied to generating high-quality protein sequences or protein backbones [Watson et al., 2023; Ingraham et al., 2023; Lisanza et al., 2024; Gruver et al., 2024]. DiffAb [Luo et al., 2022] proposed the first diffusion model to perform joint design of sequence and structure of the antibody CDR regions while conditioning on the antigen-antibody complex. AbDiffuser [Martinkus et al., 2024] improved this further by incorporating strong priors and being more memory efficient with side chain generation. DiffAbXL [Ucar et al., 2024b] is an advanced version of DiffAb, with an improved architecture.

Property Optimization Sequence-structure co-design methods enhance the antigen-targeting performance of antibodies. Recent studies further incorporate property optimization to improve their developability. DIFFFORCE [Kulyt˙e et al., 2024] introduces a differentiable force field w.r.t. the 3D atom coordinates to guide the sampling process. ABDPO [Zhou et al., 2024] fine-tunes DiffAb via direct preference optimization to optimize energies of generated antibodies. [Villegas-Morcillo et al., 2024] performs guidance in the raw space of antibodies according to developability properties during diffusion sampling. However, these methods require extensive property evaluations, limiting their scalability and efficiency.

3 Black-box Antibody Design in the Shared Latent Space of Diffusion Model

In this section, we first introduce the background of diffusion model for antibody sequence and structure co-design. Then we develop our black-box guidance strategy in the shared latent space of this co-design model.

3.1 DiffusionModel for Antibody Co-design

Our work builds upon the diffusion model for antibody sequence and structure co-design. It enables the joint generation of Complementarity-Determining Region (CDR) sequences and structures conditioning on the antibody framework and its corresponding bound antigen. Specifically, each amino acid is represented by its type si ∈ {ACDEFGHIKLMNPQRSTVWY}, Cα atom coordinate xi ∈ R3 , and amino acid orientation Oi ∈ SO(3), where i = 1, 2, . . . , N and N is the total number of the amino acids in the antibody-antigen complex.

The target CDR loop is denoted as A = {(si, xi, Oi) |i = 1, 2, . . . ,m}, given the rest of the antibody-antigen complex R = {(sj, xj,Oj) |j ̸= i, j ∈ {1, 2, . . . ,N}}. For the sake of notation, let s = {si}mi=1 , x = {xi}mi=1 ,O = {Oi}mi=1 denote the whole feature of each modality of the target CDR loop, respectively. Then, we have A = (s, x,O). The antibody generation task can be then formulated as modeling the conditional distribution P(A|R). DiffAb [Luo et al., 2022] represents the first application of Denoising Diffusion Probabilistic Model (DDPM) [Ho et al., 2020] to antibody generation, which consists of a forward diffusion process and a reverse generative process.

Forward diffusion process (t = 1, . . . , T) The antibody sequence-structure co-design framework initially assumes the three modalities are conditionally independent. Namely,

where A0 = (s0 = {s0i }mi=1 , x0 = {x0i }mi=1 , O0 = {O0 i }mi=1 ) is the noisy-free CDR loop at time step 0. Similarly, At = (st , xt ,Ot ) is the noisy CDR loop at time step t.

Then, the forward diffusion process gradually introduces noise into each modality through different distributions towards the prior distributions. Namely,

where i = 1, 2, . . . ,m is the index for the amino acid. I(·) is the one-hot operation that converts each amino acid type to a 20-dimensional one-hot vector and 1 is an all-one vector. {βt }Tt=1 is the noise schedule for the diffusion process, and we define ¯αt = ∏tτ=1 (1 - βτ ) and ¯ βt = 1 - ¯αt . Here, C(·),N(·), and IGSO(3)(·) are the categorical distribution, Gaussian distribution on R3 , and isotropic Gaussian distribution on SO(3) [Leach et al., 2022] respectively. ScaleRot scales the rotation angle with a fixed rotation axis to modify the rotation matrix [Gallier and Xu, 2003].

Reverse generative process (t = T, T - 1, . . . , 1) The reverse generative diffusion process learns to recover data by iteratively denoising from the prior distributions. The parametric models pθ (At-1 |At ,R) are employed to approximate the posterior distributions at each generation time step. To model the interplay between sequences and structures, a three-output neural network for the three modalities with a shared encoder and separate decoders [Luo et al., 2022] is adopted, which takes the whole sequence-structure context R and the noisy antibody at the previous denoising step At = (st , xt ,Ot ) as input and predicts At-1 = (st-1 , xt-1 , Ot-1 ). Namely,

In particular, the denoising process of three modalities from time step t to time step t - 1 can be defined as follows:

where i = 1, 2, . . . ,m is the index for the amino acid. f1, f2, f3 are parameterized by SE(3)-equivariant neural networks, whose i-th entry denotes the output for the i-th amino acid, respectively.

3.2 Function-guided Latent Optimization

With the pre-trained joint diffusion model defined in Section 3.1, we explore how to incorporate the functional guidance F(At ) (without loss of generality, we assume higher function reward is better), defined with respect to sequence and structure, into the denoising sampling process to generate antibodies with targeted properties.

To enable more efficient functional guidance, we no longer implement guidance in the raw data space. Instead, we consider the shared latent space of sequences and structures.

First of all, we define the shared latent space of multiple modalities. According to the definition of the three-output denoising neural network, we reformulate it as follows:

where E(·) denotes the shared encoder while D∗(·) denotes the modality-specific decoder. Denoting E (At , R) as Z t , we consider the space where Z t resides as the shared latent space of multiple modalities.

To avoid extra noise distraction, we adopt the deterministic version of the reverse generative process, a.k.a. Denoising Diffusion Implicit Models (DDIM) [Song et al., 2020]. For simplicity of notation, let DDIM also denote a one-step reverse generative process consisting of a three-output denoising network with a shared encoder (E, D1, D2, D3).

Definition 1. [DDIM for deterministic sampling] Given the noisy antibody At and the rest antigen-antibody context R, the subsequent denoised antibody At-1 can be obtained deterministically via DDIM, i.e., At-1 ∼ DDIM (D1, D2, D3, Z t , βt) where Z t = E (At , R) and βt is the t-step noise schedule as in Eq. (4). For simplicity, we denote At-1 = G(Z t ), t = 1, 2, . . . , T .

In order to obtain high reward design, we propose optimizing the shared latent code Z t to obtain the high-reward denoising CDR loop At-1 = G(Z t ) at each sampling step. Namely,

Since the shared embedding Z t encodes both sequence and structure, any updates toZ t affect all three modalities simultaneously, ensuring their synchronization throughout the optimization process.

3.3 Reward-guided Updates

Most property evaluators and denoising sampling for the CDRs are not differentiable, so we cannot perform white-box functional guidance. Motivated by the black-box optimization technique [Wierstra et al., 2014; Nesterov and Spokoiny, 2017], we propose learning the distribution of latent codes Z t to enable black-box gradient updates, thereby facilitating an efficient optimization of Z t . Let π(Z t ;Wt ) denote the distribution of latent code, where Wt is the distribution parameter. We can reformulate Eq. (6) as

Algorithm 1 LatEnt blAck-box Design (LEAD)

• 1: Input: a pre-trained antibody diffusion model featuring three-output denosing network with shared encoder (E, D1, D2, D3), the antibody-antigen backbone R, the dynamic step size {βt}Tt=1 , the property evaluator F(↑), number of evaluations per time step K, initial guidance step Tinit <= T , the step size δ. 2: Initialize: AT = (sT ∼ C (I(sT )|1/M ) , xT ∼ N(xT |0, I), OT ∼ IGSO(3) (I, 1) ). 3: for t = T, T -1, . . . , 1 do 4: if t > Tinit then 5: At-1 ∼DDPM (D1,D2,D3, E (At ,R) , βt); 6: else 7: Z t = E (At ,R); 8: for k = 1, 2, . . . , K do 9: Z tk = Z t + σδtk where δtk ∼ N (0, I); 10: At-1 k ∼DDIM (D1,D2,D3,Z tk , βt); 11: end for 12: Obtain the optimal noise δt⋆ based on Section 3.3; 13: At-1 ⋆ ∼DDIM (D1,D2,D3,Z t +σδt⋆ , βt); 14: end if 15: end for 16: Output: generated CDR loop A0⋆ .

Using the log-likelihood trick, the Monte Carlo gradient estimation w.r.t. Wt is estimated as:

where Z tk is sampled from π(Z t ;Wt ). K is the number of function queries used to refine Wt .

To further simplify, we instantiate the population distribution π as an isotropic multivariate Gaussian with mean Wt and fixed covariance σ2 I, allowing us to write J(Wt ) in terms of Wt directly. Denoting Wt = Z t = E(At ,R), Eq. (7) can be reformulated as:

Based on Eq. (8) and Eq. (9), we obtain the gradient w.r.t. Z t :

Thus, the latent code Z t can be updated as follows:

However, we find that Eq. (11) fails to achieve an effective optimization in the case of sequence derived properties (see empirical study). We hypothesize that it may result from the discrete nature of sequence space. Inspired from the fact that Eq. (11) actually updates Z t with a reward-weighted noise, we propose sampling an optimal noise based on probability derived from function rewards to be compatible with the discrete nature of data. Specifically, the K reward values can be converted into probabilities via the softmax function:

The optimized noise index is sampled from this distribution:

which is a one-hot vector. The optimized noise can be derived by δt⋆ = ∑Kj=1 γ t j δt j . The latent code is updated by:

It is intuitive to derive a hard selection version of the above strategy by defining:

We refer to this strategy as the hard selection version of LEAD, termed LEAD-H, and to Eq. (13) as the soft selection version of LEAD, termed LEAD-S.

Additionally, we find that combining LEAD-H/LEAD-S with the weighted version of LEAD (termed LEAD-W) yields better results, respectively. We hypothesize that LEAD-W can explore a larger search space, thereby enhancing the advantages of LEAD-H/LEAD-S. The combining strategy is as the following. With a slight abuse of annotation, we re-denote the optimal noise obtained by LEAD-H or LEAD-S as ζ t1 . Defining ζ t2 = 1 σK ∑Kk=1 δtkF (G(Z t + σδtk )), we further define the optimal noise as:

We perform LEAD guidance from the later stage of reverse diffusion process, namely, from Tinit (≤ T) steps. Before the guidance (t > Tinit), the denoising process replicates the original one, namely

The whole algorithm is summarized in Algorithm 1.

4 Experiment

We follow [Villegas-Morcillo et al., 2024] to consider two developability properties, i.e., hydropathy score and folding energy for antibody design.

• Hydropathy score (abbr. Hydro). It averages the hydropathy values over the generated CDR sequences and serves as a proxy for solubility and aggregation [Kyte and Doolittle, 1982]. Negative scores indicate hydrophilicity while positive scores indicate hydrophobicity, and small score is preferred.
• Folding energy difference (abbr. ∆∆G). It denotes the structural stability of generated antibodies relative to the reference. Following [Villegas-Morcillo et al., 2024], we predict ∆∆G (abbr. Pred ∆∆G) using an existing predictor released by [Shan et al., 2022], which takes as input the amino acid sequence and Cα atom coordinates. A lower value suggests improved structural stability.

Dataset We utilize the test set described in [Luo et al., 2022], which consists of 19 antibody-antigen complexes derived from the SabDab database [Dunbar et al., 2014]. The test set includes protein antigens from various pathogens, such as influenza and SARS-CoV-2.

Hyperparameters Unless otherwise specified, the latent noise scale σ is adaptively set to βt (as in Eq. (3)) and the number of evaluation per time step K is set to 20, the initial guidance step Tinit is set to 50. It is more effective to add guidance at a later stage of the denoising process as samples from the earlier steps can be too noisy to evaluate accurately.

Baselines We employ the widely adopted antibody sequence-structure co-design diffusion model framework, DiffAb [Luo et al., 2022] as the pretrained model. We consider the relevant work, property optimization under the framework of sequence-structure co-design as baselines 1 . [Villegas-Morcillo et al., 2024] performs guidance similar to our hard/soft selection strategy, but on the raw data space. We name it as GuideRaw-H and GuideRaw-S for hard and soft selection version, respectively. Except where mentioned, the experimental setup follows the setting of the original work. That is, the initial guidance step Tinit is set to 100 and the number of evaluation per time step K is set to 20.

1 Other baselines DIFFFORCE [Kulyt˙e et al., 2024] and ABDPO [Zhou et al., 2024] do not release their codes. Nevertheless, DIFFFORCE requires a differential force field model, which is inapplicable to the black-box setting of this work. ABDPO needs to finetunes DIFFAB, requiring a large amount of property evaluations.

Evaluation In addition to the properties mentioned above, we evaluate these designs using the following metrics: (i) Amino Acid Recovery (AAR), which measures the sequence identity between the reference and generated CDR sequences; (ii) Root Mean Square Deviation (RMSD), which computes the Cα atom distance between the reference and generated CDR structures. AAR and RMSD are reported for reference purposes and are not bolded in Table results. For a detailed explanation, please refer to point (4) in Section 4.1.

4.1 Single Property Optimization

For all methods, we generate 100 designs per CDR (six CDRs in total) for each of the 19 test antigen-antibody complexes. The guidance method is to optimize the hydropathy score and the predicted ∆∆G, respectively. Here, both GuideRaw and our LEAD adopt the hard selection version for a fair comparison. Other guidance strategies are studied in Section 4.4.

We report the evaluation results in Table 1. (1) The results show that our LEAD outperforms GuideRaw in terms of both target properties, which is attributed to our optimization over the shared latent code ensuring the synchronization of different modalities. (2) LEAD conducts guidance in only 50 steps, compared to the 100 steps required by GuideRaw. This demonstrates that our method achieves better performance while requiring fewer black-box function evaluations. (3) DiffAb performs unconditional generation and exhibits strong AAR and RMSD scores. This indicates that the antibody generated by this sequence-structure co-design model can recover CDR structures similar to the given reference. However, the target properties of the antibodies produced by DiffAb fall below expectations. (4) Property optimization often leads to a degradation of AAR and RMSD, as it requires adjusting the generated CDRs to meet property requirements. Specifically, the hydropathy score is determined by the sequences only. As a result, we observe that antibodies optimized by all property guidance methods exhibit significant AAR degradation but improved hydropathy scores. Notably, the RMSD scores remain relatively low across all generations, indicating that the generated CDR structures are still similar to the references.

Fig. 2 visualizes the generated antibodies. It shows that all methods can generate valid sequences and structures. GuideRaw and our LEAD obtain better hydropathy score and predicted ∆∆G than natural antibodies.

4.2 Multi-property Optimization

Optimizing multiple properties presents challenges because each property can have different value scales and units. To address this, we normalize property values to ensure they contribute comparably during optimization. We adopt a weighted sum with adjustable weights to balance trade-offs according to design priorities. Here, we optimize both the hydropathy score and ∆∆G with w ∗ ∆∆G + (1 - w) ∗ Hydro. The weight w is set as 0, 0.25, 0.5, 0.75, 1, respectively. For a fair comparison, we set up the same number of function queries for our LEAD and GuideRaw in this setting. That is, both methods set Tinit = 50 and K = 20. Still, we generate 100 generated designs per CDR for each of the 19 test antigenantibody complexes.

The results are summarized in Fig. 3. The hydropathy and energy properties exhibit a trade-off, where improving one often comes at the expense of the other. Our LEAD effectively navigates this balance, achieving superior solutions compared to the baseline approaches.

(a) Predicted ∆∆G guided antibody design

(b) Hydropathy score guided antibody design

Table 2: Average AAR, RMSD, Hydro, and Pred ∆∆G on the test complex 5xku C B A (design H-CDR1). For a fair comparison, property guidance is incorporated for all methods after 50 time steps respectively.

4.3 Query Efficiency

By guiding optimization in the shared latent space, our LEAD captures property semantics effectively and preserves the correspondence between sequence and structure, leading to more efficient guidance. We conduct experiments with different numbers of queries, 20 , 21 , 22 , 23 , 24 , 25 per guided denoising step. In this experiment, all methods generate 100 designs for H-CDR1 from one of the test complexes and perform property-guided denoising after 50 denoising steps.

The results in Fig. 4 demonstrate the efficiency of our method. To be specific, with an increased number of blackbox evaluations, the optimization process explores a wider range of possibilities, leading to a more thorough refinement of the target property. It is interesting to see that our LEAD achieves a much larger performance gain in optimizing the predicted ∆∆G than in optimizing the hydropathy score. The predicted ∆∆G is relevant to both sequences and structures, while the hydropathy score is only determined by sequences. This particularly demonstrates our method's superiority on those sequence-structure derived properties.

4.4 Guidance Strategies

Here, we explore the efficacy of different guidance strategies demonstrated in Section 3.3, namely, LEAD-H, LEADS, LEAD-W, LEAD-(W+H), LEAD-(W+S).

(1) Our LEAD with different guidance strategies achieves better results in terms of optimizing predicted ∆∆G. This property is relevant to both antibody sequence and structure.

The guidance over the shared latent space indeed enhances its optimization. (2) In terms of the hydropathy score, LEAD-W does not achieve efficient optimization, resulting in a score similar to that of unconditional generation. This is because the hydropathy score is a sequence derived property, and its gradient space is not continuous. As a result, the weighted strategy fails to perform an effective search. Instead, the discrete search strategies, LEAD-H and LEAD-S, are effective in this case. (3) The weighted strategy (Eq. (16)) can slightly boost the discrete search strategies, as demonstrated by the results of LEAD-(W+H) and LEAD-(W+S).

5 Conclusion

In this work, we introduce LEAD, a black-box guidance framework that operates in a shared latent space of sequence and structure, enabling efficient property optimization for antibody CDRs without model retraining. LEAD is generally effective at maintaining sequence-structure synchronization; however, its performance is limited by the capacity of DiffAb, which may fail to preserve meaningful alignment between sequence and structure. More advanced co-design frameworks [Yang et al., 2025; Uc¸ar et al., 2024a] may offer improved capabilities in this regard. Future work will also incorporate wet-lab experiments to validate computational designs and provide feedback to further guide optimization. Beyond antibody design, LEAD is broadly applicable to other AIdriven co-design tasks involving coupled sequence-structure modalities, e.g., protein, RNA, and small-molecule design.

Acknowledgments

This paper was supported by the Manufacturing Trade and Connectivity (MTC) Individual Research Grant (IRG) (Grant No. M24N7c0091). It is also supported by the National Research Foundation, Singapore under its National Large Language Models Funding Initiative (AISG Award No: AISGNMLP-2024-004). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore.