Simulation of Electron-Proton Scattering Events by a Feature-Augmented and Transformed Generative Adversarial Network (FAT-GAN)

1 Introduction

High-energy scattering reactions typically produce collections of particles in the final state whose momentum distributions are governed by fundamental femtometer-scale physics. One of the major goals of existing lepton-hadron scattering facilities, such as COMPASS at CERN and Jefferson Lab, as well as the future Electron-Ion Collider, is to determine the three-dimensional distributions of the hadrons' elementary quark and gluon constituent from measurements of particle production cross sections. Unfortunately, since quarks and gluons are not directly detectable experimentally, their properties must be inferred indirectly from the observed particle spectra within the theoretical framework of factorization in Quantum Chromodynamics (QCD) [Collins et al., 1989].

Since the early 1970s, Monte Carlo event generators (MCEGs) have played a vital role in facilitating studies of high-energy scattering reactions. From the experimental perspective, MCEGs are crucial for understanding the complex arrays of detectors that measure the energies and momenta of final state particles. The construction of existing MCEGs, such as Pythia [Sjostrand et al., 2008], Herwig [Bahr and others, 2008] or Sherpa [Gleisberg et al., 2009], has been driven by a combination of high-precision data from previous experiments and inputs from theory. The latter have involved a mix of perturbative QCD methods, describing the dynamics of quarks and gluons at short distances, and phenomenological models that map the transition from quark and gluon degrees of freedom to the observable hadrons ("hadronization"). An MCEG can in principle be viewed as a form of a "data compatification tool", encapsulating enormous amounts of data collected from multiple experiments, which can be regenerated from the MCEG itself. On the other hand, the reliance of existing MCEGs on theoretical assumptions of factorization and on hadronization models limits their ability to capture the full range of possible correlations between produced particles' momenta and spins.

In this work we suggest a new strategy for constructing MCEGs using modern machine learning methods involving GANs [Goodfellow et al., 2014] that can learn to generate particles in specific reactions, such as electron-proton scattering, without recourse to theoretical assumptions about femtometer-scale physics. A particular feature of GANs is their ability to generate synthetic data, such as images, by learning from real samples without knowledge of the underlying laws of the original system. The aim is to implicitly learn the primordial distributions over data which are difficult to model with an explicit parametrization for the underlying law. We propose to explore this aspect by treating the events characterized by final state particle momenta in high-energy reactions as the "images".

Typically, a GAN model is composed of a generator and a discriminator. The generator transforms random white noise through a deep neural network to produce candidate samples from the target distribution, while the discriminator learns through another deep neural network to differentiate the true samples from those produced by the generator. The GAN evolves as the generator and discriminator compete adversarially, alternatively updating their parameters during the training process. Eventually, the GAN is able to approximate the underlying cumulative distribution function (CDF) and inverse CDF transformation, and thus sample the target distribution given sufficiently large neural networks, sample size, and long enough computation time.

Although GANs have demonstrated impressive results in generating near-realistic images [Karras et al., 2018], music [Mogren, 2016], and videos [Clark et al., 2019], training a successful GAN model is known to be notoriously difficult. Many GAN models suffer from major problems including mode collapse, non-convergence, model parameter oscillation, destabilization, vanishing gradient, and overfitting due to unbalanced generator/discriminator combinations. Approaches and techniques to address these general problems have been proposed and discussed in a number of publications [Salimans et al., 2016; Arora and Zhang, 2017; Arjovsky and Bottou, 2017].

Using GANs to simulate events from particle reactions poses additional challenges for machine learning. Unlike many GAN applications, such as generating realistic and sharp looking images, where the distribution agreement between the GAN-generated samples and the true ones is often not strictly enforced, GANs for generating physical events are required to model the distributions of event features and their correlations sufficiently precisely for the nature of particle reactions to be faithfully replicated. Furthermore, events generated by GANs should not violate basic physical laws, such as baryon number, charge and momentum conservation. In order for the GAN event generator to work, a careful choice of architecture, representations, features, parameter initialization, and selection of hyper-parameters is required.

In this paper we describe the development of a GAN event generator to simulate particles in inclusive electron-proton scattering. As a proof of concept, we restrict ourselves to building a GAN that only learns how to generate specific kinds of particles in the final state, while ignoring other particles. We shall refer to such a generator as an "inclusive" generator, to distinguish it from an "exclusive" generator that generates the full spectrum of particles in the final sate. For testing and validation purposes we first use the existing theory-based Pythia8 MCEG [Sjostrand et al., 2008] to generate the training samples to train GAN. Then we train and validate GAN on data from real-life electron scattering experiment.

Our analysis shows that the difficulty of successfully training a GAN event generator lies in the complicated patterns of distributions of the physical properties of the generated particles. To improve the GAN training, we develop a FeatureAugmented and Transformed GAN (FAT-GAN), using a set of transformed features from the particle physical properties as the generated features of the generator. These transformed features represent the underlying degrees of freedom of the particles, which can simplify the learning of the generator, while avoiding generation of unphysical events. A set of augmented features is further derived from the generated features to improve the sensitivity of the discriminator. We also compare the efficiency of using Cartesian coordinates versus spherical coordinates to represent generated features in the FAT-GAN. With well-selected coordinate systems and features, our GAN model is able to regenerate particle momenta so that it mimics all the relevant momentum correlations as observed in the the original MCEG.

2 RelatedWork

In the literature, GANs have been used in a variety of applications at the Large Hadron Collider (LHC), such as simulating energy deposits of final state particles [Paganini et al., 2018a; Paganini et al., 2018b] and jets [de Oliveira et al., 2017; Musella and Pandolfi, 2018], accelerating importance sampling Monte Carlo integration [Bendavid, 2017], simulating data collections of beam studies [Erdmann et al., 2019], and reconstructing cosmic ray-induced air showers [Erdmann et al., 2018]. These efforts have focused on image representations of the aggregated experimental data as opposed to individual event-based samples. GANs have also been used to invert the detector effects such as in [Datta et al., 2018] and [Bellagente et al., 2020]. A crucial question about whether the events generated by GANs can add statistical precision beyond the training samples was investigated in [Butter et al.,

2020].

Recently there have been several attempts to investigate the possibility of directly training GANs at the event level in proton-proton collisions, such as those at the LHC. [Hashemi et al., 2019] applied a GAN to produce muon four-momenta in Z → µ+µ- events generated by Pythia. Although agreeing well in the reduced dataset, their GAN model fails to reproduce certain feature distributions. [Otten et al., 2019] reported a less satisfactory performance when a GAN was based on fully-connected deep networks in the study of the two-body decay processes e+e- → Z → l+l- and pp → t¯ t. [Butter et al., 2019] applied a GAN to simulate the 2 → 6 particle production process pp → t¯ t → (bq¯ q′ )(¯b¯ qq′ ), incorporating maximum mean discrepancy (MMD) [Gretton et al., 2007] to resolve sharp local features. [Di Sipio et al., 2020] implemented a dijet-GAN based on convolutional neural networks (CNNs) to simulate the production of pairs of jets at the LHC.

Although each of the above GANs was designed to simulate events in different reactions, they all face similar challenges of learning the unique feature distributions of the events, which often exhibit sharp edges, spikes, multiple peaks, and large variations. Building a generator capable of generating these distributions and a discriminator sensitive to their features are the keys to the success of GAN-based MCEGs.

3 Methods

3.1 Data Descriptions

We use GANs to mimic two samples of inclusive electronproton scattering events: the first is generated from the Pythia MCEG [Sjostrand et al., 2008] at a center-of-mass energy of 100 GeV, and the second is experimental data from CLAS at Jefferson Lab with beam energy of 5.5 GeV. In our initial analysis, we train the GAN only on the scattered electron momenta. Events are represented as an array of the electron four-momenta pµ = (E; p), where in Cartesian coordinates the three-momentum is given by p = (px, py, pz), and the energy is given by E = √p2x + p2y + p2z + m2 , where m is the electron mass. Throughout this paper we will work in units of GeV for all momentum and energy variables.

3.2 GANArchitecture

The generator produces both generated features and augmented features. The three generated features describe the three degrees of freedom of the scattered electron (components of the momentum p), while the augmented features, such as energy E, transverse momentum pT = √p2x +p2y , and the longitudinal to transverse momentum ratio pz/pT , can be calculated from the generated features to represent other physical characteristics of the particle. The augmented features are used to improve the sensitivity of the discriminator.

The input to the generator "G" is a 100-dimensional white noise array centered at 0 with unit standard deviation. The generator network consists of 5 hidden dense layers, each with 512 neurons, activated by a leaky Rectified Linear Unit

(ReLU) function. The last hidden layer is fully connected to a three-neuron output, activated by a linear function representing the generated features. A customized Lambda layer is then incorporated to calculate the augmented features from the generated features. Inspired by the idea of importance sampling [Ji and Li, 2016], the augmented features are carefully selected to improve the sensitivity of the discriminator in distinguishing the GAN-generated events from the Pythia input. These augmented features, together with the generated features from the generator, are concatenated and fed to the discriminator as input.

As for the generator, the neural network in the discriminator "D" also consists of 5 hidden dense layers, each with 512 neurons, activated by a leaky ReLU function. To avoid overfitting in classification, a 10% dropout rate is applied to each hidden layer. The last hidden layer is fully connected to a single-neuron output, activated by a linear function, where "1" indicates a true event and "0" is a fake event. The overall architecture of the inclusive GAN event generator is illustrated in Fig. 1.

3.3 Loss Functions

The discriminator D is trained to give D(p) = 1 for each sample p generated by Pythia, and D( ˜ p) = 0 for each sample ˜ p produced by the generator. The discriminator is optimized against the Wasserstein loss with gradient penalty [Gulrajani et al., 2017] to improve training stability and reduce the likeliness of mode collapse. The loss function LD of the discriminator is defined as

where E denotes the expectation value. The first term in Eq. (1) measures the Wasserstein distance [M. Arjovsky and Bottou, 2017]. The second term is the gradient penalty, where ̂ p is a random sample from P̂ p, defined by a uniform distribution along the straight lines between pairs of samples from the true Pythia event data and the generator's output. The coefficient λ is a harmonic parameter to balance the Wasserstein distance and the gradient penalty.

To ensure that the distributions of the event features created by the generator match with those of Pythia, we incorporate a two-sample test based on kernel MMD in our inclusive GAN event generator. To compare two distributions, the MMD employs a kernel-based statistical test method to determine if the two samples are drawn from different distributions. As a result, the loss function LG of the generator G includes a Wasserstein distance term from the discriminator D and an MMD term [Li et al., 2017],

where η is the balancing hyperparameter. The MMD term is defined as

where k(pa, pb) is a positive definite kernel function. Here we select a Gaussian kernel such that k(pa, pb) = exp[-(pa - pb)2 /2σ2 ], where σ is the hyperparameter determining the MMD resolution, tuned to the same order of magnitude as the width of the event features.

The combined network is trained adversarially for 200, 000 epochs. In each epoch, the combined network passes through the complete training dataset once. A large batch of 10, 000 events is employed to ensure that there are sufficient samples to calculate a stable MMD value in each batch. Each batch contains random examples from the Pythia event dataset. The optimizer is Adam [Kingma and Ba, 2014] with a 10-4 learning rate, β1 = 0.5, and β2 = 0.9. To balance the generator and discriminator training, the training ratio is set to 5.

3.4 Feature Representations

The physical observables characterizing the scattered electron properties are illustrated in Fig. 2, including the momentum and energy components of the electron four-vector pµ

[Fig. 2(a)-(e)], scattering angles [Fig. 2(f)-(g)], and derived quantities xbj and Q2 (see Sec. 4.2 below). The energy and momentum distributions generated by Pythia exhibit rather large variations, with the ratio between the most populated regions and those with rare events reaching up to ∼ 104 .

More seriously, a sharp edge in the E distribution arises from energy conservation, which restricts E to be less than the incident beam energy, Eb. This sharp edge is very difficult for the inclusive GAN to learn, as unphysical events can be generated with E > Eb, which the discriminator is not sensitive enough to differentiate from the eligible physical events, particularly when Eb-E is small. The sharp edge in the E distribution also leads to sharp edges in the pz and θ distributions. The difficulty of learning sharp edge distributions has also been reported in [Hashemi et al., 2019].

To address the problem of learning sharp edge distributions, we transform the momentum properties to specific generated features that allow their distributions to be generated more easily, while avoiding production of unphysical T (pz)

Figure 3: Distribution of the transformed feature T (pz).

particles. For the pz distribution, instead of directly using pz as a generated feature, we use the transformed variable T (pz) = log[(Eb - pz)/(1 GeV)] in the generator. The original distribution with a sharp edge is now converted to a distribution that is more like a Gaussian, with significantly reduced variation, as illustrated in Fig. 3.

Although T (pz) does not have actual physical meaning, the transformation ensures that the GAN will not generate events with unphysical pz values. As well as making it easier for the generator to produce, the transformed distribution T (pz) improves the sensitivity of the discriminator as a classifier. As a result, the generator learns to generate (px, py, T (pz)), and pz = exp[(Eb -T (pz))/(1 GeV)] is later calculated by the customized Lambda layer as one of the augmented features.

In addition to pz, the variables pT , E, and pz/pT are also calculated in the customized Lambda layer as augmented features in order to improve the sensitivity of the discriminator. The generated features and augmented features are concatenated as the input to the discriminator.

4 Results

In this section we compare the event feature distributions from the FAT-GAN and the Direct-Simulation GAN ("DSGAN") that directly simulates the momenta (px, py, pz). We also compare the efficiency of the FAT-GAN implementation in Cartesian coordinates ("FAT-GAN (Cart)") with the FATGAN in spherical coordinates ("FAT-GAN (Spher)"). The latter is an alternative representation to describe a particle in terms of the variables (E, θ, φ), where θ = arctan(pz/pT ) is the polar angle between pz and the transverse plane, and φ = arctan(py/px) is the azimuthal angle. The FAT-GAN (Spher) generates (T (E), T (θ), φ) in spherical coordinates as the generated features, and then px, py, pz, pT , E, and pz/pT as the augmented features. The features T (E) and T (θ) are converted from E and θ using a logarithmic transformation similar to that applied to pz to remove the sharp edges in the distribution.

4.1 Feature Distributions

The event feature distributions from the DS-GAN, FAT-GAN (Cart), and FAT-GAN (Spher) are compared in Fig. 2 with those generated from Pythia. In general the DS-GAN results do not match as well with Pythia compared to FAT-GAN

(Cart) and FAT-GAN (Spher). Moreover, the tails beyond the sharp edges in Fig. 2(c) and (d) indicate that some unphysical events are generated with E > 50 GeV. The FAT-GAN (Cart) yields match better with Pythia than do the FAT-GAN (Spher) yields, particularly for the φ distribution. The four momentum components (E; px, py, pz), as well as pT , θ and φ, are also well reproduced relative to Pythia, and have a minimal number of unphysical events.

Compared to the reaction events simulated in [Butter et al., 2019] and [Di Sipio et al., 2020], where the typical ratio between the peak and tail events is up to 10, the ratio in our scattering electron features can be up to 104 . Nevertheless, even for the rare events that are 10-3 of the number of the peak events, the FAT-GAN (Cart) agrees well with Pythia in their distributions, including the symmetry of φ shown in Fig. 2.

4.2 Inter-correlations Between Event Features

In addition to the electron momentum and energy, we examine two additional physical quantities that are typically used to characterize electron scattering, namely the squared four-momentum of the exchanged virtual photon Q2 and the Bjorken scaling variable xbj, neither of which are explicitly generated as features in the FAT-GAN. In terms of the beam and scattered electron momenta and energies, the photon virtuality can be written as -Q2 ≡ q · q = q2 0 - q2 , where q0 = Eb - E and q = (-px,-py, √E2 b -m2 - pz) are the energy and three-momentum transfer, respectively. The Bjorken variable is defined by the dimensionless ratio

and kinematically ranges from 0 to 1. As shown in Fig. 2(h) and (i), the xbj and Q2 distributions from the DS-GAN deviate significantly from those generated by Pythia. In contrast, the FAT-GAN (Cart) yields match better than those from the DS-GAN and the FAT-GAN (Spher).

The Q2 -xbj joint distributions, shown in Fig. 4, indicate that the FAT-GAN (Cart) gives a good match with the Pythia Q2 -xbj joint distribution, as indicated by the contour lines, with a χ2 value per bin of 1.58. In contrast, somewhat

(Epochs) (×103)

Figure 5: Comparison of χ2 values for xbj distributions of Pythia events and those generated by the FAT-GAN (Cart), FAT-GAN (Spher) and DS-GAN, with respect to the number of training epochs.

worse agreement (χ2 = 19.54) is observed for the FAT-GAN (Spher), and very poor agreement (χ2 = 82.41) for the DSGAN. This indicates that the FAT-GAN model in Cartesian coordinates not only learns the four-momentum vector accurately, but also their inter-correlations.

4.3 Comparison of Coordinate Representations

The Cartesian representation and the spherical representation of a particle are equivalent physically. However, the Cartesian and spherical representations exhibit different learning efficiencies in the FAT-GAN. As shown in Figs. 2 and 4, the FAT-GAN (Cart) demonstrates better agreement in the distributions of physical properties than the FAT-GAN (Spher). In Fig. 5 the convergence of the FAT-GAN (Cart), FAT-GAN (Spher) and DS-GAN is compared, as measured by the χ2 values for the xbj distributions of Pythia and GAN-generated events along training epochs. Although both FAT-GANs yield better convergence than the DS-GAN, the FAT-GAN (Cart) demonstrates better efficiency and generally lower χ2 values that are close to 1 than the FAT-GAN (Spher).

Overall, the FAT-GAN (Spher) is found to have a degraded performance compared to the FAT-GAN (Cart). The main reason is that the physical properties in spherical coordinates are less favorable than those in Cartesian coordinates for training the FAT-GAN. In particular, both the distributions of E and θ in spherical coordinates exhibit sharp edges (see Fig. 2(d) and (f)). Moreover, the φ distribution has shape boundaries at both ends, which poses additional complications for the GAN to learn.

4.4 FAT-GAN on Experimental Data

Having studied the performance of the FAT-GAN on synthetic Pythia data, we now compare the results of the FAT-GAN (Cart) simulations with inclusive scattering data for a 5.5 GeV electron beam impinging on a liquid hydrogen target in CLAS at Jefferson Lab [Mecking and others, 2003]. A clean event sample of 100k reconstructed electron events was used for the comparison with the trained FAT-GAN, illustrated in Fig. 6, which shows that the distributions of the events generated by FAT-GAN match well with those of the experimental data. Note, however, that in contrast to the smooth distributions from the Pythia MCEG in Fig. 2, the CLAS data here have not been corrected for detector effects, and therefore display artifacts such as the shoulder in the xbj distribution. Nevertheless, the comparison in Fig. 6 indicates that the physical properties pT , θ, xbj, and Q2 , which are not directly used in the training, are correctly captured by the FAT-GAN trained on the experimental event data.

5 Conclusion

We have investigated the use of GANs to simulate electrons in the final state of high-energy inclusive electron-proton collisions. We report that selecting the appropriate features as generated features or augmented features plays a critical role in building a successful GAN event generator. While the physical properties of the particles often exhibit challenging distribution patterns for a GAN to learn, using transformed features enables the generator to more easily generate and avoid unphysical events. Augmenting in addition the feature space for the discriminator to become more sensitive, our FAT-GAN demonstrates good agreement with simulated as well as experimental data in mimicking event feature distributions and their inter-correlations. We also find that the selection of the appropriate coordinate representations impacts the GAN performance. Although the FAT-GANs presented in this paper are specific to electron-proton scattering, the feature selections and transformation strategy can be generalized to GANs for simulating other reactions under different conditions, as well as learning exclusive events.

The FAT-GAN package is available at https://github.com/ ijcai2021/FAT-GAN.

Acknowledgements

We thank the CLAS Collaboration for providing the electron scattering data used in Fig. 4, and J. Qiu for helpful discussions. This work was supported by the LDRD project No. LDRD19-13, No. LDRD20-18, and No. LDRD21-22.