EDyGS: Event Enhanced Dynamic 3D Radiance Fields from Blurry Monocular Video

1 Introduction

Novel view synthesis [Zhou et al., 2016] plays a critical role in the 3D vision domain and enables more immersive experiences in applications such as augmented reality/virtual reality (AR/VR), autonomous driving, and 3D content creation. Recently, methods based on reconstructing radiance fields, such as Neural Radiance Field (NeRF) [Mildenhall et al., 2021] and 3D Gaussians Splatting (3DGS) [Kerbl et al., 2023], have demonstrated remarkable potential in novel view synthesis.

Such methods leverage multi-view consistency to reconstruct radiance fields, relying on high-quality calibrated images as inputs [Cannici and Scaramuzza, 2024]. However, such inputs are rarely available in real-world scenarios. In dynamic scenes, the presence of motion blur, caused by both camera motion and object motion, further complicates the reconstruction process, preventing NeRF and 3DGS from producing sharp radiance fields and significantly limiting their practical applicability.

Recent works [Ma et al., 2022; Lee et al., 2023; Wang et al., 2023] address motion blur caused by camera motion by estimating camera movement during exposure time. However, these methods are not well-suited for dynamic scenes, where motion blur arises from a combination of camera and object motion. Simulating camera motion alone is insufficient to ensure multi-view consistency, making it difficult to accurately represent object motion and maintain temporal coherence. A straightforward approach is to separate the sources of blur: camera motion and object motion. However, decoupling these two types of blur from blurry images is a challenging task. Some recent works [Sun et al., 2024] attempt to address this by using pre-trained networks to predict 2D-level object motion masks from blurry images. However, this approach is constrained by the severity of the blur and the absence of multi-view consistency (Figure 1 (a)).

Event cameras [Gallego et al., 2020] are high-resolution neuromorphic sensors that capture motion cues during image exposure. We observe that in dynamic scenes, event streams maintain consistency across viewpoints in static regions but lose consistency in dynamic regions due to object motion (Figure 1 (b)). By analyzing the multi-view information from event streams, we can differentiate between static and dynamic regions, further decoupling the blur caused by the two types of motion. This facilitates the reconstruction of dynamic radiance fields.

In this work, we propose EDyGS 1 , a model specifically designed to take event streams and monocular videos of dynamic scenes with motion blur as inputs and generate sharp novel views. To tackle the challenge of decoupling the two types of motion blur, we leverage the multi-view information from the event stream to construct a motion mask field that differentiates between static and dynamic regions. This allows us to handle deblurring and reconstruction separately for each region. Specifically, we introduce a motion-mask

1 Source code and Supplementary Materials are available at: https://github.com/zju-bmi-lab/EDyGS

3D Gaussian Splatting model, where each Gaussian is assigned an additional attribute to indicate whether it belongs to a static or dynamic region. By leveraging event streams to supervise this model, we create a 3D-level mask that guides the reconstruction process. We then adopt a progressive learning strategy, sequentially reconstructing the static and dynamic regions. For static regions, we optimize both the camera pose and the learnable 3D Gaussians. For dynamic regions, we train an implicit deformation field alongside learnable 3D Gaussians. Experimental results demonstrate that EDyGS outperforms existing dynamic 3D radiance field methods, offering a significant advancement in reconstructing sharp, dynamic scenes.

In summary, the major contributions of our work are:

• We introduce EDyGS, the event-enhanced dynamic 3DGS model specifically designed to effectively tackle motion-blurred monocular video.
• We propose a motion-mask 3D Gaussian Splatting model that uses event streams to reconstruct the motion mask field, separating static and dynamic regions for targeted deblurring and reconstruction.
• Experimental results demonstrate that our method achieves the state-of-the-art performance.

2 RelatedWork

Novel View Synthesis for Dynamic Scenes. Synthesizing novel views from a series of 2D images is an important and challenging task in 3D static [Liao et al., 2024; Zhang et al., 2024] and dynamic reconstruction. For dynamic reconstruction, some NeRF-based methods [Park et al., 2021a; Park et al., 2021b; Pumarola et al., 2021] incorporate a deformation field to map point coordinates to a canonical space that corresponds to a specific timestamp. They synthesize high-quality novel views, but the training and rendering speeds are slow. Besides, some Gaussian-based methods [Yang et al., 2024; Wu et al., 2024b; Duan et al., 2024; Li et al., 2024; Yang et al., 2025] are proposed, achieving both fast training and high rendering quality. Both NeRFbased and Gaussian-based methods heavily rely on sharp inputs. When dealing with images affected by motion blur due to rapid objects or camera movement, they perform poorly because they lack accurate spatial information in the image to guide the supervision.

Our method accounts for camera and object motion blur and handles the motion blur effectively.

Novel View Synthesis from Blurry Images. In static 3D scene reconstruction, some methods [Ma et al., 2022; Wang et al., 2023; Cannici and Scaramuzza, 2024; Lee et al., 2025; Chen et al., 2025a; Chen et al., 2025b] are proposed to address motion blur caused by camera movement. However, in dynamic 3D scene reconstruction, motion blur occurs due to the combined effect of camera motion and object motion. DyBluRF [Sun et al., 2024] and Deblur4DGS [Wu et al., 2024c] attempt to recover the sharp scene from blurry images. However, they rely on inaccurate blur information with a lack of multi-view consistency, thereby limiting their effectiveness. Additionally, since DyBluRF is based on NeRF, its training and rendering process are relatively slow (2 days for training on an RTX A6000 GPU).

We integrate event streams with more precise information into reconstructing 3D dynamic scenes from blurry images with 3DGS (1.5 hours for training on an RTX 3090 GPU).

Event-based Novel View Synthesis. The event camera [Brandli et al., 2014], also called the Dynamic Vision Sensor (DVS), captures the intensity change of each pixel. The high temporal resolution of event cameras enables them to record events with precise information at a high frequency [Yan et al., 2025; Wu et al., 2024d]. As a result, event cameras are commonly used in static 3D scene reconstruction tasks [Chen et al., 2021; Chen et al., 2024]. Assuming the camera motion is predefined, some 3D scene reconstruction techniques [Qi et al., 2023; Klenk et al., 2023; Low and Lee, 2023; Cannici and Scaramuzza, 2024] leverage multi-view data from events to refine the 3D scene. Other methods [Qi et al., 2024; Yu et al., 2024; Ma et al., 2024] jointly optimize both camera motion and the 3D scene. However, dynamic scenes introduce additional challenges due to object motion, which makes traditional methods unsuitable for such scenarios.

We leverage observations from event streams to separately consider camera motion blur and object motion blur, thereby optimizing 3D dynamic scenes from blurry images.

3 ProposedMethod

In this section, we demonstrate how to reconstruct dynamic 3D radiance fields using monocular videos with motion blur B = {B0,B1, ...,Bn-1} and their corresponding event streams ET0,0→Tn-1,k-1 . For each frame Bp and the corresponding event stream ETp,0→Tp,k-1 , the exposure start time and end time are Tp,0 and Tp,k-1 respectively.

We first describe the method for decoupling two types of motion blur (camera motion blur and object motion blur) in Section 3.1. We then introduce a 4D field M to represent dynamic and static regions and propose motion-mask 3D Gaussians to implement this field in Section 3.2. Finally, we demonstrate how to utilize this field to reconstruct dynamic 3D radiance fields from blurry monocular videos B and their corresponding event streams E in Section 3.3.

3.1 Decoupling Two Types of Motion Blur

In dynamic scenes, motion blur can be categorized into two types according to its source: camera motion and object motion. Camera motion blur occurs when the camera moves during the exposure time, while object motion blur arises from objects in the scene moving during the same period [Sun et al., 2024]. Distinguishing between these two types of blur in an image with motion blur is a challenging task.

To tackle this problem, we propose dividing a dynamic scene into dynamic regions and static regions. Dynamic regions correspond to parts of the scene that undergo changes over time t, such as moving objects. On the other hand, static regions remain invariant over time t, representing stationary background elements. This division allows us to associate motion blur more specifically with its source.

We find that motion blur caused by the camera movement is confined to static regions, while blur caused by the object movement is limited to dynamic regions. By leveraging this distinction between dynamic and static regions, we can decouple the two types of motion blur in an image. This decoupling is not precise but serves as a robust starting point for further analysis.

Distinction Between Dynamic and Static Regions. As

a camera moves through a dynamic scene, it naturally generates a monocular video. Each frame in the video represents a specific moment in time and a distinct viewpoint. For static regions, these areas remain unchanged over time, so they appear the same when observed from different viewpoints (or moments in time). In contrast, dynamic regions change over time, resulting in differences when viewed from different viewpoints (or moments). This fundamental distinction between dynamic and static regions allows us to utilize multi-view relationships inherent in images to identify them. Static regions will maintain consistent appearance and structure across views, while dynamic regions will exhibit changes, making them distinguishable.

However, blurry images do not provide accurate multiview relationship information [Cannici and Scaramuzza, 2024]. In contrast, event streams, with their high temporal resolution and lack of blur [Gallego et al., 2020], can provide precise information. Therefore, we can utilize the multi-view relationships of event streams to identify static and dynamic regions in a dynamic scene (Shown in Figure 1 (b)).

3.2 MotionMask Field

We propose a motion mask field M to represent the dynamic and static regions in the dynamic scene. This motion mask field represents the probability that a point x in the canonical space belongs to a dynamic region at time t,

where m ranges between 0 and 1. A value of m = 0 indicates that the point belongs entirely to a static region, while m = 1 means that the point is fully within a dynamic region.

This motion mask field plays a crucial role in decoupling motion blur caused by camera motion and object motion. By leveraging this field, we can optimize the dynamic and static components of the scene separately, ensuring accurate reconstruction and rendering for both regions.

Motion-mask 3D Gaussian Splatting. We approximate this motion mask field representation using motion-mask 3D Gaussians, an extension of the vanilla 3DGS framework [Kerbl et al., 2023]. Unlike vanilla 3DGS, which focuses solely on rendering images C, our proposed 3D Gaussians model is capable of rendering both images C and 2D mask images for each viewpoint. These masks enable the distinction between dynamic and static regions within the scene, providing an essential tool for decoupling motion blur.

To achieve this, we introduce an additional attribute m for each Gaussian. This attribute represents the probability that the corresponding Gaussian belongs to a dynamic region. The 2D mask image M is rendered using the alpha blending of the splatted 3D Gaussians onto the 2D image plane. The process is defined as:

where mt represents the dynamic probability for each Gaussian, αt is the alpha value of the Gaussian, and the product term ∏t-1 u=1 (1 -αu) accounts for the accumulated transparency of the Gaussians layered before t.

This field enables the generation of view-specific motion masks, which are crucial for capturing the temporal and spatial variation of dynamic regions in a scene. Unlike other methods [Sun et al., 2024], which obtain masks from blurry images using a pre-trained model, our approach generates masks at the 3D level, ensuring multi-view consistency.

3.3 EDyGS

Overview. Figure 2 illustrates the framework of our method. Our framework is divided into three stages: Pose Initialization, Static Region Reconstruction, and Dynamic Region Reconstruction. In the Pose Initialization stage, the event stream and the blurry monocular video are processed to generate relatively latent sharp images and corresponding initial poses. In the Static Region Reconstruction stage, our method jointly optimizes the static region, motion mask field, and initial poses. In the Dynamic Region Reconstruction stage, we focus on reconstructing the dynamic region while simultaneously refining the motion mask field and poses.

Pose Initialization. To obtain the initial trajectory of the event camera and the point cloud of the dynamic scene for 3DGS optimization, we follow the approach outlined in E2NeRF [Qi et al., 2023]. First, we preprocess the n motionblurred images {Bp}n-1 p=0 using the EDI model [Pan et al., 2019]. Specifically, for each motion-blurred image Bp, we uniformly sample k - 2 timestamps to segment the corresponding event streams ETp,0→Tp,k-1 . Using the EDI model, we then generate nk latent sharp images { ˆBi}nk-1 i=0 , which capture more texture details compared to the original blurry image. Finally, we employ COLMAP [Schonberger and Frahm, 2016] to obtain the initial trajectory {Pi}nk-1 i=0 of the event camera and initial point cloud of the dynamic scene.

Static Region Reconstruction. Given that the static regions within dynamic scenes exhibit consistency across different viewpoints and time instances, we focus solely on optimizing the static region of the Gaussian field and the trajectory of the event camera in the first stage. We first construct a differentiable path from the static region of the Gaussian field to the trajectory of the event camera. According to Equation 2, the rendered result related to the static region of the Gaussian field is given by ˆ C i st = ˆ Ci(1 -Mi), where ˆ Ci is the rendered image. Then we build the partial derivatives between ˆ C i st and the trajectory {Pi}nk-1 i=0 of event camera:

Using the above differentiable path (as detailed in the Supplementary Materials), we jointly optimize the camera trajectory and the static part of the Gaussian field. Meanwhile, to adaptively distinguish between the static and dynamic regions of the Gaussian field, we bias M towards classifying as static, so that the 3D mask can better identify the parts that satisfy multi-view consistency.

We separately utilize events and latent sharp images to supervise the joint training process, which includes the static region of the Gaussian field, the motion mask field, and the trajectory of the event camera:

where Θ is the threshold of the event camera, and λ is 0.2.

Dynamic Region Reconstruction. Following the approach in [Yang et al., 2024], we control the Gaussian using the deformation field Fθ. First, we predict the motion of each Gaussian, which transforms the canonical 3D Gaussian to the deformed space:

where sg(·) is the stop-gradient operation, γ is the positional encoding, ∆µi, ∆ri, and ∆si represent the offset of perGaussian center point, rotation, and scale. Then we use the motion mask field obtained from the first stage to suppress the motion of the static part of the Gaussian field:

Finally, we refine the motion of the Gaussian field by regularizing the motion of the static part:

This enables the dynamic part of the Gaussian field to more effectively learn the object motion in the dynamic scene from the event streams. Meanwhile, we continue to fine-tune the mask field and the camera trajectory using both events and latent sharp images, which helps to better decouple the camera motion from the object motion:

where ˆMi is the rendered 2Dmask image in the static region reconstruction stage.

4 Experiment

In this section, we introduce the datasets, experimental setup, comparisons with other methods, and ablation study to demonstrate the effectiveness of our approach. More experimental results are in the Supplementary Materials.

4.1 Datasets

Synthetic Data. Since no publicly available dynamic scene datasets contain both blurry images and event streams, we collect four synthetic scenarios. The data for these scenes is sourced from [Wu et al., 2024a]. We first generate the corresponding events by applying Vid2E [Gehrig et al., 2020] to sharp images with a resolution of 400 × 400, rendered in Blender. Next, we process nine sharp images using the simulation method in E2NeRF [Qi et al., 2023] to produce one blurry image. As a result, we obtain 60 blurry images for each scenario, along with their corresponding event streams.

Real-world Data. We use the DAVIS346C [Taverni et al., 2018] to capture six real-world dynamic scenes with both RGB frames and spatial-temporal aligned event streams. Each video is 5 seconds long, with the RGB frames having an exposure time of 100 milliseconds and a resolution of 346 × 260. The real-world dynamic scenes contain blurry images along with their corresponding events.

4.2 Experiment Settings

Metrics. We utilize standard metrics to evaluate the effectiveness of our method, including PSNR [Huynh-Thu and Ghanbari, 2008], SSIM [Wang et al., 2004], and LPIPS [Zhang et al., 2018].

Baselines. We compare our method with four approaches: (1) Deformable 3DGS (D3DGS) [Yang et al., 2024], a method used for dynamic scene reconstruction; (2) DyBluRF [Sun et al., 2024], a method for dynamic scene reconstruction from blurry video; (3) E2V [Rebecq et al., 2019]+D3DGS, a combined method that uses E2V to generate latent sharp images from events and then employs D3DGS for dynamic scene reconstruction; (4) GEM [Zhang et al., 2023]+D3DGS, a combined method that uses an eventguided deblur method (GEM) to deblur images first, then employs D3DGS for dynamic scene reconstruction.

4.3 Results

Quantitative Analysis on Synthetic Data. The results of the synthesis of novel views and input views are shown in Table 1. Compared to other methods, our quantitative results outperform them. D3DGS [Yang et al., 2024], lacking a module specifically designed for blurry image inputs, fails to reconstruct sharp dynamic scenes under blurry conditions. DyBluRF [Sun et al., 2024] leverages optical flow and depth information derived from blurry images but lacks precise spatial data, resulting in poor performance. E2V [Rebecq et al., 2019]+D3DGS and GEM [Zhang et al., 2023]+D3DGS suffer from multi-view inconsistency, preventing D3DGS from producing effective and high-quality results.

Qualitative Analysis on Synthetic Data. Figure 3 shows the visual comparison results of the input view and the novel view in synthetic scenes. Both D3DGS and DyBluRF lack precise spatial information as inputs, resulting in overall blurry images. E2V+D3DGS, due to significant image inconsistencies, deviates from the training images required by D3DGS, leading to more distorted renderings. Additionally, images generated by GEM exhibit certain inconsistencies, causing GEM+D3DGS to produce blurry details. Since these methods do not distinguish between coupled camera and object motion blur, they perform poorly in dynamic regions. Our method separates the two types of motion blur, resulting in good performance in both static and dynamic regions.

Qualitative Analysis onReal-worldData. We further validate the conclusions obtained from the synthetic datasets on the real-world datasets. Figure 4 shows the rendered images of several methods on the real-world datasets. The D3DGS rendered image is highly blurry. DyBluRF removes only a small amount of blur. The rendered images from E2V+D3DGS and GEM+D3DGS still contain some blurred details. Our method achieves better deblurring results, and the rendered images in the novel view synthesis task are also notably better since we separate static and dynamic regions for targeted deblurring and reconstruction.

Depth Visualization. Figure 5 shows the visualization of depth maps for both real-world and synthetic scenes. It can be observed that our method reconstructs dynamic scenes with well-defined geometric structures, making it beneficial for the novel view synthesis task.

5 Ablation Study

To validate the effectiveness of each component of our framework, we conduct ablation experiments as presented in Table 2 and Table 3.

Effect of EDI+event. In Table 2, when using only the event loss, the sparse spatial information of events is insufficient to effectively supervise D3DGS in modeling dynamic scenes. Utilizing latent sharp images from EDI with dense spatial information as supervision results in an average PSNR improvement of 19.68. However, these images exhibit some color distortion and noise. By incorporating the event loss alongside the latent sharp images supervision to jointly supervise D3DGS, a further increase in PSNR is observed.

Effect of optimizing poses. To achievemore accurate camera motion, we optimize the estimated poses, resulting in an average PSNR improvement of 1.33. We further validate that our method can optimize the camera motion trajectory more accurately by comparing it with the ground truth pose in synthetic scenes. The quantitative result is shown in Table 3.

Effect of motion mask. To separate object motion blur from multi-view information and camera motion blur, we use a mask to learn the dynamic components. This approach results in an average PSNR increase of 0.62 in Table 2.

6 Conclusion

In conclusion, we present EDyGS, a method for recovering a sharp dynamic scene representation from event streams and blurry monocular video. With EDyGS, we can deblur input views and generate sharp novel views of dynamic scenes. The core of our approach is the decoupling of two types of motion blur by leveraging multi-view information from the event stream, accounting for both camera and object motion blur. Experimental results demonstrate that EDyGS achieves the state-of-the-art performance in dynamic scene reconstruction with blurry images as inputs.

Limitations. Although EDI can effectively leverage event data to perform a coarse deblurring of blurry images, the latent sharp images after EDI still exhibit some gap compared to the sharp ground truth images.

Acknowledgments

This work was supported in part by the National Key Research and Development Program of China (No. 2024YDLN0005), in part by the National Natural Science Foundation of China (62376247, U20A20220, and 62334014), and in part by the grant from Key R&D Program of Zhejiang (2022C01048).

Contribution Statement

• ∗ These authors contributed equally. † Corresponding author.