Endo-GSG: Endoscopic Gaussian Splatting with Geometry-Awareness for Dynamic Tissue Reconstruction via Single-View Monocular Knowledge

1 Introduction

The rapid advancement of Artificial Intelligence (AI) is driving a global technological revolution [Ahmed et al., 2022], with surgical robotics emerging as a fast-growing domain. Modern surgical robots are increasingly capable of precise task execution [Kim et al., 2022], automated manipulation [Yip et al., 2023], and human-robot collaboration [FoschVillaronga et al., 2021] during specific surgical phases. To realize more intelligent image-guided operations and skill training, dynamic 3D surgical scene perception remains a longstanding challenge, particularly in providing robots with precise 3D awareness within the confined abdominal cavity.

*Corresponding author.

3D Gaussian Splatting (3D-GS) has recently shown remarkable potential for high-fidelity view synthesis by representing scenes as collections of Gaussian ellipsoids optimized via multi-view image supervision [Kerbl et al., 2023]. Its rasterization-based rendering pipeline enables realtime performance with high visual quality, making it wellsuited for task guidance in robotic surgery. Nevertheless, 3D-GS relies solely on photometric supervision, resulting in poor 3D reconstruction quality. Although incorporating depth supervision offers partial improvements, the rendered surface-formed by aggregating multiple Gaussian ellipsoids-still produces inaccurate point clouds and exhibits noticeable floating artifacts. While dense-view supervision helps mitigate this issue, the inherently sparse-view setting in surgical applications fundamentally constrains 3D perception in dynamic scenes.

Recent studies have explored physics-integrated Gaussian representations [Xie et al., 2023] [Zhang et al., 2024] [Yang et al., 2024a] [Zhong et al., 2024], in which each Gaussian ellipsoid is regarded as an infinitesimal element of a physical object. Although these methods enable realistic simulation of soft tissue mechanics, they are still based on the inaccurate geometry produced by classical 3D-GS pipelines. Misaligned Gaussian ellipsoids lead to unrealistic physical responses, limiting their applicability to precise tissue modeling in surgical scenarios.

To resolve these problems, several state-of-the-art (SOTA) works have introduced surface regularization [Gu´edon and Lepetit, 2024], 2D-GS-based optimization [Huang et al., 2024], and Gaussian surfels [Dai et al., 2024] to improve 3DGS reconstruction quality. However, these methods sacrifice fine-grained texture details and remain constrained by the inherent flaws of reconstruction methods. Although 3D-GSR [Lyu et al., 2024] integrates SDF fields and loose coupling to align Gaussian ellipsoids with surfaces, it still struggles to generalize to dynamic scenes. Endo-GS [Zhu et al., 2024] further introduces depth-fused pseudo-SDF supervision, yet lacks explicit geometric constraints on Gaussian opacity, limiting surface accuracy. A comparative analysis of our method and current SOTA methods is provided in Table 1.

To address these challenges, we propose Endo-GSG, a geometry-aware dynamic tissue reconstruction method based on 3D Gaussian Splatting. We decouple dynamic surgical scenes using a spatiotemporal plane, separating them into a static frame and corresponding Gaussian deformations, mapping all Gaussian kernel attributes consistently across the entire dynamic sequence. We design an SDF-Net integrated with hash encoding to output the distance of each Gaussian kernel to the surface and further propose three geometryinformed loss functions to enforce SDF-depth consistency, proximity to surface projections, and alignment with surface normals, ensuring accurate surface fitting and welldistributed Gaussian ellipsoids. Finally, instrument segmentation masks are incorporated to guide occlusion-aware optimization, effectively removing surgical tool interference and producing clean laparoscopic scene renderings.

In this work, we propose a geometry-informed regularization method to achieve surface alignment of Gaussian ellipsoids in single-view dynamic scenes. Our main contributions can be summarized as follows:

• We construct a spatiotemporal cross-plane dynamic representation field and propose an SDF-Net integrated with hash encoding, which is tightly coupled with the Gaussian splatting framework, to achieve dynamic reconstruction and precise implicit surface representation.
• We propose geometry-informed loss functions to regularize the Gaussian density on surfaces and spatial distribution of gaussian primitives, thereby achieving highfidelity reconstructions of 3D surfaces.
• We integrate a temporal instrument segmentation mechanism into our optimization scheme, which can remove

instrument occlusions and reveal covered laparoscopic scenes with globally consistent geometry awareness.

2 RelatedWork

2.1 Dynamic Tissue Reconstruction

Neural radiance fields (NeRF) [Mildenhall et al., 2021] provide an implicit, continuous scene representation for reconstruction and novel view synthesis via volumetric rendering. For dynamic modeling, D-NeRF [Pumarola et al., 2021] introduces a deformation field, while NeuS [Wang et al., 2021] integrates SDF-based surface priors to enable higher-quality geometry extraction. In surgical scenes, Endo-NeRF [Wang et al., 2022] leverages ray masking and depth supervision for single-view dynamic tissue reconstruction, and subsequent works further improve mesh extraction and robustness to unreliable depth [Zha et al., 2023; Lu et al., 2024]. Despite strong performance, NeRF-style methods remain computationally expensive due to dense sampling along rays, limiting practical efficiency in training and rendering.

2.2 Gaussian Splatting

With its efficient discretized representation and rendering capabilities, 3D-GS [Kerbl et al., 2023] has rapidly advanced as a compelling alternative to neural radiance fields. Subsequent works [Oh et al., 2025] [Liu et al., 2024a] [Zhu et al., 2024] [Yang et al., 2024b] extended 3D-GS to support 3D/4D generation and dynamic scene modeling. By representing a scene as a collection of Gaussian ellipsoids, GS adjusts the color, opacity, and shape of each ellipsoid to faithfully capture scene textures. However, due to the fundamental differences between Gaussian ellipsoids and point clouds, there remains a gap in accurately representing scene geometry. Even when Gaussian ellipsoids are aligned along surfaces, they often fail to represent the true object geometry. This discrepancy leads to significant challenges when converting Gaussian-based representations to widely used discrete formats, such as meshes, thereby hindering downstream editing tasks.

Moreover, with the rise of world models [GenesisEmbodied-AI, 2024], 3D reconstruction is expected to move beyond texture and geometry, toward embedding physical properties that support physically plausible manipulation and interaction. Thereby, several works [Xie et al., 2023] [Zhang et al., 2024] [Zhong et al., 2024] [Zong et al., 2023] were introduced to embed physical attributes into the Gaussian representation. When utilized in surgical scenes, SimEndoGS [Yang et al., 2024a] showcased the potential of physicsintegrated Gaussians in surgical applications by extending physical simulation and rendering into the endoscopic domain. However, 3D Gaussians inherently struggle to represent object surfaces accurately due to the flawed assumption that "Gaussian primitives represent the volumetric radiance rather than the explicit surface geometry", making surfacealigned modeling a critical challenge.

To address this, SuGaR [Gu´edon and Lepetit, 2024] proposed limiting the overlap of Gaussians during 3D-GS optimization. Subsequently, a mesh can be extracted using Poisson reconstruction [Kazhdan and Hoppe, 2013] , which is then refined by binding flat Gaussians distributed on its surface and optimizing for accuracy. 2D-GS [Huang et al., 2024] treats points as 2D surfaces and uses TSDF fusion to produce the final mesh. 3D-GSR [Lyu et al., 2024] introduced the use of an SDF field with a loosely coupled strategy to regularize the position and orientation of Gaussian ellipsoids. While these methods have shown effectiveness, they are not applicable to dynamic scene surface reconstruction, especially in medical scenarios with limited viewpoints. Since the above methods heavily rely on multi-view information to drive surface learning, they tend to produce unsatisfactory reconstruction results in viewpoint-limited and unstructured medical scenarios, where multi-view data is unavailable and surgical instruments often occlude the scene.

In our framework, each Gaussian ellipsoid is modeled as a physical micro-element representing matter in the real world. Physical interactions among these elements are simulated through the Material Point Method (MPM) [Jiang et al., 2016], enabling accurate modeling of deformation and contact dynamics. Concurrently, a rasterization-based pipeline facilitates real-time rendering of these interactions. This integration of physically grounded simulation and efficient rendering holds promise for advancing surgical robotics and other downstream applications that demand precise, real-time visuo-physical modeling.

3 ProposedMethod

Figure 2 shows an overview of our proposed Endo-GSG. We define input as {(Ii,Di,Mi,Pi, τi)}Ti=1 , where, Ii represents the RGB image of the i-th frame, Di is the corresponding depth map estimated from monocular endoscopy, and Mi is a binary mask that filters out surgical tools. The projection matrix Pi encodes the transformation from world coordinates to image-plane coordinates, which remains fixed across all frames under the single-view setting. The timestamp τi denotes the temporal index of each frame and is normalized as i/T . To optimize the Gaussians more effectively, we use the tuple (Ii,Di,Mi,Pi, τi)i=1 to regularize the initial point cloud from the first frame, which serves as the initialization for Gaussian modeling. The remainder of this section introduces detailed steps to achieve our tissue surface reconstruction pipeline.

3.1 Preliminaries

3D-GS [Kerbl et al., 2023] models the scene as learnable 3D Gaussians (µi,Σi, αi, ci), which are projected to screen space and accumulated by front-to-back alpha compositing. The rendered color and depth at pixel u share the same compositing weights:

where di is the depth of µi. For completeness, we briefly summarize the standard 3D-GS rendering formulation; detailed derivations and implementation specifics can be found in [Kerbl et al., 2023].

3.2 3D-GS for Dynamic Soft Tissues

To model dynamic soft tissue deformation induced by instrument traction, we represent the scene as a four-dimensional volume where a deformation network is introduced to capture temporal tissue motion. Inspired by HexPlane [Cao and Johnson, 2023], we employ six orthogonal spatio-temporal feature planes to jointly encode spatial and temporal information: three spatial planes (XY , XZ, Y Z) and three spatio-temporal planes (XT, Y T, ZT), denoted as P = {(x, y), (y, z), (x, z), (x, τ), (y, τ), (z, τ)}. Each plane has a resolution of N1 × N2 and feature tensor F ∈ Rd×N1×N2 , where d is the hidden feature dimension. For each Gaussian, we use its center µ(x, y, z) and timestamp τ to bilinearly interpolate B(·) on the four nearest voxels of each plane, aggregating the results as the final voxel feature:

This feature is passed through a single-layer MLP to predict the temporal variations of Gaussian attributes: {(∆µ,∆s,∆r,∆sh)}, which are treated as time-dependent terms. The deformed attributes are given by {(µ + ∆µ, s +∆s, r +∆r, sh +∆sh)}, and we denote them as {(˜µ, ˜ s, ˜ r, ˜ sh)} for brevity in later formulations.

3.3 Coupling Between Gaussian and SDF

3D-GS synthesizes novel views by directly rasterizing all Gaussian ellipsoids. However, this strategy lacks the ability to capture meaningful surface geometry. In single-viewpoint scenarios, the absence of multi-view constraints often leads to a disordered spatial distribution of Gaussians, despite the rendered images appearing visually plausible. To overcome this limitation, we integrate a neural implicit Signed Distance Function (SDF) field into the pipeline, which serves to regularize the spatial arrangement of Gaussians and ensure their alignment with the true tissue surface. The neural SDF is defined as a function f : R3 → R, which maps any 3D point in R3 to its signed distance from the target surface. This function is realized using a multi-resolution hash grid for efficient spatial encoding, coupled with a 2-layer MLP for regression. The underlying surface of the scene, denoted by S, can be extracted as the zero level set of the SDF and is formally defined as:

Next, we establish the coupling between the SDF field and the Gaussian ellipsoids. To enable effective surface learning for the SDF network, NeuS [Wang et al., 2021] proposes converting the SDF values into densities of ray-sampled points in a neural radiance field, which are then integrated into a volumetric rendering framework for supervision. However, in single-view scenarios, the rendered color is not a reliable surface supervision signal due to its ambiguity. To address this, we evaluate the SDF at the Gaussian centers ˜µ, i.e., f(˜µ) → SDF values, then convert these values into Gaussian densities using a differentiable bell-shaped function, and finally render the depth using Equation (5). The rendered depth D′ (u) is then compared with the ground-truth depth D(u) to provide meaningful surface learning supervision. Specifically, the bell-shaped function Φβ(f(˜ µ)) is controlled by a learnable parameter β, which is:

Through this function, we obtain the density of each Gaussian and encourage those closer to the zero level set to contribute more significantly to the overall density. This aligns with our core principle that Gaussians should be concentrated near the object surface. Moreover, the SDF field can be updated under the supervision of depth loss:

While depth supervision facilitates Gaussian density attenuation near surface regions, the inherent structural ambiguity in monocular settings hinders stable optimization. To address this, we introduce pseudo ground-truth SDF values derived from depth maps to supervise the SDF network, ensuring SDF values are aligned with depth, which is:

where, Spred is the normalized predicted SDF value, while Starget is the target SDF direction computed as the normalized depth difference between the ground truth depth map mapz and the Gaussian center's depth zcam in the camera coordinate system. The predicted SDF value Spred is trained to approximate Starget, encouraging SDF consistency in depth alignment. This strategy effectively regularizes the SDF distribution under limited multi-view information in laparoscopic scenes. Combined with Equation (5), it suppresses the density of Gaussians far from the surface.

To further mitigate floating artifacts caused by off-surface Gaussians and ensure smooth distribution along the surface [Lyu et al., 2024], we leverage the SDF field to both attract Gaussian centers toward the surface and adjust their orientations, ensuring that all ellipsoids are distributed in a scale-like pattern along the zero level set. Specifically, we perform a first-order Taylor expansion of the SDF field at the Gaussian center ˜µ to calculate the nearest projection point xg onto the surface:

where f(˜µ) denotes the SDF value and ∇f(˜µ) is the gradient of the SDF at ˜µ, normalized to a unit vector. To ensure that all Gaussians lie exactly on the surface, we aim to enforce xproj = xg. During optimization, to encourage directional gradient flow, we define a surface projection loss that effectively pulls Gaussian centers toward the surface:

Due to their anisotropic nature with distinct long and short axes, neglecting orientation consistency may result in slender ellipsoids generating fuzzy artifacts on the surface. Therefore, to ensure that each Gaussian ellipsoid is aligned with the local surface normal, we define an orientation loss by enforcing the shortest principal axis of the ellipsoid to be parallel to the SDF gradient ∇f(˜µ). Specifically, given the rotation matrix R ∈ R3×3 derived from the quaternion representation, and the scale vector s = (sx, sy, sz), we identify the local axis corresponding to the smallest scale:

where ek denotes the k-th canonical basis vector. Let n = ∇f(˜µ)/∥∇f(˜µ)∥ be the unit surface normal at the Gaussian center. To avoid the singularity that may caused by directly enforcing the dot product between vshort and n to be 1 during optimization, we define the orientation loss as:

3.4 Optimization

Occlusions from surgical instruments pose a major challenge in tissue reconstruction. To mitigate this, we incorporate binary masks to identify occluded pixels and introduce a spatiotemporal importance sampling strategy that emphasizes frequently occluded regions. Let {Mi}Ti=1 , where Mi ∈ {0, 1}, denote the binary occlusion masks across frames, with 0 for visible tissue and 1 for occluded areas. We define the corresponding importance maps {V i}Ti=1 as:

where ⊙ denotes element-wise multiplication. The term 1 -Mi restricts optimization to visible regions, while the normalized sum encodes the temporal occlusion frequency, prioritizing tissue regions that are frequently hidden. The weighting strength is controlled by the hyperparameter γ.

To train the 3D-GS model, we begin by optimizing the Gaussian attributes via minimizing both photometric and depth losses. The color loss, denoted as Lc, measures the difference between the rendered image and the ground-truth image. To suppress the influence of surgical instruments, we multiply both the photometric and depth losses by an importance map V i, which is derived from the instrument masks. Specifically, the photometric and depth losses are denoted as:

where Ii is the ground-truth RGB image, ˆ Ii is the rendered image, and ⊙ denotes element-wise multiplication. The depth loss Ld is jointly optimized with the Eikonal loss Leikonal to encourage the SDF field to learn accurate surface geometry. The Eikonal loss enforces the gradient of the SDF to remain unit-norm and is defined as:

Finally, to align the Gaussian ellipsoids precisely with the surface, we incorporate both the projection loss Lproj and orientation consistency loss Lori. The total loss for the framework is defined as:

4 Experiments

4.1 Implementation Details

We evaluate our method on two public datasets [Wang et al., 2022]: Cutting, with tissue incisions by surgical forceps, and Pulling, with upward tissue traction. Both monocular sequences contain instrument occlusions and complex nonrigid deformations. Following the dataset setup [Wang et al., 2022], stereo-derived depth maps Di and manually annotated binary masks Mi are used as pseudo supervision. The model is trained for 30,000 iterations on an RTX 4090 GPU with initial weights λ1 = 1, λ2 = 0.5, λ3 = 0.1, λ4 = 1, λ5 = 0.5, and λ6 = 0.1.

4.2 Evaluation Analysis

We comprehensively compare our method with state-of-theart approaches, including NeRF [Mildenhall et al., 2021], Endo-NeRF [Wang et al., 2022], Endo-Surf [Zha et al., 2023], 3D-GS [Kerbl et al., 2023], D-3D-GS [Yang et al., 2024b], and Endo-GS [Zhu et al., 2024], in terms of image rendering and 3D reconstruction. For rendering evaluation, we report PSNR, SSIM [Wang et al., 2004], and LPIPS [Zhang et al., 2018] to quantify visual fidelity, and measure inference speed using FPS. For Gaussian-based methods, we also track the number of 3D Gaussians to monitor splitting and attenuation, which affect rendering efficiency and model compactness. For 3D reconstruction, we assess the alignment between predicted Gaussian point clouds and RGB-D-derived ground-truth surfaces using Chamfer Distance [Fan et al., 2017], point-to-surface (P2S) metrics [Ma et al., 2020], and F-score [Tatarchenko et al., 2017]. Chamfer Distance measures global geometric accuracy; P2S metrics (mean, median, 95th percentile) evaluate fidelity and robustness; and F-score captures surface-level correspondence through precision and recall. Table 2 and Table 3 present the quantitative results on rendering and reconstruction respectively. Figure 4 shows qualitative comparisons against EndoGS, while Figure 3 visualizes multi-view Gaussian-based surface reconstructions, with the rendering outcomes under dynamic surgical scenes.

Our method, leveraging a dynamic representation network and a hash-table encoded SDF-Net, achieves superior rendering quality over existing SOTA methods, including Endo-GS [Zhu et al., 2024]. This advantage mainly stems from the strong fitting capability of hash encoding and the improved geometric interpretability of SDF-based surface modeling. With SDA, orientation, and projection losses, our method aligns Gaussians with the underlying tissue surface while preserving texture details. Although Endo-GS [Zhu et al., 2024] was trained for 60,000 iterations, our model achieves competitive or better performance with only 30,000 iterations, demonstrating faster convergence and lower training cost, as shown in Figure 5. Moreover, instrument segmentation masks enable dynamic instrument removal, and aligned comparisons show rendering quality comparable to or better than Endo-GS. In contrast, D-3D-GS [Yang et al., 2024b] achieves fast rendering speed with its lightweight dynamic network, but lacks depth guidance and effective geometric constraints, resulting in less consistent geometry and unstable surface reconstruction. 3D-GS [Kerbl et al., 2023], originally designed for static scenes, struggles with highly dynamic tissue deformations, leading to degraded reconstruction performance. NeRF-based methods, limited by ray-tracing-based rendering, consistently underperform across all metrics.

Our method focuses on tightly adhering Gaussian ellipsoids to the surface, which may sacrifice rendering some high-frequency texture details. To capture finer details, our model generates additional Gaussian ellipsoids near the surface, resulting in an increased number of points, which causes slower rendering speed. Nevertheless, after optimization, our method still satisfies the commonly adopted real-time criterion in laparoscopic video analysis, i.e., ≥30 FPS [Liu et al., 2024b]. In 3D reconstruction accuracy (Table 3), our method significantly outperforms D-3D-GS [Yang et al., 2024b] and Endo-GS [Zhu et al., 2024], owing to our specialized SDFNet pipeline and joint loss function, which optimize Gaussian sphere representation for tissue surfaces, leading to superior accuracy in 3D structural reconstruction. D-3D-GS [Yang et al., 2024b] lacks depth supervision, leading to more chaotic and weaker 3D structural reconstruction compared to EndoGS [Zhu et al., 2024].

Qualitative comparisons in Figure 3(a) show that our method tightly adheres Gaussian ellipsoids to the surface, reducing floating artifacts. Figures 3(b1)-(b2) demonstrate more accurate dynamic structure perception than SOTA methods under occlusion and sparse visual feedback, benefiting surgical navigation and physics-based simulation. Figure 4 further shows that our model handles instrument-induced occlusions effectively, recovering tissue textures and producing more accurate depth maps than Endo-GS [Zhu et al., 2024].

4.3 Ablation Study

We performed an ablation study comparing "Ours w/o SDA" (using only proj and ori losses) with the "Complete model" incorporating all losses. As shown in Table 2, adding SDA loss enables SDF-Net to better capture surface information, improving rendering quality. Figure 3(a) and Table 3 show that "Ours w/o SDA" exhibits artifacts and misalignment, while SDA loss effectively aligns the Gaussian distribution with the true surface geometry. Table 4 further demonstrates that the complete model achieves optimal performance in both rendering and 3D structure.

5 Conclusion

This work proposed Endo-GSG, a dynamic 3D reconstruction framework that couples 3D-GS with an SDF field to improve Gaussian distributions for tissue structures in unstructured surgical scenes. With a spatiotemporal deformation field, implicit SDF alignment, and geometry-informed regularization losses, Endo-GSG captures tissue dynamics and aligns Gaussians with the underlying surface under sparse-view endoscopic conditions. Experiments on complex real-world scenarios demonstrate superior rendering quality and surface accuracy over SOTA methods. Endo-GSG provides a surfacefaithful representation for postoperative surgical training and physics-integrated GS with MPM in future surgical intelligence systems, while extreme topology-breaking events and broader clinical validation remain future work.

Acknowledgements

This work is supported by the National Natural Science Foundation of China (62573310) and (62502445), in part by the Open Research Fund of The State Key Laboratory of Multimodal Artificial Intelligence Systems, in part by the Innovation and Entrepreneurship Leading Talent Program of Suzhou City (ZXL2023156), and in part by the Young Elite Scientists Sponsorship Program by CAST (2023QNRC001). The authors would also like to thank Suzhou Jupiter Semi Co., Ltd. for its support of this work.

Contribution Statement

Chao He and KuangJi Chen contributed equally to this work and are considered co-first authors. Bruce X.B. Yu and Bo Lu are the corresponding authors.