Comprehensive Review of Neural Differential Equations for Time Series Analysis

1 Introduction

The exponential growth of time series data across diverse domains has necessitated more sophisticated analytical approaches, because of the complex nonlinear nature, irregular sampling, missing values, and continuous latent dynamics. Traditional approaches [Durbin and Koopman, 2012; Box et al., 2015] assume regular sampling and often linear relationships, limiting their applicability to real-world scenarios. While deep learning methods, including Recurrent Neural Network (RNN) [Rumelhart et al., 1986; Medsker and Jain, 1999], Long Short-term Memory (LSTM) [Hochreiter and Schmidhuber, 1997], Gated Recurrent Unit (GRU) [Chung et al., 2014], and Transformer [Vaswani et al., 2017] have shown promise in capturing nonlinearity, they remain constrained by discrete-time formulations [Che et al., 2018; Sun et al., 2020; Weerakody et al., 2021]

∗Corresponding authors

Neural Differential Equations (NDEs) emerged as a paradigm shift, offering a principled framework for continuous-time modeling with neural networks. The seminal work on Neural Ordinary Differential Equations (NODEs) [Chen et al., 2018] introduced continuous-time hidden state evolution, spawning numerous advances including Neural Controlled Differential Equations (NCDEs) [Kidger et al., 2020] incorporating external control paths, and Neural Stochastic Differential Equations (NSDEs) [Han et al., 2017; Tzen and Raginsky, 2019; Li et al., 2020; Oh et al., 2024] modeling uncertainty through stochasticity.

In this survey, we provide a comprehensive review of NDE methods in time series analysis, synthesizing advancements across several interconnected areas. We begin by exploring the base model families, including NODEs, NCDEs, and NSDEs, along with their extensions and variations. Building on this, we analyze the theoretical insights that underpin these models. We also discuss the implementation of NDE methods and analyze applications. This survey aims to provide a structured synthesis for researchers and practitioners, fostering a deeper understanding of NDEs and their potential in tackling complex time series problems.

2 Preliminaries

Time series modeling seeks to represent sequential data x = (x0, x1, . . . , xn), where each xi ∈ Rdx , as a continuous latent process z(t) ∈ Rdz over a time domain [0, T]. Figure 1 illustrates the conceptual differences between conventional time series modeling and NDE-based methods. While interpolation methods simply fit curves through observations (black dots) and RNNs (often using zero-order hold) operate in discrete time steps, NDEs aim to learn the underlying continuous dynamics, dx(t)/dt (or dz(t)/dt), parameterized by a neural network [Chen et al., 2018]. This parameterization provides NDEs with a flexible framework for handling challenges common in real-world time series, such as irregular sampling, missing observations, and capturing long-horizon dependencies. Furthermore, NDEs can offer greater memory efficiency compared to standard discretetime architectures [Chen et al., 2018; Kidger et al., 2020; Rubanova et al., 2019; Oh et al., 2024].

RNNs, including LSTMs [Hochreiter and Schmidhuber, 1997] and GRUs [Chung et al., 2014], operate in discrete time, updating hidden states sequentially based on observed inputs. Transformer-based models [Vaswani et al., 2017] offer an alternative by leveraging self-attention mechanisms, though they typically rely on fixed positional encodings and do not inherently capture continuous-time dynamics.

Various modifications have been proposed to introduce a notion of time awareness in RNNs [Che et al., 2018; Rajkomar et al., 2018]. One approach adjusts updates for time intervals, while another uses decay mechanisms to approximate continuous evolution. However, these heuristics do not always generalize across different settings and, in some cases, have been found to perform comparably to standard RNNs [Mozer et al., 2017; Rubanova et al., 2019].

2.1 Neural Ordinary Differential Equations

NODEs [Chen et al., 2018] model the latent state z(t) as:

where z(0) = h(x; θh), h : Rdx → Rdz is a neural network parameterized by θh, and f (t, z(t); θf ) approximates dz(t) dt . The vector field f is typically implemented using multi-layer perceptrons or more sophisticated architectures. One can view these approaches as an infinite-layer generalization of residual networks, allowing integration over time rather than discrete stacking of layers [Chen et al., 2018; Rubanova et al., 2019; Dupont et al., 2019; Kidger et al., 2020].

2.2 Neural Controlled Differential Equations

NCDEs [Kidger et al., 2020] extend NODEs by incorporating a control path X(t) for updating state over time:

where the integral is interpreted as a Riemann-Stieltjes integral, allowing for discontinuous paths. Piecewise-smooth control paths X(t) are typically constructed using natural cubic splines [Kidger et al., 2020] or Hermite cubic splines [Morrill et al., 2021a]. Furthermore, Morrill et al. (2021b) extend Neural CDEs by incorporating rough path theory for the generalized formulation.

2.3 Neural Stochastic Differential Equations

NSDEs [Han et al., 2017; Tzen and Raginsky, 2019; Li et al., 2020] incorporate stochasticity through:

where W(t) is a Wiener process (or Brownian motion), f (·; θf ) is the drift function, and g(·; θg ) is the diffusion function. The stochastic integral follows Itˆo or Stratonovich interpretations. While NODEs describe deterministic evolution through ordinary differential equations, NSDEs model uncertainty and noise through Brownian motion terms [Jia and Benson, 2019; Liu et al., 2019]. This extension enables more robust modeling of real-world phenomena where randomness plays a crucial role [Kidger et al., 2021b; Kidger et al., 2021c; Oh et al., 2024], each offering different approaches to balance stability, expressivity, and computational tractability.

3 Theoretical Considerations

3.1 Dynamical Systems Perspective

NDEs generalize the notion of discrete layers (as in RNNs or Transformers) to a continuous-time vector field. According to Eq. (1), the latent representation z(t) evolves:

while in NCDEs, the evolution is driven by a control path:

Here, X(t) is typically a spline or piecewise interpolation of observed input data x, as explained in Eq. (2).

Stochastic extensions, such as NSDEs, incorporate Brownian motion W(t) or jump processes to model noise or uncertainty in the underlying dynamics, as shown in Eq. (3):

where W(t) is a Wiener process. Formally, these methods place the latent trajectory z(t) in a phase space:

where h : Rdx → Rdz maps raw inputs x to the initial state. By solving the equations above, one obtains a continuous phase-space trajectory that can be analyzed through classical dynamical systems tools. This continuous viewpoint also enables seamless continuous-time modeling.

Invertible Neural Networks and FlowModels. In parallel with NDEs, explicit approaches like Neural Flows have also been proposed [Lu et al., 2018; Sonoda and Murata, 2019; Grathwohl et al., 2019; Massaroli et al., 2020a; Biloˇs et al., 2021]. Rather than parameterizing the rate of change dz(t)/dt, Neural Flows directly model the solution map:

where z(0) = h(x; θh) is the initial condition. This explicit representation bypasses the need for numerical ODE solvers, potentially offering faster computations or enhanced stability via invertibility constraints [Kobyzev et al., 2020; Papamakarios et al., 2021]. Such architectures align with normalizing-flow models, which are invertible by construction and allow for efficient likelihood evaluation.

Contrasting Implicit and Explicit Methods. The distinction between NODEs/NCDEs (which define dz/dt and rely on solvers) and Neural Flows (which specify z(t) directly) parallels the classic implicit versus explicit solution dichotomy in differential equations. Implicit, solver-based methods (like NODEs, NCDEs, NSDEs) can handle irregular inputs through fixed or adaptive step sizes and are more general in capturing unknown or complex dynamics. Explicit approaches, like Neural Flows [Biloˇs et al., 2021], may yield simpler training pipelines for stable transformations, yet can suffer from constraints on network forms and from instabilities if the sought-after closed-form solution is nonexistent or approximate [Oh et al., 2025].

In this survey, we limit our scope to NDEs that require implicit solvers, excluding flow-based models and other explicit parameterizations of continuous-time dynamics.

3.2 Universal Approximation Properties

NODEs. Chen et al. (2018) established that continuoustime models can be viewed as an infinite-depth limit of residual networks, enabling them to approximate diffeomorphic transformations under mild regularity conditions, as represented in Eq. (4). However, autonomous ODE flows may limit expressivity by constraining trajectories to remain diffeomorphic transformations of initial states, thus prohibiting intersecting paths in latent space. Dupont et al. (2019) addressed these limitations through augmented ODEs, which append an auxiliary variable a(t) to the state:

Augmenting the latent dimension effectively bypasses strict diffeomorphic constraints, expanding the range of representable data manifolds. Empirically, this technique improves reconstruction and expressivity in tasks where standard Neural ODEs struggle due to their trajectory overlap constraint.

NCDEs. While NODEs rely solely on a learned vector field and initial value, Kidger et al. (2020) showed that Neural CDEs embed a control path X(t) into the model as explained in Eq. (5): By allowing data updates to arrive as increments dX(t) at arbitrary times, NCDEs implement a form of continuous recursion, demonstrating universal approximation for continuous paths under sufficiently rich control signals. Extensions to rough paths [Morrill et al., 2021b] further expand coverage to long, irregular time series, handling signals too irregular for traditional ODE integrators.

NSDEs. Stochastic extensions achieve universal approximation for broad classes of continuous-time random processes by learning drift and diffusion terms. Han et al. (2017) and Tzen and Raginsky (2019) formalized how parameterizing f and g via neural networks lets these models approximate various stochastic phenomena, such as Geometric Brownian motion or Ornstein-Uhlenbeck processes. Subsequent works generalize to jump processes and specialized SDE families [Jia and Benson, 2019; Li et al., 2020; Oh et al., 2024; Zhang et al., 2024], underscoring the versatility of NSDEs in capturing noise-driven dynamics.

3.3 Existence and Uniqueness of Solutions

Deterministic NDEs. For NODEs as explained in Eq. (4), classic results such as the Picard-Lindel¨of theorem guarantees that a unique solution z(t) exists if the vector field f is Lipschitz continuous in z. Formally, if

for some constant L > 0, then the integral equation describing z(t) has a unique solution starting from z(0) = h(x) [Chen et al., 2018]. In practice, spectral normalization or weight clipping can enforce such Lipschitz constraints, improving both existence properties and training stability [Massaroli et al., 2020b; Rackauckas et al., 2020].

In NCDEs as shown in Eq. (5), existence and uniqueness in this setting hinge on Lipschitz-like conditions for f and on the path X(t) possessing sufficient regularity (e.g., bounded variation or spline-based interpolation) [Kidger et al., 2020]. If X(t) is discontinuous or highly irregular, rough path theory provides a generalized framework under which well-posedness can still be established [Morrill et al., 2021a].

Stochastic NDEs. NSDEs embed stochasticity via Eq. (6), existence and uniqueness then require both f and g to satisfy Lipschitz and linear-growth conditions in z [Tzen and Raginsky, 2019; Oh et al., 2024]. Common conditions for the existence and uniqueness of z are Lipschitz continuity and linear growth conditions for f and g such that if there exist constants L and C such that

and

then a unique strong solution z(t) exists for all finite t. Under these conditions, finite-time blow-up is prevented, ensuring the SDE is well-posed [Jia and Benson, 2019; Li et al., 2020]. Langevin-type SDEs, linear-noise SDEs, and geometric SDEs-variants of Neural SDEs-are also designed to satisfy these conditions, thereby ensuring the existence and uniqueness of their solutions [Oh et al., 2024].

3.4 Stability and Convergence

Deterministic Stability. NODEs and NCDEs achieve stability through Lipschitz constraints on vector fields [Haber and Ruthotto, 2017; Chen et al., 2018; Kidger et al., 2020]. For NODEs, spectral norm conditions on network weights keep trajectories bounded [Kidger et al., 2021a]. NCDEs maintain stability by bounding dX/dt [Morrill et al., 2021a].

Stochastic Stability. NSDEs follow dynamics dz(t) = f (z, t; θf ) dt + g(z, t; θg ) dW(t), where stability depends on drift f and diffusion g terms [Tzen and Raginsky, 2019; Kidger et al., 2021b]. Oh et al. (2024) shows the stochastic stability of the proposed Neural SDEs (Langevin-type SDEs, linear-noise SDEs, and geometric SDEs) under suitable regularity conditions and relates the findings to their robustness against distribution shifts.

Optimal Transport and Convergence. The convergence of continuous-time methods can be analyzed through optimal transport theory, with the Wasserstein metric offering a principled measure of stability and generalization [Villani and others, 2009; Peyr´e et al., 2019]. Formally, (Ω,F, P) specifies a probability space for random variables in Lp (Ω) whenever stochastic components arise. For an Rd -valued random variable X, its law L(X) belongs to P(Rd ). The Wasserstein distance of order p measures how far two distributions µ, ν ∈ P(Rd ) are in terms of the minimum cost coupling, such as:

Here, C(µ, ν) denotes the set of couplings whose marginals match µ and ν. In the context of NDE models, particularly in stochastic or high-dimensional regimes, this metric provides a valuable tool for analyzing model robustness, generalization, and convergence properties [Kidger, 2021; Ruiz-Balet and Zuazua, 2023; Oh et al., 2024].

Solver Convergence. NDEs rely on numerical integrators such as Euler, Runge-Kutta, and Dormand-Prince for deterministic models [Chen et al., 2018; Rubanova et al., 2019] and Euler-Maruyama, Milstein, or Reversible Heun for stochastic models [Kidger et al., 2021c; Tzen and Raginsky, 2019]. On the other hand, systems that are stiff, exhibit jumps, or involve high-dimensional dynamics often necessitate specialized implicit solvers to ensure stability and computational efficiency [Jia and Benson, 2019; Rackauckas et al., 2020; Kim et al., 2021].

4 Practical Implementation

This section expands on the theoretical foundations from Section 3, focusing on the practical aspects of training, regularization, and deployment of NDEs. While discrete-time models are widely studied, continuous-time models remain underexplored despite their advantages.

4.1 Optimization of NDE-basedModels

Adjoint Method. NDEs differ significantly from discrete methods in their reliance on numerical integration. During training, gradients with respect to model parameters must be backpropagated through an ODE, CDE, or SDE solver, which raises memory and stability challenges. Traditional backpropagation is conceptually straightforward but stores all intermediate states, leading to large memory usage in long sequences or high-dimensional latents.

The adjoint sensitivity method, introduced by Chen et al. (2018), is a pivotal technique for efficiently computing gradients in NDEs by solving an adjoint equation backward in time. This method addresses the challenge of high memory consumption in gradient-based optimization by reconstructing forward states on demand, reducing the memory complexity from O(N), where N is the sequence length, to approximately O(1). The adjoint state λ(t) evolves according to a differential equation,

where λ(T) = ∂L/∂z(T) initializes the backward integration. This approach enables the computation of gradients via

making it suitable for training models with terminal loss functions. However, the adjoint sensitivity method encounters numerical challenges, particularly in stiff or chaotic systems where reverse-time integration can amplify floating-point errors. Alternatively, checkpointing techniques [Gholami et al., 2019; Zhuang et al., 2020], store selected intermediate states during the forward pass, allowing localized recomputation during backward propagation.

Integral Loss Functions. Beyond terminal loss functions, NDEs have been extended to incorporate integral loss functions distributed across the entire depth domain S. In the context of optimal control [Pontryagin, 2018], the integral loss is defined as formulated by Massaroli et al. (2020a):

where the loss combines terminal contributions L(z(S)) with intermediate terms ℓ(t, z(t)). This formulation allows the latent state z(t) to evolve through a continuum of layers, guiding the model output toward the desired trajectory over the entire depth domain S. The adjoint dynamics for such integral loss functions are modified to include an additional term, -∂ℓ/∂z(t), accounting for the distributed loss in the backward gradients [Grathwohl et al., 2018; Massaroli et al., 2020a; Finlay et al., 2020b]. This approach enhances the control and flexibility of trajectory-level learning, enabling improved performance in tasks that require supervision over the entire temporal or spatial domain.

Backpropagation through time. For discrete methods like RNNs, backpropagation is a common optimization strategy, where the model's recurrence is unfolded into a sequence of differentiable operations. However, these discrete methods face vanishing or exploding gradients over long sequences [Hochreiter and Schmidhuber, 1997; Pascanu et al., 2013; Goodfellow et al., 2016]. NDE-based variants, such as GRUODE [De Brouwer et al., 2019], ODE-RNN [Rubanova et al., 2019], ODE-LSTM [Lechner and Hasani, 2020], mitigate these issues by replacing discrete recurrence with continuous evolution, enabling robust modeling of time series while retaining mechanisms like gating for temporal dependencies.

4.2 RegularizationMethods

Advanced Techniques for NDEs. While conventional regularization techniques are widely applicable to NDE-based methods, recent advances have introduced specialized approaches tailored to continuous dynamics. Regularization based on principles of optimal transport, proposed by Finlay et al.; Finlay et al. (2020b; 2020a), simplifies the dynamics of continuous normalizing flow models, thereby accelerating training. Kelly et al. (2020) analyzed equations with differential surrogates for the computational cost of standard solvers, leveraging higher-order derivatives of solution trajectories to reduce complexity.

Continuous-Time Modifications. Standard regularization techniques have further refined NDE-based methods; stochastic sampling of end times [Ghosh et al., 2020], continuous-time dropout [Liu et al., 2020], and temporal adaptive batch normalization [Zheng et al., 2024]. These methods form a comprehensive toolkit for advancing the performance and scalability of NDE-based models.

4.3 Numerical Solution of NDEs

Comparison with Discrete Architectures. Discrete-time methods, including variants of RNNs and Transformers, rely on layer-wise or self-attention updates without numeric integration. Although this design simplifies backpropagation and typically scales well, these models often struggle to accommodate irregular sampling unless augmented by masking or gating mechanisms [Chung et al., 2014]. RNN-based architectures can also experience vanishing or exploding gradients for lengthy sequences unless they incorporate techniques such as gradient clipping or gated units [Hochreiter and Schmidhuber, 1997; Pascanu et al., 2013]. Transformers replace recurrence with attention mechanisms but depend heavily on positional encodings and expansive parameter sets, which can limit their practicality for extensive time spans with variable sampling [Vaswani et al., 2017; Rae et al., 2019; Wen et al., 2023]. By contrast, continuous-time NDEs use solvers that naturally adapt to asynchronous events, though solver overhead and potential stiffness require careful solver selection and parameter tuning.

Fixed or Adaptive Solvers. NDE-based models rely on numerical integration to evolve their latent states z(t) from initial conditions to final outputs. A fixed-step solver, such as explicit Euler or a basic Runge-Kutta method, updates z(t) at uniform intervals and is computationally simpler but may face instability if the dynamics change abruptly or if the time step is too large [Chen et al., 2018]. Adaptivestep solvers, such as Dormand-Prince method, refine the step size ∆t in response to local error estimates, offering higher accuracy for stiff or highly non-uniform dynamics [Kidger et al., 2021c]; however, this adaptive nature can cause unpredictable run times in large-scale tasks [Rackauckas et al., 2020]. In the stochastic setting of NSDEs, solvers like Euler-Maruyama or Milstein handle the Brownian increments for the diffusion term, balancing efficiency and stability under well-defined noise [Tzen and Raginsky, 2019; Oh et al., 2024].

5 Comparison of NDE-basedMethods

Table 1 summarizes the core formulations, primary tasks, and benchmark datasets associated with various NDE-based methods. These methods can be grouped into three main categories-NODE, NCDE, and NSDE-highlighting their unique characteristics and application domains.

5.1 NODEMethods

Standard form of Neural ODE [Chen et al., 2018] introduced a novel approach to continuous-time modeling by parameterizing the hidden state dynamics with a neural network, allowing for the seamless handling of irregularly sampled time series data. Augmented ODE [Dupont et al., 2019] extend this framework by augmenting the state space with auxiliary variables, enabling the modeling of complex trajectories and avoiding trajectory overlap. ODE2VAE [Yildiz et al., 2019] combines NODEs using Bayesian Neural Network (BNN) with variational autoencoders to model latent dynamics in irregular time series by learning both the continuous-time evolution and the underlying latent space.

On the other hand, GRU-ODE [De Brouwer et al., 2019] adopts continuous gating mechanisms inspired by GRU cells, where the hidden state evolves dynamically based on time gaps. u(·), and g(·) are continuous counterparts of GRU. ODE-RNN [Rubanova et al., 2019] combines ODE-based continuous evolution with discrete updates at observation points using an RNN cell. Similar to that, ODE-LSTM [Lechner and Hasani, 2020] incorporates the memory mechanisms of LSTM into the ODE framework, evolving the hidden state between observations using ODE solvers and performing discrete LSTM updates at observation points.

5.2 NCDEMethods

Neural CDE [Kidger et al., 2020] models the latent state z(t) with a piecewise-smooth control path X(t). The Riemann-Stieltjes integral allows Neural CDEs to handle irregular or asynchronous time series data effectively, making them suitable for tasks like interpolation and forecasting. Neural Rough Differential Equations (Neural RDEs) [Morrill et al., 2021a] extend Neural CDEs by using the log-signature of the control path LogSig[ri,ri+1](X) with X : [t0 : tn] and certain intervals t0 ≤ ri < ri+1 ≤ tn, capturing higher-order variations. This enables Neural RDEs to handle rough paths, improving robustness for long and complex time series. Recently, Walker et al. (2024) further extended rough path theory to Log-NCDEs. These approaches leverage continuoustime dynamics, enabling flexible modeling of time series.

Recent works [Jhin et al., 2022; Jhin et al., 2023] emphasize training-based approaches for constructing control paths in NCDEs. For instance, Attentive Neural Controlled Differential Equation (ANCDE) [Jhin et al., 2024] extend this line of research by incorporating attention mechanisms for dynamic path construction. This is implemented using two coupled NCDEs: one computes attention values, a(t) = σ(h(t)) using an activation σ, and the other applies the attentionmodulated control path, Y (t), which is a combination of vector a(t) and X(t). Similarly, Li et al. (2023) propose Neural LAD (Latent dynamics), which incorporates an attentionbased transformation of the input path gradient gv dX dt for continuous-time latent dynamics. In addition, Neural LAD employs a seasonality-trend transformation hw(t) to capture periodic and slowly varying patterns, and uses ϕγ (z(t)) to learn structural relationships within the latent state.

DualDynamics [Oh et al., 2025] combines explicit and implicit mechanisms within a unified framework to improve the modeling of irregular time series data. By integrating Neural ODEs for explicit time evolution and learnable implicit updates for latent state transitions, DualDynamics achieves a balance between interpretability and robustness. These variations of NCDEs have been extensively applied to irregularly-sampled time series or data with missing observations, demonstrating their effectiveness in a variety of tasks including classification, imputation, and forecasting.

5.3 NSDEMethods

Neural SDEs, parameterized by neural networks for both drift and diffusion terms, provide a flexible framework for modeling complex stochastic processes, with foundational contributions from Han et al. (2017) and Tzen and Raginsky (2019). Neural Jump SDE [Jia and Benson, 2019] incorporates Poisson jump processes to model discontinuous state changes. Zhang et al. (2024) extended this to temporal point processes. While requiring specialized solvers, this approach effectively captures rare, discontinuous events in NSDEs.

Latent SDE [Li et al., 2020] extends the encodingdecoding paradigm of Latent ODE [Rubanova et al., 2019] by incorporating stochastic dynamics, allowing for more flexible modeling of uncertainty and variability. Latent SDE defines a prior process ˜ z(t) and an approximate posterior process z(t), both parameterized as Neural SDEs with hθ, hϕ, and σ. During the training, the Kullback-Leibler (KL) divergence measures and minimizes the prior-posterior discrepancy while maintaining uncertainty. On the other hand, Kidger et al. (2021b) extends Wasserstein generative adversarial network (GAN) to continuous-time stochastic processes, where both generator and discriminator are formulated as Neural SDEs.

Stable Neural SDEs, introduced in Oh et al. (2024), extend Neural SDEs to address stability and robustness in complex systems: Neural Langevin-type SDE (LSDE) with additive noise for invariant measures, Neural Linear Noise SDE (LNSDE) with multiplicative noise for scale-dependent fluctuations, and Neural Geometric SDE (GSDE) with statedependent noise for exponential dynamics. Note that, these formulations utilize the augmented states with control path, z(t) = ζ(t, z(t),X(t); θζ), in order to ensure stability for irregular time series analysis. Therefore, NSDEs extend NODEs by incorporating stochasticity, enabling robust modeling of complex, uncertain, and discontinuous dynamics.

5.4 Empirical Comparison

We present a consolidated summary of diverse empirical results from Oh et al. (2024) and Oh et al. (2025), encompassing tasks such as interpolation on PhysioNet Mortality, forecasting on MuJoCo, and classification on PhysioNet Sepsis.

We summarize interpolation experiments on the 2012 PhysioNet Mortality dataset [Silva et al., 2012], which contains 37 ICU variables recorded over 48 hours. Following Rubanova et al. (2019), timestamps were rounded to the nearest minute, resulting in up to 2880 intervals. As in Shukla and Marlin (2021), observation rates varied from 50% to 90%, using 8000 instances to predict the unobserved values. Benchmark models included RNN-VAE [Chen et al., 2018], mTAND-Full [Shukla and Marlin, 2021], L-ODERNN [Chen et al., 2018], L-ODE-ODE [Rubanova et al., 2019], DualDynamics [Oh et al., 2025], Latent SDE [Li et al., 2020], and variants of Neural SDEs [Oh et al., 2024]. Well-designed NDE methods outperform conventional models in interpolation performance, as shown in Table 2.

For forecasting and classification task, we consider a variety of NDE-related methods, including GRU-∆t [Choi et al., 2016], GRU-D [Che et al., 2018], GRU-ODE [De Brouwer et al., 2019], ODE-RNN [Rubanova et al., 2019], LatentODE [Rubanova et al., 2019], Augmented-ODE [Dupont et al., 2019], Attentive co-evolving neural ordinary differential equations (ACE-NODE) [Jhin et al., 2021], Neural CDE [Kidger et al., 2020], ANCDE [Jhin et al., 2024], EXtrapolation and InTerpolation-based model (EXIT) [Jhin et al., 2022], LEArnable Path-based model (LEAP) [Jhin et al., 2023], DualDynamics [Oh et al., 2025], and variants of Neural SDEs [Oh et al., 2024] including Neural LSDE, Neural LNSDE, and Neural GSDE.

Table 3 summarizes forecasting results, which used the MuJoCo dataset [Todorov et al., 2012] with the Hopper setup from the DeepMind control suite [Tassa et al., 2018]. Following Jhin et al.; Jhin et al. (2022; 2023), models were trained to forecast 10 future steps from 50 observed points under 30%, 50%, and 70%missingness. Similar to the previous results, NDE models demonstrate both superior prediction performance under varying missingness and greater memory efficiency in general, in MuJoCo forecasting.

Table 4 summarizes classification results, which used the PhysioNet Sepsis dataset [Reyna et al., 2019], containing

40,335 patients and 34 temporal variables. Following Kidger et al. (2020), we compared classification with and without observation intensity (OI), an index reflecting patient severity. Classification performance was measured by AUROC. On the PhysioNet Sepsis task, NDE-based models consistently achieve higher AUROC with significantly lower memory usage compared to conventional baselines.

Across diverse time series related tasks, NDE-based methods consistently achieve better performance compared to conventional models. Their advantages are particularly pronounced in dynamic prediction settings and irregularly sampled data, confirming their suitability for real-world time series modeling in a continuous-time manner.

6 Discussion and Future Directions

Computational Challenges. NDEs face scalability challenges with continuous-time solvers. Fixed-step integrators or adaptive integrators, while accurate, can lead to unpredictable computation times in stiff regions [Chen et al., 2018; Kim et al., 2021; Kidger, 2021]. Specialized techniques like parallel-in-time integration and GPU acceleration offer promising solutions [Rackauckas et al., 2020; Gholami et al., 2019]. Future research should focus on developing comprehensive integration schemes that can automatically balance computational efficiency with numerical accuracy, particularly for large-scale applications in real-time settings.

Theoretical Development. Current theoretical understanding requires expansion, particularly for non-stationary and noisy data. While stability guarantees exist for NODEs and NCDEs through Lipschitz constraints [Massaroli et al., 2020b], similar guarantees for NSDEs remain incomplete [Oh et al., 2024]. Drift-diffusion analysis under strong noise conditions needs further investigation [Tzen and Raginsky, 2019; Li et al., 2020]. Development of rigorous frameworks for analyzing convergence properties and error bounds in these stochastic settings would significantly advance both theoretical foundations and practical applications of NDEs.

Physics-Informed NDEs. Physics-Informed NDEs (or Physics-Informed Neural Networks) incorporate domain knowledge, such as Partial Differential Equations (PDEs) constraints and conservation laws, to align modeled dynamics with physical systems. Recent advances in physics-informed architectures have demonstrated remarkable success in capturing multi-scale phenomena and handling noisy measurements in complex physical systems [Rudy et al., 2017; Raissi et al., 2019; Cuomo et al., 2022]. By embedding structural priors, these approaches ensure interpretability and reliability in real-world tasks.

Hybrid Architectures. Hybrid architectures combine NDEs with models like graph neural networks or Transformers to capture complex time series patterns. Graph-based NDEs handle spatio-temporal data [Poli et al., 2019; Choi et al., 2022], while attention mechanisms address longrange dependencies and irregular sampling [Jhin et al., 2024; Li et al., 2024]. Furthermore, Oh et al. (2025) integrates explicit and implicit mechanisms for irregular time series analysis. Future research directions should focus on developing more efficient training algorithms for these hybrid architectures, and exploring applications in real-world domains where both temporal dynamics and structural relationships are important.

7 Conclusion

This paper has provided a focused and comprehensive survey of NDEs, specifically for time series analysis. While existing literature includes reviews on broader topics or specific niches within this domain, a systematic consolidation and comparative analysis of the core architectural families and their variants, tailored to the unique challenges of temporal data, has been largely absent. Our work addresses this gap, offering a structured synthesis of advancements and outlining future research directions in this new paradigm.

NDEs represent a significant advancement in time series modeling, offering a principled approach to handling continuous dynamics. Through various formulations, including NODEs, NCDEs, and NSDEs, they provide flexible frameworks for complex temporal data, including cases with irregular sampling or missing data. While challenges remain in computational efficiency and theoretical understanding, ongoing developments in solver techniques, stability analysis, and hybrid architectures continue to enhance their capabilities. Beyond current applications in classification, interpolation, and forecasting, NDEs show potential in anomaly detection, reinforcement learning, andmulti-agent systems. As the field evolves, NDEs are increasingly positioned to address sophisticated time series challenges across diverse applications.

Acknowledgments

This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (RS2024-00407852); the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health and Welfare, Republic of Korea (HI19C1095); the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (RS-2025-00563597); the Ministry of Trade, Industry and Energy (MOTIE) and Korea Institute for Advancement of Technology (KIAT) through the International Cooperative R&D program (No.P0025828); and the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (RS-2024-00443780, Development of Foundation Models for Bioelectrical Signal Data and Validation of Their Clinical Applications: A Noise-and-Variability Robust, Generalizable Self-Supervised Learning Approach).