Deep learning models for clinical time-series forecasting face significant challenges in generalizability when deployed across different clinical settings. Various choices in the deep learning pipeline can affect a model's out-of-distribution performance. This abstract introduces a comprehensive benchmarking framework to assess domain generalization and adaptation under varying distribution shifts in multivariate time-series forecasting using real-world data. Preliminary empirical results on the influence of backbone model architecture and a proxy metric, the robustness slope, are presented alongside remaining challenges.