DeepShade: Enable Shade Simulation by Text-conditioned Image Generation

1 Introduction

Extreme weather is causing an increasing number of deaths worldwide, with heatwaves being a major contributing factor. According to a report [Ritchie, 2024], the frequency and intensity of extreme heat events have surged over the past two decades, with more than 178,700 deaths occurring annually (average from 2000 to 2019) as a direct result of high temperatures [Shanshan Li, 2021]. Research from the World Health Organization highlights that extreme heat is now one of the leading causes of weather-related deaths [NIEHS, 2022], disproportionately affecting vulnerable populations such as the elderly and those working or staying in outdoor areas with limited access to cooling infrastructure. This trend underscores the urgent need for adaptive measures, including heatresilient urban design and shade-aware route-planning methods [Ma, 2018], to mitigate the public health impact of rising temperatures globally.

Since shade acts as a natural shelter that reduces direct exposure to solar radiation, understanding how shade changes in real-time is crucial for preparing outdoor activities and aiding urban planning in establishing artificial shelters in areas lacking natural shade. As introduced in [Da et al., 2024a], by identifying shaded areas and integrating this information into route-planning systems, individuals can make more comfortable travel choices, reducing their risk of heat-related illnesses. However, the existing research faces significant limitations: First, shade analysis using urban simulations is only on static maps, which lacks good generalizability. Second, most of the methods are localized, relying on resourceintensive LiDAR data; they also lack scalability across different or more extensive areas. Third, they are unable to capture the real-time shade dynamics, limiting their utility in intelligent routing. These limitations impede the development of accurate shade modeling and timely planning, affecting the practical impact of existing studies.

To tackle the above challenges, a more adaptive and scalable approach to dynamically model shade variations is required. Given the success of Generative AI [Da et al., 2024b] (particularly diffusion models), in capturing spatial-temporal patterns in urban scene synthesis [Tang et al., 2024] and environment simulation [Da et al., 2025b], they present a promising direction for overcoming these challenges by generating shades for satellite images. However, to effectively leverage this method, a well-structured dataset, that accurately aligns ground-truth shade-variance with its satellite-image level geographic information, is necessary. Such a dataset is left blank in the current research domain.

In this paper, we first developed a rigorous and systematic pipeline to construct a comprehensive dataset for training shade-generation models. The dataset is designed to encompass three critical dimensions while adhering to a unified standard. The three dimensions include: (1) Geographical Diversity, covering a wide range of continents with varying latitudinal and longitudinal distributions; (2) Urban Layout Variability, capturing diverse urban configurations such as dense high-rise building places and sparse flat areas; and (3) Traffic Rule Variation, accounting for differences in driving orientations, including left-hand and right-hand traffic systems. The unified standard is that the cities must suffer from significant heat events as defined by the World Meteorological Organization (WMO), which guarantees the dataset focuses on regions with high temperatures and their associated impacts on shade dynamics are most pronounced. We secondly, designed a novel contrastive learning-based diffusion model approach, with a fine-granularity edge conditional module, that learns the shade variations based on the skeleton representations of corresponding satellite images. The model effectively maps shade dynamics to text prompts containing temporal and geographic information, allowing it to generate realistic shade predictions for arbitrary times of day or specific solar angles. This solution is unique for its ability to train on a limited satellite image dataset while generalizing to unseen buildings, provided a satellite image is available. This relaxed the requirements for detailed building height to conduct simulation and ensures adaptability and robustness across diverse urban settings.

In conclusion, this work contributes to advancing shadegeneration research by addressing critical challenges in data preparation, model design, and real-world application. First, we developed a comprehensive, globally representative dataset, meticulously crafted to align satellite imagery with the dataset. Second, we introduced a text-conditioned image generation model leveraging edge conditioning and contrastive learning, enabling accurate and generalizable shade predictions. Third, we conducted extensive experiments across diverse cities, showcasing the model's robustness in handling varying landscapes, urban layouts, and geospatial features. Finally, we demonstrated a practical application of the model in shaded route planning, where it generates shade maps for different times of the day based on satellite images and textual prompts. These contributions collectively provide a robust framework for urban planning and heat mitigation strategies.

2 RelatedWork

2.1 Building Shade Analysis

There are works investigating the shade changes from buildings and how they affect people's living. A recent study [Park et al., 2023], has quantified the cumulative cooling effects in urban environments, particularly emphasizing the benefit of high-resolution thermal imagery in hot, arid climates like Tempe, Arizona. Another recent work also used highresolution data and models like SOLWEIG [Lindberg et al., 2008] to map the thermal impact of shade at fine scales, offering valuable insights for urban planners. These maps are used in initiatives such as "cool corridor" planning [Buo et al., 2023] and [Wu et al., 2020], which aims to reduce heat exposure for pedestrians by optimizing sidewalk shade coverage. Such analysis highlights the focus on reducing urban heat in the modern environment [Buo et al., 2023]. However, current research typically focuses on specific urban locations and relies heavily on high-resolution LiDAR data, which, although accurate, is costly and complex to collect on a city-wide large scale. This has led to our work in developing universally applicable shade analysis methods that could comprehensively assess shade impacts across diverse cities.

2.2 City Simulation

There exist studies [Kamath et al., 2024] that utilized highresolution datasets, like the University of Texas Global Building Heights for Urban Studies (UT-GLOBUS [Li et al., 2020]). UT-GLOBUS combines data from sources such as LiDAR, ICESat-2 [Abdalati et al., 2010], and Global EcosystemDynamics Investigation (GEDI) [Schneider et al., 2020], augmented by machine learning models, to produce comprehensive canopy information for urban studies. Existing simulations like those conducted using GIS in the Adab neighborhood of Sanandaj [Beheshtifar and Abdollahi, 2023] demonstrate the complexity of modeling urban shade in dense areas with high-rise structures. The study used 3D GIS data to simulate shade across different seasons, highlighting the intense shading experienced by 75% of buildings in winter. However, all of the above datasets and simulations remain limited to specific locations and lack the scalability to encompass upto-date, diverse urban landscapes or support universal shade simulations across varying city layouts and climates.

2.3 Generative Models

Generative models [Da et al., 2025a] such as diffusion models [Croitoru et al., 2023] (used in image synthesis [Dhariwal and Nichol, 2021] and refinement [Du et al., 2023]), ControlNet [Zhao et al., 2024], and StyleGAN [Abdal et al., 2019] excel at producing detailed synthetic data. ControlNet uses external signals for spatial guidance, and StyleGAN enables high-resolution, realistic image generation with fine-grained style control [Azadi et al., 2018]. However, these models have yet to be applied to urban analysis (e.g., shade generation) due to challenges in acquiring accurately calibrated, geo-referenced shade data that matches satellite imagery and models sunlight angles precisely. In this work, we propose a novel framework to address this by constructing a comprehensive, calibrated shade dataset with ground truth shades.

3 Dataset Creation

The archived geographical information might be out-of-date which hinders the feasibility of simulating shade changes, but it is possible to use accessible 'satellite image + generalizable models' to infer the shade areas.

In order to develop generalizable models, we construct a formal pipeline that creates a comprehensive dataset covering three dimensions: Geographical Diversity, Urban Layout Variability, and Traffic Rule Variation. This dataset aligns satellite images with Open Street Map (OSM) [Ramm et al., 2025] building information. The pipeline involves four major steps as shown in Figure 1, which illustrates the overall framework from input to the collected outcome.

First, Metadata Retrieval obtains all necessary metadata used for the 3-D simulation. To align with the research community and utilize the large open-source data, we adopt OSM data as the metadata for shade simulation. Moreover, based on the longitude and latitude information, we can extract the building's geographical information, including "property type, address, height, levels, shape, and geometry set of locations", this typically provides information such as a list of points that form a polygon of building's 3D skeleton.

Second, Simulation Setups. With the necessary data collected, we performed the simulation to capture the shade changes for our dataset creation. In addition, we use Blender [Hess, 2013] for large-scale city-wide shade simulation. Based on scientific rules, we set up a controller as the sun's movement engine, which can generate the sun's trajectory following the sun's solar declination or angle on any day of the year, and at any time of the day. Then, we rendered the area of interest (AOI) within the simulation platform and adjusted the scale to ensure it aligns with the prevalent maps. Here, we align images with Google map tile level 13.

Third, Data Collection. We collected four types of data: (1) skeleton snapshots, (2) shaded snapshots, (3) satellite images, and (4) text descriptions. These diverse data representations enable learning models to understand shade variations and their dependencies on environmental factors.

The shaded snapshot xshade is obtained with parallel sunshine casting on the scene, capturing shades, side projections, and full structural visibility of buildings. To extract a structure-only reference, we generate a skeleton snapshot xsk by rendering the identical scene with the sun being invisible. Additionally, a satellite image xsat of the corresponding area is aligned the simulated environment with real-world conditions. Finally, we extract text descriptions T encoding temporal information, such as the sun angle and time of day, which provide an additional modality for shade generation control. We formulate the text description as follows:

where T represents the generated text prompt, θsun denotes the solar declination or angle, and tday is the timestamp. The function f(·) transforms solar and temporal attributes into natural language descriptions, which is a simple string combination function with alternative descriptive choices. The following example showcases structured text prompts for different temporal conditions:

Example Text Prompts:

Prompt 1: Solar declination: -20.7°

Prompt 2: Angle: 45°

Prompt 3: Right now, it is 6:00 PM in a day.

Last, we have the Post process step. To benefit the deep learning models, we need a ground-truth dataset for measuring the model's output and providing feedback through loss functions. However, current data from step 3 is not the ultimate shade as it includes side shadows and noises. Therefore, to obtain the actual ground truth shade, we propose to take another snapshot of the scene without sunshine, named as 'skeleton image' xsk , which only provides a pure building structure, then the pure shade can be obtained through the formula:

where xgt represents the extracted ground truth shade, xshade is the shaded snapshot taken under sunlight, capturing the shades, sides, and full building observation, and xsk is the skeleton snapshot taken without sunlight, providing only the pure building structure. The term I(xshade ≤ α) is an indicator function that removes low-intensity grey values (noise thresholding), where α is a predefined threshold. This equation effectively removes structural elements and noise, isolating the actual shaded areas to provide precise ground truth data for machine learning models.

The dataset construction process has improved alignment between simulated and real-world data while enabling controllable shade generation via images, textual descriptions, or a joint approach. This dataset has been split into train, test sets by 70%, 30%, which can help the research community with a well-defined benchmark (researcher could hold another validation set by changing the index if necessary).

4 A Text Conditioned Shaded Image Generation Model

Now given a well-constructed shade dataset D = {xshade , xsk , xsat , T}, where xshade is the shaded snapshot capturing shadowed areas under sunlight, xsk is the skeleton snapshot, xsat is the real-world satellite image after alignment, and T is the textual description encoding solar angle θsun and timestamp tday. To benefit from that, this paper also explores methods that generate accurate shade images following the text prompt and base image (which provides a base map for generation). Given the objective is to generate shade images using a base map and text conditions, we adopt a diffusion-based framework, ControlNet [Zhang et al., 2023], as our backbone since the diffusion models are effective in image synthesis and editing tasks [Dhariwal and Nichol, 2021]. As shown in Figure 2, our proposed method focuses on edge information incorporation and contrastive learning module. We will introduce the details after a reflection on the Controlnet Model in our setting.

Reflection on ControlNet in Our Task ControlNet is an extension of diffusion models designed to incorporate visual conditions into the image generation process. It is built upon pre-trained Stable Diffusion [Blattmann et al., 2023] models by introducing an additional control pathway that enables precise guidance during generation.

In our task, the vanilla ControlNet can be used to generate shade images conditioned on both structural and descriptive inputs (text description of time/angle). The base map xsk , representing the skeleton structure of the scene, serves as the primary visual condition. The text prompt T, which encodes solar angle θsun and timestamp tday, provides semantic and temporal guidance. These inputs collectively form the conditioning inputs c, driving the generation process. The generation process can be expressed as:

where xI is the generated shade image, T is the text prompt, xsk is the base map. At each layer i of the decoder D, ControlNet integrates these conditions as:

where hi represents the feature map at layer i, and hicond is the feature derived from the combined conditions c. This mechanism ensures that the generated output adheres to both the structural layout (base map) and temporal dynamics (descriptions) provided by the inputs.

The vanilla ControlNet framework could functionally generate shade images, but the quality of generation is limited, mainly because of two reasons:

First: The shade generation involves very subtle features: Shade occupies only a minor part of an image and is often nuanced and difficult to capture. Thus, identifying and processing these subtle features requires a further refinement of the model's design.

Second, The shade changes follow temporal rule: Shade changes dynamically over time, following natural rules related to solar angles and time progression. Also, the shade changes can be subtle in close timing. The vanilla ControlNet struggles to accurately simulate these temporal dependencies, often generating the wrong orientation of shades due to the failure to capture the consistent and realistic evolution of shade in response to these factors.

4.1 Edge-Enhanced Conditional Generation

To address the first challenge that the vanilla model fails to capture subtle shade outline features and often generates shadows with crooked boundaries, even for buildings with straight outlines, we propose an edge-enhanced conditional generation method. The approach incorporates edge information as an additional visual condition alongside the base map, as shown in the preparation part of the Figure 2.

Specifically, our base map xsk ∈ RH×W×3 is a threechannel mask of building footprints. We extract structural boundaries via Canny edge detection to obtain a singlechannel edge map xedge ∈ RH×W×1 . These are concatenated along the channel dimension to form a four-channel conditioning tensor (R, G, B and edge):

This four-channel tensor is supplied to the ControlNet's UNet as its concatenated input, together with the text prompt T . By providing both the global building layout and its precise edge contours, the generator is shown below:

It is guided to produce shadows with straight, well-aligned boundaries.

4.2 Contrastive Based Shade Generation

To tackle the second challenge, we innovatively propose a contrastive learning enforced learning paradigm for shade generation. We first construct a contrastive data pipeline, introduce a lightweight contrastive encoder to embed the features, and then introduce the contrastive learning-based shade generation.

Contrastive Buffer Pairs Creation

To effectively train a model capable of generating shades following the temporal rules (the generated shades between nearby time steps should be more similar than those with larger temporal gaps), we design a contrastive learning framework. This section introduces how we construct a training set of contrastive pairs: positive and negative pairs, based on their temporal and spatial relationships.

Let D = {x1, x2, . . . , xn} represent the dataset of images, where each image xi is associated with metadata such as its timestamp ti and spatial location li. For any two images (xi, xj), the pair is classified following equation :

which describes two cases:

Positive Pair: If li = lj (same spatial location) and abs(|ti - tj |) = h (timestamp difference of h hour).

Negative Pair: If li = lj (same spatial locations) or abs(|ti - tj |) > h ( the difference greater than h hour).

In order to ensure a balanced distribution of positive and negative pairs, for each image xi, we randomly select up to k+ positive pairs (k+ is a predefined maximum number of positive pairs per image 1 ). And similarly, also selects up to k- negative pairs, ensuring sufficient variety for the contrastive learning process. The resulting dataset of pairs P = {(xi, xj , Labelij ), xedge i , xsk i , T} is then used for training, the first three tuples is used for contrastive learning, and the last three is used for edge feature aggregation. For simplicity, without further clarification, the xi and xj are all satellite skeleton images in this section.

Contrastive Learning for Shade Generation

To address the challenge of maintaining temporal consistency in shade generation, we incorporate a contrastive learning framework into the training process. This framework is designed to distinguish between temporally consistent and inconsistent shade pairs by leveraging the contrastive dataset P = {(xi, xj , Labelij ), xedge i , xsk i , T}.

1 We set as 5 in our setting for learning efficiency.

During training, for each image pair (xi, xj), feature embeddings hi and hj are obtained from a pre-trained embedding network using U-net, these embeddings are used to compute a similarity matrix S ∈ RN×N , where N = 2 × B is the total number of embeddings in the batch (each pair contributing two embeddings), and each element Suv represents the cosine similarity between embeddings as below:

To optimize the temporal consistency, we apply the InfoNCE loss, which encourages embeddings of positive pairs (same location and adjacent timestamps) to be closer while pushing apart embeddings of negative pairs (differing locations or timestamps). The InfoNCE loss is expressed as:

where τ is a temperature hyperparameter (set as 0.1 in our setting) that controls the concentration of the similarity distribution. Positive pairs (Labelij = 1) contribute to higher similarity values in S, while negative pairs (Labelij = 0) are penalized to ensure their embeddings remain distinct.

The total loss function integrates the original ControlNet loss and the contrastive loss to optimize the generation of accurate and temporally consistent shade images. This is expressed as:

where LControlNet is the primary objective for generating shade images and Lcontrastive enforces temporal consistency between embeddings of positive and negative pairs. The contrastive loss Lcontrastive is scaled by λ1 = 0.1.

By minimizing this combined loss, the model achieves improved temporal coherence in shade generation and more nuanced outputs that align with real-world shading patterns over the time changes of a day.

5 Experiment

In this section, we conduct experiments to verify the effectiveness of our proposed method. We have designed three sets of experiments, the first is to verify the model's performance in dense building cities, such as Beijing, Phoenix downtown, and S˜ao Paulo, The second is to show the performance in the relatively sparse environment, such as Tempe, Brasilia, etc. Please note that these cities' selection is based on their representation in each continent, and also because they suffer from extreme heatwaves in summer, as history records. The third, we perform an ablation study to understand the contribution of different components in our methods.

5.1 Metrics

To evaluate the quality of generated shadow shades, we employ five commonly used metrics: Structural Similarity Index Measure (SSIM), Mean Squared Error (MSE), mean Intersection over Union (mIoU), Boundary Intersection over Union (B-IoU), and Learned Perceptual Image Patch Similarity (LPIPS) as in [Bakurov et al., 2022], [Tan et al., 2013] [Ghazanfari et al., 2023]. A more detailed explanation of the latter two metrics is as follows.

Boundary Intersection over Union (B-IoU) ↑ Inspired by the work [Cho, 2021], we proposed a new metric named: BIoU, which measures the alignment of boundaries between the generated and ground truth shadow masks. If Mpred and Mgt are the binary shadow masks, and K is a 3 × 3 structuring element. We extract their boundaries via morphological dilation and erosion:

Let's denote that ∂Mpred and ∂Mgt are the predicted and ground-truth boundaries, B-IoU is computed as the Intersection over Union of these boundary sets:

where |·| denotes the number of boundary pixels. The range of B-IoU is [0, 1], where higher values indicate better boundary alignment. Detailed implementation can be found in the evaluation code.

Learned Perceptual Image Patch Similarity (LPIPS) ↓ LPIPS quantifies perceptual similarity by comparing feature embeddings of the generated and ground truth images extracted from a pretrained network (e.g., AlexNet). It is computed as:

where ϕl denotes the feature map at layer l, and Hl,Wl are its height and width. LPIPS ranges from [0,∞), where lower values indicate greater perceptual similarity.

5.2 Experiment Analysis

A. The Efficient Convergence Speed. As shown in Figure. 3, the plotted curves are the reflection of the mean and standard deviation of the 5-rounds result for each method across two commonly adopted losses (reconstruction and VLB loss [Deja et al., 2022]). Our model DeepShade demonstrates a much more efficient convergence performance in comparison to baseline methods: edge-condition diffusion and vanilla controller. The training of diffusion models is essentially difficult; however, our proposed method provides a more rational way in the shade prediction setting.

B. Accurate Generation Across 12 Cities in the World.

In this part, we focus on the Table. 1. The upper part is the dense areas in the world selected dataset for testing, and the lower part is the sparse areas. We can observe that our model outperforms most of the baseline methods in the various metrics (with a total of 50 epochs of training), regardless of the dense or relatively sparse scenario land covers. It demonstrates the vivid simulation of the conditioned area using the ControlNet-based method by training on one dataset, with good transferability, it better indicates that this work has great potential for real-world applications.

C. Ablation Study. We also included an ablation study in the paper, as in Table 2, it is the training conducted in the Tempe dataset, and the test is the other 30% from the original dataset given by the default split. This result reveals the importance of each component in our method. We can see that the edge condition (our method without contrastive learning) suffers the most performance drop in comparison to the DeepShape full model structure, and if we also remove the edge conditions, the performance further decreases, the backbone model is the stable diffusion model (all of the above models are trained 5 times with mean and std reported).

6 Demonstration

Besides the quantitative analysis in experiments, it is important to demonstrate the real-world impact of the work. Thus, we design a proof-of-concept demo as shown in Figure. 4 using a subarea of Arizona State University. In this demo, we tackle the problem of integrating the shade ratio as a factor when making the routing suggestions. The input is a skeleton image containing the building outline, extracted from a satellite image of the interested area. Then, based on the time for planning, we describe as the text prompt T together with the image that is processed to xcond as in Eq. 5, the shape map will be generated as output, it will be used for shade ratio calculation by overlaying the shade map with the road using longitude and latitude ranges. Given the shade ratio, the planning is made by jointly considering the user's preference (weight) on shade and distance by a variant of the Dijkstra algorithm. The green shows a more shaded plan, while the red means the shortest path. This demo reveals the potential of a real-world application; given that shade-involved planning is crucial for areas that suffer from extreme heat waves, this demonstration shows a way that could possibly help decrease heat stroke cases and improve the health of outdoor people.

7 Conclusion

In this work, we focus on the challenge of simulating realistic urban shade patterns by introducing both a novel dataset and a generative framework. We first developed a globally diverse dataset of building layouts with aligned satellite imagery and timestamped shade snapshots, paired with text-conditioned prompts encoding solar zenith angle and time of day. We then propose DeepShade, a text-conditioned, edge-enhanced diffusion model built on ControlNet that fuses building skeletons and Canny edges and incorporates a contrastive learning module to enforce temporal consistency. We have conducted extensive experiments across multiple cities, showing that DeepShade could function reasonably well in indomain and out-of-domain tests. An ablation study further validates the importance of edge conditioning and contrastive loss, and lastly, we show a proof-of-concept application in shaded route planning, which demonstrates the work's practical utility for heat-aware urban navigation. The dataset 2 and codebase 3 are released for the promotion of research in this direction.

8 Limitation and FutureWork

The paper proposes a very first dataset that aligns shades to the satellite view of a location; however, there might be missing building issues in ground truth shades since the OSM data is not always updated as the time that the satellite image is taken. Future development to tackle this problem is important; meanwhile, the real-time shade planning would require the pre-generated shade ratios, and they should be regularly indexed to the real-world road maps to enable large-scale support for the planning systems. A more accurate and robust framework is worth exploring.

2 https://huggingface.co/datasets/DARL-ASU/DeepShade

3 https://github.com/LongchaoDa/DeepShade repo.git

Acknowledgments

The work was partially supported by NSF awards #2421839, NAIRR #240120, and used AWS through the CloudBank project, which is supported by NSF grant #1925001. Xiangrui was supported by the Fulton Fellowship at ASU. The views and conclusions in this paper are those of the authors and should not be interpreted as representing any funding agencies. Thank OpenAI for providing API credits under the Researcher Access program and Amazon Research Awards.

Contribution Statement

Longchao Da and Xiangrui Liu both contributed equally to the paper.