GeoMind: Explicit Spatial Reasoning via Dual-Reference Geometric Modeling

1 Introduction

Spatial intelligence in embodied agents relies on constructing a coherent internal representation to support mental completion and perspective taking [Feng et al., 2025; Wang et al., 2023]. In recent years, Vision-Language Models (VLMs) have become a primary foundation for embodied agents to interpret visual observations and perform language-guided spatial reasoning [Yang et al., 2025a]. However, while contemporary VLMs demonstrate strong semantic understanding, their spatial understanding remains substantially below human performance [Li et al., 2025c].This gap may be partly related to a common design choice in VLMs to encode geometric cues implicitly within unstructured latent features [Wu et al., 2024; Cheng et al., 2024], which can hinder the formation of a stable spatial state under viewpoint changes. This limitation becomes most apparent during reference-frame switching: models struggle to reconcile view-invariant spatial facts (e.g., "the TV is behind the sofa") with transient visibility and occlusion evidence, often yielding viewpoint-inconsistent predictions and hallucinations across viewpoints [Li et al., 2025a; Guan et al., 2024]. This issue arises from the absence of an explicit, queryable world state. Without such a representation, models struggle to preserve viewpoint consistency and to reason over both allocentric topology and egocentric observations.

∗Corresponding author

To bridge this gap, we introduce GeoMind, a framework that reformulates spatial QA as a generative process grounded in an explicit, language-native world model [Nejatishahidin et al., 2024]. As illustrated in Figure 1, many VLM-based approaches produce answers directly from implicit latent representations (Figure 1.C, top). In contrast, our method uses a single LLM to autoregressively generate Geometric Description Language (GDL) tokens as an explicit intermediate map (Figure 1.B), and then generates the answer conditioned on this map (Figure 1.C, bottom). GDL features a dual-reference structure to address reference-frame switching: an allocentric branch encodes viewpoint-invariant topology, while an egocentric branch captures view-dependent observational evidence. This design separates "what exists in the world" from "what is currently observed," providing a structured scaffold for cross-view reasoning.

GeoMind is supported by a scalable programmatic data engine, which constructs GDL pseudo-labels by distilling geometric signals from reconstructed 3D scenes, enabling largescale geometric supervision without expensive human annotation [Raychaudhuri and Chang, 2025]. Inference is performed under a self-consistency mechanism that constrains the model to derive answers solely from entities and relations instantiated in the GDL map. Under this strict grounding, the reasoning process becomes transparent and auditable. As shown in Figure 1, error cases can often be localized to omissions or inconsistencies in the intermediate map, and counterfactual edits to the GDL representation provide a direct way to test and revise the final answer.

Our main contributions are:

• Dual-Reference GDL Framework. We introduce a language-native cognitive map representation that explicitly disentangles allocentric (world-centric) topology from egocentric (view-dependent) visibility, providing a structured substrate for cross-view spatial reasoning.
• Self-Consistent and Auditable Reasoning. We enforce a self-consistent inference regime in which answers are derived solely from the generated GDL map, yielding coherent reasoning and enabling systematic error diagnosis through explicit map inspection and counterfactual editing.
• Scalable Programmatic Supervision. We propose an automated pipeline that distills 3D geometry into discrete symbolic supervision, enabling large-scale training without human annotation and yielding competitive performance against proprietary large-scale models on established spatial reasoning benchmarks.

2 RelatedWork

Current approaches to spatial reasoning span a spectrum from implicit adaptation to explicit modeling. The implicit stream optimizes the reasoning process itself, ranging from general VLMs (e.g., GPT-4V [Yang et al., 2023], InternVL [Chen et al., 2024]) to models fine-tuned with reinforcement learning (e.g., Spatial-SSRL [Liu et al., 2025]). However, without explicit grounding in physical geometry, these methods often fail to maintain cross-view consistency in complex 3D scenarios. This has spurred a shift towards explicit modeling, where symbolic structures serve as reasoning scaffolds.

MindCube [Yin et al., 2025] is a representative work in this direction, pioneering the "model-then-reason" paradigm by predicting 3D voxel states. Similar to other globalrepresentation approaches like 3D-LLM [Hong et al., 2023], it focuses primarily on allocentric (world-centric) topology. While effective for stable geometry, such global-only models often struggle with situated reasoning-tasks requiring a tight coupling between global position and local visibility (e.g., embodied navigation [Ma et al., 2023]). GeoMind advances this paradigm by introducing Dual-Reference GDL, which explicitly disentangles two branches: an allocentric branch for stable topology, and an egocentric branch for local observational context (e.g., visibility, occlusion). This design provides a more rigorous scaffold for perspective-taking.

To construct such grounded representations, we bridge the gap between neural perception and symbolic mapping. While foundational vision models like VGGT [Wang et al., 2025a] and DUSt3R [Wang et al., 2024] excel at feed-forward reconstruction, they lack the semantic interface required for high-level reasoning. Conversely, traditional roboticmapping (SLAM) provides metric consistency but generates dense data not natively queryable by LLMs. Recent map-driven methods have explored converting BEV maps [Xu et al., 2025] into language prompts, yet these are often tailored to domainspecific assumptions (e.g., autonomous driving). GeoMind synthesizes these threads: we leverage frozen geometric priors (VGGT) for robust 3D lifting and distill them into a lightweight map serialized as text tokens. This design allows our model to perform "in-context spatial reasoning" that combines the geometric rigor of SLAM with the semantic generalization of VLMs.

3 GeoMind

GeoMind is a neuro-symbolic model-then-reason framework for multi-view spatial QA. Instead of mapping pixels directly to answers, it introduces an explicit intermediate representation-the Geometric Description Language (GDL)-as a language-native cognitive map that encodes allocentric topology and egocentric evidence, enabling robust and auditable spatial reasoning.

Problem Formulation. Following standard multi-view spatial QA settings [Azuma et al., 2022], given an observation sequence V = {Ik}Kk=1 and a query q, our goal is to predict the answer a. Unlike end-to-end approaches that model P(a | V, q) directly, we introduce an explicit intermediate map variable M (instantiated as GDL) to enforce geometric consistency. This factorizes the inference into map construction and map-based reasoning:

In practice, we decode a MAP estimate ˆM = argmaxMP(M | V) and generate a conditioned on ( ˆM, q).

3.1 Geometry-Aware Object Abstraction

Our perception module uses a frozen VGGT encoder to extract geometry-aware patch features with cross-view consistency. Given K views V = {Ik}Kk=1 , the encoder outputs patch features Fpatch ∈ RK×Np×C , together with per-view depth maps {Zk}Kk=1 and relative camera poses {ˆTk}Kk=1 , where Np denotes the number of patch tokens per view. During training, ground-truth depth and poses are used only to construct reliable 3D lifting and supervision signals. At inference, we perform 3D lifting using {Zk, ˆTk} without requiring external pose inputs [Miao et al., 2025].

To preserve spatial layout information during patch tokenization, we inject learnable positional encodings Epos into the feature stream:

where Epos ∈ RK×Np×C encodes each patch's 2D location within its image. The frozen VGGT backbone supplies crossview coherence, while Epos retains local spatial context for subsequent object abstraction.

Geometry-Guided Slot Formation We compress multiview inputs into object slots. These object-level tokens represent merged 3D instances and serve as concise visual prompts for subsequent GDL generation (N ≪ K × Np). We adopt a lift-and-merge pipeline that synergizes mask-based proposals with 3D multi-view consistency.

Specifically, for each view index k ∈ {1, . . . ,K}, we obtain a set of instance masks Sk = {sk,j}Jk j=1 , where j indexes mask proposals in view k and Jk is the number of masks. We utilize ground-truth masks with stochastic augmentations (e.g., erosion, noise injection) during training to mitigate the domain gap, while switching to SAM for open-world inference [Ravi et al., 2024].

For each mask sk,j , we compute an appearance descriptor fk,j ∈ RC via masked average pooling:

Simultaneously, we back-project masked pixels to the world frame using Zk and ˆTk:

where Kk is the camera intrinsic matrix. The collection of lifted points forms a candidate point cloud Pk,j, from which we derive an axis-aligned 3D bounding box Bk,j . Thus, each proposal is characterized by both a semantic descriptor fk,j and a spatial support Bk,j .

Cross-View Correspondence. To associate proposals across views, we construct a pairwise cost matrix that balances geometric overlap and appearance affinity. We normalize the cosine distance to [0, 1] and define the matching cost as:

We solve for optimal one-to-one assignments using the Hungarian algorithm [Kuhn, 1955]. To enforce spatial feasibility, we apply a gating mechanism where pairs with IoU3D ≤ τiou are assigned infinite cost. Matched proposals are then merged into persistent 3D instances {Oi}.

Slot Encoding and Projection. For each merged 3D instance Oi, let Ii denote the set of indices for matched mask proposals belonging to Oi. We aggregate their per-view descriptors {fk,j}(k,j)∈Ii into an instance-level appearance embedding ¯fi ∈ RC via cross-view attention pooling:

where s(i) k,j = q⊤attWa fk,j is the attention score derived from a shared learnable query qatt and projection Wa. We further explicitly attach 3D geometry, concatenating the centroid ci ∈ R3 and box extent si ∈ R3 . A lightweight projector ϕ (implemented as an MLP) maps this representation to the LLM input space:

The resulting object tokens Z = {zi}Ni=1 are prepended to the LLM sequence as visual prompts for subsequent GDL generation.

3.2 GDL: ADual-Frame Cognitive Map

We represent the latent map variable M as a Geometric Description Language (GDL) sequence. GDL serves as a compact, text-serializable scene abstraction that the LLM can read and manipulate directly. As illustrated in Figure 3, a GDL sample consists of (i) an object registry that assigns a unique

identifier and a category label to each instance, and (ii) a set of pairwise relations written in two complementary frames.

The allocentric branch encodes viewpoint-invariant spatial topology in a shared world frame, such as coarse relative direction and distance (e.g., left of). The egocentric branch records frame-conditioned evidence tied to a specific observation, such as visibility and view-dependent ordering (e.g., vis: true). Keeping both branches makes the representation explicit about what is stable across views and what is supported by a particular view, which is central to perspective taking and cross-view consistency.

Dual-Frame Relation Schema

We use a right-handed world frame with +Y aligned with gravity and measure distances in meters; allocentric directions are defined on the XZ plane. GDL defines a relational graph over object nodes O = {oi}Ni=1 . Each node is associated with a unique identifier and a category token (e.g., "chair"), providing a stable reference for subsequent relations.

For each ordered pair (i, j), the relation is written as Rij = ⟨Aij , {E(k) ij }k⟩, where Aij denotes allocentric topology and E(k) ij denotes egocentric evidence for view k. Figure 3 shows a concrete serialized example.

Allocentric Topology (world-centric). Aij encodes viewpoint-invariant spatial memory in a unified world coordinate system. It captures stable 3D topology (e.g., relative direction, distance, and height ordering), forming a persistent map that remains consistent under viewpoint changes.

Egocentric Evidence (view-centric). E(k) ij records viewdependent evidence of the same relation in a specific observation k. It captures local projection states such as visibility, relative 2D configuration, and ordinal depth, enabling perspective-taking and grounding the map against what is visually supported in each view.

The dual-frame structure maintains a globally consistent map and grounds it in view-specific evidence. Each merged 3D instance Oi is assigned a unique object identifier (e.g., obj i) in the serialized GDL, and all relations and evidential records reference objects solely through these identifiers. During reasoning, the model is restricted to using the same identifiers via explicit in-context constraints and post-hoc validation, which prevents entity drift even when category names are ambiguous.

3.3 Programmatic Supervision Pipeline

To obtain supervision without manual annotation, we derive geometry-based GDL targets from the 3D reconstructions provided by ScanNet v2 [Dai et al., 2017] on the training set. Scene Parsing. Following standard 3D scene graph definitions [Wald et al., 2020], we parse the semantic mesh into an object set O = {oi}NGT i=1 , where NGT denotes the instance count. Each object oi is parameterized by its 3D centroid ci, extent si, semantic label yi, and a set of geometry keypoints Qi .

Allocentric Relation Graph. To avoid the computational overhead of dense fully-connected graphs [Zhou et al., 2019], we construct a sparse directed (i.e., ordered) relation graph G = (O, E) over mesh instances. For each source object oi, we define a neighbor set N(i) as its Mnbr nearest objects measured by Euclidean distance between centroids, and add edges (i, j) for all j ∈ N(i). This yields |E| = O(NGT·Mnbr) edges, reducing the supervision size from quadratic to linear in NGT. For each retained edge, we compute the displacement ∆pij = cj - ci and discretize its azimuth on the XZ plane and metric distance into categorical bins to obtain allocentric relation labels (Sec. 3.2).

Egocentric Visibility Validation. To derive view-dependent evidence, we use ground-truth camera extrinsics Tk and intrinsics Kk to project 3D keypoints into each view[Ravi et al., 2020]. A centroid-only baseline is brittle under partial occlusion and depth outliers (e.g., missing or noisy measurements around the projected centroid). We therefore adopt multikeypoint validation, which improves robustness by requiring depth-consistent support for at least one keypoint. For each p ∈ Qi, we project it to pixel up with projective depth zp, and compare zp with the sensor depth map Zk:

where Ω denotes the valid image domain. We mark oi as visible in view k if v(k) i > 0. Unless otherwise specified, we set τz = 3.0m to tolerate reconstruction misalignment and sensor noise. For visible pairs, we further compute 2D relations (e.g., left/overlap) from projected keypoint spans and infer ordinal depth (nearer/farther) by comparing median depths of validated keypoints.

Serialization. Finally, the allocentric relations and egocentric evidences are serialized into the GDL schema (Sec. 3.2), forming the ground-truth sequence MGT for the generative objective LMap.

3.4 Two-Stage Training Protocol

We formulate both cognitive map construction and question answering as conditional sequence generation. Training follows a two-stage protocol that first stabilizes GDL generation, and then introduces reasoning supervision while preserving map fidelity.

Stage 1: Cognitive Map Construction. Given multi-view observations V, we train the model to generate the groundtruth GDL map MGT = (mGT 1 , . . . ,mGT T ) by maximizing its likelihood:

This stage focuses on learning a valid and structurally consistent map representation from visual inputs.

Stage 2: Joint Map-and-Reasoning Training. We then train the model to answer the query q conditioned on the ground-truth mapMGT (teacher forcing):

where aGT = (aGT 1 , . . . , aGT S ) is the tokenized answer. The overall objective in Stage 2 is

We use λ to balance map preservation and reasoning adaptation; implementation details are provided in Sec. 4.1.

At test time, the model performs sequential generation where it first predicts a map ˆM from V and subsequently derives the answer conditioned on ( ˆM, q).

4 Experiments

4.1 Experimental Setup

Benchmark Overview. We evaluate GEOMIND on five spatial reasoning benchmarks that cover both image and video settings. Image-based benchmarks include SITE [Wang et al., 2025b], ViewSpatialBench [Li et al., 2025a], MindCube [Yin et al., 2025], and MMSIBench [Yang et al., 2025d]. VSI-Bench [Yang et al., 2025b] targets route planning in egocentric videos. For programmatic supervision, we use the ScanNet v2 [Dai et al., 2017] training split (1,513 scenes) and generate physics-based GDL labels via mesh analysis. For reasoning alignment, we aggregate ScanQA [Azuma et al., 2022], ScanRefer [Chen et al., 2020], and Nr3D [Achlioptas et al., 2020] and restrict all samples to the same training scenes to prevent leakage.

EvaluationMetrics. We follow the official evaluation protocols for each benchmark. For multiple-choice tasks in VSIBench, we report MRA, the Multiple-choice Response Accuracy. For SITE, we report CAA, the Correct Answer Accuracy, which checks both the answer text and the grounding bounding box. For open-ended QA benchmarks, we report exact match accuracy (Acc) under deterministic decoding. When required by the benchmark protocol, we filter semantic equivalents via LLM-based evaluation where applicable.

Implementation Details. We build GeoMind on the Qwen3-VL backbone with frozen vision and language components. Trainable modules include the object projector (3.9M parameters) and LoRA layers (rank 64, alpha 128, targeting attention projections, 40.4M parameters), totaling 44.3M parameters. Training employs AdamW with a twostage curriculum over 15 epochs: the first 10 epochs focus on map generation (learning rates 5 × 10-6 for LoRA and 5 × 10-5 for the projector), followed by 5 epochs of joint training with LJoint = LMap + 0.3LReasoning. We use an effective batch size of 8 with cosine annealing on 4 NVIDIA A100 (80GB) GPUs using BFloat16 precision, setting Nslots = 64 and Mnbr = 5 by default.

4.2 Main Results

Overall Performance. Table 1 summarizes the quantitative results. GEOMIND demonstrates superior performance among the compared open-weight baselines. Notably, GeoMind-8B achieves an average accuracy of 51.1%, surpassing Gemini-2.5-Pro (50.4%) and narrowing the gap to GPT-5 (52.1%) to within 1.0 percentage point. Most strikingly, on perspective-sensitive benchmarks like ViewSpatial and VSI-Bench, our 2B model substantially outperforms the spatial-specialized baseline MindCube-3B by a 37.4-point absolute margin.

Analysis of Perspective Taking. To understand the source of these gains, we break down performance on referenceframe switching tasks (Table 2). Standard VLMs struggle to generalize from camera-centric inputs to person-centric queries, exhibiting a sharp drop (e.g., Qwen3: 54.2% → 40.2%). In contrast, GEOMIND maintains robustness, achieving 53.9% on the challenging Person-Rel.Dir task. This confirms that our Allocentric Branch effectively disentangles global topology from local visibility.

Quality of Intermediate Representation. We verify that the intermediate GDL acts as a trustworthy world model. On the ScanNet validation set, GeoMind achieves a 79.8% valid JSON format rate and 84.2% structural consistency. Qualitative analysis further shows that the model correctly captures

87.3%of salient spatial relations, enabling auditable reasoning.

4.3 Ablation and Analysis

To quantify the contribution of each component, we ablate GEOMIND-8B on validation sets (Table 3). Direct QA without GDL yields the largest degradation (-6.4%Overall) and sharply reduces Route accuracy (64.8 → 52.1). This confirms that performance gains stem from reasoning over an explicit intermediate map rather than learning shortcut correlations. Removing the Allocentric Branch disproportionately hurts person-reference queries (51.6 → 41.5), while removing the Egocentric Branch mainly degrades camerareference grounding (48.6 → 43.2). This outcome validates our dual-reference design: allocentric topology is essential for reference-frame switching, whereas egocentric evidence stabilizes view-conditioned grounding.

Training and Consistency. Skipping the two-stage training curriculum decreases Overall accuracy by 3.2 points, suggesting that aligning map structure prior to joint optimization im-

proves stability. Disabling self-consistent decoding further reduces accuracy by 1.9 points, indicating that map-grounded generation effectively mitigates error propagation.

Hyperparameter Sensitivity. We further analyze model robustness in Table 4. Regarding Slot Capacity, performance peaks at Nslots = 64. Smaller budgets (e.g., N = 30) degrade performance significantly, likely due to the failure to capture all relevant entities in cluttered environments. Consequently, we establish 64 as a conservative upper bound to prevent truncation. Increasing Nslots further yields diminishing returns (51.1% vs. 50.9%), as it primarily improves coverage redundancy without enhancing the effective relation density.

In terms of Graph Sparsity, setting Mnbr = 5 proves optimal. While overly sparse graphs risk fragmenting the topology and hindering multi-hop propagation, excessive connectivity (Mnbr ≥ 10) introduces irrelevant long-range noise that dilutes geometric distinctiveness. This validates the necessity of a balanced and sparse topological representation for precise spatial routing.

4.4 Visualization of Reasoning Process

We visualize the inference trajectory of GeoMind in Figure 4. The model first converts multi-view inputs into explicit GDL tokens, where the allocentric branch encodes stable spatial topology and the egocentric branch records view-dependent evidence such as visibility and occlusion. For queries requiring mental rotation, the reasoning module uses these

grounded geometric relations to infer the correct orientation.

The examples show that the symbolic scaffold supports au-

ditable spatial reasoning beyond simple pattern matching.

5 Conclusion

We introduced GeoMind to ground spatial reasoning in an explicit, dual-reference map. By constraining inference to self-generated entities, our approach mitigates geometric hallucinations and enables transparent auditability via map inspection. Empirically, GeoMind achieves performance competitive with proprietary models, confirming that explicitly reconciling allocentric topology with egocentric evidence is essential for robust perspective-taking.

Acknowledgments

This work was supported in part by the Fundamental Research Funds for the Central Universities of China (Grant No. PA2025GDSK0036), the National Natural Science Foundation of China (Grant No.62563009), the Anhui Provincial-Level Science and Technology Innovation Tackle Plan Projects (Grant Nos.202523o09050006 and 202423k09020003), and the Project of the Provincial Major Industrial Innovation Plan of Anhui Provincial Development and Reform Commission (Grant No.AHZDCYCXJHOC2025-08).