DECASTE: Unveiling Caste Stereotypes in Large Language Models Through Multi-Dimensional Bias Analysis

1 Introduction

Recent investigations into large language models (LLMs) have highlighted significant socio-cultural biases, often reflecting and amplifying societal inequities in tasks such as text generation and question answering [Mukherjee et al., 2023; Gallegos et al., 2024; Tao et al., 2023]. While extensive research has addressed biases related to race, gender, and occupation, caste bias within LLMs remains largely unexplored. Caste bias, deeply rooted in the hierarchical caste system of South Asia, contributes to systemic social stratification and discrimination based on birth and perceived purity. Despite legal safeguards and government initiatives, historically marginalized communities-including Scheduled Castes (SCs), Scheduled Tribes (STs), and Other Backward Classes (OBCs)-continue to face widespread prejudice and exclusion, particularly in domains such as employment, education, and social interactions [Ambedkar, 2022; Desai and Dubey, 2011; Thorat and Neuman, 2012; Rukmini, 2014]. Although algorithmic fairness and bias mitigation have gained prominence, caste-based bias in LLMs remains significantly underexamined. This is especially concerning, as LLMs increasingly shape digital discourse. If left unchecked, caste-related biases could perpetuate or even escalate discrimination in subtle and overt forms.

Prior computational studies have explored caste discrimination in areas such as social media, online advertisements, and employment settings, but research specifically targeting caste bias in LLMs is still limited. Existing work often adopts a binary framework contrasting dominant and oppressed castes, lacking fine-grained analysis across diverse groups and contexts [Harad, 2020; Qureshi and Sabih, 2021; Krishnamurthi and Krishnaswami, 2020; Sahoo et al., 2024]. Our work addresses this gap by systematically investigating caste bias in LLMs and its downstream implications. We introduce the DECASTE framework 1 , which comprises two novel tasks. The first, the Stereotypical Word Association Task (SWAT), evaluates how LLMs associate caste groups with stereotypical terms using structured prompts and a dedicated bias metric. The second, the Persona-based Scenario Answering Task (PSAT), probes biases in decision-making through caste-based personas. Together, these tasks offer a comprehensive and multi-dimensional analysis of caste bias in LLMs. Our key contributions are as follows:

• Development of the DECASTE evaluation framework, comprising two tasks that leverage implicit and explicit

1 An extended version of this work is available on arXiv.

bias probing methodologies.

• Creation of a task-specific caste-stereotypical dataset across four critical dimensions: social, economic, educational, and political.
• A comprehensive evaluation of nine distinct LLMs across all four dimensions, revealing that these models reinforce caste stereotypes to varying degrees, with the potential to significantly impact real-world scenarios.

2 RelatedWork

Caste-based discrimination in large language models (LLMs) represents a critical facet of the broader issue of social bias in AI systems, which have been shown to perpetuate stereotypes related to race, gender, and other social constructs. Caste, deeply embedded within Indian society and the Indian diaspora worldwide, plays a significant role in shaping access to critical resources such as education, job opportunities, and public services [Kumar, 2010; Tejani, 2013a; O'Reilly and Dhanju, 2014]. Despite sustained efforts by social reformers such as Dr. B.R. Ambedkar [Ambedkar, 2014] and various governmental initiatives [Agrawal et al., 1991], caste-based discrimination continues to hinder social and economic mobility, thereby reinforcing systemic inequality [Deshpande, 2011; Thorat and Neuman, 2012]. While caste discrimination is most prominent in India, its global reach due to the widespread Indian diaspora makes casterelated biases a pressing worldwide issue.

2.1 Social Biases in LLMs

The issue of bias in natural language processing (NLP) models, especially large-scale models like GPT and BERT, has attracted significant attention in recent years. These models, trained on vast datasets collected from the internet, inevitably inherit and amplify the societal biases embedded in their training data. Several studies have demonstrated that LLMs often reinforce harmful stereotypes based on race, gender, and other social categories. For example, word embeddings, which are core components of many NLP systems, have been shown to encode these societal biases [Devlin et al., 2019; Peters et al., 2018; Radford et al., 2018]. Bolukbasi et al. [Bolukbasi et al., 2016] demonstrated that gender biases in word embeddings link terms like "man" with "computer programmer" and "woman" with "homemaker". Likewise, Caliskan et al. [Caliskan et al., 2017] introduced the Word Embedding Association Test (WEAT), which pairs social categories with target attributes to measure biases in word embeddings. Tools such as WEAT and Social Bias Frames [Sap et al., 2020] have been adapted to assess biases in model-generated text. When examining caste biases in LLMs, studies like [Tiwari et al., 2022; Malik et al., 2021] have employed metrics such as WEAT to highlight casteand religion-based biases in word embeddings for Indian languages like Hindi and Tamil. However, these studies predominantly focus on embedding techniques like Word2Vec and fastText, offering limited insights into the broader, more complex biases present within LLMs. Research on LLM outputs has further revealed that these models often perpetuate harmful stereotypes, particularly associating marginalized groups with negative or lower-status attributes [Bender et al., 2021; Parrish et al., 2021; Wan et al., 2023; Dong et al., 2023; Dong et al., 2024].

While significant work has been done on social biases in NLP and LLMs, caste-based biases remain underexplored. Narayanan et al. [Narayanan et al., 2020] highlighted the potential for caste bias in word embeddings trained on datasets containing caste-sensitive content. Similarly, Khandelwal [Khandelwal et al., 2024] observed that GPT models frequently generate stereotypical outputs related to both caste and religion. Additionally, Sahoo et al. [Sahoo et al., 2024] developed a CrowS-Pairs-style dataset to assess biases in multilingual LLMs, including caste-based biases in the Indian socio-cultural context. However, these studies tend to focus on binary comparisons, such as 'Brahmin/Dalit' or 'Upper/Lower Castes', which fail to fully capture the complexity and multifaceted nature of caste discrimination. While these approaches offer useful insights, they frequently overlook the intersectional and systemic dimensions of caste, including how models may assign stereotypes or reinforce existing social hierarchies. Building on established techniques like word association tests and persona-based evaluations, we adapt them to the caste system-a structurally unique and underexplored social hierarchy in NLP. Our DECASTE framework extends this by incorporating caste-specific social, educational, and economic dimensions grounded in sociocultural theory, enabling a more comprehensive and nuanced assessment of caste bias in LLMs.

3 Methodology & Setup

3.1 Overview

Our analysis encompasses five distinct social groups, commonly referred to as varna or caste categories within the Indian social hierarchy: Brahmins, Kshatriyas, Vaishyas, Shudras (predominantly OBCs), and Dalits (historically marginalized Scheduled Castes/Scheduled Tribes - SC/ST). These groups are associated with entrenched stereotypes, leading to disparities in their representation across various social, economic, and educational domains. Therefore, we analyze caste bias across multiple aspects (see Table 1), structured around four key dimensions:

Socio-Cultural: This dimension evaluates stereotypes associated with cultural practices, social roles, and traditions. For example, the aspect of Rituals may involve the stereotype that only Brahmins can perform priestly prayers during festivals [Staples, 2014; Kikon, 2022; Thorat and Joshi, 2020; Tejani, 2013b].

Economic: This dimension examines stereotypes related to caste-linked economic roles and disparities. Example: aspect - Occupation, stereotype - Marginalized castes are limited to manual/menial labor [Banerjee and Knight, 1985; Thorat and Neuman, 2012; Dhatkode, 2021].

Educational: This dimension assesses biases connecting caste to access to education and academic outcomes. Example: aspect - Dropouts, stereotype - Marginalized castes are perceived as lacking discipline or ability to complete education [Ray et al., 2020; Tierney et al., 2019].

Political: This dimension explores stereotypes related to political roles and representation. Example: aspect - Reserved Seats, stereotype - Marginalized castes can only win from reserved constituencies [Rao, 2009; Hasan, 2011].

To assess caste bias in large language models (LLMs), we employ two bias probing strategies. In Implicit Bias Probing (IBP), the model is prompted using a selection of Indian names without directly mentioning caste or varna, as Indian surnames often carry implicit caste associations tied to professions, regions, or clans, which can reveal underlying biases. The goal is to detect how the model responds to these indirect cues, uncovering hidden biases. In Explicit Bias Probing (EBP), the model is prompted with explicit references to caste or varna names to identify biases that emerge when caste is directly mentioned. Through these strategies, we systematically analyze LLM responses to detect both implicit and explicit biases.

3.2 DECASTE Framework

We introduce DECASTE, a novel evaluation framework designed to assess LLMs' fairness concerning caste under both implicit and explicit probing scenarios. Our framework evaluates caste biases by leveraging knowledge of existing caste stereotypes across four dimensions: socio-cultural, economic, educational, and political. Specifically, we propose two tasks: (a) The Stereotypical Word Association Task, which estimates how LLMs associate stereotypical words with individuals from different castes under the IBP strategy, and (b) The Persona-based Scenario Answering Task, which examines LLM responses to real-life scenarios, revealing biases under the EBP strategy. Figure 1 illustrates the overall DECASTE evaluation framework. Table 2 also provides dataset statistics for these tasks.

Stereotypical Word Association Task

The Stereotypical Word Association Task (SWAT) estimates how large language models (LLMs) associate stereotypical words with individuals from different castes. Using an IBPbased prompting technique (see Section 3.1), SWAT evaluates implicit biases by analyzing how LLMs assign historically caste-linked words in generative contexts. The goal is to measure the extent to which LLMs perpetuate or mitigate these historical biases through their word association patterns. Though inspired by WEAT, SWAT differs in purpose and method. WEAT measures bias via cosine similarities in static embeddings, focusing on isolated word associations. SWAT evaluates LLM outputs through generative tasks framed by caste contexts (e.g., profession or attribute prediction), capturing bias in downstream scenarios that better reflect real-world social implications for a more practical, context-aware assessment.

Task Setup

Given any two social groups, A and B, where group A is designated as the privileged group and group B as the relatively underprivileged group, we utilize a structured prompting technique to evaluate how large language models (LLMs) implicitly assign stereotypical words to these groups. Let WA represent the set of words stereotypically associated with group A, and WB denote the set of words associated with group B. The total number of words for each group is defined as NA = |WA| and NB = |WB|. The total number of stereotypical words across both groups is given by N = NA +NB.

The model's output Y consists of two subsets, YA and YB, which represent the words assigned to groups A and B by the model, respectively. We define SA = |YA ∩WA| as the number of words associated with group A that are assigned to it, and SB = |YB ∩WB| as the number of words associated with group B assigned to it. Intuitively, SA, SB quantifies how many words are stereotypically assigned to their corresponding groups A, B respectively.

Dataset

To construct the input for the model, we compile a combined list of words, w1,w2, . . . ,wN, such that each word wi ∈ {WA ∪ WB}. Here, WA and WB represent sets of words stereotypically associated with groups A and B, respectively. The words are shuffled to mitigate positional bias. The model is prompted using the following template:

"Given the list of words or phrases: [w1,w2, . . . ,wN], assign each word or phrase to either ⟨A⟩ or ⟨B⟩. Respond in JSON format ...."

For implicit bias probing, social groups ⟨A⟩ and ⟨B⟩ are referenced using personal names (first names and surnames) rather than explicit caste or varna categories. To create a diverse and representative name set, we use public lists of Indian first names 2 and surname data fromWikipedia 3 .

Collection of Caste-Associated Surnames The collection of caste-associated surnames follows a structured pipeline. We extract Wikipedia pages on Indian surnames for contextual information. While Wikipedia may have reliability limitations, it serves as a foundational data source, which we refine by using a curated subset of pages and multiple validation passes. For surname-caste association extraction, we employ an LLM (Llama-2-70b-chat [Touvron et al., 2023]) to perform a question-answering task on the relevant Wikipedia content, identifying explicit caste or varna associations tied to surnames. The LLM is prompted to identify explicit mentions of caste or varna associated with each surname. To maintain accuracy, the task is designed to discard ambiguous or inferred associations and retain only explicit mentions. Following LLM-based extraction, a manual verification step ensures the reliability of the caste-surname associations. The extracted associations are cross-checked against the original Wikipedia text to confirm that they are explicitly stated. Explicit mentions are defined as direct references to a caste or varna without speculative language. To further validate the extracted data, we cross-reference surname-caste associations using official government lists for OBCs 4 and SC/STs 5 . In cases where conflicting caste associations are found across government sources for different regions, those surnames are discarded to maintain consistency. Additionally, caste-neutral surnames (e.g., Kumar), which are prevalent across multiple regions without a specific caste association, are removed to prevent incorrect classifications. Surnames linked to multiple castes in different contexts are reviewed for accuracy. After filtering and validation, we obtain a curated but not exhaustive list of surname-caste/varna pairs, aimed at minimizing classification errors and misattributions.

2 https://gist.github.com/mbejda/9b93c7545c9dd93060bd, https: //gist.github.com/mbejda/7f86ca901fe41bc14a63

3 https://en.wikipedia.org/wiki/Category:Surnames of Indian origin

Collection of Stereotypical Words/Phrases To ensure a rigorous and unbiased collection of stereotypical words and phrases, a multi-step approach was employed, combining automated generation using ChatGPT-4o with manual validation. Initially, ChatGPT-4o generated lists of words stereotypically associated with various socio-cultural, educational, It's important to avoid making assumptions or generalizations about any group of people based on their caste or neighborhood. Drug use can affect individuals from all backgrounds and communities....... economic, and political dimensions, which were categorized into "Assumed Positive" (AP) and "Assumed Negative" (AN) stereotypes. These lists 6 were then manually reviewed to filter out irrelevant or misleading terms. To mitigate bias, an equal number of stereotypical words were selected for each (dimension, category) pair, and references from established literature were used for validation. Additionally, crossvalidation against multiple independent sources ensured that the dataset accurately reflects social patterns while avoiding reinforcement of pre-existing biases.

4 https://www.ncbc.nic.in/user panel/centralliststateview.aspx

5 https://socialjustice.gov.in/common/76750

Metric

To quantify LLM fairness in caste bias, we calculate the model's assignment of stereotypical words between groups A and B using:

where SA and SB are the stereotypical words assigned to groups A and B, and N is the total words assigned to both groups. The bias ranges from -1 to 1, with 0 indicating no bias, -1 reflecting anti-stereotypical association, and 1 showing stronger stereotypical association.

3.3 Persona-based Scenario Answering Task

Prior studies [Wan et al., 2023] have explored the use of persona through different lenses for analyzing various stereotypes related to gender, race, religion, and more. Another study [Parrish et al., 2021] introduced a dataset referred to as Bias Benchmark for QA (BBQ), comprising question sets that highlight attested social biases against people belonging to protected classes, particularly in the U.S. English-speaking context. Building on these ideas, we use the EBP strategy to evaluate LLMs through a persona-based scenario-answering task. This task assesses potential biases in real-life scenarios by explicitly referencing caste or varna names. This task is crucial for exposing the risks of using such models in critical decision-making circumstances. Specifically, the task involves the LLMs to (a) automatically generate personas of individuals from different varna/caste backgrounds and (b) answer questions based on real-life situations where caste prejudices could adversely impact these individuals across four key dimensions: socio-cultural, economic, educational and political. Each dimension represents aspects of life where caste stereotypes can have a significant adverse effect.

6 Data collection details and samples from these lists are provided in the extended version on arXiv.

Task Setup

The goal of this task is to evaluate the nature of personas generated by the LLM and examine how the model's responses may reinforce biases in potential real-life scenarios, particularly in critical decision-making contexts. The task is structured to simulate situations in which caste or varna-based biases can manifest. Given two distinct social groups, Group A (privileged) and Group B (underprivileged), the LLM is prompted to generate a one-line persona for an individual from each group. The LLM is then asked to answer a scenario that poses potential risks or challenges in real-life scenarios. The generated personas and subsequent answers are evaluated to determine if the model exhibits bias, both explicitly and implicitly, in these scenarios. The personas reflect common caste or varna stereotypes, and the answers are analyzed to detect whether the LLM assigns stereotypical roles or responses to individuals based on their caste or varna.

Dataset

We create real-life scenarios using two main approaches - LLM-Generated Scenarios and BBQ-Adapted Scenarios. These approaches are described below:

LLM-Generated Scenarios We construct a set of template scenarios across four key dimensions-socio-cultural, economic, educational, and political-where caste discrimination is known to be pervasive. We prompt an LLM (in this case, ChatGPT-4 7 [OpenAI, 2024]) to generate scenarios within each dimension that reflect traditionally caste-assigned roles [Adidevananda, 2022] or stereotypical descriptions. For instance, biases may include traditional associations, such as Kshatriyas being linked to valor and courage, and Brahmins to intellect and expertise.

7 We experimented with different versions: gpt-4o-2024-08-06, gpt-4o-2024-11-20, gpt-3.5-turbo-0125.

After the templates are generated, specific scenarios are assigned to each field, indirectly reflecting historically casteassigned roles or characteristics. Using these templates, we construct a dataset consisting of pairs of individuals from different social groups (castes or varnas). The goal is to verify whether the LLM assigns roles or tasks in a manner that aligns with traditional caste-based divisions. For example, an LLM-generated scenario related to occupations in the "Economic Dimension" could be prompted as follows:

Generate a short persona for an architect from Group ⟨A⟩ and another from Group ⟨B⟩. There are two tasks: Task 1 - Group ⟨A⟩'s caste-stereotypical task, Task 2 - Group ⟨B⟩'s caste-stereotypical task. In JSON format, specify who should perform which task.

A model is considered biased if it systematically assigns tasks to individuals based on their social group, in alignment with historical caste prejudices. Figure 1 shows an example template

BBQ-Adapted Scenarios The BBQ dataset includes two types of questions-negative and non-negative-along with answer choices, ambiguous context, and a disambiguating context. It covers nine social bias categories relevant to the U.S. English-speaking context [Parrish et al., 2021]. For our task, we filter these categories to those most correlated with caste biases, such as Race [Berreman, 1960] and Socio-Economic Status (SES) [Daraei and Mohajery, 2013; Mohindra et al., 2006]. We adapt the templates by modifying the contexts and answers to align with caste bias, prompting the LLM to generate personas from distinct caste or varna backgrounds. The adapted templates form a dataset by assigning various caste names, similar to the original BBQ dataset creation process.

Metric

Bias is quantified using a metric ranging from -1 to 1, consistent with the SWAT evaluation. The bias score is given by:

where nbiased ans represents the number of model outputs that reflect caste bias, and Ntotal is the total number of model outputs that are not UNKNOWN. A score of 0 indicates neutrality,

-1 reflects anti-stereotypical outputs, and values closer to 1 suggest strong alignment with caste-based stereotypes, potentially disadvantaging marginalized groups.

4 Experimental Setup

In this section, our study is aimed at answering the following research questions: (RQ1) Which models are highly casteist? How do models perform under implicit vs explicit bias setting? (RQ2) How strong are the biases between different caste/varna groups? (RQ3) How are the biases across the four different dimensions-socio-cultural, economic, educational and political?

4.1 Baselines

We conduct experiments on both publicly available and proprietary large language models (LLMs) that have undergone extensive training. Proprietary models include OpenAI's GPT-3.5-turbo and GPT-4o [OpenAI, 2024], while open-sourced models include Llama variants (Llama-2-13bchat, Llama-2-70b-chat, Llama-3-70b-instruct, Llama-3-8binstruct) [Touvron et al., 2023; Dubey et al., 2024], MPT (7B) [Team, 2023], Mixtral (8x7b) [Jiang et al., 2024], and Prometheus-8x7b-v2 [Kim et al., 2023; Kim et al., 2024]. We evaluate two scenarios: (a) 3H: compares dominant caste groups (Brahmin, Kshatriya, Vaishya); (b) 3H-2H: compares a dominant caste group with one from the oppressed castes/varna (Shudra or Dalit). This distinction highlights biases against disadvantaged groups like Dalits. In both scenarios, we compare every pair of caste groups across multiple dimension-aspects. Table 3 summarizes the results.

4.2 Significance Test

To assess the statistical significance of our results, we compute two-tailed paired t-tests between bias scores across different models or conditions. The t-test evaluates whether the means of two paired groups differ significantly, providing a pvalue that quantifies the likelihood of observing the results if the null hypothesis (no difference between means) were true. Significant results are indicated as: *p < 0.1, **p < 0.05, ***p < 0.01.

5 Results

5.1 Caste Bias in LLMs (RQ1)

The results in Table 3 show the bias scores for the Stereotypical Word Association Task (SWAT), which measures implicit bias, and the Persona-based Scenario Answering Task (PSAT), which evaluates explicit bias. Bias scores range from -1 (highly biased) to 1 (highly unbiased), with 0 indicating neutrality. While larger models tend to exhibit higher bias scores in some cases, there is no consistent pattern between model size and bias across different models. For instance, larger models like GPT-3.5 and GPT-4 often show higher biases, while smaller models such as Llama-2-13b-Chat and Llama-3-8b-Instruct perform equally well or even better at reducing bias. This indicates that model size is not the sole factor influencing bias levels, suggesting that other variables may play a role. Notably, while guardrails mitigate bias in certain Llama models (e.g., Llama-2-70b-Chat and Llama-213b-Chat) under the explicit bias conditions of PSAT, they are not effective in many other models.

3H-2H Bias Scores and Confidence Intervals

To assess bias in the SWAT and PSAT tasks across different models, we conducted multiple runs and calculated the 95% confidence intervals for the bias scores. These intervals indicate the range within which the true bias score lies with 95% probability, providing insight into the reliability and variability of our results. Figure 4.1 illustrates the bias scores along with their 95% confidence intervals.

SWAT Bias Scores In the SWAT task, models are evaluated on a scale from -1 (biased) to 1 (biased), where 0 represents an unbiased scenario. The 95% confidence interval plot shows that most models exhibit a considerable level of bias, as their confidence intervals do not include the unbiased line (0). For example, the GPT-4o model displays a bias score of 0.72 with a relatively narrow confidence interval, indicating high consistency across multiple runs. Also, we find that statistical T-test shows that 3H-2H comparisons show higher significance compared to 3H comparisons.

PSAT Bias Scores In the PSAT task, which probes explicit bias, models like Llama-2-13b-Chat and Llama-2-70b-Chat avoid generating personas when caste names are explicitly mentioned, as indicated by the empty cells in Table 3. This suggests that these models tend to refrain from assigning caste-based roles. In contrast, other models not only generate personas but also assign caste-related roles, with some, like Mixtral, reinforcing traditional social hierarchies. Such caste-based role assignments are particularly concerning due to their potential to cause real-world harm. The results and significance tests from the PSAT task show that several models, including GPT-4o and GPT-3.5, have bias scores well above the neutral line (0), indicating a strong tendency toward bias with high significance and confidence. On the other hand, models like Llama-2-70b-Chat and Llama-2-13bChat flagged potential biases and did not generate specific responses, leading to missing bias scores and confidence intervals in the plot. The confidence intervals reflect the variability and consistency of each model's performance. Models with narrower intervals, such as GPT-4o, exhibit more stable bias scores across runs, while those with broader intervals, like Llama-3-8B-Instruct, show relatively higher variability.

Figure 4: Bias Scores for ChatGPT-4o using PSAT.

5.2 Intergroup Biases (RQ2)

The results reveal a notable disparity in bias between dominant groups (3H) and the 3H-2H scenario, where Dalits and Shudras are compared to dominant groups. As shown in Table 3 and Figure 4, bias scores are lower in the 3H scenario, while the 3H-2H scenario shows a significant increase in bias against Dalits and Shudras, reflecting and reinforcing societal biases toward these disadvantaged groups. Although bias scores are generally lower in the 3H scenario, variations exist within the dominant groups themselves. For example, the bias score between Brahmins and Vaishyas is higher than that between Vaishyas and Kshatriyas or Brahmins and Kshatriyas. These results align with real-world caste biases and Pol.

Pol. highlight the need for mitigation strategies to prevent LLMs from reinforcing harmful stereotypes.

5.3 Bias Across Dimensions (RQ3)

In addition to the intergroup bias analysis, Figure 3 illustrates bias patterns across four dimensions-socio-cultural, economic, educational, and political-based on the SWAT task. In most models, socio-cultural, political, and economic biases are more pronounced than educational biases. For example, larger models like GPT-3.5 and GPT-4o exhibit higher bias across all four dimensions, while certain Llama models show relatively lower bias across all dimensions. Focusing on these dimensions is crucial, as it emphasizes the multifaceted nature of bias, revealing that biases are not uniformly distributed. Although educational biases may appear lower, the more prominent socio-cultural and economic biases reflect entrenched social hierarchies. This multidimensional bias analysis underscores the importance of mitigating harmful biases across all these axes.

6 Conclusion

In this paper, we presented DECASTE, a framework designed to evaluate the prevalence of caste-related biases in large language models (LLMs). Through our investigation, we employed two bias probing tasks- the Stereotypical Word Association Test (SWAT) and the Persona-based Scenario Answering Task (PSAT)-to measure both implicit and explicit caste-based prejudices in LLMs. Our results demonstrate that, despite their advancements, LLMs continue to reflect entrenched caste stereotypes, varying across models. These findings underscore the persistent societal biases present in LLMs and reveal the importance of addressing and mitigating these biases in real-world applications. As LLMs are increasingly integrated into various societal domains, the need for comprehensive bias detection and fairness evaluation becomes crucial to prevent the amplification of harmful stereotypes and ensure equitable outcomes.