Proceedings

Price of Fairness in Short-Term and Long-Term Algorithmic Selections

Download BibTeX Cite

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. Main Track. Pages 635-643.

https://doi.org/10.24963/ijcai.2026/72

Abstract

Algorithmic decision-making in high-stakes settings can have profound impacts on individuals and populations. While much prior work studies fairness in static settings, recent results show that enforcing static fairness constraints may exacerbate long-run disparities. Motivated by this tension, we study a stylized sequential selection problem in which a decision-maker repeatedly selects individuals, affecting both immediate utility and the population distribution over time. We introduce notions of group fairness for both the short and long term and theoretically analyze the trade-off between fairness and utility via the Price of Fairness (PoF). We characterize optimal and fair policies in the short term and show that the PoF can be large even when group distributions are nearly identical. In contrast, we show that long-term disparities can vanish under simple investment policies that achieve a low PoF. We also empirically validate these theoretical observations using both synthetic and real datasets.

  1. [Acharya et al., 2023] Krishna Acharya, Eshwar Ram Arunachaleswaran, Sampath Kannan, Aaron Roth, and Juba Ziani. Wealth dynamics over generations: Analysis and interventions. In SATML, pages 42-57, 2023.
  2. [Achiam et al., 2017] Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. Constrained policy optimization. In ICML, pages 22-31, 2017.
  3. [Altman, 1999] Eitan Altman. Constrained Markov Decision Processes. CRC Press, 1999.
  4. [Arunachaleswaran et al., 2021] Eshwar Arunachaleswaran, Sampath Kannan, Aaron Roth, and Juba Ziani. Pipeline interventions. In ITCS, pages 8:1-8:20, 2021.
  5. [Barocas et al., 2019] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and Machine Learning: Limi- tations and Opportunities. fairmlbook.org, 2019.
  6. [Berk et al., 2021] Richard Berk, Hoda Heidari, Shahin Jab- bari, Michael Kearns, and Aaron Roth. Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research, 50(1):3-44, 2021.
  7. [Bertsimas et al., 2011] Dimitris Bertsimas, Vivek Farias, and Nikolaos Trichakis. The price of fairness. Operations Research, 59(1):17-31, 2011.
  8. [Board of Governors of the Federal Reserve System, 2007] Board of Governors of the Federal Reserve System. Report to the congress on credit scoring and its effects on the availability and affordability of credit, 2007.
  9. [Calders and Verwer, 2010] Toon Calders and Sicco Verwer. Three naive bayes approaches for discrimination-free classification. Data Mining and Knowledge Discovery, 21(2):277-292, 2010.
  10. [Calvo-Fullana et al., 2024] Miguel Calvo-Fullana, Santiago Paternain, Luiz Chamon, and Alejandro Ribeiro. State aug- mented constrained reinforcement learning: Overcoming the limitations of learning with rewards. IEEE Transactions on Automatic Control, 69(7):4275-4290, 2024.
  11. [Celis et al., 2021] Elisa Celis, Anay Mehrotra, and Nisheeth Vishnoi. Fair classification with adversarial perturbations. In NeurIPS, pages 8158-8171, 2021.
  12. [Cheng et al., 2025] Zelei Cheng, Jiahao Yu, and Xinyu Xing. A survey on explainable deep reinforcement learning. CoRR, abs/2502.06869, 2025.
  13. [Chi et al., 2022] Jianfeng Chi, Jian Shen, Xinyi Dai, Weinan Zhang, Yuan Tian, and Han Zhao. Towards return parity in markov decision processes. In AISTATS, pages 1161-1178, 2022.
  14. [Corbett-Davies et al., 2017] Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algo- rithmic decision making and the cost of fairness. In KDD, pages 797-806. ACM, 2017.
  15. [Corbett-Davies et al., 2023] Sam Corbett-Davies, Johann D. Gaebler, Hamed Nilforoshan, Ravi Shroff, and Sharad Goel. The measure and mismeasure of fairness. Journal of Ma- chine Learning Research, 24:312:1-312:117, 2023.
  16. [Cousins et al., 2024] Cyrus Cousins, Kavosh Asadi, Elita Lobo, and Michael Littman. On welfare-centric fair re- inforcement learning. Reinforcement Learning Journal, 3:1124-1137, 2024.
  17. [D'Amour et al., 2020] Alexander D'Amour, Hansa Srini- vasan, James Atwood, Pallavi Baljekar, D. Sculley, and Yoni Halpern. Fairness is not static: Deeper understand- ing of long term fairness via simulation studies. In FAccT, pages 525-534, 2020.
  18. [Davis et al., 2021] Jenny Davis, Apryl Williams, and Michael Yang. Algorithmic reparation. Big Data & Society, 8(2), 2021.
  19. [Deng et al., 2023] Zhun Deng, He Sun, Steven Wu, Linjun Zhang, and David Parkes. Reinforcement learning with stepwise fairness constraints. In AISTATS, volume 206, pages 10594-10618, 2023.
  20. [Ding et al., 2020] Dongsheng Ding, Kaiqing Zhang, Tamer Basar, and Mihailo Jovanovic. Natural policy gradient primal-dual method for constrained markov decision pro- cesses. In NeurIPS, 2020.
  21. [Ding et al., 2021] Dongsheng Ding, Xiaohan Wei, Zhuoran Yang, Zhaoran Wang, and Mihailo Jovanovic. Provably ef- ficient safe exploration via primal-dual policy optimization. In AISTATS, pages 3304-3312, 2021.
  22. [Donahue and Kleinberg, 2020] Kate Donahue and Jon Kleinberg. Fairness and utilization in allocating resources with uncertain demand. In FAccT, pages 658-668, 2020.
  23. [Dong et al., 2018] Jinshuo Dong, Aaron Roth, Zachary Schutzman, Bo Waggoner, and Zhiwei Steven Wu. Strate- gic classification from revealed preferences. In EC, pages 55-70, 2018.
  24. [Doroudi et al., 2017] Shayan Doroudi, Philip Thomas, and Emma Brunskill. Importance sampling for fair policy se- lection. In UAI, 2017. IJCAI
  25. [Eaton et al., 2025] Eric Eaton, Marcel Hussing, Michael Kearns, Aaron Roth, Sikata Bela Sengupta, and Jessica Sor- rell. Intersectional fairness in reinforcement learning with large state and constraint spaces. CoRR, abs/2502.11828, 2025.
  26. [Elzayn et al., 2019] Hadi Elzayn, Shahin Jabbari, Christo- pher Jung, Michael Kearns, Seth Neel, Aaron Roth, and Zachary Schutzman. Fair algorithms for learning in alloca- tion problems. In FAccT, pages 170-179, 2019.
  27. [Ghalme et al., 2022] Ganesh Ghalme, Vineet Nair, Vishakha Patil, and Yilun Zhou. Long-term resource allocation fair- ness in average markov decision process (AMDP) environ- ment. In AAMAS, pages 525-533, 2022.
  28. [Hardt et al., 2016a] Moritz Hardt, Nimrod Megiddo, Chris- tos Papadimitriou, and Mary Wootters. Strategic classi- fication. In Madhu Sudan, editor, ITCS, pages 111-122, 2016.
  29. [Hardt et al., 2016b] Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity in supervised learning. In NeurIPS, pages 3315-3323, 2016.
  30. [Hébert-Johnson et al., 2018] Úrsula Hébert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. Multicalibration: Calibration for the (computationally- identifiable) masses. In ICML, pages 1944-1953, 2018.
  31. [Heidari and Kleinberg, 2021] Hoda Heidari and Jon Klein- berg. Allocating opportunities in a dynamic model of inter- generational mobility. In FAccT, pages 15-25, 2021.
  32. [Heidari et al., 2019a] Hoda Heidari, Michele Loi, Krishna Gummadi, and Andreas Krause. A moral framework for un- derstanding fair ML through economic models of equality of opportunity. In FAccT, pages 181-190, 2019.
  33. [Heidari et al., 2019b] Hoda Heidari, Vedant Nanda, and Kr- ishna Gummadi. On the long-term impact of algorithmic decision policies: Effort unfairness and feature segregation through social learning. In ICML, pages 2692-2701, 2019.
  34. [Hu and Zhang, 2022] Yaowei Hu and Lu Zhang. Achieving long-term fairness in sequential decision making. In AAAI, pages 9549-9557, 2022.
  35. [Jabbari et al., 2017] Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, and Aaron Roth. Fairness in reinforcement learning. In ICML, pages 1617-1626, 2017.
  36. [Joseph et al., 2016] Matthew Joseph, Michael Kearns, Jamie Morgenstern, and Aaron Roth. Fairness in learning: Classic and contextual bandits. In NeurIPS, pages 325-333, 2016.
  37. [Kearns et al., 2018] Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Wu. Preventing fairness gerrymandering: Au- diting and learning for subgroup fairness. In ICML, pages 2569-2577, 2018.
  38. [Kleinberg and Raghavan, 2019] Jon Kleinberg and Manish Raghavan. How do classifiers induce agents to invest effort strategically? In EC, pages 825-844, 2019.
  39. [Liu and Hajiesmaili, 2025] Qingsong Liu and Mohammad Hajiesmaili. Online fair allocation of reusable resources. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 9(2):1-46, 2025.
  40. [Liu et al., 2017] Yang Liu, Goran Radanovic, Christos Dim- itrakakis, Debmalya Mandal, and David C. Parkes. Cali- brated fairness in bandits. CoRR, abs/1707.01875, 2017.
  41. [Liu et al., 2018] Lydia Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. Delayed impact of fair machine learning. In ICML, pages 3156-3164, 2018. IJCAI
  42. [Liu et al., 2021] Tao Liu, Ruida Zhou, Dileep Kalathil, Pan- ganamala Kumar, and Chao Tian. Learning policies with zero or bounded constraint violation for constrained MDPs. In NeurIPS, pages 17183-17193, 2021.
  43. [Mansour et al., 2022] Yishay Mansour, Michal Moshkovitz, and Cynthia Rudin. There is no accuracy-interpretability tradeoff in reinforcement learning for mazes. CoRR, abs/2206.04266, 2022.
  44. [Menon and Williamson, 2018] Aditya Menon and Robert Williamson. The cost of fairness in binary classification. In FAccT, pages 107-118, 2018.
  45. [Mouzannar et al., 2019] Hussein Mouzannar, Mesrob Ohan- nessian, and Nathan Srebro. From fair decision making to social equality. In FAccT, pages 359-368, 2019.
  46. [Müller et al., 2024] Adrian Müller, Pragnya Alatur, Volkan Cevher, Giorgia Ramponi, and Niao He. Truly no-regret learning in constrained mdps. In ICML, 2024.
  47. [Rahmattalabi et al., 2021] Aida Rahmattalabi, Shahin Jab- bari, Himabindu Lakkaraju, Phebe Vayanos, Max Izenberg, Ryan Brown, Eric Rice, and Milind Tambe. Fair influence maximization: A welfare optimization approach. In AAAI, pages 11630-11638, 2021.
  48. [Rateike et al., 2024] Miriam Rateike, Isabel Valera, and Patrick Forré. Designing long-term group fair policies in dynamical systems. In FAccT, pages 20-50, 2024.
  49. [Reuel and Ma, 2024] Anka Reuel and Devin Ma. Fairness in reinforcement learning: A survey. In AIES, pages 1218- 1230. AAAI Press, 2024.
  50. [Salem et al., 2023] Tareq Si Salem, Georgios Iosifidis, and Giovanni Neglia. Enabling long-term fairness in dynamic resource allocation. In SIGMETRICS, pages 31-32, 2023.
  51. [Satija et al., 2023] Harsh Satija, Alessandro Lazaric, Matteo Pirotta, and Joelle Pineau. Group fairness in reinforcement learning. Transaction Machine Learning Research, 2023.
  52. [Silva et al., 2022] Andrew Silva, Rohit Chopra, and Matthew Gombolay. Cross-loss influence functions to ex- plain deep network representations. In AISTATS, volume 151, pages 1-17, 2022.
  53. [Sinha et al., 2023] Abhishek Sinha, Ativ Joshi, Rajarshi Bhattacharjee, Cameron Musco, and Mohammad Hajies- maili. No-regret algorithms for fair resource allocation. In NeurIPS, 2023.
  54. [Stoica et al., 2020] Ana-Andreea Stoica, Jessy Xinyi Han, and Augustin Chaintreau. Seeding network influence in biased networks and the benefits of diversity. In WWW, pages 2089-2098, 2020.
  55. [Tsang et al., 2019] Alan Tsang, Bryan Wilder, Eric Rice, Milind Tambe, and Yair Zick. Group-fairness in influence maximization. In IJCAI, pages 5997-6005, 2019. IJCAI
  56. [Wen and Topcu, 2018] Min Wen and Ufuk Topcu. Con- strained cross-entropy method for safe reinforcement learn- ing. In NeurIPS, pages 7461-7471, 2018.
  57. [Wen et al., 2021] Min Wen, Osbert Bastani, and Ufuk Topcu. Algorithms for fairness in sequential decision making. In AISTATS, pages 1144-1152, 2021.
  58. [Wu et al., 2025] Jiduan Wu, Rediet Abebe, Moritz Hardt, and Ana-Andreea Stoica. Policy design in long-run welfare dynamics. In ICLR, 2025.
  59. [Yin et al., 2023] Tongxin Yin, Reilly Raab, Mingyan Liu, and Yang Liu. Long-term fairness with unknown dynamics. CoRR, abs/2304.09362, 2023.
  60. [Zhang et al., 2020] Xueru Zhang, Ruibo Tu, Yang Liu, Mingyan Liu, Hedvig Kjellström, Kun Zhang, and Cheng Zhang. How do fair decisions fare in long-term qualifica- tion? In NeurIPS, 2020.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards