Proceedings

Exploring Multimodal Foundation AI and Expert-in-the-Loop for Sustainable Management of Wild Salmon Fisheries in Indigenous Rivers

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. AI and Social Good. Pages 9954-9962.

https://doi.org/10.24963/ijcai.2025/1106

Abstract

Wild salmon are essential to the ecological, economic, and cultural sustainability of the North Pacific Rim. Yet climate variability, habitat loss, and data limitations in remote ecosystems that lack basic infrastructure support pose significant challenges to effective fisheries management. This project explores the integration of multimodal foundation AI and expert-in-the-loop frameworks to enhance wild salmon monitoring and sustainable fisheries management in Indigenous rivers across Pacific Northwest. By leveraging video and sonar-based monitoring, we develop AI-powered tools for automated species identification, counting, and length measurement, reducing manual effort, expediting delivery of results, and improving decision-making accuracy. Expert validation and active learning frameworks ensure ecological relevance while reducing annotation burdens. To address unique technical and societal challenges, we bring together a cross-domain, interdisciplinary team of university researchers, fisheries biologists, Indigenous stewardship practitioners, government agencies, and conservation organizations. Through these collaborations, our research fosters ethical AI co-development, open data sharing, and culturally informed fisheries management.

  1. [Achiam et al., 2023] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Alt- man, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
  2. [Atlas et al., 2021] William I Atlas, Natalie C Ban, Jonathan W Moore, Adrian M Tuohy, Spencer Greening, Andrea J Reid, Nicole Morven, Elroy White, William G Housty, Jess A Housty, et al. Indigenous systems of management for culturally and ecologically resilient pacific salmon (oncorhynchus spp.) fisheries. BioScience, 71(2):186-204, 2021.
  3. [Atlas et al., 2023] William I Atlas, Sami Ma, Yi Ching Chou, Katrina Connors, Daniel Scurfield, Brandon Nam, Xiaoqiang Ma, Mark Cleveland, Janvier Doire, Jonathan W Moore, et al. Wild salmon enumeration and monitoring using deep learning empowered detection and tracking. Frontiers in Marine Science, 10:1200408, 2023.
  4. [Bernardin and Stiefelhagen, 2008] Keni Bernardin and Rainer Stiefelhagen. Evaluating multiple object tracking performance: The clear mot metrics. EURASIP Journal on Image and Video Processing, 2008:1-10, 2008.
  5. [Carothers et al., 2021] Courtney Carothers, Jessica Black, Stephen J Langdon, Rachel Donkersloot, Danielle Ringer, Jesse Coleman, Erika R Gavenus, Wilson Justin, Mike Williams, et al. Indigenous peoples and salmon steward- ship: a critical relationship. 26(1), 2021.
  6. [Chihaoui and Favaro, 2024] Hamadi Chihaoui and Paolo Favaro. Masked and shuffled blind spot denoising for real- world images. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 3025-3034, 2024.
  7. [Cui et al., 2019] Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie. Class-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9268-9277, 2019.
  8. [Di Lorenzo and Mantua, 2016] Emanuele Di Lorenzo and Nathan Mantua. Multi-year persistence of the 2014/15 north pacific marine heatwave. Nature Climate Change, 6(11):1042-1047, 2016.
  9. [Dorner et al., 2018] Brigitte Dorner, Matthew J Catalano, and Randall M Peterman. Spatial and temporal patterns of covariation in productivity of chinook salmon populations of the northeastern pacific ocean. Canadian Journal of Fisheries and Aquatic Sciences, 75(7):1082-1095, 2018.
  10. [Earth Economies, 2021] Earth Economies. The sociocul- tural significance of pacific salmon to tribes and first na- tions. Special Report to the Pacific Salmon Commission, 2021.
  11. [Frölicher and Laufkötter, 2018] Thomas L Frölicher and Charlotte Laufkötter. Emerging risks from marine heat waves. Nature communications, 9(1):650, 2018.
  12. [Garber and Tirer, 2024] Tomer Garber and Tom Tirer. Im- age restoration by denoising diffusion models with iter- atively preconditioned guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 25245-25254, 2024.
  13. [Goodfellow, 2016] Ian Goodfellow. Deep learning, volume 196. MIT press, 2016.
  14. [Gordon et al., 2023] Lucia Gordon, Nikhil Behari, Samuel Collier, Elizabeth Bondi-Kelly, Jackson A. Killian, Catherine Ressijac, Peter Boucher, Andrew Davies, and Milind Tambe. Find rhinos without finding rhinos: Active learning with multimodal imagery of south african rhino habitats. In Proceedings of the Thirty-Second Interna- tional Joint Conference on Artificial Intelligence, IJCAI, pages 5977-5985, 2023. IJCAI
  15. [Hao et al., 2015] Mingming Hao, Helong Yu, and Daoliang Li. The measurement of fish size by machine vision-a re- view. In International Conference on Computer and Com- puting Technologies in Agriculture, pages 15-32. Springer, 2015.
  16. [Kay et al., 2022] Justin Kay, Peter Kulits, Suzanne Stathatos, Siqi Deng, Erik Young, Sara Beery, Grant Van Horn, and Pietro Perona. The caltech fish counting dataset: a benchmark for multiple-object tracking and counting. In European Conference on Computer Vision, pages 290-311. Springer, 2022.
  17. [Kay et al., 2024] Justin Kay, Timm Haucke, Suzanne Stathatos, Siqi Deng, Erik Young, Pietro Perona, Sara Beery, and Grant Van Horn. Align and distill: Unifying and improving domain adaptive object detection. arXiv preprint arXiv:2403.12029, 2024.
  18. [Khan et al., 2023] Faizan Farooq Khan, Xiang Li, Andrew J Temple, and Mohamed Elhoseiny. Fishnet: A large- scale dataset and benchmark for fish recognition, detec- tion, and functional trait prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20496-20506, 2023.
  19. [Kilduff et al., 2015] D Patrick Kilduff, Emanuele Di Lorenzo, Louis W Botsford, and Steven LH Teo. Changing central pacific el niños reduce stability of north american salmon survival rates. Proceedings of the National Academy of Sciences, 112(35):10962-10966, 2015.
  20. [Kshitiz et al., 2023] Kshitiz, Sonu Shreshtha, Ramy Mounir, Mayank Vatsa, Richa Singh, Saket Anand, Sudeep Sarkar, and Sevaram Mali Parihar. Long-term monitoring of bird flocks in the wild. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI, pages 6344-6352, 2023. IJCAI
  21. [Liu et al., 2020] Yang Liu, Anbu Huang, Yun Luo, He Huang, Youzhi Liu, Yuanyuan Chen, Lican Feng, Tian- jian Chen, Han Yu, and Qiang Yang. Fedvision: An on- line visual object detection platform powered by federated learning. In Proceedings of the AAAI conference on artifi- cial intelligence, volume 34, pages 13172-13179, 2020.
  22. [Liu et al., 2023] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023.
  23. [Liu et al., 2024] Weizhen Liu, Jiayu Tan, Guangyu Lan, Ao Li, Dongye Li, Le Zhao, Xiaohui Yuan, and Nanqing Dong. Benchmarking fish dataset and evaluation metric in keypoint detection -towards precise fish morphological assessment in aquaculture breeding. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI, pages 7376-7384, 2024. IJCAI
  24. [Luiten et al., 2021] Jonathon Luiten, Aljosa Osep, Patrick Dendorfer, Philip HS Torr, Laura Leal-Taixé, and Bastian Leibe. Hota: A higher order metric for evaluating multi- object tracking. International Journal of Computer Vision, 129(2):548-578, 2021.
  25. [Ma et al., 2024] Sami Ma, Yi Ching Chou, Miao Zhang, Hao Fang, Haoyuan Zhao, Jiangchuan Liu, and William I Atlas. Leo satellite network access in the wild: Potentials, experiences, and challenges. IEEE Network, 2024.
  26. [Moore et al., 2021] Jonathan W Moore, Brendan M Con- nors, and Emma E Hodgson. Conservation risks and port- folio effects in mixed-stock fisheries. Fish and Fisheries, 22(5):1024-1040, 2021.
  27. [OpenAI, 2024] OpenAI. Learning to reason with llms. https://openai.com/index/learning-to-reason-with-llms/, 2024. [Accessed: 2024-10-04]. Link
  28. [Price et al., 2017] Michael HH Price, Karl K English, An- drew G Rosenberger, Misty MacDuffee, and John D Reynolds. Canada's wild salmon policy: an assessment of conservation progress in british columbia. Canadian Jour- nal of Fisheries and Aquatic Sciences, 74(10):1507-1518, 2017.
  29. [Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from nat- ural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.
  30. [Ravi et al., 2024] Nikhila Ravi, Valentin Gabeur, Yuan- Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024.
  31. [Ristani et al., 2016] Ergys Ristani, Francesco Solera, Roger Zou, Rita Cucchiara, and Carlo Tomasi. Performance mea- sures and a data set for multi-target, multi-camera track- ing. In European Conference on Computer Vision (ECCV), pages 17-35. Springer, 2016.
  32. [Schindler and Hilborn, 2015] Daniel E Schindler and Ray Hilborn. Prediction, precaution, and policy under global change. Science, 347(6225):953-954, 2015.
  33. [United Nations, 2015] United Nations. Transforming our world: The 2030 agenda for sustainable development. https://sdgs.un.org/2030agenda, 2015. Adopted by the United Nations General Assembly on 25 September 2015. [Accessed: 2025-05-21]. Link
  34. [Walsh et al., 2020] Jessica C Walsh, Jane E Pendray, Sean C Godwin, Kyle A Artelle, Holly K Kindsvater, Rachel D Field, Jennifer N Harding, Noel R Swain, and John D Reynolds. Relationships between pacific salmon and aquatic and terrestrial ecosystems: implications for ecosystem-based management. Ecology, 101(9):e03060, 2020.
  35. [Walters et al., 2008] Carl J Walters, JA Lichatowich, Ran- dall M Peterman, and John D Reynolds. Report of the skeena independent science review panel. A report to the Canadian Department of Fisheries and Oceans and the British Columbia Ministry of the Environment, 15, 2008.
  36. [Wang et al., 2024] Ao Wang, Hui Chen, Lihao Liu, Kai Chen, Zijia Lin, Jungong Han, and Guiguang Ding. Yolov10: Real-time end-to-end object detection. arXiv preprint arXiv:2405.14458, 2024.
  37. [Waples et al., 2008] Robin S Waples, George R Pess, and Tim Beechie. Evolutionary history of pacific salmon in dynamic environments. Evolutionary Applications, 1(2):189-206, 2008.
  38. [Wojke et al., 2017] Nicolai Wojke, Alex Bewley, and Diet- rich Paulus. Simple online and realtime tracking with a deep association metric. In International Conference on Image Processing (ICIP), pages 3645-3649. IEEE, 2017.
  39. [Wu et al., 2022] Xingjiao Wu, Luwei Xiao, Yixuan Sun, Junhang Zhang, Tianlong Ma, and Liang He. A survey of human-in-the-loop for machine learning. Future Gen- eration Computer Systems, 135:364-381, 2022.
  40. [Xu et al., 2024] Chi Xu, Rongsheng Qian, Hao Fang, Xiao- qiang Ma, William I Atlas, Jiangchuan Liu, and Mark A Spoljaric. Salina: Towards sustainable live sonar analytics in wild ecosystems. In Proceedings of the 22nd ACM Con- ference on Embedded Networked Sensor Systems, pages 68-81, 2024.
  41. [Yoshiyama, 1999] Ronald M Yoshiyama. A history of salmon and people in the central valley region of califor- nia. Reviews in Fisheries Science, 7(3-4):197-239, 1999.
  42. [Zhao et al., 2024] Yian Zhao, Wenyu Lv, Shangliang Xu, Jinman Wei, Guanzhong Wang, Qingqing Dang, Yi Liu, and Jie Chen. Detrs beat yolos on real-time object de- tection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16965- 16974, 2024.
  43. [Zhong et al., 2024] Tianyang Zhong, Zhengliang Liu, Yi Pan, Yutong Zhang, Yifan Zhou, Shizhe Liang, Zihao Wu, Yanjun Lyu, Peng Shu, Xiaowei Yu, et al. Evaluation of openai o1: Opportunities and challenges of agi. arXiv preprint arXiv:2409.18486, 2024.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards