Uncertainty-Aware Policy Optimization for Robust Multi-Agent Decision Making under Partial Observability
DOI:
https://doi.org/10.66238/fsrma56Keywords:
Soft actor-critic, Uncertainty modeling, Partial observability, Multi-agent reinforcement learning, Robust decision-makingAbstract
Decision-making in partially observable environments introduces uncertainty that can degrade coordination performance. This study develops an uncertainty-aware reinforcement learning framework by integrating soft actor-critic (SAC) with entropy regularization and Bayesian value estimation. The approach explicitly models uncertainty in both state representation and reward estimation, enabling more stable policy updates. Evaluation is performed on 10,000 tasks with varying levels of observation noise and incomplete information, including negotiation and multi-stage planning scenarios. The proposed method improves success rate from 69.2% to 81.5% and reduces variance in cumulative reward by 27.4% compared to standard SAC. Additionally, calibration metrics indicate better alignment between predicted and actual outcomes. The results demonstrate that uncertainty-aware optimization enhances robustness in collaborative decision systems.
References
Oroojlooy, A., & Hajinezhad, D. (2023). A review of cooperative multi-agent deep reinforcement learning. Applied Intelligence, 53(11), 13677-13722.
Xu, D., Liu, H., Qiu, D., & Ma, Q. (2026). Structured Modeling and Representation Methods for Post-Retrieval Inference Processes in Large Video Language Models.
Lambrechts, G., Bolland, A., & Ernst, D. (2022). Recurrent networks, hidden states and beliefs in partially observable environments. arXiv preprint arXiv:2208.03520.
Liu, H., Xu, D., Ma, Q., Xu, S., & Qiu, D. (2026). Memory Poisoning Propagation and Repair Mechanism in Multi-Agent Collaborative Environments.
Messaoud, S., Mokeddem, B., Xue, Z., Pang, L., An, B., Chen, H., & Chawla, S. (2024). S $^ 2$ AC: Energy-Based Reinforcement Learning with Stein Soft Actor Critic. arXiv preprint arXiv:2405.00987.
Zhang, Y., Gu, W., & Wang, J. (2025). Research on First Article Inspection (FAI)-Driven Quality Assurance Methods for Wind Turbine Installation and Operation & Maintenance and Their Effect on Reliability Improvement. Available at SSRN 6094206.
Palminteri, S. (2025). Human reinforcement learning processes and biases: computational characterization and possible applications to behavioral public policy. Mind & Society, 1-25.
Xu, D., Gui, H., & Chen, H. (2026). Research on Layered Control and Fault Recovery Mechanisms for Fast Charging Safety Diagnosis of High Voltage Battery Systems Under Charging Network Interoperability Conditions.
Pozzi, A., Incremona, A., & Toti, D. (2025). Imitation learning-driven approximation of stochastic control models. Applied Intelligence, 55(12), 838.
Wang, Y., Chen, J., Wang, Y., & Yin, X. (2026). Application of Obtainable Biological Agent Characteristics in Efficacy Stratification of Oral Anti-Obesity Drugs.
Campagner, A., Biganzoli, E. M., Balsano, C., Cereda, C., & Cabitza, F. (2025). Modeling unknowns: A vision for uncertainty-aware machine learning in healthcare. International Journal of Medical Informatics, 203, 106014.
Zhang, Y., & Wang, J. (2026). Design and Implementation of a Computer-Aided Full Lifecycle Quality Management System for Wind Farms in Upgrades, Renovations, and Subcontractor Supervision.
Kucuksayacigil, F., Zhang, Z., & Davidson, M. R. (2025). Coordinating power sector climate transitions under policy uncertainty. Nature Communications, 16(1), 3786.
Jiao, Y., Wang, A., Zhao, B., & Shi, T. (2026). The Impact of Visual Language Strategies in Public Art Creation on Community Spatial Perception and Public Behavior.
Taghavi, M., & Vahidi, J. (2025). MARL-CC: A Mathematical Framework forMulti-Agent Reinforcement Learning in ConnectedAutonomous Vehicles: Addressing Nonlinearity, Partial Observability, and Credit Assignment forOptimal Control. arXiv preprint arXiv:2511.17653.
Gao, G., Ma, X., Lu, C., & Gao, R. (2026). Reliability Analysis and Application Research of SMS Communication Systems in Medical Notification Scenarios.
Shah, M. I. A., Barrett, E., & Mason, K. (2025). Uncertainty-aware knowledge transformers for peer-to-peer energy trading with multi-agent reinforcement learning. arXiv preprint arXiv:2507.16796.
Xu, D., Chen, H., & Gui, H. (2026). Unified Online Estimation Method for SOC, SOH, and Power Capacity Considering Safety Boundary Consistency in Battery Management Systems.
Anderson, D., DeLeo, R. A., Albright, E. A., Taylor, K., Birkland, T., Zhang, M., ... & Cage, C. (2023). Policy learning and change during crisis: COVID‐19 policy responses across six states. Review of Policy Research, 40(1), 10-35.
Jiao, Y., Wang, A., Zhao, B., & Shi, T. (2026). Quantitative Study on the Construction and Application Effectiveness of Graffiti Wall Painting Teaching Models in Public Space Contexts.
Springenberg, J. T., Abdolmaleki, A., Zhang, J., Groth, O., Bloesch, M., Lampe, T., ... & Riedmiller, M. (2024). Offline actor-critic reinforcement learning scales to large models. arXiv preprint arXiv:2402.05546.
Wang, Y., Yin, X., Chen, J., & Wang, Y. (2026). Evidence-Based Study on Low-Burden Digital Phenotyping for Precision Screening of Oral Anti-Obesity Drug Efficacy.
Schetinin, V., & Jakaite, L. (2025). Bayesian Learning Strategies for Reducing Uncertainty of Decision-Making in Case of Missing Values. Machine Learning and Knowledge Extraction, 7(3), 106.
Yue, L., Xu, D., Qiu, D., Shi, Y., Xu, S., & Shah, M. (2025, December). Sequential Cooperative Multi-Agent Online Learning and Adaptive Coordination Control in Dynamic and Uncertain Environments. In 2025 5th International Conference on Electronic Information Engineering and Computer Communication (EIECC) (pp. 692-697). IEEE.
Ahmad, N. R. (2025). Monetary policy, inflation and unemployment: The interactions and their economic impacts in Pakistan. Pakistan Journal of Economics and Public Policy, 5(1), 66–81.
Ahmad, N. R. (2025). Leadership ethics in practice: Employees’ experiences of integrity and hypocrisy in public-sector organizations in Pakistan. Journal of Applied Leadership and Transparency. https://doi.org/10.63878/jalt874
Toor, M. J., Shabbir, S., Rauf, A., Nisar, A., Safdar, S., & Zaib-Un-Nisa. (n.d.). The role of structured-communication-technique in developing pragmatic language capability among ASD-children. https://doi.org/10.63878/cjssr.v3i2.785
Toor, M. J., Mehmood, H., Yasmeen, A., Honey, Q., Waseem, M., & Wasif, M. (n.d.). Exploring the effect of verbal encouraging phrases (VEP) on Chinese language learners’ academic achievement at university level.
Toor, M. J., Arafat, M., Ayoub, M., Ali, M., & Alessi Longa, F. E. (n.d.). From coherent self to fragmented persona: Digital identity fragmentation and academic self-concept among Generation Z, with comparative insights from Generation Alpha. https://doi.org/10.63878/cjssr.v4i2.2318
Katsitadze, N., Kharadze, N., & Pirtskhalaishvili, D. (2025). The impact of social media on students’ effective time management. Baltic Journal of Economic Studies, 11(1), 200–210. https://doi.org/10.30525/2256-0742/2025-11-1-200-210
Toor, M. J., Zafar, S., Omer, F., Sarwar, M., & Hussain, A. K. (n.d.). Teachers’ perspectives on effectiveness of translanguaging in enhancing English language learning at secondary school level.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Erik Andersson, Johan Svensson, Lars Johansson
This work is licensed under a Creative Commons Attribution 4.0 International License.
License Terms (Recommended – CC BY 4.0)
All articles published in Fundamental Scientific Reports in Multidisciplinary Areas are licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).
This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author(s) and source are properly cited. Users are allowed to copy, redistribute, remix, transform, and build upon the material for any purpose, including commercial use, as long as appropriate credit is given.
Authors retain copyright of their work while granting the journal the right of first publication. The journal provides immediate open access to its content on the principle that making research freely available supports a greater global exchange of knowledge.
For more information about this license, visit:
https://creativecommons.org/licenses/by/4.0/
