Value-of-Information Constrained Agentic RL for Web Agents with Tool-Call Costs and Failure Risk
DOI:
https://doi.org/10.66238/fsrma49Keywords:
Value of information, POMDP, tool-use agents, constrained RL, multi-cost optimization, web navigation, failure-aware gatingAbstract
Agentic web systems frequently decide whether to acquire additional information (open new tabs, query site search, call external tools) that incurs heterogeneous costs and may expose the agent to failure-prone pages. We formulate web interaction with information-gathering actions and propose a Value-of-Information (VoI) constrained policy that explicitly trades expected benefit against multi-cost budgets and failure risk. The proposed method, VoI-CPG, learns (i) a belief-state encoder over partially observed DOM/text, (ii) a VoI estimator predicting the expected reduction in task uncertainty from each information action, and (iii) a constrained gating mechanism that permits info actions only when VoI-to-cost ratio exceeds a budget-adaptive threshold and predicted risk remains below a limit. Experiments are recommended on tasks requiring comparison and disambiguation (e.g., product filtering, policy lookup, form completion) totaling 800–1,500 tasks, and measure success rate, tool-call count, latency, monetary/API cost, and failure frequency. VoI-CPG reduces unnecessary exploration while preserving success, making budgets predictable in deployment.
References
Gu, X., Yang, J., & Liu, M. (2025). Research on a Green Money Laundering Identification Framework and Risk Monitoring Mechanism Integrating Artificial Intelligence and Environmental Governance Data.
Brohi, S., Mastoi, Q. U. A., Jhanjhi, N. Z., & Pillai, T. R. (2025). A research landscape of agentic ai and large language models: Applications, challenges and future directions. Algorithms, 18(8), 499.
Ma, Q., Yue, L., Xu, S., Shi, Y., & Liu, H. (2026). Web Agent Agentic Reinforcement Learning Decision Model Under Multi-Cost and Failure Risk Constraints.
Fourney, A., Bansal, G., Mozannar, H., Tan, C., Salinas, E., Niedtner, F., ... & Amershi, S. (2024). Magentic-one: A generalist multi-agent system for solving complex tasks. arXiv preprint arXiv:2411.04468.
Cai, B., Bai, W., Lu, Y., & Lu, K. (2024, June). Fuzz like a Pro: Using Auditor Knowledge to Detect Financial Vulnerabilities in Smart Contracts. In 2024 International Conference on Meta Computing (ICMC) (pp. 230-240). IEEE.
Momennejad, I. (2024). Memory, space, and planning: Multiscale predictive representations. arXiv preprint arXiv:2401.09491.
Liu, S., Feng, H., & Liu, X. (2025). A Study on the Mechanism of Generative Design Tools' Impact on Visual Language Reconstruction: An Interactive Analysis of Semantic Mapping and User Cognition. Authorea Preprints.
Du, Y. (2025). Research on Deep Learning Models for Forecasting Cross-Border Trade Demand Driven by Multi-Source Time-Series Data. Journal of Science, Innovation & Social Impact, 1(2), 63-70.
Kumar, K., Ashraf, T., Thawakar, O., Anwer, R. M., Cholakkal, H., Shah, M., ... & Khan, S. (2025). Llm post-training: A deep dive into reasoning large language models. arXiv preprint arXiv:2502.21321.
Wachi, A., Shen, X., & Sui, Y. (2024). A survey of constraint formulations in safe reinforcement learning. arXiv preprint arXiv:2402.02025.
Mao, Y., Ma, X., & Li, J. (2025). Research on API Security Gateway and Data Access Control Model for Multi-Tenant Full-Stack Systems.
Qiu, Y., & Wang, J. (2023, October). A machine learning approach to credit card customer segmentation for economic stability. In Proceedings of the 4th International Conference on Economic Management and Big Data Applications, ICEMBDA (pp. 27-29).
Bedi, A., Crestani, L., El-Rayes, Y., & Marcelli, D. A Compressed Suffix Memory Approach to Improving RL Performance in Partially Observable Domains.
Zhu, W., Yao, Y., & Yang, J. (2025). Real-Time Risk Control Effects of Digital Compliance Dashboards: An Empirical Study Across Multiple Enterprises Using Process Mining, Anomaly Detection, and Interrupt Time Series.
Li, T., Xia, J., Liu, S., & Jiang, Y. (2025). Digital Transformation of Human Resources: From Consulting Frameworks to AI-Enabled Learning Management Systems.
Garg, D., VanWeelden, S., Caples, D., Draguns, A., Ravi, N., Putta, P., ... & Motwani, S. (2025). Real: Benchmarking autonomous agents on deterministic simulations of real websites. arXiv preprint arXiv:2504.11543.
Mohammadi, M., Li, Y., Lo, J., & Yip, W. (2025, August). Evaluation and benchmarking of llm agents: A survey. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2 (pp. 6129-6139).
Gu, X., Liu, M., & Yang, J. (2025). Application and Effectiveness Evaluation of Federated Learning Methods in Anti-Money Laundering Collaborative Modeling Across Inter-Institutional Transaction Networks.
Srivastava, S., Li, G., Chang, C., Garg, R., Kaur, M., Lee, C. Y., ... & Qi, P. (2025). WARC-Bench: Web Archive Based Benchmark for GUI Subtask Executions. arXiv preprint arXiv:2510.09872.
Li, T., Xia, J., Liu, S., & Hong, E. (2025). Strategic Human Resource Leadership in Global Biopharmaceutical Enterprises: Integrating HR Analytics and Cross-Cultural.
Krupp, L. A. R. S., Geißler, D. A. N. I. E. L., Woźniak, P. W., Lukowicz, P., & Karolus, J. (2025). Quantifying Web Agents-A Survey on Web Agent Performance and Efficiency.
Mao, Y., Ma, X., & Li, J. (2025). Research on Web System Anomaly Detection and Intelligent Operations Based on Log Modeling and Self-Supervised Learning.
Kuntz, T., Duzan, A., Zhao, H., Croce, F., Kolter, Z., Flammarion, N., & Andriushchenko, M. (2025). Os-harm: A benchmark for measuring safety of computer use agents. arXiv preprint arXiv:2506.14866.
Zhu, W., Yang, J., & Yao, Y. (2025, October). How Compliance Maturity Translates to Risk Reduction: A Multi-Case Comparison of Global Operations Using fsQCA and Hierarchical Bayesian Methods. In Proceedings of the 2025 2nd International Conference on Digital Economy and Computer Science (pp. 672-676).
Jones, D., Severi, G., Pouliot, M., Lopez, G., de Gruyter, J., Zanella-Beguelin, S., ... & Minnich, A. (2025). A systematization of security vulnerabilities in computer use agents. arXiv preprint arXiv:2507.05445.
Ahmad, N. R. (2025). Blockchain beyond buzzwords: Evaluating its practical application in Pakistan’s supply chain systems. Quarterly Review Journal of Social Sciences. https://doi.org/10.63878/qrjs253
Safdar, M. R. (2025). Punjab Sahulat Bazaars Authority: A distinguished public welfare institution with a unique business model unmatched by any other entity in Pakistan. Contemporary Journal of Social Sciences and Research, 3(3). https://doi.org/10.63878/cjssr.v3i3.1311
Ahmad, N. R. (2025). Monetary policy, inflation and unemployment: The interactions and their economic impacts in Pakistan. Pakistan Journal of Economics and Public Policy, 5(1), 66–81.
Ahmad, N. R. (2025). Leadership ethics in practice: Employees’ experiences of integrity and hypocrisy in public-sector organizations in Pakistan. Journal of Applied Leadership and Transparency. https://doi.org/10.63878/jalt874
Toor, M. J., Shabbir, S., Rauf, A., Nisar, A., Safdar, S., & Zaib-Un-Nisa. (n.d.). The role of structured-communication-technique in developing pragmatic language capability among ASD-children. https://doi.org/10.63878/cjssr.v3i2.785
Toor, M. J., Mehmood, H., Yasmeen, A., Honey, Q., Waseem, M., & Wasif, M. (n.d.). Exploring the effect of verbal encouraging phrases (VEP) on Chinese language learners’ academic achievement at university level.
Toor, M. J., Arafat, M., Ayoub, M., Ali, M., & Alessi Longa, F. E. (n.d.). From coherent self to fragmented persona: Digital identity fragmentation and academic self-concept among Generation Z, with comparative insights from Generation Alpha. https://doi.org/10.63878/cjssr.v4i2.2318
Katsitadze, N., Kharadze, N., & Pirtskhalaishvili, D. (2025). The impact of social media on students’ effective time management. Baltic Journal of Economic Studies, 11(1), 200–210. https://doi.org/10.30525/2256-0742/2025-11-1-200-210
Toor, M. J., Zafar, S., Omer, F., Sarwar, M., & Hussain, A. K. (n.d.). Teachers’ perspectives on effectiveness of translanguaging in enhancing English language learning at secondary school level.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 James R. Bailey, Daniel J. Steinberg, Emily K. Black
This work is licensed under a Creative Commons Attribution 4.0 International License.
License Terms (Recommended – CC BY 4.0)
All articles published in Fundamental Scientific Reports in Multidisciplinary Areas are licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).
This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author(s) and source are properly cited. Users are allowed to copy, redistribute, remix, transform, and build upon the material for any purpose, including commercial use, as long as appropriate credit is given.
Authors retain copyright of their work while granting the journal the right of first publication. The journal provides immediate open access to its content on the principle that making research freely available supports a greater global exchange of knowledge.
For more information about this license, visit:
https://creativecommons.org/licenses/by/4.0/
