The growing challenges in managing agricultural water resources in Iran reflect the inefficiency of sectoral approaches to water policymaking. This situation underscores the necessity of transitioning toward adaptive water governance, which considers the interactions among institutions, policies, and farmers’ behaviors within a multi-level governance framework. Accordingly, this study aimed to examine the institutional mechanisms influencing behavioral sustainability and improving water productivity in agricultural exploitation systems within the Miandoab Plain, one of the critical sub-basins of Lake Urmia. To achieve this objective, an agent-based analytical framework was developed to simulate farmers’ decision-making dynamics under different policy scenarios, including education, monitoring, conditional subsidies, and sustainable water supply. Subsequently, Fault Tree Analysis (FTA) was employed to trace institutional bottlenecks and identify potential failure pathways in participatory water governance. Data were collected through field questionnaires, expert interviews, and regional secondary sources. The simulation results revealed that institutional sustainability and increased farmer participation are achieved when soft and hard policy instruments are designed and implemented in a complementary and integrated manner. Education and monitoring play a pivotal role in strengthening institutional learning and rebuilding trust, and in interaction with conditional subsidies and sustainable water provision, they lead to higher physical water productivity. In contrast, one-dimensional policies lacking effective monitoring mechanisms resulted in reduced trust, increased opportunistic behavior, and weakened social capital. The Sustainable Water Supply Index, as a key variable, reflects the linkage between governance quality, equity in resource allocation, and the level of stakeholder participation.