Automated Prompt Optimization with Socratic Guidance: The Multi-Agent Framework MARS

The quality of prompts, i.e., the instructions given to large language models (LLMs), is crucial for the quality of the generated texts. Automated prompt optimization (APO) aims to overcome the limitations of manual prompt design and expand the search space for more effective prompts. However, existing APO methods encounter challenges, particularly regarding the flexibility of fixed templates and the efficiency of searching the prompt space.

A promising approach to address these challenges is the multi-agent framework MARS (Multi-Agent framework Incorporating Socratic guidance). MARS utilizes multi-agent fusion technology for automatic planning, enabling incremental, continuous optimization and evaluation of prompts. The framework consists of seven agents, each with different functions, that autonomously create an optimization path using a planner. This flexible approach allows for dynamic adaptation to the specific task.

A special feature of MARS is the integration of a Socratic dialogue pattern between a teacher, critic, and student agent. This iterative procedure allows for targeted optimization of the prompts through continuous feedback and adaptation. The teacher agent provides direction, the critic agent evaluates the generated prompts, and the student agent learns from the feedback and generates improved prompts. This interactive process enables an efficient search of the prompt space and leads to higher-quality results.

To evaluate the effectiveness of MARS, extensive experiments were conducted on various datasets. The results show that MARS achieves a significant improvement in prompt quality compared to conventional APO methods. Additional analyses demonstrate the model's advancements and offer insights into the interpretability of the optimization process.

MARS represents an important contribution to the further development of automated prompt optimization. By combining multi-agent technology and Socratic guidance, MARS offers a flexible and efficient approach for generating optimized prompts. The results of the experiments highlight the potential of MARS for improving the performance of large language models in various application areas.

The further development of APO methods like MARS is crucial for unlocking the full potential of LLMs. By automating prompt design, complex tasks can be handled more efficiently and effectively. Future research could focus on expanding the MARS framework and integrating further optimization strategies to further improve performance and adaptability to various use cases.

Bibliography: Zhang, J., Wang, Z., Zhu, H., Liu, J., Lin, Q., & Cambria, E. (2025). MARS: A Multi-Agent Framework Incorporating Socratic Guidance for Automated Prompt Optimization. arXiv preprint arXiv:2503.16874. 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP). Advances in Neural Information Processing Systems 37 (NeurIPS 2024). Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Yongchao, L. (2024). PROMST. GitHub repository. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. NeurIPS 2024. Malliaros, F. D., & Vazirgiannis, M. (2023). Mars-PO: Multi-Agent Reasoning System Preference Optimization. arXiv preprint arXiv:2311.19039. OpenReview MARS: A Multi-Agent Framework Incorporating Socratic Guidance for Automated Prompt Optimization.