AI-Powered Role-Playing Reaches New Heights: CoSER Enables Authentic Simulation of Established Literary Characters

Simulating established characters presents a significant challenge for AI-powered role-playing agents (RPLAs). A lack of authenticity in training data and the difficulty of adequately evaluating nuances in role-playing complicate the development of convincing virtual characters. A new project called CoSER (Coordinating LLM-Based Persona Simulation of Established Roles) addresses these challenges with an innovative approach based on a high-quality dataset, open models, and a novel evaluation protocol.

An Extensive Dataset from World Literature

The CoSER dataset comprises dialogues from 17,966 characters across 771 recognized books that appear on Best-Books-Ever lists. This dataset not only offers authentic dialogues with realistic subtleties but also diverse data types such as conversational contexts, character experiences, and inner thoughts of the characters. This comprehensive data foundation allows AI models to be trained with realistic and complex character traits.

"Given-Circumstance Acting": A New Method for Training and Evaluation

Inspired by the acting methodology of "Given-Circumstance Acting," a new approach for training and evaluating RPLAs has been developed. Here, the LLMs successively embody multiple characters in literary scenes. This method allows for the evaluation of the AI's ability to empathize with different roles and act according to the situation.

CoSER 8B and CoSER 70B: Advanced Open-Source Models

Based on the LLaMA-3.1 model, two advanced open-source RPLAs have been developed: CoSER 8B and CoSER 70B. Extensive experiments demonstrate the value of the CoSER dataset for the training, evaluation, and retrieval of RPLAs. CoSER 70B shows particularly impressive results, reaching or even exceeding the performance of GPT-4 in some benchmarks. For example, CoSER 70B achieves an accuracy of 75.80% in the InCharacter benchmark and 93.47% in the LifeChoice benchmark.

Future Perspectives for AI-Powered Role-Playing

CoSER represents a significant step in the development of realistic and complex AI-driven characters. The high-quality dataset, the innovative training methods, and the powerful open-source models open up new possibilities for the application of RPLAs in various areas, such as interactive stories, video games, or virtual assistants. The disclosure of the code and dataset enables further research and promotes collaboration within the AI community.

Bibliography: Wang, X., et al. "CoSER: Coordinating LLM-Based Persona Simulation of Established Roles." arXiv preprint arXiv:2502.09082 (2025). https://aclanthology.org/2024.acl-long.554.pdf https://arxiv.org/html/2402.10811v2 https://github.com/azminewasi/Awesome-LLMs-ICLR-24 https://sigir-2024.github.io/proceedings.html https://www.researchgate.net/publication/388657757_Embracing_Dialectic_Intersubjectivity_Coordination_of_Different_Perspectives_in_Content_Analysis_with_LLM_Persona_Simulation https://github.com/dair-ai/ML-Papers-of-the-Week https://www.nature.com/articles/s41599-024-03611-3 https://conf.researchr.org/track/icse-2025/icse-2025-software-engineering-in-society https://research.tudelft.nl/files/236262323/synthetic-users-insights-from-designers-interactions-with-persona-based-chatbots.pdf