Vision-Language Models and the Problem of Hallucinations: A New Approach to a Solution

Vision-language models (VLMs) have made remarkable progress in visual perception in recent years. They can analyze images, describe them, and even answer questions about them. However, a persistent problem that limits the use of VLMs in safety-critical applications is so-called hallucinations. This is where the models generate descriptions of objects, actions, or concepts that are not present in the image at all.

Previous approaches to mitigating hallucinations can be broadly divided into two categories: generation adaptation and post-hoc verification. In generation adaptation, the decoding behavior of the model is modified to better match the text output to the visual input. Post-hoc verification methods, on the other hand, use external models to evaluate the generated outputs and correct them if necessary.

Both approaches have their weaknesses. Generation adaptations are often based on heuristics and do not offer correction mechanisms for hallucinations that have already been generated. Post-hoc verification methods are complex, typically require multiple models, and tend to discard erroneous outputs completely rather than improving them.

REVERSE: A New Approach to Hallucination Mitigation

A promising new approach to solving this problem is REVERSE, a framework that combines hallucination-aware training with on-the-fly self-verification. REVERSE uses a novel dataset with over 1.3 million semi-synthetic examples of hallucinations and verification. In addition, an innovative technique called "Retrospective Resampling" is used, which enables the VLMs to detect and dynamically correct hallucinations during generation itself.

Retrospective resampling allows the model to go back and explore alternative paths during text generation if a hallucination is detected. By integrating the verification process directly into the generation, the model can critically review and correct its own output before the final description is output.

Evaluation and Results

Evaluations of REVERSE show a significant reduction in hallucinations. Compared to existing methods, REVERSE was able to achieve up to 12% and 28% better results in hallucination detection and correction, respectively, on datasets such as CHAIR-MSCOCO and HaloQuest. These results underscore the potential of REVERSE to significantly improve the reliability of VLMs.

The combination of hallucination-aware training and retrospective resampling offers a promising way to increase the accuracy and trustworthiness of vision-language models. The availability of the REVERSE dataset, model, and code opens up new opportunities for the research community to work on this important problem and advance the development of more robust and secure VLMs.

Bibliography: Wu, T.-H., Lee, H., Ge, J., Gonzalez, J. E., Darrell, T., & Chan, D. M. (2025). Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resampling. arXiv preprint arXiv:2504.13169. Sheng Liu, Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models, https://github.com/shengliu66/VTI Jihyung Kil, RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models, https://www.researchgate.net/publication/384210153_RAGTruth_A_Hallucination_Corpus_for_Developing_Trustworthy_Retrieval-Augmented_Language_Models Long Phan, A Hallucination Detection Benchmark and Method for Retrieval-Augmented Language Models, https://arxiv.org/abs/2410.15778 Lianmin Zheng, Judging LLM-Generated Texts with LLM-Based Judges: A Case Study with MATH, https://openreview.net/forum?id=LBl7Hez0fF Chen, Z., Wang, L., & Schütze, H. (2024). Towards Faithful Generation from Noisy Retrieval Results. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 1021–1041). Association for Computational Linguistics. ICLR 2024 Conference Schedule, https://iclr.cc/virtual/2024/calendar NeurIPS 2024 Papers, https://nips.cc/virtual/2024/papers.html Wang, L., Chen, Z., Schütze, H., Towards Faithful Generation from Noisy Retrieval Results, https://www.inf.uni-hamburg.de/en/inst/ab/lt/publications/2024-wang-acl24.pdf