Right Answers, Wrong Reasons: Perception Errors in Multimodal Chain-of-Thought and the Blind Spot of Outcome-Based Rewards
Keywords:
multimodal chain-of-thought, perception errors, object hallucination, process supervision, verifiable rewards, reward hacking, reasoning faithfulnessAbstract
Multimodal large language models increasingly answer visual questions through explicit reasoning chains, and the chains are typically fluent. Evidence shows they can nonetheless be perceptually wrong—misreading values off a chart's axis, miscounting geometric primitives, or citing objects absent from the image—while the final answer lands correct anyway. This survey organizes that evidence around a Process–Outcome Consistency (POC) framework: a 2×2 over process soundness and outcome correctness whose key cell, the compensated error (flawed process, correct outcome), is invisible to every metric and training signal that scores only the answer. We trace the evidence for perception errors inside coherent chains, formalize why outcome-based supervision—verifier-reward reinforcement learning, outcome reward models, answer-filtered rejection sampling—cannot select against compensated errors (the reward insensitivity principle), and review the resulting signatures: reward overoptimization, reward hacking, and reasoning that grows more articulate without growing more grounded. We map mitigation onto five leverage points—upstream perception, inference-time process shaping, training-signal redesign, post-hoc verification, process-aware evaluation—and propose reporting compensated-error and evidence-grounding rates alongside accuracy. We close on an asymmetry favoring multimodal settings: visual process errors are checkable by mundane tooling, making faithful multimodal reasoning an unusually tractable target for process supervision.
References
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 35.
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022). Large language models are zero-shot reasoners. In Advances in Neural Information Processing Systems 35.
Wang, Z., Xia, M., He, L., Chen, H., Liu, Y., et al. (2024). CharXiv: Charting gaps in realistic chart understanding in multimodal LLMs. arXiv preprint arXiv:2406.18521.
Rahmanzadehgervi, P., Bolton, L., Taesiri, M. R., & Nguyen, A. T. (2024). Vision language models are blind: Failing to translate detailed visual features into words. arXiv preprint arXiv:2407.06581.
Tong, S., Liu, Z., Zhai, Y., Ma, Y., LeCun, Y., & Xie, S. (2024). Eyes wide shut? Exploring the visual shortcomings of multimodal LLMs. arXiv preprint arXiv:2401.06209.
Rohrbach, A., Hendricks, L. A., Burns, K., Darrell, T., & Saenko, K. (2018). Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Li, Y., Du, Y., Zhou, K., Wang, J., Zhao, W. X., & Wen, J.-R. (2023). Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Cobbe, K., Klimov, O., Hesse, C., Kim, D., & Schulman, J. (2021). Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
Zelikman, E., Wu, Y., Mu, J., & Goodman, N. D. (2022). STaR: Bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems 35.
Lambert, N., Morrison, J., Pyatkin, V., Huang, S., Ivison, H., Brahman, F., et al. (2024). Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.15124.
Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., et al. (2025). DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948.
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., & Liu, T. (2025). A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. In ACM Transactions on Information Systems.
Wang, Y., Wu, S., Zhang, Y., Yan, S., Liu, Z., Luo, J., & Fei, H. (2025). Multimodal chain-of-thought reasoning: A comprehensive survey. arXiv preprint arXiv:2503.12605.
Masry, A., Long, D. X., Tan, J. Q., Joty, S., & Hoque, E. (2022). ChartQA: A benchmark for question answering about charts with visual and logical reasoning. In Findings of the Association for Computational Linguistics: ACL 2022.
Methani, N., Ganguly, P., Khapra, M. M., & Kumar, P. (2020). PlotQA: Reasoning over scientific plots. In Proceedings of the AAAI Conference on Artificial Intelligence.
Lu, P., Bansal, H., Xia, T., Liu, J., Li, C., Hajishirzi, H., Cheng, H., Chang, K.-W., Galley, M., & Gao, J. (2024). MathVista: Evaluating mathematical reasoning of foundation models in visual contexts. In Advances in Neural Information Processing Systems 36 Datasets and Benchmarks Track.
Guan, T., Liu, F., Wu, X., Xian, R., Li, Z., Liu, X., Wang, X., Chen, L., Huang, F., Yacoob, Y., Manocha, D., & Zhou, T. (2024). HallusionBench: An advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
Wang, J., Wang, Y., Xu, G., Zhang, J., Gu, Y., Jia, H., Zhang, J., Yan, M., Zhang, J., & Sang, J. (2024). AMBER: An LLM-free multi-dimensional benchmark for MLLM hallucination evaluation. arXiv preprint arXiv:2311.07397.
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020). Shortcut learning in deep neural networks. In Nature Machine Intelligence, 2, 665-673.
McCoy, R. T., Pavlick, E., & Linzen, T. (2019). Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL).
Jia, R., & Liang, P. (2017). Adversarial examples for evaluating reading comprehension systems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Zhang, J., Khayatkhoei, M., Chhikara, P., & Ilievski, F. (2025). MLLMs know where to look: Training-free perception of small visual details with multimodal LLMs. arXiv preprint arXiv:2502.17422.
Leng, S., Zhang, H., Chen, G., Li, X., Lu, S., Miao, C., & Bing, L. (2024). Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
Zhou, Y., Cui, C., Yoon, J., Zhang, L., Deng, Z., Finn, C., Bansal, M., & Yao, H. (2023). Analyzing and mitigating object hallucination in large vision-language models. arXiv preprint arXiv:2310.00754.
Yin, S., Fu, C., Zhao, S., Xu, T., Wang, H., et al. (2024). Woodpecker: Hallucination correction for multimodal large language models. arXiv preprint arXiv:2310.16045.
Favero, A., Zare, Z., Zarei, A., & Morency, L.-P. (2024). Multi-modal hallucination control by visual information grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).
Dziri, N., Lu, X., Sclar, M., Li, X. L., Jiang, L., Lin, B. Y., et al. (2023). Faith and fate: Limits of transformers on compositionality. In Advances in Neural Information Processing Systems 36.
Lanham, T., Chen, A., Radhakrishnan, A., Steiner, B., Denison, C., Hernandez, D., et al. (2023). Measuring faithfulness in chain-of-thought reasoning. arXiv preprint arXiv:2307.13702.
Turpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023). Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting. In Advances in Neural Information Processing Systems 36.
Pfau, J., Merrill, W., & Bowman, S. R. (2024). Let's think dot by dot: Simple bounds on hidden computation in transformer language models. arXiv preprint arXiv:2404.15758.
Lyu, Q., Apidianaki, M., & Callison-Burch, C. (2024). Faithful chain-of-thought reasoning. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL).
Yu, T., Yao, Y., Zhang, H., He, T., Han, Y., Cui, G., Hu, J., Liu, Z., Zheng, H.-T., Zhao, M., & Sun, M. (2024). RLHF-V: Towards trustworthy MLLMs via behavior alignment from fine-grained correctional human feedback. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2023). Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations (ICLR).
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., et al. (2022). Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems 35.
Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., et al. (2024). DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
Gao, L., Schulman, J., & Hilton, J. (2023). Scaling laws for reward model overoptimization. In Proceedings of the 40th International Conference on Machine Learning (ICML).
Baker, B., Huizinga, J., Gao, L., Dou, Z., Guan, M., et al. (2025). Monitoring reasoning models for misbehavior and the risks of promoting obfuscation. arXiv preprint arXiv:2503.11926.
Skalse, J., Howe, N. H. R., Krasheninnikov, D., & Krueger, D. (2022). Defining and characterizing reward hacking. In Advances in Neural Information Processing Systems 35.
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016). Concrete problems in AI safety. arXiv preprint arXiv:1606.06565.
Stroebl, B., Kapoor, S., & Narayanan, A. (2024). The limits of inference scaling through resampling. arXiv preprint arXiv:2411.17501.
Yu, T., Zhang, H., Li, Q., Xu, Q., Yao, Y., et al. (2024). RLAIF-V: Open-source AI feedback leads to super GPT-4V trustworthiness. arXiv preprint arXiv:2405.17220.
Xu, G., Jin, P., Wu, Z., Li, H., Song, Y., et al. (2024). LLaVA-CoT: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.10440.
Huang, W., Jia, B., Zhai, Z., Cao, S., Ye, Z., et al. (2025). Vision-R1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749.
Yu, E., Lin, K., Zhao, L., et al. (2025). Perception-R1: Pioneering perception policy with reinforcement learning. arXiv preprint arXiv:2504.07954.
Wang, W., Gao, Z., Chen, L., et al. (2025). VisualPRM: An effective process reward model for multimodal reasoning. arXiv preprint arXiv:2503.10291.
Zhang, L., Hu, A., Xu, H., Yan, M., Xu, Y., Jin, Q., Zhang, J., & Huang, F. (2024). TinyChart: Efficient chart understanding with visual token merging and program-of-thoughts learning. arXiv preprint arXiv:2404.16635.
Gupta, T., & Kembhavi, A. (2023). Visual programming: Compositional visual reasoning without training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., & Cobbe, K. (2024). Let's verify step by step. In International Conference on Learning Representations (ICLR).
Uesato, J., Kushman, N., Kumar, R., Song, F., Siegel, N., Wang, L., Creswell, A., Irving, G., & Higgins, I. (2022). Solving math word problems with process- and outcome-based feedback. arXiv preprint arXiv:2211.14275.
Xia, R., Zhang, B., Ye, H., et al. (2024). ChartX & ChartVLM: A versatile benchmark and foundation model for complicated chart reasoning. arXiv preprint arXiv:2402.12185.
Chen, J., Kong, L., Wei, H., Liu, C., & Ge, Z. (2024). OneChart: Purify the chart structural extraction via one auxiliary token. arXiv preprint arXiv:2404.09987.
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., & Finn, C. (2023). Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems 36.
