A Comprehensive Review of Multimodal Financial Stock Prediction Research
Keywords:
Multimodal Finance, Stock Prediction, Financial Text, ESG, Event-driven Modeling, Graph Relations, Large Language ModelsAbstract
With the development of financial text mining, deep time-series modeling, and large language models, multimodal stock prediction has evolved from early shallow fusion between price sequences and sentiment signals into an integrated research paradigm covering news text, technical indicators, ESG information, exogenous events, company relation graphs, and long-document semantic compression. Centered on five recent representative studies and supplemented by related work on financial text representation learning, graph neural networks, Transformer-based time-series modeling, and financial large language models, this review synthesizes the main research trajectory, core methodological categories, and key challenges in multimodal financial stock prediction. Specifically, the paper first examines the development of financial text modeling and early bimodal forecasting frameworks, then summarizes the extension of ESG signals, industry context, and exogenous events to stock prediction, and next reviews representative approaches to graph-based relation modeling and stable fusion. It further discusses recent advances in Transformers, patch-based modeling, and large language models for long-text processing and cross-modal alignment. On this basis, the review identifies common issues in label granularity, modality conflict, long-text redundancy, explicit event modeling, and interpretability, and concludes by summarizing the major research trends already visible in the literature.
References
Araci, D. (2019). FinBERT: Financial sentiment analysis with pre-trained language models. arXiv, arXiv:1908.10063.
Bollen, J., Mao, H., & Zeng, X. (2011). Twitter mood predicts the stock market. Journal of Computational Science, 2(1), 1-8.
Chen, P., Boukouvalas, Z., & Corizzo, R. (2024). A deep fusion model for stock market prediction with news headlines and time series data. Neural Computing and Applications, 36, 21229-21271.
Chen, Z., Chen, W., Smiley, C., et al. (2021). FinQA: A dataset of numerical reasoning over financial data. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 3697-3711).
Ding, X., Zhang, Y., Liu, T., & Duan, J. (2015). Deep learning for event-driven stock prediction. In Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI) (pp. 2327-2333).
Dong, Z., Fan, X., & Peng, Z. (2024). FNSPID: A comprehensive financial news dataset in time series. arXiv, arXiv:2402.06698.
Feng, F., Chen, H., He, X., Ding, J., Sun, M., & Chua, T.-S. (2019). Enhancing stock movement prediction with adversarial training. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI) (pp. 5843-5849).
Fischer, T., & Krauss, C. (2018). Deep learning with long short-term memory networks for financial market predictions. European Journal of Operational Research, *270*(2), 654-669.
Gruver, N., Finzi, M., Qehaja, M., & Wilson, A. G. (2023). Large language models are zero-shot time series forecasters. In Advances in Neural Information Processing Systems.
Gu, A., & Dao, T. (2023). Mamba: Linear-time sequence modeling with selective state spaces. arXiv, arXiv:2312.00752.
Jin, M., Wang, S., Ma, L., Chu, Z., Zhang, J. Y., & Wen, Q. (2023). Time series forecasting with pretrained foundation models: A survey. arXiv, arXiv:2312.17119.
Hu, Z., Zhao, Y., & Khushi, M. (2023). Time-LLM: Time series forecasting by reprogramming large language models. arXiv, arXiv:2310.01728.
Lee, H., Kim, J. H., & Jung, H. S. (2024). Deep-learning-based stock market prediction incorporating ESG sentiment and technical indicators. Scientific Reports, 14, Article 10262.
Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., & Long, M. (2024). iTransformer: Inverted transformers are effective for time series forecasting. In Proceedings of the International Conference on Learning Representations (ICLR).
Liu, Z., Chen, C., Wang, B., et al. (2024). Melody-GCN: A multiscale multimodal dynamic graph convolution network for stock forecasting. Pattern Recognition, 61(1).
Lu, Y., Xing, H., & Zhou, C. (2024). DocFinQA: A long-context financial reasoning dataset. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL).
Maia, M., Handschuh, S., Freitas, A., et al. (2018). WWW'18 open challenge: Financial opinion mining and question answering. In Companion Proceedings of the Web Conference 2018.
Malo, P., Sinha, A., Korhonen, P., Wallenius, J., & Takala, P. (2014). Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology, 65(4), 782-796.
Nie, Y., Nguyen, N. H., Sinthong, P., & Kalagnanam, J. (2022). A time series is worth 64 words: Long-term forecasting with transformers. arXiv, arXiv:2211.14730.
Qin, W., Song, Y., Chen, W., et al. (2021). Coupling macro-sector-micro financial indicators for learning stock representations with less uncertainty. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (pp. 9122-9129).
Sawhney, R., Agarwal, S., Wadhwa, A., & Shah, R. R. (2020). Deep attentive learning for stock movement prediction from social media text and company correlations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 8415-8426).
Shen, T., Zhang, S., Li, Y., et al. (2024). FinTextQA: A dataset for long-form financial question answering. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL).
Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (Vol. 30).
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., & Bengio, Y. (2017). Graph attention networks. arXiv, arXiv:1710.10903.
Wang, H., Xie, Z., Chiu, D. K. W., & Ho, K. K. W. (2025). Multimodal market information fusion for stock price trend prediction in the pharmaceutical sector. Applied Intelligence, 55, Article 77.
Xu, Y., & Cohen, S. B. (2018). Stock movement prediction from tweets and historical prices. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL) (pp. 1970-1979).
Yang, Y., Xie, Y., Wang, C., et al. (2023). FinGPT: Open-source financial large language models. arXiv, arXiv:2306.06031.
Zhang, Y., Dong, Z., & Xu, W. (2026). Integrative stock price trend prediction via hierarchical LLM text processing and patch-based transformer with co-attention. Expert Systems with Applications, 302, Article 130441.
Zhou, T., Zhang, P., & Yan, Z. (2023). TimesNet: Temporal 2D-variation modeling for general time series analysis. In Proceedings of the International Conference on Learning Representations (ICLR).
Zong, C., Wan, J., Cascone, L., & Zhou, H. (2025). Stock movement prediction with multimodal stable fusion via gated cross-attention mechanism. Complex & Intelligent Systems, 11, Article 396.
