Proceedings

Hierarchical Multi-Scale Gaussian Transformer for Stock Movement Prediction

Video Download BibTeX Cite

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence. Special Track on AI in FinTech. Pages 4640-4646.

https://doi.org/10.24963/ijcai.2020/640

© 2020 International Joint Conferences on Artificial Intelligence

Abstract

Predicting the price movement of finance securities like stocks is an important but challenging task, due to the uncertainty of financial markets. In this paper, we propose a novel approach based on the Transformer to tackle the stock movement prediction task. Furthermore, we present several enhancements for the proposed basic Transformer. Firstly, we propose a Multi-Scale Gaussian Prior to enhance the locality of Transformer. Secondly, we develop an Orthogonal Regularization to avoid learning redundant heads in the multi-head self-attention mechanism. Thirdly, we design a Trading Gap Splitter for Transformer to learn hierarchical features of high-frequency finance data. Compared with other popular recurrent neural networks such as LSTM, the proposed method has the advantage to mine extremely long-term dependencies from financial time series. Experimental results show our proposed models outperform several competitive methods in stock price prediction tasks for the NASDAQ exchange market and the China A-shares market.

  1. [Altman, 1992] Naomi S Altman. An introduction to ker- nel and nearest-neighbor nonparametric regression. The American Statistician, 46(3):175-185, 1992.
  2. [Breiman, 2001] Leo Breiman. Random forests. Machine learning, 45(1):5-32, 2001.
  3. [Chen et al., 2019] Deli Chen, Yanyan Zou, Keiko Hari- moto, Ruihan Bao, Xuancheng Ren, and Xu Sun. Incor- porating fine-grained events in stock movement prediction. arXiv preprint arXiv:1910.05078, 2019.
  4. [Cortes and Vapnik, 1995] Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 20(3):273-297, 1995.
  5. [Feng et al., 2019] Fuli Feng, Huimin Chen, Xiangnan He, Ji Ding, Maosong Sun, and Tat-Seng Chua. Enhancing stock movement prediction with adversarial training. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pages 5843-5849. AAAI Press, 2019. IJCAI
  6. [Guo et al., 2019] Maosheng Guo, Yu Zhang, and Ting Liu. Gaussian transformer: a lightweight approach for natural language inference. In The Thirty-Third AAAI Conference on Artificial Intelligence, 2019.
  7. [Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.
  8. [Lee et al., 2019] Mingu Lee, Jinkyu Lee, Hye Jin Jang, Byeonggeun Kim, Wonil Chang, and Kyuwoong Hwang. Orthogonality constrained multi-head attention for key- word spotting. arXiv preprint arXiv:1910.04500, 2019.
  9. [Li et al., 2018] Jian Li, Zhaopeng Tu, Baosong Yang, Michael R Lyu, and Tong Zhang. Multi-head atten- tion with disagreement regularization. arXiv preprint arXiv:1810.10183, 2018.
  10. [Li et al., 2019] Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. En- hancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In Advances in Neural Information Processing Systems, pages 5244- 5254, 2019.
  11. [Lin et al., 2013] Yuling Lin, Haixiang Guo, and Jinglu Hu. An svm-based approach for stock market trend prediction. In The 2013 international joint conference on neural net- works (IJCNN), pages 1-7. IEEE, 2013.
  12. [Nelson et al., 2017] David MQ Nelson, Adriano CM Pereira, and Renato A de Oliveira. Stock market's price movement prediction with lstm neural networks. In 2017 International Joint Conference on Neural Networks (IJCNN), pages 1419-1426. IEEE, 2017.
  13. [Qin et al., 2017] Yao Qin, Dongjin Song, Haifeng Chen, Wei Cheng, Guofei Jiang, and Garrison Cottrell. A dual- stage attention-based recurrent neural network for time se- ries prediction. arXiv preprint arXiv:1704.02971, 2017.
  14. [Rumelhart et al., 1986] David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. nature, 323(6088):533-536, 1986.
  15. [Schumaker and Chen, 2009] Robert P Schumaker and Hsinchun Chen. Textual analysis of stock market pre- diction using breaking financial news: The azfin text system. ACM Transactions on Information Systems (TOIS), 27(2):12, 2009.
  16. [Selvin et al., 2017] Sreelekshmy Selvin, R Vinayakumar, EA Gopalakrishnan, Vijay Krishna Menon, and KP So- man. Stock price prediction using lstm, rnn and cnn- sliding window model. In 2017 International Conference on Advances in Computing, Communications and Infor- matics (ICACCI), pages 1643-1647. IEEE, 2017.
  17. [Tao et al., 2018] Chongyang Tao, Shen Gao, Mingyue Shang, Wei Wu, Dongyan Zhao, and Rui Yan. Get the point of my utterance! learning towards effective re- sponses with multi-head attention mechanism. In IJCAI, pages 4418-4424, 2018. IJCAI
  18. [Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing sys- tems, pages 5998-6008, 2017.
  19. [Walczak, 2001] Steven Walczak. An empirical analysis of data requirements for financial forecasting with neural networks. Journal of management information systems, 17(4):203-222, 2001.
  20. [Wang et al., 2019] Jia Wang, Tong Sun, Benyuan Liu, Yu Cao, and Hongwei Zhu. Clvsa: A convolutional lstm based variational sequence-to-sequence model with atten- tion for predicting trends of financial markets. In Proceed- ings of the 28th International Joint Conference on Artifi- cial Intelligence, pages 3705-3711. AAAI Press, 2019. IJCAI
  21. [Weng et al., 2017] Bin Weng, Mohamed A Ahmed, and Fadel M Megahed. Stock market one-day ahead move- ment prediction using disparate data sources. Expert Sys- tems with Applications, 79:153-163, 2017.
  22. [Xingjian et al., 2015] SHI Xingjian, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. Convolutional lstm network: A machine learning ap- proach for precipitation nowcasting. In Advances in neural information processing systems, pages 802-810, 2015.
  23. [Xu and Cohen, 2018] Yumo Xu and Shay B Cohen. Stock movement prediction from tweets and historical prices. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1970-1979, 2018.
  24. [Zhang et al., 2017] Liheng Zhang, Charu Aggarwal, and Guo-Jun Qi. Stock price prediction via discovering multi-frequency trading patterns. In Proceedings of the 23rd ACM SIGKDD international conference on knowl- edge discovery and data mining, pages 2141-2149. ACM, 2017.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards