Machine Learning Investigation of Retail Demand Shocks, ETF Investing, and Limits to Arbitrage

Abstract

The paper explores the connection between the demand shocks within the retail sector, exchange traded fund (ETF) mispricing and the constraints that hindered the ability of the arbitrageurs to correct the said deviations based on a set of machine learning (ML) models estimated on a large sample of equity ETFs listed in the United States between the years 2015 (January) and 2023 (December). Using granular retail order flow data broken down through the odd-lot imbalance methodology of Boehmer et ak. (2021), social media sentiment indices based on Reddit and Google Trends, we create time-varying demand shock proxies and incorporate them into gradient-boosted tree models (XGBoost) and long short-term memory (LSTM) neural networks and random forests compared to penalised linear regressions. Evaluations based on an expanding-window out-of-sample scheme that maintains temporal sequence and removes look-ahead contamination are applied to models. We find that the most informative predictors of short-horizon ETFs premium and discount dynamics are retail demand shocks, which yield out-of-sample R² values exceeding linear benchmarks (8 to 14 percentage) and a long-short arbitrage strategy (annualised Sharpe ratio of 1.47). Significantly, the predictive advantage is concentrated: it is concentrated during periods of large market volatility, constrained by authorised participants balance sheets, and large short interest; exactly the circumstances when classical limit-to-arbitrage theory hypothesises that professional capital will be slow to rectify mispricings. These findings form part of an increasing literature relating retail investor heterogeneity to the presence of institutional arbitrage capacity and they offer practitioner-valued instruments to identify when ETF mispricing is probable not to end but to continue.