Many important variables useful for decision making cannot be observed. Examples are the latent cycle in asset returns, the volatility, and financial markets or economic sentiments. These latent (i.e., unobservable) variables have to be inferred through statistical models or observable proxies. The objectives of this doctoral thesis is to develop and test new statistical models to infer these variables, and assess the forecasting gains in the Fields of economics and finance.

In my first essay, I perform a large-scale empirical study to compare the forecasting performance of single-regime and Markov–switching GARCH (MSGARCH) models from a risk management perspective. MSGARCH models are driven by an unobservable state variable. I find that, for daily, weekly, and ten-day equity log-returns, MSGARCH models yield more accurate Value-at-Risk, Expected Shortfall, and left-tail distribution forecasts than their single–regime counterpart. Also, my results indicate that accounting for parameter uncertainty improves left-tail predictions, independently of the inclusion of the Markov-switching mechanism.

In my second essay I tackle the challenging task of computing text-based sentiment measures which serve as proxies of the unobservable sentiment about the economy. Indeed, modern calculation of textual sentiment involves a myriad of choices for the actual calibration. I introduce a general sentiment engineering framework that optimizes the design for forecasting purposes. It includes the use of the elastic net for sparse data–driven selection and weighting of thousands of sentiment values. These values are obtained by pooling the textual sentiment values across publication venues, article topics, sentiment construction methods, and time. I apply the framework to investigate the added value of textual analysis–based sentiment indices for forecasting economic growth in the US. I find that, compared to the use of high-dimensional forecasting techniques based on only economic and financial indicators, the additional use of optimized news-based sentiment values yields significant accuracy gains in forecasting the nine-month and annual growth rates of the US industrial production.

In my third essay, I design a tone-based event study framework that identifies the news in sentiment and show that the abnormal sentiment is a powerful predictor of market reaction. In my framework, I use text-mining techniques to derive the normal sentiment based on market and sector-wide news. I then focus on corporate events and track the abnormal tone dynamics around that event. This leads to a cumulative abnormal tone chart. I show that firm’s abnormal tone provides investor with relevant information on futures firm’s stock returns and media behavior.