Aspects Regarding a Deep Understanding of the Prediction for Stock Market Movements

Abstract

It is an important puzzle in the financial market to predict stock return movement direction. In this chapter, we not only propose (group) penalized logistic regression with multiple indicators to predict up- or downtrends, but also propose group penalized trinomial logit regression with multiple indicator groups to predict stock return movement direction: uptrends, sideways trends and downtrends. For the former, we construct the corresponding coordinate descent (CD) algorithm to complete variable selection and obtain parameter estimator, and introduce two-class confusion matrix, Receiver Operating Characteristic (ROC) and the area under a ROC curve (AUC) to assess two-class prediction performance. For the latter, we develop a rapidly convergent group coordinate descent (GCD) algorithm to simultaneously complete group selection and group estimation, introduce the relatively optimal Bayes classifiers to identify class indexes, and finally adopt three-class confusion matrix, Kappa, PDI, ROC surface and hypervolume under the ROC manifold (HUM) to assess three-class prediction performance.

1. Introduction

Forecasting the financial market is a major challenge in both academic and business. In recent decades, some methods have been proposed to forecast the future stock returns or trend direction. For example, time series analysis develops some statistical models such as ARIMA, ARCH and GARCH to study the past stock behavior and predict the future stock returns. Technical analysis developed by Murphy [1] and Edwards et al. [2] on financial market and stock trends applies statistical charts, technical indicators and historical data to forecast stock behavior. Fundamental analysis studies the economic factors that may influence market movements and applies financial methodologies with fundamental variables to predict stock market. For instance, Cavalcante et al. [3] sought economic factors influencing market trends. Nti et al. [4] summarized fundamental and technical analysis for stock market prediction. Machine learning techniques model nonstationary and nonlinear data to predict market behavior. Henrique et al. [5] overviewed machine learning techniques on financial market prediction by the bibliometric analysis. Jiang [6] summarized stock market prediction using deep learning methods such as LSTM, Convolutional Neural Network (CNN), Deep Neural Network (DNN) and Recurrent Neural Network (RNN) [7, 8, 9, 10].

The selection of input features plays a key role in predicting uptrends and downtrends for stock returns. The following three types of input features have been studied: A. extract-related indicators through fundamental and technical analysis [4]; B. apply text mining techniques to extract textual features from financial news or tweets [11]; C. generate candlestick charts to extract image features [12]. Recently, Jiang et al. [13] and Hu and Yang [14] combined technical indicators with (group) penalized logistic regressions to predict up- and downtrends for stock prices. Lin et al. [15] applied financial news to construct text mining-based stock prediction models. Li et al. [16] found that models including prices and news sentiments outperformed models only including either technical indicators or news sentiments. This research mainly predicted up- or downtrends. In this chapter, we not only summarize (group) penalized logistic regressions with multiple technical indicators to predict up- or downtrends, but also provide group penalized trinomial logit dynamic models with multiple technical indicator groups to predict uptrends, sideways trends and downtrends, for more details see Ref. [17].

Firstly, we propose Ridge/LASSO/ENet/SCAD/MCP penalized logistic regressions with multiple indicators to further improve the two-class prediction performance, not only apply the training set and the CD algorithm to obtain parameter estimators and probability estimators, but also employ the testing set and the learned estimators to obtain two-class confusion matrix and draw ROC curves to assess two-class prediction performance. Secondly, we propose G-LASSO/G-SCAD/G-MCP penalized logistic regressions to predict up- or downtrends, not only apply the training set to learn parameter estimators and probability estimators, but also adopt the testing set and the learned estimators to obtain confusion matrix and draw ROC curves in order to assess two-class prediction performance. Finally, we propose G-LASSO/G-SCAD/G-MCP penalized trinomial dynamic logit models to predict uptrends, sideways trends, and downtrends, not only develop the GCD algorithm to complete group selection and group estimation, but also introduce Bayes classifier to identify class labels, still adopt three-class confusion matrix, HUM, Kappa and PDI to assess three-class prediction performance to stock return movement direction.

The rest is organized as follows. Section 2 predicts uptrends and downtrends. Section 3 predicts uptrends, sideways trends and down trends. Section 4 gives conclusions and prospects. Section 5 provides discussions.

2. Predicting uptrends and downtrends

In this section, we not only propose five penalized logistic regressions with technical indicators to predict up- or downtrends, but also propose three group penalized logistic regressions to further improve two-class prediction performance.

2.1 Five penalized logistic regression prediction methods

Hu and Jiang [18] proposed logistic regression to predict up- or downtrends. Here, we propose Ridge/LASSO/ENet/SCAD/MCP penalized logistic regressions to predict up- or downtrends.

2.1.1 Five penalized logistic regressions

Introduce the direction indicator function

$$Y_t=\left\{\begin{array}{ll}1,\text{if}{C}_{t+1}>C_t,& \\ 0,\text{if}{C}_{t+1}\le C_t,& \end{array}\right.$$E1

where $C_t$ represents the closing price at the end of the t-th trading day. Then, $Y_t=1$ represents uptrends, and $Y_t=0$ represents downtrends. Now, we generalize [18] and introduce logistic regression with $p$ technical indicators:

$$P\left(X_t;{\beta }_0,\beta \right)=P\left(Y_t=1|X_t;{\beta }_0,\beta \right)=\frac{exp\left({\beta }_0+X_t^{\top }\beta \right)}{1+exp\left({\beta }_0+X_t^{\top }\beta \right)},$$E2

$$1-P\left(X_t;{\beta }_0,\beta \right)=P\left(Y_t=0|X_t;{\beta }_0,\beta \right)=\frac{1}{1+exp\left({\beta }_0+X_t^{\top }\beta \right)},$$E3

where ${\beta }_0$ is an unknown intercept term, $\beta ={\left({\beta }_1,{\beta }_2,\dots ,{\beta }_p\right)}^{\top }$ is an unknown parameter vector, and $X_t={\left(X_{t,1},X_{t,2},\dots ,X_{t,p}\right)}^{\top }$ is a predictor vector whose distribution is usually unknown. To improve two-class prediction performance, we propose the five penalized logistic regressions to predict up- or downtrends.

Let $x_t={\left(x_{t,1},x_{t,2},\dots ,x_{t,p}\right)}^{\top }$ and $y_t$ be the observation samples for $X_t$ and $Y_t$, respectively. Given the training set ${\left\{x_t,y_t\right\}}_{t=1}^{n_1}$ with the sample size $n_1$, we obtain the negative log-likelihood function

$$l\left(\beta \right)=-L\left(\beta \right)=-\sum _{t=1}^{n_1}\left\{y_t\left({\beta }_0+x_t^{\top }\beta \right)-log\left[1+exp\left({\beta }_0+x_t^{\top }\beta \right)\right]\right\},$$E4

and its penalized version

$$Q\left(\beta ;\lambda ,\gamma \right)\equiv l\left(\beta \right)+p_{\lambda ,\gamma }\left(\beta \right),$$E5

where $p_{\lambda ,\gamma }\left(\beta \right)$ is a penalty function with a tuning parameter $\lambda$ and a regularization parameter $\gamma$. Table 1 lists the five penalties.

Penalties	Formulae
Ridge	$p_{\lambda \left(\beta \right)}=\lambda \parallel \beta {\parallel }_2^2$.
LASSO	$p_{\lambda }\left(\beta \right)=\lambda \parallel \beta {\parallel }_1$.
ENet	$p_{\lambda ,\gamma }\left(\beta \right)=\frac{1}{2}\lambda \left[\left(1-\gamma \right)\parallel \beta {\parallel }_2^2+\gamma \parallel \beta {\parallel }_1\right]$, $\lambda \in \left(0,\infty \right),\gamma \in \left(0,1\right)$.
MCP	$p_{\lambda ,\gamma }\left(\beta \right)=\left\{\begin{array}{ll}\mathit{\lambda \beta }-\frac{{\beta }^2}{2\gamma },& \text{if}\beta \le \mathit{\gamma \lambda },\\ \frac{1}{2}{\mathit{\gamma \lambda }}^2,& \text{if}\beta >\mathit{\gamma \lambda },\end{array}\right.\lambda \ge 0,\gamma >1$.
SCAD	$p_{\lambda ,\gamma }\left(\beta \right)=\left\{\begin{array}{ll}\mathit{\lambda \beta },& \text{if}\beta \le \lambda ,\\ \frac{\mathit{\lambda \gamma \beta }-0.5\left({\beta }^2+{\lambda }^2\right)}{\left(\gamma -1\right)},& \text{if}\lambda <\beta \le \mathit{\lambda \gamma },\\ \frac{{\lambda }^2\left(\gamma +1\right)}{2},& \text{if}\beta >\mathit{\lambda \gamma }.\end{array}\right.\lambda \ge 0,\gamma >2$.

Table 1.

Penalized functions.

2.1.2 Parameter estimators and probability estimators

Suppose that the current parameter estimators are $\left({\widehat{\beta }}_0,\widehat{\beta }\left(m\right)\right)$. For the non-differentiable function (4), we transform it into the following weighted least-squares function:

$$l_Q\left({\beta }_0,\beta \right)=-\frac{1}{2n_1}\sum _{t=1}^{n_1}{W}_t{\left({\tilde{Y}}_t-{\beta }_0-x_t^{\top }\beta \right)}^2+C\left({\widehat{\beta }}_0,\widehat{\beta }\left(m\right)\right),$$E6

where

$${\tilde{Y}}_t={\widehat{\beta }}_0+x_t^{\top }\widehat{\beta }\left(m\right)+\frac{y_t-{\tilde{P}}_t}{{\tilde{P}}_t\left(1-{\tilde{P}}_t\right)},W_t={\tilde{P}}_t\left(1-{\tilde{P}}_t\right),{\tilde{P}}_t=\frac{exp\left(x_t^{\top }\widehat{\beta }\left(m\right)\right)}{1+exp\left(x_t^{\top }\widehat{\beta }\left(m\right)\right)},$$E7

and $C\left({\widehat{\beta }}_0,\widehat{\beta }\left(m\right)\right)$ is constant. Similarly, we replace the non-differentiable function $l\left(\beta \right)$ in (5) by the weighted least-squares function $l_Q\left({\beta }_0,\beta \right)$, apply the CD algorithm to obtain the parameter estimator

$${\widehat{\beta }}^{\lambda ,\gamma }=arg\underset{\beta }{min}\left\{l_Q\left({\beta }_0,\beta \right)+p_{\lambda ,\gamma }\left(\beta \right)\right\}.$$E8

More details refer to [18]. Table 2 lists LASSO/SCAD/MCP estimator.

Penalties	Estimators
LASSO	${\widehat{\beta }}_j^{\mathit{\text{LASSO}}}\left(Z_j;\lambda \right)=\frac{S\left(Z_j,\lambda \right)}{{\nu }_j}.$
MCP	${\widehat{\beta }}_j^{\mathit{MCP}}\left(Z_j;\lambda ,\gamma \right)=\left\{\begin{array}{ll}\frac{S\left(Z_j,\lambda \right)}{{\nu }_j-1/\gamma },& \mid Z_j\mid \le {\nu }_j\mathit{\lambda \gamma },\\ \frac{Z_j}{{\nu }_j},& \mid Z_j\mid >{\nu }_j\mathit{\lambda \gamma },\end{array}\right.\gamma >1/{\nu }_j.$
SCAD	${\widehat{\beta }}_j^{\mathit{\text{SCAD}}}\left(Z_j;\lambda ,\gamma \right)=\left\{\begin{array}{ll}\frac{S\left(Z_j,\lambda \right)}{{\nu }_j},& \mid Z_j\mid \le \lambda \left({\nu }_j+1\right),\\ \frac{S\left(Z_j,\mathit{\gamma \lambda }/\left(\gamma -1\right)\right)}{{\nu }_j-1/\left(\gamma -1\right)},& \lambda \left({\nu }_j+1\right)<\mid Z_j\mid \le {\nu }_j\mathit{\lambda \gamma },\gamma >1+1/{\nu }_j.\\ \frac{Z_j}{{\nu }_j},& \mid Z_j\mid >{\nu }_j\mathit{\lambda \gamma },\end{array}\right.$
Symbols	$\begin{array}{l}{\widehat{P}}_t=exp\left(x_t^{\top }{\widehat{\beta }}^{\lambda ,\gamma }\left(m\right)\right)/[1+exp(x_t^{\top }{\widehat{\beta }}^{\lambda ,\gamma }(m)],W_t={\widehat{P}}_t\left(1-{\widehat{P}}_t\right),t=1,\dots ,n_1,\\ W=\mathit{diag}\left\{W_1,W_2,\dots ,W_n\right\},\tilde{Y}=x^{\top }{\widehat{\beta }}^{\lambda ,\gamma }\left(m\right)+W^{-1}\left(Y-\widehat{P}\right),\widehat{P}=\left({\widehat{P}}_1,\dots ,{\widehat{P}}_n\right),\\ x_{\cdot j}={\left(x_{1j},\dots ,x_{n_{1}j}\right)}^{\top },{\nu }_j=n_1^{-1}{x}_{\cdot j}^{\top }{\mathit{Wx}}_{\cdot j},j=1,\cdots ,p,\\ Z_j=n_1^{-1}{x}_{\cdot j}^{\top }W\left(\tilde{Y}-x_{\cdot -j}{\beta }_{-j}\right)=n_1^{-1}{x}_{\cdot j}^{\top }\mathit{Wr}+{\nu }_j{\widehat{\beta }}_j^{\lambda ,\gamma }\left(m\right),\\ x_{\cdot -j}={\left(x_{\cdot 1},\cdots ,x_{\cdot j-1},0,x_{\cdot j+1},\cdots ,x_{\cdot p}\right)}^{\top },\\ {\beta }_{-j}=\left({\beta }_1,\cdots ,{\beta }_{j-1},0,{\beta }_{j+1},\cdots ,{\beta }_p\right).\end{array}$

Table 2.

Penalized functions and parameter estimators for penalized logistic regressions.

For $j$ in $\left\{1,2,\dots ,p\right\}$, the CD algorithm partially optimizes a target function $Q\left(\beta ;\lambda ,\gamma \right)$ with respect to a single parameter ${\beta }_j$ with the remaining parameters ${\beta }_l,l\ne j$ fixed at the updated values ${\widehat{\beta }}_1^{\lambda ,\gamma }\left(m+1\right),\dots ,{\widehat{\beta }}_{j-1}^{\lambda ,\gamma }\left(m+1\right),{\widehat{\beta }}_{j+1}^{\lambda ,\gamma }\left(m\right),\dots ,$ ${\widehat{\beta }}_p^{\lambda ,\gamma }\left(m\right)$, then cycling through all the parameters until convergence or a maximum iteration number $M$ is reached, and this process repeats over a grid of values for $\lambda$ to produce a path of the solution. For Ridge/LASSO/ENet penalty, variable selection is determined by the tuning parameter $\lambda$. In order to select an appropriate $\lambda$, we apply a cross-validation method to calculate the full solution path to model parameters, select a specific solution path from the full solution path, and take the binomial deviation as the risk measure. Then. we get the mean cross-validation error curve and the one-standard-deviation band. The parameter estimators for MCP/SCAD penalized logistic regression depend on $\lambda$ and the regularization parameter $\gamma$. For $\gamma$, we generally take $\gamma =3.7$. Algorithm 1 tells us how to apply the CD algorithm to calculate MCP estimator. The other four cases are similar to Algorithm 1.

Algorithm 1: The CD algorithm for MCP logistic regression.

Require: the training set ${\left\{x_t=\left(x_{t,1},x_{t,2},\cdots ,x_{t,p}\right),y_t\right\}}_{t=1}^n$, a grid of increasing $\lambda$ values $\mathrm{\Lambda }=\left\{{\lambda }_1,\dots ,{\lambda }_L\right\}$, $\gamma$, a given tolerance limit $\epsilon$ and a maximum iteration number M.

1: Initialization $\widehat{\beta }\left(0\right)=\widehat{\beta }\left({\lambda }_{\mathit{max}}={\lambda }_L,\gamma =5\right)$.

2: for each $m=0,1\cdots$, each $l\in \left\{L,L-1,\cdots ,1\right\}$, do.

3: repeat.

4: ${\widehat{\eta }}_t\Leftarrow {\beta }_0+x_t^{\top }{\widehat{\beta }}^{\lambda ,\gamma }\left(m\right)$.

5: ${\widehat{P}}_t\Leftarrow e^{{\widehat{\eta }}_t}/\left(1+e^{{\widehat{\eta }}_t}\right)$.

_6: $W\Leftarrow diag\left\{{\widehat{P}}_1\left(1-{\widehat{P}}_1\right),\cdots ,{\widehat{P}}_n\left(1-{\widehat{P}}_n\right)\right\}$.

7: $r\Leftarrow W^{-1}\left\{Y-\widehat{P}\right\}$.

8: $\tilde{Y}\Leftarrow \eta +r$.

9: while not convergent do.

10: for each $j\in \left\{1,2,\cdots ,p\right\}$ do.

11: ${\nu }_j\Leftarrow n^{-1}{x}_{\cdot j}^{\top }{\mathit{Wx}}_{\cdot j}$.

12: $\begin{array}{c}{Z}_j\Leftarrow \frac{1}{n}{x}_{\cdot j}^{\top }W\left(\tilde{Y}-x_{\cdot -j}{\beta }_{-j}\right)\\ \Leftarrow \frac{1}{n}{x}_{\cdot j}^{\top }\mathit{Wr}+v_j{\widehat{\beta }}_j^{\lambda ,\gamma }\left(m\right)\end{array}$.

13: if $\mid Z_j\mid ⩽{\nu }_j\mathit{\gamma \lambda }$ then.

14: ${\widehat{\beta }}_j^{\lambda ,\gamma }\left(m+1\right)\Leftarrow \frac{S\left(Z_j,\lambda \right)}{{\nu }_j-1/\gamma }$.

15: else.

16: ${\widehat{\beta }}_j^{\lambda ,\gamma }\left(m+1\right)\Leftarrow \frac{Z_j}{{\nu }_j}$.

17: end if.

18: $r\Leftarrow r-x_{\cdot j}^{\top }\left({\widehat{\beta }}_j^{\lambda ,\gamma }\left(m+1\right)-{\widehat{\beta }}_j^{\lambda ,\gamma }\left(m\right)\right)$.

19: end for.

20: end while.

21: until ${‖{\widehat{\beta }}^{\lambda ,\gamma }\left(m+1\right)-{\widehat{\beta }}^{\lambda ,\gamma }\left(m\right)‖}_2^2⩽\epsilon$ or do a maximum iteration number M.

22: end for.

Ensure: ${\widehat{\beta }}^{\lambda ,\gamma }$.

Now we apply the CD algorithm to the five penalized logistic regressions to obtain the final parameter estimators ${\widehat{\beta }}_0^{\lambda ,\gamma }$ and ${\widehat{\beta }}^{\lambda ,\gamma }$, then combine them with the testing set ${\left\{x_t,y_{t+1}^k\right\}}_{t=n_1+1}^{n_1+n_2}$ to compute the probability estimators

$$\widehat{P}\left(Y_t=1|X_t;{\widehat{\beta }}_0^{\lambda ,\gamma },{\widehat{\beta }}^{\lambda ,\gamma }\right)=\frac{exp\left({\widehat{\beta }}_0^{\lambda ,\gamma }+X_t^{\top }{\widehat{\beta }}^{\lambda ,\gamma }\right)}{1+exp\left({\widehat{\beta }}_0^{\lambda ,\gamma }+X_t^{\top }{\widehat{\beta }}^{\lambda ,\gamma }\right)},$$E9

$$\widehat{P}\left(Y_t=0|X_t;{\widehat{\beta }}_0^{\lambda ,\gamma },{\widehat{\beta }}^{\lambda ,\gamma }\right)=\frac{1}{1+exp\left({\widehat{\beta }}_0^{\lambda ,\gamma }+X_t^{\top }{\widehat{\beta }}^{\lambda ,\gamma }\right)}.$$E10

2.2 Three-group penalized logistic regression prediction methods

In this subsection, we propose G-LASSO/G-SCAD/G-MCP penalized logistic regressions with multiple technical indicators to predict up- or downtrends.

2.2.1 Three-group penalized logistic regressions

Suppose that the binary response $Y_t$ represents the stock price movement directions, and the p-dimensional predictor vector ${\mathbf{X}}_t={\left(X_{t,1},\dots ,X_{t,p}\right)}^{\top }$ represents $p$ technical indicators influencing stock price movement direction. The training set $\left\{\left({\mathbf{X}}_t,Y_t\right),t=1,\dots ,n_1\right\}$ is made up of the predictor vector ${\mathbf{X}}_t\in {\mathrm{ℝ}}^p$ allowing both categorical and continuous predictors and the binary response $Y_t\in \left\{0,1\right\}$. We divide the p-dimensional predictor vector ${\mathbf{X}}_t$ into the $G$ different groups ${\mathbf{x}}_t={\left({\mathbf{x}}_{t,1}^{\top },\dots ,{\mathbf{x}}_{t,g}^{\top },\dots ,{\mathbf{x}}_{t,G}^{\top }\right)}^{\top }$ with ${\mathbf{x}}_{t,g}=\left({\mathbf{X}}_{t,{\mathit{df}}_{g-1}+1},\dots ,{\mathbf{X}}_{t,{\mathit{df}}_g}\right)$ and the g-th group length ${\mathit{df}}_g$, $\mathrm{g}=1,\cdots ,G$. Then, the group logistic regression is

$$p_{\beta }\left({\mathbf{x}}_t\right)=P_{\beta }\left(Y_t=1|{\mathbf{x}}_t\right)=\frac{exp\left({\beta }_0+{\sum }_{g=1}^G{\mathbf{x}}_{t,g}^{\top }{\mathit{\beta }}_g\right)}{1+exp\left({\beta }_0+{\sum }_{g=1}^G{\mathbf{x}}_{t,g}^{\top }{\mathit{\beta }}_g\right)},$$E11

$$1-p_{\beta }\left({\mathbf{x}}_t\right)=P_{\beta }\left(Y_t=0|{\mathbf{x}}_t\right)=\frac{1}{1+exp\left({\beta }_0+{\sum }_{g=1}^G{\mathbf{x}}_{t,g}^{\top }{\mathit{\beta }}_g\right)},$$E12

where ${\beta }_0$ is the intercept, ${\mathit{\beta }}_g\in {\mathrm{ℝ}}^{{\mathrm{df}}_g}$ is the g-th parameter vector, and $\mathit{\beta }={\left({\beta }_0,{\mathit{\beta }}_1^{\top },\dots ,{\mathit{\beta }}_{\mathbf{G}}^{\top }\right)}^{\top }\in {\mathrm{ℝ}}^{p+1}$ is the whole parameter vector.

2.2.2 Parameter estimators and probability estimators

Based on the training set $\left\{\left({\mathbf{x}}_t,Y_t\right),t=1,\dots ,n_1\right\}$, we obtain the log-likelihood function

$$l\left(\beta \right)={\sum }_{t=1}^{n_1}\left\{Y_t{\eta }_{\beta }\left({\mathbf{x}}_t\right)-log\left[1+exp\left({\eta }_{\beta }\left({\mathbf{x}}_t\right)\right)\right]\right\}$$E13

with ${\eta }_{\beta }\left({\mathbf{x}}_t\right)=log\left\{\frac{p_{\beta }\left({\mathbf{x}}_t\right)}{1-p_{\beta }\left({\mathbf{x}}_t\right)}\right\}={\beta }_0+{\sum }_{g=1}^G{\mathbf{x}}_{t,g}^{\top }{\mathit{\beta }}_g,$ the penalized log-likelihood function is

$$S_{\lambda ,\gamma }\left(\mathit{\beta }\right)=-l\left(\mathit{\beta }\right)+{\sum }_{l=1}^{p}P\left({\beta }_l;\lambda ,\gamma \right),$$E14

and the group penalized log-likelihood is

$$S_{\lambda ,\gamma }\left(\mathit{\beta }\right)=-l\left(\mathit{\beta }\right)+{\sum }_{g=1}^{G}P\left(\parallel {\mathit{\beta }}_g{\parallel }_2;\lambda \sqrt{{\mathit{df}}_g},\gamma \right),$$E15

where $P\left(\parallel {\mathit{\beta }}_g{\parallel }_2;\lambda \sqrt{{\mathit{df}}_g},\gamma \right)$ with a tuning parameter $\lambda \ge 0$ and a regularization parameter $\gamma$ defines a family of penalty functions concave in $\parallel {\mathit{\beta }}_g{\parallel }_2$ . The estimator $\widehat{\mathit{\beta }}$ is the minimizer of the convex function $S_{\lambda ,\gamma }\left(\mathit{\beta }\right)$. Here, we take $P\left({\beta }_l;\lambda ,\gamma \right)$ as $P_{\mathit{\text{SCAD}}}\left({\beta }_l;\lambda ,\gamma \right)$ or $P_{\mathit{MCP}}\left({\beta }_l;\lambda ,\gamma \right)$, and take $P\left(\parallel {\mathit{\beta }}_g{\parallel }_2;\lambda \sqrt{{\mathit{df}}_g},\gamma \right)$ as $P_{\mathit{\text{gSCAD}}}\left(\parallel {\mathit{\beta }}_g{\parallel }_2;\lambda \sqrt{{\mathit{df}}_g},\gamma \right)$ or $P_{\mathit{\text{gMCP}}}\left(\parallel {\mathit{\beta }}_g{\parallel }_2;\lambda \sqrt{{\mathit{df}}_g},\gamma \right)$. Three group penalized functions are listed in Table 3. According to Breheny and Huang [19] and Fan and Li [20], the Bayes risks are not sensitive to the choice of $\gamma$. In general, we take $\gamma =3.7$ for SCAD/gSCAD (group SCAD) and $\gamma =3$ for MCP/gMCP(group MCP). According to Hu and Liu [21], we adopt the GCD algorithm to complete group selection and group estimations listed in Table 4.

Symbols	Three-group penalized functions
gLASSO	$P_{\mathit{\text{gLASSO}}}\left(\beta ;\lambda \right)=\lambda \sqrt{{\mathit{df}}_g}{‖{\mathit{\beta }}_g‖}_2.$
gSCAD	$P_{\mathit{\text{gSCAD}}}\left(\parallel {\mathit{\beta }}_g{\parallel }_2;\lambda \sqrt{{\mathit{df}}_g},\gamma \right)=\left\{\begin{array}{ll}\lambda \sqrt{{\mathit{df}}_g}{‖{\mathit{\beta }}_g‖}_2,& {‖{\mathit{\beta }}_g‖}_2\le \lambda \sqrt{{\mathit{df}}_g},\lambda \ge 0,\\ \frac{2\lambda \sqrt{{\mathit{df}}_g}\gamma {‖{\mathit{\beta }}_g‖}_2-\left({‖{\mathit{\beta }}_g‖}_2^2+{\lambda }^2{\mathit{df}}_g\right)}{2\left(\gamma -1\right)},& \lambda \sqrt{{\mathit{df}}_g}<{‖{\mathit{\beta }}_g‖}_2\le \lambda \sqrt{{\mathit{df}}_g}\gamma ,\\ \frac{{\lambda }^2{\mathit{df}}_g\left({\gamma }^2-1\right)}{2\left(\gamma -1\right)},& {‖{\mathit{\beta }}_g‖}_2>\lambda \sqrt{{\mathit{df}}_g}\gamma ,\gamma >2.\end{array}\right.$
gMCP	$P_{\mathit{\text{gMCP}}}\left(\parallel {\mathit{\beta }}_g{\parallel }_2;\lambda \sqrt{{\mathit{df}}_g},\gamma \right)=\left\{\begin{array}{ll}\lambda \sqrt{{\mathit{df}}_g}{‖{\mathit{\beta }}_g‖}_2-{‖{\mathit{\beta }}_g‖}_2^2/2\gamma ,& {‖{\mathit{\beta }}_g‖}_2\le \lambda \sqrt{{\mathit{df}}_g}\gamma ,\\ {\lambda }^2{\mathit{df}}_g\gamma /2,& {‖{\mathit{\beta }}_g‖}_2>\lambda \sqrt{{\mathit{df}}_g}\gamma ,\end{array}\right.$ $\lambda \ge 0$, $\gamma >1$.

Table 3.

gLASSO, gSCAD and gMCP functions.

Note: gLASSO stands for group LASSO.

Symbols	Three parameter estimators
gLASSO estimator	${\widehat{\beta }}^{\mathit{\text{gLASSO}}}=\frac{1}{v}S\left({\mathit{vZ}}_g,\sqrt{d_g}\lambda \right)$.
gSCAD estimator	${\widehat{\beta }}_g^{\mathit{\text{gSCAD}}}=\left\{\begin{array}{ll}\frac{1}{v}S\left({\mathit{vZ}}_g,\sqrt{d_g}\lambda \right),& \Vert Z_g\Vert \le 2\sqrt{d_g}\lambda ,\\ \frac{\gamma -1}{\gamma -2}\cdot \frac{1}{v}S\left({\mathit{vZ}}_g,\frac{\sqrt{d_g}\mathit{\lambda \gamma }}{\gamma -1}\right),& 2\sqrt{d_g}\lambda <\Vert Z_g\Vert \le \sqrt{d_g}\mathit{\lambda \gamma },\\ Z_g,& \Vert Z_g\Vert \ge \sqrt{d_g}\mathit{\lambda \gamma },\end{array}\right.$ $\gamma >2$.
gMCP estimator	${\widehat{\beta }}_g^{\mathit{\text{gMCP}}}=\left\{\begin{array}{ll}\frac{\gamma }{\gamma -1}\cdot \frac{1}{v}S\left({\mathit{vZ}}_g,\sqrt{d_g}\lambda \right),& \Vert Z_g\Vert \le \sqrt{d_g}\mathit{\lambda \gamma },\\ Z_g,& \Vert Z_g\Vert \ge \sqrt{d_g}\mathit{\lambda \gamma },\end{array}\right.$ $\gamma >1$.
Labels	$v_j=n_1^{-1}{X}_j^{\top }{\mathit{WX}}_j,j=1,\cdots ,p,Z_j=n_1^{-1}{X}_j^{\top }W\left(\tilde{Y}-X_{-j}{\beta }_{-j}\right),$
	$\tilde{Y}=X^{\top }{\beta }^{\left(m\right)}+W^{-1}\left(Y-P\right),X_{-j}=\left(X_1,\cdots X_{j-1},0,X_{j+1},\cdots ,X_p\right),$
	${\beta }_{-j}=\left({\beta }_1,\cdots ,{\beta }_{j-1},0,{\beta }_{j+1},\cdots ,{\beta }_p\right);$
	$v=\underset{t}{max}\underset{\eta }{sup}\left\{{\mathrm{\Delta }}^2{L}_t\left(\eta \right)\right\},L_t\left(\eta \right)=Y_t{\eta }_t-log\left(1+e^{{\eta }_t}\right),{\eta }_t={\beta }_0+{\sum }_{g=1}^G{X}_{t,g}^{\top }{\beta }_{\left(g\right)},t=1,\cdots ,n_1,$
	$Z_g=X_g^{\top }\left(\overline{Y}-{\mathrm{x}}_{-g}{\beta }_{-\left(g\right)}\right),g=1,\cdots ,G,\overline{Y}={\mathrm{x}}^{\top }{\beta }^{\left(m\right)}+W^{-1}\left(Y-P\right),$
	${\mathrm{x}}_{-g}=\left(X_{\left(1\right)}^{\top },\cdots X_{\left(g-1\right)}^{\top },0^{\top },X_{\left(g+1\right)}^{\top },\cdots ,X_{\left(G\right)}^{\top }\right),$
	${\beta }_{-\left(g\right)}=\left({\beta }_{\left(1\right)}^{\top },\cdots ,{\beta }_{\left(g-1\right)}^{\top },0^{\top },{\beta }_{\left(g+1\right)}^{\top },\cdots ,{\beta }_{\left(G\right)}^{\top }\right).$

Table 4.

The iterative parameter estimators for gLASSO, gSCAD and gMCP.

Note: $\mathit{X}={\left({\mathit{X}}_1,\dots ,{\mathit{X}}_{n_1}\right)}^{\top }$ is a $n_1\times p$ matrix, $\mathit{x}={\left({\mathit{x}}_1,\dots ,{\mathit{x}}_{n_1}\right)}^{\top }$ is a $n_1\times p$ matrix, $\mathit{Y}={\left(Y_1,\dots ,Y_{n_1}\right)}^{\top }$ is a $n_1\times 1$ vector, $W$ is a $n_1\times n_1$ weighted diagonal matrix, $P$ is the estimated probability of the m-th iteration, ${\beta }^{\left(m\right)}$ is the parameter estimator of the m-th iteration.

After applying the training set $\left\{\left({\mathbf{x}}_t,Y_t\right),t=1,\dots ,n_1\right\}$ and the GCD algorithm to obtain group estimation ${\widehat{\mathit{\beta }}}_g$, we apply the testing set $\left\{\left({\mathbf{x}}_t,Y_t\right),t=n_1+1,\dots ,n\right\}$ and ${\widehat{\mathit{\beta }}}_g$ to obtain the predicted probabilities for up- or downtrends:

$${\widehat{p}}_{\beta }\left({\mathbf{x}}_t\right)={\widehat{\mathrm{\mathbb{P}}}}_{\beta }\left(Y_t=1|{\mathbf{x}}_t\right)=\frac{exp\left({\widehat{\beta }}_0+{\sum }_{g=1}^G{\mathbf{x}}_{t,g}^{\top }{\widehat{\mathit{\beta }}}_g\right)}{1+exp\left({\widehat{\beta }}_0+{\sum }_{g=1}^G{\mathbf{x}}_{t,g}^{\top }{\widehat{\mathit{\beta }}}_g\right)},$$E16

$$1-{\widehat{p}}_{\beta }\left({\mathbf{x}}_t\right)={\widehat{\mathrm{\mathbb{P}}}}_{\beta }\left(Y_t=0|{\mathbf{x}}_t\right)=\frac{1}{1+exp\left({\widehat{\beta }}_0+{\sum }_{g=1}^G{\mathbf{x}}_{t,g}^{\top }{\widehat{\mathit{\beta }}}_g\right)},$$E17

where $n$ represents the sample size of the entire dataset. G-lasso estimators can be computed by R package grplasso. Now we estimate the predicted value ${\widehat{Y}}_t$ according to the following rules:

$$\text{If}{\widehat{P}}_t>c,\text{then}{\widehat{Y}}_t=1,\text{else}{\widehat{Y}}_t=0,$$E18

where $c$ is a given threshold. Raghavan et al. [22] tell us how to select the optimal threshold.

2.3 Two-class prediction performance

We introduce the two-class confusion matrix listed in Table 5 to assess two-class prediction performance.

True \ Predicted class	${\widehat{Y}}_t=1$	${\widehat{Y}}_t=2$	Total
$Y_t=1$	$V\left(1,1\right)$	$V\left(1,2\right)$	$V\left(1,\cdot \right)$
$Y_t=2$	$V\left(2,1\right)$	$V\left(2,2\right)$	$V\left(2,\cdot \right)$
Total	$V\left(\cdot ,1\right)$	$V\left(\cdot ,2\right)$	$V\left(\cdot ,\cdot \right)$

Table 5.

Two-class confusion matrix.

$V\left(k,\overline{k}\right)$ is the number of items that truly belong to class $k$ and are predicted as class $\overline{k}$.

According to Table 5, one can compute

$$\text{Accuracy}=\frac{\mathit{TP}+\mathit{TN}}{\mathit{TP}+\mathit{TN}+\mathit{FP}+\mathit{FN}},\text{Precision}=\frac{\mathit{TP}}{\mathit{TP}+\mathit{FP}}.$$

In fact, accuracy cannot reflect the losses from two types of errors. However, a ROC curve can clearly assess two-class prediction performance. For a given threshold c, $\mathit{TPR}\left(c\right)=P\left(Y_t<c\right)$ is true-positive rate, and $\mathit{FPR}\left(c\right)=P\left(1-Y_t<c\right)$ is false-positive rate. Given the different threshold $c$, we can calculate $\left\{\left(\mathit{TPR}\left(c\right),\mathit{FPR}\left(c\right)\right)\right\}$ or (Sensitivity, 1-Specificity) and draw a ROC curve, where

$$\text{Sensitivity}\left(\text{True positive rate},\mathrm{TPR}\right)=\mathit{TP}/\left(\mathit{TP}+\mathit{FN}\right)$$E19

measures the ability of classifiers to correctly identify positive samples, and

$$\text{Specificity}\left(1‐\text{False positive rate},1‐\mathrm{FPR}\right)=\mathit{TN}/\left(\mathit{TN}+\mathit{FP}\right)$$E20

measures the ability to correctly identify negative labels. ROC is a graphical method to evaluate the performance of the binary classifier. Here, we choose ggplot2 to draw the ROC curve and compute the corresponding AUC (the area under the ROC curve).

3. Predicting uptrends, sideways trends and downtrends

3.1 Three-group penalized trinomial logit prediction methods

Let $C_t$ be the closing price at the end of the t-th trading day, $Z_t=C_t-C_{t-k}$ be the $k$ period stock excess return, and

$$Y_t\left(c\right)=\left\{\begin{array}{l}1,\text{if}{Z}_t>c,\\ 2,\text{if}-c\le Z_t\le c,\\ 3,\text{if}{Z}_t<-c.\end{array}\right.$$E21

be an unordered three-category response variable. Then, $Y_t\left(c\right)=1$ represents uptrends, $Y_t\left(c\right)=2$ represents sideways trends, and $Y_t\left(c\right)=3$ represents downtrends. When $c=0$, $Y_t\left(c\right)$ is an indicator for positive returns. When $c=0.1C_{t-1}$, $Y_t\left(c\right)$ is an indicator for “large” positive returns. When $c=-0.1C_{t-1}$, $Y_t\left(c\right)$ is an indicator for “large” negative returns. More details on how to choose $c$ may refer to [23]. Suppose that the predictive vector ${\mathbf{X}}_t={\left(X_{t,1},\dots ,X_{t,p}\right)}^{\top }$ allowing both categorical and continuous predictors can represent the factors influencing stock return movement direction. One main goal is to predict stock return movement direction $Y_{t+1}\left(c\right)$ using the available information set $I_t=\sigma \left\{X_t,X_{t-1},\cdots \right\}$ at time t. If $Y_{t+1}\left(c\right)$ only depend on ${\mathbf{X}}_t$, then we can predict $Y_{t+1}\left(c\right)$ based on ${\mathbf{X}}_t$. Trinomial logit dynamic model for stock return movement direction is

$$\begin{array}{lll}& P\left(Y_{t+1}\left(c\right)=1|x_t;\beta \right)=\frac{exp\left({\beta }_0^1+{\sum }_{g=1}^G{\mathbf{x}}_{t\left(g\right)}{\mathit{\beta }}_{\left(g\right)}^1\right)}{{\sum }_{k=1}^{3}exp\left({\beta }_0^k+{\sum }_{g=1}^5{\mathbf{x}}_{t\left(g\right)}{\mathit{\beta }}_{\left(g\right)}^k\right)},& \\ & P\left(Y_{t+1}\left(c\right)=2|x_t;\beta \right)=\frac{exp\left({\beta }_0^2+{\sum }_{g=1}^G{\mathbf{x}}_{t\left(g\right)}{\mathit{\beta }}_{\left(g\right)}^2\right)}{{\sum }_{k=1}^{3}exp\left({\beta }_0^k+{\sum }_{g=1}^5{\mathbf{x}}_{t\left(g\right)}{\mathit{\beta }}_{\left(g\right)}^k\right)},& \\ & P\left(Y_{t+1}\left(c\right)=3|x_t;\beta \right)=\frac{exp\left({\beta }_0^3+{\sum }_{g=1}^G{\mathbf{x}}_{t\left(g\right)}{\mathit{\beta }}_{\left(g\right)}^3\right)}{{\sum }_{k=1}^{3}exp\left({\beta }_0^k+{\sum }_{g=1}^5{\mathbf{x}}_{t\left(g\right)}{\mathit{\beta }}_{\left(g\right)}^k\right)},& \end{array}$$E22

where the parameter vector $\beta =\left({\beta }_0^1,{\left({\beta }^1\right)}^{\top },{\beta }_0^2,{\left({\beta }^2\right)}^{\top },{\beta }_0^3,{\left({\beta }^3\right)}^{\top }\right)$ with the k-th class intercept ${\beta }_0^k\in R$ and the g-th group parameter vector from the k-th class ${\mathit{\beta }}_{\left(g\right)}^k={\left({\beta }_{{\mathit{df}}_{g-1}+1},\dots ,{\beta }_{{\mathit{df}}_g}\right)}^{\top },$ $g=1,\cdots ,5$. The k-th class parameter vector is ${\beta }^k={\left({\left({\beta }_{\left(1\right)}^k\right)}^{\top },\cdots ,{\left({\beta }_{\left(G\right)}^k\right)}^{\top }\right)}^{\top },k=\mathrm{1,2,3}.$ According to the training set $\left\{\left({\mathbf{X}}_t,Y_{t+1}\left(c\right)\right),t=1,\dots ,n_1\right\}$ and $y_{t+1}^k=I\left(Y_{t+1}\left(c\right)=k\right)$, we obtain the group trinomial logit log-likelihood function

$$l\left(\beta \right)={\sum }_{t=1}^{n_1}\left\{{\sum }_{k=1}^3{y}_{t+1}^k\left({\beta }_0^k+{\mathrm{x}}_t{\beta }^k\right)-log\left[{\sum }_{k=1}^{3}exp\left({\beta }_0^k+{\mathrm{x}}_t{\beta }^k\right)\right]\right\}$$E23

and the corresponding group penalized trinomial logit log-likelihood function

$$Q^{\lambda ,\gamma }\left(\beta \right)\equiv R_{n_1}\left(\beta \right)+{\sum }_{k=1}^3{\sum }_{g=1}^G\mathbf{P}\left(\parallel {\beta }_{\left(g\right)}^k\parallel ;\lambda \sqrt{p_g},\gamma \right),$$E24

where $R_{n_1}\left(\beta \right)=\frac{1}{n_1}{\sum }_{t=1}^{n_1}{\gamma }_{\beta }\left(x_t,y_{t+1}^k\right)=-\frac{1}{N}{\sum }_{t=1}^{n_1}\left\{{\sum }_{k=1}^3{y}_{t+1}^k\left({\beta }_0^k+x_t{\beta }^k\right)-log\left[{\sum }_{k=1}^{3}exp\left({\beta }_0^k+x_t{\beta }^k\right)\right]\right\}$ is the empirical risk to $R\left(\beta \right)=E{\gamma }_{\beta }\left(x,y\right)$ with $y=\left(y^1,y^2,y^3\right)$ and the loss function

$${\gamma }_{\beta }\left(x,y\right)=-\left\{{\sum }_{k=1}^3{y}^k\left({\beta }_0^k+x{\beta }^k\right)-log\left[{\sum }_{k=1}^{3}exp\left({\beta }_0^k+x{\beta }^k\right)\right]\right\},$$E25

$\mathbf{P}\left(\parallel {\beta }_{\left(g\right)}^k\parallel ;\lambda \sqrt{p_g},\gamma \right)$ defines a family of penalty functions. Next, we introduce the three group penalized criterions:

1. G-LASSO penalized function

   $$Q_{G-\mathit{\text{LASSO}}}^{\lambda ,\gamma }\left(\beta \right)\equiv R_{n_1}\left(\beta \right)+{\sum }_{k=1}^3{\sum }_{g=1}^G\lambda \sqrt{p_g}\parallel {\beta }_{\left(g\right)}^k\parallel ;$$E26

2. G-SCAD penalized function

   $$Q_{G-\mathit{\text{SCAD}}}^{\lambda ,\gamma }\left(\beta \right)\equiv R_{n_1}\left(\beta \right)+{\sum }_{k=1}^3{\sum }_{g=1}^G{\mathbf{P}}_{G-\mathit{\text{SCAD}}}\left(\parallel {\beta }_{\left(g\right)}^k\parallel ;\lambda \sqrt{p_g},\gamma \right)$$E27

   for $\lambda \ge 0$ and $\gamma >2$, where

   $$\begin{array}{l}{\mathbf{P}}_{G-\mathit{\text{SCAD}}}\left(\parallel {\beta }_{\left(g\right)}^k\parallel ;\lambda \sqrt{p_g},\gamma \right)=\lambda \sqrt{p_g}{\int }_0^{\parallel {\beta }_{\left(g\right)}^k\parallel }min\left\{1,{\left(\gamma -\lambda \sqrt{p_g}x\right)}_{+}/\left(\gamma -1\right)\right\}\mathit{dx}\\ =\left\{\begin{array}{l}\lambda \sqrt{p_g}\parallel {\beta }_{\left(g\right)}^k\parallel ,\text{if}\parallel {\beta }_{\left(g\right)}^k\parallel \le \lambda \sqrt{p_g},\\ \frac{2\lambda \sqrt{p_g}\gamma \parallel {\beta }_{\left(g\right)}^k\parallel -\left(\parallel {\beta }_{\left(g\right)}^k{\parallel }^2+{\lambda }^2{p}_g\right)}{2\left(\gamma -1\right)},\text{if}\lambda \sqrt{p_l}<\parallel {\beta }_{\left(g\right)}^k\parallel \le \lambda \sqrt{p_g}\gamma ,\\ \frac{{\lambda }^2{p}_g\left({\gamma }^2-1\right)}{2\left(\gamma -1\right)},\text{if}\parallel {\beta }_{\left(g\right)}^k\parallel >\lambda \sqrt{p_g}\gamma \end{array}\right.\end{array}$$

   and $x_{+}=x{1}_{\left\{x\ge 0\right\}}$ represents the nonnegative part of $x$.

3. G-MCP penalized function

$$Q_{G-\mathit{MCP}}^{\lambda ,\gamma }\left(\beta \right)\equiv R_{n_1}\left(\beta \right)+{\sum }_{k=1}^3{\sum }_{g=1}^G{\mathbf{P}}_{G-\mathit{MCP}}\left(\parallel {\beta }_{\left(g\right)}^k\parallel ;\lambda \sqrt{p_g},\gamma \right)$$E28

for $\lambda \ge 0$ and $\gamma >1$, where

$$\begin{array}{l}{\mathbf{P}}_{G‐\mathit{MCP}}\left(\parallel {\beta }_{\left(g\right)}^k\parallel ;\lambda \sqrt{p_g},\gamma \right)=\lambda \sqrt{p_g}{\int }_0^{\parallel {\beta }_{\left(g\right)}^k\parallel }{\left(1-x/\left(\lambda \sqrt{p_g}\gamma \right)\right)}_{+}\mathit{dx}\\ =\lambda \sqrt{p_g}\cdot \left(\parallel {\beta }_{\left(g\right)}^k\parallel -\parallel {\beta }_{\left(g\right)}^k{\parallel }^2/\left(2\lambda \sqrt{p_g}\gamma \right)\right)I\left(\parallel {\beta }_{\left(g\right)}^k\parallel <\mathit{\lambda \gamma }\right)+{\lambda }^2{p}_g\gamma /2\cdot I\left(\parallel {\beta }_{\left(g\right)}^k\parallel \ge \mathit{\lambda \gamma }\right)\\ =\left\{\begin{array}{l}\lambda \sqrt{p_g}\parallel {\beta }_{\left(g\right)}^k\parallel -\parallel {\beta }_{\left(g\right)}^k{\parallel }^2/2\gamma ,\text{if}\parallel {\beta }_{\left(g\right)}^k\parallel \le \lambda \sqrt{p_g}\gamma ,\\ {\lambda }^2{p}_g\gamma /2,\text{if}\parallel {\beta }_{\left(g\right)}^k\parallel >\lambda \sqrt{p_g}\gamma .\end{array}\right.\end{array}$$

3.2 GCD algorithm completes group selection and group estimations

For $g$ in $\left\{1,2,\dots ,G\right\}$, GCD partially optimizes the target function

$$\begin{array}{l}{Q}^{\lambda ,\gamma }\left(\beta \right)\equiv -\frac{1}{n_1}{\sum }_{t=1}^{n_1}\left\{{\sum }_{k=1}^3{y}_{t+1}^k\left({\beta }_0^k+x_t{\beta }^k\right)-log\left[{\sum }_{k=1}^{3}exp\left({\beta }_0^k+x_t{\beta }^k\right)\right]\right\}\\ +{\sum }_{k=1}^3{\sum }_{g=1}^G\mathbf{P}\left(\parallel {\beta }_{\left(g\right)}^k\parallel ;\lambda \sqrt{p_g},\gamma \right)\end{array}$$

for a single group ${\beta }_{\left(g\right)}$ with the other groups ${\beta }_{\left(\overline{g}\right)},\overline{g}\ne g$ fixed at the updated values ${\tilde{\beta }}_{\left(1\right)}^k\left(m+1\right),\dots ,{\tilde{\beta }}_{\left(g-1\right)}^k\left(m+1\right),{\tilde{\beta }}_{\left(g+1\right)}^k\left(m\right),\dots ,$ ${\tilde{\beta }}_{\left(G\right)}^k\left(m\right)$, then iteratively cycling through all the groups until convergence or a maximum iteration is reached. Now, we develop the GCD algorithm for G-LASSO/G-SCAD/G-MCP penalized trinomial logit dynamic models. We perform partial Newton steps by forming the partial quadratic approximation

$$R_{\mathit{Qk}}\left({\beta }_0^k,{\left({\beta }^k\right)}^{\top }\right)=\frac{1}{2n_1}{\sum }_{t=1}^{n_1}{\omega }_t^k{\left({\zeta }_t^k-{\beta }_0^k-x_t^{\top }{\beta }^k\right)}^2-C\left({\left\{{\tilde{\beta }}_0^{\overline{k}},{\tilde{\beta }}_1^{\overline{k}}\right\}}_{\overline{k}=1}^3\right)$$

to the empirical risk $R_{n_1}\left(\beta \right)=-\frac{1}{n_1}{\sum }_{t=1}^N\left\{{\sum }_{k=1}^3{y}_{t+1}^k\left({\beta }_0^k+x_t{\beta }^k\right)-log\left[{\sum }_{k=1}^{3}exp\left({\beta }_0^k+x_t{\beta }^k\right)\right]\right\}$ or Taylor expansion about current estimates ${\left\{{\tilde{\beta }}_0^{\overline{k}},{\tilde{\beta }}_1^{\overline{k}}\right\}}_{\overline{k}=1}^3$, allowing only ${\left({\beta }_0^k,{\left({\beta }^k\right)}^{\top }\right)}^{\top }$ to vary for a single class at a time, where ${\omega }_t^k={\widehat{P}}_t^k\left(1-{\widehat{P}}_t^k\right)$, ${\widehat{P}}_t^k=exp\left({\tilde{\beta }}_0^k+x_t{\tilde{\beta }}^k\right)/\left[{\sum }_{k=1}^{3}exp\left({\tilde{\beta }}_0^k+x_t{\tilde{\beta }}^k\right)\right]$, and ${\zeta }_t^k={\tilde{\beta }}_0^k+x_t^{\top }{\tilde{\beta }}^k+\left(y_{t+1}^k-{\widehat{P}}_t^k\right)/{\omega }_t^k$. For each given $\left(\lambda ,\gamma \right)$, we create an outer loop which cycles over $k$ and compute $R_{\mathit{Qk}}\left({\beta }_0^k,{\left({\beta }^k\right)}^{\top }\right)$ about current estimates ${\left\{{\tilde{\beta }}_0^{\overline{k}},{\tilde{\beta }}_1^{\overline{k}}\right\}}_{\overline{k}=1}^3$. Then, we apply the GCD algorithm to solve the negative group penalized weighted least-squares problem

$$\underset{\left({\beta }_0^k,{\left({\beta }^k\right)}^{\top }\right)\in R^{{\sum }_{g=1}^G{p}_l+1}}{min}\left\{R_{\mathit{Qk}}({\beta }_0^k,{\left({\beta }^k\right)}^{\top })+{\sum }_{g=1}^G\mathbf{P}(\parallel {\beta }_{\left(g\right)}^k\parallel ;\lambda \sqrt{p_l},\gamma )\right\}.$$

The main four steps are as follows:

Step 1. Set $m=0$ and ${\tilde{\beta }}^k\left(0\right)={\left({\tilde{\beta }}_0^k\left(0\right),{\left({\tilde{\beta }}_{\left(1\right)}^k\left(0\right)\right)}^{\top },\dots ,{\left({\tilde{\beta }}_{\left(G\right)}^k\left(0\right)\right)}^{\top }\right)}^{\top }$ be the initial value. Calculate ${\widehat{P}}^k\left(0\right)={\left({\widehat{P}}_1^k\left(0\right),{\widehat{P}}_2^k\left(0\right),\dots ,{\widehat{P}}_{n_1}^k\left(0\right)\right)}^{\top }$ with

$${\widehat{P}}_t^k\left(0\right)=\frac{exp\left({\tilde{\beta }}_0^k\left(0\right)+{\sum }_{g=1}^G{x}_{t\left(g\right)}{\tilde{\beta }}_{\left(g\right)}^k\left(0\right)\right)}{{\sum }_{k=1}^{3}exp\left({\tilde{\beta }}_0^k\left(0\right)+{\sum }_{g=1}^G{x}_{t\left(g\right)}{\tilde{\beta }}_{\left(g\right)}^k\left(0\right)\right)},W^k\left(0\right)=diag\left({\omega }_1^k\left(0\right),{\omega }_2^k\left(0\right),\dots ,{\omega }_{n_1}^k\left(0\right)\right)$$

with ${\omega }_t^k\left(0\right)={\widehat{P}}_t^k\left(0\right)\left(1-{\widehat{P}}_t^k\left(0\right)\right),$ and obtain the initial residual vector $r^k\left(0\right)={\left(r_1^k\left(0\right),r_2^k\left(0\right),\dots ,r_N^k\left(0\right)\right)}^{\top }$ with $r_t^k\left(0\right)=\left(y_{t+1}^k-{\widehat{P}}_t^k\left(0\right)\right)/w_t^k\left(0\right).$.

Step 2. At the g-th step of $m+1$ iterations, $g=1,\dots ,G$, carry out (A)-(C):

1. Calculate ${\omega }_t^k\left(m\right)={\widehat{P}}_t^k\left(m\right)\left(1-{\widehat{P}}_t^k\left(m\right)\right),{\widehat{P}}^k\left(m\right)={\left({\widehat{P}}_1^k\left(m\right),{\widehat{P}}_2^k\left(m\right),\dots ,{\widehat{P}}_{n_1}^k\left(m\right)\right)}^{\top },W^k\left(m\right)$ $=diag\left({\omega }_1^k\left(m\right),{\omega }_2^k\left(m\right),\dots ,{\omega }_{n_1}^k\left(m\right)\right)$ with ${\widehat{P}}_t^k\left(m\right)=\frac{exp\left({\tilde{\beta }}_0^k\left(m\right)+{\sum }_{\overline{g}=1}^{g-1}{x}_{t\left(\overline{g}\right)}{\tilde{\beta }}_{\left(\overline{g}\right)}^k\left(m\right)+{\sum }_{\overline{g}=g}^G{x}_{t\left(\overline{g}\right)}{\tilde{\beta }}_{\left(\overline{g}\right)}^k\left(m-1\right)\right)}{{\sum }_{k=1}^{3}exp\left({\tilde{\beta }}_0^k\left(m\right)+{\sum }_{\overline{g}=1}^{g-1}{x}_{t\left(\overline{g}\right)}{\tilde{\beta }}_{\left(\overline{g}\right)}^k\left(m\right)+{\sum }_{\overline{g}=g}^G{x}_{t\left(\overline{g}\right)}{\tilde{\beta }}_{\left(\overline{g}\right)}^k\left(m-1\right)\right)},$ and obtain the m-th iterations’ residual vector $r^k\left(m\right)={\left(r_1^k\left(m\right),r_2^k\left(m\right),\dots ,r_N^k\left(m\right)\right)}^{\top }$ with

   $$r_t^k\left(m\right)=\left(y_{t+1}^k-{\widehat{P}}_t^k\left(m\right)\right)/w_t^k\left(m\right).$$

   Then, compute the $p_l\times p_l$ matrix ${\nu }_{\left(g\right)}^k\left(m\right)=n_1^{-1}{x}_{\left(g\right)}^{\top }{W}^k\left(m\right)x_{\left(g\right)},$ $g=1,2,\dots ,G,$ and the $p_l\times 1$ vector $z_{\left(g\right)}^k\left(m\right)=n_1^{-1}{x}_{\left(g\right)}^{\top }{W}^k\left(m\right)r^k\left(m\right)+{\nu }_{\left(g\right)}^k\left(m\right){\tilde{\beta }}_{\left(g\right)}^k\left(m\right),$ $g=1,2,\dots ,G.$

2. Update the g-th group estimator ${\tilde{\beta }}_{\left(g\right)}^k\left(m+1\right)$:

   1. G-LASSO estimator

      $${\tilde{\beta }}_{\left(g\right)}^k\left(m+1\right)\leftarrow \frac{S\left(z_{\left(g\right)}^k\left(m\right),\lambda \sqrt{p_g}\right)}{{\nu }_{\left(g\right)}^k\left(m\right)},$$E29

      where the soft-thresholding operator

      $$S\left(z_{\left(g\right)}^k\left(m\right),\lambda \sqrt{p_g}\right)=S\left(\parallel z_{\left(g\right)}^k\left(m\right)\parallel ,\lambda \sqrt{p_g}\right)\frac{z_{\left(g\right)}^k\left(m\right)}{\parallel z_{\left(g\right)}^k\left(m\right)\parallel }={\left(1-\lambda \sqrt{p_g}/\parallel z_{\left(g\right)}^k\left(m\right)\parallel \right)}_{+}{z}_{\left(g\right)}^k\left(m\right).$$

   2. G-SCAD estimator

      $${\tilde{\beta }}_{\left(g\right)}^k\left(m+1\right)\leftarrow \left\{\begin{array}{l}\frac{S\left(z_{\left(g\right)}^k\left(m\right),\lambda \sqrt{p_g}\right)}{{\nu }_{\left(g\right)}^k\left(m\right)},\text{if}\parallel z_{\left(g\right)}^k\left(m\right)\parallel \le \lambda \sqrt{p_g}\left(\parallel {\nu }_{\left(g\right)}^k\left(m\right)\parallel +1\right),\\ \frac{S\left(z_{\left(g\right)}^k\left(m\right),\lambda \sqrt{p_g}\gamma /\left(\gamma -1\right)\right)}{{\nu }_{\left(g\right)}^k\left(m\right)-1/\left(\gamma -1\right)},\text{if}\lambda \sqrt{p_g}\left(\parallel {\nu }_{\left(g\right)}^k\left(m\right)\parallel +1\right)<\parallel z_{\left(g\right)}^k\left(m\right)\parallel \le \parallel {\nu }_{\left(g\right)}^k\left(m\right)\parallel \lambda \sqrt{p_g}\gamma ,\\ \frac{z_{\left(g\right)}^k\left(m\right)}{{\nu }_{\left(g\right)}^k\left(m\right)},\text{if}\parallel z_{\left(g\right)}^k\left(m\right)\parallel >\parallel {\nu }_{\left(g\right)}^k\left(m\right)\parallel \lambda \sqrt{p_g}\gamma ,\gamma >1+1/c_{\ast }\left({\beta }_{\left(g\right)}^k\right),\end{array}\right.$$E30

      where $c_{\ast }\left({\beta }_{\left(g\right)}^k\right)$ denotes the minimum eigenvalue of $N^{-1}{x}_{\left(g\right)}^{\top }{W}^k\left(m\right)x_{\left(g\right)}$.

   3. G-MCP estimator

      $${\tilde{\beta }}_{\left(g\right)}^k\left(m+1\right)\leftarrow \left\{\begin{array}{l}\frac{S\left(z_{\left(g\right)}^k\left(m\right),\lambda \sqrt{p_g}\right)}{{\nu }_{\left(g\right)}^k\left(m\right)-1/\gamma },\text{if}\parallel z_{\left(g\right)}^k\left(m\right)\parallel \le \parallel {\nu }_{\left(g\right)}^k\left(m\right)\parallel \lambda \sqrt{p_g}\gamma ,\\ \frac{z_{\left(g\right)}^k\left(m\right)}{{\nu }_{\left(g\right)}^k\left(m\right)},\text{if}\parallel z_{\left(g\right)}^k\left(m\right)\parallel >\parallel {\nu }_{\left(g\right)}^k\left(m\right)\parallel \lambda \sqrt{p_g}\gamma ,\gamma >1/c_{\ast }\left({\beta }_{\left(g\right)}^k\right).\end{array}\right.$$E31

(C) update $r^k\left(m+1\right)\leftarrow r^k\left(m\right)-x^{\top }\left({\tilde{\beta }}_{\left(g\right)}^k\left(m+1\right)-{\tilde{\beta }}_{\left(g\right)}^k\left(m\right)\right)$.

Step 3. Update $m\leftarrow m+1$.

Step 4. Repeat steps 2–3 until convergence or the maximum iteration number is reached, the iterations stop and the final estimators are obtained.

For $\lambda$ depending on the sample size, the group number, and the degree of freedom within each group, we can apply the grid search method to $\lambda$ on a grid $\left\{{\lambda }_0,\dots ,{\lambda }_{K+1}\right\}$. Let ${\lambda }_{\mathit{min}}={\lambda }_0>{\lambda }_1>\dots >{\lambda }_K>{\lambda }_{K+1}={\lambda }_{\mathit{max}}=\underset{1\le g\le s}{max}\parallel z_{\left(g\right)}^k\left(m\right)\parallel ,$ where $z_{\left(g\right)}^k\left(m\right)=N^{-1}{x}_{\left(g\right)}{W}^k\left(m\right)r^k\left(m\right)+{\nu }_{\left(g\right)}^k\left(m\right){\tilde{\beta }}_{\left(g\right)}^k\left(m\right)$ and $W^k\left(m\right),r^k\left(m\right),{\nu }_{\left(g\right)}^k\left(m\right)$ and ${\tilde{\beta }}_{\left(g\right)}^k\left(m\right)$. For any given $\gamma$, the algorithm starts at ${\lambda }_{\mathit{min}}$ and proceeding toward ${\lambda }_{\mathit{max}}$. Each time sets the initial value of ${\beta }_{\left(g\right)}^k$ to be ${\tilde{\beta }}_{\left(g\right)}^k\left(m\right)$ estimated from the previous grid point to ensure that the initial values will never be far from the solution. To find the optimal $\lambda$ on a grid $\left\{{\lambda }_0,\dots ,{\lambda }_{K+1}\right\}$, we divide the stock dataset into S subsets ${\left\{S_k\right\}}_{k=1}^S$ with the same sample size and apply the S-fold cross-validation procedure to choose the optimal $\lambda$.

3.3 Three-class prediction performance

We firstly combine the GCD algorithm with the training set ${\left\{x_t,y_{t+1}^k\right\}}_{t=1}^{n_1}$ to learn G-LASSO/G-SCAD/G-MCP penalized trinomial logit dynamic models, then combine group estimators

$$\tilde{\beta }={\left({\tilde{\beta }}_0^1,{\left({\tilde{\beta }}^1\right)}^{\top },{\tilde{\beta }}_0^2,{\left({\tilde{\beta }}^2\right)}^{\top },{\tilde{\beta }}_0^3,{\left({\tilde{\beta }}^3\right)}^{\top }\right)}^{\top }$$

with the testing set ${\left\{x_t,y_{t+1}^k\right\}}_{t=n_1+1}^{n_1+n_2}$ to compute class probability estimations

$$\tilde{P}\left(y_{t+1}^k=k|X_t;\tilde{\beta }\right)=\frac{exp\left({\tilde{\beta }}_0^k+x_t{\tilde{\beta }}^k\right)}{{\sum }_{\overline{k}=1}^{3}exp\left({\tilde{\beta }}_0^{\overline{k}}+x_t{\tilde{\beta }}^{\overline{k}}\right)},k=\mathrm{1,2,3},$$E32

and finally introduce the Bayes classifier to predict class indexes

$${\widehat{Y}}_{t+1}\left(c\right)=k\text{if}\tilde{P}\left(y_{t+1}^k=k|X_t;\tilde{\beta }\right)=\underset{\overline{k}\in \left\{\mathrm{1,2,3}\right\}}{max}\tilde{P}\left(y_{t+1}^k=\overline{k}|X_t;\tilde{\beta }\right).$$E33

Table 6 lists three-class confusion matrix for assessing the prediction performance.

True \ Predicted class	${\widehat{Y}}_t=1$	${\widehat{Y}}_t=2$	${\widehat{Y}}_t=3$	Total
$Y_t=1$	$V\left(1,1\right)$	$V\left(1,2\right)$	$V\left(1,3\right)$	$V\left(1,\cdot \right)$
$Y_t=2$	$V\left(2,1\right)$	$V\left(2,2\right)$	$V\left(2,3\right)$	$V\left(2,\cdot \right)$
$Y_t=3$	$V\left(3,1\right)$	$V\left(3,2\right)$	$V\left(3,3\right)$	$V\left(3,\cdot \right)$
Total	$V\left(\cdot ,1\right)$	$V\left(\cdot ,2\right)$	$V\left(\cdot ,3\right)$	$V\left(\cdot ,\cdot \right)$

Table 6.

Three-class confusion matrix.

$V\left(k,\overline{k}\right)$ is the number of items that truly belong to class $k$ and are predicted as class $\overline{k}$. $V\left(k,\cdot \right)$ is the number of items that truly belong to class $k$. $V\left(\cdot ,\overline{k}\right)$ is the number of items that were predicted as class $\overline{k}$.

From Table 1, one can compute Accuracy. For unordered multi-category response variables, some traditional accuracy measures such as AUC are not applicable, a few important statistical concepts for assessing multi-category classification accuracy have been proposed. For example, $\mathbf{\text{Kappa}}\mathbf{\text{coefficient}}\left(\mathbf{\text{Kappa}}\right)=\left({\sum }_{k=1}^K{p}_{\mathit{kk}}-{\sum }_{k=1}^K{p}_{k+}{p}_{+k}\right)/\left(1-{\sum }_{k=1}^K{p}_{k+}{p}_{+k}\right)$ with $p_{\mathit{kk}}=P\left(Y_t=k,{\widehat{Y}}_t=k\right),p_{k+}=P\left(Y_t=k\right),p_{+k}=P\left({\widehat{Y}}_t=k\right)$ measures multi-category classification accuracy, and $\mathbf{\text{Kappa}}\in \left(0,0.2\right),$ (0.2,0.4), (0.4,0.6), (0.6,0.8), (0.8,1.0) represent no consistency, slightly consistency, mild consistency, reasonably consistency, strong consistency, almost identical, respectively. PDI evaluates the probability of an event related to simultaneously classifying $n$ subjects from $K$ categories. For a fixed category $k$, define the class-specific PDI to be ${\mathrm{PDI}}_k=P(p_{\mathit{kk}}>p_{\mathit{jk}},j\ne k{Y}_t=k)$ with $p_{\mathit{ij}}$ be the probability of placing a subject from category $i$ into category $j$. The overall PDI is $\mathrm{PDI}={\sum }_{k=1}^K{\mathrm{PDI}}_k/K$.

According to the three-class confusion matrices and the three testing sets, we compute sensitivity and specificity for class 1–3. Plotting true classification rates in three dimensions (3D) obtains the 3D ROC surface in the unit cube. HUM is a reflection of the intrinsic accuracy of three-class classifier. Here, we choose R package mcca to draw the ROC surfaces. The HUM value is higher, and the prediction performance is better. More details see Refs. [24, 25].

4. Conclusions and prospects

In this chapter, we combine different technical indicators with five penalized logistic regressions, three-group penalized logistic regressions and three-group penalized trinomial logit regressions to predict stock return movement direction. The future research focuses on adding more variables such as fundamental factors, specific news, national policies and market sentiment to the proposed regressions, and further improves the prediction performance to stock return movement direction.

5. Discussions

In the aforementioned methods, it is very important to select important (group) variables by tuning two parameters $\lambda$ and $\gamma$. The different tuning parameters will bring the different prediction performance. However, after we choose the relative optimum tuning parameters $\lambda$ and $\gamma$ by some model selection criterion such as cross-validation and BIC, the proposed methods perform very robust in terms of accuracy, AUC or HUM, Kappa and PDI. In addition, the proposed CD or GCD algorithm converges very rapidly. They can be generalized to high-dimensional or super high-dimensional cases by containing more variables influencing stock movement directions. From Refs. [13, 14, 15], we can found that the proposed methods outperform some classical machine learning or deep learning methods in terms of robustness and effectiveness and calculation. Therefore, these prediction methods are worth of further promotion.