Triple the gamma: A unifying shrinkage prior for variance and variable selection in sparse state space and TVP models
Abstract
EconStor is a publication server for scholarly economic literature, provided as a non-commercial public service by the ZBW.
Full text
Cadonna, Annalisa; Frühwirth-Schnatter, Sylvia; Knaus, Peter Article Triple the gamma: A unifying shrinkage prior for variance and variable selection in sparse state space and TVP models Econometrics Provided in Cooperation with: MDPI – Multidisciplinary Digital Publishing Institute, Basel Suggested Citation: Cadonna, Annalisa; Frühwirth-Schnatter, Sylvia; Knaus, Peter (2020) : Triple the gamma: A unifying shrinkage prior for variance and variable selection in sparse state space and TVP models, Econometrics, ISSN 2225-1146, MDPI, Basel, Vol. 8, Iss. 2, pp. 1-36, https://doi.org/10.3390/econometrics8020020 This Version is available at: https://hdl.handle.net/10419/247568 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by/4.0/
econometrics Article Triple the Gamma—A Unifying Shrinkage Prior for Variance and Variable Selection in Sparse State Space and TVP Models Annalisa Cadonna, Sylvia Frühwirth-Schnatter * and Peter Knaus Department of Finance, Accounting and Statistics, WU Vienna University of Economics and Business, 1020 Vienna, Austria; [email protected] (A.C.); peter[email protected] (P.K.) *Correspondence: sfr[email protected] Received: 9 December 2019; Accepted: 29 April 2020; Published: 20 May 2020 Abstract: Time-varying parameter (TVP) models are very flexible in capturing gradual changes in the effect of explanatory variables on the outcome variable. However, in particular when the number of explanatory variables is large, there is a known risk of overfitting and poor predictive performance, since the effect of some explanatory variables is constant over time. We propose a new prior for variance shrinkage in TVP models, called triple gamma. The triple gamma prior encompasses a number of priors that have been suggested previously, such as the Bayesian Lasso, the double gamma prior and the Horseshoe prior. We present the desirable properties of such a prior and its relationship to Bayesian Model Averaging for variance selection. The features of the triple gamma prior are then illustrated in the context of time varying parameter vector autoregressive models, both for simulated dataset and for a series of macroeconomics variables in the Euro Area. Keywords: Bayesian model averaging; horseshoe prior; lasso prior; sparsity; stochastic volatility; triple gamma prior; VAR models 1. Introduction Model selection in a high-dimensional setting is a common challenge in statistical and econometric inference. The introduction of Bayesian model averaging (BMA) techniques in the statistical literature (Brown et al. 2002;Cottet et al. 2008;Raftery et al. 1997) has led to many interesting applications, see, among others, (Frühwirth-Schnatter and Tüchler 2008;Kleijn and van Dijk 2006;Koop and Potter 2004;Sala-i-Martin et al. 2004) for early references in econometrics. Selecting explanatory variables for possibly very high-dimensional regression problems though shrinkage priors is an attractive alternative to BMA which relies on discrete mixture priors, see Bhadra et al. (2019) for an excellent review. There is a vast and growing literature on shrinkage priors for regression problems that focuses on the following aspects. First, how to choose sensible priors for high-dimensional model selection problems in a Bayesian framework, second, how to design efficient algorithms to cope with the associated computational challenges and third, to investigate, both from a theoretical and a practical viewpoint, how such priors perform in high-dimensional problems. A striking duality exists in this very active area between Bayesian and traditional approaches. For many shrinkage priors, the mode of the posterior distribution obtained in a Bayesian analysis can be regarded as a point estimate from a regularization approach, see Fahrmeir et al. (2010) and Polson and Scott (2012a) . One such example is the popular Lasso (Tibshirani 1996) which is equivalent to a double-exponential shrinkage prior in a Bayesian context (Park and Casella 2008). However, the two approaches differ when it comes to selecting penalty parameters that impact the sparsity of the Econometrics 2020,8, 20; doi:10.3390/econometrics8020020 www.mdpi.com/journal/econometrics
Econometrics 2020,8, 20 2 of 36 solution. One advantage of the Bayesian framework in this context is that the penalty parameters are considered to be unknown hyperparameters which can be learned from the data. Such “global-local” shrinkage priors (Polson and Scott 2011) adjust to the overall degree of sparsity that is required in a specific application through a global shrinkage parameter and separate signal from noise through local, individual shrinkage parameters. While the inclusion of potentially many explanatory variables though shrinkage priors in regression models is addressed in a vast literature, the use of shrinkage priors for more general econometric models in time series analysis, such as state space models and time-varying parameter (TVP) models is, in comparison, less well-studied. Sparsity in the context of such models refers to the presence of a few large variances among many (nearly) zero variances in the latent state processes that drive the observed time series data. A common goal in this setting is to recover a few dynamic states, driven by such a state space model, among many (nearly) constant coefficients. As shown by Frühwirth-Schnatter and Wagner (2010), this variance selection problem can be cast into a variable selection problem in the non-centered parametrization of a state space model. Once this link has been established, shrinkage priors that are known to perform well in high-dimensional regression problems can be applied to variance selection in state space models, as demonstrated for the Lasso (Belmonte et al. 2014) and the normal-gamma (Bitto and Frühwirth-Schnatter 2019;Griffin and Brown 2017). Despite this already existing variety, we introduce a new shrinkage prior for variance selection in sparse state space and TVP models in the present paper called triple gamma prior, as it has a representation involving three gamma distributions. This prior can be related to various shrinkage priors that were found to be useful for high-dimensional regression problems, such as the generalized beta mixture prior (Armagan et al. 2011), and contains the popular Horseshoe prior (Carvalho et al. 2009 2010) as a special case. Furthermore, the halft and the half Cauchy (Gelman 2006;Polson and Scott 2012b) , suggested as robust alternatives to the inverse gamma distribution for variance parameters in hierarchical models, as well as the Lasso and the double gamma, are special cases of the triple gamma. In this context, the triple gamma can also be regarded as an extension of the scaled beta2 distribution (Pérez et al. 2017). Among Bayesian shrinkage priors, usually a clear distinction is made between two-group mixture or spike-and-slab priors and continuous shrinkage priors, of which the triple gamma is a special case. An important contribution of the present paper is to show that the triple gamma provides a bridge between these two approaches and has the following property which is favourable both in sparse and dense situations. One of the hyperparameters allows high concentration over the region in the shrinkage profile that is relevant for shrinking noise, while the other hyperparameter allows high concentration over the region that prevents overshrinking of signals. This allows the triple gamma prior to exhibit behavior that very much resembles Bayesian model averaging based on discrete spike-and-slab priors, with a strong prior concentration at the corner solutions where some of the variances are nearly zero. While this is reminiscent of the Horseshoe prior, the shrinkage profile induced by the triple gamma is more flexible than that of a Horseshoe. Thanks to the estimation of the hyperparemters, it is not constrained to be symmetric around one half, enabling adaption to varying degrees of sparsity in the data. The triple gamma prior also scores well from a computational perspective. While exploring the full posterior distribution for spike-and-slab priors leads to computational challenges due to the combinatorial complexity of the model space, Bayesian inference based on Markov chain Monte Carlo (MCMC) methods is straightforward for continuous shrinkage priors, exploiting their Gaussian-scale mixture representation (Bitto and Frühwirth-Schnatter 2019;Makalic and Schmidt 2016). An extension of these schemes to the triple gamma prior is fairly straightforward. We will study the empirical performance of the triple gamma for a challenging setting in econometric time series analysis, namely for time-varying parameter vector autoregressive models with stochastic volatility (TVP-VAR-SV models). Since the influential paper of Primiceri (2005)
Econometrics 2020,8, 20 3 of 36 (see Del Negro and Primiceri (2015) for a corrigendum), this model has become a benchmark for analyzing relationships between macroeconomic variables that evolve over time, see Nakajima (2011), Koop and Korobilis (2013), Eisenstat et al. (2014), Chan and Eisenstat (2016), Feldkircher et al. (2017) and Carriero et al. (2019) , among many others. Due to the high dimensionality of the time-varying parameters, even for moderately sized systems, shrinkage priors such as the triple gamma prior are instrumental for efficient inference. The rest of the paper is organized as follows—in Section 2, we define the triple gamma prior and discuss some of its properties. The close relationship between the triple gamma and spike-and-slab priors applied in a BMA context is investigated in Section 3.2. Section 4introduces an efficient MCMC scheme and Section 5provides applications to TVP-VAR-SV models. Section 6concludes the paper. 2. The Triple Gamma as a Prior for Variance Parameters 2.1. Motivation and Definition To motivate the triple gamma prior, consider the state space form of a TVP model for a univariate time series yt. For t=1, . . . , T, we have that βt=βt−1+wt,wt∼ Nd(0,Q), yt=xtβt+εt,εt∼ N 0, σ2 t,(1) where Q=Diag (θ1, . . . , θd) and the initial value of the state process follows a normal distribution, β0∼ Nd(β,Q) , with initial mean β= (β1 , . . . , βd)> . xt= (xt1 , . . . , xtd) is a d -dimensional row vector containing the explanatory variables at time t . The variables xtj can be exogenous control variables and/or be equal to lagged values of yt . Usually, one of the variables, say xt1 , corresponds to the intercept, but an intercept need not be present. This approach can be straightforwardly adapted to the multivariate case as for the TVP-VAR-SV model that will be considered in Section 5. The error variance σ2 t in the observation equation is either homoscedastic ( σ2 t≡σ2 for all t= 1, . . . , T ) or follows a stochastic volatility (SV) specification (Jacquier et al. 1994), where the log volatility ht=log σ2 tfollows an AR(1) process. Specifically, ht|ht−1,µ,φ,σ2 η∼ N µ+φ(ht−1−µ),σ2 η. (2) For Bayesian inference, priors have to be chosen for the unknown variances θ1 , . . . , θd and the unknown initial means β1 , . . . , βd . In order to shrink dynamic coefficients to static ones and, in this way, avoid overfitting, a shrinkage prior is placed on θj that puts a lot of prior mass close to zero. One such prior is the double gamma prior, employed recently by Bitto and Frühwirth-Schnatter (2019). The double gamma prior can be expressed as a scale-mixture of gamma distributions, with the following hierarchical representation: θj|ξ2 j∼ G 1 2,1 2ξ2 j!,ξ2 j|aξ,κ2 B∼ G aξ,aξκ2 B 2!. (3) In the double gamma prior, each innovation variance θj is mixed over its own scale parameter ξ2 j , each of which has an independent gamma distribution, with a common hyperparameter κ2 B . Moreover, the ξ2 j ’s play the role of local (component specific) shrinkage parameters, while the parameter κ2 B is a (common) global shrinkage parameter.
Econometrics 2020,8, 20 4 of 36 We propose an extension of the double gamma prior to a triple gamma prior, where another layer is added to the hierarchy: θj|ξ2 j∼ G 1 2,1 2ξ2 j!,ξ2 j|aξ,κ2 j∼ G aξ,aξκ2 j 2!,κ2 j|cξ,κ2 B∼ G cξ,cξ κ2 B!. (4) The main difference with the double gamma prior is that the prior scale of the ξ2 j ’s is not identical, but each ξ2 j depends on its component specific scale κ2 j . We will show in Section 2.2 that the triple gamma prior can be represented as a global-local shrinkage prior in the sense of Polson and Scott (2012a) where the local shrinkage parameters ξ2 j arise from an F2aξ, 2cξ distribution. Hence, the triple gamma prior contains the Horseshoe prior and many other well-known shrinkage priors as special cases, as will be discussed in Section 2.3. The shrinkage behaviour of the triple gamma prior becomes even more apparent when we rewrite model (1) in the non-centered parametrization introduced in Frühwirth-Schnatter and Wagner (2010): ˜ βt=˜ βt−1+˜ wt,˜ wt∼ Nd(0,Id), yt=xtβ+xtDiag pθ1, . . . , pθd˜ βt+εt,εt∼ N 0, σ2 t,(5) with ˜ β0∼ Nd(0,Id) , where Id is the d -dimensional identity matrix. Both representations are equivalent and we can specify a prior either on the variances θj in (1) or the scale parameters √θj in (5). Using the fact that θj/ξ2 j∼χ2 1 and the χ2 1 -distribution can be represented as χ2 1=Z2 j , where Zj∼ N (0, 1) follows a standard normal distribution, we can match prior (4) to the non-centered parametrization (5). This yields √θj|ξ2 j∼ N 0, ξ2 j,ξ2 j|aξ,κ2 j∼ G aξ,aξκ2 j 2!,κ2 j|cξ,κ2 B∼ G cξ,cξ κ2 B!. (6) In (6), we could force √θj to take on only positive values, however, we do not impose such a constraint and allow √θjto take on negative values. Since the half-normal √θj∼ N 0, ξ2 jI{√θj> 0 } also implies that θj∼ξ2 jχ2 1 , the question arises whether the negative half is of importance. Whenever inference is performed under the non-centered parametrization (5), as is done in Section 4, restricting the prior to the positive half will lead to automatic truncation of the full conditional posterior p(√θj|˜ β0 , . . . , ˜ βT , y , ·) to the positive part during MCMC sampling. If the positive and the negative mode of the marginal posterior p(√θj|y) are well-separated, then this will not matter. However, if the true value of θj is close or equal to zero and p(√θj|y) is concentrated at zero, this truncation will introduce a bias, because the negative half is not accounted for. Interestingly, prior (6) is related to the so-called normal-gamma-gamma prior consider by Griffin and Brown (2017) in the context of defining hierarchical shrinkage priors for regression models. This relation is helpful in choosing a prior on the fixed coefficients β1 , . . . , βd . To allow shrinkage of these coefficients toward insignificant ones in a TVP model, we extend Bitto and Frühwirth-Schnatter (2019) further by assuming such a normal-gamma-gamma prior on β1, . . . , βd: βj|τ2 j∼ N 0, τ2 j,τ2 j|aτ,λ2 j∼ G aτ,aτλ2 j 2!,λ2 j|cτ,λ2 B∼ G cτ,cτ λ2 B!. (7) In Section 2.4, we will discuss hierarchical versions of both priors, by putting a hyperprior on the parameters κ2 B,λ2 B,aξ,aτ,cξ, and cτ.
Econometrics 2020,8, 20 5 of 36 2.2. Properties of the Triple Gamma Prior In this section, we study the mathematical properties of the triple gamma prior. It is shown in Theorem 1that the triple gamma prior is a global-local shrinkage prior where the local shrinkage parameters arise from the F2aξ, 2cξ distribution. Furthermore, a closed form of the marginal shrinkage prior p(√θj|φξ,aξ,cξ)is given in Theorem 1, which is proven in Appendix A. Theorem 1. For the triple gamma prior defined in (4), with aξ>0and cξ>0, the following holds: (a) It has following representation as a local-global shrinkage prior: √θj|ψ2 j,κ2 B∼ N 0, 2 κ2 B ψ2 j!,ψ2 j|aξ,cξ∼F2aξ, 2cξ. (8) (b) The marginal prior p(√θj|φξ,aξ,cξ)takes the following form with φξ=2cξ κ2 Baξ, p(√θj|φξ,aξ,cξ) = Γ(cξ+1 2) p2πφξB(aξ,cξ)Ucξ+1 2,3 2−aξ,θj 2φξ, (9) where U (a,b,z)is the confluent hyper-geometric function of the second kind: U(a,b,z)=1 Γ(a)Z∞ 0e−ztta−1(1+t)b−a−1dt. In Figure 1, we can see the marginal prior distribution of √θj under the triple gamma prior aξ=cξ= 0.1 and under other well-known shrinkage priors which are special cases of the triple gamma, see Table 1. Theorem 1also allows us to give a closed form for the prior p(θj|φξ , aξ , cξ) = p(√θj|φξ,aξ,cξ)/√θj.1 Global-local shrinkage priors are typically compared in terms of the concentration around the origin and the tail behaviour. For the triple gamma prior p(√θj|φξ , aξ , cξ) , the two shape parameters aξand cξplay a crucial role in this respect, see Theorem 2which is proven in Appendix A. Theorem 2. The triple gamma prior (9) satisfies the following: (a) For 0<aξ<0.5 and small values of √θj, p(√θj|φξ,aξ,cξ) = Γ(1 2−aξ) √π(2φξ)aξB(aξ,cξ) 1 √θj!1−2aξ +O(1). 1 Let f√θj(x) and F√θj(x) be, respectively, the pdf and cdf of the random variable √θj . The cdf Fθj(x) of the random variable θjis given by Fθj(x) = Pr(θj≤x) = Pr(−√x≤√θj≤√x) = F√θj(√x)−F√θj(−√x) = 2F√θj(√x), since f√θj(x)is symmetric around 0. The pdf fθj(x)is obtained by taking the first derivative of Fθj(x)with respect to x: fθj(x) = dFθj(x) dx=f√θj(√x)/√x.
Econometrics 2020,8, 20 6 of 36 (b) For aξ=0.5 and small values of √θj, p(√θj|φξ,aξ,cξ) = 1 p2πφξB(0.5, cξ)−log θj+log(2φξ)−ψ(cξ+0.5)+O(|θjlog θj|), where ψ(·)is the digamma function. (c) For aξ>0.5, lim √θj→0 p(√θj|φξ,aξ,cξ) = Γ(cξ+1 2)Γ(aξ−1 2) p2πφξΓ(cξ)Γ(aξ). (d) As √θj→∞, p(√θj|φξ,aξ,cξ) = Γ(cξ+1 2)(2φξ)cξ √πB(aξ,cξ) 1 √θj!2cξ+1"1+O 1 θj!#. From Theorem 2, Part (a) and (b), we find that the triple gamma prior p(√θj|φξ , aξ , cξ) has a pole at the origin, if aξ≤ 0.5. According to Part (a), the pole is more pronounced, the closer aξ gets to 0. For aξ> 0.5, we find from Part (c) that p(√θj|φξ , aξ , cξ) is bounded at zero by a positive upper bound which is finite, as long as 0 <cξ<∞ . Part (d) shows that the triple gamma prior p(√θj|φξ , aξ , cξ) has polynomial tails, with the shape parameter cξ controlling the tail index. Prior moments E((√θj)k|φξ , aξ , cξ) exist up to k< 2 cξ . Hence, the triple gamma prior has no finite moments for cξ<1/2. Finally, additional useful representations of the triple gamma prior as a global-local shrinkage prior are summarized in Lemma 1which is proven in Appendix A. Representation (a) shows that the triple gamma is an extension of the double gamma prior where the Gaussian prior √θj|ξ2 j∼ N( 0, ξ2 j) is substituted by a heavier-tailed Studentt prior, making the prior more robust to large values of √θj . Representation (b) and (c) will be useful for MCMC inference in Section 4. Representations (c) and (d) show that for a triple gamma prior with finite aξ and cξ , φξ acts as a global shrinkage parameter, in addition to 2/κ2 B. −1.0 −0.5 0.0 0.5 1.0 −4 −2 0 2 4 logp(θ) θ 6 7 8 9 10 11 −18 −16 −14 −12 −10 −8 −6 logp(θ) θ Triple gamma Horseshoe Double gamma Lasso Figure 1. Marginal prior distribution of √θj under the triple gamma prior with aξ=cξ= 0.1 with κ2 B= 2, in comparison to the Horseshoe prior with φξ= 1, the double gamma prior with aξ= 0.1 and κ2 B= 2 and the Lasso prior with κ2 B= 2. Spike (left-hand side) and tail (right-hand side) of the marginal prior.
Econometrics 2020,8, 20 7 of 36 Lemma 1. For aξ>0and cξ>0, the triple gamma prior (4) has the following alternative representations: (a) √θj|˜ ξ2 j,cξ,κ2 B∼t2cξ 0, 2 κ2 B ˜ ξ2 j!,˜ ξ2 j|aξ∼ Gaξ,aξ, (10) (b) √θj|ˇ ξ2 j,cξ,κ2 B∼t2cξ 0, 2 aξκ2 B ˇ ξ2 j!,ˇ ξ2 j|aξ∼ Gaξ, 1. (11) Additional representations for 0<aξ<∞and 0<cξ<∞based on φξ=2cξ κ2 Baξare (c) √θj|ˇ ξ2 j,ˇ κ2 j,φξ∼ N 0, φξˇ ξ2 j/ˇ κ2 j,ˇ ξ2 j|aξ∼ Gaξ, 1,ˇ κ2 j|cξ∼ Gcξ, 1, (12) (d) √θj|˜ ψ2 j,φξ∼ N 0, φξ˜ ψ2 j,˜ ψ2 j|aξ,cξ∼ BP aξ,cξ, (13) where BP aξ,cξis the beta-prime distribution.2 2.3. Relation of the Triple Gamma to Other Shrinkage Priors The triple gamma prior can be related to the very active research on shrinkage priors in a Bayesian framework in various ways. On the one hand, popular priors for variance parameters introduced as robust alternatives to the inverse gamma prior are special cases of the triple gamma, see Table 1. For instance, in (8), ψ2 j converges a.s. to 1, as aξ→∞ and cξ→∞ , and the triple gamma reduces to a normal distribution for √θj , applied for univariate TVP models (Frühwirth-Schnatter 2004) and unobserved component state space model (Frühwirth-Schnatter and Wagner 2010). For cξ→∞ , F2aξ, 2cξ converges to the Gaξ,aξ distribution and the triple gamma reduces to the Bayesian Lasso for aξ= 1 (Belmonte et al. 2014) and otherwise to the double gamma (Bitto and Frühwirth-Schnatter 2019) applied in sparse TVP models. Gelman (2006) introduced the halft and the half-Cauchy prior for variance parameters in hierarchical models, by assuming that √θj follows a “folded” t -distribution, that is, a t -distribution truncated to [ 0, ∞) , see also Polson and Scott (2012b). In (10), ˜ ξ2 j converges a.s. to 1 as aξ→∞ and the triple gamma reduces to a t2cξ - distribution and to the Cauchy distribution for cξ= 1 / 2, however without being “folded”, since we allow √θjto take on negative values, as explained in Section 2.1. 2Note that the X∼ BP (a,b)-distribution has pdf f(x) = 1 B(a,b) xa−1 (1+x)a+b. Furthermore, Y=X/(1+X)follows the B(a,b)-distribution.
Econometrics 2020,8, 20 8 of 36 Table 1. Priors on √θj which are equivalent to (top) or special cases of (bottom) the triple gamma prior. Prior for √θjaξcξκ2 Bφξ N0, ψ2 j,ψ2 j∼GG aξ,cξ,φξnormal-gamma-gamma aξcξ2cξ φξaξφξ N0, 1 κj−1,κj∼ T PBaξ,cξ,φξgeneralized beta mixture aξcξ2cξ φξaξφξ N0, ψ2 j,ψ2 j∼SBeta2 aξ,cξ,φξhierarchical scaled beta2 aξcξ2cξ φξaξφξ DE 0, √2ψj,ψ2 j∼ Gcξ,1 λ2normal-exponential-gamma 1 cξ2λ2cξ1 λ2 N0, τ2ψ2 j,ψj∼t1Horseshoe 1 21 22 τ2τ2 N0, 1 κj−1,κj∼ B(1/2, 1)Strawderman-Berger 1 21 4 1 N0, τ2˜ ξj,˜ ξj∼ Gaξ,aξdouble gamma aξ∞2 τ2N0, τ2˜ ξj,˜ ξj∼ E (1)Lasso 1 ∞2 τ2tν0, τ2half-t∞ν 22 τ2t10, τ2half-Cauchy ∞1 22 τ2N(0, B0)normal ∞ ∞ 2 B0On the other hand, the triple gamma prior is related to popular shrinkage priors in regression models. It extends the generalized beta mixture prior introduced by Armagan et al. (2011) for variable selection in regression models, βj|ξ2 j∼ N 0, ξ2 j,ξ2 j∼ Gaξ,λj,λj∼ Gcξ,φξ, to variance selection in state space and TVP models. This is evident from rewriting (4) as ξ2 j∼ Gaξ,λj , λj∼ Gcξ,φξ . We exploit this relationship in Section 3.1 to investigate the shrinkage profile of a triple gamma prior. Using Armagan et al. (2011, Definition 2), the triple gamma prior can be written as √θj|ρj∼ N 0, 1/ρj−1,ρj|aξ,cξ,φξ∼ T PBaξ,cξ,φξ, (14) where T PBaξ,cξ,φξis the three-parameter beta distribution with density: p(ρj) = 1 B(aξ,cξ)(φξ)cξρcξ−1 j(1−ρj)aξ−11+ (φξ−1)ρj−(aξ+cξ). (15) From (14) and (15), it becomes evident that the Strawderman-Berger prior √θj|ρj∼ N0, 1/ρj−1 , ρj∼ B(1/2, 1) (Berger 1980;Strawderman 1971) is that special case of the triple gamma prior where φξ=1, aξ=1/2, and cξ=1. The special case of a triple gamma, where aξ=cξ= 1 / 2, corresponds to a Horseshoe prior (Carvalho et al. 2009 2010) on √θj with global shrinkage parameter τ2= 2 /κ2 B , since ψ2 j∼F(1, 1) implies that ψj∼t1 . The Horseshoe prior has been introduced for variable selection in regression models and has been shown to have excellent theoretical properties in this context for the “nearly black” case (van der Pas et al. 2014). The triple gamma is a generalization of the Horseshoe prior, with a similar shrinkage profile, however with much more mass close to the corner solutions. Most importantly, as will be discussed in Section 3.1, this leads to a BMA-type behaviour of the triple gamma prior for small values of aξand cξ.
Econometrics 2020,8, 20 15 of 36 where φτ= 2 cτ/(λ2 Baτ) and φξ= 2 cξ/(κ2 Baξ) . All conditional distributions in our MCMC scheme are available in closed form, except for the ones for aξ , cξ , aτ and cτ , for which we will resort to a Metropolis-Hastings (MH) step within Gibbs. Several conditional distributions are the same as for the double gamma prior and we apply Algorithm 1 of Bitto and Frühwirth-Schnatter (2019). We provide more details on the derivation of the various densities in Appendix B. Algorithm 1. MCMC inference for TVP models under the triple gamma prior. Choose starting values for all global shrinkage parameters (aτ , cτ , λ2 B , aξ , cξ , κ2 B) and local shrinkage parameters {ˇ τ2 j,ˇ λ2 j,ˇ ξ2 j,ˇ κ2 j}d j=1, and repeat the following steps: (a) Define for j= 1, . . . , d , τ2 j=φτˇ τ2 j/ˇ λ2 j and ξ2 j=φξˇ ξ2 j/ˇ κ2 j and sample from the posterior p(˜ β0 , . . . , ˜ βT , β1 , . . . , βd , √θ1 , . . . , √θd|{ξ2 j , τ2 j}d j=1 , y) using Algorithm 1, Steps (a), (b), and (c) in Bitto and Frühwirth-Schnatter (2019). In the homoscedastic case, use Step (f) of this algorithm to sample from σ2|z−σ2 , y . For the SV model (2), sample the parameters µ , φ , and σ2 η as in Kastner and Frühwirth-Schnatter (2014), for example, using the R-package stochvol (Kastner 2016). (b) Use the prior p(√θj|ˇ κ2 j , aξ , cξ) , marginalized w.r.t. ˇ ξ2 j , to sample aξ from p(aξ|z−aξ , y) via a random walk MH step on z=log(aξ/( 0.5 −aξ)) . Propose aξ,(∗)= 0.5 ez∗/( 1 +ez∗) , where z∗∼ N z(m−1),v2 and z(m−1)=log(aξ,(m−1)/( 0.5 −aξ,(m−1))) depends on the previous value aξ,(m−1) of aξ , accept aξ,(∗) with probability min (1, qa(aξ,(∗)) qa(aξ,(m−1))),qa(aξ) = p(aξ|z−aξ,y)aξ(0.5 −aξ), and update φξ= 2 cξ/(κ2 Baξ) . Explicit forms for p(aξ|z−aξ , y) and log qa(aξ) are provided in (A3) and (A4). Similarly, use the prior p(βj|ˇ λ2 j , aτ , cτ) , marginalized w.r.t. to ˇ τ2 j , to sample aτ via a random walk MH step and update φτ=2cτ/(aτλ2 B). (c) Sample ˇ ξ2 j, j =1, . . . , d, from a generalized inverse Gaussian distribution, see (A5): ˇ ξ2 j|ˇ κ2 j,θj,aξ,φξ∼ GIG aξ−1 2, 2, ˇ κ2 jθj φξ!. (25) Similarly, update ˇ τ2 j, j =1, . . . , d, conditional on aτ: ˇ τ2 j|βj,ˇ λ2 j,aτ,φτ∼ GIG aτ−1 2, 2, ˇ λ2 jβ2 j φτ!. (d) Use the marginal Studentt distribution p(√θj|ˇ ξ2 j , cξ , κ2 B) given in (11) to sample cξ from p(cξ|z−cξ , y) via a random walk MH step on z=log(cξ/(0.5 −cξ)) . Propose cξ,(∗)= 0.5 ez∗/( 1 +ez∗) , where z∗∼ N z(m−1),v2 and z(m−1)=log(cξ,(m−1)/( 0.5 −cξ,(m−1))) depends on the previous value cξ,(m−1)of cξ, accept cξ,(∗)with probability min (1, qc(cξ,(∗)) qc(cξ,(m−1))),qc(cξ) = p(cξ|z−cξ,y)cξ(0.5 −cξ), and update φξ= 2 cξ/(κ2 Baξ) . Explicit forms for p(cξ|z−cξ , y) and log qc(cξ) are provided in (A6) and (A7). Similarly, to sample cτ via a random walk MH step use the marginal distribution of βj|ˇ τ2 j , aτ , cτ with respect to ˇ λ2 jand update φτ=2cτ/(aτλ2 B).
Econometrics 2020,8, 20 16 of 36 (e) Sample ˇ κ2 j, for j =1, . . . , d, from following gamma distribution, see (A8): ˇ κ2 j|θj,ˇ ξ2 j,cξ,φξ∼ G 1 2+cξ,θj 2φξˇ ξ2 j +1!. (26) Similarly, update ˇ λ2 j, j =1, . . . , d, conditional on cτ: ˇ λ2 j|βj,ˇ τ2 j,cτ,φτ∼ G 1 2+cτ,β2 j 2φτˇ τ2 j +1!. (f) Sample d2 from d2|aξ , cξ , κ2 B∼ Gaξ+cξ,κ2 B+2cξ aξ , see (A9); sample from κ2 B from following gamma distribution, κ2 B|{θj,ˇ κ2 j,ˇ ξ2 j}d j=1,aξ,cξ,d2∼ G d 2+aξ,aξ 4cξ d ∑ j=1 ˇ κ2 j ˇ ξ2 j θj+d2!, (27) see (A10), and update φξ=2cξ/(κ2 Baξ). Similarly, sample e2from e2|aτ,cτ,λ2 B∼ Gaτ+cτ,λ2 B+2cτ aτ, sample λ2 Bfrom λ2 B|{βj,ˇ λ2 j,ˇ τ2 j}d j=1,aτ,cτ,e2∼ G d 2+aτ,aτ 4cτ d ∑ j=1 ˇ λ2 j ˇ τ2 j β2 j+e2!, and update φτ=2cτ/(aτλ2 B). The MCMC scheme in Algorithm 1is not a full conditional scheme, as several steps are based on partially marginalized distributions. That means that the sampling order matters. For instance, in Step (b), we marginalize w.r.t. ˇ ξ2 1 , . . . , ˇ ξ2 d , hence we need to update ˇ ξ2 1 , . . . , ˇ ξ2 d after sampling aξ , before we update cξ in Step (d) conditional on ˇ ξ2 1 , . . . , ˇ ξ2 d . Similarly, due to marginalization in Step (d), we need to update ˇ κ2 1 , . . . , ˇ κ2 d , before we update d2 in Step (f). Furthermore, both Step (b) and Step (d) are based on the marginal prior of κ2 B , given in (17). Hence, in Step (f), d2 has to be updated from d2|aξ,cξ,κ2 B, before κ2 Bis updated conditional on d2. For a symmetric triple gamma prior, where aξ=cξ , the MCMC scheme in Algorithm 1has to be modified only slightly. Either qa(aξ) in Step (b) is adjusted and Step (d) is skipped, setting cξ=aξ , or qc(cξ) in Step (d) is adjusted and Step (b) is skipped, setting aξ=cξ . In Appendix B, we provide details in (A11) for the first case and in (A12) for the second case. Similar modifications are needed, if aτ=cτ. All other steps in Algorithm 1remain the same for aξ=cξand/or aτ=cτ. 5. Applications to TVP-VAR-SV Models 5.1. Model In this section, we consider a generalization of the TVP model (1), where yt is a m -dimensional time series, observed for t= 1, . . . , T . The time series yt is assumed to follow a time-varying parameter vector autoregressive model with stochastic volatility (TVP-VAR-SV) of order p: yt=ct+Φ1,tyt−1+Φ2,tyt−2+. . . Φp,tyt−p+εt,εt∼ Nm(0,Σt), (28)
Econometrics 2020,8, 20 17 of 36 where ct is the m -dimensional time-varying intercept, Φj,t , for j= 1, . . . , p is an m×m matrix of time-varying coefficients, and Σt is the time-varying variance covariance matrix of the error term. The TVP-VAR-SV model can be written in a more compact notation as the following TVP model: yt= (Im⊗xt)βt+εt,εt∼ Nm(0,Σt), (29) where xt= (y0 t−1 , . . . , y0 t−p , 1 ) is a row vector of length mp + 1 and the time-varying parameter βt is defined as βt= (β1 t0 , . . . , βm t0)0 , where βi t= (Φ1,ti• , . . . , Φp,ti• , ct,i)0 . Here, Φj,ti• denotes the i -th row of the matrix Φj,t and ct,i denotes the i -th element of ct . Since the influential paper of Primiceri (2005) (see Del Negro and Primiceri (2015) for a corrigendum), this model has become a benchmark for analyzing relationships between macroeconomic variables that evolve over time, see Nakajima (2011), Koop and Korobilis (2013) ,Eisenstat et al. (2014), Chan and Eisenstat (2016), Feldkircher et al. (2017) and Carriero et al. (2019), among many others. Following Frühwirth-Schnatter and Tüchler (2008), we use a Cholesky decomposition of the time-varying covariance matrix Σt , that is Σt=AtDtA0 t , where Dt is a diagonal matrix and At is lower unitriangular matrix, see Carriero et al. (2019) and Bitto and Frühwirth-Schnatter (2019) for related models. We denote with aij,t the element at the i -th row and j -th column of At , and with σ2 i,t the i -th diagonal element of Dt=Diag σ2 1,t···σ2 m,t . In total, we have m(m− 1 )/ 2 +m(mp + 1 ) (potentially) time-varying parameters. Using the Cholesky decomposition, we can rewrite the system as: yt= (Im⊗xt)βt+Atηt,ηt∼ Nm(0,Dt), (30) where ηt= (η1,t , . . . , ηm,t)> . The idiosyncratic shocks ηi,t∼ N 0, σ2 i,t follow independent SV processes as in (2), with row specific parameters. Specifically, with hi,t=log σ2 i,t , we have that the logarithm of the elements of the diagonal matrix Dtfollow independent AR(1) processes: hi,t=µi+φi(hi,t−1−µi) + νi,t,νi,t∼ N 0, σ2 η,i, for i= 1, . . . , m . Here, µi is the mean, φi is the persistence parameter, and σ2 η,i is the variance of the i th log-volatility hi,t. It is possible to write the TVP-VAR-SV model (30) as a system of m univariate TVP models as in (1): y1,t=xtβ1 t+η1,t,η1,t∼ N 0, σ2 1,t, y2,t=xtβ2 t+a21,tη1,t+η2,t,η2,t∼ N 0, σ2 2,t, y3,t=xtβ3 t+a31,tη1,t+a32,tη2,t+η3,t,η3,t∼ N 0, σ2 3,t, ··· ym,t=xtβm t+am1,tη1,t+. . . +am,m−1,tηm−1,t+ηm,t,ηm,t∼ N 0, σ2 m,t. Note that for i> 1, the i -th equation of this system is a TVP model where the residuals of the preceding i−1 equations are added as explanatory variables: yi,t=xtβi t+ i−1 ∑ j=1 aij,tηj,t+ηi,t,ηi,t∼ N 0, σ2 i,t,
Econometrics 2020,8, 20 18 of 36 and all time-varying parameters follow a random walk as in the TVP model (1): βi j,t=βi j,t−1+vij,t,vij,t∼ N 0, θβ ij, for i=1, . . . , m, and j=1, . . . , mp +1, aij,t=aij,t−1+wij,t,wij,t∼ N 0, θa ij, for i=1, . . . , m, and j=1, . . . , i−1, with initial values βi j,0 ∼ N ββ ij,θβ ij and aij,0 ∼ N βa ij,θa ij . Here, βi j,t denotes the j th element of the vector βi t. To achieve shrinkage for each VAR coefficient βi j,t as well as for each Cholesky factor aij,t , we proceed as in Section 2and introduce shrinkage priors for the initial expectations ββ ij and βa ij as well as the variances θβ ij and θa ij . We do this independently for each equation of the system. Within each equation, the ββ ij s and βa ij s are assumed to follow independent shrinkage priors to allow for flexibility in the prior structure, and similarly for θβ ij and θa ij: βx ij ∼ N 0, φτ,x iˇ τx,2 ij /ˇ λx,2 ij ,ˇ τx,2 ij ∼ Gaτ,x i, 1,ˇ λx,2 ij ∼ Gcτ,x i, 1,φτ,x i=2cτ,x i/(λx,2 B,iaτ,x i),(31) √θx ij ∼ N 0, φξ,x iˇ ξx,2 ij /ˇ κx,2 ij ,ˇ ξx,2 ij ∼ Gaξ,x i, 1,ˇ κx,2 ij ∼ Gcξ,x i, 1,φξ,x i=2cξ,x i/(κx,2 B,iaξ,x i), where x=β for the VAR-coefficients and x=a for the elements of At . Following Section 2.4, the priors for the global shrinkage parameters in the ith equation read λx,2 B,i|aτ,x i,cτ,x i∼F2aτ,x i, 2cτ,x i, 2aτ,x i∼ B(αaτ,βaτ), 2cτ,x i∼ B(αcτ,βcτ), (32) κx,2 B,i|aξ,x i, 2cξ,x i∼F2aξ,x i, 2cξ,x i, 2aξ,x i∼ B(αaξ,βaξ), 2cξ,x i∼ B(αcξ,βcξ). 5.2. A Brief Sketch of the TVP-VAR-SV MCMC Algorithm Our algorithm exploits the aforementioned unitriangular decomposition to estimate the model parameters equation-by-equation. Due to the prior structure introduced in (31), the estimation of βi t and the aij,t ’s is separated into two blocks, with the algorithm cycling through the m equations, alternating between sampling βi t conditional on Σt and sampling the aij,t s and di,t s conditional on the VAR coefficients βi t. Given a set of initial values, the algorithm repeats the following steps: Algorithm 2. MCMC inference for TVP-VAR-SV models under the triple gamma prior. Choose starting values for all global and local shrinkage parameters in prior (31) for each equation and repeat the following steps: For i =1, . . . , m, update all the unknowns in the ith equation: (a) Conditional on Atand Dt, create ˇ yi,t=yi,t−∑i−1 j=1aij,tηj,tand define the following TVP model: ˇ yi,t=xtβi t+ηi,t,ηi,t∼ N 0, σ2 i,t. Apply Algorithm 1(sans the step for the variance of the observation equation) to this univariate TVP model, to draw from the conditional posterior distribution of the time-varying VAR-coeffcients βi t ,for t= 0, . . . , T ,their initial expectations ββ ij , the process variances θβ ij , the local shrinkage parameters ˇ τβ,2 ij , ˇ λβ,2 ij , ˇ ξβ,2 ij , ˇ κβ,2 ij , as well as the global shrinkage parameters λβ,2 B,i , κβ,2 B,i , aτ,β i , cτ,β i , aξ,β i , and cξ,β i. (b) For i >1, create y? i,t=yi,t−xtβi t, conditional on βi t, and define the following TVP model: y? i,t= i−1 ∑ j=1 aij,tηj,t+ηi,t,ηi,t∼ N 0, σ2 i,t,
Econometrics 2020,8, 20 19 of 36 where the residuals from the previous i− 1equations, (η1,t , . . . , ηi−1,t) , are used as explanatory variables and no intercept is present. Apply Algorithm 1to this univariate TVP model, to sample the volatilities σ2 i,t and the time-varying coefficients aij,t in the i th row of At for t= 0, . . . , T from the respective conditional posteriors, as well as the initial expectations βa ij , the process variances θa ij , the local shrinkage parameters ˇ τa,2 ij , ˇ λa,2 ij , ˇ ξa,2 ij , ˇ κa,2 ij and the global shrinkage parameters λa,2 B,i,κa,2 B,i,aτ,a i,cτ,a i,aξ,a i, and cξ,a i. In the following applications, we run our algorithm for M= 200, 000 iterations, discarding the first 100, 000 iterations as burn-in, and then keeping the output of one every 100 iterations. 5.3. Illustrative Example with Simulated Data To illustrate the merit of our methodology in the context of TVP-VAR-SV models, we simulate data from two TVP-VAR-SV models with T= 200 points in time, p= 1 lags and m= 7 equations, with varying degrees of sparsity. In the dense regime, approximately 30% of the values of β and θ (here referring to the means of the initial states and the variances of the innovations as defined in Section 2, respectively) are truly zero, while in the sparse regime approximately 90% are truly zero. We show results for the triple gamma prior, the Horseshoe prior, the double gamma and the Lasso. Regarding the priors on the hyperparameters, we use prior (32) with αaτ=αcτ=αaξ=αcξ= 1 and βaτ=βcτ=βaξ=βcξ= 6 for the triple gamma. The probability density function of the corresponding beta prior is monotonically increasing, with a maximum at 0.5. This prior places positive mass in a neighborhood of the Horseshoe, but allows for more flexibility. In practice, placing a prior on the spike and slab parameters of the triple gamma, instead of fixing them to 0.5 as in the Horseshoe, allows us to learn the shrinkage profile from the data, including asymmetric profiles. We assume that the global shrinkage parameters λβ,2 B,i , κβ,2 B,i , λa,2 B,i , and κa,2 B,i follow a F(1, 1) distribution for the Horseshoe prior which corresponds to the prior in Carvalho et al. (2009) and a G(0.001, 0.001) distribution for the Lasso and the double gamma prior, as suggested in Belmonte et al. (2014) and Bitto and Frühwirth-Schnatter (2019). Concerning the spike parameters aτ,a i , aξ,a i , aτ,β i , and aξ,β i of the double gamma, we employ a rescaled beta prior to force them to be smaller than 0.5. Specifically, we use a B( 4, 6 ) prior which places most of its mass between 0.05 and 0.4, a range that Bitto and Frühwirth-Schnatter (2019) have found to induce desirable shrinkage characteristics. 0 50 100 150 200 −0.015 −0.005 0.005 0.015 Triple Gamma 0 50 100 150 200 Horseshoe 0 50 100 150 200 Double Gamma 0 50 100 150 200 Lasso Figure 5. Posterior path against time for a constant non-significant parameter βi j,t in the sparse regime. Figure 5shows the posterior path of a permanently non-significant state, that is a state where the true βi j,t= 0 for t= 1, . . . , T , in the sparse regime. The entire set of states for the triple gamma prior
Econometrics 2020,8, 20 20 of 36 can be found in Appendix C. Note that, while the zero line is contained in the 95% posterior credible interval for all priors, said interval is thinner under the triple gamma prior and the double gamma prior than under the Lasso and the Horseshoe prior. We calculate the posterior inclusion probabilities based on the thresholding approach introduced in Section 3.2, comparing the triple gamma prior to widely used special cases. In a variance selection context, the posterior inclusion probabilities reflect the uncertainty on whether a state should be time varying or constant over time. Figure 6shows the posterior inclusion probabilities for the variance of the innovations ( θβ ij ’s) under four different shrinkage priors, for the sparse and the dense scenario, respectively. The cells are shaded in gray when the corresponding true state parameter is time-varying ( θβ ij 6= 0), while the background is white when the corresponding true state parameter is not time-varying ( θβ ij = 0). In this simulated example, the posterior inclusion probabilities under the triple gamma prior are consistently higher for the variances that are actually different from 0, even when they are very small. This outcome is in line with the analytical results derived in Section 2.2, which show that the tails of the triple gamma prior are heavier than those of the other priors. 0.0 0.4 0.8 V1 θ = 1.688 V2 θ = 0.225 0.0 0.4 0.8 0.0 0.4 0.8 V3 θ = 0.045 θ = 3.45 V4 0.0 0.4 0.8 0.0 0.4 0.8 V5V6 θ = 0.015 0.0 0.4 0.8 TG HS DG LS 0.0 0.4 0.8 V7 TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS 0.0 0.2 0.4 V1 θ = 0.026 θ = 0.003 θ = 0.025 θ = 0.013 θ = 0 V2 θ = 0θ = 0.155 θ = 0.063 θ = 0.072 0.0 0.2 0.4 0.0 0.2 0.4 V3 θ = 0.001 θ = 0.006 θ = 0.065 θ = 0.017 θ = 0.003 V4 θ = 0.019 θ = 0.344 θ = 0.003 θ = 0 0.0 0.2 0.4 0.0 0.2 0.4 V5 θ = 0.015 θ = 0.001 θ = 0.026 θ = 0.011 V6 θ = 0.089 θ = 0.085 θ = 0.004 θ = 0.084 0.0 0.2 0.4 θ = 0.018 TG HS DG LS 0.0 0.2 0.4 V7 θ = 0.049 TG HS DG LS θ = 0.066 TG HS DG LS θ = 0.118 TG HS DG LS θ = 0.145 TG HS DG LS θ = 0.122 TG HS DG LS TG HS DG LS θ = 0.043 Figure 6. Posterior inclusion probability for the θβ ij ’s in the sparse and dense regime, under the triple gamma prior, the Horseshoe prior, the Lasso prior and the double gamma prior. The true values of the θβ ij’s are reported in each cell. 5.4. Modeling Area Macroeconomic and Financial Variables in the Euro Area Our application investigates a subset of the area wide model of the European Union of Fagan et al. (2005), which comprises quarterly macroeconomic data spanning from 1970 to 2017. We include seven of the variables present in the dataset, namely real output (YER), prices (YED), short-term interest rate (STN), investment (ITR), consumption (PCR), exchange rate (EEN) and unemployment (URX). A more detailed description of the data and the transformations performed to make the time series stationary can be found in Table A1 in Appendix D. To stay in line with the literature, for example, Feldkircher et al. (2017), we estimate a TVP-VAR-SV model with p= 2 lags on all endogenous variables. The hyperparameter choices are the same as in Section 5.3. As in the example with simulated data, we run the algorithm for M= 200, 000 iterations, discarding the first 100, 000 iterations as burn-in, and then keeping the output of one every 100 iterations. Figures 7and 8display the posterior inclusion probabilities for the means of the initial states and the innovation variances of the VAR coefficients, respectively. A few things about Figure 7are noteworthy. First, the posterior inclusion probabilities on the diagonal, meaning those belonging to the parameter of each equation’s own autoregressive term, appear to be those that are the highest, while off diagonal elements are more likely to be excluded. Second, the equation for the short-term
Econometrics 2020,8, 20 21 of 36 interest rate is characterized by a large amount of parameters with a high inclusion probability, across all priors. Third, the first lag tends to have higher posterior inclusion probabilities than the second lag, which is in line with the literature. In most cases, the triple gamma prior can be seen to have either the largest or the smallest posterior inclusion probability compared to the other priors. This can be seen as a reflection of the fact that the triple gamma prior places more mass on the edges of the shrinkage profile, as illustrated in Section 3. Now, we shift our focus to the posterior inclusion probabilities for the θβ ij ’s plotted in Figure 8. Compared to the means of the inital states, almost all inclusion probabilities are essentially zero. This lack of variability is unsurprising, as it is well known (see, e.g., Feldkircher et al. (2017)) that stochastic volatility in a TVP-VAR model for macroeconomic variables can explain a large part of the variability in the data. However, the triple gamma prior appears to allow posterior distributions that place slightly more mass on models with some time variation, in particular with respect to the financial variables. Figures 9and 10 display the posterior median of ββ ij and √θβ ij , respectively. Here the triple gamma can be seen to be quite conservative, both in terms of which parameters to include, as well as their magnitude. In particular the medians of the √θβ ij are interesting, as they are closest to zero under the triple gamma prior, despite having the highest posterior inclusion probabilities among all considered priors. In Figures A3 and A4 in Appendix D, all the posterior paths of Φ1,t and Φ2,t under the triple gamma prior are shown. 0.0 0.4 0.8 YER YER YED STN ITR PCR EEN URX YED 0.0 0.4 0.8 0.0 0.4 0.8 STNITR 0.0 0.4 0.8 0.0 0.4 0.8 PCREEN 0.0 0.4 0.8 TG HS DG LS 0.0 0.4 0.8 URX TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS 0.0 0.4 0.8 YER YER YED STN ITR PCR EEN URX YED 0.0 0.4 0.8 0.0 0.4 0.8 STNITR 0.0 0.4 0.8 0.0 0.4 0.8 PCREEN 0.0 0.4 0.8 TG HS DG LS 0.0 0.4 0.8 URX TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS Figure 7. Posterior inclusion probability for state parameters ββ ij associated with the first lag (on the left ) and with the second lag (on the right ), for the Euro Area data under the triple gamma prior, the Horseshoe prior, the double gamma prior and the Lasso prior.
Econometrics 2020,8, 20 22 of 36 0.00 0.04 0.08 YER YER YED STN ITR PCR EEN URX YED 0.00 0.04 0.08 0.00 0.04 0.08 STNITR 0.00 0.04 0.08 0.00 0.04 0.08 PCREEN 0.00 0.04 0.08 TG HS DG LS 0.00 0.04 0.08 URX TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS 0.00 0.04 0.08 YER YER YED STN ITR PCR EEN URX YED 0.00 0.04 0.08 0.00 0.04 0.08 STNITR 0.00 0.04 0.08 0.00 0.04 0.08 PCREEN 0.00 0.04 0.08 TG HS DG LS 0.00 0.04 0.08 URX TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS TG HS DG LS Figure 8. Posterior inclusion probability for θβ ij ’s associated with the first lag on the left and with the second lag on the right , for the Euro Area data under the triple gamma prior, the Horseshoe prior, the double gamma prior and the Lasso prior. Triple gamma URX EEN PCR ITR STN YED YER Int YER YED STN ITR PCR EEN URX YER YED STN ITR PCR EEN URX −0.8 −0.6 −0.4 −0.2 0.0 0.2 0.4 0.6 0.8 Horseshoe URX EEN PCR ITR STN YED YER Int YER YED STN ITR PCR EEN URX YER YED STN ITR PCR EEN URX −0.8 −0.6 −0.4 −0.2 0.0 0.2 0.4 0.6 0.8 Double gamma URX EEN PCR ITR STN YED YER Int YER YED STN ITR PCR EEN URX YER YED STN ITR PCR EEN URX −0.8 −0.6 −0.4 −0.2 0.0 0.2 0.4 0.6 0.8 Lasso URX EEN PCR ITR STN YED YER Int YER YED STN ITR PCR EEN URX YER YED STN ITR PCR EEN URX −0.8 −0.6 −0.4 −0.2 0.0 0.2 0.4 0.6 0.8 Figure 9. Posterior median of ββ ij under the triple gamma, Horseshoe, double gamma and Lasso for the Euro area model. The vertical lines delimit the intercept, first and second lag, respectively.
Econometrics 2020,8, 20 23 of 36 Triple gamma URX EEN PCR ITR STN YED YER Int YER YED STN ITR PCR EEN URX YER YED STN ITR PCR EEN URX 0.000 0.005 0.010 0.015 0.020 0.025 0.030 0.035 0.040 Horseshoe URX EEN PCR ITR STN YED YER Int YER YED STN ITR PCR EEN URX YER YED STN ITR PCR EEN URX 0.000 0.005 0.010 0.015 0.020 0.025 0.030 0.035 0.040 Double gamma URX EEN PCR ITR STN YED YER Int YER YED STN ITR PCR EEN URX YER YED STN ITR PCR EEN URX 0.000 0.005 0.010 0.015 0.020 0.025 0.030 0.035 0.040 Lasso URX EEN PCR ITR STN YED YER Int YER YED STN ITR PCR EEN URX YER YED STN ITR PCR EEN URX 0.000 0.005 0.010 0.015 0.020 0.025 0.030 0.035 0.040 Figure 10. Posterior median of √θβ ij under the triple gamma, Horseshoe, double gamma and Lasso for the Euro area model. The vertical lines delimit the intercept, first and second lag, respectively. 6. Conclusions In the present paper, shrinkage for time-varying parameter (TVP) models was investigated within a Bayesian framework with the goal to automatically reduce time-varying parameters to static ones, if the model is overfitting. This goal was achieved by suggesting the triple gamma prior as a new shrinkage priors for the process variances of varying coefficients, extending previous work using spike-and-slab priors, the Bayesian Lasso, or the double gamma prior. The triple gamma prior is related to the normal-gamma-gamma prior applied for variable selection in highly structured regression models (Griffin and Brown 2017). It contains the well-known Horseshoe prior as a special case, however it is more flexible, with two shape parameters that control concentration at zero and the tail behaviour. This leads to a BMA-type behaviour which allows not only variance shrinkage, but also variance selection. In our application, we considered time-varying parameter VAR models with stochastic volatility. Overall, our findings suggest that the family of triple gamma priors introduced in this paper for sparse TVP models is successful in avoiding overfitting, if coefficients are, indeed, static or even insignificant. The framework developed in this paper is very general and holds the promise to be useful for introducing sparsity in other TVP and state space models in many different settings. A number of extensions seem to be worth pursuing. First of all, the triple gamma prior is relevant not only for TVP models, but for any model containing variance parameters such as random-effect models or Bayesian p-splines models (Scheipl and Kneib 2009). Second, in particular, in ultra-sparse settings, modifications of the triple gamma prior seem sensible. Currently, the hyperprior for the global shrinkage parameter of the triple gamma prior is selected in a way that it implies a uniform prior on “model size”. A generalization of Theorem 3would allow the choice of hyper priors that induce higher sparsity. Furthermore, in the variable selection literature, special priors such as the Horseshoe+ (Bhadra et al. 2017a) were suggested for very sparse, ultra-sparse high dimensional settings. Exploiting
Econometrics 2020,8, 20 24 of 36 once more the non-centered parametrization of a state space model, it is straightforward to extend this prior to variance selection using following hierarchical representation: √θj|κ2 j,ξ2 j∼ N 0, 2 κ2 B κ2 jξ2 j!,κj∼t1,ξj∼t1. We leave these extensions for future research. Finally, an important limitation of our approach is that shrinking a variance toward zero implies that a coefficient is fixed over the entire observation period of the time series. In future research we will investigate dynamic shrinkage priors (Kalli and Griffin 2014;Kowal et al. 2019;Roˇcková and McAlinn 2020) where coefficients can be both fixed and dynamic. Author Contributions: The authors contributed equally to the work. All authors have read and agreed to the published version of the manuscript. Conflicts of Interest: The authors declare that there is no conflict of interest. Appendix A. Proofs Proof of Theorem 1.To proof Part (a), rewrite prior (6) in the following way by rescaling ξ2 jand κ2 j: √θj|˜ ξ2 j,˜ κ2 j,κ2 B∼ N 0, 2 κ2 B ˜ ξ2 j ˜ κ2 j!,˜ ξ2 j|aξ∼ Gaξ,aξ,˜ κ2 j|cξ∼ Gcξ,cξ, (A1) and use the fact that in (A1) the random variable ψ2 j=˜ ξ2 j/˜ κ2 jfollows the F-distribution: ψ2 j= ˜ ξ2 j ˜ κ2 j∼Gaξ,aξ Gcξ,cξ=dF2aξ, 2cξ, where p(ψ2 j)is given by: p(ψ2 j) = 1 B(aξ,cξ)aξ cξψ2 jaξ−11+aξ cξψ2 j−(aξ+cξ) . (A2) This yields (8). Using that ηj=1/ψ2 j∼F2cξ, 2aξ, we obtain from (8) that p(√θj|κ2 B,aξ,cξ) = qκ2 B(cξ)cξ √4π(aξ)cξB(aξ,cξ)Z∞ 0exp −θjκ2 Bηj 4!ηcξ−1 2 j 1+cξηj aξ!−(aξ+cξ) dηj. A change of variable with yj=cξηj/aξproves Part (b): p(√θj|φξ,aξ,cξ) = 1 p2πφξB(aξ,cξ)Z∞ 0exp −θj 2φξyjycξ−1 2 j1+yj−(aξ+cξ)dyj =Γ(cξ+1 2) p2πφξB(aξ,cξ)Ucξ+1 2,3 2−aξ,θj 2φξ, where φξ=2cξ κ2 Baξ.
Econometrics 2020,8, 20 31 of 36 Appendix D. Application Appendix D.1. Data Overview Table A1. Data overview. Variable Abbreviation Description Tcode Real output YER Gross domestic product (GDP) at market prices in millions of Euros, chain linked volume, calendar and seasonally adjusted data, reference year 1995. 1 Prices YED GDP deflator, index base year 1995. Defined as the ratio of nominal and real GDP. 1 Short-term interest rate STN Nominal short-term interest rate, Euribor 3-month, percent per annum 2 Investment ITR Gross fixed capital formation in millions of Euros, chain linked volume, calendar and seasonally adjusted data, reference year 1995. 1 Consumption PCR Individual consumption expenditure in millions of Euros, chain linked volume, calendar and seasonally adjusted data, reference year 1995. 1 Exchange rate EEN Nominal effective exchange rate, Euro area-19 countries vis-à-vis the NEER-38 group of main trading partners , index base Q1 1999. 1 Unemployment URX Unemployment rate, percentage of civilian work force, total across age and sex, seasonally adjusted, but not working day adjusted. 2 Note: Data was retrieved from https://eabcn.org/page/area-wide-model. Tcode = 1 indicates that differences of logs were taken, while Tcode = 2 implies that the raw data was used.
Econometrics 2020,8, 20 32 of 36 Appendix D.2. Posterior Paths −4 0 2 4 YER YER YED STN ITR PCR EEN URX YED −4 0 2 4 −4 0 2 4 STNITR −4 0 2 4 −4 0 2 4 PCREEN −4 0 2 4 1970 1990 2010 −4 0 2 4 URX 1970 1990 20101970 1990 20101970 1990 20101970 1990 20101970 1990 20101970 1990 2010 Figure A3. Each cell represents the corresponding state of the matrix Φ1,t , for t= 1, . . . , T , for the data described in Section 5.4. The solid line is the median and the shaded areas represent 50% and 95% posterior credible intervals under the triple gamma prior.
Econometrics 2020,8, 20 33 of 36 −4 0 2 4 YER YER YED STN ITR PCR EEN URX YED −4 0 2 4 −4 0 2 4 STNITR −4 0 2 4 −4 0 2 4 PCREEN −4 0 2 4 1970 1990 2010 −4 0 2 4 URX 1970 1990 20101970 1990 20101970 1990 20101970 1990 20101970 1990 20101970 1990 2010 Figure A4. Each cell represents the corresponding state of the matrix Φ2,t , for t= 1, . . . , T , for the data described in Section 5.4. The solid line is the median and the shaded areas represent 50% and 95% posterior credible intervals under the triple gamma prior.
Econometrics 2020,8, 20 34 of 36 References Abramowitz, Milton, and Irene A. Stegun, eds. 1973. Handbook of Mathematical Functions. New York: Dover Publications. Armagan, Artin, David B. Dunson, and Merlise Clyde. 2011. Generalized beta mixtures of Gaussians. In Advances in Neural Information Processing Systems. Vancouver: NIPS, pp. 523–31. Belmonte, Miguel, Gary Koop, and Dimitris Korobolis. 2014. Hierarchical shrinkage in time-varying parameter models. Journal of Forecasting 33: 80–94. [CrossRef] Berger, James O. 1980. A robust generalized Bayes estimator and confidence region for a multivariate normal mean. The Annals of Statistics 8: 716–61. [CrossRef] Bhadra, Anindya, Jyotishka Datta, Nicholas G. Polson, and Brandon Willard. 2017a. The horseshoe + estimator of ultra-sparse signals. Bayesian Analysis 12: 1105–31. [CrossRef] Bhadra, Anindya, Jyotishka Datta, Nicholas G. Polson, and Brandon Willard. 2017b. Horseshoe regularization for feature subset selection. arXiv: arXiv:1702.07400. Bhadra, Anindya, Jyotishka Datta, Nicholas G. Polson, and Brandon Willard. 2019. Lasso meets horseshoe: A survey. Statistical Science 34: 405–27. [CrossRef] Bitto, Angela, and Sylvia Frühwirth-Schnatter. 2019. Achieving shrinkage in a time-varying parameter model framework. Journal of Econometrics 210: 75–97. [CrossRef] Brown, Philip J., Marina Vannucci, and Tom Fearn. 2002. Bayes model averaging with selection of regressors. Journal of the Royal Statistical Society, Ser. B 64: 519–36. [CrossRef] Carriero, Andrea, Todd E. Clark, and Massimiliano Marcellino. 2019. Large Bayesian vector autoregressions with stochastic volatility and non-conjugate priors. Journal of Econometrics 212: 137–54. [CrossRef] Carvalho, Carlos M., Nicholas G. Polson, and James G. Scott. 2009. Handling sparsity via the horseshoe. Journal of Machine Learing Research W&CP 5: 73–80. Carvalho, Carlos M., Nicholas G. Polson, and James G. Scott. 2010. The horseshoe estimator for sparse signals. Biometrika 97: 465–80. [CrossRef] Chan, Joshua C. C., and Eric Eisenstat. 2016. Bayesian model comparison for time-varying parameter VARs with stochastic volatilty. Journal of Applied Econometrics 218: 1–24. Cottet, Remy, Robert J. Kohn, and David J. Nott. 2008. Variable selection and model averaging in semiparametric overdispersed generalized linear models. Journal of the American Statistical Association 103: 661–71. [CrossRef] Del Negro, Marco, and Giorgio E. Primiceri. 2015. Time Varying Structural Vector Autoregressions and Monetary Policy: A Corrigendum. The Review of Economic Studies 82: 1342–45. [CrossRef] Eisenstat, Eric, Joshua C.C. Chan, and Rodney W. Strachan. 2014. Stochastic model specification search for time-varying parameter VARs. SSRN Electronic Journal 01/2014. [CrossRef] Fagan, Gabriel, Jerome Henry, and Ricardo Mestre. 2005. An area-wide model for the euro area. Economic Modelling 22: 39–59. [CrossRef] Fahrmeir, Ludwig, Thomas Kneib, and Susanne Konrath. 2010. Bayesian regularisation in structured additive regression: A unifying perspective on shrinkage, smoothing and predictor selection. Statistics and Computing 20: 203–19. [CrossRef] Feldkircher, Martin, Florian Huber, and Gregor Kastner. 2017. Sophisticated and small versus simple and sizeable: When does it pay off to introduce drifting coefficients in Bayesian VARs. arXiv: arXiv:1711.00564. Fernández, Carmen, Eduardo Ley, and Mark F. J. Steel. 2001. Benchmark priors for Bayesian model averaging. Journal of Econometrics 100: 381–427. [CrossRef] Figueiredo, Mario A. T. 2003. Adaptive sparseness for supervised learning. IEEE Transaction on Pattern Analysis and Machine Intelligence 25: 1150–59. [CrossRef] Frühwirth-Schnatter, Sylvia. 2004. Efficient Bayesian parameter estimation. In State Space and Unobserved Component Models: Theory and Applications. Edited by Andrew Harvey, Siem Jan Koopman and Neil Shephard. Cambridge: Cambridge University Press, pp. 123–51. Frühwirth-Schnatter, Sylvia, and Regina Tüchler. 2008. Bayesian parsimonious covariance estimation for hierarchical linear mixed models. Statistics and Computing 18: 1–13. [CrossRef] Frühwirth-Schnatter, Sylvia, and Helga Wagner. 2010. Stochastic model specification search for Gaussian and partially non-Gaussian state space models. Journal of Econometrics 154: 85–100. [CrossRef]
Econometrics 2020,8, 20 35 of 36 Gelman, Andrew. 2006. Prior distributions for variance parameters in hierarchical models (Comment on Article by Browne and Draper). Bayesian Analysis 1: 515–34. [CrossRef] Griffin, Jim E., and Phil J. Brown. 2011. Bayesian hyper-lassos with non-convex penalization. Australian & New Zealand Journal of Statistics 53: 423–42. Griffin, Jim E., and Phil J. Brown. 2017. Hierarchical shrinkage priors for regression models. Bayesian Analysis 12: 135–59. [CrossRef] Jacquier, Eric, Nicholas G. Polson, and Peter E. Rossi. 1994. Bayesian analysis of stochastic volatility models. Journal of Business &Economic Statistics 12: 371–417. Johnstone, Iain M., and Bernard W. Silverman. 2004. Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences. The Annals of Statistics 32: 1594–649. [CrossRef] Kalli, Maria, and Jim E. Griffin. 2014. Time-varying sparsity in dynamic regression models. Journal of Econometrics 178: 779–93. [CrossRef] Kastner, Gregor. 2016. Dealing with stochastic volatility in time series using the R package stochvol. Journal of Statistical Software 69: 1–30. [CrossRef] Kastner, Gregor, and Sylvia Frühwirth-Schnatter. 2014. Ancillarity-sufficiency interweaving strategy (ASIS) for boosting MCMC estimation of stochastic volatility models. Computational Statistics and Data Analysis 76: 408–23. [CrossRef] Kleijn, Richard, and Herman K. van Dijk. 2006. Bayes model averaging of cyclical decompositions in economic time series. Journal of Applied Econometrics 21: 191–212. [CrossRef] Koop, Gary, and Dimitris Korobilis. 2013. Large time-varying parameter VARs. Journal of Econometrics 177: 185–98. [CrossRef] Koop, Gary, and Simon M. Potter. 2004. Forecasting in dynamic factor models using Bayesian model averaging. Econometrics Journal 7: 550–65. [CrossRef] Kowal, Daniel R., David S. Matteson, and David Ruppert. 2019. Dynamic shrinkage processes. Journal of the Royal Statistical Society, Ser. B 81: 673–806. [CrossRef] Ley, Eduardo, and Mark F. J. Steel. 2009. On the effect of prior assumptions in Bayesian model averaging with applications to growth regression. Journal of Applied Econometrics 24: 651–74. [CrossRef] Makalic, Enes, and Daniel F. Schmidt. 2016. A simple sampler for the horseshoe estimator. IEEE Signal Processing Letters 23: 179–82. [CrossRef] Nakajima, Jouchi. 2011. Time-varying parameter VAR model with stochastic volatility: An overview of methodology and empirical applications. Monetary and Economic Studies 29: 107–42. Park, Trevor, and George Casella. 2008. The Bayesian Lasso. Journal of the American Statistical Association 103: 681–86. [CrossRef] Pérez, Maria-Eglée, Luis Raúl Pericchi, and Isabel Cristina Ramírez. 2017. The scaled beta2 distribution as a robust prior for scales. Bayesian Analysis 12: 615–37. [CrossRef] Polson, Nicholas G., and James G. Scott. 2011. Shrink globally, act locally: Sparse Bayesian regularization and prediction. In Bayesian Statistics 9. Edited by José M. Bernardo, M. J. Bayarri, James O. Berger, Phil Dawid, David Heckerman, Adrian F. M. Smith and Mike West. Oxford: Oxford University Press, pp. 501–38. Polson, Nicholas G., and James G. Scott. 2012a. Local shrinkage rules, Lévy processes, and regularized regression. Journal of the Royal Statistical Society, Ser. B 74: 287–311. [CrossRef] Polson, Nicholas G., and James G. Scott. 2012b. On the half-Cauchy prior for a global scale parameter. Bayesian Analysis 7: 887–902. [CrossRef] Primiceri, Giorgio E. 2005. Time varying structural vector autoregressions and monetary policy. Review of Economic Studies 72: 821–52. [CrossRef] Raftery, Adrian E., David Madigan, and Jennifer A. Hoeting. 1997. Bayesian model averaging for linear regression models. Journal of the American Statistical Association 92: 179–91. [CrossRef] Roˇcková, Veronika, and Kenichiro McAlinn. 2020. Dynamic variable selection with spike-and-slab process priors. Bayesian Analysis. Sala-i-Martin, Xavier, Gernot Doppelhofer, and Ronald I. Miller. 2004. Determinants of long-term growth: A Bayesian averaging of classical estimates (BACE) approach. The American Economic Review 94: 813–35. [CrossRef] Scheipl, Fabian, and Thomas Kneib. 2009. Locally adaptive Bayesian p-splines with a normal-exponential-gamma prior. Computational Statistics and Data Analysis 53: 3533–52. [CrossRef]
Econometrics 2020,8, 20 36 of 36 Strawderman, William E. 1971. Proper Bayes minimax estimators of the multivariate normal mean. The Annals of Statistics 42: 385–88. [CrossRef] Tibshirani, Ryan. 1996. Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, Ser. B 58: 267–88. [CrossRef] van der Pas, Stéphanie, Bas Kleijn, and Aad van der Vaart. 2014. The horseshoe estimator: Posterior concentration around nearly black vectors. Electronic Journal of Statistics 8: 2585–618. [CrossRef] Zhang, Yan, Brian J. Reich, and Howard D. Bondell. 2017. High dimensional linear regression via the R2-D2 shrinkage prior. Technical report. arXiv: arXiv:1609.00046v2. c 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).