class: center, middle, inverse, title-slide # State Space Modelling ### Kevin Kotzé --- <!-- layout: true --> <!-- background-image: url(image/logo.svg) --> <!-- background-position: 2% 98% --> <!-- background-size: 10% --> --- # Contents 1. Introduction 1. Mathematics of State Space modelling 1. Practicalities 1. Examples 1. Filtering and Predicting 1. Diagnostics 1. Forecasting 1. Multivariate models 1. State Space and Box-Jenkins methods --- # Introduction to state space modelling - Provides an encompassing framework to time series modelling - Particularly useful when dealing with structural & dynamic time series, as well as models for TVP - Unobserved variables reflect the state of the system that evolve through time - Examples: - Macroeconomic analysis & state of business cycle - Other unobserved variables incl. expectations, reservation wages, permanent income, etc. - Dynamic Stochastic General Equilibrium models - Stochastic volatility models - State of system is determined by an unobserved vector `\(\{\alpha_{1}, \ldots,\alpha_{n}\}\)` which is associated with observations `\(\{y_{1}, \ldots,y_{n}\}\)` - Relationship between the `\(\alpha_{t}\)`'s and the `\(y_{t}\)`'s is specified by the state space model --- # Basic State Space Model - A time series `\(\{y_{1},\ldots,y_{n}\}\)` may be expressed in the additive form `\begin{eqnarray} y_{t}= \mu_{t} + \gamma_{t} + \varepsilon_{t}, & \;\;\;\; & t=1,\ldots,T \end{eqnarray}` - where: - `\(\mu_{t}\)` is a slowly varying component called the *trend* - `\(\gamma_{t}\)` is a periodic component called the *seasonal* - `\(\varepsilon_{t}\)` is an irregular component called the *error* - To model `\(\mu_{t}\)` and `\(\gamma_{t}\)` suppose a scalar series `\(\alpha_{t}\)` follows a random walk `\begin{eqnarray} \alpha_{t+1}= \alpha_{t} + \eta_{t}, & \;\;\;\; & \eta_{t} \sim \mbox{i.i.d.} \mathcal{N}(0, W_{\eta}) \end{eqnarray}` --- # Basic State Space Model - If we assume that no seasonal is present `\((\gamma_{t}=0)\)`, and - `\(\mu_{t}=\alpha_{t}\)` (where `\(\alpha_{t}\)` is a random walk), we can rewrite the previous expressions as `\begin{eqnarray} y_{t}= \alpha_{t} + \varepsilon_{t}, & \;\;\;\; & \varepsilon_{t} \sim \mbox{i.i.d.} \mathcal{N}(0, V_{\varepsilon}) \\ \alpha_{t+1}= \alpha_{t} + \eta_{t}, & \;\;\;\; & \eta \sim \mbox{i.i.d.} \mathcal{N}(0, W_{\eta}) \end{eqnarray}` - Such a model has state space characteristics as we have: - a **measurement equation** that describes the relation between the observed variables `\(\{y_{1},\ldots,y_{n}\}\)` and the unobserved state variables `\((\alpha_{t})\)` - a **state equation** that reflects the dynamics of the unobserved state variables `\(\{\alpha_{1},\ldots,\alpha_{n}\}\)` --- # Basic State Space Model - Objective of state space modelling is to infer properties of `\(\alpha_{t}\)`'s from the observed `\(\{y_{1},\ldots,y_{n}\}\)` - To accomplish this process we can make use of a groovy technique `\(\hspace{5cm}\)`The Kalman Filter! --- # General Exposition - The state space form provides a unified representation of - ARIMA models, unobserved components models - dynamic time series models, time varying parameters - non-parametric regressions, spline regressions - So this form is probably worth remembering! --- # Measurement equation - Consider `\(m\)` state variables that may be subject to distortions and noise (i.e. shocks) - State variables are contained in an `\(m \times 1\)` vector, `\(\alpha_t\)` - The `\(N\)` variables that are observed are defined by an `\(N \times 1\)` vector, `\(y_t\)` - Then we can define the measurement equation as `\begin{equation} y_{t} = F_{t}\alpha_{t} + S_{t}\varepsilon_{t} \end{equation}` - where `\(F_t\)` is a fixed matrix of order `\(N \times m\)` - `\(r\)` is the dimension of the measurement equation disturbance vector - `\(\varepsilon_t\)` is a `\(r \times 1\)` vector with zero mean & covariance matrix, `\(V\)` - `\(S_t\)` is also a fixed matrix of order `\(N \times r\)` --- # State equation - The state equation could then be described as `\begin{equation} \alpha_{t+1} = G_{t}\alpha_{t} + R_{t}\eta_{t} \end{equation}` - where `\(G_t\)` and `\(R_t\)` are fixed matrices of order `\(m \times m\)` and `\(m \times g\)` - `\(g\)` refers to the dimension of the state equation disturbance vector - `\(\eta_t\)` is a `\(g \times 1\)` vector with zero mean and covariance matrix, `\(W\)` --- # Disturbances - Disturbances in the measurement and state equations are assumed to be uncorrelated `\begin{eqnarray} \left\{ \begin{array}{ll} \varepsilon_t\\ \eta_t\\ \end{array} \right\} \sim \mbox{i.i.d.} \; \mathcal{N} \left[ 0, \left( \begin{array}{ll} V & 0\\ 0 & W\\ \end{array} \right)\right] \end{eqnarray}` - And they are also uncorrelated with the initial state vector `\(\alpha_0\)` `\begin{eqnarray} \mathbb{E}\left[\alpha_{0} \eta_{t}^{\prime}\right]=0, & \mathbb{E}\left[\alpha_{0} \varepsilon_{t}^{\prime}\right]=0 \end{eqnarray}` --- # Unknowns - The covariance matrix of the error terms may be referred to as `\(\Omega\)` - The coefficient matrix may be referred to as `\(\Phi\)` `\begin{eqnarray} \Phi = \left\{ \begin{array}{ll} F_t\\ G_t\\ \end{array} \right\} , \; \; \; \; \Omega = \left\{ \begin{array}{ll} V & 0\\ 0 & W\\ \end{array} \right\} \end{eqnarray}` - which represent the unknowns in any standard regression model --- # The practicalities of formulating a model - There are several software packages that facilitate State Space models - **R** packages include `dlm`, `sspir`, `StructTS`, `RWinBugs`, `MASS`, `RStan`, etc. --- # Example: Local Level Model - Simplest example of state space model - Level component is allowed to vary over time - Hence the measurement and state equations are `\begin{eqnarray} y_{t} = \mu_{t} + \varepsilon_{t}, & \;\;\;\; & \varepsilon_{t} \sim \mathsf{i.i.d.} \mathcal{N}(0,V_{\varepsilon}) \\ \mu_{t+1} = \mu_{t} + \xi_{t}, & \;\;\;\; & \xi_{t} \sim \mathsf{i.i.d.} \mathcal{N}(0,W_\xi) \end{eqnarray}` - Or alternatively `\begin{eqnarray} \alpha_t = \mu_t, \; \eta_t = \xi_t, \; F_t = G_t = S_t = R_t = 1, \; W = W_\xi, \; V = V_{\varepsilon}, \end{eqnarray}` --- background-image: url(image/llm_smo.svg) background-position: top background-size: 90% 90% class: clear, center, bottom Figure : Local level model - SA inflation (1960Q2-2014Q1) --- # Results: Local Level Model - In terms of the statistical results for this model - negative log-likelihood is 79.6 - `\(\hat{V}\)` is `\(0.124\)` - `\(\hat{W}\)` is `\(0.029\)` - `\(\mu\)`, at the final state at period 2014Q1 is 2.86% per year - To compare different state space models use Akaike Information Criterion (AIC): `\begin{eqnarray} AIC =\left[ -2 \log \ell + 2(n) \right] = 163.29 \end{eqnarray}` - Compensates for the number of parameters in a model - Smaller (or more negative) values denote better fitting models --- # Example: Local Level Trend Model - The local linear trend model has two state equations to include a slope component, `\(\upsilon_{t}\)` - It may be derived from the general specification by defining `\begin{eqnarray} y_{t} = \mu_{t} + \varepsilon_{t}, & \;\;\;\; & \varepsilon_{t} \sim \mathsf{i.i.d.} \mathcal{N}(0,V_{\varepsilon}) \\ \mu_{t+1} = \mu_{t} + \upsilon_{t} + \xi_{t}, & \;\;\;\; & \xi_{t} \sim \mathsf{i.i.d.} \mathcal{N}(0,W_{\xi}) \\ \upsilon_{t+1} = \upsilon_{t} +\zeta_{t}, & \;\;\;\; & \zeta_{t} \sim \mathsf{i.i.d.} \mathcal{N}(0,W_{\zeta}) \end{eqnarray}` - Or alternatively `\begin{eqnarray} \alpha_{t} = \binom{\mu_{t}}{\upsilon_{t}}, \; \eta_{t} = \binom{\xi_{t}}{\zeta_{t}}, \; G_{t} = \left( \begin{array}{cc} 1 & 1 \\ 0 & 1 \end{array} \right), \; F_{t} = \binom{1}{0}, \; S_{t} = 1, \end{eqnarray}` `\begin{eqnarray} W = \left( \begin{array}{cc} W_{\xi} & 0 \\ 0 & W_{\zeta} \end{array} \right), \; R_{t} = \left( \begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \right), \; V = V_{\varepsilon} \end{eqnarray}` --- background-image: url(image/llmt_smo.svg) background-position: top background-size: 90% 90% class: clear, center, bottom Figure : Local level model - SA inflation (1960Q2-2014Q1) --- background-image: url(image/llmt_slop.svg) background-position: top background-size: 90% 90% class: clear, center, bottom Figure : Local level trend model - Stochastic slope --- # Example: Local Level Trend with Seasonal - Time series are often influenced by seasonal characteristics incorporated in the state space framework `\begin{eqnarray} y_{t} = \mu_{t} +\gamma_{1,t} + \varepsilon_{t}, & \;\;\;\; & \varepsilon_{t} \sim \mathsf{i.i.d.} \mathcal{N}(0,V_{\varepsilon}) \\ \mu_{t+1} = \mu_{t} + \upsilon_{t} + \xi_{t}, & \;\;\;\; & \xi_{t} \sim \mathsf{i.i.d.} \mathcal{N}(0,W_\xi) \\ \upsilon_{t+1} = \upsilon_{t} +\zeta_{t}, & \;\;\;\; & \zeta_{t} \sim \mathsf{i.i.d.} \mathcal{N}(0,W_\zeta) \\ \gamma_{1,t+1} = -\gamma_{1,t} -\gamma_{2,t} -\gamma_{3,t} +\omega_{t}, & \;\;\;\; & \omega_{t} \sim \mathsf{i.i.d.} \mathcal{N}(0,W_\omega) \\ \gamma_{2,t+1} = \gamma_{1,t},\\ \gamma_{3,t+1} = \gamma_{2,t} \end{eqnarray}` - where the `\(\gamma\)` terms refer to the seasonal components - disturbance `\(\omega\)` allows for the seasonal to change over time - require several state equations (i.e. frequency -1) - the last two equations suggest that `\(\gamma_1\)` follows `\(\gamma_2\)` and `\(\gamma_2\)` follows `\(\gamma_3\)` --- # Example: Local Level Trend with Seasonal Or alternatively, `\begin{eqnarray} \alpha_{t} = \left( \begin{array}{c} \mu_{t} \\ \upsilon_t \\ \gamma_{1,t} \\ \gamma_{2,t} \\ \gamma_{3,t} \\ \end{array} \right), \; \eta_{t} = \left( \begin{array}{c} \xi_{t} \\ \zeta_{t} \\ \omega_{t} \end{array} \right), \; G_{t} = \left( \begin{array}{ccccc} 1 & 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0\\ 0 & 0 & -1 & -1 & -1 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ \end{array} \right) \end{eqnarray}` `\begin{eqnarray} F_{t} = \left( \begin{array}{c} 1 \\ 0 \\ 1 \\ 0 \\ \end{array} \right), \; S_{t} = 1, \; W = \left( \begin{array}{ccc} W_{\xi} & 0 & 0 \\ 0 & W_\zeta & 0\\ 0 & 0 & W_\omega \end{array} \right), \; R_{t} = \left( \begin{array}{ccc} 1 & 0 & 0 \\ 0 & 1 & 0\\ 0 & 0 & 1\\ 0 & 0 & 0\\ 0 & 0 & 0 \end{array} \right)\; \end{eqnarray}` `\(V = V_{\varepsilon}\)` --- background-image: url(image/llmts_smo.svg) background-position: top background-size: 90% 90% class: clear, center, bottom Figure : Local level model with seasonal- SA inflation (1960Q2-2014Q1) --- background-image: url(image/llmts_seas.svg) background-position: top background-size: 90% 90% class: clear, center, bottom Figure : Local level trend model with seasonal - Stochastic seasonal --- # Local level model with intervention variable - Assess the impact of a structural change on a particular time series over time - Performed by adding intervention variables to any of the above models - Includes *level shift*, where the value of the level of the time series exhibits a permanent change - Or *slope shift*, where the value that is attached to the slope experiences a permanent change - Or *pulse* effect, where the value of the level suddenly changes and then returns to previous levels --- # Local level model with intervention variable - To determine the impact of a structural change on a local level model one could estimate: `\begin{eqnarray} y_{t} = \mu_{t} + \varepsilon_{t}, & \;\;\;\; & \varepsilon_{t} \sim \mathsf{i.i.d.} \mathcal{N}(0,V_{\varepsilon}) \\ \mu_{t+1} = \mu_{t} + \lambda_t w_{t} + \xi_{t}, & \;\;\;\; & \xi_{t} \sim \mathsf{i.i.d.} \mathcal{N}(0,W_{\xi}) \\ \lambda_{t+1} = \lambda_t \end{eqnarray}` --- # Local level model with explanatory variable - It's easy to add the explanatory variables to the measurement equation of the model - Just add an additional state equation for each additional explanatory variable while maintaining the unobserved components `\begin{eqnarray} y_{t} = \mu_{t} + \beta_t x_{t}+ \varepsilon_{t}, & \; & \varepsilon_{t} \sim \mathsf{i.i.d.} \mathcal{N}(0,V_{\varepsilon}) \\ \mu_{t+1} = \mu_{t} + \xi_{t}, & \; & \xi_{t} \sim \mathsf{i.i.d.} \mathcal{N}(0,W_\xi) \\ \beta_{t+1} = \beta_{t} +\tau_{t}, & \; & \tau_{t} \sim \mathsf{i.i.d.} \mathcal{N}(0,W_\tau) \end{eqnarray}` --- # Example: Model with explanatory variables - Consider per capita consumption of spirits in the UK - Per capita income, and the relative price of spirits from 1870 to 1938 - Allow for a stochastic trend element that describes changes in tastes and habits - Allow for `\(\beta\)` parameters to be time-varying --- background-image: url(image/llm_reg.svg) background-position: top background-size: 90% 90% class: clear, center, bottom Figure : Local level model with explanatory variables --- # Results: Model with explanatory variables - The output for the regressive variables are interpreted as ordinary regression coefficients - `\(\bar{\beta} = -0.927\)` indicates that a one percent increase in price leads to a fall in spirit consumption of `\(-0.9\)` (on average) - Decline in coefficient indicates consumers have become more price sensitive (but only by a small amount) - The decline in the level suggests that change in taste occurred, away from the hard stuff --- # Confidence Intervals - State equation variables are estimated - provides standard errors - Allows for the construction of confidence intervals for each of the state components - Thus allowing for the an evaluation of uncertainty at each point in time --- background-image: url(image/llm_conf.svg) background-position: top background-size: 90% 90% class: clear, center, bottom Figure : Local level trend model with confidence intervals - SA inflation (1960Q2-2014Q1) --- # Kalman Filter and Disturbance Smoother - The estimation of the state vectors are carried out by performing: - a *forward pass* - use of a recursive Kalman Filter may be used to find the filtered state - the *backward pass* - state and disturbance smoothers are applied to the output of the Kalman Filter - Kalman filter provides the optimal values of that state at point `\(t\)` considering present and past observations: - provides estimates for the filtered state - filtered state error variances - Disturbance smoother derives estimates that relate the current state vector to the current observation: - provides estimates for the smoothed state - provides the smoothed state estimation error variances - smoothed irregular components --- # Kalman Filter - Given the state-space model `\begin{eqnarray} y_{t} &=& F_{t}\alpha_{t} + S_{t}\varepsilon_{t} \\ \alpha_{t+1} &=& G_{t}\alpha_{t} + R_{t}\eta_{t} \end{eqnarray}` - The expression for the Kalman filtered state is: `\begin{eqnarray} \alpha_{t+1} = \alpha_{t} + K_{t}(y_{t} - F_{t}'\alpha_{t}) \end{eqnarray}` - For the local level model this given as: `\begin{eqnarray} \mu_{t+1} = \mu_{t} + K_{t}(y_{t} - \mu_{t}) \end{eqnarray}` --- background-image: url(image/kf1.svg) background-position: top background-size: 90% 90% class: clear, center, bottom Figure : Workings of the Kalman Filter --- # Kalman Filter - At time point `\(t = 1980\)` - Filtered level, `\(\alpha_{t}\)`, is based on past observation - If we did not observed value `\(y_t\)`, best estimate for `\(\alpha_{t}+1\)` would be `\(\alpha_{t}\)` - However, at `\(t\)` we do observe `\(y_t\)` - This value is fed into the Kalman filter algorithm - The discrepancy between `\(y_t\)` and `\(\alpha_{t}\)` in 1980 is used to update the estimate for `\(\alpha_{t+1}\)` - Since the discrepancy `\(y_{t} - \alpha_{t}\)` is a negative number, the estimate decreases - Note that in the above expression for the Kalman filter, the larger is `\(K_t\)` the greater is the effect of `\(y_t\)` on subsequent values of `\(\alpha_t\)` --- # Kalman Gain - Note that in the local level model, `\(\varepsilon_t = y_{t} - \alpha_{t}\)` - Call `\((y_{t} - \alpha_{t})\)` the one-step ahead *innovation* errors that have variance `\(\sigma_{\varepsilon,t}\)` - In the local level model the variance of the *prediction* errors `\((\alpha_{t+1} - \alpha_t)\)` for the state equation are denoted `\(P_t\)` - `\(K_t\)` determines how much the innovation errors at `\(t\)` influence the estimate of the state at time `\((t+1)\)` --- # Kalman Gain - `\(K_t\)` is influenced by two issues: - If `\(y_t\)` is highly volatile (with many outliers) then `\(K_t\)` should tend towards zero so that `\(y_t\)` has a small effect on `\(\alpha_{t+1}\)` - If uncertainty about the future state (based on past states) is large; then `\(K_t\)` must be large so that `\(y_t\)` has a large effect on `\(\alpha_{t+1}\)` - Hence, `\begin{eqnarray} K_t = \frac{P_t}{\sigma_{\varepsilon,t}} \end{eqnarray}` --- # Kalman Smoother - The Kalman filter uses values from `\(y_{t}\)` and `\(\alpha_{t}\)` to estimate values for `\(\alpha_{t+1}\)` - Hence the Kalman filter values appear to project `\(y_t\)` by one period - To shift these values back a period we use a smoother - Similar procedure to that of Kalman filter, but we start at the end of the sample and work to the first observation - Could use the specification: `\begin{eqnarray} \alpha^s_{t-1} = \alpha_{t} + J_{t-1}(\alpha^s_{t} - \alpha_{t}) \end{eqnarray}` - where `\(\alpha^s_{t}\)` is the smoothed estimate and `\(\alpha_t\)` is the filtered estimate - Value for `\(J_{t-1}\)` is determined by the ratio between the variance in `\((\alpha^s_{t} - \alpha_{t})\)` and the variance in `\(\alpha_{t}\)` --- background-image: url(image/KGerror.svg) background-position: top background-size: 90% 90% class: clear, center, bottom Figure : Kalman gain and one-step ahead prediction errors --- # Example - Kalman filter in local level mod - Residuals should be: - independent - homoscedastic - normally distributed - Make use of standardised prediction errors `\begin{eqnarray} e_t = \frac{\xi_t}{\sqrt{P_{t}}} \end{eqnarray}` - Example: Inflation gap (i.e. the difference between core and actual inflation) with a local level trend model with seasonal and intervention components, and an explanatory variable (output gap) --- background-image: url(image/diagnost1.svg) background-position: top background-size: 90% 90% class: clear, center, bottom Figure : Diagnostic Tests - Example --- # Testing for Independence (Box-Ljung) - The residual autocorrelation from lag `\(k\)` `\begin{eqnarray} r_k = \frac{\sum_{t=1}^{T-k} (e_t - \bar{e})(e_{t+k} - \bar{e} )}{\sum_{t=1}^{T} (e_t - \bar{e})^2} \end{eqnarray}` - The Box-Ljung statistic may then be expressed as `\begin{eqnarray} Q(k) = T(T+2) \sum_{l=1}^{k} \frac{r_l^2}{T-l} \end{eqnarray}` - for lags `\(l=1, \ldots ,k\)` - Compare to a `\(\chi^2\)` distribution with `\((k-w+1)\)` degrees of freedom (where `\(w\)` is the number of hyperparameters or disturbance variances) - When calculated value `\(<\)` critical value, the null of independence is not rejected - residuals are not serially correlated --- # Homoscedasticity of the residuals - Compare the variance of the residuals in the first third with the variance of the residuals in the last third of the series `\begin{eqnarray} H(h)=\frac{\sum_{t=T-h+1}^{T} e^2_t}{\sum_{t=d+1}^{d+h} e^2_t} \end{eqnarray}` - where `\(d\)` is the number of diffuse initial state values - `\(h\)` is the nearest integer to `\((T-d)/3\)` - Compare to an `\(F\)`-distribution with `\((h,h)\)` degrees of freedom - When `\(1< H(h) < F(h,h; 0.025)\)`, the null of equal variances is not rejected - no departure from homoskedasticity in residuals --- # Normality of the residuals - Consider the skewness and kurtosis of the residual distribution `\begin{eqnarray} N=T\left(\frac{S^2}{6} + \frac{(K-3)^2}{24}\right) \end{eqnarray}` - with the skewness , `\(S\)`, and the kurtosis, `\(K\)`, being defined as `\begin{eqnarray} S=\frac{\frac{1}{T} \sum_{t=1}^{T} (e_t - \bar{e})^3}{\sqrt{\left(\frac{1}{T} \sum_{t=1}^{T} (e_t - \bar{e})^2\right)^3}} \;\;\;\; K=\frac{\frac{1}{T} \sum_{t=1}^{T} (e_t - \bar{e})^4}{\left(\frac{1}{T} \sum_{t=1}^{T} (e_t - \bar{e})^2\right)^2} \end{eqnarray}` - When `\(N<\chi^2_{(2;0.05)}\)` the null hypothesis of normality is not rejected - residuals are not normally distributed --- # Forecasting - To compute forecasts continue with the Kalman filter - Assume that the last observation in the in-sample estimation period is `\(n\)` `\begin{eqnarray} a_n = a_{n-1} + K_{n-1}(y_{n-1} - F^{\prime}_{n-1} a_{n-1}) \end{eqnarray}` - The last observation, `\(y_n\)`, can then be used to update the filtered state at time point `\(t=n+1\)` as follows `\begin{eqnarray} a_{n+1} = a_{n-1} + K_{n-1}(y_{n} - F^{\prime}_{n} a_{n}) \end{eqnarray}` - From `\(n+1\)` onwards the filtered state no longer changes and by letting `\(\bar{a}_{n+1} = a_{n+1}\)` the forecasts simply become `\(\bar{a}_{n+1+j} = \bar{a}_{n+f}\)`, where `\(F\)` refers to the number of time points for the forecast (i.e. the lead time) --- background-image: url(image/llm_fore.svg) background-position: top background-size: 90% 90% class: clear, center, bottom Figure : Filtered trend and forecasts - SA inflation (1960Q1-2014Q1) --- # Multivariate model - Consider the case of a bivariate local linear trend model - where the explanatory variable `\(x_t\)` is applied to both `\(y_t\)` - The measurement equations would be `\begin{eqnarray} y_{t}^{(1)} = \mu_{t}^{(1)} +\beta_{t}^{(1)} x_t + \varepsilon_{t}^{(1)},\\ y_{t}^{(2)} = \mu_{t}^{(2)} +\beta_{t}^{(2)} x_t + \varepsilon_{t}^{(2)}, \end{eqnarray}` --- # Multivariate model - And the following six state equations `\begin{eqnarray} \mu_{t+1}^{(1)} = \mu_{t}^{(1)} + \upsilon_{t}^{(1)} + \xi_{t}^{(1)}, \\ \upsilon_{t+1}^{(1)} = \upsilon_{t}^{(1)} +\zeta_{t}^{(1)}, \\ \beta_{t+1}^{(1)} = \beta_{t}^{(1)},\\ \mu_{t+1}^{(2)} = \mu_{t}^{(2)} + \upsilon_{t}^{(2)} + \xi_{t}^{(2)}, \\ \upsilon_{t+1}^{(2)} = \upsilon_{t}^{(2)} +\zeta_{t}^{(2)}, \\ \beta_{t+1}^{(2)} = \beta_{t}^{(2)} \end{eqnarray}` --- # Common levels and slope - When the slope components are correlated, i.e. `\(\text{cov}(\zeta^{(1)},\zeta^{(2)}) \ne 0\)` - The level of dependence is measured by the correlation between the two disturbances `\begin{eqnarray} \text{corr}(\zeta^{(1)},\zeta^{(2)}) = \frac{\text{cov}(\zeta^{(1)} , \zeta^{(2)} )} {\sqrt{ \sigma^2_{\zeta^{(1)}} \sigma^2_{\zeta^{(2)}} }} \end{eqnarray}` - where `\(-1 < \text{corr}(\zeta^{(1)}, \zeta^{(2)}) \leq 1\)` - When this expression is close to 1: - the one slope is a linear combination of the other slope --- # Common levels and slope - In addition, the following variance matrix has rank one `\begin{eqnarray} \left( \begin{array}{cc} \sigma_{\zeta^{(1)}}^{2} & \text{cov}(\zeta^{(1)},\zeta^{(2)}) \\ \text{cov}(\zeta^{(1)},\zeta^{(2)}) & \sigma_{\zeta^{(2)}}^{2} \end{array} \right) \; \end{eqnarray}` - The same arguments apply to the disturbances of the other components (including the irregular vector `\(\varepsilon_t\)`) --- # Example: Bivariate model - Bivariate model that considers whether the output and inflation gaps move together `\begin{eqnarray} \left( \begin{array}{c} \pi_t \\ y_t \end{array} \right) = \left( \begin{array}{c} \mu_t^\pi \\ \mu_t^y \end{array} \right) + \left( \begin{array}{c} \psi_t^\pi \\ \psi_t^y \end{array} \right) + \left( \begin{array}{c} \varepsilon_t^\pi \\ \varepsilon_t^y \end{array} \right) \end{eqnarray}` - where the `\(\mu_t^\pi\)` and `\(\mu_t^y\)` trend components are integrated random walk processes --- # Example: Bivariate model - To investigate whether the cyclical components exhibit similar behaviour: `\begin{eqnarray} \pi_t = \mu^\pi_t + \beta \psi_t^y + \psi_t^{\pi \dagger} + \varepsilon_t^{\pi} \end{eqnarray}` - Compare the respective models, one with the restriction and one without the restriction - Determine whether the inclusion of the restriction improves the model fit - In this case the restriction does not improve the fit of the model --- background-image: url(image/llm_multi_inf.svg) background-position: top background-size: 80% 80% class: clear, center, bottom Figure : Bivariate model for USA Inflation and GDP --- # Non-stationary ARIMA models - Typical Box-Jenkins approach makes the series stationary - Removing the trend by first differencing to create `\(y_t^\star\)` `\begin{eqnarray} y_t^\star = \Delta y_t = y_t - y_{t-1} \end{eqnarray}` - Removing the seasonal with periodicity `\(s\)` by differencing `\begin{eqnarray} y_t^\star = \Delta_s y_t = y_t - y_{t-s} \end{eqnarray}` - Or remove both the trend and the seasonal `\begin{eqnarray} y_t^\star = \Delta \Delta_s y_t = (y_t - y_{t-s}) - (y_{t-1} - y_{t-s-1}) \end{eqnarray}` - Where the variable is still not stationary take second difference `\begin{eqnarray} y_t^\star = \Delta^2 \Delta_s^2 y_t, \end{eqnarray}` - After sufficient differencing the appropriate `\(AR(p)\)`, `\(MA(q)\)` or `\(ARMA(p,q)\)` is identified --- # Similarities: State-Space and Box-Jenkins - Recall that the local level model has the form `\begin{eqnarray} y_t = \mu_t + \varepsilon_t \end{eqnarray}` `\begin{eqnarray} \mu_t = \mu_{t-1} + \eta_t \end{eqnarray}` - Where the first difference of `\(y_t\)` yields `\begin{eqnarray} \Delta y_{t} = y_t - y_{t-1} = \mu_{t} - \mu_{t-1} + \varepsilon_t - \varepsilon_{t-1} \end{eqnarray}` - Since state equation `\((\mu_t = \mu_{t-1} + \eta_t)\)` implies that `\begin{eqnarray} \mu_t - \mu_{t-1} = \eta_t \end{eqnarray}` - We can rewrite the first difference for `\(\Delta y_t\)` as `\begin{eqnarray} \Delta y_{t} = y_t - y_{t-1} = \eta_{t-1} + \varepsilon_t - \varepsilon_{t-1} \end{eqnarray}` - Which is stationary and has the correlogram of an MA(1) --- # Differences: State-Space and Box-Jenkins - State space approach explicitly models the non-stationarity of trend and the seasonal components - provides simultaneous decomposition of a time series into the respective dynamic components - Box-Jenkins treats these as nuisance components which need to be removed - primarily concerned with short-run dynamics and forecasts - Advantages of state space: - Never really sure as to when a series is non-stationary - Easier to deal with missing data, time-varying regression coefficients and multivariate extensions