PACF and AR(p) models

In the last few posts we have seen that random walk can be written in recursive form, which suggests that random walk is AR(1) process. We have also became familiar with the partial auto-correlation functions. Here in this post we show that PACF can provide an intuition on the order of AR which should be used in modeling the data.

Random walk and PACF

If we calculate PACF of a time series generated using random walk, we see that PACF(1) is noticeably different from zero, while PACF for longer lags is quite close to zero. This single peak hints that we need indeed AR(1) model for this particular data:

\begin{equation} x_t = \alpha_1 x_{t-1} + \xi_t , \quad \text{given} \quad x_0 = 0 . \end{equation}

In the app below we have implicitly set \( \alpha_1 = 1 \). Lower values of \( \alpha_1 \) would make the process stationary, but this should not have any effect on the PACF plot.

Note that all values of PACF become close to zero if we apply differencing procedure. This is expected, because in this case we revert back to AR(0) model - white noise.

Another thing to notice in the app is that we do not plot PACF(0). As with ACF(0) it will also be always equal to 1. We still plot ACF(0) in ACF plots due to tradition, but we don't plot PACF(0) so that we could easily count the peaks and know the order of the AR process to be used to model the data.

AR(5) interactive app

Below we have a generalized app available, which allows you to consider AR process up to fifth order:

\begin{equation} x_t = \sum_{i=1}^{5} \alpha_i x_{t-i} + \xi_t = \vec{\alpha} \cdot \vec{x}^{(5)} + \xi_t . \end{equation}

Let the model parameters be encoded in vector \( \vec{\alpha} \). Let \( \vec{x}^{(p)} \) contain last \( p \) samples of \( x_i \), starting from the most recent.

Now observe how both ACF and PACF change, when you change the order of the time series. As soon as more terms of \( \vec{\alpha} \) are noticeably non-zero PACF changes to show more peaks.

Note that as you add more positive terms to \( \vec{\alpha} \) notification about stationarity may change. The process ought to be stationary if:

\begin{equation} \sum_i \alpha_i < 1 \quad \text{and} \quad \max( |\alpha_i| ) < 1 . \end{equation}

Note that the further your parameter set is from the boundary between stationary and non-stationary regime the faster ACF decays.