ARMA(p, q) models

In the last few posts we have talked about how physical models relate to "economical" AR(p) models. Yet our end goal is to about ARFIMA models of which "AR" makes up just about 1/3rd. In this post we introduce you to MA(q) models and show how they combine AR(p) models to make up ARMA(p, q) models.

MA(q) model

Quite likely you are familiar with the concept of average. Moving average is a generalization of this concept - it calculates averages not over all sampled values, but over certain subset of subsequent samples. In practice it is one of the simplest tools how to remove excessive randomness in the time series. After applying moving average to some time series we often get smoother representation of the same series.

MA(q) model applies this to the last q values of the noise terms.

\begin{equation} x_t = \beta_1 \xi_{t-1} + \beta_2 \xi_{t-2} + \ldots + \beta_q \xi_{t-q} + \xi_t = \sum_{i=1}^{q} \beta_i \xi_{t-i} + \xi_t = \vec{\beta} \cdot \vec{\xi}^{(q)} + \xi_t. \end{equation}

In the above \( \vec{\xi}^{(q)} \) stands for the \( q \) most recent samples of noise, starting from the most recent.

This form is quite like the one of the AR(p) process, but instead of the new value being dependent on the previous samples it is conditioned on the history of noise.

Note that in case of MA(q) process ACF plot becomes more informative. Now the number of peaks (minus one) in the ACF plot seems to indicate the order of MA process.

ARMA(p, q) model

In practice crude models as AR(p) or MA(q) rarely work well. So they are often combined to make ARMA(p, q) model, samples of which mathematically are defined as:

\begin{equation} x_t = \vec{\alpha} \cdot \vec{x}^{(p)} + \vec{\beta} \cdot \vec{\xi}^{(q)} + \xi_t . \end{equation}

With this model we have a system agnostic tool to do forecasts for real life systems (although adding tool to deal with seasonality would help too). Now everything has become a lot more complicated (though reality can be even more complicated [1]), but feel free to explore using the app below.

References

• E. E. Holmes, M. D. Scheuerell, E. J. Ward. Applied Time Series Analysis for Fisheries and Environmental Sciences. Edition 2021.