Comparing basic statistical properties of IID (independently and identically distributed) time series with autoregressive and random walk time series
This post is a supplement to my earlier post on IID: please read it before you start reading this one.
Consider the AR(1) model
Basic descriptive properties of an IID process where ρ = 0, AR(1) time series with ρ ∈ {0.3, 0.6, 0.9}, and a random walk (ρ = 1) are compared using time plots and autocorrelation functions are compared in this period. R code is provided.
Time plots
An IID series Y1, as an AR(1) time series with ρ = 0, shows no pattern at all, randomly and frequently fluctuating around the mean of 0. It has a strong tendency to revert back to the mean.
For Y2 to Y4, as the value of ρ increases from 0.3 to 0.9, the time series gets smoother and less frequent, reflecting an increasing degree of dependence on its own past. The degree of mean-reversion also declines as the value of ρ gets higher.
A random walk Y5 shows a trend which can change its direction randomly (called stochastic trend). It shows an increasing variability over time, as shown in the first result in (9), with a little tendency to revert back to its mean over time (mean-aversion).
Autocorrelation Functions
The autocorrelation function of a time series plots Corr(Yt,Yt-k) against the lag value of k. It provides a visual summary of the dependence of structure of a time series. For example, Corr(Yt,Yt-1) measures how much the values of Y 1-period apart are correlated. The blue horizontal band (dotted) is 95% confidence band, and a value of autocorrelation inside this band means that the correlation is statistically no different from 0 at the 5% level of significance.
An IID time series Y1 has all of the autocorrelation values practically negligible and statistically 0.
As the value of ρ increases from 0.3 to 0.9, the degree of dependence Y on its own past increases, as more autocorrelation values get practically larger and statistically different from 0.
A random walk time series Y5 has all autocorrelation values extremely close to 1, indicative of extremely high degree of dependence on its own past (persistence). This is a reflection of the second property given in (9).
This application presents the basic statistical properties of an IID time series, in comparison with those of AR(1) and random walk. It illustrates that the degree of dependence on past (or predictability) changes as the value of AR(1) coefficient changes from 0 to 1, i.e., from an IID time series to a random walk. As explained my previous post on IID, a time series is predictable when the degree of dependence is moderately strong, characterized by the value of ρ greater than 0 but less than 1.
R code
The time series and plots are generated using the following R code:
set.seed(1234) n=500 # Sample size # IID Y1 = rnorm(n) # AR(1) with rho = 0.3, 0.6, and 0.9 Y2 = arima.sim(list(order=c(1,0,0), ar=0.3), n) Y3 = arima.sim(list(order=c(1,0,0), ar=0.6), n) Y4 = arima.sim(list(order=c(1,0,0), ar=0.9), n) # Random Walk Y5 = cumsum(rnorm(n)) par(mfrow=c(3,1)) # Time plots plot.ts(Y1,main="IID",lwd=2) plot.ts(Y2,main="AR(1) with rho=0.3",lwd=2) plot.ts(Y3,main="AR(1) with rho=0.6",lwd=2) plot.ts(Y4,main="AR(1) with rho=0.9",lwd=2) plot.ts(Y5,main="Random Walk",lwd=2) # Autocorrelation functions acf(Y1,main="IID"); acf(Y2,main="AR(1) with rho=0.3"); acf(Y3,main="AR(1) with rho=0.6"); acf(Y4,main="AR(1) with rho=0.9"); acf(Y5,main="Random Walk");