A time series is a set of random variables each at a specific time t. {Xt}t = 1n is a time series {xt}t = 1n is the realized path of the time series trend: Xt = mt + Yt, E(Yt)=0 seasonal:Xt = st + Yt, E(Yt)=0, st = st + d model with trend and seasonal characteristics is Xt = mt + st + Yt, E(Yt)=0, st = st + dthe elimination of the trend in the model can be through estimation and differencing Differencing operators:Back-shift Operator:Bt = Xt − 1Lag-Operator :∇(Xt)=Xt − Xt − 1 Lag-j Operator :∇j(Xt)=Xt − Xt − j Power of Operators: Bj(Xt)=Xt − j ∇j(Xt)=∇(∇j − 1Xt − 1) examples : ∇(Xt)=Xt − Xt − 1 = (1 − B)Xt ∇²(Xt)=Xt − 2Xt − 1 + Xt − 2 = (1 − B)²(Xt) generally:∇j(Xt)=(1 − B)jXt, j ∈ ℕ Power calculation of usually easier removing trend by differencing : In general: Xt = mt + Yt with mt = Σj = 0kajtj and E(Yt)=0. Then,∇k(Xt)=k!Ck + ∇kYt(notice )E(∇kYt)=0 so, ∇kXt does not have any trend removing seasonality by differencing : Assume the seasonal model Xt = st + Yt. where, st = st − d and E(Yt)=0, the lag-d differencing will remove the seasonal component: ∇d(Xt)=∇d(Yt) Elimination of trend of seasonality: Model with trend and seasonality : assume Xt = mt + st + Yt , where mt = Σj = 0kajTj, st = st − d and E(Yt)=0 1. Remove Seasonality : ∇dXt = ∇dmt + ∇dst + ∇dYt = ∇dmt + ∇dYt where $\nabla _d m_t=\Sigma ^k _{j=0} _j t^j $ is still a kth order polynomial 2. Remove trend : $ \nabla ^k (\nabla _d X_t)= k! _k + \nabla ^k (\nabla _d Y_t) $ where E[∇k(∇dYt)] = 0 Notice ∇k∇dXt = (1 − B)k(1 − Bd)Xt = (1 − Bd)(1 − B)kXt = ∇d∇kXt Natural log transformation is often used to supress increasing fluction/variation and MUST be applied before any differencing operators Ex : data air 1.log trans 2. ∇₁₂3.∇¹ with steps 2 and 3 interchangable Strictly stationary: {Xt} if ∀s ≥ 1, F(Xt + 1, ..., Xt + s) is independent of t. weakly stationary : Define mean function of {Xt}:μX(t)=E(Xt),∀t, and covariance function of {Xt}:γX(r, s)=Cov(Xr, Xs)=E[Xr − μX(r)][Xs − μX(s)] A time series {Xt} is weakly stationary if μX(t)<∞ is independent of t ; and , E(Xt²)<∞ and γX(t + h, t) is independent of t , ∀h ≥ 0 Let {Xt} be a [weakly] stationary time series : Autocovariance function (ACVF) of {Xt} is defined as γX(h)=Cov(Xt + h, Xt) and the autocorrelation function (ACF) of {Xt} is defined as $\rho _X (h)= \frac {\gamma _X (h)}{\gamma _X (0)} $ ACF is more preferred because it is scale free Properties of ACVF of a stationary process : 1. γ(0)≥0 2. |γ(h)| ≤ γ(0)∀h ∈ ℤ3.γ(h)=γ(−h),∀h ∈ ℤ Properties of ACF of a stationary process : 1.ρ(0)=12.|ρ(h)| ≤ 1∀h ∈ ℤ3.ρ(h)=ρ(−h),∀h ∈ ℤ Assume observing a stationary time series {xt}t = 1n sample mean: $= \frac 1n \Sigma ^n _1 x_t $ . Sample ACVF: $ (h)= \frac 1n \Sigma ^{n-|h|} _{t=1} (X_{t+|h|}- )(X_t - ),|h| < n $ Sample ACF: $ (h) = \frac { (h)} {(0)} $ A linear time series is represented by Xt = μ + Σj = −∞∞ψjZt − j, ∀t where, Zt ∼ WN(0, σ²) and {ψj} is a sequence of constants s.t. Σj = −∞∞|ψj|<∞ In practice Zt is interpreted as the information at time t, so {Zt} is called the innovation process or shock process Properties of linear time series: 1.E(Xt)=μ∀t.2.Var(Xt)=Var(Σj = −∞∞ψjZt − j)=Σj = −∞∞ψj²Var(Zt − j)=σ²Σj = −∞∞ψj² < ∞, ∀t. 3.Cov{X + t + h, Xt}=Σi = −∞∞Σj = −∞∞ψiψjCov(Zt + h − j, Zt − j)=σ²Σj = −∞∞ψj + hψj, ∀h. A linear time series is weakly stationary with ACF ρ(h){Xt} is called an ARMA(p,q) process if it is stationary and satifies Xt − ... − ϕpXt − p = Zt + ... + θqZt − q∀t, {Zt}∼WN(0, σ²) and Φ(z)=1 − ... − ϕpzp, Θ(z)=1 + ... + θqzq have no common factors (Unit roots are less than zero). Causality : ∃{ϕ}s.t.Σj = 0∞|ϕj|<∞,andΣj = 0∞ϕjZt − j, ∀t(Φ(z)≠0∀z ∈ ℂ : |z|≤1). A causal ARMA(p,q) process is a linear time series. A stationary solution {Xt} of the ARMA equation Φ(B)Xt = Θ(B)ZtexistsiffΦ(z)≠0∀z ∈ ℂ : |z|=1 and when the condition is satisfied, the stationary solution is also unique. When Φ(z) has roots inside the unit circle, there exists a non-causal stationary solution but it is too complicated to study (so we conventionally assume stationarity = causality for ARMA processes). An ARMA (p,q) process {Xt} is invertible if ∃{πj}s.t.Σj = 0∞|πj|<∞andZt = Σj = 0∞πjXt − j, ∀t(Θ(z)≠0∀z ∈ ℂ : |z|≤1). For mean of AR equation E(Xt)=μ = Σjp = 1ϕjμ which is equivalent to (1 − Σj = 1pϕj)μ = 0 the only solution to above equation is μ ≡ 0. For k = 0 ,..., p we obtain the following with p+1 equations: $\gamma (0)- \phi _1 \gamma (1)-... - \phi_p \gamma (p)= \sigma ^2; \gamma (1) -\phi _1 \gamma (0) -...- \phi _p \gamma (p-1) =0 ;\gamma (2) -\phi _1 \gamma (1) -...- \phi _p \gamma (p-2) = 0 ; ... \gamma (p) -... - \hi _p \gamma (0)=0. \gamma (k), k \ge p+1 $ is computed recursively by γ(k)=ϕ₁γ(k − 1)+...+ϕpγ(k − p). The best linear predictor of Xh in terms of {Xt₁, ...Xtm} is $ P(X_h | X_t_1, ... , X_t_m)= a_0 + \Sigma ^m _{i=1} a_i X_t_i := \hat X _h $ where the coefficients {a₀, ..., am} minimize the mean square prediction error $E(\hat X _h - X_h ) ^2. \hat X_h = \Sigma ^p _{i=1} \phi _i X_{h-i}, h > p. PACF: \forall h \in ^+ : \alpha (h) = Corr (X_h - P(X_h | X_{h-1} , ... , X_1),X_0 -P(X_0 | X_{h-1} , ... , X_1) = Corr(X_h - \hat X_h, X_0 - \hat X_0). $ Special case α(1)=ρ(1). MA(q) process is always weakly stationary. Defining θ₀ = 1, the ACVF of MA(q) is γ(h)=σ²Σj = |h|qθjθj − |h|, |h|≤q, 0o.w. the ACF : ρ(h). A stationary process is q correlated if ∀t, s : |t − s|>q. MA(q) is q-correlated