Stationary Signals
A stationary signal is defined by the
stability of its statistical properties over time. Specifically, this implies
that the signal’s mean, variance, and autocorrelation function remain
consistent, regardless of the time in which the signal is evaluated. There are
two main types of stationarity:
·
Strict-sense
stationarity refers to the property of
a signal in which all statistical characteristics remain unchanged over time.
This encompasses all moments of the distribution, indicating that the entire
probability distribution remains constant throughout the observed period.
·
Wide-sense
stationarity requires that the first
two moments, specifically the mean and variance, remain constant over time. The
autocorrelation function is dependent only on the time difference between two
points, not on the exact time points themselves.
Importance in Signal Processing
Stationary signals exhibit statistical properties that remain constant
over time, which allows for more reliable data modeling and forecasting. This
stability significantly influences the choice of analytical methods, as
techniques such as Fourier transforms and spectral analysis are often applied
to stationary data. In practical applications, the design of systems, such as
communication networks and control systems, relies heavily on the
predictability of these signals to ensure optimal performance and accuracy. By
thoroughly understanding the characteristics and behaviors of stationary
signals, we can establish more effective algorithms and frameworks. This, in
turn, enhances the ability to process complex datasets, conduct meaningful
statistical analyses, and derive actionable insights across various domains,
ranging from signal processing to financial modeling.
Examples and Applications
White noise is a classic example of a stationary signal characterized by the
property that every value within its range is equally probable. This randomness
means that the signal does not exhibit any predictable patterns or trends over
time. In essence, each individual sample in a white noise sequence is
independent of the others, leading to a consistent statistical distribution.
Colored noise exhibits distinct statistical properties compared to white noise,
characterized by its varying frequency distributions. However, it can still be
classified as stationary if the probabilities of its events remain consistent
over time. This means that, despite the differences in how colored noise is
structured, its statistical characteristics do not fluctuate, allowing for
predictable patterns in its behavior.
Many real-world signals, such as
electrocardiogram (ECG) and electroencephalogram (EEG) signals, display
non-stationary characteristics. Nevertheless, for analytical purposes, it is
possible to approximate these signals as stationary over brief time intervals.
Unit Root Tests
A unit root test is the standard way to check if a signal is
stationary or if it has a stochastic trend.
If the signal is non-stationary, then we are in presence of a
trend-driven signal. On the contrary, if the signal is stationary, the
variability is mostly cyclical or random around a constant mean.
The most known unit root tests are the Augmented Dickey-Fuller
(ADF) test and the Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test.
The ADF test is used to determine whether a time series has a unit root,
which indicates non-stationarity. In contrast, the KPSS test assesses whether a
time series is stationary around a mean or deterministic trend. These tests
differ in how their hypotheses are formulated (Table 1).
Table 1. Hypothesis Structure of the Augmented Dickey–Fuller and KPSS
Unit Root Tests
|
Unit
Root Test
|
Null
Hypothesis (H0)
|
Alternative
Hypothesis (H1)
|
|
ADF
|
The
series has a unit root (it is non-stationary).
|
The
series does not have a unit root (it is stationary).
|
|
KPSS
|
The
series is stationary.
|
The
series is non-stationary (it has a unit root).
|
Legend of Table 1. In
ADF, a small p‑value implies rejecting the unit‑root null (evidence for
stationarity). In KPSS, a small p‑value implies rejecting the stationarity null
(evidence for non‑stationarity).
Therefore, ADF and KPSS lead to different interpretations. In the ADF
test, if the p-value falls below the chosen significance level, we can reject
the null hypothesis, indicating that the series is considered stationary.
Conversely, in the KPSS test, if the p-value is lower than the significance
level, we reject the null hypothesis, concluding that the series is
non-stationary.
Because the ADF and KPSS tests have complementary roles, it is
recommended to use both when assessing stationarity to enhance the reliability
of the results. For example, it is possible to observe conflicting outcomes,
where the ADF test indicates non-stationarity while the KPSS test suggests
stationarity (Table 2).
Table 2. Joint Interpretation of ADF and KPSS Test Outcomes and
Recommended Actions
|
ADF
|
KPSS
|
Action
|
|
Stationary
|
Stationary
|
Likely stationary.
|
|
Non-stationary
|
Non-stationary
|
Likely non-stationary
signal: consider differencing.
|
|
Non-stationary
|
Stationary
|
1) Re-evaluate
trend/intercept specification in ADF;
2) consider if ADF test is over-rejecting;
3) check bandwidth in KPSS.
|
|
Stationary
|
Non-Stationary
|
1) check lag length
selection in ADF test;
2) consider structural breaks;
3) examine for level shifts;
4) try differencing.
|
Legend of Table 2. The
table provides a diagnostic interpretation of combined ADF and KPSS test
outcomes. Conflicting results should be viewed as indicators of possible model
misspecification, deterministic trends, or structural breaks, rather than as
definitive evidence for or against stationarity.
Mind that Table 2 should be interpreted as a decision and diagnostics
matrix, not as a strict classification rule.
Figure 1. Workflow for Unit Root Testing (ADF & KPSS)
Legend of Figure 1. The figure summarizes the diagnostic workflow used to classify a time series as stationary, trend‑stationary, or unit‑root non‑stationary based on the combined outcomes of the ADF and KPSS tests.
Let’s delve into the case-by-case
interpretation.
ADF: Stationary | KPSS: Stationary
This is the least ambiguous outcome providing consistent evidence of
stationarity: both tests agree that the series is stationary. The data likely
fluctuate around a constant mean or a deterministic trend, depending on model
specification. Small-sample distortions are still possible, but no immediate
corrective action is required. The typical next step is to proceed with
modeling in levels and ensure that the ADF specification (intercept vs. trend)
matches the data-generating process.
ADF: Non‑stationary | KPSS: Non‑stationary
This is the other unambiguous case giving consistent evidence of
non-stationarity. The signal is likely difference-stationary. This outcome
provides strong justification for differencing the signal. In addition, we can
consider higher-order integration (I(2)) or seasonal unit roots may be present.
ADF: Non‑stationary | KPSS: Stationary
This apparent conflict is often misunderstood. Here the ADF fails to
reject a unit root, while the KPSS fails to reject stationarity. However, it
does not mean that the tests are contradicting each other in a “pathological”
sense. Rather, we should consider that the signal may be trend-stationary, but
the ADF test is underpowered or mis-specified. It happens because the ADF has
low power against near-unit root or trend-stationary alternatives. If the
deterministic trend is incorrectly omitted or mis-specified, the ADF may
falsely indicate non-stationarity. The KPSS, with stationarity as the null
hypothesis, may still accept trend stationarity. This outcome suggests that the
stochastic component may be stationary once a deterministic trend is properly
modelled. In addition, structural breaks or level shifts can also cause ADF to
lose power. Therefore, the recommended diagnostic actions include a)
re-estimate the ADF with a trend term (if omitted), or different lag lengths;
b) test for structural breaks (e.g., Perron-type tests); c) visually inspect
the signal for deterministic trends, level shifts, or regime changes. The
bottom line is that this outcome encourages for model refinement, not
differencing by default!
ADF: Stationary | KPSS: Non‑stationary
This scenario describes a less common conflict where the ADF rejects the
unit root and the KPSS rejects stationarity. In this case, the ADF may be
over-rejecting due to inappropriate trend specification, small-sample bias, or the
use of too many lags. Conversely, we need to consider that the KPSS is
sensitive to bandwidth selection and short-run autocorrelation. Overall, the
signal may be close to exhibiting a unit root or could be influenced by
long-memory behaviour. However, the evidence for stationarity in this scenario
is weak. We should reassess the ADF specification (considering an intercept
versus an intercept plus trend), evaluate the robustness of the KPSS test
against various bandwidth choices, and explore alternative tests, such as the
DF-GLS test.
General recommendations for Unit
Root Testing and Interpretation
Unit root tests such as ADF and KPSS should never be applied
mechanically. Their outcomes depend critically on preliminary data exploration,
model specification, and tuning choices. The following recommendations provide
a structured workflow for reliable and interpretable results.
1. Always Start with a Visual Inspection of the Signal
Before conducting any formal statistical test, plot the time series.
A simple time‑series plot can reveal key features that strongly
influence unit root testing:
- Trends (linear, nonlinear, or piecewise),
- Seasonal or cyclical patterns,
- Structural breaks or change‑points,
- Outliers or extreme observations,
- Variance changes over time.
These features are not merely descriptive; they directly inform:
- whether to include an intercept or trend in the test,
- whether the series should be segmented,
- whether a single global model is appropriate at all.
Unit root tests are conditional on their specification. A poor
specification often reflects ignoring information that is clearly visible in
the plot.
2. Lag Length Selection in the ADF Test Is Crucial
The ADF test accounts for serial correlation by including lagged
differences of the series. The number of lags matters greatly:
- Too few lags → residual autocorrelation → size distortions (false rejections or
failures to reject).
- Too many lags → loss of power → failure to detect stationarity when it exists.
Because of this sensitivity, lag length should not be chosen
arbitrarily.
Recommended practice
- Use information criteria such as:
- AIC
(more permissive, favors longer lag structures),
- BIC
(more conservative, favors parsimony).
- Compare results across reasonable lag choices as a robustness
check.
In tutorial terms: ADF is not “one test,” but a family of tests indexed
by lag length.
3. Carefully Choose Whether to Include an Intercept and/or Trend
Both ADF and KPSS tests allow different deterministic components:
- No intercept, no trend (rarely appropriate),
- Intercept only (stationarity around a constant mean),
- Intercept + linear trend (trend stationarity).
The correct specification depends on the data‑generating process,
not on convenience.
Guiding principles
- If the series fluctuates around a nonzero level → include an
intercept.
- If there is a visible long‑term trend → include a trend.
- If unsure, test multiple specifications and interpret results
jointly.
For KPSS:
- One version tests level stationarity,
- Another tests trend stationarity.
Mismatch between the true deterministic structure and the test
specification is one of the most common causes of misleading unit root results.
4. Sensitivity of KPSS to Level Shifts and Structural Breaks
The KPSS test is particularly sensitive to level shifts, i.e.,
sudden and permanent changes in the mean of the series.
This has an important consequence:
- Even if a series is otherwise stable and mean‑reverting,
- A single structural break can cause KPSS to reject stationarity.
By contrast, the ADF test is often less sensitive to such shifts and may
fail to detect them.
Implication
- A KPSS rejection does not necessarily imply a unit root.
- It may instead reflect unmodeled structural change.
Recommended diagnostic step
- Explicitly test for breaks using procedures such as:
- Bai–Perron multiple break tests,
- Change‑point detection methods.
For teaching purposes: KPSS often answers the question
“Is the series stationary under a stable regime?”
5. Bandwidth Selection in the KPSS Test Matters
The KPSS test relies on a long‑run variance estimator, which
requires choosing a bandwidth parameter.
- Small bandwidth → underestimation of long‑run variance → over‑rejection of
stationarity.
- Large bandwidth → loss of sensitivity → failure to detect non‑stationarity.
Because there is no universally optimal choice, results should be
checked for robustness across reasonable bandwidth values.
In practice, KPSS outcomes can change noticeably with bandwidth choice,
especially in finite samples.
6. Deterministic vs. Stochastic Trends in the ADF Framework
One of the conceptual strengths of the ADF framework is that it allows
us to distinguish between:
- Deterministic trends
– predictable functions of time (e.g., linear trend), – removable by
detrending.
- Stochastic trends (unit roots)
– intrinsic to the process, – shocks have permanent effects.
However, this distinction depends entirely on model specification.
Important caveat
- Including a deterministic trend in the ADF regression can
sometimes:
- absorb low‑frequency stochastic behavior,
- make a truly unit‑root process appear stationary.
This is especially problematic in:
- small samples,
- near‑unit‑root processes,
- series with structural breaks.
Hence, rejecting a unit root in a trend‑augmented ADF test does not
automatically imply that the trend is purely deterministic.
7. Key Pedagogical Takeaway
Unit root tests are conditional diagnostics, not automatic truth
detectors.
Reliable interpretation requires:
- visual inspection,
- careful specification,
- sensitivity analysis,
- and economic or scientific context.
For tutorial purposes, it is helpful to emphasize that:
- conflicting test outcomes are informative,
- model refinement is part of the analysis,
- and differencing should be a considered decision, not a reflex.
What if the signal is
non-stationary
If a time series is found to be non‑stationary, this indicates
that its statistical properties, such as the mean, variance, or autocovariance,
change over time. In practice, non‑stationarity usually arises from the
presence of a trend, structural changes, or a unit root.
Implications of Non‑Stationarity
A non‑stationary signal often reflects one or more of the following
features:
- A deterministic trend, where the series evolves smoothly
over time according to a fixed functional form.
- A stochastic trend (unit root process), where shocks
accumulate and have permanent effects.
- Structural breaks or level shifts, where the underlying data‑generating process
changes at specific points in time.
When unit root tests (e.g. ADF and KPSS jointly) indicate non‑stationarity,
it is no longer appropriate to assume that deviations from the mean are short‑lived
or that long‑run moments are constant.
Consequences of a Unit Root
The presence of a unit root has several important implications:
- The series follows a random‑walk–like behavior, possibly
with drift.
- Shocks have permanent effects: disturbances do not decay over time but
permanently alter the level of the series.
- Variance increases over time, meaning that uncertainty grows with the
forecast horizon.
- Standard inference procedures that assume stationarity may become
invalid.
In this context, non‑stationarity is not merely a technical issue but it
fundamentally changes how we interpret persistence and long‑run behavior in the
data.
Trend Modeling as a Natural Response
Evidence of non‑stationarity justifies the use of explicit trend
modeling, especially when the objective is to separate long‑term evolution
from shorter‑term dynamics.
Common approaches include:
- Parametric trends (linear or polynomial),
- Non‑parametric or semi‑parametric trends, such as:
- LOESS smoothing,
- Generalized Additive Models (GAMs),
- Penalized splines.
These methods allow the analyst to capture smooth but flexible long‑run
movements without forcing the data into a rigid functional form.
Differencing vs. Detrending
If the goal is to analyze cycles or short‑run fluctuations,
additional transformations may be required:
- Differencing removes stochastic trends and is appropriate when the series is
difference‑stationary.
- Detrending removes deterministic trends while preserving low‑frequency
dynamics.
The choice between differencing and detrending is crucial:
- Over‑differencing may eliminate meaningful low‑frequency
information.
- Inappropriate detrending may leave residual non‑stationarity.
Hence, transformation should be guided by both statistical tests and
substantive knowledge of the data‑generating process.
Interpreting Cycles in Non‑Stationary Series
Importantly, non‑stationarity does not imply that cycles are
meaningless or ineffective. Instead, it alters how cycles should be
interpreted:
- In stationary series, cycles represent temporary deviations around
a stable equilibrium.
- In unit‑root processes, cycles are superimposed on a drifting
baseline, and their amplitudes or durations may change over time.
Thus, cyclical components in non‑stationary signals should be
interpreted relative to the evolving trend, not as oscillations around a
fixed mean.
Conclusions
The presence of a unit root does not invalidate cyclical analysis, but
it requires a shift in interpretation:
cycles occur around a stochastic trend, shocks are permanent, and
uncertainty accumulates over time.
For tutorial purposes, this perspective reinforces a central lesson:
- Unit root tests are diagnostic tools, not automatic transformation rules.
- Non‑stationarity motivates thoughtful modeling choices—trend
specification, structural break analysis, and transformation—rather than
mechanical differencing.
Appendix: Checklist for
Unit Root Testing and Interpretation (ADF & KPSS)
This checklist that turns the recommendations above described into a practical,
step‑by‑step workflow. It is designed to be used before, during, and
after running ADF and KPSS tests, and works well as a handout or methods
box.
✅ Checklist for Unit Root Testing and Interpretation
(ADF & KPSS)
Step 1 — Preliminary Data Inspection
☐ Plot the time series.
☐ Look for:
- overall trends (linear or nonlinear),
- cyclical or seasonal patterns,
- sudden level shifts or change‑points,
- outliers or volatility changes.
☐ Ask: Does a single global model seem appropriate, or are there
regime changes?
Step 2 — Decide on Deterministic Components
☐ Based on the plot, decide whether to include:
- ☐ an intercept (non‑zero mean),
- ☐ a deterministic trend (visible long‑run
movement).
☐ Ensure consistency between:
- ADF specification (none / intercept / intercept + trend),
- KPSS version (level stationarity vs. trend stationarity).
Step 3 — Select Lag Length for the ADF Test
☐ Do not fix lag length arbitrarily.
☐ Use
information criteria:
- ☐ AIC (more flexible),
- ☐ BIC (more parsimonious).
☐ Check for residual autocorrelation after estimation.
☐ Assess robustness across nearby lag choices.
Step 4 — Run ADF and KPSS Tests Jointly
☐ Run ADF test under the chosen specification.
☐ Run KPSS test
with matching assumptions (level or trend).
☐ Record:
- test statistics,
- p‑values,
- lag length (ADF),
- bandwidth choice (KPSS).
Step 5 — Interpret Joint Outcomes
☐ ADF stationary & KPSS stationary
→ Likely stationary process.
☐ ADF non‑stationary & KPSS non‑stationary
→ Likely unit root; consider differencing.
☐ ADF non‑stationary & KPSS stationary
→ Possible trend stationarity or low ADF power; refine model.
☐ ADF stationary & KPSS non‑stationary
→ Potential over‑rejection or long‑memory; reassess assumptions.
Step 6 — Check for Structural Breaks
☐ If KPSS rejects stationarity unexpectedly, ask:
- Are there level shifts or regime changes?
☐ Apply structural break tests (e.g., Bai–Perron).
☐ Consider segmenting the series or modeling breaks explicitly.
Step 7 — Tune KPSS Bandwidth
☐ Review bandwidth selection method.
☐ Test
sensitivity to alternative bandwidth choices.
☐ Be cautious of
over‑rejection caused by small bandwidths.
Step 8 — Reassess Trend Type
☐ Ask: Is the trend likely:
- ☐ deterministic (removable by detrending), or
- ☐ stochastic (unit root; shocks are permanent)?
☐ Remember:
- Including a deterministic trend in ADF can mask a stochastic trend.
- Small samples and near‑unit‑root processes require caution.
Step 9 — Decide on Transformation
☐ If unit root is supported → consider differencing.
☐ If trend‑stationary
→ consider detrending or flexible trend modeling (LOESS, GAM).
☐ Avoid
mechanical differencing without diagnostic justification.
Step 10 — Interpret Cycles Appropriately
☐ If stationary → cycles are deviations around a stable
equilibrium.
☐ If non‑stationary
→ cycles evolve around a drifting baseline.
☐ Interpret
persistence and shocks accordingly.
Final comments
☐ Unit root tests are diagnostic tools, not
automatic decision rules.
☐ Conflicting
results signal model refinement, not failure.
☐ Visualization,
specification, and robustness checks are essential.
Comments
Post a Comment