1 INTRODUCTION 1
1.1 Four Important Practical Problems 2
1.1.1 Forecasting Time Series, 2
1.1.2 Estimation of Transfer Functions, 3
1.1.3 Analysis of Effects of Unusual Intervention Events To a System, 4
1.1.4 Discrete Control Systems, 5
1.2 Stochastic and Deterministic Dynamic Mathematical
Models 7
1.2.1 Stationary and Nonstationary Stochastic Models
for Forecasting and Control, 7
1.2.2 Transfer Function Models, 12
1.2.3 Models for Discrete Control Systems, 14
1.3 Basic Ideas in Model Building 16
1.3.1 Parsimony, 16
1.3.2 Iterative Stages in the Selection of a Model, 16
Part I Stochastic Models and Their Forecasting 19
2 AUTOCORRELATION FUNCTION AND SPECTRUM OF
STATIONARY PROCESSES 21
2.1 Autocorrelation Properties of Stationary Models 21
2.1.1 Time Series and Stochastic Processes, 21
2.1.2 Stationary Stochastic Processes, 23
2.1.3 Positive Definiteness and the Autocovariance Matrix, 26
2.1.4 Autocovariance and Autocorrelation Functions, 29
2.1.5 Estimation of Autocovariance and Autocorrelation Functions, 30
2.1.6 Standard Error of Autocorrelation Estimates, 32
2.2 Spectral Properties of Stationary Models 35
2.2.1 Periodogram of a Time Series, 35
2.2.2 Analysis of Variance, 36
2.2.3 Spectrum and Spectral Density Function, 37
2.2.4 Simple Examples of Autocorrelation and Spectral Density Functions, 41
2.2.5 Advantages and Disadvantages of the
Autocorrelation and Spectral Density Functions, 43
A2.1 Link Between the Sample Spectrum and
Autocovariance Function Estimate 44
3 LINEAR STATIONARY MODELS 46
3.1 General Linear Process 46
3.1.1 Two Equivalent Forms for the Linear Process, 46
3.1.2 Autocovariance Generating Function of a Linear Process, 49
3.1.3 Stationarity and Invertibility Conditions for a Linear Process, 50
3.1.4 Autoregressive and Moving Average Processes, 52
3.2 Autoregressive Processes 54
3.2.1 Stationarity Conditions for Autoregressive Processes, 54
3.2.2 Autocorrelation Function and Spectrum of Autoregressiue Processes, 55
3.2.3 First-Order Autoregressive (Markov) Process, 58
3.2.4 Second-Order Autoregressive Process, 60
3.2.5 Partial Autocorrelation Function, 64
3.2.6 Estimation of the Partial Autocorrelation Function, 67
3.2.7 Standard Errors of Partial Autocorrelation Estimates, 68
3.3 Moving Average Processes 69
3.3.1 Invertibility Conditions for Moving Average Processes, 69
3.3.2 Autocorrelation Function and Spectrum of Moving Average Processes, 70
3.3.3 First-Order Moving Average Process, 72
3.3.4 Second-Order Moving Average Process, 73
3.3.5 Duality Between Autoregressive and Moving Average Processes, 75
3.4 Mixed Autoregressive-Moving Average Processes 77
3.4.1 Stationarity and Invertibility Properties, 77
3.4.2 Autocorrelation Function and Spectrum of Mixed Processes, 78
3.4.3 First-Order Autoregressive-First-Order Moving Average Process, 80
3.4.4 Summary, 83
A3.1 Autocovariances, Autocovariance Generating
Function, and Stationarity Conditions for a
General Linear Process 85
A3.2 Recursive Method for Calculating Estimates of
Autoregressive Parameters 87
4 LINEAR NONSTATIONARY MODELS 89
4.1 Autoregressive Integrated Moving Average Processes 89
4.1.1 Nonstationary First-Order Autoregressive Process, 89
4.1.2 General Model for a Nonstationary Process Exhibiting Homogeneity, 92
4.1.3 General Form of the Autoregressive Integrated Moving Average Process, 96
4.2 Three Explicit Forms for the Autoregressive Integrated Moving Average Model 99
4.2.1 Difference Equation Form of the Model, 99
4.2.2 Random Shock Form of the Model, I00
4.2.3 Inverted Form of the Model, 106
4.3 Integrated Moving Average Processes 109
4.3.1 Integrated Moving Average Process of Order (0, 1, 1), 110
4.3.2 Integrated Moving Average Process of Order (0, 2, 2), 114
4.3.3 General Integrated Moving Average Process of Order (0, d, q), 118
A4.1 Linear Difference Equations 120
A4.2 IMA(0, 1, 1) Process With Deterministic Drift 125
A4.3 ARIMA Processes With Added Noise 126
A4.3.1 Sum of Two Independent Moving Average Processes, 126
A4.3.2 Effect of Added Noise on the General Model, 127
A4.3.3 Example for an IMA(O, 1, 1) Process with Added White Noise, 128
A4.3.4 Relation Between the IMA(O, 1, 1) Process and a Random Walk, 129
A4.3.5 Autocovariance Function of the General Model
with Added Correlated Noise, 129
5 FORECASTING 131
5. ! Minimum Mean Square Error Forecasts and Their Properties 131
5.1.1 Derivation of the Minimum Mean Square Error Forecasts, 133
5.1.2 Three Basic Forms for the Forecast, 135
5.2 Calculating and Updating Forecasts 139
5.2.1 Convenient Format for the Forecasts, 139
5.2.2 Calculation of the ¦· Weights, 139
5.2.3 Use of the ¦· Weights in Updating the Forecasts, 141
5.2.4 Calculation of the Probability Limits of the Forecasts at Any Lead Time, 142
5.3 Forecast Function and Forecast Weights 145
5.3.1 Eventual Forecast Function Determined by the Autoregressive Operator, 146
5.3.2 Role of the Mooing Average Operator in Fixing the Initial Values, 147
5.3.3 Lead l Forecast Weights, 148
5.4 Examples of Forecast Functions and Their Updating 151
5.4.1 Forecasting an IMA(O, 1, 1) Process, 151
5.4.2 Forecasting an IMA(O, 2, 2) Process, 154
5.4.3 Forecasting a General IMA(O, d, q) Process, 156
5.4.4 Forecasting Autoregressive Processes, 157
5.4.5 Forecasting a (1, O, 1) Process, 160
5.4.6 Forecasting a (1, 1, 1) Process, 162
5.5 Use of State Space Model Formulation for Exact Forecasting 163
5.5.1 State Space Model Representation for the ARIMA Process, 163
5.5.2 Kalman Filtering Relations for Use in Prediction, 164
5.6 Summary 166
A5.1 Correlations Between Forecast Errors 169
A5.1.1 Autocorrelation Function of Forecast Errors at
Different Origins, 169
A5.1.2 Correlation Between Forecast Errors at the
Same Origin with Different Lead Times, 170
A5.2 Forecast Weights for Any Lead Time 172
A5.3 Forecasting in Terms of the General Integrated
Form 174
A5.3.1 General Method of Obtaining the Integrated
Form, 174
A5.3.2 Updating the General Integrated Form, 176
A5.3.3 Comparison with the Discounted Least Squares
Method, 176
Part II Stochastic Model Building 181
6 MODELDENTIFICATION 183
6. l Objectives of Identification 183
6.1.1 Stages in the Identification Procedure, 184
6.2 Identification Techniques 184
6.2.1 Use of the Autocorrelation and Partial
Autocorrelation Functions in Identification, 184
6.2.2 Standard Errors for Estimated Autocorrelations
and Partial Autocorrelations, 188
6.2.3 Identification of Some Actual Time Series, 188
6.2.4 Some Additional Model Identification Tools, 197
6.3 Initial Estimates for the Parameters 202
6.3.1 Uniqueness of Estimates Obtained from the
Autocovariance Function, 202
6.3.2 Initial Estimates for Moving Average Processes, 202
6.3.3 Initial Estimates for Autoregressive Processes, 204
6.3.4 Initial Estimates for Mixed Autoregressive-Moving
Average Processes, 206
6.3.5 Choice Between Stationary and Nonstationary
Models in Doubtful Cases, 207
6.3.6 More Formal Tests for Unit Roots in ARIMA
Models, 208
6.3.7 Initial Estimate of Residual Variance, 211
6.3.8 Approximate Standard Error for , 212
6.4 Model Multiplicity 214
6.4.1 Multiplicity of Autoregressive-Moving Average
Models, 214
6.4.2 Multiple Moment Solutions for Moving Average
Parameters, 216
6.4.3 Use of the Backward Process to Determine
Starting Values, 218
A6.1 Expected Behavior of the Estimated Autocorrelation
Function for a Nonstationary Process 218
A6.2 General Method for Obtaining Initial Estimates of
the Parameters of a Mixed Autoregressive-Moving
Average Process 220
7MODELESTIMATION 224
7. l Study of the Likelihood and Sum of Squares
Functions 224
7.1.1 Likelihood Function, 224
7.1.2 Conditional Likelihood for an ARIMA Process, 226
7.1.3 Choice of Starting Values for Conditional Calculation, 227
7.1.4 Unconditional Likelihood; Sum of Squares
Function; Least Squares Estimates, 228
7.1.5 General Procedure for Calculating the Unconditional Sum of Squares, 233
7.1.6 Graphical Study of the Sum of Squares Function, 238
7.1.7 Description of "Well-Behaved" Estimation
Situations; Confidence Regions, 241
7.2 Nonlinear Estimation 248
7.2.1 General Method of Approach, 248
7.2.2 Numerical Estimates of the Derivatives, 249
7.2.3 Direct Evaluation of the Derivatives, 251
7.2.4 General Least Squares Algorithm for the Conditional Model, 252
7.2.5 Summary of Models Fitted to Series A to F, 255
7.2.6 Large-Sample Information Matrices and Covariance Estimates, 256
7.3 Some Estimation Results for Specific Models 259
7.3.1 Autoregressive Processes, 260
7.3.2 Moving Average Processes, 262
7.3.3 Mixed Processes, 262
7.3.4 Separation of Linear and Nonlinear Components in Estimation, 263
7.3.5 Parameter Redundancy, 264
7.4 Estimation Using Bayes' Theorem 267
7.4.1 Bayes' Theorem, 267
7.4.2 Bayesian Estimation of Parameters, 269
7.4.3 Autoregressive Processes, 270
7.4.4 Moving Average Processes, 272
7.4.5 Mixed processes, 274
7.5 Likelihood Function Based on The State Space Model 275
A7.1 Review of Normal Distribution Theory 279
A7.1.1 Partitioning of a Positive-Definite Quadratic Form, 279
A7.1.2 Two Useful Integrals, 280
A7.1.3 Normal Distribution, 281
A7.1.4 Student's t-Distribution, 283
A7.2 Review of Linear Least Squares Theory 286
A7.2.1 Normal Equations, 286
A7.2.2 Estimation of Residual Variance, 287
A7.2.3 Covariance Matrix of Estimates, 288
A7.2.4 Confidence Regions, 288
A7.2.5 Correlated Errors, 288
A7.3 Exact Likelihood Function for Moving Average and Mixed Processes 289
A7.4 Exact Likelihood Function for an Autoregressive Process 296
A7.5 Examples of the Effect of Parameter Estimation
Errors on Probability Limits for Forecasts 304
A7.6 Special Note on Estimation of Moving Average Parameters 307
8 MODEL DIAGNOSTIC CHECKING 308
8.1 Checking the Stochastic Model 308
8.1.1 General Philosophy, 308
8.1.2 Overfitting, 309
8.2 Diagnostic Checks Applied to Residuals 312
8.2.1 Autocorrelation Check, 312
8.2.2 Portmanteau Lack-of-Fit Test, 314
8.2.3 Model Inadequacy Arising from Changes in Parameter Values, 317
8.2.4 Score Tests for Model Checking, 318
8.2.5 Cumulative Periodogram Check, 321
8.3 Use of Residuals to Modify the Model 324
8.3.1 Nature of the Correlations in the Residuals When
an Incorrect Model Is Used, 324
8.3.2 Use of Residuals to Modify the Model, 325
9SEASONALMODELS 327
9.! Parsimonious Models for Seasonal Time Series 327
9.1.1 Fitting versus Forecasting, 328
9.1.2 Seasonal Models Involving Adaptive Sines and Cosines, 329
9.1.3 General Multiplicative Seasonal Model, 330
9.2 Representation of the Airline Data by a Multiplicative
(0, 1, 1) ~ (0, 1, 1)12 Seasonal Model 333
9.2.1 Multiplicative (0, l, l) ~ (0, l, 1)12 Model, 333
9.2.2 Forecasting, 334
9.2.3 Identification, 341
9.2.4 Estimation, 344
9.2.5 Diagnostic Checking, 349
9.3 Some Aspects of More General Seasonal Models 351
9.3.1 Multiplicative and Nonmultiplicative Models, 351
9.3.2 Identification, 353
9.3.3 Estimation, 355
9.3.4 Eventual Forecast Functions for Various Seasonal Models, 355
9.3.5 Choice of Transformation, 358
9.4 Structural Component Models and Deterministic
Seasonal Components 359
9.4.1 Deterministic Seasonal and Trend Components and Common Factors, 360
9.4.2 Models with Regression Terms and Time Series Error Terms, 361
A9.1 Autocovariances for Some Seasonal Models 366
Part III Transfer Function Model Building 371
10 TRANSFER FUNCTION MODELS 373
10.1 Linear Transfer Function Models 373
10.1.1 Discrete Transfer Function, 374
10.1.2 Continuous Dynamic Models Represented by Differential Equations, 376
10.2 Discrete Dynamic Models Represented by Difference Equations 381
10.2.1 General Form of the Difference Equation, 381
10.2.2 Nature of the Transfer Function, 383
10.2.3 First- and Second-Order Discrete Transfer Function Models, 384
10.2.4 Recursive Computation of Output for Any Input, 390
10.2.5 Transfer Function Models with Added Noise, 392
10.3 Relation Between Discrete and Continuous Models 392
10.3.1 Response to a Pulsed Input, 393
10.3.2 Relationships for First- and Second-Order Coincident Systems, 395
10.3.3 Approximating General Continuous Models by Discrete Models, 398
A10.1 Continuous Models With Pulsed Inputs 399
A10.2 Nonlinear Transfer Functions and Linearization 404
11 IDENTIFICATION, FITTING, AND CHECKING OF
TRANSFER FUNCTION MODELS 407
ll.1 Cross Correlation Function 408
11.1.1 Properties of the Cross Covariance and Cross Correlation Functions. 408
11.1.2 Estimation of the Cross Covariance and Cross Correlation Functions, 411
11.1.3 Approximate Standard Errors of Cross Correlation Estimates, 413
11.2 Identification of Transfer Function Models 415
11.2.1 Identification of Transfer Function Models by Prewhitening the Input, 417
11.2.2 Example of the Identification of a Transfer Function Model, 419
11.2.3 Identification of the Noise Model, 422
11.2.4 Some General Considerations in Identifying Transfer Function Models, 424
11.3 Fitting and Checking Transfer Function Models 426
11.3.1 Conditional Sum of Squares Function, 426
11.3.2 Nonlinear Estimation, 429
11.3.3 Use of Residuals for Diagnostic Checking, 431
11.3.4 Specific Checks Applied to the Residuals, 432
11.4 Some Examples of Fitting and Checking Transfer Function Models 435
11.4.1 Fitting and Checking of the Gas Furnace Model, 435
11.4.2 Simulated Example with Two Inputs, 441
11.5 Forecasting Using Leading Indicators 444
11.5.1 Minimum Mean Square Error Forecast, 444
11.5.2 Forecast of C02 Output from Gas Furnace, 448
11.5.3 Forecast of Nonstationary Sales Data Using a Leading Indicator, 451
11.6 Some Aspects of the Design of Experiments to Estimate Transfer Functions 453
A11.1 Use of Cross Spectral Analysis for Transfer
Function Model Identification 455
All.I.1 Identification of Single Input Transfer Function Models, 455
All.l.2 Identification of Multiple Input Transfer Function Models, 456
AI1.2 Choice of Input to Provide Optimal Parameter Estimates 457
All.2.1 Design of Optimal Inputs for a Simple System, 457
All.2.2 Numerical Example, 460
12 INTERVENTION ANALYSIS MODELS AND OUTLIER
DETECTION 462
12.1 Intervention Analysis Methods 462
12.1.1 Models for Intervention Analysis, 462
12.1.2 Example of Intervention Analysis, 465
12.1.3 Nature of the MLE for a Simple Level Change Parameter Model, 466
12.2 Outlier Analysis for Time Series 469
12.2.1 Models for Additive and Innovational Outliers, 469
12.2.2 Estir m ation of Outlier Effect for Known Timing of the Outlier, 470
12.2.3 Iterative Procedure for Outlier Detection, 471
12.2.4 Examples of Analysis of Outliers, 473
12.3 Estimation for ARMA Models With Missing Values 474
Part IV Design of Discrete Control Schemes 481
13 ASPECTS OF PROCESS CONTROL 483
13.1 Process Monitoring and Process Adjustment 484
13.1.1 Process Monitoring, 484
13.1.2 Process Adjustment, 487
13.2 Process Adjustment Using Feedback Control~488
13.2.1 Feedback Adjustment Chart, 489
13.2.2 Modeling the Feedback Loop, 492
13.2.3 Simple Models for Disturbances and Dynamics, 493
13.2.4 General Minimum Mean Square Error Feedback Control Schemes, 497
13.2.5 Manual Adjustment for Discrete Proportional-Integral Schemes, 499
13.2.6 Complementary Roles of Monitoring and Adjustment, 503
13.3 Excessive Adjustment Sometimes Required by MMSE Control 505
13.3.1 Constrained Control, 506
13.4 Minimum Cost Control With Fixed Costs of Adjustment And Monitoring 508
13.4.1 Bounded Adjustment Scheme for Fixed Adjustment Cost, 508
13.4.2 Indirect Approach for Obtaining a Bounded Adjustment Scheme, 510
13.4.3 Inclusion of the Cost of Monitoring, 511
13.5 Monitoring Values of Parameters of Forecasting
and Feedback Adjustment Schemes 514
A13.1 Feedback Control Schemes Where the
Adjustment Variance Is Restricted 516
A13.1.1 Derivation of Optimal Adjustment, 517
A13.2 Choice of the Sampling Interval 526
A13.2.1 Illustration of the Effect of Reducing Sampling Frequency, 526
A13.2.2 Sampling an IMA(O, I, I) Process, 526
Part V Charts and Tables 531
COLLECTION OF TABLES AND CHARTS 533
COLLECTION OF TIME SERIES USED FOR EXAMPLES IN
THE TEXT AND IN EXERCISES 540
REFERENCES 556
Part VI
EXERCISES AND PROBLEMS 569
INDEX 589
鄂公网安备 42010302001612号 