All drawings appearing in this Recommendation have been done in Autocad. Recommendation E.506 FORECASTING INTERNATIONAL TRAFFIC1) 1 Introduction This Recommendation is the first in a series of three Recommendations that cover international telecommunications forecasting. In the operation and administration of the international telephone network, proper and successful development depends to a large degree upon estimates for the future. Accordingly, for the planning of equipment and circuit provision and of telephone plant investments, it is necessary that Administrations forecast the traffic which the network will carry. In view of the heavy capital investments in the international network, the economic importance of the most reliable forecast is evident. The purpose of this Recommendation is to give guidance on some of the prerequisites for forecasting international telecommunications traffic. Base data, not only traffic and call data but also economic, social and demographic data, are of vital importance for forecasting. These data series may be incomplete; strategies are recommended for dealing with missing data. Different forecasting approaches are presented including direct and composite methods, matrix forecasting, and top down and bottom up procedures. Recommendation E.507 provides guidelines for building forecasting models and contains an overview of various forecasting techniques. Recommendation E.508 covers the forecasting of new international telecommunications services. 2 Base data for forecasting An output of the international traffic forecasting process is the estimated number of circuits required for each period in the forecast horizon. To obtain these values, traffic engineering techniques are applied to forecast Erlangs, a measure of traffic. Figure 1/E.506 outlines two different approaches for determining forecasted Erlangs. The two different strategies for forecasting are the direct strategy and the composite strategy. The first step in either process is to collect raw data. These raw data, perhaps adjusted, will be the base data used to generate the traffic forecasts. Base data may be hourly, daily, monthly, quarterly, or annual. Most Administrations use monthly accounting data for forecasting purposes. With the direct strategy, the traffic carried in Erlangs, or measured usage, for each relation would be regarded as the base data in forecasting traffic growth. These data may be adjusted to account for such occurrences as regeneration (see Recommendation E.500). Figure 1/E.506 - T0200800-87 In both strategies (direct and composite) it is necessary to convert the carried traffic into offered traffic Erlangs. The conversion formula can be found in Recommendation E.501 for the direct strategy and in this Recommendation for the composite strategy. Composite forecasting uses historical international accounting data of monthly paid minute traffic as the base data. The data may be adjusted by a number of factors, either before or after the forecasting process, that are used for converting paid minutes on the basis of the accounting data into busy-hour Erlang forecasts. As seen in Figure 1/E.506, the forecasting process is common to both the direct and composite strategy. However, the actual methods or models used in the process vary. Forecasts can be generated, for example, using traffic matrix methods (see S 4), econometric models or autoregressive models (see S 3, Recommendation E.507). There are various other data that are input to the forecasting process. Examples of these are explanatory variables, market segmentation information and price elasticities. Wherever possible, both the direct and composite forecasting strategies should be used and compared. This comparison may reveal irregularities not evident from the use of only one method. Where these are significant, in particular in the case of the busy hour, the causes for the differences should be identified before the resulting forecast is adopted. 1) The old Recommendation E.506 which appeared in the Red Book was split into two Recommendations, revised E.506 and new E.507 and considerable new material was added to both. Fascicle II.3 - Rec. E.506 PAGE1 In econometric modelling especially, explanatory variables are used to forecast international traffic. Some of the most important of these variables are the following: - exports, - imports, - degree of automation, - quality of service, - time differences between countries, - tariffs, - consumer price index, and - gross national product. Other explanatory variables, such as foreign business travellers and nationals living in other countries, may also be important to consider. It is recommended that data bases for explanatory variables should be as comprehensive as possible to provide more information to the forecasting process. Forecasts may be based on market segmentation. Base data may be segmented, for example, along regional lines, by business, non-business, or by type of service. Price elasticities should also be examined, if possible, to quantify the impact of tariffs on the forecasting data. 3 Composite strategy - Conversion method The monthly paid-minutes traffic is converted to busy-hour Erlangs for dimensioning purposes by the application of a number of traffic related conversion factors for each service category. The conversion is carried out in accordance with the formula: A = Mdh/60e (3-1) where A is the estimated mean traffic in the busy hour, M is the monthly paid-minutes, d is day-to-month ratio, h is the busy hour-to-day ratio, and e is the efficiency factor. The formula is described in detail in Annex A. 4 Procedures for traffic matrix forecasting 4.1 Introduction To use traffic matrix or point-to-point forecasts the following procedures may be used: - Direct point-to-point forecasts, - Kruithof's method, - Extension of Kruithof's method, - Weighted least squares method. It is also possible to develop a Kalman Filter model for point-to-point traffic taking into account the aggregated forecasts. Tu and Pack describe such a model in [16]. The forecasting procedures can be used to produce forecasts of internal traffic within groups of countries, for example, the Nordic countries. Another application is to produce forecasts for national traffic on various levels. 4.2 Direct point-to-point forecasts It is possible to produce better forecasts for accumulated traffic than forecast of traffic on a lower level. Hence, forecasts of outgoing traffic (row sum) or incoming traffic (column sum) between one country and a group of countries will give a relatively higher precision than the separate forecasts between countries. In this situation it is possible to adjust the individual forecasts by taking into account the aggregated forecasts. On the other hand, if the forecasts of the different elements in the traffic matrix turn out to be as good as the accumulated forecasts, then it is not necessary to adjust the forecasts. Evaluation of the relative precision of forecasts may be carried out by comparing the ratios s¢(X)/X where X is the forecast and eq \o(\s\up4(^),s)(X) the estimated forecast error. 4.3 Kruithof's method Kruithof's method [11] is well known. The method uses the last known traffic matrix and forecasts of the row and column sum to make forecasts of the traffic matrix. This is carried out by an efficient iteration procedure. Kruithof's method does not take into account the change over time in the PAGE4 Fascicle II.3 - Rec. E.506 point-to-point traffic. Because Kruithof's method only uses the last known traffic matrix, information on the previous traffic matrices does not contribute to the forecasts. This would be disadvantageous. Especially when the growth of the distinct point-to-point traffic varies. Also when the traffic matrices reflect seasonal data, Kruithof's method may give poor forecasts. 4.4 Extension of Kruithof's method The traditional Kruithof's method is a projection of the traffic based on the last known traffic matrix and forecasts of the row and column sums. It is possible to extend Kruithof's method by taking into account not only forecasts of the row and column but also forecasts of point-to-point traffic. Kruithof's method is then used to adjust the point-to-point traffic forecasts to obtain consistency with the forecasts of row and column sums. The extended Kruithof's method is superior to the traditional Kruithof's method and is therefore recommended. 4.5 Weighted least squares method Weighted least squares method is again an extension of the last method. Let {Cij}, {Ci.} and {C.j} be forecasts of point-to-point traffic, row sums and column sums respectively. The extended Kruithof's method assumes that the row and column sums are "true" and adjust {Cij} to obtain consistency. The weighted least squares method [2] is based on the assumption that both the point-to-point forecasts and the row and column sum forecasts are uncertain. A reasonable way to solve the problem is to give the various forecasts different weights. Let the weighted least squares forecasts be {Dij}. The square sum Q is defined by: Q = eq \i\su(ij, ,aij)(Cij - Dij)2+eq \i\su(i, , ) bi(Ci. - Di.)2+eq \i\su(j, , ) cj(C.j - D.j)2 (4-1) where {aij}, {bi }, {cj} are chosen constants or weights. The weighted least squares forecast is found by: MinQ(Dij) Dij Fascicle II.3 - Rec. E.506 PAGE1 subject to Di. = eq \i\su(j, , ) Dij i = 1, 2, . . . (4-2) and D.j = eq \i\su(i, , ) Dij j = 1, 2, . . . A natural choice of weights is the inverse of the variance of the forecasts. One way to find an estimate of the standard deviation of the forecasts is to perform ex-post forecasting and then calculate the root mean square error. The properties of this method are analyzed in [14]. 5 Top down and bottom up methods 5.1 Choice of model The object is to produce forecasts for the traffic between countries. For this to be a sensible procedure, it is necessary that the traffic between the countries should not be too small, so that the forecasts may be accurate. A method of this type is usually denoted as "bottom up". Alternatively, when there is a small amount of traffic between the countries in question, it is better to start out with forecasting the traffic for a larger group of countries. These forecasts are often used as a basis for forecasts for the traffic to each country. This is done by a correction procedure to be described in more detail below. Methods of this type are called "top down". The following comments concern the preference of one method over another. Let sT2 be the variance of the aggregated forecast, and si2 be the variance of the local forecast No. i and gij be the covariance of the local forecast No. i and j. If the following inequality is true: eq \o(\s\up4(^),s)\s(2,T) < eq \i\su(i, , ) eq \o(\s\up 4(^),s)\s(2,i) + eq \i\su(i ¹, , )\I\su( j, , )gij (5-1) then, in general, it is not recommended to use the bottom up method, but to use the top down method. In many situations it is possible to use a more advanced forecasting model on the aggregated level. Also, the data on an aggregated level may be more consistent and less influenced by stochastic changes compared to data on a lower level. Hence, in most cases the inequality stated above will be satisfied for small countries. 5.2 Bottom up method As outlined in S 5.1 the bottom up method is defined as a procedure for making separate forecasts of the traffic between different countries directly. If the inequality given in S 5.1 is not satisfied, which may be the case for large countries, it is sufficient to use the bottom up method. Hence, one of the forecasting models mentioned in Recommendation E.507 can be used to produce traffic forecasts for different countries. 5.3 Top down procedure In most cases the top down procedure is recommended for producing forecasts of international traffic for a small country. In Annex D a detailed example of such a forecasting procedure is given. The first step in the procedure is to find a forecasting model on the aggregated level, which may be a rather sophisticated model. Let XT be the traffic forecasts on the aggregated level and sT the estimated standard deviation of the forecasts. The next step is to develop separate forecasting models of traffic to different countries. Let Xi be the traffic forecast to the ith country and s¢i the standard deviation. Now, the separate forecasts [Xi] have to be corrected by taking into account the aggregated forecasts XT. We know that in general XT ¹ eq \i\su(i, , ) Xi (5-2) Let the corrections of [Xi] be [X`i], and the corrected aggregated forecast then be X`T = S X`i. The procedure for finding [X`i] is described in Annex C. 6 Forecasting methods when observations are missing 6.1 Introduction Most forecasting models are based on equally spaced time series. If one observation or a set of observations are missing, it is necessary either to use an estimate of missing observations and then use the forecasting model or to modify the forecasting model. All smoothing models are applied on equally spaced observations. Also autoregressive integrated moving average (ARIMA)-models operate on equally spaced time series, while regression models work on irregularly spaced observations PAGE4 Fascicle II.3 - Rec. E.506 without modifications. In the literature it is shown that most forecasting methods can be formulated as dynamic linear models (DLM). The Kalman Filter is a linear method to estimate states in a time series which is modelled as a dynamic linear model. The Kalman Filter introduces a recursive procedure to calculate the forecasts in a DLM which is optimal in the sense of minimizing the mean squared one step ahead forecast error. The Kalman Filter also gives an optimal solution in the case of missing data. 6.2 Adjustment procedure based on comparable observations In situations when some observations are missing, it may be possible to use related data for estimating the missing observations. For instance, if measurements are carried out on a set of trunk groups in the same area, then the traffic measurements on various trunk groups are correlated, which means that traffic measurements on a given trunk group to a certain degree explain traffic measurements on other trunk groups. When there is high correlation between two time series of traffic measurements, the relative change in level and trend will be of the same size. Suppose that a time series xt of equidistant observations from 1 to n has an inside gap . xt is, for instance, the yearly increase. The gap consists of k missing observations between r and r + k + 1. A procedure for estimating the missing observations is given by the following steps: i) Examine similar time series to the series with missing observations and calculate the cross correlation. ii) Identify time series with high cross correlation at lag zero. iii) Calculate the growth factor Dr+i between r and r + k of the similar time series yt: Dr+i = eq \f( yr+i - yr, yr+k+1 - yr) i = 1, 2, . . . k (6-1) iv) Estimates of the missing observations are then given by: eq \o(\s\up4(^,x))!Unexpected End of Expression.r+i = xr + Dr+i (xr+k+1 - xr) i = 1, 2, . . . k (6-2) Fascicle II.3 - Rec. E.506 PAGE1 Example Suppose we want to forecast the time series xt. The series is observed from 1 to 10, but the observations at time 6, 7 and 8 are missing. However a related time series yt is measured. The measurements are given in Table 1/506. TABLE 1/E.506 Measurements of two related time series; one with missing observations t 1 2 3 4 5 6 7 8 9 10 xt 100 112 125 140 152 - - - 206 221 yt 300 338 380 422 460 496 532 574 PAGE4 Fascicle II.3 - Rec. E.506 622 670 The last known observation of xt before the gap at time 5 is 152, while the first known observation after the gap at time 9 is 206. Hence r = 5 and k = 3. The calculation gives: D6 = eq \f( 496 - 460, 622 - 460) = eq \f( 36, 162) D7 = eq \f( 532 - 460, 622 - 460) = eq \f( 72, 162) D8 = eq \f( 574 - 460, 622 - 460) = eq \f( 114, 162) eq \o(\s\up4(^),x)6 = 152 +eq \f( 36, 162) (206 - 152) = 164 eq \o(\s\up4(^),x)7 = 152 +eq \f( 72, 162) (206 - 152) = 176 eq \o(\s\up4(^),x)8 = 152 +eq \f( 114, 162) (206 - 152) = 190 6.3 Modification of forecasting models The other possibility for handling missing observations is to extend the forecasting models with specific procedures. When observations are missing, a modified procedure, instead of the ordinary forecasting model, is used to estimate the traffic. Fascicle II.3 - Rec. E.506 PAGE1 To illustrate such a procedure we look at simple exponential smoothing. The simple exponential smoothing model is expressed by: eq \o(\s\up4(^),m)t = (1 - a) yt + aeq \o(\s\up4(^),m)t-1 (6-3) where yt is the measured traffic at time t eq \o(\s\up4(^),m)t is the estimated level at time t a is the discount factor [and (1 - a) is the smoothing parameter]. Equation (6-3) is a recursive formula. The recursion starts at time 1 and ends at n if no observation is missing. Then a one step ahead forecast is given by: eq \o(\s\up4(^),y)t (1) = eq \o(\s\up4(^),m)t (6-4) If some observations lying in between 1 and n are missing, then it is necessary to modify the recursion procedure. Suppose now that y1, y2, . . ., yr, yr+k +1, yr+k +2, . . ., yn are known and yr+1, yr+2, . . ., yr+k are unknown. n the time series has a gap consisting of k missing observations. The following modified forecasting model for simple exponential smoothing is proposed in Aldrin [2]. (1 - a) yt + a eq \o(\s\up4(^),m)t-1 t = 1, 2, . . . , r eq \o(\s\up4(^),m)t = (1 - ak) yt + akeq \o(\s\up4(^),m)t t = r+k+1 (6-5) (1 - a) yt + a eq \o(\s\up4(^),m)t-1 t = r+k+2, . . . , n where ak = eq \f( a,1 + k(1-a)2) (6-6) By using the (6-5) and (6-6) it is possible to skip the recursive procedure in the gap between r and r + k + 1. In Aldrin [2] similar procedures are proposed for the following forecasting models: - Holt's method, - Double exponential smoothing, - Discounted least squares method with level and trend, - Holt-Winters seasonal methods. Wright [17] and [18] also suggests specific procedures to modify the smoothing models when observations are missing. As mentioned in the first paragraph, regression models are invariant of missing observations. When using the least squares method, all observations are given the same weight. Hence, missing observations do not affect the estimation procedure and forecast are made in the usual way. PAGE4 Fascicle II.3 - Rec. E.506 On the other hand it is necessary to modify ARIMA models when observations are missing. In the literature several procedures are suggested in the presence of missing data. The basic idea is to formulate the ARIMA model as a dynamic linear model. Then the likelihood function is easy to obtain and the parameters in the model can be estimated recursively. References to work on this field are Jones [9] and [10], Harvey and Pierse [8], Ansley and Kohn [3] and Aldrin [2]. State space models or dynamic linear models and the Kalman Filter are a large class of models. Smoothing models, ARIMA models and regression models may be formulated as dynamic linear models. This is shown, for instance, in Abraham and Ledolter [1]. Using dynamic linear models and the Kalman Filter the parameters in the model are estimated in a recursive way. The description is given, for instance, in Harrison and Stevens [7], Pack and Whitaker [13], Moreland [12], Szelag [15] and Chemouil and Garnier [6]. In Jones [9] and [10], Barham and Dunstan [4], Harvey and Pierse [8], Aldrin [2] and Bílviken [5] it is shown how the dynamic linear models and the Kalman Filter handle missing observations. Fascicle II.3 - Rec. E.506 PAGE1