Journal Archive

Karaj City is the capital of Alborz Province, Iran. Its population is about 1.97 million, making it the fourthlargest city in Iran after Tehran, Mashhad and Esfahan. It is situated 20 kilometers west of Tehran, at the foothills of the Alborz Mountains. Its coordinates are 50 degrees, 55 minutes and 15 seconds east longitude and 35 degrees, 45 minutes and 50 seconds north latitude. Its area is about 858 km². The annual rainfall of the area is about 261 mm, and the mean annual temperature is between 5 and 13 degree centigrade (Ilanloo, 2011). A monitoring station was considered for this work which is located in the main metro station of the Karaj to Tehran metro (Fig. 1).

Fig. 1. 
The location of study area.

An artificial neural network is an idea to process information that is inspired by biological nervous systems and processes information like the human brain. The overall performance of ANN can be observed in Equations 2 and 3 (Hambli, 2011; Haykin, 1999).

Where yim and vim are the input and output of i-th neuron in m-th of the hidden layer, f is activation function. L is the number of connections to previously hidden layers and bim represent the weight and bias.

Several different types of ANNs are available. We used Radial Basis function (RBF) and Multilayer Perceptron (MLP) neural networks in this study. We developed all the programs with MATLAB software (R2012a) produced by the Math Work Company.

MLP neural networks have the ability to determine the number of hidden layers, the number of neurons in each layer and transfer functions used in the layers. These functions can be log sigmoid functions, the one most usually applied in ANN (according to Eq. 4) or tan. Sigmoid functions (according to Eq. 5).

Where θ is the slope of the transfer function (θ=0.9).

Several algorithms for training MLP networks exist. In the simplest implementation of these networks, weight and bias are updated in the direction in which efficiency decreases (the opposite direction of the slope). Equation 6 illustrates a repeat of this algorithm (Rumelhart and McClelland, 1986).

Where x_k is weight and bias vector; g_k is the slope of the function and a_k is the learning rate. Fig. 2 indicates the MLP neural network used in this study.

Fig. 2. 
Schematic of the MLP network in this study.

Table 2 indicates the parameters of the MLP network design, where the parameter ‘show’ indicates the number of iterations after which the training status is displayed; α is the speed of learning; the goal is the target error rate. β is the coefficient of momentum and epochs are the frequency of training. Training stop when it reaches the number of levels determined in epochs, or when the amount of the performance function is less than the goal parameter. The learning rate is multiplied by the slope value and used to update weights and bias. If the value of this parameter is too large, the training process will not have enough stability and if it is too small, the algorithm will need a long time to converge. Momentum ratio (β) receives a value between 0 and 1. When the momentum ratio is zero, weight changes are only from the performance function slope and when it is one, weight changes are based on previous weight changes and the slope is ignored.

First of all, in the performance of MLP neural networks, the weights are selected randomly and then are applied to the input of the neural network along with randomized bias. After that output prediction are compared with observed output data, and finally the mean square error (MSE) is calculated between the observed data and predicted data. If the error value is less than the desired error set of the network, training stops, otherwise, weights and bias will be changed to reduce errors.

Radial functions are simply a class of functions. In principle they could be employed in any sort of model, linear or nonlinear. Fig. 3 presents a RBF network, each of n components of the input vector x feed forward to m basis functions whose outputs are linearly combined with weights wjj=1m into the network output f(x) (Orr, 1996).

Fig. 3. 
Diagram of RBF network.

Compared to MLP neural networks, RBF neural networks need less time to design and more neurons are necessary. When there are many training vectors, these networks have the best performance (Cohen and Intrator, 2002). The procedure in these networks is: the training process continues by increasing the number of hidden layer neurons until the performance function reaches the target value or until it reaches the maximum number of neurons (the number of data).

The RBF neural networks have an easy architecture. Their structure includes an input layer, a single hidden layer, and an output layer, which at each output node makes available a linear combination of the outputs of the hidden-layer nodes. Training an RBF is comprised of two steps. First, the basic functions are established using an algorithm to cluster data in the training set. Kohohen self-organizing maps (SOMs) or a k-means clustering algorithm has been most typically used. Kohohen SOMs (Kohohen, 1984) are a form of ‘selforganizing’ neural network that learn to differentiate patterns within input data. A SOM will, consequently, cluster an input data according to perceived patterns without having to be given a corresponding output response. K means clustering and organizing all objects into a predefined number of groups by minimizing the total squared Euclidean distance for every object regarding its nearest cluster Centre. Nevertheless, other techniques, such as orthogonal least squares and maxi min algorithms, have also been applied (Song, 1996). Next, the weights linking the hidden and the output layer are calculated directly using simple matrix inversion and multiplication. The direct calculation of weights in an RBF makes it far quicker to train than an equivalent MLP (Dawson and Wibly, 2001).

To determine the amount of error in predicting PM_2.5 and to evaluate the performance of the models, we applied a Root Mean Squared Error (RMSE) and a Mean Bias Error (MBE) which are indicated in Equations 7 and 8. Also, we applied the Nash-Sutcliffe Efficiency Coefficient (E), coefficient of determination (R²) and the Index of Agreement (IA), between the observed and predicted data to illustrate the validity of the model (Feng et al., 2015; Voukantsis et al., 2011; Krause et al., 2005).

Where P and M are the predicted and the observed values of PM_2.5 at the time t, respectively, and M¯ and P¯ are the average of predicted and observed values, respectively and n is the number of data.

Several mathematical methods have been used to measure the concentration of air pollutants such as Markov chain model, which was used in this study. Markov chain is a mathematical method for modeling of probabilistic processes. Two features characterize a Markov chain: (a) state space and (b) level. If we define the Karaj weather as a system, its state space (S) in a given hour will be one of three positions in Eq. 12 (Chung and AitSahlia, 2003).

Where g is the duration of the day with good air quality. s is sensitive hours and u represents the duration of the day with unhealthy air quality. A Markov chain level specifies that the current state of the system depends on how many of the previous states. To determine the most suitable level in a Markov chain several tests are available. We used the Akaike information criterion (AIC) and the Bayesian information criterion (BIC) tests (Eq. 13 and 14). These tests were performed for different levels and the most suitable criteria were selected based on the lowest AIC or BIC. The AIC and the BIC tests are based on likelihood functions and the likelihood values of a Markov chain, which from zero likelihood (L₀), the first (L₁), second (L₂) and third (L₃) according to equations 15 to 18 (Taylor and Karlin, 1998).

Where S is the number of states, m is the order of the Markov chain; n and P^ are the transition count (number of data) and estimated transition probabilities, respectively. The n_ij is the observed transition count for a binary time series. For example, the transition count n₀₀ specifies the number of consecutive pairs 0’s in time series (Wilks, 2006). More information is available by Wilks, 2006.

In this study, according to the results of the AIC and BIC tests, a Markov chain was defined from the first level. Equation 19 indicates its mathematical expression (Logofet and Lensnaya, 2000).

According to equation 18, the state of a variable in time t, X_t, only depends on its state at the time t-1, X_t-1, and it does not depend on the path through which the system reaches its current state. The behavior of a Markov chain can be summarized in the form of a matrix of transition probabilities where each of its elements represents the probability of transition from one mode in the past to another mode later. Transition probability matrix is a k×k matrix, where k is the number of members of a state space. Eq. 20 expresses a case of a transfer matrix and Eq. 21 expresses the first level of a three-state Markov Chain transition probability matrix used in this study (Shamshad et al., 2005).

Where subtitles g, s and u as mentioned previously represent time with good, sensitive and unhealthy air quality, respectively, as the first subtitle(subscript) is related to time t-1 and the second subtitle (subscript) is related to time t (e.g. P_uu is the probability of occurrence of two consecutive unhealthy hours). Each element of this matrix is determined based on Eq. 22.

Where n denotes the hours, for example P_gu is the probability of occurrence of one hour unhealthy air quality after one hour healthy air quality.

By determining the transition probability matrix of the Markov chain, several analyses can be carried out with the most important of which is the continuing pollution of unhealthy PM_2.5. Considering that after the occurrence of a continuation of n hours of good air quality certainly an hour of sensitive quality or unhealthy certainly will occur. Eq. 23 is extracted to calculate the probability of n hours of continuing good air quality, and similarly Eq. 24 and 25 are extracted to calculate the probability of sustainability of n hours of unhealthy and sensitive air quality.

One of the major issues affecting the performance of a neural network is to select the input parameters to train the network. For this purpose, various algorithms have been used in previous studies (Niska et al., 2006; Eleuteri et al., 2005; Kohavi and John, 1997). The use of these methods has limitations and errors; therefore, in this study the decision on the choice of these parameters was based on their performance in the network training. Therefore, various models were applied and their performance were compared (Table 4). In all of these models, we applied two hidden layers with 15 neurons in the each layer (15 neurons in the first layer and 15 neurons in second layer) and type of transfer functions in the layers (First layer: tan-sigmoid transfer function; and second layer: log-sigmoid transfer function) were considered, so that the impact of changes in input parameters on the network was tangible. For example, our ANN6 model was made by using NO, PM₁₀, SO₂ and temperature as inputs with two hidden layers, 15 neurons in the first layer and 15 neurons in the second layer and transfer functions Tansig and Logsig.

We trained the network using 1,300 data, to predict hourly concentrations of PM_2.5 in Karaj City. To determine the most suitable network in predicting output, we changed the characteristics of the network such as, inputs, number of neurons in the hidden layers, type of transfer functions, learning rate and momentum factors. To choose the most accurate and reliable model, the amount of errors, R², IA and E were computed. As a result, it was revealed that the network in which input parameters were CO, NO_x, PM₁₀ and temperature, (ANN7), had a better performance than the other scenarios (Table 5).

After selecting the optimal input parameters and appropriate number of hidden layers (ANN7), the effects of the other main factors in ANN performance (normalizing data) were evaluated. Networks using input data CO, NO_x, PM₁₀ and temperature were trained once by normalizing data and once without it. The number of neurons in the hidden layers in the MLP networks was changed using trial and error and were automatically changed in RBF neural networks from 0 to 1,488 (number of data). Table 6 indicates the characteristic of ANN7 applied in our study.

Fig. 4 indicates the performance of various networks. Normalization of the data improved network performance. Increasing the number of neurons in the hidden layer in the MLP and the RBF networks reduced forecast error and increased the coefficient of determination. A network made with two hidden layers including 19 neurons in the first layer and 16 neurons in the second layer, using the normalized data (FN19/16) had a coefficient of determination, efficiency (E), RMSE and MBE equal to 0.92, 0.981, 1.25 and 0.0545, respectively. The network had the best performance among the MLP networks which we developed (Fig. 4).

Fig. 4. 
The performance of MLP networks with various structures (N: normalized data, 0: without normalization).

Fig. 5 illustrated the comparison of the observed data with the predicted data using MLP network with 188 data test for PM_2.5 parameter. The coefficient of determination was 0.92 and the RMSE was 1.25. The result indicates the good workability of the model.

Fig. 5. 
A comparison between observed and predicted PM_2.5 data using FN19/16.

Fig. 6 indicates the effect of the learning rate and momentum factor in the performance of MLP networks. First, we fixed the rate of learning (on each of the values: α=0.05, 0.2, 0.4, 0.65, 0.8, 0.95). We increased the momentum coefficient (β) from 0.05 to 0.95 in steps of 0.05 and trained and tested each step 20 times. Each of the network errors in predicting PM_2.5 values were averaged and considered as the error value for a model with factor of momentum and learning rate. These tests were made on the MLP model with two hidden layers (FN19/16). The results indicated that increasing the learning rate caused weakens of the network performance. On the other hand, lower values in the learning rate made the learning process time-consuming. Increasing momentum from 0.05 to 0.95 first increased errors, then improved performance and reduced errors. Therefore, we concluded that determining changes of weight by using any of the slopes of efficiency function or previous weight changes alone improves network performance compared to when the both of these factors are involved in determining the new weight. To select the momentum factor in the range of 0 to 1 improved network performance in predicting PM_2.5 in our research.

Fig. 6. 
The effect of learning rate and momentum factors in the RMSE rates in the MLP neural network.

Correct designing of the parameters of an MLP neural network such as input parameter, the number of layers, the number of hidden layer neurons, transfer functions, learning rate and momentum factors and normalizing data also increases the accuracy. We considered all mentioned parameters and determined the effect of learning rate and momentum factor in the amount of RMSE in the MLP neural network (Fig. 6) which causes a good performance of predicting PM_2.5 in the Karaj City. Voukantsis et al. (2011) used a principal component analysis to select input parameters for MLP neural network and predicted PM₁₀ and PM_2.5. They obtained IA=0.8. We carried out other method for selecting input parameters and selected those input parameters which each of input parameter had a low correlation coefficient with PM_2.5 parameter. Index of agreement (IA) in our study for ANN7 was 0.84. The advantage of their work was to select a few meteorological parameters as input to the ANN. Our input parameters for MLP and RBF were NO, NO₂, NO_x, PM₁₀, SO₂ and temperature. However, meteorological data was not available for authors. Bahari et al. (2014) predicted PM_2.5 concentrations, in one station, in Tehran using an MLP neural network. Their input parameters were temperature, wind speed, wind direction, relative humidity, and cloud cover and inversion strength. They did not describe the method of selecting input parameters to the ANNs. The R² of the study were between 0.61 and 0.79. However, the R² in our study for MLP and RBF was 0.92 and 0.93, respectively.

We compared the results our MLP neural network with the results of Voukantsis et al. (2011) and Feng et al. (2015) and found that our model presented a suitable performance in predicting PM_2.5 concentrations in Karaj City. Feng et al. (2015) obtained RMSE rate between 28 to 36 for one day and two days PM_2.5 prediction using an MLP neural network. The amount of RSME in the MLP neural network was 1.25 for FN19/16 model. Their RSME results could be due to the smaller number of data used in their studies and may need a longer data collection. In fact, they changed in concentrations of suspended particles over the year lead to a reduction in network accuracy in predicting the amount of this parameter.

In RBF networks that are formed from a hidden layer and Radial Basis transfer function, the number of neurons starts from zero and increases. At each stage, error calculated and reported. This process continues until the error decreases to zero, or the number of neurons is equal to the number of input data. Fig. 7 indicates the changes of prediction error for PM_2.5 due to the increase of the number of neurons of hidden layer using this method. The root mean square error of the neurons in this method decreased with increasing neurons from 7.88 to 2e-06 and the coefficient of determination reached 0.93. The results indicate the proper functioning of this network in predicting the concentration of PM_2.5, without requiring any design. The coefficient of determination between observed data and predicted data reached 0.93 which indicates the reliability of RBF in predicting of PM_2.5 (Fig. 8).

Fig. 7. 
The changes of RMSE through increasing the number of neurons in RBF network.

Fig. 8. 
A comparison between observed and predicted PM_2.5 data using RBF network.

The training of neural network structure of artificial neural network which was used in our study was, according to design of continuous statistical model by using past data. This artificial neural network presents a numerical description of a mathematical structure which is able to predict the physical condition of air pollution for 24 or 48 hours in advance. By increasing the length of existing data, we can increase the possibility of predictions durations.

Equation 26 presents the transition probability matrix and Fig. 9 indicates its graphical representation for 9 different transition mode. The results of this matrix state that, in the event of good or sensitive air quality, there is the possibility of repetition of the above condition (probability 71% and 89% respectively). However, in the event of pollution and poor quality in one of night and day hours, the possibility of repeating or transferring to sensitive state is somehow the same value (respectively 48 and 51 percent probability) and there is no high chance for the air to be healthy in the following hours. Generally, in the case of healthy, fresh air, the likelihood of its re-occurrence, unhealthy state, the possibility of switching to a sensitive air quality, and in the case of sensitive weather, the likelihood of its continuation get the highest value.

Fig. 9. 
The probability of continuation of n hour (a) good, (b) sensitive, (c) unhealthy AQ statement.

Table 7 presents the number of incidents and the probability of selection for each of the two data sets initial calculations (1,300 data) and test (data 188) with the error between them. By calculating the transition probability matrix for 188 test data, showed that the matrix with slight error is similar to the calculated matrix for the raw data. For example, the possibility of transfer of clean air quality to sensitive state (gs) was calculated as 0.28 that compared with the probability of the situation in test data that was equal to 0.2758, with absolute error of 0.014 has acceptable accuracy and precision. As a result, the probability of occurrence of PM_2.5 pollution in different periods is predictable using this method.

Fig. 9(a-c) indicates the continuation of n hour for 2 hours to 1 day for each of air quality conditions. The vertical axis is the likelihood to maintain a state and the horizontal axis presents duration time. For example, in the first curve (Fig. 9(a)), P(n) is the probability that a good air quality status is maintained for n hours and then the situation will change. Fig. 9(c) illustrates the high probability of remaining two-hour unhealthy state (25%) the poor status of PM_2.5 pollutant. However, the likelihood of continuation of this situation to fourth and fifth hours is reduced and there will be no substantial risk of the continuity of too much pollution. Also, due to the low likelihood of continuation of healthy condition and opposite to that its relative high likelihood in some hours or daily continuation of sensitive quality, we will not experience favorable weather conditions in terms of particulate pollution, especially for sensitive groups.

The MLP, RBF and Markov chain are independent model. The MLP and RBF belong to a family of artificial intelligence neural network, which they can predict future physical quantities of PM_2.5 in a city. The RBF results may be accurate than MLP model. The Markov chain model can describe the probability of occurrence of PM_2.5 in different periods and also indicates the quality of air in a city in three forms, including good quality, sensitive and unhealthy conditions.