Evaluation of the Effectiveness and Efficiency of Atmospheric Particulates Reduction Policy: The Case of South Korea

1. INTRODUCTION

The rising levels of atmospheric particulates have become public health threats to growing portion of the world population. A 2021 WHO report on air quality and health estimates that air pollution was behind approximately 7 million premature deaths annually while in 2019 over 90% of the world population was living in air conditions that does not meet WHO air quality guidelines (WHO, 2021)¹. A research published in Nature in 2017 renders a similar diagnosis (Zhang et al., 2017). Among 3.45 million premature deaths due to air pollution in 2007, 1.19 million were in China, another 0.58 million in India, and still another 0.45 million in the other Asian countries, indicating the fact that a larger share of such deaths is taking place in Asia. In fact, the number of premature deaths due to air pollution accounts for 5-7% of total annual deaths in the world.

Alarmed by the harms of such magnitude done by air pollution, governments around the world have introduced various air pollution reduction measures. Certain municipalities in California² have adopted an array of particulates reduction strategies including regulation of outdoors waste incineration, introduction of transportation means with little to no exhaust gas emissions, and scrappages of old vehicles in order to control emissions from fixed sources, road mobile sources as well as non-road mobile sources (SCAQMD, 2012). Japan also strives to reduce air pollution by promoting low-emission vehicles, regulating vehicle emission levels and introducing eco-driving system³. China, a country exposed to one of the highest levels of atmospheric particulates concentration, has put into practice industrial dust control technologies, scrappage of old vehicles and removal of arsenics in air particulates (MEE, 2012).

While the type of policy tools varies among different countries, what stands out is the fact that the policy efforts of most countries are in the form of a policy mix. A policy mix, in its original usage, refers to a combination of various economic policy tools governments implement so as to pursue both economic growth and stability. Afterward, its application has expanded to the environmental field to denote governmental efforts characterized by proactive combinations among related policy tools or prioritization of policy tools in order to minimize contradictions (Lafferty and Hovden, 2003). Put differently, a policy mix refers to the pursuit of an optimal policy combination motivated by comprehensive understanding of the interrelations and interactions amongst existing policies and their trade-offs (Flanagan et al., 2010; Sorrel and Sijm, 2003). This is due to the particular characteristic of the issue: the existence of various sources of atmospheric particulates and channels of influx. Inevitably, any measures that aim at controlling those sources and channels involve countermeasures that target each type of sources and influx channel.

Yet, the process of introducing various policy instruments often is the source of issues that hinder effectiveness and efficiency of a policy. For example, if a certain policy measure that is effective but financially costly is prioritized over others, the overall policy response is consequently more likely to be inefficient despite its effectiveness. On the other hand, if a policy measure gives more weight to an instrument that is more efficient but relatively less effective, then such policy response is more likely to fall short of achieving its desired level of effectiveness. Hence, the objective of devising a policy mix in the atmospheric particulates reduction strategy boils down to choosing the combination of policy instruments that achieves the best effectiveness and efficiency possible.

However, identifying the exact levels of effectiveness and efficiency of individual policy instruments is not easy. Regardless of whether the effect of a certain policy instrument can be observed, it is a challenging task to determine whether such effect indeed did originate from the relevant policy instrument. Accordingly, for the same reason, it is hard to estimate the level of efficiency based on its effectiveness vis-à-vis the cost incurred. Yet, such difficulty does not obviate the need for information on the effectiveness and efficiency of individual policy instruments. In order to come up with an optimal policy combination, it is essential to secure reliable information on the effectiveness and efficiency of individual policy instruments despite the inherent difficulty involved confirming the exact levels of effectiveness and efficiency. On the other hand, a government intervention to any policy issues in most cases is translated into expenditure of resources such as manpower, organizations and budget, which in turn is directly related to the issue of allocation of social resources. On this account, developing and selecting more efficient and effective policy instruments is not only the most demanding task but frequently the source of controversy in the policy-making stage (Weimer and Vining, 2015; Christensen, 2006; Salamon, 2002; Kettl, 1988; Rhoads, 1985; Stokey and Zeckhauser, 1978; Kirschen, 1964).

The purpose of this study is to examine the effectiveness and efficiency of policy instruments implemented under the 1^st Basic Plan on Metropolitan Area Air Quality Improvement (the Plan) on the basis of the discussion above⁴. To the extent that the policy response for air quality improvement was in the form of a combination of various policy instruments, it is difficult to determine the precise levels effect and efficiency of each of those policy instruments. Nevertheless, Korea’s policy mix implementation characterized by the fact that the lion’s share of the budget was devoted to a small number of policy instruments makes easier the attempt to determine the effectiveness and efficiency of those select policy instruments. Therefore, this study first examines the degree of seriousness of Korea’s atmospheric particulates issue and policy instruments implemented in order to improve the air quality under the Plan; secondly, explain the process of estimating policy effects of such instruments. Further, since detailed information on the use of budget per policy instrument is available, it is also possible to estimate the efficiency of the policy instruments once policy effect is estimated. Lastly, the effectiveness of the Plan assessed, and subsequently, better particulates reduction policy design is proposed based upon observed level of effectiveness and efficiency of each policy instruments.

2. THE CLEAN AIR ACT FOR SEOUL METROPOLITAN AREA

The need for and economic benefits of air quality improvement have been noted by many previous researches with a substantial history of various governments devising and implementing their own policy responses. According to OECD (2008), the U.S., for instance, expects that 1 USD spent on air quality improvement policy as mandated in the Clean Air Act would incur a benefit worth 4 USD while Canada projects that reducing PM₁₀, PM_2.5⁵ and O₃ emissions will bring about health benefits worth 3.6 billion USD annually. EU also estimates the net economic benefits for the next 20 years of its EC clean Air for Europe program to amount to 42 to 168 billion in USD. Such projections of large economic benefits, on the flip side, mean that deterioration in air quality can cause damages of corresponding magnitude.

Republic of Korea is not an exception in feeling an urgent need for air quality improvement. Particularly, the Seoul Metropolitan Area houses 46% of the total national population while its geographic area represent only 11.8% of the country’s total⁶. The number of cars in the area skyrocketed from 0.27 million units in 1980 to 8.26 million units in 2010, greatly aggravating the concentration of atmospheric particulates such that 32% of national NOx emissions, 23% of PM₁₀, 38% of VoCs and 45% of CO originate in the area. Certain studies estimate that the scale of damages done by such air pollution in the metropolitan area as of early 2000s could be as large as 8 billion in USD (GRI, 2003; Cho et al., 2002).

For these reasons, the ‘Special Act on Seoul Metropolitan Air Quality Improvement’ was enacted in 2005, with its 1^st Basic Plan for Metropolitan Area Air Quality Management (the Plan) effective from 2005 to 2014 focusing on reducing emissions of NOx, SOx, VOCs, and PM₁₀, among other atmospheric particulates. As for PM₁₀, the subject of this study, the Plan aimed at reducing the emissions from the 2001 level of 14,681 metric tons to 8,999 metric tons by 2014 and its concentration level from 69 μg/m³ in 2003 to 40 μg/m³ in 2014 (Kim and Lee, 2018; MOE, 2013)⁷. To this end, the Management by Objective (MBO) approach was adopted in which the municipal governments in the metropolitan area endowed with the freedom to establish and implement specific instruments they independently devise reallocate per-pollutant emissions quota based the original quota the Ministry of Environment assign. As Table 1 shows, sources of atmospheric particulates are classified largely into point sources, green energy and urban sources, area sources, road mobile and non-road mobile sources. Various representative policy tools for each category that have been proposed for each category are also shown.

Table 1.

Key tools included in the Special Act on Seoul Metropolitan Air Quality Improvement.

The collection of Korea’s atmospheric particulates reduction policy instruments has the following unique characteristics. First, there are different factors inducing particulates concentration in the atmosphere, such as influx from abroad, pollution from domestic sources domestic, and chemical reaction in the atmosphere (Kim, 2006). In particular, because Korea is adjacent to China and the direction of atmospheric circulation is from China to the Korean peninsula, the amount of atmospheric particulates influx from China into the country is presumed to be substantial (Park and Han, 2014). However, generation of particulates from chemical reactions in the atmosphere is hard to manage, and blocking out the influx of particulates from abroad also requires active efforts on the part of governments of all involved countries. For such reasons, policy measures that intend to manage particulates generation in the atmosphere and to stall the inflow of particulates from foreign sources are hard to become realistic options. Hence, the scope of South Korea’s atmospheric particulates reduction policy is limited to efforts to suppress pollution from the domestic sources.

Second, Korea’s policy prioritized reduction in particulates generation from road mobile sources and non-road mobile sources over the other domestic sources of pollution⁸. Specifically, the Ministry of Environment aimed to achieve reduction of 4,506 metric tons (79%) from the road and non-road mobile sources comparing to the total of 5,682 metric tons of annually generated particulates (MOE, 2005). Such strategy is also confirmed when the use of relevant budget is examined. The figures of budget expenditure shown in the Table 2 reveal a disproportionately large amount of the resources that has been put into programs targeting reduction of the emissions from the road or non-road mobile sources. Of particular note is the fact that 75% of the total budget was spent on lowering emissions level of diesel vehicles (subsidy programs for Diesel Particulate Filter installation). Another 19% was spent on promoting the use of low-emissions vehicles (scrapping subsidies for old diesel vehicles) so that a total of 94% of the budget was expended for the purpose of reducing emissions from diesel vehicles.

Table 2.

The use of allocated budget for the First Air Quality Improvement Plan9.(Unit: million ￦).

Third, during the period of the Plan, there were no specific regulations related to particulates emissions in place, nor did there have been any specific changes with respect to the use of public transportation uses that could have affected the policy. Since there were no other particulates reduction policy in effect other than the one under the Plan, it can be assumed that most of the changes in domestic particulates emissions were based on the policy interventions implemented as mandated by the Plan, provided that there were not significant changes in the amount of influx from the abroad and in meteorological factors. While the budget could have been used for different purposes - implementing various regulations, development and supplying environment friendly energy, monitoring of manufacturing sites and arsenic particulate emissions, creating a green belt or establishing an overall emissions management system, Korea’s strategy of particulates reduction can be adequately summarized as reducing emissions of particulates from road mobile sources based on the subsidy program for DPF installations and diesel scrappage scheme.

Indeed, it has been reported that annual PM₁₀ emissions in the Seoul metropolitan area decreased from 14,605 metric tons in 2005 to 9,519 metric tons in 2010, while the concentration of the particulate matter in Seoul reduced from 56 μg/m³ to 41 μg/m³, coming very close to the Plan’s original target (MOE, 2013). To look at changes per source, among the 4,396 metric tons of total amount of reduced emissions, 89 metric tons were from point pollution sources, 61 metric tons from area pollution sources, 4,393 metric tons from road mobile sources and 26 metric tons from non-road mobile sources. Thus, looking at the absolute figures, we may reasonably determine that most of the goals that government originally intended were achieved, and the air quality in the Seoul Metropolitan Area improved greatly. However, whether this change is caused by the measures implemented under the Plan is a matter that needs further investigation. For example, it is necessary to carefully examine whether the decrease in the particulates concentration in the Seoul metropolitan area is due to actual reduction in emissions from the road mobile sources, rather than due to a third factor such as decreased in the influx from the abroad that is above control. Along the same line, it is necessary to confirm whether the decrease in emissions from road mobile sources is due to the DPF installation policy or due to certain factors outside the policy. As such, we can discuss the effectiveness and efficiency of the individual policies only when it is possible to estimate the effects of individual policy instruments while controlling for the effects of third factors.

3. METHODS OF ANALYSES

As discussed earlier and shown in Fig. 1, the Korean government’s primary policy instrument against atmospheric particulates is the subsidy program for DPF installation. The logic behind is that DPF installations will reduce emissions from the road mobile sources, which subsequently will result in lower PM₁₀ concentration in the atmosphere. Such strategy is well confirmed by the pattern of budget allocation and emissions reduction targets per source. When put in a slightly more general term, it can be said that there is a policy structure where sub-level policy instrument reduces the emissions from a target type of source, which in turn leads to reduction in the atmospheric particulates concentration.

Fig. 1.

Primary particulates reduction strategy under the 1st Plan.

Given such structure, this study attempts to first identify the net effect of the overall policy; second, the sources of the identified policy effects; and finally assess the effects of the individual policy instruments that intend to reduce particulates emissions from each type of sources. Therefore, this study seeks to conduct three analyses in sequence in order to estimate the relative effect of each policy instrument.

First, we want to estimate how much of the decrease in the concentration of atmospheric particulates can be attributed to the Plan. The reason for prioritizing the estimation the net effect is that it is possible to trace the source of reduced emissions only after the net effect of the policy is estimated. For example, when there is a policy effect of 10, it is possible to disaggregate policy effects such that a reduction of 5 determined to have come from road mobile sources and another 3 from point sources. Conversely, in a case where the intended policy effect is deemed to have not occurred, it will be necessary to identify the cause of the policy failure and examine the related policy instruments.

Second, after the first analysis confirms whether there is a certain degree of policy effect or not, we want to ascertain the sources of change in emissions that are behind such result. To that end, we tried to estimate the effect of each emissions source on the particulates concentration level while controlling for other factors. As discussed above, the Plan was implemented in the form a policy mix that combines instruments to control emissions from each type of sources and focuses especially on the amount of particulates emissions from the road mobile sources. A problem inherent in the above strategy is that the policy is bound to be mostly ineffective if the amount of particulates emissions from other sources surpasses that of the road mobile sources. In such a situation, future particulates reduction policy will need to address emission sources that has a greater impact on the particulates concentration level, rather than the road mobile sources. Hence, prior to determining the effectiveness of the policy measures, we sought to determine whether changes in the emissions levels from each type of sources originally aimed by the policy instruments have ultimately led to changes in the particulates concentration level.

Third, when the level of contribution from each type of sources in emissions reduction is confirmed, we attempt to single out a policy measure that contributed the most to the particulates emissions reduction. For example, in addition to the DPF installation policy, there are other various policy instruments such as reducing the traffic volume and the number of vehicles. We want to see how effective the policy instrument pushed by the government was in comparison to other policy instruments available. The overall analysis process mentioned above is summarized in Fig. 2.

Fig. 2.

Logical frame for estimating the policy effects of the dpf installation policy.

According to the process mentioned above, we attempted to estimate the policy effects of the Plan. The evaluation of effectiveness generally refers to estimated size of effects that have been generated by a project or a policy, meaning the difference in the size of changes between an experiment group, a sub-group to whom a policy intervention was introduced, and a control group, the rest without the treatment of the policy intervention. In other words, if we regard as the effects generated purely by a policy intervention the difference between a change in an environment with the policy intervention and a change in an environment identical in all aspects but for the policy intervention (counterfactual) (Angrist and Krueger, 1999; Heckman et al., 1997; Holland, 1986; Rubin, 1974)¹⁰. Although it is important to secure homogeneity in the initial conditions of the experiment group and control group in order to ensure that the measured difference can appropriately be inferred as generated by the policy intervention rather than by the characteristics of objects¹¹, since it practically impossible to simultaneously assign an identical object both to the experiment group and the control group, the method of random assignment is the best way to configure the two groups as identical as possible stochastically as well as on average (Hollister and Hill, 1995; Fisher, 1937).

The method of random assignment, however, is also not a practically feasible option in policy evaluation in that random assignment must be carried out in the stage of determining experiment and control groups prior to introducing a policy intervention¹². Another method that can be used in lieu of random assignment is the propensity score matching and synthetic control methods that obtain similarity between an experiment group and a control group through ex-post application of statistical techniques. Other commonly used methods are the difference-in-differences analysis model, panel model, or Heckman’s 2 stage model¹³. All of these methods take as a given the heterogeneity in initial conditions and estimate the effect of policy intervention by controlling a third variable or its mechanism that causes a difference between the two groups.

This paper attempts to estimate the policy effects using the difference-in differences model. The difference-indifferences model estimates the difference in the outcomes between the two groups before and after a policy intervention while controlling the effect of third factors other than the treatment. If the difference between the two groups before a policy intervention is different from the difference between the groups after the intervention, we estimate the difference in differences as caused by the policy intervention¹⁴. As for the subject of this paper - attempting to estimate the effectiveness of the policy intervention aiming at reduction of atmospheric particulate concentration, the Seoul metropolitan area as the target area of the policy is distinguished from the non-metropolitan areas, and there were differences in the air qualities of those areas that existed prior to the policy intervention. While the method used in this study as a statistical method has its own weaknesses arising from the issues of data and appropriateness of control when compared to the method of random assignment, in light of the constraints inherent in the as-is environment that cannot be identical with the laboratory environment, it is the best available option.

In estimating the difference in the changes in atmospheric particulate mass concentrations using the difference-in-differences model, dummy variables were created distinguish between the Seoul metropolitan area and non-metropolitan areas, the states of pre- and post-policy intervention. An interaction term of the two dummy variables was created in addition such that the ρ value of the Formula 1 indicates the size of the policy effect, the ‘difference in the particulate mass concentrations in the Seoul metropolitan area before and after the policy intervention’ - ‘difference in the particulate mass concentrations in the non-metropolitan before and after the policy intervention’ while controlling for the characteristics inherent in different geographic locations and influence of a third variable.

Formula 1.

Yit=β1+β2(treati)+β3 (timet)+ρ (treati·timet)+εit

Consequently, as the difference between the pre-policy air quality and the post-policy air quality in the metropolitan area is estimated as β₃+ρ and that in the non-metropolitan area as β₃, it is proper to deem ρ as indicative of the policy effect. Although the policy was implemented in 2005, the estimation points in time were set at 2005, 2006 and 2007 since we are uncertain of whether the policy effect has occurred and thereby made observable immediately or after a certain period.

Further, in order to determine the exact sources of emissions reduction that contributed to the estimated policy effect, a panel data analysis was used to estimate the relationships between particulates concentration level and the amount of emissions from each type of sources. In that a panel data analysis eliminates unobservable heterogeneity of individual factors as well as macroscopic variability inherent in different time periods, the method is a better option to obtain a more efficient estimator. At the same time, greater credibility as to the estimation of the policy effect by re-estimating the effect can be gained from conducting a panel data analysis in addition to the above DID analysis.

Lastly, in order to assess the exact levels of contributions made by each policy instrument, each type of sources was further divided into their component sources. The amount of particulates emissions can be calculated using existing formulas. For example, in the case of emissions from the road mobile sources, the amount of emissions is a function of emission coefficient, the driving distance and the number of vehicles in operation. In other words, while the previous analysis required controlling over the third factors in order to estimate the emissions amount, since emissions amount is computed based on a particular formula, the size of policy effects of each policy instrument exerted over particular emission component can be independently estimated. For example, since the DPF installation is a mean to reduce the emissions from the road mobile sources by diminishing the emission coefficient, if the value of the coefficient decreased only by little, we can deem the policy tool as relatively ineffective.

Fig. 3.

Scheme of analyses.

Though the Plan employed an array of different policy instruments, as mentioned in Fig. 1, an overwhelming majority of the budget for the Plan was used only on a few. Hence, the appropriateness of the policy design as the government proposed and implemented can be assessed in the process of undertaking the analyses planned for this study. In other words, an indirect assessment of the policy design is attempted in addition to evaluating the effectiveness and efficiency of the policy instruments.

4. DETERMINANTS OF PM₁₀ CONCENTRATION

In order to estimate the policy effects as well as the emissions amount from the source that chiefly affects the concentration level, any 3^rd factors that may affect the PM₁₀ concentration level must be controlled for. Fig. 4 depicts a generalization of determinants of the PM₁₀ concentration. As can be seen, the level of PM₁₀ concentration is determined by the factors that affect the influx from or outflow to the external environment, factors that generates or eliminates the particulates internally, and factors that affect the process of particulates generation by chemical reactions in the atmosphere.

Fig. 4.

Determinant factors of PM10 concentration. Source: Kim (2006).

Although it is possible to generalize the determinants of the PM₁₀ concentration, in the process of its application, the geographic characteristics must be taken into account. For example, the PM₁₀ level in South Korea, located adjacent to China, is more susceptible to and greatly affected by the influx of atmospheric particulates that originate from China. On the other hand, countries that are not located as close by, will experience little effect of such PM₁₀ influx. In a similar fashion, the emissions from the road mobile sources in Seoul metropolitan areas is greater than those from the other sources. Further, in certain areas, emissions from point sources surpass those from the road mobile sources. As such, regional characteristics must be appropriately considered in the process of examining determinant of atmospheric particulates concentration.

First, in terms of external influx into South Korea, it is suspected that the influx from China is the primary factor that affects the PM₁₀ level in the country. This is because China consumes enormous amounts of coals, among other fossil fuels, that generate immense amount of exhaust emissions (Kim, 2006; Kim et al., 1999). Many previous researches have already examined how much domestic PM₁₀ concentration level is affected by external influx of atmospheric particulates into Korea, and such researches does estimate that a substantial part of the domestic PM₁₀ concentration level is due to the influx from external sources (Kwon et al., 2016; Nawahda et al., 2012; Song and Calmichael, 2001). As such, this study attempted to control for the effects of the external influx, using the number of days of continued yellow dust from abroad that Government records¹⁵.

Second, government data were used to represent the amount of domestic emissions. According to the government classification system of emission sources - point, area, road mobile and non-road mobile sources, data for each type of sources per region and per year were aggregated in order to come up with annualized data that properly reflect regional characteristics. In particular, as there has been a modification in the formula for calculating the amount of emissions that yield different figures of estimated emission amount at different points in time during the period of the Plan, this study uses only one formula throughout the process of estimating emissions amount in order to enhance the accuracy of the analysis results.

Fig. 5.

Emissions amount per city and province in South Korea in 2012. Source: National Pollutants Emission Service (http://airemiss.nier.go.kr). *Metropolitan areas emphasized while the rest are non-metropolitan areas.

Third, since the amount of emissions of each region reflects the characteristics of the relevant city or province, we viewed specific characteristics of each city as a determinant of PM₁₀ concentration. Many studies on the relationship between urban characteristics and air pollution levels have found that the traffic volume and the size of population within a geographic boundary affects the level of air pollution in area (Choi, 2009; Mats et al., 2008; Jerrett et al., 2007). However, as the correlation coefficient between the number of registered vehicles and the population size within a certain area is as high as 0.98 while the correlation coefficient between the number of registered vehicles and the number of enterprises expected to cause atmospheric particulates emissions, except for the services sector, is 0.89, we viewed that the population size in a certain region properly represent the severity of traffic volume and industrial activities and used it as the only regional characteristic that is controlled for. As a result, the socioeconomic difference between the metropolitan area and the non-metropolitan area was statistically controlled by population that reflects socioeconomic status.

Fourth, the generation of atmospheric particulates by chemical reactions in the atmosphere, its outflow, and removal of particulates within a country are closely related with the meteorological conditions of the country¹⁶. Hence, regional weather characteristics were also attempted to be controlled for. In particular, wind speed and precipitation are generally considered as the main meteorological factors that affect the particulates concentration. This is because wind speed is responsible for the influx as well as efflux of atmospheric particulates while precipitation lowers the PM₁₀ concentration level by washing down the particulates in the atmosphere (Chae, 2009). While there is a contrasting view as to the role of relative humidity - one view that it increases the concentration level because of moisture in the air absorbing particulates (Shin et al., 2007) whereas another views it decreases the level by washing down particulates (Chae, 2009), regardless of the direction, it is regarded to have a statistically significant effect on the PM₁₀ concentration level. Finally, we viewed average temperature of a region as another determinant of particulates concentration as it affects the people’s behavior pattern as well as other meteorological elements within the region.

Fifth, we assume oil price as an important external factor that may affect the particulates concentration level by affecting people’s behavior. Since higher price of oil leads to greater number of people using public transportation, the emissions from the use of private cars is likely to decrease¹⁷. At the same time, the level of domestic production also fluctuates due to its sensitivity to changes in the oil prices. Higher oil prices are likely to lead to lower emissions of particulates from point sources. Therefore, oil price was included in the study as an external determinant as it changes of people’s driving and production behavior. As a variable to represent the oil prices, the Dubai crude oil price was used. The selection of the Dubai crude oil price is justified based on the correlation coefficient of 0.97 or higher between the price of Dubai crude oil and those of gasoline and diesel.

All of the data used in this study are from the official data made available by the Korean government for 16 cities and provinces between 2000 and 2014^18,19. Further, annual average figures were used because certain data were available only in those terms while certain data such as PM₁₀ concentration levels were also available on daily as well as monthly basis. Thus, for the purpose of ensuring uniformity, annual average data were used to construct the panel data used in the analysis and accordingly the data does not consider intra-year variations.

5. RESULTS OF ANALYSES

5. 1 Analysis of the Policy Effect in the Target Area: Difference-in-Differences Analysis

Because the policy instruments of the Plan were put into effect in 2005 with its scope limited to the Seoul metropolitan area, the case can be well regarded as a natural experiment. Thus, it is possible to gauge the presence as well as the size of the policy effect by examining the differences in the atmospheric particulates concentration levels between the target area and the non-target area as well as between a pre-intervention point in time and a post-intervention point in time - before and after 2005, the year of policy implementation. Fig. 6 compares the time-series trends of the atmospheric particulates concentration levels in the metropolitan area and the non-metropolitan areas. In the case of the metropolitan area, a decrease in the particulates concentration level before and after the introduction of the policy is obvious. While there was a decrease in the level in the non-metropolitan areas, the size of the decrease is slightly smaller. In addition, given the fact that the downturn has started after their peaks in 2002, the policy effect of the plan is better interpreted to have accelerated the speed of emissions reduction, rather than reversing a pattern of increase into a pattern of decrease. Particularly, while there was an apparent gap between the patterns shown in the metropolitan area and the non-metropolitan areas prior to 2005, post-2005 levels show similar patterns in both the metropolitan area and the non-metropolitan areas. However, as discussed above, factors such as population size, characteristics of type of emission sources, meteorological factors, and external factors must be also taken into account in order to accurately evaluate the policy effect.

Fig. 6.

Trends in particulates concentration levels in metropolitan areas and non-metropolitan areas.

Estimating the policy effect while controlling for other factors that may affect particulates concentration level produces the results shown in Table 3. Expressed in standardized coefficients, the effects of policy measures implemented under the Special Act on Seoul Metropolitan Air Quality Improvement are not statistically significant. In other words, the extent of emissions reduction in the metropolitan area, was not sufficiently differentiated so as to give statistical significance from that of the non-metropolitan. The results are the same when the time of policy effect expression is adjusted to 2006 and 2007. However, what was discovered that the role of the road mobile sources and the number of days of continued yellow dusts are remarkable for atmospheric particulates concentration. Meteorological factors such as temperature and relative humidity also appear to play a role. Further, oil prices have been shown to have the strongest impact on the reduction of particulates concentration level²⁰.

Table 3.

Estimation of policy effects using the difference-in differences model (Using standardized coefficients).

5. 2 The Relationship between Emissions and PM₁₀ Concentration

Although the PM₁₀ concentration level in the Seoul metropolitan area did show a downward trend since the policy implementation, the size of the reduction was not differentiated enough from that of non-metropolitan areas, and therefore a clear policy effect could not be observed in the process of estimation. Thus, it becomes necessary as mentioned in the research design, to determine which type of emission sources affects particulates concentration level. If the effect of the policy instrument the government has chosen is insignificant, we can conclude that the policy response has been wrongly designed. Further, if the emissions from the road mobile sources does have a statistically significant impact on the particulates concentration level, we must examine relevant policy instruments that are behind such result. In this study, a panel data analysis was employed to investigate the effects of emission amounts on the PM₁₀ concentration level.

Table 4 shows the coefficient estimates obtained from applying the fixed-effects model to the relationship between the PM₁₀ concentration levels and emissions from each type of sources. The fixed-effect model differs from the difference-in-differences model in that the former controls for the regional characteristics and time-specific characteristics (two-way fixed effects model) the difference-in-differences model does not observe or thus is unable to take into account. Particularly, from Model 1 to Model 4 the fixed-effect models can affirm the robustness of coefficient estimates by estimating per-source emission levels, thought to be the primary factor affecting the concentration, while sequentially controlling for regional characteristics, meteorological factors, external and other factors in the process. In addition, using standardized coefficients, the study attempts to identify the relative size of each factor’s impact on the PM₁₀ concentration levels.

Table 4.

Panel regression analysis of the PM10 concentration (Using standardized coefficients).

First, it shows that only the road mobile emissions had consistent effect on the PM₁₀ concentration. In all four models, concentration level increased as emissions from the road mobile sources associated with vehicle operation increased. The standardized coefficients are relatively large and stable, implying the need for regulating this particular type of sources of emissions. In addition, the number of registered residents, an urban characteristic, appear to have a statistically significant and, in a relative sense, the greatest effect on PM₁₀ concentration, meaning that when all other factors are held constant, greater number of people is likely to translate into greater frequency of activities in daily life, increasing the PM₁₀ concentration.

Model 3 and Model 4 take into consideration the meteorological factors. It was found that among several factors, the number of days of continued yellow dust influx has a statistically significant effect on PM₁₀ concentration and the amount of such influx is substantial. Further, it appears that higher international oil price as an external factor decreases the concentration. Thus, it can be interpreted that a rise in the oil price will also raise the fuel prices, such as those of gasoline and diesel, that in turn lead to reduction in the frequency of vehicle use as well as total driving distance along with increased use of public transportation. Resulting was the reduction in the PM₁₀ emissions from road mobile sources and in the emissions of atmospheric arsenic. However, in that effects of the oil price on PM₁₀ concentration could have been partly reflected in the estimation of the emissions from the road mobile sources, there is a possibility that the estimate at hand is somewhat underestimated. Also, unlike the other models, Model 4 does not control for the time characteristics because the factors of number of days of continued yellow dust influx and international oil price have been measured on the annual basis²¹.

The panel analysis revealed that only emissions amount of the road mobile sources has consistent effect on the PM₁₀ concentration level regardless of whether various other factors are held constant. As shown in Fig. 5, except for some cities, the emissions from the road mobile sources accounts for the largest shares of total emissions. Thus, it is reasonable to predict that among domestic sources, emissions from the road mobile sources play a major role. The data listed in Table 4 further confirms the role of emissions from the road mobile sources as the primary factor that affects the particulates concentration level even as all other variables are controlled for. This result indicates that the design of the policy that emphasized reduction in the emissions from the road mobile sources is not ill founded.

Having proved that the emissions from the road mobile sources does have a statistically significant effect on the particulates concentration level, why the government policy instrument did not succeed in reducing particulates concentration level in the policy target area needs to be explained. In other words, the effects of individual policy instruments that target reduced emissions from the road mobile sources must be assessed.

5. 3 The Effect of the DPF Installation Subsidy as a Policy Instrument

Thus far we have confirmed a reduction in the particulates concentration level in the Seoul metropolitan area as well as in the non-metropolitan areas. We have also found that emissions from the road mobile sources are the primary determinant of particulates concentration level. Consequently, it is reasonable to test the hypothesis that the particulates concentration levels in the target area and the non-target areas decreased because the emissions from the road mobile sources decreased in both areas.

As shown in Fig. 7, the changes in emissions from each type of vehicles from 2000 to 2012 reveals that almost all emission sources had comparable changes in both the metropolitan and non-metropolitan areas. Particularly, emissions from buses and vans started to decrease prior to 2005, while emissions from RVs, freight vehicles, and special vehicles showed little difference between the metropolitan and the non-metropolitan area despite observable ups and downs. Emissions from passenger vehicles, on the other hand, increased after the policy intervention albeit a slight difference between those in the metropolitan and the non-metropolitan areas. In short, although emissions from all type of vehicles with the exception of passenger vehicles decreased since 2005, when the Plan targeting emissions reduction in the metropolitan area was implemented, the difference in the amounts of reduced emissions between the metropolitan area and the non-metropolitan area is not different²².

Fig. 7.

The trends of emissions from road mobile sources in 16 cities and provinces (Metropolitan area in thick line).

The above findings necessitate a return to the discussion of the policy instruments that generate policy effects in order to account for why there were no policy effects sufficiently differentiated between the Seoul metropolitan and the non-metropolitan areas. As mentioned earlier, the government’s policy to reduce emissions from the road mobile sources focused on the strategy of installation of diesel particulate filter (DPF) in diesel-powered vehicles. An audit report on the Plans by the Board of Audit and Inspection of Korea shed a light on a critical aspect in the implementation of the program: while the rate of DPF installation on vehicles of high exhaust gas emissions (exhaust concentration levels above 50%) decreased annually, the rate of installation on low-emissions vehicles (exhaust concentration under 10%) showed annual increase (the Board of Audit and Inspection of Korea, 2016: 28). Needless to say, the resulting policy effect of expanding DPF installations was in reality far less than originally expected.

However, it is still hard to reconcile the fact that the PM₁₀ emissions in both the metropolitan and the non-metropolitan areas decreased to a similar extent and in a similar pattern with the fact that in the Seoul metropolitan area the subsidy programs for engine conversion and early vehicle scrappage programs were implemented based on the Plan and the emissions from road mobile sources decreased to a great extent. An alternative explanation could be that a third factor’s effect that reduced emissions in both areas outdid the Plan’s policy effect in the Seoul metropolitan area. To examine this hypothesis, we need to assess the emissions from on-road mobile sources. The formula for estimation is below:

Formula 2

Thus, the emissions from the road mobile sources is a function of emission coefficient of each car model, driving distance and the number of cars of the same model, while emission coefficient is a function of car model, fuel and speed and driving distance is a function of car model and driving route. The number of cars is the number of on-road vehicles of each car model. The installation of DPF mentioned earlier and engine conversion program is a representative policy instruments chosen for the purpose of reducing emission coefficient. Early vehicle scrappage program, on the other hand, is a policy instrument that reduces the number of vehicles on the road.

What is important, as implied in the formula, is that emissions from the road mobile source can be reduced by reducing emission coefficients as well as cutting down the driving distance and the number of cars on the road. For example, Fig. 8 and Fig. 9 depicting the rate of change in the number of registered vehicles and average daily driving distance in each province and major city show that the rate of increase in the number of registered vehicles in metropolitan areas is not markedly higher than those in the non-metropolitan areas. In fact, the rate of increase is the lowest in Seoul. That is, the number of registered vehicles rose in all provinces and cities, and it can be inferred that the PM₁₀ emissions has increased accordingly. Yet, the daily average driving distances in all provinces and major cities, an important factor in calculating the emissions, have decreased since around 2005 displaying a similar overall fashion among them.

Fig. 8.

The rate of change in the number of registered vehicles per city and province (as of 2004). Source: The Korea Transportation Safety Authority23, Han et al. (2017).

Fig. 9.

Daily average driving distance per city and province. Source: stat.molit.go.kr, Han et al. (2017).

In sum, the most reasonable inference is that DPF installation and other emissions reduction programs did decrease the emission coefficients while the number of registered vehicles has increased and the average driving distance decreased. In such a circumstance where the effects of the factors that affect both the metropolitan and the non-metropolitan areas overshadow the effect of the policy instrument that was differentially applied between the control and the target areas, reduction in emissions depends on not just one factor, but on multiple factors as well as their relevant sizes of effect, complicating the attempt to estimate the policy effect.

Nonetheless, such inference does not mean that the government policy to reduce the emissions coefficient has failed. Rather, more appropriate conclusion is that the policy instrument implemented with the objective of reducing emission coefficient has not produced a notable outcome in light of the changes in the non-metropolitan area despite the fact that it was the government’s primary policy instrument that exhausted the majority of the budget. Therefore, its effectiveness is less than satisfactory even as there does exist sufficient causality between the policy instrument and the policy objective. Furthermore, the results obtained from analyses indicate that the rising oil price is more likely to be the factor that was behind the reduced nationwide driving distance. As can be seen in Table 4, higher oil price is likely to lead to lower particulates concentration level. As the oil price has hovered its record high since 2005, it is likely that people were motivated to use public transportations rather than driving private cars. Consequently, despite the fact that the lion’s share of the budget was used to reduce emission coefficients of the road mobile sources, the primary factor that had the most dominant effect on the actual decrease in the particulates concentration level was the decrease in the total nationwide driving distance due to the rise in the oil price.

The Table 5 summarizes the discussions above. In all geographic areas, the number of registered vehicles increased while the average driving distance per car decreased. On the other hand, the emission coefficients in the target area and the control areas moved in different directions - a decrease in the emission coefficient in the metropolitan area, the presumed policy effect. As iterated earlier, what appears puzzling here is the fact that emissions from the road mobile sources have decreased in both the metropolitan and the non-metropolitan areas to a similar extent. Given the factors in the formula, a likely explanation is a case where the extent of decrease in the driving distance is overwhelmingly dominant so as to obscure the differences in the number of registered vehicles and the differences in the size of decreases in emission coefficients, the policy effect, between the metropolitan and the non-metropolitan areas. In other words, although the Plan must have reduced the emission coefficients in the metropolitan area, the fact that such policy effect nonetheless did not result in critical difference between the metropolitan and the non-metropolitan areas means that it was not a cost-efficient policy instrument vis-à-vis the nationwide decrease in the average driving distance. Also, assuming that such result did arise from differences in driving distances, it can be inferred that more efficient policy instruments should have been expansion of rotated no driving day campaign, promotion of public transportation and decrease in the driving distance due to higher oil prices rather than the DPF installation program, the cost of which accounted for an overwhelming portion of the budget expenditure²⁴.

Fig. 10.

Gasoline and diesel oil price movements in Korea. Source: http://www.opinet.co.kr.

Table 5.

Changes in the components of road mobile emission sources.

6. CONCLUSION

Among the broader range of atmospheric environmental issues that have attracted global attention in recent years, this study attempts to evaluate South Korea’s atmospheric particulates reduction policy that was in effect from 2005 to 2015. Specifically, it attempted to test whether the policy was effective and efficient in reducing atmospheric particulates and proposed more effective and efficient policy instruments. The examination of Korea’s case has following implications:

First, as the scope of this study is limited to Korea’s atmospheric particulates reduction policy which was devised based on Korea’s specific contextual setting, it must be noted that there may exist substantial contextual difference when applied to different countries. For one instance, while the primary domestic determinant of particulates concentration level is the emissions from the road mobile sources, it is likely that some other type of emission sources is the main culprit in other countries. Nevertheless, the most significant implication of this study is that the effectiveness and the efficiency of each policy mix used in individual countries can be examined according to the structure of analyses and process presented in this research. This study was conducted with the purpose of determining the policy effect of the Plan, identifying emissions amount per different type of emission sources that affect particulates concentration level, and evaluating the effectiveness and the efficiency of the relevant policy instruments. Such structure and process is equally applicable to policy design and evaluation in other countries.

Second, while the exact components of policy mix vary among different countries, most of the policy mixes in place include measures that aim at reducing emissions from the road mobile sources. This study has identified effective and efficient policy instruments that can reduce emissions from the road mobile sources, and such conclusion is applicable to other countries. This is because, unlike other type of emissions sources, policy instrument available for consideration must target diesel fueled vehicles by reducing the emission coefficients, travel distance, or the number of those vehicles on the road. Particularly, since reducing the travel distance is a tool of superior efficiency that requires no cost on the part of the government, the strategy can be adopted in countries where it is hard to secure budget for the purpose of alleviating atmospheric particulates concentration. However, while the means of raising the diesel fuel prices was deemed to be efficient in the case of Korea since its social benefit is expected to be larger than the social cost, the actual ratio between social benefit and social cost is strictly contingent upon the specific contextual factors each country face, and therefore, introduction of such measure may be difficult whereas the social cost exceeds the social benefit. As such, careful assessment of expected social benefits and costs must be conducted prior to introduction.

Third, due to the unpredictability inherent in the atmospheric environment, evaluation of policies that aim at improving air quality is conducted on the basis of simulations or statistical methods that compare differences between the states with and without government intervention. Regardless of which method is used, the most important task in estimating policy effect is to obtain a control group that is as similar to the treatment group as possible. In a field that takes the ever-changing atmospheric environment as its subject, more refined evaluation methods must be continuously devised.

Lastly, when dealing with issues characterized by sporadic changes and unpredictability such as air quality, oftentimes, there is insufficient information to determine causality and effectiveness of a policy. Nevertheless, when a policy needs to be consistently pursued on a long-term basis, utilization of scientific information and planning as well as careful coordination among relevant policy instruments are essential. This is because the longer a policy’s time scope is, the bigger the opportunity cost and the cost of policy implementation become. Moreover, discordance among policy instruments due to lack of coordination can cancel out intended policy effects. Of no less importance, proactive efforts must be made to accumulate information on various aspects of a policy regardless of sources - pilot studies as well as existing policy instruments - and to utilize them in all stages of policy analysis, formation and coordination.

However, this study has the following limitations in the research process. First, this study focused on the targets and effects mentioned in the 1^st Basic Plan and examined whether PM₁₀ was reduced, but DPF could have additional effects except for PM₁₀ reduction. Second, secondary generation for PM₁₀ was not considered as a measurement problem. Third, although there are costs paid by the private sector during the plan, there is a limit that considers only government budgets as an input factor due to measurement problems. Fourth, there is a limit to not being able to consider a new emission source by changing the emission calculation method. This limitation was partially considered by controlling the year in the process of estimating the policy effect through panel analysis.

Acknowledgments

This research was supported by the Technology Development Program to Solve Climate Changes through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT (2019M1A2A2103953).

References

• Angrist, J.D., Krueger, A.B. (1999) Empirical strategies in labor economics. Handbook of Labor Economics, 3, 1277-1366. [https://doi.org/10.1016/S1573-4463(99)03004-7]
• BAI (The Board of Audit and Inspection of Korea) (2016) Audit Report: Status of Seoul Metropolitan Area Atmospheric Environment Improvement Project.
• Card, D., Krueger, A.B. (1993) Minimum wages and employment: A case study of the fast food industry in New Jersey and Pennsylvania, NBER working paper. [https://doi.org/10.3386/w4509]
• Chae, H.J. (2009) Effect on the PM₁₀ Concentration by Wind Velocity and Wind Direction. Journal of Environmental and Sanitary Engineering, 24(3), 37-54.
• Cho, S.H., Kim, S.C., Dellink, R., Jang, H.J., Cheon, H.W. (2002) A modeling approach to climate change in Korea III-A CGE analysis of GHG mitigation.
• Choi, K.C., Lee, K.J., Ahn, S.C., Shin, K.W. (2009) An Evaluation and Management Strategy of Environmental Zone for Improving Air Quality in the Seoul Metropolitan Area. Journal Of The Korean Society Of Civil Engineers, 29(6D), 693-702.
• Christensen, T. (2006) Smart policy. The Oxford Handbook of Public Policy. Oxford University Press, Oxford, 448-468.
• Fisher, R.A. (1937) The design of experiments (2nd Ed).
• Flanagan, K., Uyarra, E., Laranja, M. (2011) Reconceptualising the ‘policy mix’ for innovation. Research Policy, 40(5), 702-713. [https://doi.org/10.1016/j.respol.2011.02.005]
• GRI (Gyeonggi Research Institute) (2003) Estimating Social Costs of Air Pollutions and Developing Emission Control Strategies for Kyonggi-Do.
• Han, H., Jung, C.H., Kum, H.S., Kim, Y.P. (2017) The Revisit on the PM₁₀ Reduction Policy in Korea: Focusing on Policy Target, Tools and Effect of 1st Air Quality Management Plan in Seoul Metropolitan Area. Journal of Environmental Policy and Administration, 25(1), 49-79. [https://doi.org/10.15301/jepa.2017.25.1.49]
• Heckman, J.J., Ichimura, H., Todd, P.E. (1997) Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme. The Review of Economic Studies, 64, 605-654. [https://doi.org/10.2307/2971733]
• Heckman, J.J., Willis, R.J. (1977) A beta-logistic model for the analysis of sequential labor force participation by married women. Journal of Political Economy, 85(1), 27-58. [https://doi.org/10.1086/260544]
• Holland, P.W. (1986) Statistics and causal inference. Journal of the American statistical Association, 81, 945-960. [https://doi.org/10.1080/01621459.1986.10478354]
• Hollister, R.G., Hill, J. (1995) Problems in the evaluation of community-wide initiatives. New Approaches to Evaluating Community Initiatives, 1, 127-172.
• Jerrett, M., Arain, M.A., Kanaroglou, P., Beckerman, B., Crouse, D., Gilbert, N.L., Brook, J.R., Finkelstein, N., Finkelstein, M.M. (2007) Modeling the intraurban variability of ambient traffic pollution in Toronto, Canada. Journal of Toxicology and Environmental Health, Part A, 70(3-4), 200-212. [https://doi.org/10.1080/15287390600883018]
• Kang, M.O. (2010) Social costs and policy challenges in the transport sector. Korea Environment Institute.
• Kettl, D.F. (1988) Government by proxy: (Mis?) managing federal programs. Cq Press, 170pp.
• Kim, J.H., Kim, H.J., Lee, J.C. (2020) Effects of Road Environment Improvements in Hydrogen Economy. Korea Research Institute for Human Settlements.
• Kim, W.S., Choi, Y.J., Kim, J.A., Jeon, H.N. (2011) A Study on the Establishment of Pollution Reduction Measures to Achieve the Air Quality Improvement Goals in Seoul. Seoul Institute.
• Kim, Y.P. (2006) Air Pollution in Seoul Caused by Aerosols. Journal of Korean Society for Atmospheric Environment, 22(5), 535-553.
• Kim, Y.P., Lee, G. (2018) Trend of Air Quality in Seoul: Policy and Science. Aerosol and Air Quality Research, 18(9), 2141-2156. [https://doi.org/10.4209/aaqr.2018.03.0081]
• Kim, Y.P., Moon, K.C., Lee, J.H., Baik, N.J. (1999) Concentrations of carbonaceous species in particles at Seoul and Cheju in Korea. Atmospheric Environment, 33(17), 2751-2758. [https://doi.org/10.1016/S1352-2310(98)00313-6]
• King, G., Gakidou, E., Ravishankar, N., Moore, R.T., Lakin, J., Vargas, M., Téllez-Rojo, M.M., Hernández Ávila, J.E., Ávila, M.H., Llamas, H.H. (2007) A “politically robust” experimental design for public policy evaluation, with application to the Mexican universal health insurance program. Journal of Policy Analysis and Management, 26(3), 479-506. [https://doi.org/10.1002/pam.20279]
• Kirschen, E.S. (1964) Economic policy in our time.
• Kwon, S.M., Lee, J.B., Choi, Y.R., Lee, S.M., Eo, S.M. (2016) PM-2.5 Emission Source Inventory and Detailed Monitoring Study: Summary Report. Seoul Metropolitan Government.
• Lafferty, W., Hovden, E. (2003) Environmental policy integration: towards an analytical framework. Environmental Politics, 12(3), 1-22. [https://doi.org/10.1080/09644010412331308254]
• Lee, J.M., Han, S.Y. (2009) A Study on the Comparison between Fuel Taxes and Social Costs in Road Transport. Journal of Regulation Studies, 18(2), 65-94.
• Mats, R., Francesco, F., Massimo, S., Daniela, P., Mara, P., Andrea, R., Nussio, F., Carlo, A.P. (2008) Comparison of regression models with land-use and emissions data to predict the spatial distribution of traffic-relate air pollution in Rome. Journal of Exposure Science and Environmental Epidemiology, 18, 192-199. [https://doi.org/10.1038/sj.jes.7500571]
• MEE (Ministry of Ecology and Environment) (2012) 重點區域大氣汚染防治12.5規劃.
• MOE (Ministry of Environment) (2005) the Seoul Metropolitan Air Quality Management Plan.
• MOE (Ministry of Environment) (2013) the 2nd Seoul Metropolitan Air Quality Management Plan (2015-2024).
• MoLTMA (Ministry of Land, Transport and Maritime Affairs) (2010) Press release: Transportation sharing rates by means of transportation.
• Nawahda, A., Yamashita, K., Ohara, T., Kurokawa, J., Yamaji, K. (2012) Evaluation of premature mortality caused by exposure to PM_2.5 and ozone in East Asia: 2000, 2005, 2020. Water, Air, & Soil Pollution, 223(6), 3445-3459. [https://doi.org/10.1007/s11270-012-1123-7]
• OECD (Organisation for Economic Co-operation and Development) (2008) Environmental outlook to 2030.
• Park, R.S., Han, K.M. (2014) Contribution of long-range transported air pollution from China to particulate matter over Korean Peninsula. Magazine of Korean Society of Hazard Mitigation.
• Rhoads, S.E. (1985) The economist’s view of the world. Cambridge University Press.
• Rubin, D.B. (1974) Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66, 688-701. [https://doi.org/10.1037/h0037350]
• Salamon, L.M. (2002) The tools of government: A guide to the new governance, Oxford University Press.
• SCAQMD (South Coast Air Quality Management District) (2012) Air Quality Management Plan.
• Shin, M.K., Lee, C.D., Ha, H.S., Choe, C.S., Kim, Y.H. (2007) The influence of meteorological factors on PM₁₀ concentration in Incheon. Journal of Korean Society for Atmospheric Environment, 23(3), 322-331. [https://doi.org/10.5572/KOSAE.2007.23.3.322]
• Song, C.H., Carmichael, G.R. (2001) A three-dimensional modeling investigation of the evolution processes of dust and sea-salt particles in east Asia. Journal of Geophysical Research: Atmospheres, 106(D16), 18131-18154. [https://doi.org/10.1029/2000JD900352]
• Sorrell, S., Sijm, J. (2003) Carbon trading in the policy mix. Oxford review of Economic Policy, 19(3), 420-437. [https://doi.org/10.1093/oxrep/19.3.420]
• Stokey, E., Zeckhauser, R. (1978) Primer for policy analysis, WW Norton.
• Weimer, D.L., Vining, A. (2015) Policy analysis: Concepts and practice, Routledge. [https://doi.org/10.4324/9781315663289]
• WHO (World Health Organization) (2021) WHO global air quality guidelines.
• Zhang, Q., Jiang, X., Tong, D., Davis, S.J., Zhao, H., Feng, T., Zheng, B., Lu, Z., Streets, D.G., Ni, R. (2017) Transboundary health impacts of transported global air pollution and international trade. Nature, 543(7647), 705-709. [https://doi.org/10.1038/nature21712]
• 環境省·産業省·国土交通省 (2012) 「低公害車ガイドブック2012」.