Figure 1. A representation of a traditional model of a policy decision.
What follows is material excerpted from an article by Terry J. Keating, a graduate
student in public health at the University of North Carolina,
Chapel Hill. The article, Performance Criteria for
Environmental Policy: Cars and Smog in Los Angeles, CA, was
presented at the 18th Annual Research Conference of the
Association for Public Policy Analysis and Management, Pittsburgh,
PA, October 31-November 2, 1996. Material that is quoted directly
is in green. Terry was most kind in giving me the electronic file
for his paper, and permission to use the material here.
Return to Main Environmental Justice Page
Mr. Keating conducted a series of interviews with individuals in
Los Angeles with differing concerns over the issue of motor
vehicle emissions, and found that different people thought about
the problem in very different ways. He developed two diagrams,
which will appear below, representing the way in which risk
assessors thought about the issues as compared to the general
public. First I'll include some of the background provided by Mr.
Keating.
Values, the attributes of our world that we believe are
functionally important, morally good, or personally desirable
[Andrews and Waits, 1978], are derived from our individual
perspectives. Our perspectives are shaped by our age, ethnicity,
gender, socioeconomic status, education, health, religion,
occupation, etc. These differences in personal experience lead us
to different beliefs about what is important, good, or desirable.
These beliefs, or values, determine, not only what we believe
makes good public policy, but also what conditions represent a
public policy problem.
The technocratic institutions or processes in which many public
policy issues are addressed are based on a belief that it is
possible to separate facts from values. This belief is apparent in
the traditional model of policy making in which experts provide
objective technical information to political decision makers who
then evaluate the implications of the technical information in
light of their own values or their understanding of the values of
those they represent [e.g., Quade, 1982; Stokey and Zeckhauser,
1978; Weimer and Vining, 1992]. This traditional model of a policy
decision is diagrammed in Figure 1....
Facts and Values: The Theory
Evaluating complex societal and technological policy alternatives
often requires a great deal of specialized knowledge. To deal with
complex policy problems, our society has developed technocratic
institutions and processes that rely on technical experts to
define the policy alternatives and the criteria by which the
alternatives will be evaluated [Fischer, 1990; Stewart, et al.,
1984; Wheaton and Wheaton, 1972]. While the knowledge and
understanding that individual experts bring to the policy process
are necessary for the development of competent policies, the
values that individual experts bring to the process may not be
representative of the affected public, the policy makers, or
experts with other specialized knowledge [Yankelovich, 1991; Wheaton and Wheaton, 1972].
Instead of the model in Figure 1, Mr. Keating argues that any
facts (entering via the "Environmental Model") will also be
influenced by values (of the person doing the modeling,) so a more
appropriate diagram is given by Figure 2.
This is where I must differ somewhat with Mr. Keating. He also
refers to "presumably objective models of the real world" (part I
did not include.) The term "values" implies that decisions on how
to model a system are made in completely subjective ways. But in
many cases the "values" are arrived at by agreement or consensus
among a large number of experts, and decisions are made before
examining any data, so that the analysis of a particular data set,
say for one chemical, will not be biased relative to the analysis
of the data for some other chemical. The idea is that whatever
biases are inherent in analyzing the risks will be applied equally
to all chemicals (or all of those within a particular class)
rather than letting one's subjective feelings influence the
analysis. (This is the ideal; it is not always achieved.)
While it is not possible to make a completely objective model, it
is possible to limit the extent to which values affect some
analyses, and to explicitly state those "values" when
presenting the results to those who set policy. This later
point is key–it means that we try to avoid having hidden
values or "agendas" influencing analyses of risk.
Returning to Mr. Keating's introduction:
The environmental model in this traditional model of policy
making, however, is not solely the product of objective knowledge.
A model, by nature, is an abstraction, produced by generalization,
deletion, and distortion [Jeffries and Arnold, 1987]. In making
decisions about what phenomena to include and how to represent
them, it is impossible for the modeler to avoid making value
judgments [Shrader-Frechette, 1985]. Thus, the structure and
formulation of the model is a function of the real world and the
values and knowledge of the modeler....
... a number of methods have been proposed to address public
policy decisions in a manner that is both competent and fair in
its use of expert knowledge and its representation of societal
values [Renn and Webler, 1992; Renn, et al., 1993; Andrews, 1992;
Maguire, 1994; Maguire and Boiney, 1994; Keeney, et al., 1990].
However, while many of these approaches recognize the different
value perspectives of different stakeholders, most rely on an
"objective" environmental model developed without the input of
stakeholders, following the traditional model of policy decisions
presented in Figure 1.
Stakeholder groups with contrasting value perspectives are likely
to hold different positions on complex policy issues. These
differences may be due to differences in their preferences or
differences in their environmental models, i.e., their definitions
of the problem. The environmental models may differ in terms of
... the parameter values or probabilities that they think are
appropriate, etc....
The points made in the first paragraph are certainly true. But I
would argue that an "expert" is essentially someone who has been
trained to make judgments about what to include or exclude from
a model. Further, these choices are not (or should not be) made
with a specific intent to influence the outcome of the model, but
with the intent of winnowing out those phenomena that are truly
important. For example, in calculating the motion of an object,
one should include the force of gravity exerted by all sources.
The planet Jupiter does exert some gravitational force on a cannon
ball on Earth. But this force is so small that it can be safely
neglected since other uncertainties, like variations in wind speed
and air pressure, far exceed it. So while excluding this force is
a "distortion" and a "value judgment" in modeling the flight of a
cannon ball, it hardly represents "bias" in the common sense of
the word.
Now my example here is extreme. Sometimes it is much less clear if
a phenomenon is important or not. In this case, however, one can
test the importance of the phenomenon by building two models, one
with and one without it, and seeing what happens. EPA's proposed
Cancer Risk Assessment Guidelines suggest the development of
multiple models, with the choices for each explicitly stated, for
submission to the policy decision process.
The second paragraph suggests that one could perform the modeling
with input from all stakeholders. In the case of air pollution in
LA, this would be every single resident of LA, plus the auto
makers and the oil companies. Clearly, we could at best only
include some stakeholders' input. Who do we pick? How many
from each side? Another point that is glossed over by the first
paragraph is that a model cannot include every single
nuance of biology, or physics. If we are to perform any kind of
quantitative analyses, some assumptions (value choices) must be
made about what aspects of biology to include and which to
exclude. If we try to include too much biology, the analysis will
become impossible. I would argue that only a person who has had
much training in the relevant biology–an expert–should
be involved in making those decisions.
The last paragraph lists some very good reasons why the values of
all stakeholders should be represented in the decision-making
process. However I am disturbed by the suggestion of using "the
parameter values or probabilities that they think are
appropriate." Let's take, for example, the probability of cancer
due to a lifetime exposure to 10 parts per billion benzene in the
air. Should we use the value that just anyone thinks is
appropriate? It is true that value judgments are made when a
group of experts decides what biological processes they think are
important in calculating this probability from the existing data,
and what assumptions are appropriate, but I would think that one
should not include the opinion of a corporate executive of a car
company who may have had one course in biology twenty years
ago.
Mr. Keating then describes in some detail the context of the problem: car-related air pollution in LA, and his methods for
assessing the views of various stakeholders. This involved two
rounds of interviews in which the people were asked to list
objectives that they thought important for policy decisions. These
objectives were divided into two basic categories, "fundamental
objectives" and "attributes." The fundamental objectives are over-
arching goals like "Minimizing (adverse) Economic Impacts" while
the attributes where specific, measurable criteria that apply to
the fundamental objectives, like "Fuel Costs." Depending on how
many specific attributes, and which fundamental objectives, a
person mentioned, the importance of the relevant fundamental
objective was determined.
In-depth interviews were conducted with 26 participants from eight
organizations or perspectives: a state air quality regulatory
agency (referred to in the figures and tables as state) , a local
air quality regulatory agency (local), a metropolitan planning
agency (plan), a municipal government (govt), an environmental
group (eco), an automobile manufacturer (auto), a petroleum
company (oil), and an elected representative (elect). The
interviews were conducted in two rounds: In the first round, the
participants were interviewed in groups together with the other
participants from the same organization. In the second round, the
participants were interviewed individually. The two rounds of
interviews were conducted about ten weeks apart, and only one
participant from the first round was not able to participate in
the second round.
Cars and Smog: The Context; Research Methods
Results
Stakeholder groups were able to specify more detailed attributes
for some objectives and not for others. Minimizing air pollution
and minimizing economic impacts were included as objectives in all
of the hierarchies. However, approximately 20 different
attributes, shown in Table 1, were suggested as measures for each
of these objectives by different groups. Thus, while there is
agreement that these are important considerations, there is
disagreement about how to define or bound these complex
effects.
| "Minimizing Air Pollution" | "Minimizing Economic Impacts" |
|---|---|
| Ambient Standards | Net Societal Benefits |
| Emission Reductions | Capital Investment |
| Cancer Risk | Infrastructure Investment |
| Non-Cancer Risk | Profit/Cost of Production Ratio |
| Population Exposure | Cost of Regulatory Compliance |
| Mortality | Job Opportunities Missed |
| Physiological Changes | Jobs Created/Lost by Sector/Region/Ethnicity |
| Hospitalization | Interstate/International Trade Balance |
| Restricted Activity | Health Care Costs |
| Public Perceptions | Worker Productivity |
| Visibility | Crop Damage |
| Odor | Liability Costs |
| Irritation | Cost Effectiveness ($/ton) |
| Global Climate Change | Availability of Consumer Choice |
| Materials Damage | Lifestyle or Operational Change |
| Crop Yield | Vehicle Purchase and Maintenance Costs |
| Forest Impacts | Fuel Costs |
| Wildlife Impacts | Change in Government Revenue |
The structure and content of the objective hierarchies constructed
by the different stakeholders in the first round of interviews
were quite similar. Many of the objectives were common to the
different hierarchies, but the objectives were not always
organized in a similar manner. Figure 3 presents a comparison of
the fundamental objectives identified by each of the groups. The
lightest shading indicates that the elements of the listed
fundamental objective were identified by the group but organized
under another fundamental objective. The intermediate shading
indicates that elements of the listed fundamental objective were
identified by the group but as a separate fundamental objective.
The darkest shading indicates that the listed fundamental
objective was identified by the group as a fundamental objective.
The extent of shading in this figure indicates the extent of
agreement about what elements should be included in the objectives
hierarchy. The intensity of the shading indicates the extent of
agreement about why these elements are important.
The results displayed in Figure 3 imply that the stakeholders tend
to agree about what aspects of the problem are important, but do
not always agree about why aspects of the problem are important.
For example, "attaining the National Ambient Air Quality Standard"
was an attribute that was suggested to measure four different
objectives with different levels of specificity: 1) minimizing air
pollution, 2) minimizing the adverse health effects of air
pollution, 3) complying with legal requirements, and 4) maximizing
the perceived quality of life in the region. In the case of the
latter two responses, different attributes were suggested for
measuring air pollution and its adverse health effects.
Mr. Keating goes on to discuss further analysis of the responses,
and to interpret the results. I will not include any more material
from the paper. However, subsequent to the paper, Mr. Keating
presented two diagrams showing the conceptual models of general
citizens, and the South Coast Air Quality Management Division
(SCAQMD) Staff. These were presented at a meeting of the Research
Triangle Chapter of the Society of Risk Analysis in February,
1997, and are shown below. I will leave their interpretation to
you, the reader.
Figure 4. Conceptual model of
citizens' perspective.
While I do disagree in some points with Mr. Keating, as I argue
above, I think that these diagrams do a great job of representing
"reality" and why communication between government officials or
scientists and general citizens can be difficult and contentious.
Individuals from the two groups can be using the same words, but
the meaning of the words exists in the context of these models. It
is my hope that these illustrations will provide a better
understanding of perspectives on the problem, and a framework for
constructive dialogue.