The NEP and Measurement Validity The NEP and Measurement Validity
This web site was designed for a course in Research Methods at Cornell University's Health and Human Services Department. For a more detailed description of the New Environmental Paradigm, environmental attitudes, and various scales used to measure environmental attitudes, please see the web site Measuring Environmental Attitudes: The NEP . This page builds on that web site and examines measurement validity addressed in a study on the NEP by Dunlap and Van Liere.
In general, scales are meant to "weigh" an object. In social science, scales are used to "weigh" or gauge a behavior or a personality quality like self-esteem, for example. In the late 1970s, many researchers began to examine environmental attitude and potential ways to "gauge" this concept. Think about it for a moment, if you wanted to measure environmental attitude...
As you can see, it is very tricky to measure--let alone define--something like environmental attitude. Getting the "right" answers to the questions above means operationalizing the construct environmental attitude accurately--defining exactly what you mean by environmental attitude and developing a scale that captures this concept. In other words, developing a scale that is "valid" and accurately able to measure the concept of interest--in this case environmental attitude.
When we think about measurement validity we are essentially talking about construct validity--"the approximate truth of the conclusion that your operationalization accurately reflects its construct" (Trochim web site). Clearly, if we want to measure environmental attitude, we first need to operationalize it or define exactly what we think an individual's environmental attitude might be. Luckily, after a literature search on environmental attitude, we have found one study that has operationalized environmental attitude and developed a scale to measure it.
In the 1960s and 1970s, social scientists' interest in the concept environmental attitude increased. There was a great deal of concern relating to the environment during this decade: the Ohio Cayahoga River caught fire in 1969 capturing national attention; the first Earth Day was held in 1970; the National Environmental Policy Act was signed that same year; and energy conservation became a primary goal in the mid and late 1970s as oil embargoes severely impacted the nation. As a result of these and many other incidents, funding for research directed at the environment and human interaction with the environment became more of a priority.
In 1978, social scientists Dunlap and Van Liere published an article in The Journal of Environmental Education that summarized their efforts to measure a fairly new environmental mind-set they and other researchers believed was becoming a predominant influence. At the time, many social scientists believed that a "paradigmatic" shift--a change in many people's way of thinking--was occurring. People were becoming disenchanted with the so-called "Dominant Social Paradigm," which emphasized human ability to control and manage the environment, limitless natural resources, private property rights, and unlimited industrial growth.
The New Environmental Paradigm, on the other hand, emphasized environmental protection, limited industrial growth, and population control, among other issues. The two social scientists developed the New Environmental Paradigm scale to measure this mind-set. Since its development, the scale has been used in many other studies--both replicating as well as modifying the scale. Many of the studies conducted since then have questioned whether in fact a paradigmatic shift is occurring or has occurred. But most researchers agree that the scale developed by Dunlap and Van Liere is considered one valid measure of environmental attitude and comprises the 12-items listed below. Agreement and disagreement with these statements constitute acceptance or rejection of the NEP.
Validity is "a set of standards by which research can be judged" or "the best available approximation to the truth or falsity of a given inference, proposition, or conclusion" (Trochim web site). Validity can be divided into the following areas: Conclusion Validity, Internal Validity, Construct Validity, and External Validity.
I will not attempt to define all of the above kinds of validity. For a detailed explanation of validity types, please see the Knowledge Base constructed by Professor William Trochim. The kind of validity this web site is concerned with is Measurement Validity--and falls mostly under the domain of Construct Validity. Dunlap and Van Liere attempted to prove that their NEP scale was valid by addressing three elements of measurement validity: construct, predictive, and face validity.
Construct validity is often considered the most difficult kind of validity to achieve--it essentially comprises both predictive and face validity. I will do my best to differentiate different types of validity, but be aware that many kinds of validity overlap and are sometimes difficult to distinguish. By construct validity we mean assessing how well an idea or concept is translated from the "land of theory" in your head to the land of reality into an actual measure or scale. In terms of the NEP, achieving construct validity (and thus achieving measurement validity) meant that Dunlap and Van Liere had to translate exactly what they meant by the new environmental paradigm, as well as develop an actual scale that could accurately measure whether or not this paradigm was in fact part of an individual's attitude-makeup.
There are essentially three conditions that must be met to ensure construct validity:
How did Dunlap and Van Liere achieve construct validity for their scale? Well, in order to meet the first condition, they reviewed the literature to find out more about how others defined construct validity. In addition, they consulted scientists and ecologists to determine if their definition and development of the NEP and scale items met with agreement among experts. [By the way, this discussion on establishing construct validity also overlaps with Content Validity--see Trochim's web site for more information on this kind of validity.] What I have just outlined above falls under Face Validity below, so skip to that section for more detailed information. They met the second condition by predicting results they might achieve with their scale [skip to Predictive Validity for more information]. By achieving predictive validity, they also essentially achieved the third condition--their data from two samples supported their theoretical views.
What is predictive validity? Recall from above that validity means "the approximate truth of the conclusion that your operationalization accurately reflects its construct." The definition of predictive validity can be found in the name--PREDICTIVE. How well does the NEP Scale predict what it theoretically should predict? Well, Dunlap and Van Liere were able to test their scale on two samples--a sample of the general public as well as a sample of environmental group members. They theorized that the environmental group members would score BETTER than the general public on the NEP scale. And they were right--the mean total scale score for environmental group members was 43.8. This compares with a mean scale score of 36.3 for the general public.
What exactly did the two researchers accomplish with two distinct samples and an explicit statement that the environmental group members would score better than the general public? First, they were able to demonstrate meeting the second condition for establishing construct validity. They offered a theory about the results they would receive from these two samples--one sample would score better than another sample based on environmental attitudes. The two researchers were correct in their prediction--the environmental groups did score higher than the general public.
They not only established predictive validity--meeting the second condition for construct validity--they were also able to demonstrate concurrent validity. Concurrent Validity is established when a scale is able to "distinguish between two groups that it theoretically should be able to distinguish between" (Trochim web site). We would expect, as did Dunlap and Van Liere, that environmental group members would care MORE about the environment--and thus score higher on the NEP scale--than the members of the general public.
Dunlap and Van Liere also tested the NEP scale against other measures of environmental attitude and behavior. Their additional scales contained a list of environmental activities and lists of state and federal environmental programs. Respondents were asked to report how often they performed the behaviors and how much they supported the various state and federal programs. The data from these scales was compared with the NEP scale data to see if respondents who performed environmental behaviors and supported government environmental programs also scored well on the NEP scale. Again, Dunlap and Van Liere found that their NEP scale did in fact correlate well with their other measures of environmental attitude and behavior.
Now lets look at face validity--the most subjective and weakest method to establishing measurement validity. Face Validity essentially looks at whether the scale appears to be a good measure of the construct "on its face." As mentioned earlier, Dunlap and Van Liere established face validity by conducting a literature review of what they considered to be crucial aspects of this new environmental paradigm and developing a list of scale items that constitute the paradigm. In addition to the literature review, environmental scientists and ecologists also aided in developing and writing scale questions. By submitting the NEP scale for review by experts in environmental issues, the two researchers were able to bolster face validity.
Lets take a moment to recap exactly what Dunlap and Van Liere did to ensure measurement validity of their New Environmental Paradigm Scale. First, the researchers operationalized--or explicitly defined--what they meant by the NEP and how they were going to measure the NEP. Second, they worked with a panel of experts who approved of the content of their scale. Third, they used two separate population samples--an environmental group sample and a general public sample. Fourth, the researchers also used several scales to measure environmental attitude. Fifth, they theorized how their scale would work in relation to the other scales and with the different population samples--in other words, what kind of data they would expect to get. And last but not least, their data actually supported their theory and predictions.
Remember what the three conditions are to ensure construct validity?
What do you think? Did Dunlap and Van Liere meet these conditions? I would say yes. They provided a theoretical framework for their concept, demonstrated that their operationalization acted the way they predicted it would, and produced data to support their theoretical views.