QUESTION ORDER EFFECTS IN SUSTAINABLE BEHAVIOR NORM MEASUREMENT : GENDER DIFFERENCES

Sustainable behavior is an important and increasingly popular field of research, but the question order effects that may occur in descriptive and injunctive sustainable behavior norm measurements remain largely unexplored. Even less attention is drawn to gender differences that may be present in question order effects. In the present study, we have investigated the question order effects and their gender differences in measurements of descriptive and injunctive social norms directed at sustainable behavior. For the purposes of the study, three questionnaires were constructed, each questionnaire consisting of 16 items half of which were directed at injunctive and half at descriptive social norms. The questionnaires differed only in the order in which the items were presented. Engineering, art, and social science students from various Lithuanian universities participated in the study (N = 296). The participants were equally distributed by gender, their mean age was 20 years (SD = 1.5). Participants were asked to fill in the questionnaires during lectures, and each participant was randomly assigned to fill in one of three types of questionnaires. The results have shown that the evaluations of injunctive and descriptive social norms were sensitive to the question order. When priming participants with one type of social norm, the evaluation of the other type of social norm was affected. Presenting questionnaire items in random order did not cancel priming; rather, it made both types of social norms salient at the same time, and therefore norm measurements of this group did not differ from those that were obtained under priming. The further analysis indicated that the question order effects were present only in the female sample. We recommend presenting questionnaire items directed at descriptive and injunctive social norms in random order. When measuring just one type of social norm, we would recommend adding items directed at the other type of social norm as well, in order to obtain data that would be more comparable with other studies.


Introduction
There are many circumstances under which unwanted and hard to predict question effects can occur (Holbrook, Krosnick, Moore, & Tourangeau, 2007;Yang & Wyckoff, 2010;Lasorsa, 2003).It is known that question order (Ramirez & Straus, 2006;Schwarz & Hippler, 1995;Siminski, 2006), visual presentation (Christian & Dillman, 2004), structure, length, word familiarity (Blair, Sudman, Bradburn, & Stocking, 1977), and response order (Chan, 1991;McDermott & Frankovic, 2003) can have an impact on respondents' answers.The present study is concerned with the type of order effects that are induced by the relative position of an item in a questionnaire and the context of the previous questions (Ramirez & Straus, 2006;Siminski, 2006).These types of order effects can easily distort and thus invalidate the obtained data (Yang & Wyckoff, 2010).Therefore, for researchers striving for objectivity, question order effects become an important factor in their research and the research of others.
Question order effects occur when a respondent reacts to a question while having been primed with previous questions (Yang & Wyckoff, 2010;Lasorsa, 2003;Schröder & Thagard, 2013;Van De Walle & Van Ryzin, 2011).Responding to a question involves remembering and making judgments (Yang & Wyckoff, 2010).However, in answering the further questions this process can be shortened if some thoughts and judgments are already salient and ready to be used from the answers to the preceding questions (Yang & Wyckoff, 2010;Van De Walle & Van Ryzin, 2011).Thus, answers to questions on a similar topic can largely depend on the answer to the first question or prior judgments in general.
Descriptive norms depict the perceived behavior of others (Brauer & Chaurand, 2010;Cialdini et al., 1990;FeldmanHall et al., 2012;Gerber & Rogers, 2009;Reno et al., 1993).By observing the behavior of others, i.e. by evaluating the descriptive norms of behavior, one can determine which behaviors are actually allowed or frowned upon.This serves as a kind of heuristic to determine the most beneficial and adaptive behavior in a given situation (Cialdini et al., 1990).This may even lead to people behaving the way they perceive that others behave, even if this behavior seems to disagree with injunctive norms (Croson, Handy, & Shang, 2009;Gerber & Rogers, 2009).
There is ample evidence suggesting that priming with certain social norms can elicit behavioral change (Carrico & Riemer, 2011;Ferguson, Branscombe, & Reynolds, 2011;Goldstein, Cialdini, & Griskevicius, 2008;Huffman, Van Der Werff, Henning, & Watrous-Rodriguez, 2014).Since filling in a questionnaire is a type of behavior, one is left with the question: what order effects can occur when one is primed with a certain type of social norm before answering questions directed at another type of social norm?That is, what order effects can occur if one is primed with questions directed at injunctive norms before having to answer questions directed at descriptive norms, and vice versa?
In the present study, we aim to measure the question order effects of descriptive and injunctive social norms describing sustainable behavior.Sustainable behavior can be defined as socially responsible behavior directed at preserving the natural environment (Luchs & Mooradian, 2011).Sustainable behavior is an important and increasingly popular field of research (Ernst & Wenzel, 2014;Pelletier, Lavergne, & Sharp, 2008); therefore, methodological insights into how to improve the research instruments used in these types of studies can be very valuable.Various social norm measurements are used extensively in studies investigating sustainable behavior (Aguilar-Luzón, García-Martínez, Calvo-Salguero, & Salinas, 2012;de Groot & Steg, 2010;Interis & Haab, 2014;Şener & Hazer, 2008;Smith et al., 2012); however, the question order effects that may occur in these measurements remain largely unexplored.
Even less attention is paid to gender differences in question order effects, even though there are some data to suggest that in some cases males might be less affected by question order effects and to these gender differences might be related to the topics of the questions (Yang & Wyckoff, 2010;Kalton, Collins, & Brook, 1978).For example, the topics that are of greater importance to females and that elicit a more emotional reaction from them may induce question order effects (Yang & Wyckoff, 2010).Furthermore, gender differences may be present in sustainable behavior (Luchs & Mooradian, 2011); females generally tend to be more concerned about the environment.Therefore, it is reasonable to explore whether there are gender differences in question order effects in measuring social norms directed at sustainable behavior.

Method
Instruments.The questionnaires for the study were constructed similarly to the previous research (Poškus, 2014).For the purposes of the study, eight statements about sustainable behavior were constructed.These statements were as follows: "one should recycle trash"; "one should use biodegradable products"; "one should not litter"; "one should choose eco-friendly products"; "one should avoid driving a car just by oneself"; "one should use energyefficient electronics"; "one should conserve water"; "one should choose transportation that consumes less fuel".Each of these eight statements was reworded to be directed at descriptive and injunctive social norms.For example, the statement "one should recycle trash" was reworded into "people should recycle trash" to represent an injunctive norm, and into "most people recycle trash" to represent a descriptive norm.Using these newly constructed items, three questionnaires were constructed.Each questionnaire consisted of 16 items, half of which were directed at descriptive sustainability norms and half at injunctive ones.The questionnaires differed only by the order in which the statements were presented: 1) items directed at descriptive norms presented first, 2) items directed at injunctive norms presented first, 3) items presented in random order (the order of items did not differ among the participants).
The questionnaire items were scored on a Likert scale ranging from 1 (completely agree) to 5 (completely disagree).The sum of each norm scale was computed by adding up all of the values of the scale and dividing by the number of items.In this questionnaire, the lower the scale score, the more a social norm is expressed.In most cases, the data were not normally distributed, so we opted to use non-parametric statistics.The internal consistency (alpha) of the question-naire descriptive and injunctive norm scales varied among experimental conditions; however, in most cases the scales were fairly consistent.When the questionnaire items directed at injunctive norms were presented first, the internal consistency was 0.72 for the injunctive norm scale and 0.66 for the descriptive norm scale.When the items directed at descriptive norms were presented first, the internal consistency was 0.82 for the injunctive norm scale and 0.74 for the descriptive norm scale.When the questionnaire items were presented in random order, the internal consistency was 0.81 for the injunctive norm scale and 0.54 for the descriptive norm scale.
Participants.A convenience sample of 296 engineering, art, and social science students from various Lithuanian universities participated in the study.The participants were equally distributed by gender, their mean age was 20 years (SD = 1.5).Participants were almost equally distributed among the experimental conditions, with a difference of at most 3 participants among the conditions.
Procedure.Study participants were asked to fill in the questionnaires during lectures.Each participant was randomly assigned to fill in one of three types of questionnaires.They were informed that the questionnaire was completely anonymous and that they should be as honest as possible.They were asked not to look into each other's questionnaires and not to discuss the questionnaire with their friends until after everyone had completed the survey.Participants finished filling in the questionnaires in roughly 7 minutes and then were debriefed about the aim of the study.

Results
In most cases, the data were not normally distributed; therefore, non-parametric tests were used for hypothesis testing.In order to determine whether the question order effects in the descriptive and injunctive social norm measurements were present in the experimental conditions, we conducted a Kruskal-Wallis test.It was found that injunctive (x 2 (2) = 9.51; p < 0.01) and descriptive (x 2 (2) = 14.54; p < 0.01) norm measurements differed significantly among the experimental conditions.To further analyze the data, we conducted pairwise comparisons of both injunctive and descriptive norm scales in all experimental conditions.For this purpose, we used the Mann-Whitney U test, correcting the significance for multiple comparisons (Table 1).
In the case of injunctive norm measurements, when the items directed at descriptive norms were presented first, as well as when items were presented in random order, the norm measurements differed from the condition where the injunctive items were presented first.This shows that when respondents were primed with descriptive norms, their reported injunctive norms shifted towards a more strict view of what people should do.It is important to note that presenting the questionnaire items in random order did not cancel the priming.In fact, presenting items in random order had exactly the same effect on injunctive norm measurements as if the respondents were primed with descriptive norms first.
A similar pattern can be observed in descriptive norm measurements.Descriptive norm measurements did not differ among the groups that received injunctive norm items first or that received items in random order.This shows that presenting items in random order does not actually cancel priming; rather, it makes salient both types of social norms at the same time.
In order to determine whether the questionnaire scales were rated differently by male and female participants, two Mann-Whitney tests were conducted.The injunctive norm measurements in the condition when injunctive norm items were presented    first (when no priming was present) did not differ between the genders (U = 1078.5;Z = -1.02;p > 0.05).Similarly, descriptive norm measurements, in the condition when descriptive norm items were presented first, were also compared by gender and no significant differences were found (U = 1004; Z = -1.40;p > 0.05).This shows that when no priming is present, male and female participants' ratings of the questionnaire's observed under priming.It was also found that question order effects were present for both types of social norms; however, Poškus (2014) found that injunctive norm measurements did not significantly differ based on their relative position in the questionnaire.This presents an interesting problem: what question order should researchers choose when measuring injunctive and descriptive social norms?It may be the case that while measuring just one type of social norm one might get substantially different data than if both injunctive and descriptive norms are measured.In the light of this problem, it would be highly recommended not to compare norm measurements among samples if their measurement procedures and instruments are not identical -this may be especially important for meta-analyses.However, if one wants to compare norm measurements obtained in different conditions (e.g., different questionnaire, different question order), it is unclear what adjustments to data should be made because of the unpredictable nature of question order effects (Holbrook et al., 2007;Yang & Wyckoff, 2010).

Table 2. Pairwise comparisons of norm scales between experimental conditions in the
It is noteworthy that the present study differs from that of Poškus (2014) in that in the present study we chose to use the sustainability social norms as our stimulus.We also opted to construct less statements, making the overall questionnaire shorter, which may have been the cause of the results that were observed in the random question order experimental condition.However, to determine this with certainty, more research is needed.
One of the aims of this study was to investigate the possible gender differences in question order effects.Previous research descriptive and injunctive norm scales do not differ and the test is not biased towards one gender.
To test whether gender plays a role in question order effects, we conducted Kruskal-Wallis tests for the male and female samples separately.The tests indicated that injunctive (x 2 (2) = 10.68;p < 0.01) and descriptive (x 2 (2) = 15.68;p < 0.01) norm measurements among the experimental groups differed significantly only in the female portion of the sample, thus indicating that question order effects may be stronger for female populations.To further analyze the data, we conducted pairwise comparisons of the norm scales among experimental conditions in the female sample (Table 2).
The order effects that were present for the whole sample remained the same for the female portion of the sample; however, the effect sizes increased.This indicates that, at least in the present sample, females are more susceptible to question order effects when measuring the descriptive and injunctive social norms of sustainable behavior.

Discussion
We have found that question order effects do occur when measuring injunctive and descriptive social norms, sustainable beha vior norms not being an exception.However, while the previous research which investigated question order effects in measuring descriptive and injunctive social norms had found that presenting items in random order negates order effects in norm measurements (Poškus, 2014), we found that presenting items in random order actually made both types of social norms salient at the same time.Presenting items in random order produced results similar to those on the topic is quite scarce and offers no general insight, except that some specific topics, which may be of greater importance to females, may induce question order effects (Yang & Wyckoff, 2010), while in other cases males might have more clear opinions which make their responses more resilient to question order (Kalton et al., 1978).We have found that gender plays a substantial role in question order effects when measuring descriptive and injunctive sustainable behavior norms; order effects were present for the female sample but were absent in the male sample.
As all research, the present study is not without limitations.Firstly, the results of this study, although providing an important insight, are hard to generalize, since question order effects in measuring descriptive and injunctive social norms may vary by the topic and/or by scale length.Secondly, we used a convenience sample, which partly prevents from generalizing the results to broader populations.Also, we have utilized a questionnaire designed just for the purposes of this study, and it should be noted that, in the random question order condition, the descriptive norm scale was not as consistent as under other conditions.Although we provide insights on how similar questionnaires should be constructed and ordered, we are unable to comment on actual instruments used in measuring social norms.
Gender differences in question order effects are hard to explain, especially in the case of sustainable behavior social norms, since the topic is not of bigger importance to either gender; rather, sustainability concerns everyone.However, since sustainable behavior can be predicted by personality traits which, in turn, can be predicted by gender (Luchs & Mooradian, 2011), it may be the case that gender and personality factors drive the susceptibility to question order effects when evaluating sustainability social norms as well.In other words, the tendency to be more susceptible to the question order effects may be a result of innate factors and not of social contexts or cognitive states.This, however, remains to be tested by further research.
Most researchers suggest that, in order to get less biased data, one should present questionnaire items in random order (Poškus, 2014;Ramirez & Straus, 2006;Siminski, 2006).In the present study, presenting questionnaire items directed at descriptive and injunctive sustainability norms in random order made both norm types salient and thus influenced responses to both types of questionnaire items.We would nonetheless recommend presenting questionnaire items directed at descriptive and injunctive social norms in random order.This, we believe, is a better alternative than priming respondents with just one type of social norm and producing a one-sided bias.If researchers are interested in measuring just one type of social norm, we would recommend considering adding items directed at the other type of social norm, in order to obtain data that would be more comparable with other studies.order of questions in a survey on citizen satisfaction with public services: Lessons from a split-ballot experiment.Public Administration, 89 (4), 1436-1450. doi: 10.1111/j.1467-9299.2011
Įteikta 2014 12 15 Injunctive norms provide a basis for one's behavior by describing what is allowed by society and what is not, what behaviors are appropriate and what are not