Tuesday, September 30, 2014

American public opinion’s awareness and support for the ICC: a comparison between the Chicago Council and the online IPSOS surveys

The issue

This paper aims to assess the current trend in American public opinion’s support and acceptance of the ICC by comparing the results of the most recent opinion polls concerning the ICC in 2010 and 2012. The surveys considered have two different sponsors which have used different methods in conducting them.
This comparison of the two approaches is intended to check if the separate methods applied have resulted in different or similar results and to determine the significance of the differences. The sources analyzed are the Chicago Council Survey and the IPSOS Public Affairs. The latter is a social research and corporate reputation specialists group which is helping the American Bar Association-ICC project to determine and analyze US public opinion about the Court.
The Chicago Council survey and its findings

The Chicago Council on Global Affairs has conducted different surveys on this topic respectively in 2008, 2010 and 2012 whose results show a significant, although modest increase in the number of American people interested in and sensitive to the involvement of the US in the Court’s work and to potential ratification of the Court’s Rome Statute.

From 68% of Americans willing to ratify the Rome Statute in 2008 [1], in 2010 the percentage reached 70% of those who not only would ratify the Statute [2] but also would support any attempt to strengthen the ICC as international institution[3]. The last survey conducted by the same organization in September 2012 did not show any difference from the previous figure, confirming the 70% of Americans are in favor of the Court.(p.44)

Methodology and criteria applied by Chicago Council

The Council’s method in its biennial surveys on American attitudes towards US foreign policy is based on a direct questionnaire involving a representative national sample of adults, including an additional sample of Hispanic respondents. The margin of sampling error for the full sample varies and fluctuates and so does the amount of time needed to complete it.
In 2008 the Council asked the respondents if the US should participate in international treaties such as the treaty establishing the International Criminal Court. The result was that 68% of Americans were in favor.
In 2010, The Council limited its survey about the ICC to two questions:
1. Should the US participate in the Rome Statute?
2.   Do you think that international institutions, such as the International Criminal Court, should be strengthened or not?
As above mentioned, the percentage increased to 70%. In the 2012 survey, the Council repeated the question regarding the participation of the US in the ICC’s treaty and the 70% figure did not change.  

In its polling data research, the Council usually assigns the task of conducting the survey to polling and research firms in the United States. For instance, in the 2012 report, the GfK Custom Research, a market research firm based in California, fielded the survey to a total of 3,135 respondents. Of these there were 2,747 for the general population sample and 388 for the eighteen to twenty-nine-year-old additional sample. Those completing the survey were 1,984: 1,790 general population, 194 eighteen to twenty-nine-year-old additional sample. This yielded a completion rate of 63 percent: 65% general population, 50% eighteen to twenty-nine-year-old additional sample. The survey had a total sample size of 1,877 American adults. Of the total 1,984 completed cases, 107 (88 general population, 19 eighteen to twenty-nine-year-old in a special sample) were excluded based on four predetermined criteria. The survey was fielded using a randomly selected sample of GfK’s large-scale, nationwide research group called KnowledgePanel®.

Unlike groups whose members decide voluntarily to participate in a survey, in standard polls like the one by the Chicago survey, individuals can become members of the group to be interviewed only after being randomly selected, no one can just volunteer to be a member. This selection method provides results which more near represent the US population with a consistently higher degree of accuracy than results obtainable from volunteer opt-in groups.
In the Chicago Council  poll, the group of people interviewed was recruited using random digit dialing (RDD) telephone sampling and also address-based sampling (ABS) for households without landline telephone numbers.
Individual residing at randomly residing addresses were invited to join the panel through mailings and for non responders a telephone call in English and in Spanish was made when a telephone number could be matched to the sample address.
RDD provides some probability of selection for every U.S. household with a telephone.
Household members who were randomly selected could indicate their willingness to join the panel by returning a complete acceptance form in a postage-paid envelope or calling a toll-free hotline or visiting a dedicated recruitment website.  
Households that agree to participate in the panel are provided with free Web access and an Internet appliance (if necessary), which uses a telephone line to connect to the Internet and uses the television as a monitor. Thus, the sample is not limited to those in the population who already have Internet access.

The distribution of the sample closely tracked the distribution of United States Census reports for the U.S. population eighteen years of age or older on age, race, Hispanic ethnicity, geographical region, employment status, income, education, etc. To reduce the effects of any non response and non coverage bias in panel estimates, a post statistical adjustment of the sample weights is applied so that sample weights sum up to the population sizes within each group.
This is possible by using demographic distributions from the most recent data from the Current Population Survey (CPS).
The post stratification variables include age, race, gender, Hispanic ethnicity, and education.
The completion rate for respondents is 65% with some variation depending on survey length, topic, and other fielding characteristics. In contrast, online groups of volunteer respondents typically achieve a survey completion rate in the 2% to 16% rate.[4]

The IPSOS Public Affairs opinion polls about the ICC
Ipsos is a global independent market research company specialized in six areas including international justice and global affairs. This specialization is also one of the reasons why the American Bar Association chose IPSOS to conduct surveys related to its ICC project. The ABA-ICC project was established to raise awareness in the US about the work of the ICC to prosecute those responsible for the most heinous crimes.

Ipsos methodology

The Ipsos survey used an online voluntary questionnaire. Its results were weighted to balance demographics and ensure that the sample's composition reflected that of the U.S. adult population according to Census data and to provide results approximating the population from which the volunteer respondents came. To reduce the effects of any non-response and non-coverage bias in panel estimates, a post statistical adjustment of the weights was applied so that they sum up to the population sizes within each group. Statistical margins of error are not applicable to online polls because they are based on samples drawn from opt-in online panels, not on random samples that mirror the population within a statistical probability ratio. In addition to that, the self-selection of respondents is likely to attract more people opposed to the Court than the traditional random selection does.

Ipsos findings

In February 2014, IPSOS Public Affairs conducted an online survey interviewing a national sample of 1,005 adults and asking a more detailed series of questions regarding specific aspects of the ICC than the Chicago Global Council survey. However, Ipsos filtered out some data so as to include just those respondents reporting that they know ‘A great deal’, ‘A fair amount’, or ‘ A little bit’ about the ICC at question 1 cutting out those who say they know ‘Nothing at all’. This reduced the filtered base size to 379 adults called audience “aware of ICC “. The filtering was based on a question that asked the respondents how much they knew about the Court and provided four possible answers. The result was that only 4% of the sample panel knew “a great deal” in contrast with the 60% of it who knew nothing at all. 8% of it knew “a fair amount” and 28% a” little bit”.

Question 2 gave five statements panelists could agree with or not.
1. The US is currently a member of the ICC,
2.  It is important for the United States to participate in international organizations that support human       rights and that hold individuals accountable for mass atrocities
3. Joining international organizations concerned with human rights and holding individuals accountable for mass atrocities is a risk to the United States because it could hurt our autonomy.
4. We should dedicate US resources (financial military, intelligence etc) to international organizations that support human rights and hold individuals accountable for mass atrocities.
5. It is not in our best interest to dedicate US resources (financial military, intelligence etc) to international organizations that support human rights and hold individuals accountable for mass atrocities.
The results were that 68% of respondents did not know if the US is currently a member of the ICC, 8% of them knew that it is not a member and 28% thought that it is a member.
By contrast, in the group of people aware of the ICC, the majority thought that the US is a member, (52%), a minority knew that it is not (6%) and the rest did not know the answer.
Answers to the other statements regarding the participation of the US in human rights organizations corroborate the favorable trend expressed by other public opinion polls with 60% of Americans without knowledge of the ICC, who would like the US to participate and join human rights organizations, and with 71% of those aware of the ICC who share the same opinion.
Furthermore, turning to a statement about possible risks in American participation in the ICC, question 3, only 24% of the unaware audience agreed with this statement, whereas 39% did not know and 37% disagreed with it. The situation changed in the audience aware of the ICC, where the majority disagreed with the statement (44%) and 39% agreed with it.

In the following series of questions, interviewers inserted in the survey a more detailed paragraph with information about the ICC and its goals and the American opinion about a future American involvement in it. This information identified issues in the current debate over ICC such as sovereignty issues and politically motivated prosecution of American soldiers at the ICC. Results show that the majority of Americans unaware of the ICC, when informed about the debate, did not know if the US should become a member of the ICC (42%), 34% thought that it should and 24% thought that it should not join the Court.
On the contrary, among those aware of the ICC, the majority agreed with an American membership in the Court, 25% disagreed and 23% did not know if the US should or should not join the Court.

Moreover, when people were asked about the different degrees of American participation in the ICC’s work, they were provided with different alternatives.
The first was the possibility to give moderate resources to support ICC’s actions without formally joining the Court. About this option, 44% of all respondents agreed and 19% disagreed, while 37% did not know.
Among the aware group, 57% agreed, 22% did not agree and 21% did not know.
The second option concerned the sovereignty issue. A majority of respondents did not know if joining the Court would compromise American sovereignty (43%), 31% thought it would not and the rest of Americans (26%) agreed that US participation in the ICC would affect American sovereignty. Instead, among those previously aware of the ICC, 40% disagreed with the statement, 36% agreed and 34% did not know if American sovereignty would be compromised.

The third option asked about a stronger involvement of the US governmental resources to support ICC actions without joining the Court.
Results are similar to the ones of the previous question: the majority of all respondents did not know if this could be a viable choice (43%); 28% thought that the US should become more involved without joining the Court and 29% disagreed with this option. Among those aware of the ICC, 44% agreed with the statement, 30% disagreed and 26% did not know.
The last option asked if the Court should become a full member of the ICC.
Similarly to the other options, the majority of respondents did not know what to answer (47%), 22% agreed with a full membership option and 31% disagreed. Consistent with the previous results, among those aware of the ICC, there is an almost equal distribution of the answers as 36% agreed with a full membership 34% disagreed and 30% did not know if the US should become a full member of the ICC.

The last set of questions involved the situation in Kenya and the charges against its President Kenyatta and the Deputy President Ruto. In particular, people were asked if sitting heads of states should have immunity from the ICC for mass atrocities charges, and whether President Kenyatta and Deputy President Ruto should participate in person in the trial, or through video or defense lawyers.To the first question, the vast majority of Americans (60%) replied that they would deny immunity to sitting heads of States, whereas only 9% would grant it during their time in office,
Among the aware of the ICC audience, a similar pattern can be identified with 66% of Americans against the immunity for sitting heads of States and just 18% in favor of that. Similarly, regarding President Kenyatta and Deputy President Ruto, 51% of all Americans thought that they should stand trial in person, 18% thought that they could participate via video conference, 9% thought that they could just send their lawyers to the Hague and 28% did not know the answer. Among the restricted sample of those aware of the ICC, for 58% of Americans they should stand trial in person, for 24% they could do the same through a video conference, for 15% their lawyers could participate on their behalf.

From this short summary of the IPSOS survey conducted in 2014, it can certainly be argued that although a majority of Americans will be willing to support the ICC without a direct membership, a wide proportion of them is still unsure and uncertain of what could be the best option for the country, and a considerable proportion of people disagree with the US involvement in the ICC’s actions.
Furthermore, turning to those who know the Court, the difference between the supporters and detractors of the ICC is not very remarkable, whereas a good percentage of people could not make a choice.        

Ipsos vs Chicago Council.
Similarities and differences in the opinion poll results and methodology

Comparing the results of the two opinion poll data survey, it is easy to identify three main differences in method and consequently in results.
In the Ipsos report, only 22% of all respondents answered in a positive way, while 31% were against and a majority of people could not make a choice. In the Chicago Council report instead, 70% of rspondents would welcome a ratification of the Rome Statute

The Chicago Council method, moreover, is based on randomly sample of selected people (knowledge panel). They are chosen among the American population and they are selected using a variety of criteria: such as age, race origin, income. (first difference)
The panel is then composed by respondents who accepted to be interviewed online and were given access to the Internet in case they do not have it.
The accuracy thus is very high as well as the quality of the survey as every sample unit has a known probability, and therefore it is clear of the errors coming from voluntary opt-in groups.
 The Ipsos survey was based on online volunteer panelists who decide to take the survey without being chosen before and as a result there is not a statistical probability ratio that can mirror the American population.

The second difference is the presence of verification key checks that control the way respondents take the survey. In the Chicago Council report, for instance, people who spend less than ten minutes to finish the survey were excluded. The Ipsos survey did not use any control indicator and the sample considered is significantly small as people who take the survey may decide to opt out during the survey.
The third difference is the completion rate or response rate. The completion rate is the number of people who answered the survey divided by the number of people in the sample. It is usually expressed in the form of a percentage. For the Chicago Council survey it is 65%, for Ipsos online polls is only 15%.

In results, what stands out most from this comparison is the deep gap between the Chicago Council’s and Ipsos’s findings regarding the question of the full participation of the US in the Court.
In the Ipsos survey the question most comparable to the Chicago Council’s report is whether the US should become a full member of the International Criminal Court.
Only 22% of all respondents answered in a positive way, as compared to 31% against and a majority of people who could not make a choice. In the Chicago Council report instead, 70% of respondents would welcome a ratification of the Rome Statute.

Assessing the trend
In conclusion, the Chicago Council report based on the randomly selected group may be more reliable and accurate than the Ipsos one because of the latter’s opt in method and therefore, the 70% of Americans who would be willing to ratify the Rome Statute of the ICC could be a more accurate assessment.
However, the more recent Ipsos survey may be showing a real change over time in US attitude toward the ICC. Also, the questions addressed by this last report were more specific and detailed than the Chicago poll, and therefore more likely to provide some insight into public knowledge about it.

Written for AMICC  by Miriam Morfino on 09/19/2014


[1] Cfr., Chicago Council Survey (2008), Anxious Americans seek a new direction in United States Foreign Policy, p.13 fig.4.
[2] Cfr., Chicago Council Survey (2010), Constrained Internationalism: Adapting to New realities, p. 16 fig. 9.
[3] Cfr., Ibidem p. 22 fig. 15.
[4] Cfr., GFK 2013, Knowledge Panel® Design Summary  (p.1-5).

No comments: