Exposing the fallacy of Spain’s “full democracy”

Every year numerous media outlets report on indices such as those published by The Economist, which rank Spain as a “full democracy”, scoring higher than countries like France and Belgium. These reports counter claims of repression from organisations and official bodies such as Amnesty International, the UN Working Group on Arbitrary Detention and the Council of Europe’s Human Rights Committee, which appear to have little bearing when it comes to compiling these indices.

A team of Catalan economists has now published a report (scroll down for English version) on the reliability of such studies which concludes that these indices lack sufficiently solid mechanisms to account for the biases, prejudices and outright attempts at manipulation that can influence the opinions of the experts involved. Following the report, two hundred academics, jurists and other public figures have signed a petition objecting to Spain’s status as a “full democracy”.

They argue that the numerous “recent human rights violations” are inconsistent with this classification. According to economist Eduard Gracia, who spoke during the presentation of the report, “it’s unacceptable that opinion-based indices should be presented as if they were factual, with as much authority as the UN’s”. He went on to say that “like they’ve done in the case of Spain, they provide poor evidence that doesn’t disprove much more solid, readily-available data on the situation.”

The report is entitled “Prejudice and manipulation under the cloak of science: Spain’s dubious ‘full democracy’ rating and the shaky foundations of democracy indices”. It was written by Eduard Gracia, Oriol Martínez, Jaume Pérez, Josep Reyner, David Ros and Lluís Verbon. The report examines the methodology employed by two of the leading indices on democratic quality, The Economist Intelligence Unit (EIU) and the University of Gothenburg’s Varieties of Democracy Institute (V-Dem). A 2020 report by the latter ranked Spain ahead of Finland, Ireland, the United Kingdom, the Netherlands, France, Germany, Iceland and Austria.

The methodology underlying democratic quality indices

In addition to conducting quantitative analyses of these indices, Gracia and his team asked those who compile the EIU and the V-Dem to specify the methodology they follow. According to the report, of the two, only the V-Dem participated in an “open, frank and transparent” dialogue on the matter. The report states that both indices are based on interviews with experts, although the V-Dem is the more rigorous of the two in that in takes into account a larger number of parameters, making any shortcomings in its conclusions all the more surprising. Although the report’s authors see the architecture of the V-Dem indices as essentially solid, they conclude that there is very little control over the classification criteria. Gracia states that “this particular index takes into account many different parameters to assess the quality of democracy. However, it lacks any explanation to justify why an expert has chosen a 5 or a 9 when assessing a particular parameter.”.

For safety reasons, the selection process to decide which experts participate in the indices is kept under wraps. Furthermore, those chosen are under no obligation to specify the thinking behind their assessment or justify any changes they may subsequently make, which in certain instances have benefited Spain’s ranking. The report states that “if country ratings cannot be guaranteed to abide to common standard criteria, the entire concept of ranking countries against each other makes little sense”. The major issue is that if the experts do not have to justify their opinions, there is no way to eliminate bias or prejudice: “The ratings become just glorified personal opinions at best and, at worst, a fantastic opportunity for manipulation”.

The report’s authors consider it particularly worrying that the experts are selected anonymously and randomly for places such as Catalonia, since the fact that the victims of repression in a particular state form part of a minority, it makes it more likely that the experts who are chosen will adopt the majority opinion with regard to said minority.

Regarding the EIU analysis, the report draws a comparison between the objective parameters used in Dynamic Patterns of Human Rights Practices, a study on the state of human rights which measures extrajudicial killings, levels of repression and international sanctions, and the subjective parameters used by the EIU, such as the Civil Liberties Index, which is based on subjective opinions. They EIU fares badly in the comparison: as little as 40% of the EIU’s subjective index of civil liberties is supported by objective data. In the case of Spain, the objective criteria indicated that in 2017 it had approximately the same level of respect for human and civil rights as Hungary. Instead, the EIU’s subjective view suggested that Spain fared much better.

Invisible repression since 2017

The report places special emphasis on changes in the V-Dem’s indices since 2017, when the crackdown on independence began to take hold. In its 2017 report, the V-Dem reduced Spain’s score as a liberal democracy by 8%. However, in the following years the score has steadily increased.

Indeed, the researchers were struck by the fact that reports in the index’s database had been revised in subsequent years, resulting in the Spanish state obtaining a more generous score than the one which had originally been published since 2017, without any justification for these changes. The authors state that, “Surprisingly, in 2020 the scores for previous years were amended after having bounced back from the significant drop in 2017, thus making it appear less serious.”. Speaking during the presentation of the report, Josep Reyner declared that “when we explicitly enquired as to the reason for this change, no information was forthcoming.”. According to the researchers, the V-Dem team were unable to explain the changes in Spain’s score, though it reiterated its confidence in its experts.

More surprising cases

It is not only Spain’s position in these indices which has raised eyebrows. The V-Dem pushed the United States’ rating to the highest level in the country’s history when Barack Obama took office, a score that was maintained until 2016, with Donald Trump’s victory. The report’s authors find such a high score surprising, given that Barack Obama maintained George Bush’s War on Terror policy, failed to appreciably change racist policing practices, failed to close Guantanamo, and the constitutional system remained unchanged during his tenure.

Likewise, the researchers were unable to find any explanation based on objective data which would justify the US’s score falling to almost the lowest level since the Vietnam War during Trump’s term in office. The report states that “it is true that human rights were severely degraded during the Trump presidency, but to such an extent? Objectively, it is difficult to justify these ratings on the basis of observable facts alone”.

They also mention the case of Belgium, which —according to the EIU— lags far behind Spain in terms of human rights. The report’s authors believe that this could be accounted for by the prolonged periods during which Belgium lacked a government, its short-lived ministerial cabinets and its four-party coalition government. This is in contrast to the Spanish case, with more stable, single-party governments.

Countries which have pursued “socially tough, yet economically orthodox austerity policies” to overcome economic crises, such as Ireland, Portugal, Greece, Cyprus, Lithuania, Finland, Italy and Spain, are also better rated by EIU’s indicators. This is in contrast to countries such as Hungary, Poland, Slovakia, Slovenia and Croatia, which are often led by populist parties or coalitions. In other indices, the influence of the economy also appears to play a key role, according to the experts consulted. A 2017 study of Freedom House’s democratic scores concluded that countries aligned with US foreign policy tend to score higher on Freedom House than on other indicators.

To justify such cases, the authors of the report point out the influence exerted by the media’s narrative regarding every country at a particular moment in time. This could explain the differences between the indices during Obama and Trump’s presidencies, as well as the fact that countries with populist leaders generally receive lower scores, as is the case of Viktor Orbán in Hungary. Neither Mariano Rajoy nor Pedro Sánchez have ever been depicted by the media as populist leaders, leading the study’s authors to believe that this may have some bearing. “Are those anonymous experts able (or willing) to separate the mainstream media’s hype on a country’s government from the objective assessment of that country’s democratic credentials?”, they ask.

A political tool

In light of such criticisms, the authors of the report consider that these indices have become “political tools” which can end up being used by the media and governments to reinforce a specific narrative regarding every country. In Spain’s case, they are used to bolster up the idea that there are no political prisoners or politicians in exile. They are concerned with the possibility that, since the institutions which compile these rankings wish to be influential, this may leave their evaluation processes susceptible to the influence of the powerful and the media which support their views, essentially telling them what they want to hear.

The authors conclude that “sadly, by adhering to a flawed methodology, these index agencies add confusion in a field where their avowed, original aim was precisely to improve transparency”. They argue that what ought to be taken into account when making such assessments are the facts, not the opinions of the experts. “It is even sadder that such a faulty tool should give the government of a flawed democracy with insufficient respect for human rights, such as Spain, the means to claim that it is, against all evidence, a “full democracy”.

The petition

The petition, meanwhile, calls out and deplores the current state of affairs. It appeals to the authors of the indices to change their methodology by employing more robust checks in order to decrease the risk of bias. It has been signed by economists, university lecturers, professors, philologists, psychologists and politicians, among others. During the publication of the report the economist David Ros stated that “we’ve launched this petition because we, too, are outraged”.

Prominent signatories to the petition include the UB’s vice-rector, Agustí Alcoberro; ANC President Elisenda Paluzie; former MP and lawyer Eusebi Campdepadrós; jurist and professor Hector López Bofill; lawyer and Junts MP Jaume Alonso-Cuevillas; jurist Jaume Renyer; Junts MP Joan Canadell; economist Jordi Goula; former MP and engineer Josep Puig; former MP and university professor Montserrat Palau; lawyer and Senior Consul of the Barcelona Consulate of the Sea Jordi Domingo; and the President of the Barcelona Chamber of Commerce, Mònica Roca, among others.