Spain gets worst democracy score in 40 years

V-Dem is an international academic consortium, based at the University of Gothenburg (Swedem), that measures democracy. Its annual ‘Democracy Report’ is a reliable barometer of the democratic quality of different countries – across multiple dimensions and indicators – around the world. Unlike some indices of democracy, V-Dem’s methods and data are completely transparent and available; anyone can download the data and examine each variable by herself.

Spain and the 2019 report

V-Dem recently published its 2019 report and raw data. Soon after publication, the Spanish Government marketing arm, Global Spain (a branch of the Spanish Government’s Ministry of Foreign Affairs) began tweeting data and infographics based on V-Dem.

These tweets and charts, in both English and Spanish, claimed that Spain’s rank and score in the ‘liberal democracy index’ had improved over the last year.

Global Spain (and its leader, [Irene Lozano](https://twitter.com/lozanoirene)) published other tweets and graphics related to V-Dem as well. For example, several tweets pointed out that Spain’s ‘judicial independence’ was among the ‘top 10’ in the world.

In addition to publishing on [its website], the official Spanish language account of ‘España Global’ tweeted infographics and numbers, highlighting the the ‘improvement’ from last year and the high score in ‘judicial independence’.

The campaign was picked up by others, too. Both private Twitter users and official government bodies like ‘España en la UE’ tweeted about the ‘democratic improvements’. And the Spanish Ministry of Foreign Affairs (as well as multiple politicians) retweeted.

Reading all these tweets, it’s clear that Spain’s democracy improved last year, right?

But…

There’s one problem with all these tweets and charts about Spain’s big improvements in democratic quality and top-10 rankings in judicial independence. They’re not true:

• Spain’s V-Dem score did not improve last year (in fact, it decreased slightly).
• Spain’s ranking did not change over the last year year.
• And Spain’s ‘judicial independence’ is not ranked in the top 10 in the world by V-Dem.

It’s simply false. It’s a ‘bulo’. It’s ‘fake news’

THE DATA

Global Spain made an honest mistake in interpreting the V-Dem data. Or rather, three mistakes

Mistake 1: Spain’s ‘Liberal Democracy Index’ did not increase over the last year; it decreased.

The V-Dem has an index measure (an aggregation of other measures) which they call the ‘Liberal Democracy Index’. According to Global Spain, this measure improved from 7,03 to 7,42 over the last year.

This is not true. The correct data are below.

The above chart shows that Spain’s Liberal Democracy Index score did not improve over the last year. In fact, it got worse, going from 0.747 to 0.742.

According to Global Spain’s blog post on the V-Dem data, ‘La consolidación democrática de España durante 40 años de Constitución es indudable’ (‘The democratic consolidation of Spain during the 40 years of the Constitution is beyond doubt’). However, according to the V-Dem data, Spain’s most recent score is its worse score in 40 years, the worst score since the Constitution came into effect.

Mistake 2: No, Spain’s ranking in the ‘Liberal Democracy Index’ did not improve over last year; it remained the same

According to Global Spain, Spain improved 9 places (from 35th to 26th) in the ranking of world countries in terms of liberal democracy.

However, this is also false. Spain’s rank in the 2019 report (with data collected in 2018) is 26th. Its rank the previous year was also 26th. There was no improvement.

The below is the correct data:

Mistake 3: Spain is not ranked 7th in the world in ‘judicial independence’. It is ranked 17th or 18th.

In Global Spain’s Spanish-language infographic, as well as multiple tweets on the subject, it was explicitly stated that Spain was a ‘top 10’ country in terms of ‘judicial independence’ (see below).

This, again, is false.

There are two variables in the V-Dem which directly measure judicial independence. These are (i) v2juhcind (high court independence) and v2juncind (low court independence). In these two variables, Spain ranks 17th and 18th, respectively, not 7th. The below chart shows the correct data.

What Global Spain reported as ‘judicial independence’ was actually a misleading translation of the ‘Judicial constraints on the executive’ index. This index indeed includes some measures on judicial independence, but it also includes other measures (such as the executive’s respect for the Constitution, compliance with courts, etc.). Here is Global Spain’s infographic showing the misleading translation:

How did these mistakes happen?

Global Spain reported false information. But I call this a ‘mistake’ and not a ‘lie’ because it is entirely plausible that they simply were not aware that the information they were reporting was false.

Here is what I suspect happened: someone at Global Spain decided to compare the 2019 V-Dem democracy report with the 2018 report. They looked at the pdf files from two different years, saw the scores and rankings, and turned those into an infographic. Sounds fine, right? Wrong.

Whoever did this failed to examine V-Dem methodology and raw data. If they had examined this, they would have noted that the 2019 report was based on the version 9 methodology, whereas the 2018 report was based on the version 8 methodology. Given the two methodologies, one cannot compare V9 with V8. Rather, to make comparisons over time, one should use only the most recent methodology (V9), which includes historical data.

This is a bit confusing, right? When I examined the raw data, and saw the discrepancies with the historical pdf reports, I also got confused. So, I wrote to V-Dem and asked why it is that the 2018 pdf report contained different values than the V9 raw data.

The V-Dem Institute team wrote back. They explained that ‘Comparing absolute index values between different versions of our datasets’ (exactly what Global Spain had done) ‘can be misleading, due to several different factors’. They then listed reasons, including:

1. ‘Coders add data with every update’.
2. ‘We currently allow our coders to update and change their ratings back in time if they have come across new and more accurate information. This should lead to a continuous improvement of the data quality over time’. So V9 should be considered ‘improved’ relative to V8.

In other words, the more recent versions not only contain more recent data, but also more accurate data of the past. And because the statistical methods across different versions are different, one should not compare data across different versions (which is what Global Spain did). Rather, one should compare data across time, using the same version (which is what I’ve done in this article).

The V-Dem team explained further. ‘Users should always use our newest dataset since it has improved data quality over previous dataset versions’. Since my question to V-Dem specifically pertained to data on Spain, they addressed the issue as well: ‘If Spanish State employees wanted to know if Liberal Democracy improved from 2017 to 2018 they should look at v9 data for both years in which Spain ranked 26 in both years, but reduced their scores from 0.747 in 2017 to 0.742 in 2018. However, this change is well inside confidence intervals so it cannot be considered much of a decrease.’

In other words, Spain’s ‘Liberal Democracy Index’ score, like its ranking, did not improve last year. In the case of Global Spain’s efforts to show an improvement, ‘if the government employees as described by you compare v8 2017 scores with either v9 2018 or v9 2017 scores that would be very misleading in both cases due to the V-Dem methodology.’

Conclusion

Global Spain published, and then heavily publicized, an ‘improvement’ in its ranking and scores as a liberal democracy via the V-Dem data. It also stated, and publicized, that it was a top-10 country in the world according to V-Dem’s in the area of ‘judicial independence’. Both the improvement and the top-10 judicial independence scores are false. Spain’s ‘Liberal Democracy Index’ score did not improve over the last year, and Spain is not in the top 10 countries in the world in ‘judicial independence’ according to the V-Dem.

Like I said, it is entirely plausible that this was a simple mistake on the part of Global Spain, and not an attempt to mislead. And if this was indeed a mistake, I hope that Global Spain will make as much of an effort to publish the correctcomparison data (as I’ve reported in this article, and as is publicly available on V-Dem’s website) as they did to publish the misleading data last week.