Indices, indexes, rankings and ratings. Never have there been more measures competing for attention than today. At the end of May, two ranking reports with relevance to Arab markets were released within two days of each other. The International Finance Corporation released its Doing Business in the Arab World 2012 (DBAW 2012) report and the Swiss-based global business school IMD released its 2012 World Competitiveness Yearbook (WCY).
The DBAW report tells us, in a nutshell, that Saudi Arabia has the best framework of business regulations for domestic companies in the Middle East and North Africa (MENA) — ranked first among 20 Arab countries and 12th worldwide — and that Morocco was the country which made the greatest progress in improving its business environment in 2010/11, on a regional and global scope.
The WCY tells us that Qatar is the most competitive Arab economy — ranked 10th in IMD’s global review of 59 countries by 329 measurements — and that the United Arab Emirates was the country that made the greatest progress globally in competitiveness in 2011, advancing from 28th place to 16th.
The two reports are presumably a boon for the corporate executive who wants to choose a new market or manufacturing location. The IFC measures the level of development that regulatory frameworks have reached in 183 countries and the WCY exhaustively examines 59 countries for competitiveness on the basis of business and government efficiency, business performance and infrastructure. That is, of course, if the said executive has enough time on his hands to pore over hundreds of pages and evaluate the information that is provided.
DBAW is not a slim specimen of the report genre at 123 pages, and the WCY feeds nicely into the assumption that smart people will never stop reading, at over 300 pages (that’s excluding the expanded, detailed country profiles). Also for consideration is the Global Competitiveness Report 2011/12 by the World Economic Forum (WEF), another popular global reference and home of the Global Competitiveness Index (GCI). Covering 142 economies, it is a tome burgeoning with lists after lists, sub-rankings upon sub-rankings, filling nearly 550 pages. Thankfully there are electronic versions.
Curious details
A curio about the WCY is that the 15 most competitive economies in its ranking collectively represent just more than eight percent of the world population and do so only because of the United States, the WCY’s second most competitive economy, contributes some 55 percent of this headcount. Moreover, the top 15 in the WCY are not at all countries driven to improve their comparative economic edges by population expansion; Malaysia, number 14 in the list, stands in as number 75 in global population growth and all other 14 highly competitive countries are on record for comparatively low population growth.
Another note to ponder about the leading competitiveness reports is that both their producers, although each collaborates with research partners around the world, are based in Switzerland (ranked third by WCY and first by WEF). After banking services and those time measurement complications, the reports appear to form another market where Swiss qualities seem strong enough to inculcate oversized or even monopolistic positions.
Of course competitiveness reports are just one type of ranking report that have gone viral with the information age. Measurability is the undercarriage of management; convergence of global markets and production locations puts corporations and large organizations, whether in economy, public sector or civil society in an absolute need for comparative reports. This need has been answered by an army of knowledge economy producers, whose reports rank everything from financial centers to human happiness in comparative reports and handy lists.
Limited criteria, skewed results
In step with technological and scientific progress, the methodologies of global research reports have evolved to ever more sophisticated models. However, even the reports produced by top-tier institutions such as the IFC and leading academic entities such as IMD often disclose rankings or developments that smack of distortions or otherwise can startle the reader.
For example, a category of the DBAW 2012 report ranks countries for the ease with which businesses can access electricity. Lebanon is ranked in seventh place of 20 Arab countries and in one of the top 50 places worldwide for that particular category. Anyone who has ever done business in Lebanon or ridden in an elevator in the country knows of the annoying frequency of power cuts. The DBAW criteria, focusing on the time, cost and number of interactions required to get an electricity connection, do not mention reliability of electricity supply. But what good is knowing that it takes only a short time to get a warehouse connected to the grid when the grid operator cannot deliver the power?
In the WCY, the UAE acheived an improvement of 26 positions, from 34th to eighth, in a particular bracket of the business efficiency sub-category. Strong gains in a single year may be possible, but this particularly large leap upward was reported in the “attitudes and values” bracket. Attitudes change over time, the assumption goes usually, not overnight.
One will find room for questions in every index and rankings report, whether its subject is a single index based on measuring one performance area, a composite index collating several pillars into one ranking and offering the sub-rankings for better understanding, or a meta-index that consolidates other research reports under a captivating header such as Transparency International’s Corruption Perceptions Index.
Besides having to digest and present more data than is comprehensible over a morning coffee, flaw factors in rankings include methodological omissions and at least three other components that one should be aware of: biases, data uncertainty and simplifications. The sources where one might look for these flaw-factors are business leaders, governments and media.
Surveys of business leaders and decision makers are a standard component of many rankings reports, for example the competitiveness reports by both IMD and WEF. Business leaders are usually opinionated and often biased. How common is it to meet a manufacturer or trader who will tell you that his competitor’s products are a better deal than his own, or that his own country of production is not a great location to manufacture his product?
Margins of error in rankings are not the first topic discussed by the providers of rankings who have themselves an incentive to appear as reliable and authoritative as possible. However, producers of the reports are well aware of the bias issue, which a researcher on the WEF’s report called “the halo effect”.
Secondly, not all data should be taken as equal. When governments produce projections of next year’s gross domestic product (GDP), any experienced business leader will assume that not all estimates will hit the bull’s eye.
Population estimates for a large country are very difficult or even impossible to verify, with all sorts of potential digressions in subsequent calculations. Plus, while we like to think of our governments as being honest, data supplied by government institutions and close affiliates to competitiveness researchers may, just may, in extremely rare cases, be a little bit more polished than they should be [think Greece pre-financial crisis].
But the biggest nightmare is the media. Take any rankings report and the media will redact its detailed and complex findings into something akin to a list of test results on the wall of classroom in third grade. Where the researchers explain that their report shows small actual changes resulting in a country’s rise in the percentiles of a ranking for a complex area such as labor market efficiency but that the change is not indicative of much at all, the media will scream that ‘Country X’ gained 20 ranks and is now top of the heap.
Stripping away the sensationalism, the rankings report in the knowledge economy is the business leaders’ Hamlet experience — to trust or not to trust, that is the question. According to Hisham el-Agamy, an Executive Director at IMD in charge of the Middle East Africa, Southeast Asia and Southeast Europe, governments and business investors use a competitiveness report as a “map to understand the country” but “any serious institution which wants to invest in a country does not only consider a single ranking.”
