OBI Response to Blog Posting

Editor's Note:

There was some controversy at the 23rd Annual ICGFM Conference surrounding the assessment of the Government of Honduras by the Open budget Initiative. Mr. Vivek Ramkumar had summary data on hand when he received questions at the conference. He has kindly provided ICGFM with a comprehensive answer.

As the manger of the International Budget Partnership’s (IBP) Open Budget Initiative, I would like to respond to the ICGFM’s headline (“Is the Open Budget Initiative Rating Accurate?”) for its May 20^th blog posting of my responses to questions on the Open Budget Index 2008 (OBI) findings raised at its recent meeting in Miami, Florida.

The Open Budget Index 2008 is a comparative measure of the overall commitment of the governments in 85 countries to budget transparency. The OBI 2008 is calculated using a subset of data collected through the Open Budget Survey 2008—a comprehensive analysis and survey that evaluates whether central governments give the public access to budget information and opportunities to participate in the budget process. The Survey also examines the ability of legislatures and auditors to hold their governments accountable.

The Survey is implemented through a rigorous data collection and review process that spans nearly two years. Although the Survey is based on a questionnaire, it is not an opinion poll. All researchers’ responses to the 123 questions in the survey must be based on verifiable evidence, such as reference to specific budget documents, the country’s laws, or documented communications with government officials.

Within each country, a researcher or team of researchers, who are drawn from civil society or academia and are independent from the government and political parties, complete the survey and provide the required evidence for their answers. Their work is analyzed and reviewed by IBP staff to cross check it against available information. This include those budget documents that countries made available on the Internet, data collected by the Bank Information Center (a Washington, DC-based nonprofit that monitors the activities of international financial institutions); the International Monetary Fund (IMF) Reports on the Observance of Standards and Codes (ROSCs), covering fiscal transparency; IMF Article IV reports; World Bank documents and publications, including Public Expenditure Reviews; and the Organization for Economic Co-operation and Development-World Bank budgeting practices database.

Next, the completed questionnaire for each country is subject to a thorough peer review by two in-country budget experts, who were independent of both government and the researcher’s institutions.

Finally, governments were invited to comment on the completed questionnaires prior to publication. The decision to extend an invitation to a government to comment was based on a request by the research organization that completed the questionnaire for that country. In total, 61 countries were invited to participate. (For more details on the Survey’s methodology and findings, visit www.openbudgetindex.org.)

In addition to the thorough review process, IBP also conducted two further tests to check the reliability and robustness of the data. First, the Survey results were compared with the results of other indices of governance and transparency to see how the Survey fares as an overall indicator of the governance situation in a country. The Survey results showed a strong, positive correlation with the World Bank’s World Governance Indicator on Voice & Accountability (0.737), the Global Integrity Index produced by Global Integrity (0.681), and the Democracy Index produced by Freedom House (0.691). These positive results suggest that OBI is a relatively good proxy for broader measures of governance and the quality of institutions in the countries covered.

IBP also constructed a “unanimity score,” a measure to capture the degree of debate between the researcher and the two peer reviewers responsible for completing the questionnaire in each country. The scores for this Measure of Unanimity are available in the Survey report at http://openbudgetindex.org/files/FinalFullReportEnglish_lores.pdf.

The Open Budget Survey is intended to provoke public debate about budget transparency, public participation in budget debates, and accountability of budget institutions. As such, the research process frequently results in debate among the experts in the country responsible for completing and reviewing the questionnaires about important public financial management topics. IBP attempts to capture this debate through the “unanimity score” and by publishing within each questionnaire the exchanges between researchers and reviewers that led to the selection of a final answer to each Survey question (available at www.openbudgetindex.org).

Since the February 1, 2009, release of the Survey, challenges to OBI 2008 scores have come almost exclusively from governments that earned low scores. I am happy to report that in some cases these governments have made positive changes that should result in more budget information being available to the public. However, since these changes were made after the cutoff date (September 28, 2007) for data collection for the 2008 Survey, these efforts were not reflected in their OBI 2008 scores. If these efforts are sustained, though, they will be reflected in higher scores on the next round of the Open Budget Survey in 2010.

As I explained in my video response to the questions raised by the representative from Honduras about his country’s low OBI 2008 score, there are specific actions that the government could take that would improve their performance. Honduras’ OBI 2008 score of 11 out of a possible 100 is primarily due to the fact that as of September 27, 2007 the government did not produce a Pre-Budget Statement that outlines the macroeconomic framework of the budget, a Citizens Budget (a presentation of the budget that is delivered in a way that is easily understood and widely accessible), or a Mid-Year Report that compares the actual expenditures and revenues to those budgeted. The government did produce an Executive’s Budget Proposal but did not release it to the public. By producing and making these documents available to the public, Honduras could raise its OBI score significantly by 2010. A relatively quick and low-cost first step would be to post the Executive’s Budget Proposal, which is already being produced for internal purposes, on the government’s website. A summary of Honduras’s results on the Open Budget Survey 2008 is available at http://openbudgetindex.org/files/cs_honduras.pdf and the completed questionnaire for the country can be found at http://openbudgetindex.org/files/IBPQuestionnaire2008Honduras.pdf.

Finally, I would like to reiterate the point I make in the video that the IBP believes that governments must be strong and effective to meet the needs of their people, particularly the poor. This belief drives the Open Budget Initiative, which seeks not only to measure levels of budget transparency and accountability at particular points in time but also to provide guidance on how to improve.